(r11) ELMArchitectureOverview < Deployment

Engineering Lifecycle Management Wiki - Deployment
Community information and contribution guidelines
~~EditAttach~~Printable
TWiki > Deployment Web > DeploymentPlanningAndDesign > ELMArchitectureOverview
Revision 11 - 2024-11-18 - 13:54:24 - DanielMoul
<div id="header-title" style="padding: 10px 15px; border-width:1px; border-style:solid; border-color:#FFD28C; background-image: url(<nop>https://jazz.net/wiki/pub/Deployment/WebPreferences/TLASE.jpg); background-size: cover; font-size:120%">
---+!! IBM Engineering Lifecycle Management (ELM) Architecture Overview 
%DKGRAY% Authors: Main.TimFeeney, Main.DanielMoul, Main.RichWatts, Main.MichaelRowe <br>
Build basis: ELM 7.x.
%ENDCOLOR%</div></sticky>

<!-- Page contents top of page on right hand side in box -->
<sticky><div style="float:right; border-width:1px; border-style:solid; border-color:#DFDFDF; background-color:#F6F6F6; margin:0 0 15px 15px; padding: 0 15px 0 15px;">
%TOC{title="Page contents"}%
</div></sticky>

<sticky><div style="margin:15px;"></sticky>

Designing and developing tomorrow’s products and systems involve a level of complexity and require a level of coordination among many teams and specialties like never before. _Products_ have become _systems_ which have become _systems of systems_. While systems have included mechanical, electrical and software aspects for many decades, software (embedded, in the data center, in the cloud) provides an ever-growing portion of the essential features, and a correspondingly larger portion of the effort in requirements, design, implementation, verification, and validation. 

The cost of failure of these systems is high: human injury or death, impact to the environment, financial penalties, reputational damage. As a result, regulations and industry standards have arisen that reduce the chance and impact of failure and at the same time assert constraints on the organizations developing and operating these systems, which must _comply_ with these constraints. Key enablers of compliance must be built into an engineering solution to enable teams to be productive while at the same time operating correctly in terms of _process_ and _outcomes_.

Every engineering toolchain involves tools from multiple vendors and open-source projects. Decades of experience with IT integration projects and the growth of the internet have reinforced the lessons that “just enough” integration with good standards in a federated data system allow for scale (and evolution in ways not planned or expected in the beginning).
   * Open Services for Lifecycle Collaboration (OSLC)
   * W3 open data platform underpinning an engineering linked data knowledge graph (sometimes called the “Digital Thread”)
   * REST-based APIs
   * Federated servers and data

A modern engineering foundation provides the following:
   * Version and configuration management, including of the relationships among resources
   * Expression of hierarchical decomposition of these configurations
   * Change management with auditability
   * Transparency of information with appropriate access controls
   * Openness, so other tools and data repositories can participate in the engineering linked data knowledge graph
   * Reporting in any configuration context, including tools and data repositories from many providers

IBM Engineering Lifecycle Management (ELM) implements these principles. Growing out of the lessons of Eclipse, which enabled multiple providers to contribute capabilities into developers’ workstation toolset, the IBM Jazz project brought development collaboration to teams. Now after more than 15 years and many millions of hours of use by developers and engineers of many domains worldwide, we focus the ELM solution on helping teams address the unprecedented challenges summarized above.

---++ High-level architecture 

ELM Applications: 
   * Are distributed 
   * Loosely coupled
   * Provide RESTful services
   * Share a standard set of services
   * Provide addressable resources
   * Leverage traceability (from the ground up)

A simplified diagram of the component relationships within ELM. 

<img src="%ATTACHURLPATH%/ELMcomponentRelationships.png" alt="ELMcomponentRelationships.png" width="750" height="400"/>

Jazz-based applications are !Java EE client/server (MVC-like) applications built on the OSGi (Eclipse Equinox) framework.

“Jazz-based” refers to an application built on the Jazz Foundation SDK. These applications are written in Java, extend the Eclipse (OSGI) Equinox framework, and follow a model-view-controller client/server implementation pattern.

The suite includes applications that are not jazz-based. These include:
   * Rhapsody (C++), 
   * DOORS (C++), 
   * LQE, LDX, and JRS (Java, but not jazz-based)
   * New in 2024: Rhapsody Systems Engineering (TypeScript)

---++ Major components 

The ELM solution is comprised of the following applications running on a common Jazz Team Server (JTS). These applications are Requirements Management (RM), Change and Configuration Management (CCM) and Quality Management (QM).  These applications are used by a set of products included in the ELM solution. These products are: 

   * *[[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/doors-next][IBM Engineering DOORS Next (DN):]]* helps teams define and use requirements effectively across the project lifecycle including rich editors for both textual and visual requirements capabilities for lean requirements definition, including the creation of use cases, sketches and storyboards.
   * *[[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/workflow-management][IBM Engineering Workflow Management (EWM):]]* integrates change management, source control management, continuous builds (software delivery automation), project planning and reporting/dashboard capabilities.
   * *[[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/test-management][IBM Engineering Test Management (ETM):]]* provides a shared test management hub for test planning, creation, and execution as well as workflow control, tracking, and end-to-end traceability.
   * *[[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/design-rhapsody][IBM Engineering System Design Rhapsody - Model Manager (RMM):]]* enables teams to share, collaborate and manage design information across the application development lifecycle. It is provided as an extension to an Engineering Workflow Management (EWM) server, rather than as a separate application.
*Jazz Reporting Service (JRS)* is the default reporting option for ELM and consists of the following components.
   * *Report Builder:* Provides an easy-to-use interface where you can create your own reports or run predefined reports. 
   * *Data Collection Component (DCC):* Gathers the data from across your projects into the data warehouse for your reports. 
   * *Lifecycle Query Engine (LQE):* Indexes the data from across your projects for near-live reporting and reporting on configurations.

In addition to these applications, the ELM solution provides two types of optional reporting capabilities. 

   * *[[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/lifecycle-optimization-insights][IBM Engineering Lifecycle Optimization – Engineering Insights (ENI):]]* allows creation and viewing of chart and dashboard style reports. It provides users with an additional set of predefined development intelligence reports, as well as the ability to author new reports.
   * *[[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/lifecycle-optimization-publishing][IBM Engineering Lifecycle Optimization – Publishing (PUB):]]* provides support for authoring and generation of document style reports. 

In support of federated version/configuration management, which enables parallel development and strategic reuse patterns including product line engineering (PLE), ELM includes these optional components:
   * *Global Configuration Management (GCM):* defines hierarchical, federated configurations (streams and baselines) for contributors implementing OSLC Configuration Management.
   * *Link Index Provider (LDX):* provides a store for links between engineering artifacts participating in global configurations.

In addition to these applications, the ELM solution includes a set of process and practices and supporting tool extensions, all packaged in an *IBM Engineering Lifecycle Optimization – Method Composer* configuration. The ELM processes and practices include everything the client needs to execute the solution. They describe the context and usage model for the products that support the solution -- step-by-step guidance on how to apply the solution, including product guidance in tool mentors.  The practices are self-documenting and include process templates for enacting the solution.

---++ Licensing

ELM licenses generally enable capabilities in multiple ELM components. The primary licenses provide secondary-license capabilities in the other components. As an organization adds more ELM components to their deployment, existing licenses become more capable. For example, in the three main components:

Primary licenses enable full capabilities for the role, 
   * IBM Engineering DOORS Next _Analyst_
   * IBM Engineering Workflow Management _Developer_ 
   * IBM Engineering Test Management _Quality Professional_

Secondary licenses enable read, comment, approve capabilities in the other applications
   * DN _Contributor_
   * EWM _Contributor_
   * ETM _Contributor_

EWM _Stakeholder_ is a low-cost tertiary license, providing work item capabilities for people in other organizational roles.

See [[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/lifecycle-management/7.1?topic=licenses-client-access-license-management][Client access license management]] for more details.

EWM, ETM and DOORS Next share a common installer that deploys the shared JTS plus the CCM, QM and RM applications. The trial licenses that are installed in the JTS provide access to the capabilities of these products. This means that you can install any of these products to enable the full set of engineering lifecycle management capabilities provided by all the products. 

---++ Data architecture 
This section is under construction
 
---++ Integration architecture 

<img src="%ATTACHURLPATH%/JazzIntegrationArchitecture.png" alt="JazzIntegrationArchitecture.png" width="800" />

The ELM applications are designed to be extendable and integrated into existing engineering management systems.  This is enabled by the Jazz Team Server (JTS).   The JTS provides foundational services that enable groups of tools to work together.  Powering much of integration architecture are standard RESTful APIs and standard resource definitions which enable participating tools to easily share data. Foundation also includes reusable building blocks that speed the development of new features and the adoption of existing tools.  Detailed information is available for integration APIs is available at – [[https://jazz.net/wiki/bin/view/Deployment/ELMProductAPILanding][ELM Product API Landing page]]. 

Protocols used for communication between components.
   * To ensure security, ELM supports Transport Layer Security (TLS) 1.2 and 1.3 for encryption of messages running between applications and over the internet.  We recommend deployments use TLS 1.3 which provides [[https://en.wikipedia.org/wiki/Forward_secrecy][perfect forward secrecy]]. 
   * Support for hypertext transport protocol security (https) is provided by !WebSphere liberty.

APIs and interfaces (and support for 3rd party integrations) - Each of the ELM applications supports multiple methods for extension and integration.  
   * *OSLC*: Supports linked data, allowing traceability and collaboration across the products within the solution.  For OSLC domains we support the following specifications:
      1. Detailed information for enabling OSLC integrations for configuration Management enabled applications can be found at [[https://jazz.net/wiki/bin/view/Deployment/IntegratingWithConfigurationManagementEnabledCLMApplications][Integrating With Configuration Management Enabled CLM Applications]].
      1. Note for all OSLC specifications, you should perform the OSLC discovery process to validate the resource shape and other requirements. A good rule of thumb for this process can be found [[https://michaelrowe01.com/index.php/day-job/ibm-elm/getting-a-resource-shape/][here]].

|   %BLUE% *Table 1: Supported OSLC Specifications* %ENDCOLOR%   ||
| %BLACK% *Domain* | *Specification links* %ENDCOLOR% |
| Core | [[https://docs.oasis-open-projects.org/oslc-op/core/v3.0/os/oslc-core.html][3.0 OS]], [[https://archive.open-services.net/bin/view/Main/OslcCoreSpecification.html][2.0 Final]] |
| Query | [[https://docs.oasis-open-projects.org/oslc-op/query/v3.0/os/oslc-query.html][3.0 OS]] |
| Architecture Management | [[https://archive.open-services.net/specifications/architecture-management-2.0/index.html][2.0]] |
| Change Management | [[https://docs.oasis-open-projects.org/oslc-op/cm/v3.0/errata01/os/change-mgt-spec.html][3.0 OS with Approved Errata 0.1]], [[https://archive.open-services.net/specifications/change-management-2.0/index.html][2.0]] |
| Configuration Management | [[https://docs.oasis-open-projects.org/oslc-op/config/v1.0/os/oslc-config-mgt.html][1.0 OS]] |
| Quality Management | [[https://docs.oasis-open-projects.org/oslc-op/qm/v2.1/os/quality-management-spec.html][2.1 OS]], [[https://archive.open-services.net/specifications/quality-management-2.0/index.html][2.0]] |
| Requirements Management | [[https://docs.oasis-open-projects.org/oslc-op/rm/v2.1/os/requirements-management-spec.html][2.1 OS]], [[https://archive.open-services.net/specifications/requirements-management-2.0/index.html][2.0]] | 
| Tracked Resource Set | [[https://docs.oasis-open-projects.org/oslc-op/trs/v3.0/os/tracked-resource-set.html][3.0 OS]] |


   * EWM, ETM, DNG, and Rhapsody each support a !RestAPI called Reportable !Rest.  These public APIs allow for access a subset of application data, and are documented on [[https://jazz.net/wiki/bin/view/Deployment/ELMProductAPILanding][ELM Product API Landing]] page. 
   * Both server and client-based SDKs are available for JTS, GCM, DNG, and EWM, documented on the [[https://jazz.net/wiki/bin/view/Deployment/ELMProductAPILanding][ELM Product API Landing]] page. 

Other supported integration methods include:
   * *TRS Feeds*: Provides integrated data transfer through feeds
   * *Data Collection*: Provides integrated data transfer through ETL
   * *Data Transfer*: Provides data transfer via import/export or synchronization

Security / Authorization architecture
   * There are multiple methods of authenticating access to a jazz-based ELM application.  By implementing a Jazz Authentication Server (JAS), you can greatly simplify application-level access with IBM Liberty’s OIDC feature and OAuth 2.0 bearer tokens.  
   * If you choose not to use the JAS you can implement OAuth 1.0a via registered “friend” applications.  More information on the Jazz Security Architecture and JAS server can be found [[https://jazz.net/wiki/bin/view/Deployment/AboutJazzAuthorizationServer][here]].
Managing Integrations
   * Integrating your ELM applications with other processes and application is a key driver of value to your teams.  As such, you should develop an appropriate governance process or set of protocols to ensure a smooth experience for all stakeholders.  
   * Things to consider when managing integrations include:
      1. Performance and frequency – understand the load they may have on the system
      1. Scope of data – make sure you ask (or provide) only the data necessary for the specific transaction.  If you are looking for specific artifacts or assets, be specific instead of running large open-ended queries, as they may impact other processes and users.
      1. Enable monitoring for your integrations.
   * More details on managing your integration we provided the following [[https://jazz.net/wiki/bin/view/Deployment/AutomationsBestPractices][best practices]].

Tracking / Monitoring
   * Building integrations without considering performance and monitoring can leave you with an inappropriately sized or non-performant environment.  Integrations should be monitored via the [[https://jazz.net/wiki/bin/view/Deployment/CreateCustomScenarios][Resource-intensive Scenario]] process.

---++ Deployment architecture

The ELM solution components are built on top of Jazz Team Server, which provides the foundational services that enable a group of applications to work together as a single logical server.  The figure below shows a typical department installation topology. Note that each application shares a Jazz Team Server.  The deployment can leverage a single database server to host the individual database instances. This helps to reduce administrative costs including simplifying backup procedures, although is not required.  The diagram below shows the various product components or applications.  These applications represent capabilities and do not map 1:1 to the products.  For example, Engineering Test Management contains capabilities that are found in each of the applications (CCM, RM, QM and JTS) shown below.  An ETM Quality Professional license has access to defining textual requirements in the RM application, work items and plans in the CCM application and all capabilities in the QM and JTS applications.

<img src="%PUBURL%/%WEB%/StandardTopologiesOverview/Departamental-7.1.png" alt="test" width="1000" height="600" />

The ELM solution may be deployed in one of the standard topologies (or hybrid thereof) described in [[https://jazz.net/wiki/bin/view/Deployment/StandardTopologiesOverview][Standard deployment topologies overview]] while also applying one or more of the topology patterns noted in [[https://jazz.net/wiki/bin/view/Deployment/StandardTopologyPatternsOverview][Standard deployment topology patterns overview]] used to meet different scalability, performance, redundancy, security or other non-functional needs.
The ELM solution may be deployed on-premises on physical and/or virtual servers as well as part of a cloud-based deployment using services provided IBM, Amazon, Google, or Microsoft.
 
---++ Technology stack  

The table below identifies the primary elements for our technology stack. It is not intended to be exhaustive.

|   %BLUE% *Table 2: ELM Technology Stack* %ENDCOLOR%   ||
| %BLACK% *Stack Component* | *ELM Technology* %ENDCOLOR% |
| UI | Dojo, React,jQuery |
| Integration | OSLC (REST) |
| Languages | Java, !JavaScript, C/C++, PHP, CSS, HTML, XML, Python, Ruby, !TypeScript |
| Frameworks | Eclipse Equinox (OSGi) |
| Authentication | Kerberos/SPNEGO, Jazz Security Architecture (OIDC), SAML |
| Semantic Model | OSLC (OASIS Std) |
| Storage | Relational DB (DB2, Oracle), Triplestore Index (Jena) |
| Search | Lucene |
| Pub/Sub | OSLC TRS (provider and consumer) |
| Licensing | IBM Common Licensing |
| Logging and Monitoring | JMX MBeans / Instana |
| Install | IBM Installation Manager |

---++ Security architecture 

The Jazz Team Server application is packaged as a Java Enterprise Edition (Java EE) Web Application and as such runs in a supported Java EE container using a standard _authentication, authorization, and permission model_. The application also uses a set of predefined roles that are assigned to users, for _authorization_ to access specific URLs or to perform specific low-level operations (e.g. read, write, administer).  The “container” is another term for “application server”, e.g. IBM !WebSphere Liberty.  The authentication itself is managed by the container; if a user fails to authenticate by providing a valid user id and password, then the container rejects the request without the request ever reaching the application.  When the user successfully authenticates, the container subsequently forwards the successfully authenticated request to the application (Jazz Team Server in this case) along with the current user’s authorization (group memberships or role) for processing. Within the operation that processes the request, the application first checks that the user is assigned a role with requisite authority to perform the operation.

The configuration of the authentication and authorization Is done at the container level, at login time the application server retrieves the user information from the external user directory.  Lightweight Directory Access Protocol (LDAP) is an external user registry supported by the Jazz Team Server application.  To support authorization, the group membership of the users must be managed in the user registry.  

Jazz Team Server authentication is further explained in Tech Note [[http://jazz.net/library/article/75][TN0013]].  Support for Common Access Card (CAC) Smart Card and generic certificate-based authentication is provided, more information available [[https://jazz.net/library/article/90284][Configuring certificate authentication for Rational solution for Collaborative Lifecycle Management on Liberty Profile]].

Additional authentication methods have been introduced with the addition of the Jazz Authorization Server (JAS).  The additional authentication methods are SAML and OIDC.  More information on these configurations can be found in the [[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/lifecycle-management/7.0.3?topic=server-enabling-saml-as-identity-provider][documentation]].

Once a user is authenticated and authorized to access the Jazz repository, the application, e.g., Engineering Workflow Management, performs process-level authorization based on a user's role within a project and what permissions and operational behaviors have been allowed for that role.  Roles identify the functions of team members and can be defined at the project level or within a team area.  Permissions are then assigned for specific operations to roles at the project level or within a team area.  A user may be assigned more than one role, which are additive.  Behaviors define preconditions and follow-up actions for individual operations.  This behavior encourages or enforces process rules for your project and your teams.  Behavior is defined in the project’s process configuration and can be customized by team areas. Permissions and behaviors as described here is the general ELM model, however, how it is implemented and exposed to the user varies between the applications.

Licensing also impacts the security model and whether an authenticated user can perform actions they are authorized to do.  The licensing model used in ELM is described in [[https://www.ibm.com/docs/en/engineering-lifecycle-management-suite/lifecycle-management/7.0.3?topic=aelms-managing-licenses-in-engineering-lifecycle-management-engineering-lifecycle-management][Managing Licenses and Inside Jazz Licensing]].

   * Authentication and authorization mechanisms.
   * Encryption and data protection.
   * Security best practices implemented.

---++ Performance, scale, and monitoring considerations
The ELM Performance team publishes results and guidance on sizing and tuning from performance testing in our catalog of [[PerformanceDatasheetsAndSizingGuidelines][Performance sizing guides and datasheets]]. Enterprise monitoring is a best practice we highly recommend. See the [[DeploymentMonitoring][Deployment monitoring]] page. 

---+++!! Related topics: [[DeploymentPlanning][Deployment planning: Where to start?]]

---+++!! External links:          
   * None

<sticky></div></sticky>
Attachments
Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
png	ELMcomponentRelationships.png	manage	146.2 K	2024-08-13 - 19:21	TimFeeney
png	JazzIntegrationArchitecture.png	manage	255.4 K	2024-09-10 - 18:43	AlessandraCristinaAndrade	According to V7.1
Copyright © by IBM and non-IBM contributing authors. All material on this collaboration platform is the property of the contributing authors.
Contributions are governed by our Terms of Use. Please read the following disclaimer.
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.