March 23, 2018
IBM Continuous Engineering Connector for IoT in Rational Engineering Lifecycle Manager
Use Jazz capabilities when developing IoT solutions components that run on IBM Bluemix
IBM Continuous Engineering Connector for IoT in Rational Engineering Lifecycle Manager 0.9
IBM Continuous Engineering Connector for IoT (CE4IoT) - Technical Preview User Guide
The Internet of Things (IoT) is driving businesses to transform themselves into connected, digital enterprises to take advantage of new opportunities for optimizing operations, increasing customer engagement and loyalty, developing new revenue streams and business models, disrupting existing markets, and innovating faster with an IoT feedback loop that informs business and engineering leaders. Clients recognize that they need to be the disruptors in their industries ... or they will be disrupted.
But being fast to market, which is challenging in its own right, is not enough. IoT solutions that interact with the physical world via sensors and actuators can cause serious harm, financial loss, or reputational damage if the solution fails to perform or performs incorrectly. Doing enterprise-scale transformations to realize enterprise-level business value requires enterprise-appropriate systems and software engineering processes.
Use CE4IoT to bridge the gap
IoT solutions have many components which typically are hosted on multiple runtimes and developed by multiple teams. Coordinating across these teams while designing, building and evolving an enterprise solution is especially challenging.
IBM IoT Continuous Engineering (CE) tools and practices can be employed to design and build dependable enterprise IoT solutions. This technical preview enables teams to extend their development practices to include the teams building IoT solution components using the IBM Watson IoT Platform. CE4IoT enables you to work with a design view of your IoT solution, including devices, rules and Node-RED flows. This design view enables you to link these platform artifacts to any other element in the Rational Continuous Engineering solution including requirements, work items, test cases, and UML or SysML model elements. It also enables you to use Rational Engineering Lifecycle Management (RELM) for traceability and impact analysis of potential changes to your IoT solutions.
With CE4IoT you can make use of Continuous Engineering capabilities when developing enterprise-level IoT solutions. CE4IoT makes this possible by placing objects in IBM Rhapsody Design Manager that represent IBM Watson IoT Platform resources, enabling them to participate in change management, impact analysis, requirements-driven development, requirements-driven testing and other common development practices. These objects include links to the actual resources in the IBM Watson IoT Platform and Node-RED, simplifying navigation between these environments.
- The Continuous Engineering solution runs in the Cloud or in your data center. See the discussion in the Installation Guide for considerations related to deployment and firewalls.
- IoTPPublisher.java runs in the Continuous Engineering environment. It contacts iotCEAdapter requesting information about the resources available in the IBM Watson IoT Platform and Node-RED application.
- iotCEAdapter runs in an IBM-hosted Bluemix organization (1). This server queries the IBM Watson IoT Platform and Node-RED application to discover resources of interest (device definitions, rules and Node-RED flows). Your IBM Watson IoT Platform and Node-RED instances run in your own Bluemix organization (2).
- IoTPPublisher.java identifies new or changed resources and updates the objects in IBM Rational Design Manager representing these resources.
- Once the IoT resources are represented in the Continuous Engineering solution, you can use existing CE capabilities and engineering practices.
Introduction to the IoT Platform Domain
The IoT Platform Domain defines a design-level view of Watson IoT Platform resources in IBM Rhapsody Design manager. This view provides an abstraction of the IoT Platform organization and application implementation details in order to facilitate the design and lifecycle management of your IoT devices and Node-RED flows. The IoT Platform domain defines the analysis and design artifacts that can be integrated with other lifecycle management artifacts including requirements, work items, test cases and other UML or SysML model elements.
The IoT Platform domain is organized into two parts or ontologies: Bluemix and the IoT Platform, as shown in the following UML model sketches:
The Bluemix ontology covers design artifacts that are common to all Bluemix projects. A Bluemix Organization establishes the unit of organization and user management for the Spaces that it contains. Spaces can be used to scope applications and provisioned services for different purposes or lifecycle phases. For example, a typical organization might have different spaces for managing different lifecycle phases such as development, testing, preproduction and production. Applications represent the deployed instances of the Bluemix runtime platform with its bound services. Applications typically have routes that provide access to the running application. All of the Bluemix classes, and many of the Watson IoT Platform artifacts described below specialize Platform Resource which provides a platform URI property that supports direct navigation from Design Manager to the represented Watson IoT Platform resource.
Note: this technical preview imports Node-RED applications from Bluemix. The Bluemix organizations, spaces, and other applications and services are not currently included.
The IoT Platform ontology extends the Bluemix ontology with artifacts that represent design artifacts derived from Watson IoT Platform devices and applications. IoTP Organization extends Bluemix Organization to address IoT Platform specific properties. Like a Bluemix Organization. IoTP Organization is the unit of user management. An IoTP Organization manages Device Types, each with their own Device Properties. Device Type and Device Property classes abstract the Watson IoT Platform device type and device schema elements. A Device represents an instance of a Device Type, and something that would be deployed to the Watson IoT Platform to collect real-time data.
The IoTP Organization also manages Rules for real-time analytics. A Rule has a condition property which is a string representation of the RTI rule condition. The design view enables you to see and manage dependencies between the rules and the devices they reference.
A Node-RED Application is a specialization of a Bluemix Application and represents the Node-RED applications in a Bluemix organization. This allows you to manage the dependencies between your Node-RED flows and the devices they use.
CE4IoT defines an explorer viewpoint in Design Manager called IoT Platform that supports user navigation of the IoT Platform domain. The viewpoint can be used in the Design Manager Explorer view, or when selecting an platform resource in a selection dialog while creating a link to other lifecycle artifacts such as requirements. The viewpoint simplifies user navigation by making visible containment and other associations in the domain ontologies to. For example, the following figure shows the organization, rules, device types, device properties, Node-RED applications and flows for a Low Flow Washing machine in the panel at the left side of the window:
The figure above also shows the Design Manger Properties tab, which can be used to view the details of the selected artifact. Note that these artifacts are read-only in the technical preview.
Preparing to Use CE4IoT
In order to use CE4IoT, you must first create and configure the appropriate project areas. You must at least create one Design Management project area for administration of the Connector properties, for example, called IoTPubConig. Additionally, you need to create or reuse one or a set of related project areas for other lifecycle artifacts including requirements, work item and test cases, and designs. The easiest way to do this is to create a Lifecycle Project using the Quality Professional, Analyst, Developer - with Design Management template, which will create and associate all the required project areas so that you can create links between the different lifecycle artifacts. A Lifecycle Project simplifies user management; you can add users to all the associated project areas in one step. Note however that you will still need to manage each project area to assign project roles in order to establish the appropriate permissions.
Once the project areas are created, you must associate the IoT Platform domain with the Design Management project area you created. This configures the project area to support the Watson IoT Platform artifacts described above. To associate the domain:
- Open the Design Management project area you just created for your IoT design artifacts
- From the Application Administration menu, select Manage This Project Area
- Click on the Design Domain link on left side of the page
- Under the Data Management heading, select Actively managed with Design Manager
- Under the Domains heading, click Associate... and add the IoT Platform Domain from the Available Domains list
- Save the changes to the project area
- Click Explore Project to open the Design Management project area dashboard
- On the Associated Domains widget, click Versions
- Click the Set Versions to Latest Baselines button, then click the OK button to set the version of the IoT Platform Domain to the latest baseline.
All the project areas for you need for lifecycle management of your Watson IoT Platform artifacts are now created, configured, and ready to use.
Publishing the Watson IoT Platform Data to your project areas
Configuring the IoTPPublisher
Preparing for configuration
- Make sure you have all domains and viewpoints installed as described in the installation guide
- Make sure you configured the configuration project area as described in the installation guide, for simplicity we will refer to it as IoTPubConig project area
- Make sure you have your lifecycle project areas set up and that the user id you will use for
publishing has read-write access to the Design Manager project area (the pre-defined Architect role is sufficient).
This project area must be associated with a specific version of the IoT Platform Domain.
The design manager project area the data is published to is called "target project area"
Obtaining the required RTI and Watson IoT Platform API keys and tokens
To access the information from your instance of the Watson IoT Platform you need to have an API Key and Token.
You can generate these by doing the following:
- Go to the relevant instance of the Watson IoT Platform service (directly or via the service on your Bluemix dashboard)
- On the side bar of the dashboard click APPS and then Generate API Key
- Copy the API Key and Authentication Token - you will need it for the configuration
You need to ask for RTI API keys in order to read the RTI rules as well as the device schemas. Until you get the keys you will not be able to read the device types from the Watson IoT Platform. As of December 2016 this is a manual process. To request RTI keys contact your IBM representative (you will need to specify your Watson IoT Platform organization ID).
Entering the configuration data
IMPORTANT: To avoid data corruption, make sure to stop any running IoTPPublisher processes before creating or changing the configuration.
- Open the IoTPubConfig project area
- Make sure you see the "IoT Publisher Configuration Domain" as an associated domain and that it has a version assigned
- Create an IoT Project Area resource (Designs menu) and fill in the data
- schedule: the auto update time interval in seconds
- target DM Server URI: the URI of the server to which the data is published. For example https://myserver.com:9622/dm. Uncheck the Is URI checkbox in the dialog when entering the URI, otherwise a # character is added. Tip: you need to make some change in order to OK the dialog
- target Configuration: the configuration to which you want the data published. This is an optional setting, if you leave it blank the data will be published to the default configuration
- target Project Area: the name of the project area the publisher is publishing to (e.g My Project created as a lifecycle project above)
- userid and password: the username and password used by the publisher when writing data to the target project area
- You can add one or more Node-RED apps to reference their flows by clicking Add... in the Publishes Apps group:
- route: the URL of the application (without the /red)
- bmxAPIKey and bmxAPIToken: the username and password to access the application if relevant. By default node-RED apps are not protected (leave blank in that case)
- Node-RED app Adapter URI: the name of the IBM-hosted CE IoT adapter (Bluemix iotCEAdapter project). Typically should not be changed unless the iotCEAdapter has been deployed someplace else.
- You can add one or more Watson IoT Platform organizations to read from by clicking Add... in the Publishes Organizations group:
- apiKey and apiToken: the API Key and Token to have to access the Watson IoT Platform organization
- orgId: the Watson IoT Platform org id
- rtiKey and rtiToken: the RTI Key and Token if you have them.
- IoT Adapter URI: the name of the CE IoT adapter, this should not be changed in general
- Save the IoT Project Area settings.
Running the IoTPPublisher
- On the server hosting Design Manager (or on another server with network access to your Design Manager server) open a terminal or command prompt
- cd to the folder containing the file IoTPPublisher-jar-with-dependencies.jar
- Run the following command:
java -jar IoTPPublisher-jar-with-dependencies.jar --dmServer [DMSERVER] --user [USER] --password [PASSWORD] --dmConfigProject [IOT_CONFIG_PROJECT_AREA]
- [DMSERVER] is the URI of the DM Server that hosts that IoT Config Project (IOT_CONFIG_PROJECT_AREA), for example https://myserver.com:9603/dm
- o Note: you can get the [DMSERVER] URI by opening the project area in Design Manager and copying the URL from the browser address field
- [USER] and [PASSWORD] are the credentials to read the data from IoTPubConfig
- [IOT_CONFIG_PROJECT_AREA] is the name of the configuration project area (which we refer to as IoTPubConfig in this doc)
- Recommended: Monitor the output messages and make sure there are no errors in the first publishing cycles
- Recommended: Open the target DM project area web UI and validate the data is there by going to the Designs->Explorer
Using the Watson IoT Platform Data In CE Projects
Now that Watson IoT Platform data is in your Design Management project area, you can use this information to manage the lifecycle of your IoT artifacts. Although the artifacts are different, lifecycle management of IoT artifacts is not that different than lifecycle management of similar analysis and design artifacts that can be created using IBM Rational Rhapsody. The Continuous Engineering (CE) lifecycle management capabilities can be used to facilitate the design and implementation of your Watson IoT Platform applications. These same capabilities can also be used to in embedded software development to deploy solution components to the Watson IoT Platform, expanding the scope of traditional Systems and Software Engineering (SSE) solutions to the IoT. It is beyond the scope of this user guide to explain the CE capabilities. Rather it is assumed that CE users are already familiar with these capabilities and this user guide will provide a summary of relevant aspects of Watson IoT Platform artifact lifecycle management.
Administering IoT Platform Design Projects
Administering an IoT platform design project is the same as administering any Design Management project area. The administrator opens the IoT platform design project area, and uses the Application Administration menu items to manage the project area, the application or users. Please note that the following administration topics are specific to project areas that use the IoT Platform domain. Additional details for setup administration are covered in the installation guide.
- Members added to the project area should have appropriate process roles and permissions. The Basic process template defines a number of roles. Typically the Architect role is used for analysis and design activities.
- Typically Design Management project areas will have Uses Associations with related project areas for Change Requests, Quality Management Artifacts, and Requirements. This enables the creation of links between Watson IoT Platform and other lifecycle resources. These are created automatically when creating a lifecycle project as described above
- Project areas used for Watson IoT Platform design must be Actively Managed by Design Manager and include an association to the IoT Platform Domain
Exploring IoT Platform resources
In this technical preview, you can explore IoT platform design artifacts described in Introduction to the IoT Platform Domain using Design Manager dashboards, explorer, and properties views.
The Design Manager dashboard is the typical starting point for managing design artifacts. Like other Jazz applications, you can create project, team and individual dashboards containing widgets that are configured to support different stakeholder needs. Dashboards typically provide information about the design project, recent comments, pending reviews, team members and many other views. For example, the Low Flow Washer (Design Management) project dashboard shows the default widget views.
Dashboards can include widgets that display information from any associated project area. You can create multiple tabs to organize lifecycle and design information to meet your specific needs.
The Design Manager Explorer provides a way to navigate to your IoT platform design artifacts and provides a summary view of key properties. To facilitate navigating potentially complex artifact hierarchies, Design Manager provides different Viewpoints that organize the the artifacts in different ways for different usages. CE4IoT provides the IoT Platform viewpoint which organizes IoT artifacts according to the relationships in the IoT Platform domain as described in section Introduction to the IoT Platform Domain. The explorer viewpoint is primarily used in these three places:
- The Design Manager Explorer view
- In the sidebar of the Design Manager Properties view
- In the selection dialog for selecting IoT platform resources when creating links to other lifecycle artifacts (see the next section)
For example, the Low Flow Washer (Design Management) Explorer view shows a hierarchical list of the artifacts and their properties:
Selecting the LowFlowWashingMachine device type opens the Properties view which displays the details of the selected artifact:
Creating links to IoT platform resources
The IoT Platform Domain defines the design view of Watson IoT Platform resources as OSLC Architecture Management resources. This allow you to link the IoT platform artifacts to any other OSLC provider that supports links to Architecture Management resources. The various OSLC domain specifications define standard links to Architecture Management resources, and these properties are supported by the IBM CE products. This allows you to create links from IoT platform design artifacts to requirements, change requests (or any RTC work item type), test cases, etc.-- the same way you can link from UML and SysML models to these life-cycle artifacts. When creating a link to an IoT platform artifact, Design Manager displays a selection dialog that you can use to select one or more target elements to link to. For example, if you are viewing a requirement in Rational DOORS Next Generation (RDNG), you can create a Satisfied By Architecture Element link to one or more IoT platform resources:
Notice that the Search Type is Explorer, and the explorer viewpoint is IoT Platform. The selection dialog is similar to the explorer list view described in the previous section. Search Type can also be set to Search to search for the desired artifacts, or Pinboard to show the artifacts that you added to your Design Manager pinboard.
Previewing IoT Platform resources in CLM application
Once you have created links between lifecycle artifacts, including IoT platform design artifacts, OSLC consumer applications can use OSLC preview to view artifacts managed by other tools. For example, if we created a Satisfied By Architecture Element link from a requirement to an IoT platform artifact such as the LowFlowWashingMachine, RDNG can show a preview of the device type. Hover over the link to see the preview:
OSLC preview provides three different views, the first two of which are shown in the figure above.
- Artifact icon and label
- Small preview dialog
- Large preview dialog
OSLC preview makes it easy to view linked artifacts across tools in order to understand their relationships. You can click on any of these artifact links to navigate to the artifact in the other tool.
Using traceability and impact analysis
One of the primary drivers of continuous engineering is the management of change: change in the marketplace, change in requirements, change in the supply chain (dependencies), change in available resources, change in base technologies, change resulting from discovered defects, etc. Managing change is one of the primary purposes of the IBM's Continuous Engineering Solution. CE enables collaborative interaction across the whole team and across the whole solution delivery lifecycle to maximize the positive impact of change while minimizing its negative impact. The primary means of managing change is through comprehensive change and impact analysis across the lifecycle.
CE provides a number of ways to manage change, from fine-grained impact analysis in individual tools to comprehensive impact analysis across tools using RELM.
Individual tools often provide their own fine-grained traceability and impact analysis capabilities. Impact analysis in these tools often extends only to the "edge" of the tool. That is, the tool can show traceability between its managed resources and links it knows about to resources in other tools, but it cannot navigate beyond these links into the resources managed by other tools. Rather this is what RELM provides. Individual tools provide additional details supporting richer impact analysis of changes to resources the tool does manage, since each tool knows more about the resources it manages and has greater access to them.
Design Manager provides the ability to create Impact Analysis Diagrams that visually show the relationships between all downstream artifacts referenced from a focus artifact. You can use the Navigation Tree to quickly navigate to different resources and relationships within the analysis diagram. You can use different views of the dependencies including hierarchical, tree, radial tree, forced directed and circular. You can filter the views to hide uninteresting artifacts, and can create reusable impact analysis profiles that save your preferences. For example, the figure below shows an impact analysis diagram focused on the LowFlowWashingMachine:
This view shows the numOfRinseCycles device property is selected so that the view is positioned to show the focus element, LowFlowWashingMachine and the selected property. This helps you zero in on the model elements whose change you want to explore. See Analyzing the impact of change on designs in the Design Manager documentation for further information.
Another technique for managing change is through Suspect traceability (for projects that do not use configuration management) or Link validity (for projects that do use configuration management). Suspect traceability and link validity indicates the potential need for changes when related or linked artifacts change. This provides a convenient way to detect the potential for change impact, and to clear suspect links once the impact has been addressed. See the documentation links for further information. For example, the figure below shows a number of suspect and valid links for a Node-RED application:
Creating Design Reviews
Design reviews are a good way to ensure the whole team understands the potential impact of proposed changes to and from design elements. Like RDNG, Design Manager supports the creation of explicit reviews. A review has a name and instructions to the participants performing the review. You can add review participants and assign them Reviewer, Approver, or Subscriber roles. You can then add specific IoT platform design artifacts to the review to tell reviewers what they should focus on. Reviewers can open the review, review the resources, and provide comments on the resources themselves, or in the review.
A review has a lifecycle including the states created, started, reviewed and finalized. Reviews are saved so that you can see who reviewed and approved what and when. For example, the following figure shows a review of an IoTP Organization and Device Type:
Query and Reporting on IoT platform resources
There are a number of ways to find and report on IoT platform design resources. The simplest is to use Quick Search which allows you to do simple text searches on design artifacts. Design Manager supports the creation of reports using Rational Publishing Engine report templates.
Design Manager also contributes resources to the Lifecycle Query Engine which enables the use of the Jazz Reporting Service to build queries and create reports on not only IoT platform design artifacts, but also including other lifecycle artifacts contributed by other tools.
Note: Jazz Reporting Service support for Design Management in in technology preview for CE 6.0.3. Therefore this feature is not available for this version of the CE4IoT technical preview.
The CE products support versioning and configuration management for many lifecycle artifacts, not just source code. Local and global configurations allow you to create versions of artifacts linked to other artifacts including your IoT platform design. This facilitates strategic reuse, product line engineering, parallel development, and controlled change management of your solution.
This version of the CE4IoT technical preview enables your IoT design views to be versioned and contribute to global configurations. Note however that this does not support or involve versioning and configuration management of the Bluemix IoT platform resources themselves. That is, you can version the design views and create baselines to preserve configurations of the design views by enabling configuration management on your IoT platform Design Manager project areas. However, this does not version the actual Watson IoT Platform artifacts from which the design views were extracted.
Creating and Configuring CE4IoT RELM views for IoT Platform traceability and impact analysis
Rational Engineering Lifecycle Manager (RELM) helps teams manage the complexity of IoT systems and solution development. RELM enables users to create queries and views that span across CE products in order to perform system-wide, full lifecycle traceability and impact analysis. The CE4IoT technical preview enables you to use RELM capabilities to include your IoT design artifacts in traceability and impact analysis. With RELM you can construct the specific queries and views that support your particular needs. With CE4IoT you can also include your IoT platform artifacts with RELM in order take a design centered view of traceability and impact analysis. That is, you can look at potential changes in your solution design, and determine what requirements and test cases might be impacted, what other IoT design artifacts might need to change, and what work will need to be done to complete the change. You can also look at the impact that changes in requirements might have on IoT platform design artifacts.
CE4IoT includes a number of custom RELM artifact elements that can be used to represent Watson IoT Platform artifacts in RELM views. See Import the IoT4CE Custom Artifact Elements for details on how to import the CE4IoT artifact elements into the RELM application so that they can be used to create queries and views of IoT platform artifacts. For example, the figure below shows a RELM view of the Low Flow Washing Machine system traceability view:
In this view you can see the flows in a Node-RED application, the device types they use, the properties of that device type, the requirements a change in the properties implement, the work required to satisfy the requirement, and how the result will be tested.
Create RELM Views
RELM can be used to create many types of view. What follows is a simple example of how to go about using the CE4IoT custome artifact elements you imported.
From the menu select Views > Create > View
Name the new view 'Change Request Analysis' and click Finish:
Add a Starting Artifact Element
Click the 'x' in the upper right hand corner of the tips box to close it:
In the right-hand sidebar, click the twisty next to the Artifacts section to collapse it:
In the Custom Artifact Elements palette, scroll down until you can see a Node Red Application. Drag the Node-Red Application onto the page:
When you drag the element onto the page it becomes a container for elements of that type and a Set Parameters dialog opens which allows you to filter the items that appear in that container. (If you do not select any parameters then the container will show all Node Red Applications)
Filter to Show Only a Specific Artifact Element
In the Set Parameters dialog, click the pencil icon to the right of the Architecture Resource field:
In the Select Parameter Value dialog, type ConnectedWashingMachine into the Search by ID or Text field:
Select the ConnectedWashingMachine Node Red Application Architecture Resource:
NOTE: Your server will only have one - this screenshot was from a sandbox where there were multiple instances of the demo staged.
Click Add to set the parameter
Click OK to close the Edit Parameters dialog
Add Linked Elements
Right click the Node Red Application container on the page and select Show Links To:
In the Custom Artifact Elements section, expand the Other CLM Applications folder and select Flow. In the Link types section on the right, select has flow:
Other connected elements may be added in the same way. Lines, fonts, colors and so on can all be modified to create a pleasing view - see the RELM training for more details.
For additional information on RELM see:
- RELM documentation in IBM Knowledge Center
- Course material for setting up, configuring and creating custom views (coming soon)
- YouTube video: Building Views with Rational Engineering Lifecycle Manager v5.0
The IBM Continuous Engineering development tools and methods combined with the IBM Watson IoT Platform and related Bluemix services help development teams to design and implement complex, reliable and secure IoT solutions. The Continuous Engineering Connector for IoT technical preview extends these capabilities to support scalable development of IBM Watson IoT Platform solutions. Features include:
- Suite of end-to-end capabilities (requirements, design, tracking and planning, change management, testing and test management, Watson IoT Platform, Bluemix services, etc.)
- Traceability across the development lifecycle
- Enabling teams using different tools and development practices to collaborate in building reliable IoT solutions
- Tailored solution offerings that match the problem space and end user skill levels.
Use CE4IoT to help you accomplish the following:
- Turn existing products into connected products
- Innovate faster with an IoT feedback loop to business leaders and engineering, including use of Watson services for advanced analytics
- Achieve consistency and robustness across the design artifacts and development process to enable efficient, cost effective, and reliable (high quality) implementation
- Design complex system-of-systems solutions (requirements management, systems engineering, MBSE)
- Manage change effectively across the IoT solution lifecycle (change requests following customized workflow, traceability and impact analysis of lifecycle artifacts)
- Manage risk (particularly deployment risks) and respond more effectively when risks materialize
- Enable validation of the solution architecture against the system requirements and use cases
- Improve development collaboration across geographical and organizational boundaries
- Develop your solution design ground truth necessary to estimate costs and effort
For further information, see:
- Learn more about the IBM IoT Continuous Engineering Solution
- Learn more about the Internet of Things
- Learn more about the IBM Watson IoT Platform
- Learn more about product development for the Internet of Things
- Learn more about IoT development using Delivering large-scale Internet of Things and analytics applications, a IBM Bluemix Garage Method