Filtering the LQE type system model when reporting on configurations
If you use Lifecycle Query Engine (LQE) to report on versioned artifacts with configuration management, you are already aware of the challenges related to maintaining consistency in type system definitions across your IBM Engineering Lifecycle Management (ELM) applications, particularly for requirements in DOORS Next. Other Jazz.net articles describe practices to manage the DOORS Next type system, including how to define consistent URIs for types and attributes.
However, there are some situations where type system changes are unavoidable. For example, as a result of process evolution or transformation, you might need to deprecate or revise attributes and labels, causing inconsistencies between current streams and older baselines. Another example might be where you maintain variant streams for different clients or partners who want to see their own labels and values reflected in reports on their data. What can you do in these situations?
In this article, we explore how new capabilities in ELM 7.0 enable you to define filtered subsets of the default LQE type system model to make it easier manage conflicting versioned types, including:
- An overview of filtering the LQE type system model
- Creating filtered type system models
- Authoring reports using filtered type system models
- Running reports
- Maintaining filtered type system models
- Summary of best practices and additional information
Note: Although Report Builder shows LQE and LQE scoped by a configuration as separate entries in administration and selection fields, they are actually the same data source and share the same type system model. The difference between them lies in the initial filtering of project areas, where LQE excludes configuration-enabled project areas while LQE scoped by a configuration excludes project areas that are not configuration-enabled, and the mandatory configuration context at run time required with LQE scoped by a configuration.
Filtered type system models are available only for reporting on versioned artifacts and configurations with LQE scoped by a configuration, where the challenges are most acute.
Important: The new filtering capabilities are not a replacement for controlling the evolution and consistency of your type system, which remains a best practice and is essential for enabling report reuse. Adding filtered type system models impacts administration, usage, and potentially performance; plan their use thoughtfully.
What does it mean to filter the LQE type system
By default, LQE constructs an overall type system model for reporting, merging type resource information from all configurations, components, and project areas in your ELM environment. Where types differ across configurations (by URI, label, or presence/absence), LQE attempts to resolve conflicts and identifies them to report creators in Report Builder.
When you author reports in Report Builder, you start with this comprehensive default type system model; as you make choices to limit project areas, artifact types, and so on, those choices filter the types based on your selections. That becomes more challenging when you have conflicting or duplicate types, as shown below.
As of ELM 7.0, you can create additional type system models that filter the default before you get to the authoring step. The filter is based on a specific global configuration, using only the type definitions from its contributions. A report author can then start authoring from the filtered type system model, which has already excluded conflicting type versions, and proceed to make choices as usual to further filter types and data.
For an example of why you might need to define filtered type system models and more detailed instructions, see Reporting on versioned type systems in the IBM Knowledge Centre.
Like the default model, each additional type system model requires disk space and memory resources from the LQE server. Be selective when creating new ones. Consider whether report authors could achieve similar results through choices in Report Builder like scoping by project area, artifact type, components, or other criteria. Monitor your LQE server for performance.
Does this create a new data source?
Currently, Report Builder and the Knowledge Centre refer to these filtered type system models as “configuration-scoped data sources”, and Report Builder displays them to report authors in the same menu as the Data Warehouse and LQE data source options. To clarify, the data source is still the LQE instance where the data resides; what you are really creating is a different model of the type system for the LQE instance, instead of the comprehensive default type system model. Future releases of ELM might change the terminology and presentation to clarify this point.
It’s important to understand that the model filters the type information for reporting, not the actual data. For example, if you made a significant process and type system change in version 3, you might create a type system model based on the version 3 global configuration to use only the updated types and exclude the previous versions. You could author a report using that version 3 model, and run it against any global configuration that shares those type system characteristics – namely version 3 or later. To report on releases prior to version 3, you would use the default type system model, or define a different filtered model based on a pre-v3 configuration.
Creating filtered type system models
First determine whether you need to filter the type system model, and if so, by what configurations. Your project, process, or reporting leads are likely in the best position to make this decision based on their understanding of type system evolution and reporting needs.
Each filtered model is based on a single configuration, typically a global configuration to provide greater context and reuse. Choose configurations with type definitions that are most representative of those you plan to report on; remember, the selected configuration provides the type system, but you will use it to report on other configurations as well. A best practice is to choose a global baseline, where you can be confident the types will not change. Limit the number of type system models you define for simplicity and to minimize performance impact to the LQE server.
The LQE administrator creates the filtered type system models in Report Builder administration, starting from the data source entry for LQE scoped by a configuration. Expand the Advanced section to start the process. For detailed instructions, see this topic in the IBM Knowledge Centre.
Use the dialog to choose the configuration that has the type system you want to use for the filtered model. Although you can choose a configuration from any domain, typically a global configuration provides most versatility. Choose baselines rather than streams, as noted above. The selection dialog shown below is from ELM 7.0.1.
In the new Data Source entry, provide a meaningful name and description for the filtered type system model. If you have more than one LQE server, ensure the model name also reflects the LQE instance. Report Builder displays this name to report authors, who need to know when to use which filtered models. The default name is very similar to LQE scoped by a configuration, which might be confusing.
To avoid confusion and unexpected report results, clearly communicate the intended use for each type system model as well as its target configurations and reports.
Leave the other properties as is, and save the entry. LQE creates and saves a new type system model, filtering the default model based on the selected configuration. The default type system model remains unchanged. The filtered type system models appear in the list of Report Builder data sources for the administrator, as shown below, and also to report authors in Report Builder. The screen capture below also shows an example of default name and description values.
LQE regularly refreshes its default type system model; however, it does not automatically refresh the filtered models. If you need to update the models to reflect changes in the type system, you must manually refresh them. See Updating type filter models for more details.
Authoring reports using filtered type system models
In the current version of Report Builder, entries for the filtered type system models appear in the Data Source selection menu.
Report authors must understand what the different entries represent, and when they should choose a filtered model instead of the default. The descriptions are not visible from the Report Builder pages, so consider how best to communicate this information to your reporting teams.
After you select the type system model, Report Builder scopes the list of available artifact types and attributes based on that selection, greatly reducing conflict and duplication.
Note: It is still possible to have type conflicts, for example if the global configuration used to filter the model has contributions with inconsistent type definitions. Follow best practices for using URIs and maintaining type system consistency to minimize conflict and duplication.
Like any other report, you can then continue to limit scope by selecting project areas, artifact types, relationships, and conditions.
Provide a meaningful name and description for the report that will help report users understand when they should run this report and which configurations are valid targets.
Running reports based on filtered type system models
Report users don’t typically interact with or care about type system models. However, they do need to know what reports are valid to run against what configurations. If the report is based on a filtered model, and you run that report against a configuration with a different type system, you might get unexpected results. For example, if the report includes an attribute that does not exist in the target configuration, that column would be empty; if the report set a condition on that attribute, there might be no results at all.
Note: The same behaviour is true for reports that use the default model for LQE scoped by a configuration. For example, if a report queries for hardware requirements and the target configuration does not include hardware requirements, there will be no results.
In Report Builder, you can group reports by data source (which currently includes the filtered type system model entries) to understand the report context and review the description.
For users that interact with reports differently, like in dashboards, or who need to sort by tags due to number of reports, grouping by data source might not be sufficient. As noted above, provide meaningful names and descriptions for reports, including intended target configurations, to avoid confusion.
Users must also understand that although the report is based on the type system from a specific configuration, they can run it against other configurations with similar type systems as well. For example, you might define a report that is valid for all instances or versions of configurations for a particular client or partner. To aid users, clearly document the intended use of your filtered type system models and reports, and the different configuration contexts where they are valid. If users get unexpected results, they can verify the data source for the report and their target configuration to see if they align, and modify or select a different report if appropriate.
Modifying filtered type systems and validating reports
If your reporting needs change, you can modify an existing filtered type system model.
If you chose a stream as the basis for type filtering and the type definitions have changed, the LQE administrator must refresh the type system model from the data source entry to pick up those changes. The IBM Knowledge Centre provides detailed steps. The data source entry for the filtered model includes the time of the most recent refresh.
You can also change the configuration used for filtering, if necessary; do this with caution as it can impact existing reports. Click Choose configuration (shown in the screen capture above) to select a new configuration. Note: the selection dialog does not show the current settings, so ensure you know the correct domain, project area, and component settings if you’re just updating the configuration.
When you select a different configuration, the data source name and description revert to default text reflecting the new configuration name. Change those fields to meaningful values before clicking Save to save the changes and update or regenerate the type system model.
After any change to the filtered type system model, the LQE administrator or reporting lead should validate the reports based on that model to ensure the report definition is consistent with the updated model. Sort the reports by data source, as described earlier, then select the reports for the affected type system model and click the validate (checkmark) button. Checkmarks indicate valid reports, while an x indicates reports with potential issues, for example that use types that are not included in the updated type system. The screen capture below shows results for validating reports against both the default type system model (under Lifecycle Query Engine scoped by a configuration) and the filtered model (under LQE – using Aviary Base 1.0 types).
Report authors must then review and amend the impacted reports to work correctly with the updated type system model.
If you no longer need the filtered type system model or the reports based on it, you can archive it like any other data source. Users will not be able to run reports based on that data source or see them in the All Reports list; authors cannot use archived type system models to build new reports. Ensure you remove affected widgets from dashboards. LQE removes archived type system models from memory, but continues to store them on disk so you can restore them if needed. Updating or archiving a filtered model does not affect the LQE default type system model in any way.
Summary of best practices
Used correctly, filtered type system models can help you manage situations where you cannot avoid type system conflicts. Follow these best practices to get the most value from this capability:
- Continue to follow best practices for managing type consistency and URIs, as described in articles referenced below.
- Be selective in defining filtered type system models to minimize impact on server resources.
- Choose baselines over streams to avoid type change, and typically choose global configurations for broader scope and reuse.
- Clearly communicate intended use of filtered type system models and the reports based on them to report authors and users to minimize issues with unexpected results.
- If you must update the filtered model, validate the reports that use it and update where necessary.
For more information
- Reporting on versioned type systems in the IBM Knowledge Centre
- Maintaining the DOORS Next type system on Jazz.net
- Defining URIs for type resources on Jazz.net
- Reporting on data in configurations in the IBM Knowledge Centre
About the author
Kathryn Fryer is a Senior Solution Architect focused on the IBM Engineering Lifecycle Management solution, particularly around global configuration management. Drawing on 30 years at IBM in software development, management, and user experience, Kathryn works closely with clients and internal development and offering teams to develop usage models, aid adoption, and improve product offerings. She can be contacted at fryerk@ca.ibm.com.
© Copyright IBM Corporation 2020