Switch to Log based TRS

Authors: WilliamChatham
Build basis: IBM Engineering Requirements Management DOORS Next, V7.0.2, V7.0.3 and (temporarily) V7.1.

The objective of page is to provide an overview and assist to Customers who wish to move from TRS Event based to TRS Log based. DOORS Next 7.1 authoritative information will be in the product documentation. For those upgrading to 7.1, we recommend starting with the interactive upgrade guide.

Why switch to Log based TRS?

TRS Event-based mode:

• Bugs leading to loss of TRS change log updates
• Server shutdown/crash can lose changes (Need to be fixed with a TRS Validation)
• Very difficult to diagnose problems

TRS Log-based mode:

• Improves TRS inconsistency (out of date configurations, skipped resources, missing/extra requirements, out of date requirements.
• Improves error-detection (Better serviceability).
• Improves TRS Validation (detect/correct a wider range of problems).
• Note performance is very much the same between Log based and Event based.
  

Log based TRS Key points:

• A full RM TRS Rebase and LQE RM Datasource Reindex/LDX Reindex is required when switching to Log based TRS from Event based TRS and vice versa (See Steps to improve Rebase and Reindex Performance below).
• in V7.0.2 and V7.0.3 You can switch back to Event based TRS at any time, This is not possible in V7.1 as Event based TRS is no longer supported in this release.
• You should use the latest available ifix when switching to log based TRS to have the latest fixes installed.
• IBM Recommends that You test Log Based(Switch and usage) in a comparable test environment before switching in production.
  • You can then gauge the time needed for Rebase and reindex.
    
    

Version specific switch to log based TRS

V7.1

New V7.1 installs, the TRS is set to Log based by default.
Customers who upgrade to V7.1 must also switch to Log based as Event based is no longer supported in V7.1.

V7.1 Switch process:

See the Interactive upgrade here:Upgrading Engineering Lifecycle Management Suite

From Engineering Lifecycle Management version 7.1 onwards, only log-based TRS mode is supported. If you were using event-based mode in Engineering Lifecycle Management 7.0.3 or any earlier versions, after upgrading to Engineering Lifecycle Management version 7.1 it is automatically updated to use log-based TRS mode. However, you need to perform TRS rebase and re-indexing for Link Index Provider and Lifecycle Query Engine.

V7.0.3

New V7.0.3 installs, the TRS is set to Log based by default.

Customers who upgrade to V7.0.3 the TRS that is set in the previous release is what is used in the upgraded V7.0.3.

Customers can switch to Log based TRS from Event based TRS in V7.0.3GA and later releases.

• For V7.0.3 the switch Does not require a Key from Support.

Update with the most recent 7.0.3 iFix before switching to log-based TRS.

V7.0.3 Switch process:

1. Navigate to https://:/rm/admin.
2. Select Advanced Properties. Search for Enable log-based TRS maintenance and set this to True.
3. Save the changes and restart the server as per the message returned : The configuration changes were saved successfully. To complete the updates, you must restart the server.
4. Clear the TRS Validation Selections cache folder on DOORS Next application server.
5. The folder location is specified by the "Selections Validation disk cache directory" advanced property in DOORS Next. (Skip this step if the property is not set (i.e. its value is empty).
6. To perform this step:
7. Stop the DN server.
8. On the DN server, go to the location specified in "Selections Validation disk cache directory" advanced property.
9. Manually delete all files in that folder.
10. Restart the server.
11. After the restart confirm Log-based TRS is enabled by checking the Advanced Properties to ensure Enable log-based TRS maintenance is set to true.
12. Also access the rmTRS.log and confirm Log-based TRS changelog maintenance is enabled is recorded, example below:

 2024-08-27T15:00:07,123+0000 [][EventAdmin Async Event Dispatcher Thread] INFO m.fronting.server.services.trs.internal.TrsService - Log-based TRS changelog maintenance is enabled.
 2024-08-27T15:00:07,358+0000 [][EventAdmin Async Event Dispatcher Thread] WARN  com.ibm.rdm.fronting.server.services.trs - Log-based mode is TRUE

Post Switch

1. Perform a full RM TRS Rebase.
2. Perform a LQE RM Datasource reindex as per: Perform a Rebase and Reindex for IBM Engineering Requirements Management DOORS Next
3. Perform a LDX Reindex as per: Perform a Rebase and Reindex for IBM Engineering Requirements Management DOORS Next
4. [can be done later than step 1 above]Perform a TRS Validation. Located on TRS Feed: DOORS Next Resources from https://server:port/rm/admin#action=jazz.viewPage&id=com.ibm.team.repository.reports.diagnostics
   
   

NOTE What is important for the switch is the rebase and the LQE RM datasource reindex and LDX RM datasource reindex.

V7.0.2

Customers can switch to Log based TRS from Event based TRS from V7.0.2 ifix025 and later releases.

• For V7.0.2 the switch requires a key from IBM Support.

Update with the most recent 7.0.2 iFix before switching to log-based TRS.

V7.0.2 Switch process:

1. Navigate to https://:/rm/admin.
2. Select Advanced Properties. Search for Log-based TRS activation key and enter the key: To be provided by IBM Support.
3. Also search for Enable log-based TRS maintenance and set this to True.
4. Save the changes and restart the server as per the message returned : The configuration changes were saved successfully. To complete the updates, you must restart the server.
5. Clear the TRS Validation Selections cache folder on DOORS Next application server.
6. The folder location is specified by the "Selections Validation disk cache directory" advanced property in DOORS Next. (Skip this step if the property is not set (i.e. its value is empty).
7. To perform this step:
8. Stop the DN server.
9. On the DN server, go to the location specified in "Selections Validation disk cache directory" advanced property.
10. Manually delete all files in that folder.
11. Restart the server.
12. After the restart confirm Log-based TRS is enabled by checking the Advanced Properties to ensure Enable log-based TRS maintenance is set to true.
13. Also access the rmTRS.log and confirm Log-based TRS changelog maintenance is enabled is recorded, example below:

2024-08-27T15:00:07,123+0000 [][EventAdmin Async Event Dispatcher Thread] INFO m.fronting.server.services.trs.internal.TrsService - Log-based TRS changelog maintenance is enabled
2024-08-27T15:00:07,358+0000 [][EventAdmin Async Event Dispatcher Thread] WARN  com.ibm.rdm.fronting.server.services.trs - Log-based mode is TRUE

Post Switch

1. Perform a full RM TRS Rebase.
2. Perform a LQE RM Datasource reindex as per: Perform a Rebase and Reindex for IBM Engineering Requirements Management DOORS Next
3. Perform a LDX Reindex as per: Perform a Rebase and Reindex for IBM Engineering Requirements Management DOORS Next
4. [can be done later than step 1 above]Perform a TRS Validation. Located on TRS Feed: DOORS Next Resources from https://server:port/rm/admin#action=jazz.viewPage&id=com.ibm.team.repository.reports.diagnostics
   
   

NOTE What is important for the switch is the rebase and the LQE RM datasource reindex and LDX RM datasource reindex.

Known issues:

Log based specific Known issues to be aware of:

No	Known Issue	Defect	Fixed in
1.	DT253617	TRS does not update changelog due to repeated exceptions dealing with project create event (145905)	V7.0.2 ifix027 V7.0.3 GA
2.	DT258794	TrsDeltaMonitorTask - Unable to process repository changes (146646)	V7.0.2 ifix028 V7.0.3 ifix003
3.	DT387232	IllegalstateException: This stopwatch is already stopped in log-based mode(147405)	V7.0.2 ifix030 V7.0.3 ifix007
4.	DT409193	TRS validation of dnext resources on Perf FGC environment has discrepancies when large module contains artifact bound more than once(148018)	V7.0.2 ifix032 V7.0.3 ifix010. Note: Can result in missing Data/links, Validation failures and skipped resources (Duplicate key).
5.	DT260176	Archive/Restore compensation in log-based mode is slow (146346)	V7.0.2 ifix029 V7.0.3 ifix004
6.	DT389482	Archiving/unarchiving large containers can block "normal" TRS updates for a very long time (147438)	V7.0.2 ifix030 V7.0.3 ifix007
7.	DT380590	Missing report data due to checksum value mismatch in subsequent RDF patches(147195)	V7.0.2 ifix030 V7.0.3 ifix007. Note: Although the checksums are incorrect, the content of the selection resource is correct, so report content is not affected by this bug.

• 1. , 2. , 3. , 4. should be in place to reduce the risk of TRS update failures.
• 5. and 6. may not be a concern if you do not plan on utilizing archive/restores of projects with a large number of configurations.

Log based and Event based Known issues to be aware of:

Known Issue	Defect	Fixed in
DT382088	DOORS Next Generation TRS Validation: Query performance is poor(147283)	V7.0.2 ifix030 V7.0.3 GA.
DT363651	ERM TRS Validation reports false positive Extra and missing resources in the feed (146949)	V7.0.2 ifix031 V7.0.3 ifix008
DT380779	TRS validation incorrectly reports missing resources (opt-out only; event-based and log-based)(147159)	V7.0.2 ifix031 V7.0.3 GA.
DT260140	TRS Validation results Summary reports Selection failures that are repaired(572188)	V7.0.2 ifix031 V7.0.3 ifix009.
DT261704	TRS Validation fails with reported error: Event with patch for Selections resource unknown to validation(572499)	V7.1. Note: Issue is rare and there is a workaround (Validation).
DT379224	TRS Validation fails due to Error encountered Validating TRS change log in com.ibm.rdm.fronting.server.trs.validation': Failed deleting TRS Selection cache file…(574208)	Open. Note: To occur, this requires a file system full during a validation and there is a workaround).

• IBM Recommends that you use the latest available ifix when switching to log based TRS to have the latest fixes installed.
  
  

Steps to improve DOORS Next Rebase and LQE/LDX Reindex Performance:

DOORS Next maintains a Tracked Resource Set(TRS). During the process of switching to the Log-Based TRS mode, it is necessary to rebase this TRS. During the process of rebase, all active configurations will be accessed as part of building the new base. This operation is time consuming, and if the rebase fails, will need to be restarted from the beginning. We advise that to ensure this procedure runs optimally, that you alter the cache that relates to unarchived configurations. This will also ensure that the users who are accessing their own configurations do not suffer when their configuration is expelled from the cache, during this time.

1. Set your configuration cache to the total number of unarchived configurations: Tuning the configuration cache for IBM Engineering Requirements Management DOORS Next V7.X

Lifecycle Query Engine Best Practices

Refer to the LQE/LDX Best Practices page for detailed guidance and best practice for exceptional use cases, such as how to optimally reindex large data sources. For example, it is possible to adjust the standard Lifecycle Query Engine configuration parameters to maximize throughput from the Tracked Resource Set(TRS) providing-applications. These settings are intentionally shared for those situations where the need to reindex, or perform an exceptional task is a trade-off for putting additional load on the applications that provide the TRS.

The two key changes to make are below and facilitate for more threads to run concurrently. The documentation states that 4 threads may be sufficient, but utilising all 16 reindexer threads, under the right circumstances will speed up the overall process.

1. Increase RAM/CPU/heap to handle re-indexer threads: See: How to speed up the reindex of Lifecycle Query Engine (LQE) data sources
2. Increase your LQE/LDX throughput for DOORS Next Datasources: See: Step 3 - Increasing Threads within a data source
   
   

Perform a full RM TRS Rebase and LQE RM Datasource reindex and LDX Reindex as per: Perform a DOORS Next rebase and data source reindex for IBM Engineering Requirements Management DOORS Next

Virtualized Windows customers should note: Initial indexing is slower than expected when LQE/LDX is running on a virtualized Microsoft Windows server

Additional contributors: