TRS Validation

Authors: WilliamChatham, PetruAcsinte, YuriyYermakov, IanGreen, PaulEllis, IanBarnard
Build basis: V7.0.2, V7.0.3 and V7.1.

The objective of page is to provide an overview of the TRS Validation including troubleshooting.

Why Validate?

---

The purpose of the Tracked Resource Set (TRS) validation is to increase your confidence in the quality of your reports and help troubleshoot missing, stale, or extra artifacts seen in reports from the Lifecycle Query Engine (LQE) or LDX (Lifecycle index). Validation helps you diagnose and troubleshoot problems and avoid reindexing data providers. Reports can be affected in the following way:

• A report is missing artifacts.
• A report includes outdated (stale) artifacts.
• A report shows the same artifact more than once, possibly with different attribute values or with its old and new name.
• An application view and a corresponding report in Jazz Reporting Service (JRS) show a different number of artifacts.
• A report includes deleted artifacts.
• LQE or LDX shows skipped resources.
  

See here for more details on validation:Validating TRS feeds and the LQE or LDX index

TRS Validation is also a much better alternative to performing a Full TRS rebase and LQE Datasource reindex which can take a long time to complete.

How to run a TRS Validation:

---

Running a TRS Validation may take a long time to complete and will put additional load on the server.

• We recommend running a validation when server usage is low, such as during a scheduled maintenance window.
• There is no way to stop a validation once it starts except by shutting down the application server.
  • Note: The validation will not automatically resume when the server restarts, the validation must be started again after clearing the TRS cache (As its in an unknown state).
• Validation results are based on the data from when the validation starts, so if users make changes these changes will not be picked up and validated.
  • This is why we recommend running during a scheduled maintenance, otherwise, inaccurate validation results can be reported.
  • Running a validation during working hours is best to run against baselines only as these are immutable so cannot be affected by changes.

Procedure to run a TRS Validation:

Open the applications administration page: https://host:port/{app}/admin. On the toolbar, click TRS Feed Diagnostics. In the Validate dialog box, click Validate artifacts and configurations.

Validation options explained:

---

Validate artifacts option:

---

The Validate artifacts option can be used to verify that resources in the TRS can be fetched from the current application. Use this option if LQE is detecting 'skipped' resources. This option is mandatory when validating all configurations in the TRS. When validating specific configurations, only configurations and their members are validated.

Resolve problems option:

---

Sometimes, applications might choose not to provide any repair capabilities in TRS Validation. So, this option is available only when application supports TRS feed repair. You can use the Resolve problems option in the Validate dialog box to automatically resolve discrepancies, if any. The checkbox is selected by default. This option is displayed only for the TRS feeds that support repair operation.

When there are unresolved discrepancies, you can see a list of

• Missing or extra artifacts.
• Unknown resource for Selections patch.
• Wrong patch eTag.
• Unexpected artifact added by a patch, etc.

The validation results are appended to the existing validation summary. You can view the number of repaired, missing, and extra artifacts in the feed. These artifacts are listed in separate sections.

1. If all issues are resolved, then the validation result shows a green check mark.
2. If there are discrepancies found that were fixed, then the validation result shows a Yellow Triangle.
3. If there are discrepancies to fix, then the validation result shows a red cross.

If you see discrepancies in reports after you validate the feed, consider running a full validation. This operation takes longer because it examines every entry in the TRS feed, discards all caches and starts with the beginning. The artifacts that cannot be repaired are listed as missing or extra artifacts. If you still see discrepancies, even after running the full validation, contact IBM Support.

Clear cached validation data option:

---

When you are validating TRS feed, selecting this option, erases cached data that is recorded during previous validations that pertains to the current validation. Cached data helps to improve TRS Validation performance.

Use this option in the following cases:

1. If the application server is restarted while a validation as running as it is in an unknown state.
2. After the TRS has been rebased or truncated, either when done manually, or as part of a scheduled periodic task.
3. Following a validation run with incompatible options (because this means the validation cache is stale), such as, after you accidentally cleared the Validate configurations checkbox on the previous validation but now want to validate configurations, or if the validation API was used to validate (because this can't validate configurations). Also select this option if you suspect that previous validations reported incorrect failures due to inconsistent validation options, such as when you tried validating a configuration with an outdated configuration cache entry. (This might happen when you stop validating all configurations when you are validating TRS and then decide to validate configurations)
4. Selections Validation disk cache directory location change.
   
   

Throttling the TRS validation

TRS validation of the TRS providers, such as DOORS Next, or Engineering Test Management (ETM), is expected to be run as frequently as required. If the validation is regularly resolving problems when run monthly, or if you are experiencing user reported incidents that match the "Why Validate?" section above, then you need to validate more frequently, for example, weekly.

Whether to use throttling depends very much on your environment. Here are considerations, for evaluating whether you need to throttle:

• How many TRS providers do you need to validate within your IBM ELM ecosystem...at the same time?
• How many CPU do the machines running the TRS providers have?
• How many IBM HTTP Servers (IHS) do you have/How many IHS connections are available?
• What other off-hours automation are you running? This includes custom, in-house as well as say DCC jobs.
• Active Services is showing that each TRS validation thread is taking many minutes/hours for each thread

If, for example, you have multiple large ELM repositories of DOORS Next and ETM on the same shared resources, then you need to pay attention to the amount of time a validation takes/the overall CPU on the application server used and also if other users running their own scripts are complaining of longer durations to complete their scripts.

To throttle TRS validation feeds, you should:

• Login to https://host:port/{application}/admin
• Find Maximum threads for resource validation
• Find TRS Validation thread limit for repository queries
• Specify 25% of your available CPU for both these settings.

Note: the default for both values is 0, which equates to 100% of your CPU. The 25% recommendation is a starting point. Reduced CPU allocation could result in a longer time to complete. However, if the number of TRS validation threads was causing issues, then the overall time may well improve when throttled. Little's Law has been encountered when all maintenance jobs are scheduled off-peak for users, creating a new peak for maintenance.

When not to run validation

The Tracked Resource Set can become inconsistent in certain circumstances when configuration restores and archives are performed whilst the server is restarted. If there is a server restart during archiving or restoration of configurations, then you must run TRS Validate.

However, you should not validate during archive/restore, as you will be making changes that could cause the validate to report false positives, or fix (reverse) the action the restore/archive was performing.

Reading Validation results:

---

The results of each validation is stored in the GUI (By default, results are retained for 28 days) and the summary confirms how many records, artifacts and selections were validated and what failed validation and what was resolved. The validation options you selected are also recorded. Example below:

So in the example above:

• Out of 4788 Artifacts validated 4786 were successful and 2 failed validation and those 2 were resolved.
• Out of 27 Selections validated 13 were successful and 14 failed validation and could not be resolved so will need to be reviewed.
  
  

Troubleshooting:

---

Logging:

The TRS Validation logs out to the application log e.g. rm.log which can roll over very quickly on a busy server so its best to have the validation log out to its own log (rmTRS-Validation.log). You can do this by adding the following to the log4j2.xml located here: [installDir]/server/conf/{app}/. Log4j2.xml changes are dynamically read, so there is no need to restart the server.

In the Appenders section, define a new appender by introducing:

<RollingFile name="rmTRSValidation" filename="${dir}/${app}TRS-Validation.log" filePattern="${dir}/${app}TRS-Validation-%i.log">
   <PatternLayout pattern="${pattern}"/>
   <Policies>
      <SizeBasedTriggeringPolicy size="10 MB"/>
   </Policies>
   <DefaultRolloverStrategy max="5"/>
</RollingFile>

In the Loggers section, include the following three loggers:

<Logger name="com.ibm.rdm.fronting.server.services.trs.internal.validation" additivity="false" level="INFO">
   <AppenderRef ref="rmTRSValidation"/>
</Logger>
<Logger name="com.ibm.team.repository.service.internal.ReportsDiagnosticsRestService" additivity="false" level="INFO">
   <AppenderRef ref="rmTRSValidation"/>
</Logger>
<Logger name="com.ibm.team.repository.service.internal.trs" additivity="false" level="INFO">
   <AppenderRef ref="rmTRSValidation"/>
</Logger>

DOORS Next specific TRS Validation:

---

• Engineering Requirements Management (ERM) DOORS Next TRS Validation

Engineering Test Management specific TRS Validation:

---

• Engineering Test Management (ETM) TRS Validation

External links:

• Validating TRS feeds and the LQE or LDX index
• Troubleshooting configuration-aware reporting

Additional contributors: BarryReilly