(r3) ConfigurationAwareReportingTroubleshooting < Deployment

Engineering Lifecycle Management Wiki - Deployment
Community information and contribution guidelines
~~EditAttach~~Printable
TWiki > Deployment Web > DeploymentTroubleshooting > ReportingTroubleshooting > ConfigurationAwareReportingTroubleshooting
Revision 3 - 2023-08-03 - 13:18:41 - RosaNaranjo
<div id="header-title" style="padding: 10px 15px; border-width:1px; border-style:solid; border-color:#FFD28C; background-image: url(<nop>https://jazz.net/wiki/pub/Deployment/WebPreferences/TLASE.jpg); background-size: cover; font-size:120%">
---+!! Configuration aware reporting Troubleshooting<img src="https://jazz.net/wiki/pub/Deployment/WebPreferences/uc.png" alt="uc.png" width="50" height="50" align="right"> 
%DKGRAY% Authors: Main.ErnestMah, Main.RosaNaranjo <br>
Build basis: Lifecycle Query Engine (LQE), Lifecycle Index (LDX) 7.0.x
%ENDCOLOR%</div></sticky>

<!-- Page contents top of page on right hand side in box -->
<sticky><div style="float:right; border-width:1px; border-style:solid; border-color:#DFDFDF; background-color:#F6F6F6; margin:0 0 15px 15px; padding: 0 15px 0 15px;">
%TOC{title="Page contents"}%
</div></sticky>

<sticky><div style="margin:15px;"></sticky>

The objective of page is to provide an overview of troubleshooting techniques for configuration-aware reporting.

---++ Architecture
<div style="text-align:left;"><img src="%ATTACHURLPATH%/2023-08-01_12-59-00.png" alt="2023-08-01_12-59-00.png" width="50%" height="50%" /><br><br>
</div>

---++ Troubleshooting

*How can I make sure the set of data from a data provider is complete in LQE/LDX*
<br>Validation!!!!
<br>The TRS feed is essentially the contract between the data provider and the data consumer.  
<br>A data provider’s TRS defines the set of data that should be indexed by a consumer (LQE/LDX).
   * Data provider validation operation checks the TRS is accurate to the Data provider's representation.
   * Data consumer validation checks that the consumer (LQE/LDX) index (datasources) has the same set of Data.
<br>
*Limitation*
   * Does not check the content of resources  <elaborate?>
      * Global configurations can contain other global or local configurations
      * Local configurations have selection resources that define the set of versioned artifacts that are a part of the baseline or stream

*What can happen from the Data Provider (TRS)?*
   * Data Provider server or Network path temporarily down
      * LQE/LDX app logs issue fetching TRS and will retry at next interval

   * TRS change log truncation - last read change log event not present in TRS
      * LQE/LDX cannot proceed safely with indexing as there could be missing events – must reindex
<br>
*What can lead to change log truncation?*
   * a defect during change log maintenance
   * a data provider or LQE outage longer than 7 days
      * <question>why do I need to mention the 606 IFix003 improvement?  6.0.6 ifix003 – Increase resiliency of overall system by increasing change log event retention period to 14 days for the n-1 version of the TRS base document or 28 days from the current version
   * TRS content prior to LQE/LDX's last known event is modified
      *  LQE (7.0) will review one interval prior’s set of change events to see if there are any discrepancies.  If there are, it will log the discrepancies and attempt to process the events.
   * TRS provides patches that are invalid
      * Before state (eTag) does not match what is in LQE/LDX
      * LQE (7.0) will refetch the resource at a later interval to recover
    * TRS content is not accurate compared to the data provider’s repository
       * Action required is a data provider TRS validation.  for example, RM TRS validation.
<br>
<div style="text-align:left;"><img src="%ATTACHURLPATH%/2023-08-01_13-23-00.png" alt="2023-08-01_13-23-00.png" width="590" height="295" /><br>
</div>
<br>
*What can happen from the Data Consumer (LQE/LDX)?*
   * LQE/LDX server or Network path temporarily down
      * LQE/LDX resumes indexing from last known event per data provider at the next interval
<br>
   * Content of the LQE/LDX does not match what is supplied in TRS
<br>
_what is the segue here?  Is the below the answer to the question above_
<br>
*LQE/LDX data validation*
   * Compares the set of resources in the index with the TRS
   * Utilized eTags for comparisons
      * EWM TaP TRS contains eTags for all artifacts
      * Configurations modified with patch events in the changelog contain eTags (Global Config Management, RM, ETM) (RMM, SCM Future)
<br>
*What can happen fetching resources identified by TRS?*
   * Server or network temporarily down (transient)
      * LQE/LDX captures resources it is unable to fetch during the interval into the Skipped Resources list
      * Administrators can manually retry the entire skip list for recovery
      * LQE/LDX auto retries for skipped resources (*needs to be enabled by LQE or LDX Admin)
   * Application fails to return the artifact
      * Check Skipped Resource Diagnostic page from Data Provider
      * data provider complimentary diagnostics coordinated with skipped resource list _what does this mean_
      * Manage skipped resources
         * Ignore skipped resources
         * remove from TRS feed
         * provide comments for ignoring skipped resources _did we implement this?_
<br>
*Remediation of missing data*
   * Identify the data provider (ELM application) that owns the missing data
   * Ensure validation has been run at the data provider level - this operation compares the data provider TRS to the actual data in the repository
   * Validation results that show repairs can provide data provider developers an idea of where to look for issues.
   * Repairs done by the validation operation may fix the issue but *best practice* is to run a corresponding LQE/LDX validation for that data provider. 
 <br>
*In general, the idea is to use validation to keep reporting data in good shape, and fix the primary cause of any regular recurring validation issues.  And, over time, the frequency of the validation can be reduced.*
<br>
---+++ Report Builder - Data Completeness Checker
   * Focus on a single global or local configuration to determine if everything that is referenced by the configuration is available to be considered by the report
      * Unable to determine if the query itself would have included it or filtered it out (if a requirement is skipped, or missing it can’t know what are the properties/relationships of that requirement)
      * Reduces the scope of what it reports to only those referenced by the configuration selected.  Does not focus on all skipped resources.
      * looks for resources that the configuration is looking for but aren’t included in the TRS itself
   * Running it once for that configuration is enough to know the status of what may be missing which could affect any reports run against that same configuration
   * Do nott need to run it for every report
   * Slows down reports by up to 20%, which is why it should be only run on demand, not enabled across the entire system.
<br>
*What can be done with this information?*
Skipped Resources
   * Inspect the URL.  Based on the URL, are these artifacts involved in the report at all? If not, you can ignore them
   * If the artifacts are involved in the report, retry the skipped resource from LQE and observe if it can now be retrieved.
   * If not, report the issue against the ELM application providing the data
<br>
Missing Artifacts
   * This represents artifacts that were never found in the TRS from the data provider.
   * If a data provider validation does not properly fix the TRS to include this artifact, report the issue against the data provider
   * If it is in the TRS from the data provider, then run LQE/LDX validation against the datasource from the data provider
<br>
Not up-to-date artifacts
   * Go to the LQE admin view for the datasource
   * Go to the Invalid update/patch tab and attempt to repair it there
   * Work with the data provider to see if there is a way to repair the selection resource for that configuration (L3 TRS Selections Validation/Repair Tool tool for RM)
<br>
What can be done to check that selection resources are correct?
   * RM L3 TRS Selections Validation/Repair Tool (trsSelections.jar)
   * ETM - force refetch of selections via ETM TRS console
<br>
---+++LQE Query Execution details
[[Detailed Lifecycle Query Engine status information available in Report Builder][https://jazz.net/pub/new-noteworthy/jrs/7.0.2/7.0.2/index.html#11]]

---++ Heading 1

---+++++!! Related topics: [[DeploymentWebHome][Deployment web home]], [[DeploymentWebHome][Deployment web home]]

---+++++!! External links:          

   * [[https://www.ibm.com][IBM]]

<sticky></div></sticky>
    * 2023-08-01_13-23-00.png: <br />
     <img src="%ATTACHURLPATH%/2023-08-01_13-23-00.png" alt="2023-08-01_13-23-00.png" width="590" height="295" />
Attachments
Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
png	2023-08-01_12-59-00.png	manage	218.4 K	2023-08-01 - 17:02	RosaNaranjo	TRS Reporting Architecture
png	2023-08-01_13-23-00.png	manage	29.1 K	2023-08-01 - 17:23	RosaNaranjo
Copyright © by IBM and non-IBM contributing authors. All material on this collaboration platform is the property of the contributing authors.
Contributions are governed by our Terms of Use. Please read the following disclaimer.
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.