ELM 7.3.0 TRS Resumable Validation Performance Report

Introduction

In IBM Engineering Lifecycle Management (ELM) 7.3.0, a new Resumable Tracked Resource Set (TRS) Validation feature is introduced. Tracked Resource Set (TRS) validation is an essential administrative process that ensures data consistency and synchronization between Lifecycle Query Engine (LQE) / Link Index Provider (LDX) and application repositories. In large enterprise repositories containing millions of resources, standard TRS validation can run for several hours. Previously, any unexpected disruption, scheduled maintenance, or server restart required cancelling and restarting the entire validation process from the beginning.

With the Resumable TRS Validation feature, TRS validation can be cleanly paused and resumed within a single end-to-end run without data loss, duplication, or re-processing already validated phases. If a validation run is interrupted by a planned or unplanned server restart, processing can resume from the last persisted checkpoint once the server is back online.

This article consolidates the comprehensive performance, scalability, and correctness validation results for Resumable TRS Validation across the ELM 7.3.0 solution, encompassing both:

1. LQE-Side Resumable Window Validation: LQE-driven validation processing pre-built validation lists in configurable memory-to-disk windows across application data providers (DOORS Next, ETM, and EWM).
2. Application-Side Resumable Validation: Validation executed on application repositories (such as DOORS Next, ETM, and EWM). In ELM 7.3.0, individual applications have not yet adopted application-controlled chunking; instead, the entire resumable capability (checkpoint persistence, state serialization, pause/resume/restart recovery) is handled and managed directly by the Jazz Foundation layer.

Architectural Implementation in ELM 7.3.0

The Resumable TRS Validation feature operates across two primary architectural tiers:

• LQE Window Validation Engine: When resumable validation is enabled on LQE, LQE processes validation lists in configurable chunk windows. URIs fetched from the TRS feed are retained in an in-memory buffer up to the configured window threshold before being sorted and flushed to temporary chunk files on disk, followed by multi-way merge passes against the LQE Relational Store.
• Jazz Foundation Application Resiliency: At the application server tier, individual applications have not yet adopted application-controlled chunking. The resumable engine is implemented and managed within the Jazz Foundation layer. Foundation manages phase state transitions, checkpoint serialization to a designated persistent volume (`validation--state.json`), and pause/resume/recovery lifecycle controls.

Configuration Properties & Parameters

1. LQE-Side Window Validation Configuration Parameters (Configured on LQE Server)

These parameters apply specifically to LQE-side Window Validation and are configured independently per data provider in LQE administration:

2. Application-Side Resumable TRS Validation Properties (Configured on JTS / Application Server)

These properties apply specifically to application-side TRS validation (DOORS Next, ETM, EWM) and are configured through Jazz Team Server (JTS) Advanced Properties:

Operational Clarification for Administrators: You do not need to configure both tables unless you are executing validations across both tiers. If you are validating from the LQE administration console, configure the LQE parameters (Table 1). If you are validating from an application side (e.g., DOORS Next, ETM, EWM), configure the JTS properties (Table 2).

Standard Disclaimer

The information in this document is distributed AS IS. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer’s ability to evaluate and integrate them into the customer’s operational environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk. Any pointers in this publication to external Web sites are provided for convenience only and do not in any manner serve as an endorsement of these Web sites. Any performance data contained in this document was determined in a controlled environment, and therefore, the results that may be obtained in other operating environments may vary significantly. Users of this document should verify the applicable data for their specific environment.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multi-programming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

This testing was conducted to characterize the performance, durability, overhead, and recovery time of the Resumable TRS Validation feature across ELM repositories. The data presented should be evaluated as comparative performance guidance rather than an absolute SLA for all customer environments.

Key Findings

Performance, scalability, and interruption recovery testing of the Resumable TRS Validation feature across LQE and application tiers yielded the following key conclusions:

• LQE-Side Validation Results:
  • Performance Impact of Enabling Window Validation: Using the default window configuration (50K/50K/64), performance was compared with Window Validation disabled and enabled. ETM (+2.4% / +1m 40s) and EWM (+7.1% / +4m 22s) showed no measurable performance overhead, as the observed differences remained within normal run-to-run variability. However, for the DOORS Next (RM) data provider, which contained approximately 3.7 million resources, enabling Window Validation increased the mean validation duration from 1h 09m 47s to 1h 29m 08s, an increase of 19m 21s (+27.7%). This increase exceeded the normal run-to-run variability threshold for RM.
  • Pause-and-Resume Overhead: After excluding the planned pause intervals, pausing and resuming validation introduced no measurable additional active processing overhead across the RM, ETM, and EWM data providers. Validation correctness was maintained in all executions, with 0 missing resources and 0 stale resources.
  • Storage Footprint: During chunk files creation Validation directory max disk consumption was ~2 GB for ~3.7M resources .

• DOORS Next (DNG) Application Results:
  • Zero Performance Regression in Normal Execution: Enabling the resumable validation feature does not slow down normal, uninterrupted validation. With the feature enabled, uninterrupted validation took 51m 26s (Run 2), which is practically identical to running with the feature disabled (56m 15s in Run 1) and the pre-delivery baseline (53m 23s). Across multiple repeated test runs, normal execution naturally varied by about 3 to 5 minutes, confirming that background checkpoint tracking adds no measurable performance penalty during normal execution.
  • Durable Interruption Recovery: Validation state persisted intact across 10-minute UI pause intervals and full application server restarts, resuming cleanly from saved checkpoints without duplicate work.
  • Transparent Overhead Analysis: All reported validation durations reflect active validation processing time (excluding the 10-minute hold windows). Pause/Resume (1h 07m 47s) and Pause/Server Restart/Resume (1h 10m 03s) added ~11–14 minutes of active processing overhead for checkpoint serialization and recovery verification—vastly superior to the multi-hour cost of restarting validations from scratch.
  • Clean Cancellation: Cancelling paused validations halted all processing, released memory, and allowed subsequent fresh runs to start from scratch and complete cleanly in 48m 25s.
  • Storage Footprint: State directory disk consumption was ~200 MB for ~1.5M resources (~500 MB to 1 GB recommended allocation).

LQE-Side TRS Resumable Testing

This section summarizes the LQE TRS Resumable Validation results across three data providers. The evaluation focused on performance impact, pause-and-resume behaviour, and validation correctness. All testing was conducted on the ELM 7.3.0 M10 release using an Oracle database backend and an LQE Relational Store (RS) database.

Data Shapes & Provider Scale

All validation activities were executed against a pre-indexed LQE Relational Store (LQE RS) that was fully rebased with each application server.

The default Window configuration used throughout testing:

• Window Count: 50,000
• LQE Validation Batch Size: 50,000
• LQE Validation Merge Fan-In: 64

LQE-side validation Results summary

Window Validation (50K/50K/64) was tested across three data providers covering ~2.1 and 3.7 million resources.

Performance: The default (50K/50K/64) window configuration is performance-neutral for ETM (QM) and EWM (CCM) data providers. For DN (RM), a significant overhead of ~27.7%(+19m 21s) was observed — 5.8× above the normal run-to-run variation.

LQE Pause-and-Resume Timing Breakdown:

Pause/Resume: Across all three data providers, the pause-and-resume functionality operated as expected. After excluding the intentional pause duration, the effective processing time remained consistent with comparable windowed validation runs. No additional overhead was introduced by the pause/resume operation itself.

Correctness Summary: All validation runs across all three providers and scenarios completed with 0 missing resources and 0 stale resources.

DOORS Next Resources (RM) Data Provider Validation

Scenario 1 — Feature OFF vs. Window Validation (Feature ON)

Individual Run Results (DOORS Next RM):

Scenario 2 — Pause and Resume Validation (RM)

Configuration: Window count=50,000 / LQE Batch=50,000 / FanIn=64

• Total validation duration (incl. pause): 1h 39m 30s
• Pause duration: ~14m 24s
• Active processing time (total – pause): ~1h 25m 06s

Observations

Scenario 1: Enabling Window Validation for the RM data provider resulted in a measurable performance overhead. The mean validation duration increased from 1h 09m 47s with the feature disabled to 1h 29m 08s with the feature enabled, representing an increase of 19m 21s (+27.7%). This overhead was approximately 5.8× higher than the expected run-to-run variability threshold, confirming a significant performance impact.

Scenario 2: After excluding the planned pause duration of 14m 24s, the active processing time was approximately 1h 25m 06s. This result was within the Scenario 1 Window Validation run range of 1h 24m 17s to 1h 28m 33s, indicating that the pause-and-resume operation did not introduce additional processing overhead. Validation correctness was maintained with zero missing resources and zero stale resources.

CPU Utilization:

• Oracle database server average CPU utilization remained low at approximately 5% throughout the validation run.
• LQE server average CPU utilization remained low at approximately 1.1% throughout the validation run.
• DN application server average CPU utilization remained low at approximately 0.5% throughout the validation run.

Disk Utilization:

• Oracle database server average disk utilization was approximately 7.2% throughout the validation run.
• LQE server average disk utilization was approximately 0.8% throughout the validation run.
• DN application server average disk utilization was approximately 0.4% throughout the validation run.

JVM Heap Utilization:

• Heap utilization remained well below the configured 32 GB JVM heap throughout the validation run, with consistently low GC pause times of approximately 0.2 seconds.

ETM Resources (QM) Data Provider Validation

Scenario 1 — Feature OFF vs. Window Validation (Feature ON)

Individual Run Results (ETM QM):

Scenario 2 — Pause and Resume Validation (QM)

Configuration: Window count=50,000 / LQE Batch=50,000 / FanIn=64

• Total validation duration (incl. pause): 1h 20m 59s
• Pause duration: ~10m 18s
• Active processing time (total – pause): ~1h 10m 41s

Observations

Scenario 1: Enabling Window Validation for the ETM(QM) data provider did not result in a measurable performance impact. The mean validation duration increased from 1h 11m 01s with the feature disabled to 1h 12m 42s with the feature enabled, representing an increase of 1m 40s (+2.4%). This increase was only 0.25× the expected run-to-run variability threshold, confirming that the difference was within normal execution variation.

Scenario 2: After excluding the planned pause interval of approximately 10m 18s, the active processing time was approximately 1h 10m 41s. This was consistent with the Scenario 1 Window Validation run range of 1h 10m 42s to 1h 14m 02s, indicating that the pause-and-resume operation did not introduce additional processing overhead. Validation correctness was maintained, with zero missing resources and zero stale resources.

Resource Utilization (ETM Validation)

CPU Utilization:

• Oracle database server average CPU utilization remained low at approximately 5% throughout the validation run.
• LQE server average CPU utilization remained low at approximately 1.5% throughout the validation run.
• ETM application server average CPU utilization remained low at approximately 2% throughout the validation run.

Disk Utilization:

• Oracle database server average disk utilization was approximately 8% throughout the validation run.
• LQE server average disk utilization was approximately 1% throughout the validation run.
• ETM application server average disk utilization was approximately 0.3% throughout the validation run.

JVM Heap Utilization:

• Heap utilization remained well below the configured 32 GB JVM heap throughout the validation run, with consistently low GC pause times of approximately 0.2 seconds.

EWM Tracking & Planning Resources (CCM) Data Provider Validation

Scenario 1 — Feature OFF vs. Window Validation (Feature ON)

Individual Run Results (EWM CCM):

Scenario 2 — Pause and Resume Validation (EWM)

Configuration: Window count=50,000 / LQE Batch=50,000 / FanIn=64

• Total validation duration (incl. pause): 1h 15m 08s
• Pause duration: ~14m 00s
• Active processing time (total – pause): ~1h 01m 08s

Observations

Scenario 1: Enabling Window Validation for the EWM (CCM) data provider did not result in a measurable performance impact. The mean validation duration increased from 1h 01m 35s with the feature disabled to 1h 05m 56s with the feature enabled, representing an increase of 4m 22s (+7.1%). This increase was only 0.14× the expected run-to-run variability threshold, confirming that the difference was within normal execution variation.

Scenario 2: After excluding the planned pause interval of approximately 14m, the active processing time was approximately 1h 01m 08s. Although this was slightly higher than the typical Scenario 1 Window Validation run range of 55m 00s to 57m 02s, the difference remained within normal run-to-run variability. Therefore, no measurable additional processing overhead attributable to the pause-and-resume operation was observed. Validation correctness was maintained, with zero missing resources and zero stale resources.

Resource Utilization (EWM Validation)

CPU Utilization:

• Oracle database server average CPU utilization remained low at approximately 11.8% throughout the validation run.
• LQE server average CPU utilization remained low at approximately 1.5% throughout the validation run.
• EWM application server average CPU utilization remained low at approximately 2.9% throughout the validation run.

Disk Utilization:

• Oracle database server average disk utilization was approximately 13.1% throughout the validation run.
• LQE server average disk utilization was approximately 0.6% throughout the validation run.
• EWM application server average disk utilization was approximately 0.003% throughout the validation run.

JVM Heap Utilization:

• Heap utilization remained well below the configured 32 GB JVM heap throughout the validation run, with consistently low GC pause times of approximately 0.2 seconds.

DOORS Next (DN) Application TRS Resumable Testing

This section documents the performance characterization and interruption recovery validation of the Resumable TRS Validation feature executing on the IBM Engineering Requirements Management DOORS Next (DNG) application repository.

DN Results Summary

Performance testing was executed on the ~1.5M resource DOORS Next repository across baseline, uninterrupted, and interruption scenarios. In all interruption tests (Run 3, Run 4, and Run 5), an intentional 10-minute pause interval was introduced during active processing. The validation times shown in the table below reflect the *active validation execution time* (excluding the 10-minute pause interval):

Note: Each feature-enabled scenario was executed 3 to 4 times to establish confidence, consistently showing a normal run-to-run variation of 3 to 5 minutes.

Detailed DOORS Next Scenario Breakdown

1. Baseline & Uninterrupted Validation (Run 1 & Run 2)

• Flow: `Start Validation` → `Uninterrupted Execution` → `Complete`
• Observations: Run 1 (Feature Disabled) completed in 56m 15s; Run 2 (Feature Enabled, uninterrupted) completed in 51m 26s.
• Conclusion: Active checkpoint tracking incurs negligible overhead during continuous, uninterrupted validation runs, remaining well within the normal run-to-run variation bounds of 3–5 minutes.

2. Pause and Resume Lifecycle (Run 3)

• Flow: `Start Validation` → `Pause (UI)` → `Hold 10 Minutes` → `Resume (UI)` → `Complete`
• Pause Duration: 10 minutes (intentional hold interval)
• Active Validation Time: 1h 07m 47s (excludes the 10-minute hold duration)
• Observations: Triggering Pause transitioned validation to `PAUSED` cleanly, halting queries against DOORS Next. The validation was held in this paused state for 10 minutes. Upon clicking Resume, processing resumed directly from the saved checkpoint without repeating earlier phases. The active duration of 1h 07m 47s reflects ~11 minutes of checkpoint serialization, state flush, and worker re-synchronization overhead.

3. Pause, Cancel, and Fresh Restart (Run 4)

• Flow: `Start Validation` → `Pause (UI)` → `Hold 10 Minutes` → `Cancel (UI)` → `Start Fresh` → `Complete`
• Pause Duration: 10 minutes (intentional hold interval)
• Active Validation Time: 48m 25s
• Observations: Validation was paused, held for 10 minutes, and then cancelled. Cancellation immediately halted background workers, freed allocated memory, and reset checkpoint tracking. Triggering a new validation started completely fresh from scratch with no stale checkpoint data carried over, completing successfully in 48m 25s.

4. Pause, Server Restart, and Post-Restart Recovery (Run 5)

• Flow: `Start Validation` → `Pause (UI)` → `DOORS Next Server Restart` → `Resume (UI)` → `Complete`
• Pause & Maintenance Window: 10 minutes (intentional hold during server restart)
• Active Validation Time: 1h 10m 03s (exact active validation time, excluding the 10-minute pause window)
• Observations: Validation was paused during `VALIDATE_RESOURCES`, and the DOORS Next application server (`rm3`) was restarted. After the server came back online, the checkpoint state file on persistent disk was verified intact. Clicking Resume in the admin UI re-engaged validation from the exact pre-restart checkpoint, completing with status `COMPLETED`.

ETM TRS Resumable Testing

Note: ETM application-tier resumable validation test results will be added here upon completion of testing.

EWM TRS Resumable Testing

Note: EWM application-tier resumable validation test results will be added here upon completion of testing.

Test Environment Details & Hardware Specifications

The performance test topology comprises physical enterprise servers hosting the complete ELM 7.3.0 solution stack:

Abbreviations

• JTS: Jazz Team Server
• GCM: Global Configuration Management
• DN / RM: DOORS Next / Requirements Management
• ETM / QM: Engineering Test Management / Quality Management
• EWM / CCM: Engineering Workflow Management / Change and Configuration Management
• LQE RS: Lifecycle Query Engine (Relational Store)
• TRS: Tracked Resource Set
• SLA: Service Level Agreement