Collaborative Lifecycle Management performance report: Data Collection Component Performance (DCC) 5.0 release

Introduction

This article presents the results of the Data Collection Component (DCC) performance testing for the Rational solution for Collaborative Lifecycle Managment (CLM) 5.0 release. The DCC is introduced in the 5.0 release in order to address concerns with the current ETL solutions. The DCC can help to improve the performance of the DM and JAVA ETL by optimizing round tips to a data source and using concurrent processing to fetch data. The DCC also improved the utilization of the system resources of the server. This article will show the performance benchmark of the DCC testing and also provide a comparison with JAVA ETL and DM ETL.

Disclaimer

The information in this document is distributed AS IS. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer’s ability to evaluate and integrate them into the customer’s operational environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk. Any pointers in this publication to external Web sites are provided for convenience only and do not in any manner serve as an endorsement of these Web sites. Any performance data contained in this document was determined in a controlled environment, and therefore, the results that may be obtained in other operating environments may vary significantly. Users of this document should verify the applicable data for their specific environment.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multi-programming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

This testing was done as a way to compare and characterize the differences in performance between different versions of the product. The results shown here should thus be looked at as a comparison of the contrasting performance between different versions, and not as an absolute benchmark of performance.

What our tests measure

We use predominantly automated tooling such as Rational Performance Tester (RPT) to simulate a workload normally generated by client software such as the Eclipse client or web browsers. All response times listed are those measured by our automated tooling and not a client.

The diagram below describes at a very high level which aspects of the entire end-to-end experience (human end-user to server and back again) that our performance tests simulate. The tests described in this article simulate a segment of the end-to-end transaction as indicated in the middle of the diagram. Performance tests are server-side and capture response times for this segment of the transaction.

Methodology

The testing is based on 3 CLM repositories with the same data volume which are generated using data population tool and use DCC server, Data Manager tool client, and CLM JAVA ETL component to load the data into data warehouse. The testing runs the JAVA, DM, and DCC ETL based on one CLM repository in order to get the performance comparison of DCC compared with JAVA and DM ETL. Another run was done with the DCC ETL based on 3 CLM repositories in order to get the performance comparison of DCC with JAVA, DM ETL and DCC based on 3 CLM repositories.

Findings

Performance goals

• Make comparison of performance with DCC, JAVA and DM ETL based on one CLM repository.

• Make comparison of the performance with DCC, JAVA and DM ETL based on 3 CLM repositories concurrently.

Findings

• The DCC has two major improvements on :

• • Compared with JAVA and DM ETL loading apps in sequence, DCC ETL loads the applications in parallel. The total duration of DCC depends on the slowest app data loading. The performance of DCC is clearly improved compared with the JAVA/DM ETL loading in sequence.

• • For specific application data loading, DCC ETL does optimization by merging the same REST services to reduce the number of requests sent to the CLM Server. DCC is optimized to find the REST requests with the same relative URL path and merge them together and then send the REST request once to get the data that JAVA/DM ETL fetch by sending many REST requests. Thus for a specific application DCC ETL also has a performance improvement.

• Against one CLM repository of performance test data, DCC ETL has better performance than JAVA and DM ETL. For the specific ETL job (CCM,QM, RM and star job) the DCC has better performance than JAVA and DM ETL. Because DCC merges some REST requests the ETL job sends fewer REST requests than JAVA and DM ETL. For the details, please refer to the Results. For the total duration of one repository (the duration time of ETL to complete the JFS, QM, RRC, RTC, and star jobs) DCC has more significant improvement than JAVA and DM ETL. Because DCC runs the ETL jobs concurrently the total duration is the maximum time among the ETL jobs. But the JAVA and DM ETL run the ETL jobs in sequence. For the detail comparison data, please refer to Results

• Against three CLM repositories of performance test data (each repository has the same data volume and is deployed with same configuration) DCC ETL shows a major performance improvement because DCC provides the capability to load the data from multiple CLM repositories concurrently. For the detail comparison data, please refer to Results

Topology

The topology under test is based on Standard Topology (E1) Enterprise - Distributed / Linux / DB2.

The specifications of machines under test are listed in the table below. Server tuning details are listed in Appendix A

Test data was generated using automation. The test environment for the latest release was upgraded from the earlier one by using the CLM upgrade process. Please see the system configuration in the under table. And refer to the server tuning details listed in Appendix A

IBM Tivoli Directory Server was used for managing user authentication.

Data volume and shape

The data volume listed in Appendix B

Network connectivity

All server machines and test clients are located on the same subnet. The LAN has 1000 Mbps of maximum bandwidth and less than 0.3 ms latency in ping.

Results

ETL Comparison of DCC\JAVA\DM based on one CLM repository

This part is targeted to determine the performance gain of DCC ETL compared with JAVA ETL and DM ETL based on one CLM repository. Before CLM 5.0, CLM provided JAVA ETL to load the CLM data from the applications one by one who registered on the JTS server. One typical CLM deployment has one JTS, CCM, QM and RM application registered on the JTS. The JAVA ETL will load the data in sequence with following apps: JTS, CCM,QM, RM and Star. The total duration is the sum of ETL duration of all applications. So does DM ETL. DCC provides the capability to load the applications data in parallel. Thus the total duration is the slowest duration among the ETLs of specific applications.

The chart shows the performance gain of DCC compared with JAVA and DM ETL based on one CLM repository. The DCC has two major improvements:

1. DCC performance has a significant improvement per the comparison of JAVA and DCC ETL based on performance test data. The duration reduced from 48 hours to 16 hours. For the duration of specific applications the DCC also has a significant improvement. It improved about 30% on QM loading, 20% on RM loading, 60% on CCM loading and 90% on Star job.

2. DCC duration also has a significant improvement per the comarison of DM and DCC ETL based on performance test data. The duration reduced from 139 hours to 16 hours. The major improvements are the RRC ETL and RQM ETL. RQM loading improved about 60% and RRC loading improved about 85%.

Row 1. DCC : The DCC load the CLM apps data concurrently based on one CLM repository.

Row 2. JAVA : JAVA ETL load the CLM apps data sequentially based on one CLM repository.

Row 3. DM : DM ETL load the CLM apps data sequentially based on one CLM repository.

• Precondition: CCM ETL has one build named as Workitembaseline which records the latest info of each workitem by getting the latest workitem history record. When the workitembaseline ETL build is running, the ETL gets the latest info (status, state, priority, severity, etc.) by requesting the latest WI history with the query condition that the change time of the WI history is earlier than the ETL build start time. If there is no ETL schedule one day, the latest info of each workitem on that day are not loaded in the worktiembaseline table of DW. JAVA ETL will fill the WI latest info on the days that have no ETL running. For example, if there is no JAVA ETL run on Jan 1st, the workitemBaseline table won't have chance to load the workitem latest info into DW. However, the next JAVA ETL run on Jan 2nd will insert the missing data on Jan 1st. The performance team always uses the same data set to do the ETL performance so that the performance results are comparable. This feature will cause the ETL get more workitembaseline data for the JAVA Full ETL load along with time passed. That means the baseline data of 4.0.6 is slightly more than that in 4.0.5, and that the data of 4.0.5 is slightly more than that in 4.0.4, etc. To improve the comparability of performance data release by release, we insert one pseudo record so that the ETL only inserts a single day's baseline. We get the same number of workitembaseline ELT builds inserted by this way.

ETL Comparison of DCC\JAVA\DM based on three CLM repositories

This part is targeted to determine the performance gain of DCC ETL compared with JAVA ETL and DM ETL based on multiple CLM repositories. In theory, the DCC will save more time of ETL along with more CLM repositories to be loaded because DCC can load all of the CLM applications concurrently.

In this test senario DCC loads 3 CLM repositories data (one CCM, QM, and RM application registered per CLM repository) concurrently. The DCC has two major improvements:

1. DCC performance has a significant improvement per the comparison of JAVA and DCC ETL based on 3 CLM repositories. The duration reduced from 144 hours to 34 hours. And for the duration of specific applications, the DCC also has a significant improvement. It improved about 30% on QM loading, 20% on RM loading, 60% on CCM loading, and 90% on Star job.

2. DCC duration also has a significant improvement per the comparison of DM and DCC ETL based on 3 CLM repositories. The duration reduced from 416 hours to 16 hours (96% improved). For the duration of specific applications the DCC also has a significant improvement. It improved about 70% on QM loading, 90% on RM loading, and 55% on CCM loading.

Row1. DCC_3_Repo : The DCC loads 3 CLM repositories data concurrently.

Row2. JAVA_3_Repo_Sequently : The JAVA ETL loads 3 CLM repositories in sequence. For one CLM repository loading JAVA ETL loads the apps in sequence as well.

Row3. DM_3_Repo_Sequently: The DM ETL loads 3 CLM repositories in sequence. For one CLM repository loading the DM ETL loads the apps in sequence as well.

Appendix A - Server Tuning Details

    db2 update db cfg using LOGFILSIZ 40960
    db2 update db cfg using LOGPRIMARY 50
    db2 update db cfg using LOGSECOND 100

-verbose:gc -XX:+PrintGCDetails -Xverbosegclog:gc.log -Xgcpolicy:gencon 
-Xmx8g -Xms8g -Xmn1g -Xcompressedrefs -Xgc:preferredHeapBase=0x100000000
-XX:MaxDirectMemorySize=1g

    db2 update db cfg using LOGFILSIZ 40960

Appendix B - Data Volume and Shape

N/A: Not applicable.

For more information

• Collaborative Lifecycle Management 2012 Sizing Report (Standard Topology E1)

About the authors

PengPengWang

Questions and comments:

• What other performance information would you like to see here?
• Do you have performance scenarios to share?
• Do you have scenarios that are not addressed in documentation?
• Where are you having problems in performance?

Warning: Can't find topic Deployment.PerformanceDatasheetReaderComments