(r18) CollaborativeLifecycleManagementPerformanceReportRQM60Release < Deployment

~~EditAttach~~Printable
r18 - 2015-05-20 - 15:31:56 - Main.hhuoYou are here: TWiki >
Deployment Web > DeploymentPlanningAndDesign > PerformanceDatasheetsAndSizingGuidelines > CollaborativeLifecycleManagementPerformanceReportRQM60Release
<div id="header-title" style="padding: 10px 15px; border-width:1px; border-style:solid; border-color:#FFD28C; background-image: url(<nop>https://jazz.net/wiki/pub/Deployment/WebPreferences/TLASE.jpg); background-size: cover; font-size:120%">
<!--
   * Set ALLOWTOPICCHANGE = Main.TWikiDeploymentDatasheetsAuthorsGroup, Main.TWikiAdminGroup, Main.GrantCovell 
-->
---+!! <img src="https://jazz.net/wiki/pub/Deployment/WebPreferences/uc.png" alt="uc.png" width="50" height="50" align="right">(DRAFT) Collaborative Lifecycle Management performance report: Rational Quality Manager 6.0 release (DRAFT) </br>
%DKGRAY% Authors: [[Main.HongyanHuo][Hongyan Huo]], [[Main.VaughnRokosz][Vaughn Rokosz]]</br>
Last updated: May 20, 2015</br>
Build basis: Rational Quality Manager 6.0 
%ENDCOLOR%</div></sticky>

<!-- Page contents top of page on right hand side in box -->
<sticky><div style="float:right; border-width:1px; border-style:solid; border-color:#DFDFDF; background-color:#F6F6F6; margin:0 0 15px 15px; padding: 0 15px 0 15px;">
%TOC{title="Page contents"}%
</div></sticky>

<sticky><div style="margin:15px;"></sticky>

---++ Introduction

This report compares the performance of <b>Rational Quality Manager (RQM) </b> version 6.0 to the previous 5.0 release.    The goal of the test was the verify that the performance of the 6.0 operations were the same or better than their 5.0.2 equivalents.

The test methodology involves these steps:
   * Collect the standard one-hour performance test data using 1,000 concurrent users from RQM 5.0 (repeat 3 times)
   * Repeat the same standard one-hour performance test load against RQM 6.0 using 1,000 concurrent users (repeat 3 times)
   * Compare the 6 runs 

The 6.0 tests do not include any of the new 6.0 features (such as the configuration management capabilities), which allows the performance workload and data repositories to be the same for 5.0.2 and 6.0. 

---+++!! Disclaimer

%INCLUDE{"PerformanceDatasheetDisclaimerEndToEnd"}%

---++ Summary of performance results

Most operations in 6.0 are equivalent or better than their [[CollaborativeLifecycleManagementPerformanceReportRQM50Release][5.0]] equivalents.   The chart below is a high-level summary, showing a histogram of the absolute time differences (in milliseconds) between the 6.0 and 5.0 operations.    We consider "equivalent" to be if the response time difference when comparing an operation between the two releases is within 100 ms.    Out of the 97 operations we measured, all but one are equivalent or better.    20 show significant improvement.

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/histogram_pageDegrImpr.png" />

There are improvements in dashboard loading, test plan loading, and editing of various artifacts.

The one operation which has degraded is still relatively fast (around 1 second total):
   * <b>Search For Strings</b> ([[#FullTextSearch][Full Text Search]]) - degraded by 900 ms (to 1.1 s)
   
There are defects tracking this issue, and are under consideration to be addressed in future releases.

For more details on the 1000 user workload, see this section:   [[#User_roles_test_cases_and_worklo][User roles, test cases and workload characterization ]].

For the detailed performance results for each use case, see this section:    [[#DetailedResults][Detailed performance results]].

---+++ Summary of OS resource utilization
Comparing system resource data for both 6.0 release and [[CollaborativeLifecycleManagementPerformanceReportRQM50Release][5.0]], 6.0 has
   * Slightly higher CPU on application server than 5.0 
   * Reduced disk and CPU utilization on DB server than 5.0
   * Higher system memory consumptions on both DB and application servers than 5.0
   * Similar network I/O with 5.0

The details are provided in section [[#Resource_utilization][Resource utilization]]

---++ Appendix A:  Topology

The topology under test is based on [[RecommendedCLMDeploymentTopologies#CLM_E1_Enterprise_Distributed_Li][Standard Topology (E1) Enterprise - Distributed / Linux / DB2.]]

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM50Release/ServerOverview.png" alt="ServerOverview.png"width="40%" height="40%" />

The specifications of machines under test are listed in the table below. Server tuning details listed in *[[#Appendix_A][Appendix A]]*

<table class="gray-table">
	<tbody>
		<tr>
			<th valign="top">Function</th>
			<th valign="top">Number of Machines</th>
			<th valign="top">Machine Type</th>
			<th valign="top">CPU / Machine</th>
			<th valign="top">Total # of CPU vCores/Machine</th>
			<th valign="top">Memory/Machine</th>
			<th valign="top">Disk</th>
			<th valign="top">Disk capacity</th>
			<th valign="top">Network interface</th>
			<th valign="top">OS and Version</th>
		</tr>
		<tr>
			<td>Proxy Server (IBM HTTP Server and !WebSphere Plugin)</td>
			<td>1</td>
			<td>IBM System x3250 M4</td>
			<td>1 x Intel Xeon E3-1240 3.4GHz (quad-core)</td>
			<td>8</td>
			<td>16GB</td>
			<td>RAID 1 -- SAS Disk x 2</td>
			<td>299GB</td>
			<td>Gigabit Ethernet</td>
			<td>Red Hat Enterprise Linux Server release 6.5</td>
		</tr>
		<tr>
			<td>JTS Server</td>
			<td>1</td>
			<td>IBM System x3550 M4</td>
			<td>2 x Intel Xeon E5-2640 2.5GHz (six-core)</td>
			<td>24</td>
			<td>32GB</td>
			<td>RAID 5 -- SAS Disk x 2</td>
			<td>897GB</td>
			<td>Gigabit Ethernet</td>
			<td>Red Hat Enterprise Linux Server release 6.5</td>
		</tr>
		<tr>
			<td>QM Server</td>
			<td>1</td>
			<td>IBM System x3550 M4</td>
			<td>2 x Intel Xeon E5-2640 2.5GHz (six-core)</td>
			<td>24</td>
			<td>32GB</td>
			<td>RAID 5 -- SAS Disk x 2</td>
			<td>897GB</td>
			<td>Gigabit Ethernet</td>
			<td>Red Hat Enterprise Linux Server release 6.5</td>
		</tr>
		<tr>
			<td>Database Server</td>
			<td>1</td>
			<td>IBM System x3650 M4</td>
			<td>2 x Intel Xeon E5-2640 2.5GHz (six-core)</td>
			<td>24</td>
			<td>64GB</td>
			<td>RAID 5 -- SAS Disk x 2</td>
			<td>2.4TB</td>
			<td>Gigabit Ethernet</td>
			<td>Red Hat Enterprise Linux Server release 6.1</td>
		</tr>
		<tr>
			<td>RPT workbench</td>
			<td>1</td>
			<td>IBM System x3550 M4</td>
			<td>2 x Intel Xeon E5-2640 2.5GHz (six-core)</td>
			<td>24</td>
			<td>32GB</td>
			<td>RAID 5 -- SAS Disk x 2</td>
			<td>897GB</td>
			<td>Gigabit Ethernet</td>
			<td>Red Hat Enterprise Linux Server release 6.4</td>
		</tr>
		<tr>
			<td>RPT Agents</td>
			<td>6</td>
			<td>VM image</td>
			<td>4 x Intel Xeon X5650 CPU (1-Core 2.67GHz)</td>
			<td>1</td>
			<td>2GB</td>
			<td>N/A</td>
			<td>30GB</td>
			<td>Gigabit Ethernet</td>
			<td>Red Hat Enterprise Linux Server release 6.5</td>
		</tr>
		<tr>
			<td>Network switches</td>
			<td>N/A</td>
			<td>Cisco 2960G-24TC-L</td>
			<td>N/A</td>
			<td>N/A</td>
			<td>N/A</td>
			<td>N/A</td>
			<td>N/A</td>
			<td>Gigabit Ethernet</td>
			<td>24 Ethernet 10/100/1000 ports</td>
		</tr>
	</tbody>
</table>

N/A: Not applicable.
vCores = Cores with hyperthreading


---+++Network connectivity

All server machines and test clients are located on the same subnet. The LAN has 1000 Mbps of maximum bandwidth and less than 0.3 ms latency in ping.

---+++Data volume and shape

The artifacts were migrated from 5.0GA repository which contains a total of 579,142 artifacts in *one* large project.

The repository contained the following data:
<font color="red">to be validated</font> 
   * 50 test plans
   * 30,000 test scripts
   * 30,000 test cases
   * 120,000 test case execution records
   * 360,000 test case results
   * 3,000 test suites
   * 5,000 work items(defects)
   * 200 test environments
   * 600 test phases
   * 30 build definitions
   * 6,262 execution sequences
   * 3,000 test suite execution records
   * 15,000 test suite execution results
   * 6,000 build records

   * Database size = 15 GB
   * QM index size = 1.3 GB



---++ Appendix B:  Methodology
Rational Performance Tester(RPT) was used to simulate the workload created using the web client. Each user completed a random use case from a set of
available use cases. A Rational Performance Tester script is created for  each use case. The scripts are organized by pages and each page represents a user action.

The work load is role based as each of the areas defined under sequence of actions which are separated into individual user groups within an RPT schedule.  
The settings of the RPT schedule is shown below:

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM406Release/userload.jpg" width="35%" height="35%"/>

---+++User roles, test cases and workload characterization
*User Roles*
#UserRoles
<table class="gray-table">
	<tbody>
		<tr>
			<th align="center" height="40"><b>Use role</b></th>
			<th align="center" height="40"><b>% of Total</b></th>
			<th align="center" height="40"><b>Related Actions</b></th>
		</tr>
		<tr>
			<td>QE Manager</td>
			<td align="center">8</td>
			<td>Test plan create, Browse test plan and test case, Browse test script, Simple test plan copy, Defect search, View dashboard</td>
		</tr>
		<tr>
			<td>Test Lead</td>
			<td align="center">19</td>
			<td>Edit Test Environments, Edit test plan, Create test case, Bulk edit of test cases, Full text search, Browse test script, Test Execution, Defect search</td>
		</tr>
		<tr>
			<td>Tester</td>
			<td align="center">68</td>
			<td>Defect create, Defect modify, Defect search, Edit test case, Create test script, Edit test script, Test Execution, Browse test execution record</td>
		</tr>
		<tr>
		        <td>Dashboard Viewer</td>
			<td align="center">5</td>
			<td>View dashboard(with login and logout)</td>
		</tr>
	</tbody>
</table>

*Test Cases*
#TestCases

<table class="gray-table">
	<tbody>
		<tr>
			<th align="center" height="40"><b>Use Role</b></th>
			<th align="left" height="40"><b>Percentage of the user role</b></th>
			<th align="left" height="40"><b>Sequence of Operations</b></th>
		</tr>
		<tr>
			<td rowspan=6>QE Manager</td>
			<td align="center">1</td>
			<td>[[#TestPlanCreate][Test plan create]]:user creates test plan, then adds description, business objectives, test objectives, 2 test schedules, test estimate quality objectives and entry and exit criteria.</td>
		</tr>
		<tr>
			<td align="center">26</td>
			<td>[[#BrowseTestPlansAndTestCases][Browse test plans and test cases]]: user browses assets by: View Test Plans, then configure View Builder for name search; open test plan found, review various sections, then close. Search for test case by name, opens test case found, review various sections, then close.</td>
		</tr>
		<tr>
			<td align="center">26</td>
			<td>[[#BrowseTestScript][Browse test script]]: user search for test script by name, open it, reviews it, then closes.</td>
		</tr>
		<tr>
			<td align="center">1</td>
			<td>[[#SimpleTestPlanCopy][Simple test plan copy]]: user search test plan by name, then select one, then make a copy.</td>
		</tr>
		<tr>
			<td align="center">23</td>
			<td>[[#DefectSearch][Defect search]]: user searches for specific defect by number, user reviews the defect (pause), then closes.</td>
		</tr>
		<tr>
			<td align="center">20</td>
			<td>[[#ViewDashboard][View Dashboard]]: user views dashboard</td>
		</tr>
		<tr>
			<td rowspan=8>Test Lead</td>
			<td align="center">8</td>
			<td>[[#EditTestEnvironment][Edit Test Environment]]: user lists all test environments, and then selects one of the environments and modifies it.</td>
		</tr>
		<tr>
			<td align="center">15</td>
			<td>[[#EditTestPlan][Edit test plan]]: list all test plans; from query result, open a test plan for editing, add a test case to the test plan, a few other sections of the test plan are edited and then the test plan is saved.</td>
		</tr>
		<tr>
			<td align="center">4</td>
			<td>[[#TestCaseCreate][Create test case]]: user create test case by: opening the Create Test Case page, enters data for a new test case, and then saves the test case.</td>
		</tr>
		<tr>
			<td align="center">1</td>
			<td>[[#BulkEditofTestCases][Bulk edit of test cases]]: user searches for test cases with root name and edits all found with owner change.</td>
		</tr>
		<tr>
			<td align="center">3</td>
			<td>[[#FullTextSearch][Full text search]]: user does a full text search of all assets in repository using root name, then opens one of found items.</td>
		</tr>
		<tr>
			<td align="center">32</td>
			<td>[[#BrowseTestScript][Browse test script]]: user search for test script by name, open it, reviews it, then closes.</td>
		</tr>
		<tr>
			<td align="center">26</td>
			<td>[[#TestExecutionFor4Steps][Test Execution]]: selects “View Test Execution Records”, by name, starts execution,  enters pass/fail verdict, reviews results, sets points then saves. </td>
		</tr>
		<tr>
			<td align="center">11</td>
			<td>[[#DefectSearch][Defect search]]: user searches for specific defect by number, user reviews the defect (pause), then closes.</td>
		</tr>
		<tr>
			<td rowspan=8>Tester</td>
			<td align="center">8</td>
			<td>[[#CreateDefect][Defect create]]: user creates defect by: opening the Create Defect page, enters data for a new defect, and then saves the defect. </td>
		</tr>
		<tr>
			<td align="center">5</td>
			<td>[[#DefectModify][Defect modify]]: user searches for specific defect by number, modifies it then saves it.</td>
		</tr>
		<tr>
			<td align="center">14</td>
			<td>[[#DefectSearch][Defect search]]: user searches for specific defect by number, user reviews the defect (pause), then closes.</td>
		</tr>
		<tr>
			<td align="center">6</td>
			<td>[[#EditTestCase][Edit test case]]: user searches Test Case by name, the test case is then opened in the editor, then a test script is added to the test case (user clicks next a few times (server size paging feature) before selecting test script), The test case is then saved.</td>
		</tr>
		<tr>
			<td align="center">4</td>
			<td>[[#TestScriptCreate][Create test script]]: user creates test case by: selecting Create Test Script page, enters data for a new test script, and then saves the test script. </td>
		</tr>
		<tr>
			<td align="center">8</td>
			<td>[[#EditTestScript][Edit test script]]: user selects Test Script by name. test script then opened for editing, modified and then saved.</td>
		</tr>
		<tr>
			<td align="center">42</td>
			<td>[[#TestExecutionFor4Steps][Test Execution]]: selects “View Test Execution Records”, by name, starts execution,  enters pass/fail verdict, reviews results, sets points then saves. </td>
		</tr>
		<tr>
			<td align="center">7</td>
			<td>[[#TestExecutionRecordBrowsing][Browse test execution record]]: user browses TERs by:  name, then selects the TER and opens the most recent results.</td>
		</tr>
		<tr>
			<td>Dashboard Viewer</td>
			<td align="center">100</td>
			<td>[[#ViewDashboardWithLogin][View dashboard(with login and logout)]]: user logs in, views dashboard, then logs out. This user provides some login/logout behavior to the workload </td>
		</tr>
	</tbody>
</table>

---+++Response time comparison
The page performance is measured as mean value (or average) of its response time in the result data. For the majority of the pages under tests, there is little variation between runs, and the mean values are close to median in the sample for the load. 


---++ Appendix C:   Detailed performance results
<font color="red">all charts to be reviewed</font>
#DetailedResults

---+++Average page response time comparison breakdown
*NOTE*

For all usecase comparison charts, the unit is millisecond, and for the data, smaller is better.

#CreateDefect Create Defect

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/createDefect.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#TestPlanCreate Create Test Plan

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/createTestPlan.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#TestCaseCreate Create Test Case

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/createTestCase.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#TestScriptCreate Create Test Script

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/createTestScript.png"/>

[[#TestCases][Back to Test Cases & workload characterization]]

#BrowseTestPlansAndTestCases Browse Test Plans & Test Cases

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/browseTPsTCs.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#BrowseTestScript Browse Test Scripts

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/browseTestScript.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#BulkEditofTestCases Bulk Edit of Test Cases

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/bulkEditTC.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#DefectSearch Defect Search

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/searchDefect.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#DefectModify Defect Modify

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/modifyDefect.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#EditTestCase Edit Test Case

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/editTestCase.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#EditTestEnvironment Edit Test Environment

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/editTestEnv.png"/>

[[#TestCases][Back to Test Cases & workload characterization]]

#EditTestPlan Edit Test Plan

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/editTestPlan.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#EditTestScript Edit Test Script

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/editTestScript.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#FullTextSearch Full Text Search

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/fullTextSearch.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#SimpleTestPlanCopy Simple Test Plan Copy

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/simpleCopy.png"/>

[[#TestCases][Back to Test Cases & workload characterization]]

#TestExecutionFor4Steps Test Execution For 4 Steps

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/executeTestScript.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#TestExecutionRecordBrowsing Test Execution Record Browsing

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/browseTERs.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#ViewDashboard View Dashboard

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/viewDashboard.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

#ViewDashboardWithLogin View Dashboard with Login

<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/viewDashboardDV.png" />

[[#TestCases][Back to Test Cases & workload characterization]]

---+++RPT network transfer comparison
<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/RPT_network_v60_vs_v50.png" />

---+++RPT script execution comparison
%TABLE{ sort="off" headerbg="#3399FF" cellpadding="2" cellspacing="2" dataalign="center" caption="RPT script executions 6.0 vs 5.0" tableborder="2" tableframe="border" tablerules="none"}%
|Page name|v6.0 counts|v5.0 counts|
|[[#ViewDashboardWithLogin][Login_Prompt_viewDashboard_DV]]|504|499|
|[[#ViewDashboardWithLogin][Submit_User_Creds_viewDashboard_DV]]|496|495|
|[[#ViewDashboardWithLogin][Logout_viewDashboard_DV]]|498|497|
|[[#EditTestScript][View TestScript Page]]|205|207|
|[[#EditTestScript][InsertImage4]]|206|209|
|[[#EditTestScript][InsertImage2]]|209|207|
|[[#EditTestScript][InsertImage3]]|209|204|
|[[#EditTestScript][InsertImage1]]|214|212|
|[[#EditTestScript][Post_Data_editTestScript]]|219|215|
|[[#EditTestScript][Search Scripts By Name]]|205|208|
|[[#EditTestScript][Sort By Modified]]|203|205|
|[[#EditTestScript][Click Next - Edit Test Script]]|203|209|
|[[#EditTestScript][Open Selected TestScript]]|205|212|
|[[#EditTestPlan][Open TestPlan Page]]|165|168|
|[[#EditTestPlan][Search TestPlan By Name]]|162|169|
|[[#EditTestPlan][Open Test Plan]]|161|172|
|[[#EditTestPlan][Post_Data_editTestPlan]]|160|165|
|[[#EditTestEnvironment][Open LabManagement Page]]|77|90|
|[[#EditTestEnvironment][Search Test Environment]]|76|86|
|[[#EditTestEnvironment][Post_Data_editTestEnv]]|71|86|
|[[#EditTestEnvironment][Update Configuration]]|73|84|
|[[#TestExecutionFor4Steps][Open TER Page]]|1,447|1,460|
|[[#TestExecutionFor4Steps][Search TER By Name]]|1,456|1,478|
|[[#TestExecutionFor4Steps][Execute The TER]]|1,459|1,462|
|[[#TestExecutionFor4Steps][Execute Step1]]|1,445|1,439|
|[[#TestExecutionFor4Steps][Execute Step2]]|1,423|1,453|
|[[#TestExecutionFor4Steps][Execute Step4]]|1,420|1,463|
|[[#TestExecutionFor4Steps][Execute Step3]]|1,430|1,462|
|[[#TestExecutionFor4Steps][Show Results]]|1,429|1,461|
|[[#EditTestCase][Open TestCase Page]]|170|170|
|[[#EditTestCase][Sort TC By State]]|173|170|
|[[#EditTestCase][Search TC By Name]]|172|171|
|[[#EditTestCase][Next Page]]|174|169|
|[[#EditTestCase][Open TestCase]]|170|172|
|[[#EditTestCase][View TestCase]]|165|165|
|[[#EditTestCase][Click Add TS]]|167|163|
|[[#EditTestCase][Search TS By Name]]|166|158|
|[[#EditTestCase][Add TestScript]]|166|158|
|[[#EditTestCase][Post_Data_editTestCase]]|153|151|
|[[#BrowseTestScript][ViewTestScript]]|498|467|
|[[#BrowseTestScript][SearchTestScripts]]|499|463|
|[[#BrowseTestScript][SortTestScripts]]|496|462|
|[[#BrowseTestScript][ClickNext - Browse TestScript]]|501|466|
|[[#BrowseTestScript][OpenTestScript]]|499|468|
|[[#DefectModify][CCM WI Home]]|160|132|
|[[#DefectModify][Open Defect - Modify Defect]]|161|133|
|[[#DefectModify][Post_Data_modifyDefect]]|158|136|
|[[#DefectSearch][CCM WI HOME]]|610|608|
|[[#DefectSearch][Search Defect By Number]]|158|133|
|[[#TestScriptCreate][OpenCreateTestScriptPage]]|91|106|
|[[#TestScriptCreate][Post_Data_createTestScript]]|90|101|
|[[#ViewDashboard][Refresh Dashboard Page]]|81|86|
|[[#CreateDefect][RQM_WorkItemDropdown]]|193|200|
|[[#CreateDefect][createDefectDialog_createDefect]]|193|202|
|[[#CreateDefect][Post_Data_createDefect]]|194|211|
|[[#TestExecutionRecordBrowsing][ViewTERsPage]]|214|227|
|[[#TestExecutionRecordBrowsing][SearchTER]]|907|898|
|[[#TestExecutionRecordBrowsing][ClickNext]]|898|891|
|[[#TestExecutionRecordBrowsing][openPreviousResults]]|895|885|
|[[#TestExecutionRecordBrowsing][ClickLatestResult]]|901|881|
|[[#TestExecutionRecordBrowsing][SortbyMilestone]]|900|898|
|[[#DefectSearch][Search Defect By Number]]|602|617|
|[[#DefectSearch][openDefect]]|610|618|
|[[#FullTextSearch][SearchForString]]|32|44|
|[[#FullTextSearch][OpenFoundItem]]|31|45|
|[[#BulkEditofTestCases][Open TestCase Page]]|13|12|
|[[#BulkEditofTestCases][Search TC By Name]]|14|12|
|[[#BulkEditofTestCases][Select All TestCases]]|15|13|
|[[#BulkEditofTestCases][Change Owner]]|16|13|
|[[#TestPlanCreate][LoadNewTestPlanForm]]|4|5|
|[[#TestPlanCreate][Post_Data_createTestPlan]]|4|5|
|[[#TestCaseCreate][OpenCreateTestCasePage]]|45|47|
|[[#TestCaseCreate][Post_Data_createTestCase]]|45|48|
|[[#BrowseTestPlansAndTestCases][Open TestPlan View]]|93|91|
|[[#BrowseTestPlansAndTestCases][Search TP]]|99|91|
|[[#BrowseTestPlansAndTestCases][Open TP]]|96|91|
|[[#BrowseTestPlansAndTestCases][Review Test Objects]]|93|90|
|[[#BrowseTestPlansAndTestCases][Review Requirement]]|90|92|
|[[#BrowseTestPlansAndTestCases][Review Schedule]]|93|96|
|[[#BrowseTestPlansAndTestCases][Review Estimation]]|98|95|
|[[#BrowseTestPlansAndTestCases][Review Quality Objective]]|100|94|
|[[#BrowseTestPlansAndTestCases][Review TestCases]]|100|92|
|[[#BrowseTestPlansAndTestCases][TestCase View]]|99|90|
|[[#BrowseTestPlansAndTestCases][Search TestCase]]|104|88|
|[[#BrowseTestPlansAndTestCases][Sort By State]]|101|89|
|[[#BrowseTestPlansAndTestCases][Click Next]]|97|88|
|[[#BrowseTestPlansAndTestCases][Open TestCase]]|99|87|
|[[#BrowseTestPlansAndTestCases][Review TestScript]]|96|85|
|[[#SimpleTestPlanCopy][OpenViewTestPlanPage]]|3|4|
|[[#SimpleTestPlanCopy][SearchTPbyName]]|3|4|
|[[#SimpleTestPlanCopy][SelectTPandDuplicate]]|2|3|
|[[#SimpleTestPlanCopy][startDuplicate]]|2|3|

---+++Resource utilization

%TABLE{ sort="off" headerbg="#3399FF" cellpadding="2" cellspacing="2" dataalign="center" caption="OS resource utilization 6.0 vs 5.0 - overview" tableborder="2" tableframe="border" tablerules="none"}%
||6.0 vs 5.0|
|CPU|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/CPU_v60_vs_v50.png" />|
|Disk|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Disk_v60_vs_v50.png" />|
|Memory|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Mem_v60_vs_v50.png" />|
|Network|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Network_v60_vs_v50.png" />|

%TABLE{ sort="off" headerbg="#3399FF" cellpadding="2" cellspacing="2" dataalign="center" caption="QM server resource utilization 6.0 vs 5.0" tableborder="2" tableframe="border" tablerules="none"}%
||6.0|5.0|
|CPU|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/CPU_QM_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/CPU_QM_CLM5.png" />|
|Disk|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Disk_QM_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Disk_QM_CLM5.png" />|
|Memory|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Mem_QM_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Mem_QM_CLM5.png"/>|
|Network|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Network_QM_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Network_QM_CLM5.png"/>|

%TABLE{ sort="off" headerbg="#3399FF" cellpadding="2" cellspacing="2" dataalign="center" caption="DB server resource utilization 6.0 vs 5.0" tableborder="2" tableframe="border" tablerules="none"}%
||6.0|5.0|
|CPU|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/CPU_DB_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/CPU_DB_CLM5.png"/>|
|Disk|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Disk_DB_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Disk_DB_CLM5.png"/>|
|Memory|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Mem_DB_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Mem_DB_CLM5.png"/>|
|Network|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Network_DB_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/Network_DB_CLM5.png"/>|

---+++Garbage collection - JVM GC Chart
For JVM parameter please refer to [[#Appendix_A][Appendix A]]

Verbose garbage collection is enabled to log the GC activities.  These GC logs showed very little variation between runs.  There is also no discernible difference between versions.  Below is one example of the output from the GC log for each application.

%TABLE{ sort="off" headerbg="#3399FF" cellpadding="2" cellspacing="2" dataalign="center" caption="WAS JVM Garbage Collection Charts 6.0 vs 5.0" tableborder="2" tableframe="border" tablerules="none"}%
||6.0|5.0|
|QM|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/GC_QM_CLM6.png" />|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM50Release/CLM50GA_QM_regression_run017_qm_gc_chart.PNG" width="80%"  />|
|JTS|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM60Release/GC_JTS_CLM6.png"/>|<img src="https://jazz.net/wiki/pub/Deployment/CollaborativeLifecycleManagementPerformanceReportRQM50Release/CLM50GA_QM_regression_run017_jts_gc_chart.PNG" width="80%"  />|


---++Appendix D - Key configuration parameters

<table class="gray-table">
	<tbody>
		<tr>
			<th align="left" width="200"><strong>Product</strong><br></th>
			<th align="left" width="100"><strong>Version</strong></th>
			<th align="left" width="400"><strong>Highlights for configurations under test</strong></th>
		</tr>
		<tr>
			<td style="vertical-align: top;">IBM HTTP Server for !WebSphere Application Server</td>
			<td style="vertical-align: top;">8.5.0.1</td>
			<td style="vertical-align: top;">IBM HTTP Server functions as a reverse proxy server implemented via Web server plug-in for !WebSphere Application Server.

Configuration details can be found from the [[http://pic.dhe.ibm.com/infocenter/clmhelp/v4r0/index.jsp?topic=%2Fcom.ibm.jazz.install.doc%2Ftopics%2Ft_config_reverse_proxy_ihs.html][CLM infocenter]].

*HTTP server (httpd.conf)*:

   * !MaxClients: increase value for high-volume loads [[http://publib.boulder.ibm.com/httpserv/ihsdiag/ihs_performance.html][(adjust value based on user load)]]
   * !ThreadsPerChild = 25

*OS Configuration*:

   * max user processes = unlimited
      </td>
		</tr>
		<tr>
			<td style="vertical-align: top;">IBM !WebSphere Application Server Network Deployment</td>
			<td style="vertical-align: top;">8.5.0.1</td>
			<td style="vertical-align: top;"><strong>JVM settings:</strong>

   * GC policy and arguments, max and init heap sizes:

				<verbatim>
 -Xgcpolicy:gencon -Xmx8g -Xms8g -Xmn2g -Xss786K -Xcompressedrefs -Xgc:preferredHeapBase=0x100000000
 -verbose:gc -Xverbosegclog:gc.log -XX:MaxDirectMemorySize=1G
</verbatim>

				*Thread pools:*

   * Maximum !WebContainer = Minimum !WebContainer = 500


				*OS Configuration:*

				System wide resources for the app server process owner: 
   * max user processes = unlimited
   * open files = 65536

			</td>
		</tr>
		<tr>
			<td>DB2</td>
			<td>ESE 10.1.0.3</td>
			<td> </td>
		</tr>
		<tr>
			<td>LDAP server</td>
			<td> </td>
			<td> </td>
		</tr>
		<tr>
			<td>License server</td>
			<td> </td>
			<td>N/A</td>
		</tr>
		<tr>
			<td>RPT workbench</td>
			<td>8.2.1.5</td>
			<td>Defaults</td>
		</tr>
		<tr>
			<td>RPT agents</td>
			<td>8.2.1.5</td>
			<td>Defaults</td>
		</tr>
		<tr>
			<td>Network</td>
			<td> </td>
			<td>Shared subnet within test lab</td>
		</tr>
	</tbody>
</table>

---++++!! For more information

   * [[SizingReportCLM2012][Collaborative Lifecycle Management 2012 Sizing Report (Standard Topology E1)]]

--------------------
---++++!! About the authors: 
Main.HongyanHuo
---++++!! Related topics: [[CollaborativeLifecycleManagementPerformanceReportRQM50Release][Collaborative Lifecycle Management performance report: Rational Quality Manager 5.0 release]], [[PerformanceDatasheetsAndSizingGuidelines][Performance datasheets]]

--------------------
---+++++!! Questions and comments:
   * What other performance information would you like to see here?
   * Do you have performance scenarios to share? 
   * Do you have scenarios that are not addressed in documentation?
   * Where are you having problems in performance?

%COMMENT{type="below" target="PerformanceDatasheetReaderComments" button="Submit"}%

%INCLUDE{"PerformanceDatasheetReaderComments"}%

<sticky></div></sticky>
Deployment
Community information and contribution guidelines
Status icon key:
To do
Under construction
New
Updated
Constant change
None - stable page
Smaller versions of status icons for inline text:
Copyright © by IBM and non-IBM contributing authors. All material on this collaboration platform is the property of the contributing authors.
Contributions are governed by our Terms of Use. Please read the following disclaimer.
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.