SizingAndTuningGuideForRationalDOORSNextGenerationVersion60 < Deployment

Engineering Lifecycle Management Wiki - Deployment
Community information and contribution guidelines
TWiki > Deployment Web > DeploymentPlanningAndDesign > PerformanceDatasheetsAndSizingGuidelines > SizingAndTuningGuideForRationalDOORSNextGenerationVersion60
Revision 30 - 2016-07-05 - 12:55:43 - Main.byrnesle
<div id="header-title" style="padding: 10px 15px; border-width:1px; border-style:solid; border-color:#FFD28C; background-image: url(<nop>https://jazz.net/wiki/pub/Deployment/WebPreferences/TLASE.jpg); background-size: cover; font-size:120%">
---+!! Sizing and tuning guide for Rational DOORS Next Generation 6.0:   configuration management  
%DKGRAY% Authors: Main.VaughnRokosz, Main.GustafSvensson, Main.LeeByrnes<br>
Build basis:  IBM Rational DOORS Next Generation 6.0 
%ENDCOLOR%</div></sticky>

<!-- Page contents top of page on right hand side in box -->
<sticky><div style="float:right; border-width:1px; border-style:solid; border-color:#DFDFDF; background-color:#F6F6F6; margin:0 0 15px 15px; padding: 0 15px 0 15px;">
%TOC{title="Page contents"}%
</div></sticky>

<sticky><div style="margin:15px;"></sticky>


---++ Introduction

The 6.0 release of IBM® Rational® DOORS® Next Generation (RDNG) introduces a major new feature:   configuration management.    You can now manage the evolution of requirements using techniques similar to those which Rational Team Concert supports for managing source code.     You can organize requirements into streams and baselines, and you can make changes to requirements within the context of change sets which can then be delivered to streams.    You can easily reuse requirements by branching streams or baselines, and you can apply versioning across your entire CLM solution through global configurations.

This 6.0 sizing guide focuses on the new configuration management feature, and examines the [[#HardwareImpact][impact of different hardware configurations]] on the scalability of a deployment (where scalability refers to the maximum throughput/user load which can be achieved for a given repository size without significant response degradation).    This guide also looks at the [[#CountImpact][impact of artifact counts]] on the configuration management operations.

If you are not using configuration management, please refer to [[SizingAndTuningGuideForRationalDOORSNextGenerationVersion50][Sizing and tuning guide for Rational DOORS Next Generation 5.0]].    The sizing recommendations for 6.0 without configuration management are the same as for 5.0.

---+++ Standard disclaimer

The information in this document is distributed AS IS. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk. Any pointers in this publication to external Web sites are provided for convenience only and do not in any manner serve as an endorsement of these Web sites. Any performance data contained in this document was determined in a controlled environment, and therefore, the results that may be obtained in other operating environments may vary significantly. Users of this document should verify the applicable data for their specific environment. 

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the number of other applications running in the customer environment, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 

This testing was done as a way to compare and characterize the differences in performance between different configurations of the 6.0 product. The results shown here should thus be looked at as a comparison of the contrasting performance between different configurations, and not as an absolute benchmark of performance. 


#HardwareImpact
---++ How hardware configurations impact scalability

We conducted a series of tests with different hardware configurations, to assess the impact of memory and disk drives on scalability.    These tests cover similar configurations as in the [[SizingAndTuningGuideForRationalDOORSNextGenerationVersion50][5.0 sizing guide]], but the [[#WorkloadCharacterization][workload used for 6.0]] included configuration management operations.    We carried out these tests by setting up a [[#TestTopology][specific hardware configuration]] and [[#DataShape][repository size]], and then slowly increasing the number of simulated users until we started to see increasing response times.     

The overall conclusions for 6.0 are similar to the 5.0 conclusions.   As a general rule, the system can support more users (and more artifacts) as the amount of RAM on the DNG server increases.    The number of supported users can also be increased if the disk I/O speed is increased (e.g. by using SSD drives instead of regular hard disks).     The reason for this concerns the index files used by the DNG server to provide fast access to artifact data.    The DNG server will cache this index in memory if possible, which is why increasing the amount of RAM can improve performance.    The amount of RAM available to cache the DNG index is roughly the total amount of RAM minus the maximum size of the JVM heap.    If the disk size of the DNG index (in server/conf/rm/indices) is less than that number, then the DNG index will be cached in memory, which will allow for the fastest read access to DNG data.     The DNG index will be accessed from the file system when there isn't enough memory to cache the index completely (but also on write operations).   This is why disk I/O matters, and why an SSD drive (when combined with enough RAM) provides the best performance and scale.

#FailureMode
---+++ A note on failure modes

What we see in all cases is that the response times slowly increase as the user load increases up to a point - and at that point, there is a sudden, sharp increase in response times.   At the same time, the CPU utilization on the DNG server increases to 100%.    This is the classic symptom of lock contention in Java.    As the load increases, the simulated users start to ask for exclusive access to the same data, and this causes a queue to form.    The bottleneck forms around access to the DNG index information.    We have observed that the user load at which this happens is not consistent, and so there is a margin of error on the user estimates of +/- 50-100 users.

This failure mode is in part a consequence of the performance simulation itself, which issues a stream of requests against the server at a steady rate, without pause.    Our simulated users never take breaks for coffee, or go to meetings, or go to lunch!  This represents a worst case scenario, where the server can never catch up.     In production usage, it would be common for bursts of operations to be followed by slow periods, during which the server can catch up.    

---+++ Tuning recommendations for all configurations
---++++ JVM nursery sizing:   1/3 of max JVM heap

When allocating memory for the Java Virtual machine for the DNG server, allocate 1/3 of the total heap to the nursery (divide the value of -Xmx by 3, and use that for -Xmn).

The nursery is a section of JVM memory used to efficiently manage short-lived objects, such as those associated with handling an incoming HTTP request.    As the number of concurrent DNG users grows, there is more demand on the nursery to process the incoming transactions.     If the nursery is too small, memory for those objects will be copied into sections of JVM memory which have more management overhead.     By allocating more memory to the nursery, you can improve performance at higher load levels by ensuring that short-lived objects stay in the nursery.

---++++ Tuning the configuration management cache

Some information about streams and baselines is cached in memory.    The cache size can be changed to improve the performance for higher user loads (with the tradeoff being additional memory usage).     You control the cache settings through JVM startup parameters (e.g. through the -D flag or by using custom JVM properties).

The cache stores information which DNG associates with each configuration (where a configuration is a stream, baseline, or change set).    For best performance, set the cache size to the number of concurrent users.    This supports the worst-case scenario where all active users are working in separate configurations.    The default value for this cache is 200.

To set as a JVM startup parameter:

<verbatim>
-Dcom.ibm.rdm.configcache.size=400
</verbatim>

Assume that each active entry in the cache will consume 2M of RAM.

---++++ Keeping database statistics up-to-date

Information about streams, baselines, and changesets is stored in tables in the database.   The DNG server will query these tables when users work in a DNG project which is enabled for configuration management.   For best performance, be sure that the database statistics are kept up to date as the size of the repository grows.    This is especially critical when you are first adopting configuration management.    Some of the tables will start out empty, and then grow as users work with streams, baselines, and change sets.     Keeping the database statistics updated ensures that the queries against those tables will be properly optimized.

Databases generally manage statistics automatically; for example, in a scheduled overnight operation.  However, to ensure that the database is fully optimized, you can manually run the statistics as follows. 
DB2
<verbatim>DB2 REORGCHK UPDATE STATISTICS ON TABLE ALL </verbatim>

Oracle
<verbatim>EXEC DBMS_STATS.GATHER_DATABASE_STATS</verbatim>

See also [[http://www-01.ibm.com/support/docview.wss?uid=swg21975746][Slow server performance and CRRRW7556E on IBM Rational DOORS Next Generation version 6.0.x]] for further information on keeping databases tuned for configuration management, if you are experiencing performance issues.

---++++ Available TCP/IP ports
The default number of TCP/IP ports that are available on AIX and Linux operating systems is too low and must be increased. Windows operating systems have a higher default limit, but that limit might still be too low for large concurrent user loads. Use the following instructions to increase the port range.
   * On AIX and Linux systems, set the limit as follows: ulimit -n 65000 
   * On Windows 2003 Server, follow these steps:
      * Open the registry. 
      * Navigate to HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Services/Tcpip/Parameters and create a dWord named !MaxUserPort. Set its value to 65000. 
      * Restart the computer.
   * On Windows 2008 Server, follow the instructions in this [[http://support.microsoft.com/kb/929851][Microsoft support article]] to change the dynamic port range. For the start port setting, use 2000. Set the number of ports to 63535.

---++++ Thread pool size
The !WebSphere Application Server thread pool size must be increased from the default of 50 to 0.75 times the expected user load. For example, if you have 400 concurrent users, the thread pool maximum should be set to 300.

---++++ More suggestions
In a 3-server topology, where Jazz Team Server and the DNG server are on separate hardware, adjust two settings in the fronting.properties file for loads higher than 200 concurrent users. 
   * Ensure that com.ibm.rdm.fronting.server.connection.route.max equals the number of users
   * Ensure that com.ibm.rdm.fronting.server.connection.max is twice the value of number of users
When you use a proxy or reverse proxy server, you might need to increase the maximum allowed connections on the proxy server, depending on the concurrent user load. For more information, see [[http://www-01.ibm.com/support/docview.wss?uid=swg21167658][Tuning IBM HTTP Server to maximize the number of client connections to WebSphere Application Server]] .

---+++ Repositories with up to 500,000 artifacts

The chart below summarizes the results of testing with a repository of 500,000 artifacts, using a standard configuration management workload.    Here we looked at the impact of the amount of RAM on the response times and maximum user limit.    The system with 32G RAM performed the worst.   Given that the 500K repository has a 39G index, and our 32G system has only 16G or less available for caching, this is not unexpected.    Increasing the amount of RAM to 64G allows for most of the index to be cached in memory.   Performance for both the 64G and 128G systems are roughly the same, as is the supported user load (around 450).    

The maximum user load for the 64G system was unexpectedly high (600 users vs. 450 for 128G).     This is related to the [[#FailureMode][inconsistencies discussed earlier]] regarding the exact point at which bottleneck around the DNG index forms.   We therefore use 450 users as the limit for the 64G system, to be conservative.

<img src="%ATTACHURLPATH%/500KResults.png" alt="500KResults.png" width="75%" height="75%" />

Note:  in this graph, we have left out the data points collected when the system was overloaded.   The line for the 32G system, for example, stops at 350 users.     If you want to see all of the data points, please refer to the [[#SummaryTable][ summary table]].

The following table shows the various hardware configurations with RAM and heap sizes and the maximum number of concurrent users that are supported in each hardware configuration.

<table border="1" cellpadding="0" cellspacing="0" id="_749" style="margin-left:5px">
   <tbody>
      <tr>
         <td colspan="1" id="_750" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_751" style="margin-left: 5px; "><b> RM server hardware and JVM configuration</b></p>
         </td>
         <td colspan="1" id="_754" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_755" style="margin-left: 5px; "><b>Jazz Team Server</b></p>
         </td>
         <td colspan="1" id="_756" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_757" style="margin-left: 5px; "><b> DB2 server</b></p>
         </td>
         <td colspan="1" id="_758" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_759" style="margin-left: 5px; "><b>Number of users supported</b></p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_760" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_761" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_764" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_765" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_766" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_767" style="margin-left: 5px; ">64 GB RAM </p>
         </td>
         <td colspan="1" id="_768" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_769" style="margin-left: 5px; ">350</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_770" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_771" style="margin-left: 5px; ">64 GB RAM with 24 GB JVM heap</p>
         </td>
         <td colspan="1" id="_774" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_775" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_776" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_777" style="margin-left: 5px; ">64 GB RAM</p>
         </td>
         <td colspan="1" id="_778" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_779" style="margin-left: 5px; ">450</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_780" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_781" style="margin-left: 5px; ">128 GB RAM with 24 GB JVM heap</p>
         </td>
         <td colspan="1" id="_784" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_785" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_786" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_787" style="margin-left: 5px; ">64 GB RAM</p>
         </td>
         <td colspan="1" id="_788" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_789" style="margin-left: 5px; ">450</p>
         </td>
      </tr>
   </tbody>
</table>

---+++ Repositories with up to 1M artifacts

The chart below shows response times as a function of user load for a repository with 1 million artifacts, for a variety of hardware configurations.    Here are the main observations:
   * More than 32G of RAM is needed to cache the index (75G) associated with 1M artifacts, so the response times and scale limits are lowest in 32G configurations (even with an SSD drive).
   * An SSD drive improves scalability slightly, but there are enough inconsistencies in when the bottlenecks form to make it hard to draw solid conclusions as far as scalability goes beyond 450 users.   But the response times at 450 users are best for the 64G SSD configuration (with the 128G HDD configuration coming in second).
   * The runs beyond 450u are not considered repeatable.   We are using 450 users as the limit, to be conservative.

<img src="%ATTACHURLPATH%/1MResults.png" alt="1MResults.png" width="75%" height="75%" />

The following table outlines the different hardware and JVM heap configurations and the maximum number of users that are supported in each configuration.

<table border="1" cellpadding="0" cellspacing="0" id="_813" style="margin-left:5px">
   <tbody>
      <tr>
         <td colspan="1" id="_814" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_815" style="margin-left: 5px; "><b> RM server hardware and JVM configuration</b></p>
         </td>
         <td colspan="1" id="_818" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_819" style="margin-left: 5px; "><b>Jazz Team Server</b></p>
         </td>
         <td colspan="1" id="_820" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_821" style="margin-left: 5px; "><b> DB2 server</b></p>
         </td>
         <td colspan="1" id="_822" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_823" style="margin-left: 5px; "><b>Number of users supported</b></p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_824" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_825" style="margin-left: 5px; ">HDD – 32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_828" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_829" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_830" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_831" style="margin-left: 5px; ">64 GB RAM </p>
         </td>
         <td colspan="1" id="_832" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_833" style="margin-left: 5px; ">350</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_834" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_835" style="margin-left: 5px; ">SSD – 32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_838" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_839" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_840" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_841" style="margin-left: 5px; ">64 GB RAM </p>
         </td>
         <td colspan="1" id="_842" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_843" style="margin-left: 5px; ">400</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_854" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_855" style="margin-left: 5px; ">HDD – 64 GB RAM with 24 GB JVM heap</p>
         </td>
         <td colspan="1" id="_858" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_859" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_860" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_861" style="margin-left: 5px; ">64 GB RAM</p>
         </td>
         <td colspan="1" id="_862" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_863" style="margin-left: 5px; ">450</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_864" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_865" style="margin-left: 5px; ">SSD – 64 GB RAM with 24 GB JVM heap</p>
         </td>
         <td colspan="1" id="_868" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_869" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_870" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_871" style="margin-left: 5px; ">64 GB RAM</p>
         </td>
         <td colspan="1" id="_872" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_873" style="margin-left: 5px; ">450</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_874" rowspan="1" style="width: 262px; vertical-align: center; ">
            <p dir="ltr" id="_875" style="margin-left: 5px; ">HDD – 128 GB RAM with 24 GB JVM heap</p>
         </td>
         <td colspan="1" id="_878" rowspan="1" style="width: 243px; vertical-align: center; ">
            <p dir="ltr" id="_879" style="margin-left: 5px; ">32 GB RAM with 16 GB JVM heap</p>
         </td>
         <td colspan="1" id="_880" rowspan="1" style="width: 124px; vertical-align: center; ">
            <p dir="ltr" id="_881" style="margin-left: 5px; ">64 GB RAM</p>
         </td>
         <td colspan="1" id="_882" rowspan="1" style="width: 129px; vertical-align: top; ">
            <p dir="ltr" id="_883" style="margin-left: 5px; ">450</p>
         </td>
      </tr>
   </tbody>
</table>

#SummaryTable
---+++ Response times and CPU utilization - all configurations

The table below summarizes the overall average response time and the CPU utilization of the DNG server at different user levels, for the tested hardware configurations.   The cells in red indicate that the system was past its limit.

<img src="%ATTACHURLPATH%/SummaryTable.png" alt="SummaryTable.png" width="100%" height="100%" />

#CountImpact
---++ How artifact counts impact configuration management operations

We looked at several of the new configuration management use cases in single-user mode, to see how the response times would be impacted by changing the number of artifacts involved.   Here's a summary of what we found:

<table border="1" cellpadding="0" cellspacing="0" id="_223" style="margin-left:5px">
   <tbody>
      <tr>
         <td colspan="1" id="_224" rowspan="1" style="width: 300px; vertical-align: center; ">
            <p dir="ltr" id="_225" style="margin-left: 5px; "><b> Use case</b></p>
         </td>
         <td colspan="1" id="_226" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_227" style="margin-left: 5px; "><b>Sensitivity to counts<br></p>
         </td>
         <td colspan="1" id="_228" rowspan="1" style="width: 200px; vertical-align: center; ">
            <p dir="ltr" id="_229" style="margin-left: 5px; "><b>Upper limit of testing<br></p>
         </td>
         <td colspan="1" id="_230" rowspan="1" style="width: 250px; vertical-align: center; ">
            <p dir="ltr" id="_231" style="margin-left: 5px; "><b>Typical response time<br></p>
         </td>
      </tr>
        <tr>
          <td valign="top">Open configuration UI<br>
          </td>
          <td valign="top">low<br>
          </td>
          <td valign="top">n/a<br>
          </td>
          <td valign="top">2-4 seconds<br>
          </td>
        </tr>
        <tr>
          <td  valign="top">Configuration UI: list baselines<br>
          </td>
          <td  valign="top">low<br>
          </td>
          <td  valign="top">62 baselines<br>
          </td>
          <td valign="top">4 seconds<br>
          </td>
        </tr>
        <tr>
          <td  valign="top">Configuration UI: list streams<br>
          </td>
          <td  valign="top">low<br>
          </td>
          <td valign="top">33 streams<br>
          </td>
          <td  valign="top">2-3 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Configuration UI: list change sets<br>
          </td>
          <td  valign="top">low<br>
          </td>
          <td  valign="top">132 change sets<br>
          </td>
          <td  valign="top">2-3 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Open "switch configuration" UI<br>
          </td>
          <td valign="top">low<br>
          </td>
          <td valign="top">n/a<br>
          </td>
          <td valign="top">4 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Switch configuration:&nbsp; search for baseline<br>
          </td>
          <td valign="top">medium<br>
          </td>
          <td valign="top">328 baselines<br>
          </td>
          <td valign="top">up to 12 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Switch configuration:&nbsp; search for stream<br>
          </td>
          <td valign="top">medium<br>
          </td>
          <td valign="top">109 streams<br>
          </td>
          <td valign="top">up to 9 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Switch configuration:&nbsp; search for change sets<br>
          </td>
          <td valign="top">medium<br>
          </td>
          <td valign="top">2508 change sets<br>
          </td>
          <td valign="top">up to 11 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Compare stream: confirm source/target<br>
          </td>
          <td valign="top">low<br>
          </td>
          <td valign="top">n/a<br>
          </td>
          <td valign="top">2 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Compare stream: compare project properties<br>
          </td>
          <td valign="top">high<br>
          </td>
          <td valign="top">1300 change sets<br>
          </td>
          <td valign="top">Increases from 3 to 42 seconds<br>
          </td>
        </tr>
          <tr>
          <td valign="top">Compare stream: compare artifacts<br>
          </td>
          <td valign="top">high<br>
          </td>
          <td valign="top">1300 change sets<br>
          </td>
          <td valign="top">Increases from 8 to 62 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Deliver change set: confirm source/target<br>
          </td>
          <td valign="top">high<br>
          </td>
          <td valign="top">n/a<br>
          </td>
          <td valign="top">Increases from 5 to 20 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Deliver change set:&nbsp; compare artifacts<br>
          </td>
          <td valign="top">high<br>
          </td>
          <td valign="top">825 changes in change set<br>
          </td>
          <td valign="top">Increases from 8 to 33 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Deliver change set:&nbsp; Finish<br>
          </td>
          <td valign="top">high<br>
          </td>
          <td valign="top">825 changes in change set<br>
          </td>
          <td valign="top">Increases from 7 to 19 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Deliver to stream:&nbsp;&nbsp; select menu<br>
          </td>
          <td valign="top">high<br>
          </td>
          <td valign="top">1017 changes delivered<br>
          </td>
          <td valign="top">Increases from 15 to 35 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Deliver to stream:&nbsp; compare artifacts<br>
          </td>
          <td valign="top">high<br>
          </td>
          <td valign="top">1017 changes delivered </td>
          <td valign="top">Increases from 5 to 45 seconds<br>
          </td>
        </tr>
        <tr>
          <td valign="top">Deliver to stream:&nbsp; Finish<br>
          </td>
          <td valign="top">high<br>
          </td>
          <td valign="top">1017 changes delivered </td>
          <td valign="top">Increases from 15 to 25 seconds<br>
          </td>
        </tr>
   </tbody>
</table>

---+++ Change set delivery as a function of change set size

This use case looked at how delivering a change set was impacted by the number of changes in the change set.    The test sequence involved creating a change set, and then making a controlled number of changes to a module.    Then, we delivered the change set and measured the response times of the main operations:
   * Time to display the "Confirm source/target" UI (after selected the "Deliver change set" menu option)
   * Time to display the "Artifacts changed" UI
   * Time to "Finish" the deliver

When making controlled changes to an artifact, we first opened a module containing 1500 artifacts, and then copied 25 existing artifacts and pasted them into the module.    The response times for the different deliver steps are shown in the graph below.

<img src="%ATTACHURLPATH%/DeliverChangeSet.png" alt="DeliverChangeSet.png" width="50%" height="50%" />

---+++ Deliver changes to a stream as a function of change count

This use case looked at how delivering changes from one stream to another was impacted by the number of differences between the two streams.    We looked at the following operations in the "Deliver":

   * Time to display the "Confirm source/target" UI (this appears after selecting the "Compare stream" menu option)
   * Time to display the "Modified artifacts UI" UI 
   * Time to display "Finish" the deliver UI 

The time to deliver is highly sensitive to the number of differences between the streams, especially when displaying all of the differences, and when finishing the deliver (which then applies all of the changes to the target stream).   

<img src="%ATTACHURLPATH%/DeliverToStream.png" alt="DeliverToStream.png" width="50%" height="50%" />

---+++ Compare streams as a function of the number of change sets

This use case looked at how comparing two streams was impacted by the number of differences (change sets) between the two streams.   In particular, this compared a stream to its parent.     Change sets were delivered to the parent stream, while keeping the child stream unchanged.   

We looked at the following operations in the "Compare streams" operation:
   * Time to display the "Confirm source/target" UI (this appears after selecting the "Compare stream" menu option)
   * Time to display the "Compare project properties" UI
   * Time to display the "Compare folders" UI
   * Time to display the "Compare artifacts" UI

In these measurements, there were no changes to project properties or to the folders, so those two UIs did not display anything.    The time to display the "compare artifacts" UI increased as the number of changes increased (since there is more data to display).   

<img src="%ATTACHURLPATH%/CompareStream.png" alt="CompareStream.png" width="50%" height="50%" />

---+++ Searching for baselines, streams, and change sets

This use case looked at the cost of searching for baselines, streams, and change sets as a function of count.   We did this in the "Switch configuration" UI, and then searching for a configuration.   We looked at each step in the process:
   * Time to open the initial dialog after selecting the "Switch" menu
   * Time to select the Requirements Manager Configuration type and waiting for the dialog to refresh
   * Time to enter a search string and display a filtered set of results 

There is some sensitivity when searching to the artifact counts, but this is not pronounced.    The other operations are not sensitive to counts.

<img src="%ATTACHURLPATH%/Switch_FindBaselines.png" alt="Switch_FindBaselines.png" width="50%" height="50%" />

<img src="%ATTACHURLPATH%/Switch_FindStreams.png" alt="Switch_FindStreams.png" width="50%" height="50%" />

<img src="%ATTACHURLPATH%/Switch_FindChangeSets.png" alt="Switch_FindChangeSets.png" width="50%" height="50%" />

---++ Appendix A:  Topology

The topology that was used in our testing is based on the [[https://jazz.net/wiki/bin/view/Deployment/RecommendedCLMDeploymentTopologies5#CLM_E1_Enterprise_Distributed_Li][standard (E1) Enterprise - Distributed / Linux / DB2 topology]]

<img src="%ATTACHURLPATH%/TestTopology.png" alt="TestTopology.png" width="50%" height="50%" />

Key:
   * RM Server: Rational DOORS Next Generation Server
   * JTS: Jazz™ Team Server

The performance tests described here did not include linking scenarios, and so the back link indexer and the global configuration application were not deployed into our test environment.

#TestTopology
---++ Appendix B:  Tested hardware and software configurations

All of the hardware that was tested was 64 bit and supported hyperthreading.   The following table shows the server hardware configurations that were used for all of the tests of all repository sizes.

<table id="_61" style="margin-left: 5px; " cellpadding="0" cellspacing="0" border="1">
   <tbody>
      <tr>
         <td colspan="1" id="_62" rowspan="1" style="width: 99px; vertical-align: top; ">
            <p dir="ltr" id="_63" style="margin-left: 5px; "><b>Function</b></p>
         </td>
         <td colspan="1" id="_64" rowspan="1" style="width: 75px; vertical-align: top; ">
            <p dir="ltr" id="_65" style="margin-left: 5px; "><b>Number of machines</b></p>
         </td>
         <td colspan="1" id="_66" rowspan="1" style="width: 71px; vertical-align: top; ">
            <p dir="ltr" id="_67" style="margin-left: 5px; "><b>Machine type</b></p>
         </td>
         <td colspan="1" id="_68" rowspan="1" style="width: 143px; vertical-align: top; ">
            <p dir="ltr" id="_69" style="margin-left: 5px; "><b>Processor/machine</b></p>
         </td>
         <td colspan="1" id="_70" rowspan="1" style="width: 141px; vertical-align: top; ">
            <p dir="ltr" id="_71" style="margin-left: 5px; "><b>Total processor cores/machines</b></p>
         </td>
         <td colspan="2" id="_72" rowspan="1" style="width: 140px; vertical-align: top; ">
            <p dir="ltr" id="_73" style="margin-left: 5px; "><b>Memory/machine</b></p>
         </td>
         <td colspan="1" id="_74" rowspan="1" style="width: 96px; vertical-align: top; ">
            <p dir="ltr" id="_75" style="margin-left: 5px; "><b>Network interface</b></p>
         </td>
         <td colspan="1" id="_76" rowspan="1" style="width: 106px; vertical-align: top; ">
            <p dir="ltr" id="_77" style="margin-left: 5px; "><b>OS and version</b></p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_78" rowspan="1" style="width: 99px; vertical-align: center; ">
            <p dir="ltr" id="_79" style="margin-left: 5px; ">Proxy Server (IBM HTTP Server and !WebSphere Plugin)</p>
         </td>
         <td colspan="1" id="_80" rowspan="1" style="width: 75px; vertical-align: center; ">
            <p dir="ltr" id="_81" style="margin-left: 5px; ">1</p>
         </td>
         <td colspan="1" id="_82" rowspan="1" style="width: 71px; vertical-align: center; ">
            <p dir="ltr" id="_83" style="margin-left: 5px; ">IBM System x3250 M4</p>
         </td>
         <td colspan="1" id="_84" rowspan="1" style="width: 143px; vertical-align: center; ">
            <p dir="ltr" id="_85" style="margin-left: 5px; ">1 x Intel Xeon E3-1240v2 3.4 GHz (quad-core)</p>
         </td>
         <td colspan="1" id="_86" rowspan="1" style="width: 141px; vertical-align: center; ">
            <p dir="ltr" id="_87" style="margin-left: 5px; ">8</p>
         </td>
         <td colspan="1" id="_88" rowspan="1" style="width: 140px; vertical-align: center; ">
            <p dir="ltr" id="_89" style="margin-left: 5px; ">16 GB</p>
         </td>
         <td colspan="2" id="_90" rowspan="1" style="width: 96px; vertical-align: center; ">
            <p dir="ltr" id="_91" style="margin-left: 5px; ">Gigabit Ethernet</p>
         </td>
         <td colspan="1" id="_92" rowspan="1" style="width: 106px; vertical-align: center; ">
            <p dir="ltr" id="_93" style="margin-left: 5px; ">Red Hat Enterprise Linux Server release 6.3 (Santiago)</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_94" rowspan="1" style="width: 99px; vertical-align: center; ">
            <p dir="ltr" id="_95" style="margin-left: 5px; ">Jazz Team Server</p>
            <p dir="ltr" id="_96" style="margin-left: 5px; "> !WebSphere Application Server 8.5.5.1</p>
         </td>
         <td colspan="1" id="_97" rowspan="1" style="width: 75px; vertical-align: center; ">
            <p dir="ltr" id="_98" style="margin-left: 5px; ">1</p>
         </td>
         <td colspan="1" id="_99" rowspan="1" style="width: 71px; vertical-align: center; ">
            <p dir="ltr" id="_100" style="margin-left: 5px; ">IBM System x3550 M4</p>
         </td>
         <td colspan="1" id="_101" rowspan="1" style="width: 143px; vertical-align: center; ">
            <p dir="ltr" id="_102" style="margin-left: 5px; ">2 x Intel Xeon E5-2640 2.5 GHz (six-core)</p>
         </td>
         <td colspan="1" id="_103" rowspan="1" style="width: 141px; vertical-align: center; ">
            <p dir="ltr" id="_104" style="margin-left: 5px; ">24</p>
         </td>
         <td colspan="1" id="_105" rowspan="1" style="width: 140px; vertical-align: center; ">
            <p dir="ltr" id="_106" style="margin-left: 5px; ">32 GB</p>
         </td>
         <td colspan="2" id="_107" rowspan="1" style="width: 96px; vertical-align: center; ">
            <p dir="ltr" id="_108" style="margin-left: 5px; ">Gigabit Ethernet</p>
         </td>
         <td colspan="1" id="_109" rowspan="1" style="width: 106px; vertical-align: center; ">
            <p dir="ltr" id="_110" style="margin-left: 5px; ">Red Hat Enterprise Linux Server release 6.3 (Santiago)</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_111" rowspan="1" style="width: 99px; vertical-align: center; ">
            <p dir="ltr" id="_112" style="margin-left: 5px; ">RM server</p>
            <p dir="ltr" id="_113" style="margin-left: 5px; "> !WebSphere Application Server 8.5.5.1</p>
         </td>
         <td colspan="1" id="_114" rowspan="1" style="width: 75px; vertical-align: center; ">
            <p dir="ltr" id="_115" style="margin-left: 5px; ">1</p>
         </td>
         <td colspan="1" id="_116" rowspan="1" style="width: 71px; vertical-align: center; ">
            <p dir="ltr" id="_117" style="margin-left: 5px; ">IBM System x3550 M4</p>
         </td>
         <td colspan="1" id="_118" rowspan="1" style="width: 143px; vertical-align: center; ">
            <p dir="ltr" id="_119" style="margin-left: 5px; ">2 x Intel Xeon E5-2640 2.5 GHz (six-core)</p>
         </td>
         <td colspan="1" id="_120" rowspan="1" style="width: 141px; vertical-align: center; ">
            <p dir="ltr" id="_121" style="margin-left: 5px; ">24</p>
         </td>
         <td colspan="1" id="_122" rowspan="1" style="width: 140px; vertical-align: center; ">
            <p dir="ltr" id="_123" style="margin-left: 5px; ">Varied, depending on the repository size and hard disk type.</p>
            <p dir="ltr" id="_124" style="margin-left: 5px; ">32 GB to 128 GB</p>
         </td>
         <td colspan="2" id="_125" rowspan="1" style="width: 96px; vertical-align: center; ">
            <p dir="ltr" id="_126" style="margin-left: 5px; ">Gigabit Ethernet</p>
         </td>
         <td colspan="1" id="_127" rowspan="1" style="width: 106px; vertical-align: center; ">
            <p dir="ltr" id="_128" style="margin-left: 5px; ">Red Hat Enterprise Linux Server release 6.3 (Santiago)</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_129" rowspan="1" style="width: 99px; vertical-align: center; ">
            <p dir="ltr" id="_130" style="margin-left: 5px; ">Database server</p>
            <p dir="ltr" id="_131" style="margin-left: 5px; ">DB2 10.1.2</p>
         </td>
         <td colspan="1" id="_132" rowspan="1" style="width: 75px; vertical-align: center; ">
            <p dir="ltr" id="_133" style="margin-left: 5px; ">1</p>
         </td>
         <td colspan="1" id="_134" rowspan="1" style="width: 71px; vertical-align: center; ">
            <p dir="ltr" id="_135" style="margin-left: 5px; ">IBM System x3650 M4</p>
         </td>
         <td colspan="1" id="_136" rowspan="1" style="width: 143px; vertical-align: center; ">
            <p dir="ltr" id="_137" style="margin-left: 5px; ">2 x Intel Xeon E5-2640 2.5 GHz (six-core)</p>
         </td>
         <td colspan="1" id="_138" rowspan="1" style="width: 141px; vertical-align: center; ">
            <p dir="ltr" id="_139" style="margin-left: 5px; ">24</p>
         </td>
         <td colspan="1" id="_140" rowspan="1" style="width: 140px; vertical-align: center; ">
            <p dir="ltr" id="_141" style="margin-left: 5px; ">64 GB</p>
         </td>
         <td colspan="2" id="_142" rowspan="1" style="width: 96px; vertical-align: center; ">
            <p dir="ltr" id="_143" style="margin-left: 5px; ">Gigabit Ethernet</p>
         </td>
         <td colspan="1" id="_144" rowspan="1" style="width: 106px; vertical-align: center; ">
            <p dir="ltr" id="_145" style="margin-left: 5px; ">Red Hat Enterprise Linux Server release 6.3 (Santiago)</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_146" rowspan="1" style="width: 99px; vertical-align: center; ">
            <p dir="ltr" id="_147" style="margin-left: 5px; ">Network switches</p>
         </td>
         <td colspan="1" id="_148" rowspan="1" style="width: 75px; vertical-align: center; ">
            <p dir="ltr" id="_149" style="margin-left: 5px; ">N/A</p>
         </td>
         <td colspan="1" id="_150" rowspan="1" style="width: 71px; vertical-align: center; ">
            <p dir="ltr" id="_151" style="margin-left: 5px; ">Cisco 2960G-24TC-L</p>
         </td>
         <td colspan="1" id="_152" rowspan="1" style="width: 143px; vertical-align: center; ">
            <p dir="ltr" id="_153" style="margin-left: 5px; ">N/A</p>
         </td>
         <td colspan="1" id="_154" rowspan="1" style="width: 141px; vertical-align: center; ">
            <p dir="ltr" id="_155" style="margin-left: 5px; ">N/A</p>
         </td>
         <td colspan="1" id="_156" rowspan="1" style="width: 140px; vertical-align: center; ">
            <p dir="ltr" id="_157" style="margin-left: 5px; ">&nbsp;</p>
         </td>
         <td colspan="2" id="_158" rowspan="1" style="width: 96px; vertical-align: center; ">
            <p dir="ltr" id="_159" style="margin-left: 5px; ">Gigabit Ethernet</p>
         </td>
         <td colspan="1" id="_160" rowspan="1" style="width: 106px; vertical-align: center; ">
            <p dir="ltr" id="_161" style="margin-left: 5px; ">24 Ethernet 10/100/1000 ports</p>
         </td>
      </tr>
   </tbody>
</table>

#DataShape
---++ Appendix C:  Test data shape and volume

The  test data that was used for these sizing guide tests represents extremely large repositories compared to most environments that are currently deployed.    The 6.0 tests used repositories containing 500K and 1 million artifacts.   The artifacts, modules, comments, links, and other elements were evenly distributed among the projects. Each project had data as shown in the 
following table.

<table id="_165" style="margin-left: 5px; " cellpadding="0" cellspacing="0" border="1">
   <tbody>
      <tr>
         <td colspan="1" id="_166" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_167" style="margin-left: 5px; "><b>&nbsp;Artifact type</b></p>
         </td>
         <td colspan="1" id="_168" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_169" style="margin-left: 5px; "><b>&nbsp;Number</b></p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_170" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_171" style="margin-left: 5px; ">Large modules (10,000 artifacts)</p>
         </td>
         <td colspan="1" id="_172" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_173" style="margin-left: 5px; ">2</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_174" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_175" style="margin-left: 5px; ">Medium modules (1500 artifacts)</p>
         </td>
         <td colspan="1" id="_176" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_177" style="margin-left: 5px; ">40</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_178" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_179" style="margin-left: 5px; ">Small modules (500 artifacts)</p>
         </td>
         <td colspan="1" id="_180" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_181" style="margin-left: 5px; ">10</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_182" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_183" style="margin-left: 5px; ">Folders</p>
         </td>
         <td colspan="1" id="_184" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_185" style="margin-left: 5px; ">119</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_186" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_187" style="margin-left: 5px; ">Module artifacts</p>
         </td>
         <td colspan="1" id="_188" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_189" style="margin-left: 5px; ">85000</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_190" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_191" style="margin-left: 5px; ">Non-module artifacts</p>
         </td>
         <td colspan="1" id="_192" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_193" style="margin-left: 5px; ">1181</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_194" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_195" style="margin-left: 5px; ">Comments</p>
         </td>
         <td colspan="1" id="_196" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_197" style="margin-left: 5px; ">260582</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_198" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_199" style="margin-left: 5px; ">Links</p>
         </td>
         <td colspan="1" id="_200" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_201" style="margin-left: 5px; ">304029</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_202" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_203" style="margin-left: 5px; ">Collections</p>
         </td>
         <td colspan="1" id="_204" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_205" style="margin-left: 5px; ">14</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_206" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_207" style="margin-left: 5px; ">Public tags</p>
         </td>
         <td colspan="1" id="_208" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_209" style="margin-left: 5px; ">300</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_210" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_211" style="margin-left: 5px; ">Private tags</p>
         </td>
         <td colspan="1" id="_212" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_213" style="margin-left: 5px; ">50</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_214" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_215" style="margin-left: 5px; ">Views</p>
         </td>
         <td colspan="1" id="_216" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_217" style="margin-left: 5px; ">200</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_218" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_219" style="margin-left: 5px; ">Terms</p>
         </td>
         <td colspan="1" id="_220" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_221" style="margin-left: 5px; ">238</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_218" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_219" style="margin-left: 5px; ">Streams</p>
         </td>
         <td colspan="1" id="_220" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_221" style="margin-left: 5px; ">15</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_218" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_219" style="margin-left: 5px; ">Baselines</p>
         </td>
         <td colspan="1" id="_220" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_221" style="margin-left: 5px; ">15</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_218" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_219" style="margin-left: 5px; ">Change sets</p>
         </td>
         <td colspan="1" id="_220" rowspan="1" style="width: 68px; vertical-align: center; ">
            <p dir="ltr" id="_221" style="margin-left: 5px; ">600</p>
         </td>
      </tr>
   </tbody>
</table>

Each repository that was tested contained a different number of projects. In each project, the data was distributed as shown in the previous table. The categories of RM artifacts in the repository were evenly distributed among all of the available projects in the repositories.

<table border="1" cellpadding="0" cellspacing="0" id="_223" style="margin-left:5px">
   <tbody>
      <tr>
         <td colspan="1" id="_224" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_225" style="margin-left: 5px; "><b> Artifact type</b></p>
         </td>
         <td colspan="1" id="_226" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_227" style="margin-left: 5px; "><b>500,000-artifact<br></br>         repository</b></p>
         </td>
         <td colspan="1" id="_228" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_229" style="margin-left: 5px; "><b> 1-million-artifact<br></br>         repository</b></p>
         </td>
         <td colspan="1" id="_230" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_231" style="margin-left: 5px; "><b> 2-million-artifact<br></br>         repository</b></p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_234" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_235" style="margin-left: 5px; ">Projects</p>
         </td>
         <td colspan="1" id="_236" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_237" style="margin-left: 5px; ">6</p>
         </td>
         <td colspan="1" id="_238" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_239" style="margin-left: 5px; ">12</p>
         </td>
         <td colspan="1" id="_240" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_241" style="margin-left: 5px; ">24</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_244" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_245" style="margin-left: 5px; ">Large modules (10,000 artifacts)</p>
         </td>
         <td colspan="1" id="_246" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_247" style="margin-left: 5px; ">12</p>
         </td>
         <td colspan="1" id="_248" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_249" style="margin-left: 5px; ">24</p>
         </td>
         <td colspan="1" id="_250" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_251" style="margin-left: 5px; ">48</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_254" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_255" style="margin-left: 5px; ">Medium modules (1500 artifacts)</p>
         </td>
         <td colspan="1" id="_256" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_257" style="margin-left: 5px; ">240</p>
         </td>
         <td colspan="1" id="_258" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_259" style="margin-left: 5px; ">480</p>
         </td>
         <td colspan="1" id="_260" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_261" style="margin-left: 5px; ">960</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_264" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_265" style="margin-left: 5px; ">Small modules (500 artifacts)</p>
         </td>
         <td colspan="1" id="_266" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_267" style="margin-left: 5px; ">60</p>
         </td>
         <td colspan="1" id="_268" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_269" style="margin-left: 5px; ">120</p>
         </td>
         <td colspan="1" id="_270" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_271" style="margin-left: 5px; ">240</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_274" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_275" style="margin-left: 5px; ">Folders</p>
         </td>
         <td colspan="1" id="_276" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_277" style="margin-left: 5px; ">714</p>
         </td>
         <td colspan="1" id="_278" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_279" style="margin-left: 5px; ">1428</p>
         </td>
         <td colspan="1" id="_280" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_281" style="margin-left: 5px; ">2856</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_284" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_285" style="margin-left: 5px; ">Module artifacts</p>
         </td>
         <td colspan="1" id="_286" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_287" style="margin-left: 5px; ">510,000</p>
         </td>
         <td colspan="1" id="_288" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_289" style="margin-left: 5px; ">1,020,000</p>
         </td>
         <td colspan="1" id="_290" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_291" style="margin-left: 5px; ">2,040,000</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_294" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_295" style="margin-left: 5px; ">Non-module artifacts</p>
         </td>
         <td colspan="1" id="_296" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_297" style="margin-left: 5px; ">7086</p>
         </td>
         <td colspan="1" id="_298" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_299" style="margin-left: 5px; ">14,172</p>
         </td>
         <td colspan="1" id="_300" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_301" style="margin-left: 5px; ">28344</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_304" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_305" style="margin-left: 5px; ">Comments</p>
         </td>
         <td colspan="1" id="_306" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_307" style="margin-left: 5px; ">1,563,492</p>
         </td>
         <td colspan="1" id="_308" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_309" style="margin-left: 5px; ">3,126,984</p>
         </td>
         <td colspan="1" id="_310" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_311" style="margin-left: 5px; ">6,253,968</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_314" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_315" style="margin-left: 5px; ">Links</p>
         </td>
         <td colspan="1" id="_316" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_317" style="margin-left: 5px; ">1,824,174</p>
         </td>
         <td colspan="1" id="_318" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_319" style="margin-left: 5px; ">3,648,348</p>
         </td>
         <td colspan="1" id="_320" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_321" style="margin-left: 5px; ">7,296,696</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_324" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_325" style="margin-left: 5px; ">Collections</p>
         </td>
         <td colspan="1" id="_326" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_327" style="margin-left: 5px; ">84</p>
         </td>
         <td colspan="1" id="_328" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_329" style="margin-left: 5px; ">168</p>
         </td>
         <td colspan="1" id="_330" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_331" style="margin-left: 5px; ">336</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_334" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_335" style="margin-left: 5px; ">Public tags</p>
         </td>
         <td colspan="1" id="_336" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_337" style="margin-left: 5px; ">1800</p>
         </td>
         <td colspan="1" id="_338" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_339" style="margin-left: 5px; ">3600</p>
         </td>
         <td colspan="1" id="_340" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_341" style="margin-left: 5px; ">7200</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_344" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_345" style="margin-left: 5px; ">Private tags</p>
         </td>
         <td colspan="1" id="_346" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_347" style="margin-left: 5px; ">300</p>
         </td>
         <td colspan="1" id="_348" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_349" style="margin-left: 5px; ">600</p>
         </td>
         <td colspan="1" id="_350" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_351" style="margin-left: 5px; ">1200</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_354" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_355" style="margin-left: 5px; ">Views</p>
         </td>
         <td colspan="1" id="_356" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_357" style="margin-left: 5px; ">1200</p>
         </td>
         <td colspan="1" id="_358" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_359" style="margin-left: 5px; ">2400</p>
         </td>
         <td colspan="1" id="_360" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_361" style="margin-left: 5px; ">4800</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_364" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_365" style="margin-left: 5px; ">Terms</p>
         </td>
         <td colspan="1" id="_366" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_367" style="margin-left: 5px; ">1428</p>
         </td>
         <td colspan="1" id="_368" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_369" style="margin-left: 5px; ">2856</p>
         </td>
         <td colspan="1" id="_370" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_371" style="margin-left: 5px; ">5712</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_364" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_365" style="margin-left: 5px; ">Streams</p>
         </td>
         <td colspan="1" id="_366" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_367" style="margin-left: 5px; ">90</p>
         </td>
         <td colspan="1" id="_368" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_369" style="margin-left: 5px; ">180</p>
         </td>
         <td colspan="1" id="_370" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_371" style="margin-left: 5px; ">360</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_364" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_365" style="margin-left: 5px; ">Baselines</p>
         </td>
         <td colspan="1" id="_366" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_367" style="margin-left: 5px; ">90</p>
         </td>
         <td colspan="1" id="_368" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_369" style="margin-left: 5px; ">180</p>
         </td>
         <td colspan="1" id="_370" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_371" style="margin-left: 5px; ">360</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_364" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_365" style="margin-left: 5px; ">Change sets</p>
         </td>
         <td colspan="1" id="_366" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_367" style="margin-left: 5px; ">600</p>
         </td>
         <td colspan="1" id="_368" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_369" style="margin-left: 5px; ">1200</p>
         </td>
         <td colspan="1" id="_370" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_371" style="margin-left: 5px; ">3600</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_374" rowspan="1" style="width: 222px; vertical-align: center; ">
            <p dir="ltr" id="_375" style="margin-left: 5px; "> Index size on disk</p>
         </td>
         <td colspan="1" id="_376" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_377" style="margin-left: 5px; "> 39 GB</p>
         </td>
         <td colspan="1" id="_378" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_379" style="margin-left: 5px; "> 75 GB</p>
         </td>
         <td colspan="1" id="_380" rowspan="1" style="width: 114px; vertical-align: center; ">
            <p dir="ltr" id="_381" style="margin-left: 5px; "> 157 GB</p>
         </td>
      </tr>
   </tbody>
</table>

#WorkloadCharacterization
---++ Appendix D:  Workload characterization

IBM® Rational® Performance Tester was used to simulate the workload. A Rational Performance Tester script was created for each use case. The scripts are organized by pages; each page represents a user action. Users were distributed into many user groups and each user group repeatedly runs one script (use case). The RM team used the user load stages in Rational Performance Tester to capture the performance for various user loads. 

We started with 100 simulated users, and increased the load by 100 users every 90 minutes.   After the 400 user stage, we increased the user load in increments of 50 up until failure.

In each stage, the users set up in 15 minutes. After they log in, they are allowed another 15 minutes to get settled. Then, tests are iterated for 45 minutes. During that time, tests were run with a 1 minute “think time” between operations for each user.

This table shows the use cases and the number of users who were repeatedly running each script.    Change sets are created in the default stream.

<table border="1" cellpadding="0" cellspacing="0" id="_397" style="margin-left:5px">
   <tbody>
      <tr>
         <th colspan="1" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_398" style="margin-left: 5px; "><b>Use case</b></p>
         </th>
         <th colspan="1" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_399" style="margin-left: 5px; "><b>Description</b></p>
         </th>
         <th colspan="1" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_400" style="margin-left: 5px; "><b>Number of users</b></p>
         </th>
      </tr>
      <tr>
         <td colspan="1" id="_401" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_402" style="margin-left: 5px; ">Edit stream</p>
         </td>
         <td colspan="1" id="_403" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_404" style="margin-left: 5px; ">Select the default configuration.   Navigate to the streams, baselines, and change set tabs..</p>
         </td>
         <td colspan="1" id="_405" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_406" style="margin-left: 5px; ">2%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_401" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_402" style="margin-left: 5px; ">Create stream/baseline</p>
         </td>
         <td colspan="1" id="_403" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_404" style="margin-left: 5px; ">Create a stream and a baseline as children of the default stream.   Runs 5 times per hour per user.</p>
         </td>
         <td colspan="1" id="_405" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_406" style="margin-left: 5px; ">1%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_401" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_402" style="margin-left: 5px; ">Select random baseline</p>
         </td>
         <td colspan="1" id="_403" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_404" style="margin-left: 5px; ">Open the Switch configuration UI;   select a random baseline and switch to it.</p>
         </td>
         <td colspan="1" id="_405" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_406" style="margin-left: 5px; ">1%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_401" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_402" style="margin-left: 5px; ">Select random stream</p>
         </td>
         <td colspan="1" id="_403" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_404" style="margin-left: 5px; ">Open the Switch configuration UI;   select a random stream and switch to it.</p>
         </td>
         <td colspan="1" id="_405" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_406" style="margin-left: 5px; ">1%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_401" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_402" style="margin-left: 5px; ">Copy/Paste/Move/Delete</p>
         </td>
         <td colspan="1" id="_403" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_404" style="margin-left: 5px; ">Create a change set.   Open a module that contains 1500 artifacts, select 25 artifacts, move them by using the copy and paste functions, and then delete the copied artifacts.   Deliver the change set.</p>
         </td>
         <td colspan="1" id="_405" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_406" style="margin-left: 5px; ">1%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_407" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_408" style="margin-left: 5px; ">Create an artifact</p>
         </td>
         <td colspan="1" id="_409" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_410" style="margin-left: 5px; ">Create a change set.  Create non-module artifacts.  Deliver the change set.</p>
         </td>
         <td colspan="1" id="_411" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_412" style="margin-left: 5px; ">3%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_413" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_414" style="margin-left: 5px; ">Create a collection</p>
         </td>
         <td colspan="1" id="_415" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_416" style="margin-left: 5px; ">Create a change set.  Create collections that contain 10 artifacts.  Deliver the change set.</p>
         </td>
         <td colspan="1" id="_417" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_418" style="margin-left: 5px; ">4%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_419" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_420" style="margin-left: 5px; ">Create a module artifact end-to-end scenario</p>
         </td>
         <td colspan="1" id="_421" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_422" style="margin-left: 5px; ">Create a change set.   Open a medium module that contains 1500 artifacts, create a module artifact, edit the new artifact, and delete the new artifact.   Deliver the change set 75% of the time;   discard the change set 25% of the time.</p>
         </td>
         <td colspan="1" id="_423" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_424" style="margin-left: 5px; ">12%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_425" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_426" style="margin-left: 5px; ">Create a small module artifact end-to-end scenario</p>
         </td>
         <td colspan="1" id="_427" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_428" style="margin-left: 5px; ">create a change set. Open a small module that contains 500 artifacts, create a module artifact, edit that new artifact, and delete the new artifact.   Deliver the change set 50% of the time;   discard the change set 50% of the time.</p>
         </td>
         <td colspan="1" id="_429" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_430" style="margin-left: 5px; ">6%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_431" rowspan="1" style="width: 167px; vertical-align: center; ">
            <p dir="ltr" id="_432" style="margin-left: 5px; ">Create a comment in a module artifact</p>
         </td>
         <td colspan="1" id="_433" rowspan="1" style="width: 562px; vertical-align: center; ">
            <p dir="ltr" id="_434" style="margin-left: 5px; ">Create a change set.   Open a medium module that contains 1500 artifacts, open an artifact that is in the module, expand the Comments section of the artifact, and create a comment.    Deliver the change set 83% of the time;   discard the change set 17% of the time.</p>
         </td>
         <td colspan="1" id="_435" rowspan="1" style="width: 142px; vertical-align: center; ">
            <p dir="ltr" id="_436" style="margin-left: 5px; ">18%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_449" rowspan="1" style="width: 167px; vertical-align: top; ">
            <p dir="ltr" id="_450" style="margin-left: 5px; ">Hover over a module artifact and edit it</p>
         </td>
         <td colspan="1" id="_451" rowspan="1" style="width: 562px; vertical-align: top; ">
            <p dir="ltr" id="_452" style="margin-left: 5px; ">Create a change set.  Open a module that contains 1500 artifacts and hover over an artifact. When the rich hover is displayed, edit the artifact text.   Deliver the change set.</p>
         </td>
         <td colspan="1" id="_453" rowspan="1" style="width: 142px; vertical-align: top; ">
            <p dir="ltr" id="_454" style="margin-left: 5px; ">2%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_455" rowspan="1" style="width: 167px; vertical-align: top; ">
            <p dir="ltr" id="_456" style="margin-left: 5px; ">Hover over and open a collection</p>
         </td>
         <td colspan="1" id="_457" rowspan="1" style="width: 562px; vertical-align: top; ">
            <p dir="ltr" id="_458" style="margin-left: 5px; ">Display all of the collections, hover over a collection, and then open it.</p>
         </td>
         <td colspan="1" id="_459" rowspan="1" style="width: 142px; vertical-align: top; ">
            <p dir="ltr" id="_460" style="margin-left: 5px; ">1%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_461" rowspan="1" style="width: 167px; vertical-align: top; ">
            <p dir="ltr" id="_462" style="margin-left: 5px; ">Manage folders</p>
         </td>
         <td colspan="1" id="_463" rowspan="1" style="width: 562px; vertical-align: top; ">
            <p dir="ltr" id="_464" style="margin-left: 5px; ">Click “Show Artifacts” to display folder tree and then create a folder. Move the new folder into another folder and then delete the folder that you just created.</p>
         </td>
         <td colspan="1" id="_465" rowspan="1" style="width: 142px; vertical-align: top; ">
            <p dir="ltr" id="_466" style="margin-left: 5px; ">1%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_473" rowspan="1" style="width: 167px; vertical-align: top; ">
            <p dir="ltr" id="_474" style="margin-left: 5px; ">Open the project dashboard</p>
         </td>
         <td colspan="1" id="_475" rowspan="1" style="width: 562px; vertical-align: top; ">
            <p dir="ltr" id="_476" style="margin-left: 5px; ">Open a dashboard that displays the default dashboard.</p>
         </td>
         <td colspan="1" id="_477" rowspan="1" style="width: 142px; vertical-align: top; ">
            <p dir="ltr" id="_478" style="margin-left: 5px; ">5%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_479" rowspan="1" style="width: 167px; vertical-align: top; ">
            <p dir="ltr" id="_480" style="margin-left: 5px; ">Search by ID and string</p>
         </td>
         <td colspan="1" id="_481" rowspan="1" style="width: 562px; vertical-align: top; ">
            <p dir="ltr" id="_482" style="margin-left: 5px; ">Open a project, select a folder, search for an artifact by its numeric ID, and click a search result to display an artifact.</p>
            <p dir="ltr" id="_483" style="margin-left: 5px; ">Search for artifacts by using a generic string search that produces about 50 results.</p>
         </td>
         <td colspan="1" id="_484" rowspan="1" style="width: 142px; vertical-align: top; ">
            <p dir="ltr" id="_485" style="margin-left: 5px; ">9%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_486" rowspan="1" style="width: 167px; vertical-align: top; ">
            <p dir="ltr" id="_487" style="margin-left: 5px; ">Scroll 20 pages in a module</p>
         </td>
         <td colspan="1" id="_488" rowspan="1" style="width: 562px; vertical-align: top; ">
            <p dir="ltr" id="_489" style="margin-left: 5px; ">Open a module that contains 1500 artifacts and then scroll through 20 pages.</p>
         </td>
         <td colspan="1" id="_490" rowspan="1" style="width: 142px; vertical-align: top; ">
            <p dir="ltr" id="_491" style="margin-left: 5px; ">16%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_492" rowspan="1" style="width: 167px; vertical-align: top; ">
            <p dir="ltr" id="_493" style="margin-left: 5px; ">Switch the module view</p>
         </td>
         <td colspan="1" id="_494" rowspan="1" style="width: 562px; vertical-align: top; ">
            <p dir="ltr" id="_495" style="margin-left: 5px; ">Open a module that contains 1500 artifacts and then change the view to add columns that display user-defined attributes.</p>
         </td>
         <td colspan="1" id="_496" rowspan="1" style="width: 142px; vertical-align: top; ">
            <p dir="ltr" id="_497" style="margin-left: 5px; ">14%</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_498" rowspan="1" style="width: 167px; vertical-align: top; ">
            <p dir="ltr" id="_499" style="margin-left: 5px; ">Upload a 4 MB file as a new artifact</p>
         </td>
         <td colspan="1" id="_500" rowspan="1" style="width: 562px; vertical-align: top; ">
            <p dir="ltr" id="_501" style="margin-left: 5px; ">Upload a file and create an artifact.</p>
         </td>
         <td colspan="1" id="_502" rowspan="1" style="width: 142px; vertical-align: top; ">
            <p dir="ltr" id="_503" style="margin-left: 5px; ">4%</p>
         </td>
      </tr>
   </tbody>
</table>

#ThroughputOfTests
---+++ Throughput of tests
As explained in the workload characterization, the test load was aggressive and it simulated the load of very active users. After the users were logged in, they continuously ran the scripts, waiting for only 1 minute before running each page. During the tests, transaction throughput statistics were gathered. Those statistics were the basis for the high-level overview of server activity that is shown in the next table.

<table border="1" cellpadding="0" cellspacing="0" id="_525" style="margin-left:5px">
   <tbody>
      <tr>
         <td colspan="1" id="_526" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_527" style="margin-left: 5px; "><b>Server activity</b></p>
         </td>
         <td colspan="1" id="_528" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_529" style="margin-left: 5px; "><b>100 users in 1 hour</b></p>
         </td>
         <td colspan="1" id="_530" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_531" style="margin-left: 5px; "><b>100 users in 8 hours</b></p>
         </td>
         <td colspan="1" id="_532" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_533" style="margin-left: 5px; "><b>400 users in 8 hours</b></p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_534" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_535" style="margin-left: 5px; ">Number of artifacts created</p>
         </td>
         <td colspan="1" id="_536" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_537" style="margin-left: 5px; ">369</p>
         </td>
         <td colspan="1" id="_538" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_539" style="margin-left: 5px; ">2952</p>
         </td>
         <td colspan="1" id="_540" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_541" style="margin-left: 5px; ">11808</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_542" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_543" style="margin-left: 5px; ">Number of artifacts opened </p>
         </td>
         <td colspan="1" id="_544" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_545" style="margin-left: 5px; ">280</p>
         </td>
         <td colspan="1" id="_546" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_547" style="margin-left: 5px; ">2240</p>
         </td>
         <td colspan="1" id="_548" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_549" style="margin-left: 5px; ">8960</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_550" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_551" style="margin-left: 5px; ">Number of artifacts edited or deleted</p>
         </td>
         <td colspan="1" id="_552" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_553" style="margin-left: 5px; ">308</p>
         </td>
         <td colspan="1" id="_554" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_555" style="margin-left: 5px; ">2464</p>
         </td>
         <td colspan="1" id="_556" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_557" style="margin-left: 5px; ">9856</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_558" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_559" style="margin-left: 5px; ">Display a list of modules</p>
         </td>
         <td colspan="1" id="_560" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_561" style="margin-left: 5px; ">591</p>
         </td>
         <td colspan="1" id="_562" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_563" style="margin-left: 5px; ">4728</p>
         </td>
         <td colspan="1" id="_564" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_565" style="margin-left: 5px; ">18912</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_566" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_567" style="margin-left: 5px; ">Comments created</p>
         </td>
         <td colspan="1" id="_568" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_569" style="margin-left: 5px; ">162</p>
         </td>
         <td colspan="1" id="_570" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_571" style="margin-left: 5px; ">1296</p>
         </td>
         <td colspan="1" id="_572" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_573" style="margin-left: 5px; ">5184</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_582" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_583" style="margin-left: 5px; ">Modules opened</p>
         </td>
         <td colspan="1" id="_584" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_585" style="margin-left: 5px; ">588</p>
         </td>
         <td colspan="1" id="_586" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_587" style="margin-left: 5px; ">4704</p>
         </td>
         <td colspan="1" id="_588" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_589" style="margin-left: 5px; ">18816</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_590" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_591" style="margin-left: 5px; ">Search by ID and open the artifact</p>
         </td>
         <td colspan="1" id="_592" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_593" style="margin-left: 5px; ">133</p>
         </td>
         <td colspan="1" id="_594" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_595" style="margin-left: 5px; ">1064</p>
         </td>
         <td colspan="1" id="_596" rowspan="1" style="  width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_597" style="margin-left: 5px; ">4256</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_598" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_599" style="margin-left: 5px; ">Search by string</p>
         </td>
         <td colspan="1" id="_600" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_601" style="margin-left: 5px; ">132</p>
         </td>
         <td colspan="1" id="_602" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_603" style="margin-left: 5px; ">1056</p>
         </td>
         <td colspan="1" id="_604" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_605" style="margin-left: 5px; ">4224</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_606" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_607" style="margin-left: 5px; ">Switch module view to filter by attribute</p>
         </td>
         <td colspan="1" id="_608" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_609" style="margin-left: 5px; ">270</p>
         </td>
         <td colspan="1" id="_610" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_611" style="margin-left: 5px; ">2160</p>
         </td>
         <td colspan="1" id="_612" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_613" style="margin-left: 5px; ">8640</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_614" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_615" style="margin-left: 5px; ">Number of module pages scrolled</p>
         </td>
         <td colspan="1" id="_616" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_617" style="margin-left: 5px; ">834</p>
         </td>
         <td colspan="1" id="_618" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_619" style="margin-left: 5px; ">6672</p>
         </td>
         <td colspan="1" id="_620" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_621" style="margin-left: 5px; ">26688</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_622" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_623" style="margin-left: 5px; ">Create stream</p>
         </td>
         <td colspan="1" id="_624" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_625" style="margin-left: 5px; ">8</p>
         </td>
         <td colspan="1" id="_626" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_627" style="margin-left: 5px; ">64</p>
         </td>
         <td colspan="1" id="_628" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_629" style="margin-left: 5px; ">256</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_574" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_575" style="margin-left: 5px; ">Create baseline</p>
         </td>
         <td colspan="1" id="_576" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_577" style="margin-left: 5px; ">8          </p>
         </td>
         <td colspan="1" id="_578" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_579" style="margin-left: 5px; ">64</p>
         </td>
         <td colspan="1" id="_580" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_581" style="margin-left: 5px; ">256</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_574" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_575" style="margin-left: 5px; ">Switch to a stream</p>
         </td>
         <td colspan="1" id="_576" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_577" style="margin-left: 5px; ">104        </p>
         </td>
         <td colspan="1" id="_578" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_579" style="margin-left: 5px; ">832</p>
         </td>
         <td colspan="1" id="_580" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_581" style="margin-left: 5px; ">3328</p>
         </td>
      </tr>      <tr>
         <td colspan="1" id="_574" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_575" style="margin-left: 5px; ">Switch to a baseline</p>
         </td>
         <td colspan="1" id="_576" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_577" style="margin-left: 5px; ">80         </p>
         </td>
         <td colspan="1" id="_578" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_579" style="margin-left: 5px; ">640</p>
         </td>
         <td colspan="1" id="_580" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_581" style="margin-left: 5px; ">2560</p>
         </td>
      </tr>      <tr>
         <td colspan="1" id="_574" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_575" style="margin-left: 5px; ">Browse the current configuration</p>
         </td>
         <td colspan="1" id="_576" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_577" style="margin-left: 5px; ">101          </p>
         </td>
         <td colspan="1" id="_578" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_579" style="margin-left: 5px; ">808</p>
         </td>
         <td colspan="1" id="_580" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_581" style="margin-left: 5px; ">3232</p>
         </td>
      </tr>      <tr>
         <td colspan="1" id="_574" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_575" style="margin-left: 5px; ">Create change set</p>
         </td>
         <td colspan="1" id="_576" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_577" style="margin-left: 5px; ">360         </p>
         </td>
         <td colspan="1" id="_578" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_579" style="margin-left: 5px; ">2880</p>
         </td>
         <td colspan="1" id="_580" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_581" style="margin-left: 5px; ">11520</p>
         </td>
      </tr>      <tr>
         <td colspan="1" id="_574" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_575" style="margin-left: 5px; ">Discard change set</p>
         </td>
         <td colspan="1" id="_576" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_577" style="margin-left: 5px; ">68          </p>
         </td>
         <td colspan="1" id="_578" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_579" style="margin-left: 5px; ">544</p>
         </td>
         <td colspan="1" id="_580" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_581" style="margin-left: 5px; ">2176</p>
         </td>
      </tr>      <tr>
         <td colspan="1" id="_574" rowspan="1" style="  width: 339px; vertical-align: top; ">
            <p dir="ltr" id="_575" style="margin-left: 5px; ">Deliverchange set</p>
         </td>
         <td colspan="1" id="_576" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_577" style="margin-left: 5px; ">290 </p>
         </td>
         <td colspan="1" id="_578" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_579" style="margin-left: 5px; ">2320</p>
         </td>
         <td colspan="1" id="_580" rowspan="1" style="width: 214px; vertical-align: top; ">
            <p dir="ltr" id="_581" style="margin-left: 5px; ">9280</p>
         </td>
      </tr>
   </tbody>
</table>

---++ Appendix E:  Server tuning

---+++ JVM heap and GC tuning

In general, the team used 50% of the available RAM as the JVM heap size and did not exceed the maximum heap size of 24 GB. Because the Jazz Team Server had 32 GB of RAM, the team used 16 GB for the maximum heap size. The JVM arguments that the team used to set the JVM heap sizes and the GC policy are as follows: 

<verbatim> -Xgcpolicy:gencon –Xmx16g –Xms16g –Xmn4G -Xcompressedrefs -Xgc:preferredHeapBase=0x100000000 </verbatim>

For the DNG server, we used a nursery (-Xmn) of 1/3 the maximum heap size.

---++ Appendix F:  Database server hardware 

The database server that was used for the testing was IBM DB2 enterprise version 10.1.2. The same hardware and database settings were used for all of the performance tests, irrespective of the repository size. 

For a database server, the team used an IBM System x3650 M4 with 2 x Intel Xeon E5-2640 2.5 GHz (six-core) and 64G of total RAM . The total number of processor cores/machines was 24. In the tests, one server was used for both the Jazz Team Server and RM databases.    The database server that was used for the testing was IBM DB2 enterprise version 10.1.2. The same hardware and database settings were used for all of the performance tests, irrespective of the repository size.

In our tests, performance bottlenecks always formed first on the DNG server, due to the interactions around the disk-based indices.   The database server was not the limited factor.    However, the 6.0 release does use the database server more heavily for storing version information, so we expect that a higher end database server with fast disks)  will improve performance as the number of versions and configurations increases.

---+++ Hard disk capacity and configuration
The database servers should use the RAID 10 configuration with the RAID controller configured with "write back cache."     The team observed performance degradation in some use cases when using "write-through" caching.  Use a high capacity and high performing hard disk (10K RPM or better). 

<table border="1" cellpadding="0" cellspacing="0" id="_656" style="margin-left:5px">
   <tbody>
      <tr>
         <td colspan="1" id="_657" rowspan="1" style="width: 155px; vertical-align: center; ">
            <p dir="ltr" id="_658" style="margin-left: 5px; "><b> Repository</b></p>
         </td>
         <td colspan="1" id="_659" rowspan="1" style="width: 462px; vertical-align: center; ">
            <p dir="ltr" id="_660" style="margin-left: 5px; "><b>Combined hard disk usage of the Jazz Team Server and RM databases</b></p>
         </td>
         <td colspan="1" id="_661" rowspan="1" style="width: 254px; vertical-align: center; ">
            <p dir="ltr" id="_662" style="margin-left: 5px; "><b> Hard disk size </b></p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_663" rowspan="1" style="width: 155px; vertical-align: center; ">
            <p dir="ltr" id="_664" style="margin-left: 5px; ">Up to 500,000 artifacts</p>
         </td>
         <td colspan="1" id="_665" rowspan="1" style="width: 462px; vertical-align: center; ">
            <p dir="ltr" id="_666" style="margin-left: 5px; ">~100 GB</p>
         </td>
         <td colspan="1" id="_667" rowspan="1" style="width: 254px; vertical-align: center; ">
            <p dir="ltr" id="_668" style="margin-left: 5px; ">300 GB</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_669" rowspan="1" style="width: 155px; vertical-align: center; ">
            <p dir="ltr" id="_670" style="margin-left: 5px; ">Up to1 million artifacts</p>
         </td>
         <td colspan="1" id="_671" rowspan="1" style="width: 462px; vertical-align: center; ">
            <p dir="ltr" id="_672" style="margin-left: 5px; ">~150 GB</p>
         </td>
         <td colspan="1" id="_673" rowspan="1" style="width: 254px; vertical-align: center; ">
            <p dir="ltr" id="_674" style="margin-left: 5px; ">400 GB</p>
         </td>
      </tr>
      <tr>
         <td colspan="1" id="_675" rowspan="1" style="width: 155px; vertical-align: center; ">
            <p dir="ltr" id="_676" style="margin-left: 5px; ">Up to 2 million artifacts</p>
         </td>
         <td colspan="1" id="_677" rowspan="1" style="width: 462px; vertical-align: center; ">
            <p dir="ltr" id="_678" style="margin-left: 5px; ">~300 GB</p>
         </td>
         <td colspan="1" id="_679" rowspan="1" style="width: 254px; vertical-align: center; ">
            <p dir="ltr" id="_680" style="margin-left: 5px; ">700 GB</p>
         </td>
      </tr>
   </tbody>
</table>

---++ Appendix G:  Proxy server hardware and configuration (for repositories of all sizes)
The team used the same server hardware to test repositories of all sizes. In general, the proxy server does not slow the performance of the RM server.

---+++ Processor
The team used IBM System x3250 M4 with 1 x Intel Xeon E3-1240v2 3.4 GHz (quad-core). The total number of processor cores/machines was 8. 

---+++ IHS tuning
For the IHS systems that were used during the tests, in the server pool regulation section (worker.c) of httpd.conf file, these parameters were configured:

| !ThreadLimit |  25  |
| !ServerLimit |  100  |
| !StartServers |  2  |
| !MaxClients |  2500  |
| !SpareThreads |  25  |
| !MaxSpareThreads |  500  |
| !ThreadsPerChild |  25  |
| !MaxRequestsPerChild |  0  |

#AppendixH
---++ Appendix H: Server performance benchmarks 

Various server performance benchmarks were captured by using Sysbench. The performance benchmarks were captured on the RM server.

---+++ Using the CPU workload: sysbench --test=cpu --cpu-max-prime=20000 --num-threads=2 run
<verbatim>
[root@rtpclmperf54 ~]# sysbench --test=cpu --cpu-max-prime=20000 --num-threads=2 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 2

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          14.1474s
    total number of events:              10000
    total time taken by event execution: 28.2911
    per-request statistics:
         min:                                  2.81ms
         avg:                                  2.83ms
         max:                                  4.57ms
         approx.  95 percentile:               2.83ms

Threads fairness:
    events (avg/stddev):           5000.0000/6.00
    execution time (avg/stddev):   14.1456/0.00
</verbatim>

---+++ Using the threads workload: sysbench --test=threads --thread-locks=1 --max-time=20s run
<verbatim>
[root@rtpclmperf54 ~]# sysbench --test=threads --thread-locks=1 --max-time=20s run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing thread subsystem performance test
Thread yields per test: 1000 Locks used: 1
Threads started!
Done.


Test execution summary:
    total time:                          2.4065s
    total number of events:              10000
    total time taken by event execution: 2.4051
    per-request statistics:
         min:                                  0.24ms
         avg:                                  0.24ms
         max:                                  0.66ms
         approx.  95 percentile:               0.25ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   2.4051/0.00
</verbatim>

---+++ Using the mutex workload: sysbench --test=mutex --num-threads=64 run
<verbatim>
[root@rtpclmperf54 ~]# sysbench --test=mutex --num-threads=64 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Doing mutex performance test
Threads started!
Done.


Test execution summary:
    total time:                          1.7035s
    total number of events:              64
    total time taken by event execution: 105.7884
    per-request statistics:
         min:                               1526.68ms
         avg:                               1652.94ms
         max:                               1701.69ms
         approx.  95 percentile:            1699.11ms

Threads fairness:
    events (avg/stddev):           1.0000/0.00
    execution time (avg/stddev):   1.6529/0.04
</verbatim>
---+++ Using the memory workload: sysbench --test=memory --num-threads=4 run
<verbatim>
[root@rtpclmperf54 ~]# sysbench --test=memory --num-threads=4 run 
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 4

Doing memory operations speed test
Memory block size: 1K

Memory transfer size: 102400M

Memory operations type: write
Memory scope type: global
Threads started!
Done.

Operations performed: 104857600 (1279460.43 ops/sec)

102400.00 MB transferred (1249.47 MB/sec)


Test execution summary:
    total time:                          81.9545s
    total number of events:              104857600
    total time taken by event execution: 161.3530
    per-request statistics:
         min:                                  0.00ms
         avg:                                  0.00ms
         max:                                  2.03ms
         approx.  95 percentile:               0.00ms

Threads fairness:
    events (avg/stddev):           26214400.0000/947576.28
    execution time (avg/stddev):   40.3383/0.67
</verbatim>

#AppendixI
---++ Appendix I: Hard disk performance benchmarks 

---+++ HDD performance of RM server measured using the fileio workload
<verbatim>

[root@rtpclmperf54 home]# sysbench --test=fileio --file-total-size=150G --file-test-mode=rndrw --max-time=300 --max-requests=0 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 0
128 files, 1.1719Gb each
150Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed:  83823 Read, 55882 Write, 178816 Other = 318521 Total
Read 1.279Gb  Written 873.16Mb  Total transferred 2.1317Gb  (7.2762Mb/sec)
  465.67 Requests/sec executed

Test execution summary:
    total time:                          300.0057s
    total number of events:              139705
    total time taken by event execution: 292.0346
    per-request statistics:
         min:                                  0.00ms
         avg:                                  2.09ms
         max:                                116.53ms
         approx.  95 percentile:               5.80ms

Threads fairness:
    events (avg/stddev):           139705.0000/0.00
    execution time (avg/stddev):   292.0346/0.00

</verbatim>
---+++ SSD performance of RM server measured using the fileio workload
<verbatim>

[root@rtpclmperf54 data]# sysbench --test=fileio --file-total-size=150G --file-test-mode=rndrw --max-time=300 --max-requests=0 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 0
128 files, 1.1719Gb each
150Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed:  1039203 Read, 692802 Write, 2216960 Other = 3948965 Total
Read 15.857Gb  Written 10.571Gb  Total transferred 26.428Gb  (90.209Mb/sec)
 5773.35 Requests/sec executed

Test execution summary:
    total time:                          300.0002s
    total number of events:              1732005
    total time taken by event execution: 197.4660
    per-request statistics:
         min:                                  0.00ms
         avg:                                  0.11ms
         max:                                  9.51ms
         approx.  95 percentile:               0.25ms

Threads fairness:
    events (avg/stddev):           1732005.0000/0.00
    execution time (avg/stddev):   197.4660/0.00

</verbatim>
---+++++!! Related topics: [[DeploymentWebHome][Deployment web home]], [[DeploymentWebHome][Deployment web home]]

---+++++!! External links:          

   * [[https://www.ibm.com][IBM]]

---+++++!! Additional contributors: 
<sticky></div></sticky>
Attachments
Topic attachments
I	Attachment	Action	Size	Date	Who	Comment
png	1MResults.png	manage	51.6 K	2015-05-28 - 15:14	UnknownUser	Test results for 1M repository
png	500KResults.png	manage	24.1 K	2015-05-28 - 15:15	UnknownUser	Test results for 500K repository
png	CompareStream.png	manage	8.1 K	2015-05-12 - 16:32	VaughnRokosz	Response times: compare streams
png	DeliverChangeSet.png	manage	8.3 K	2015-05-12 - 16:31	VaughnRokosz	Response times: deliver change set
png	DeliverToStream.png	manage	8.0 K	2015-05-12 - 17:54	VaughnRokosz	Response times: deliver between streams
png	SummaryTable.png	manage	31.7 K	2015-05-28 - 15:16	UnknownUser	Summary of test results across all configurations
png	Switch_FindBaselines.png	manage	7.0 K	2015-05-12 - 17:55	VaughnRokosz	Response times: switch configuration -> search for baseline
png	Switch_FindChangeSets.png	manage	6.9 K	2015-05-12 - 17:55	VaughnRokosz	Response times: switch configuration -> search for change sets
png	Switch_FindStreams.png	manage	6.8 K	2015-05-12 - 17:56	VaughnRokosz	Response times: switch configuration -> search for streams
png	TestTopology.png	manage	21.5 K	2015-05-11 - 17:50	VaughnRokosz	TestTopology
Copyright © by IBM and non-IBM contributing authors. All material on this collaboration platform is the property of the contributing authors.
Contributions are governed by our Terms of Use. Please read the following disclaimer.
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.