CLM Sizing Strategy

Introduction

Whether new users or seasoned experts, customers using IBM Jazz products all want the same thing: They want to use the Jazz products without worrying that their deployment implementation will slow them down, that it will keep up with them as they add users and grow. A frequent question we hear, whether it’s from a new administrator setting up Collaborative Lifecycle Management (CLM) for the first time, or an experienced administrator tuning their Systems and Software Engineering (SSE) toolset, is “How many users will my environment support?”

Back when Rational Team Concert (RTC) was in its infancy we built a comprehensive performance test environment based on what we thought was a representative workload. It was in fact based upon the workload the RTC and Jazz teams itself used to develop the product. We published what we learned in our first Sizing Guide. Later sizing guides include: Collaborative Lifecycle Management 2011 Sizing Guide and Collaborative Lifecycle Management 2012 Sizing Report (Standard Topology E1). As features were added and the release grew, we started to hear about what folks were doing in the field. The Jazz products, RTC especially, are so flexible that customers were using them with wonderfully different workloads than we had anticipated.

Consequently, we stepped back from proclaiming a one-size fits all approach, and moved to presenting case studies and specific test reports about the user workload simulations and the loads we tested. We have published these reports on the jazz.net Deployment wiki at Performance datasheets. We have tried to make a distinction between performance reports and sizing guides. Performance reports document a specific test with defined hardware, datashape and workload, whereas sizing guides suggest patterns or categories of hardware, datashape and workload. Sizing reports are not specific and general descriptions of topologies and estimations of workloads they may support.

Throughout the many 4.0.x release cycles, we were still asked “How many users will my environment support?” Our reluctance to answer this apparently straightforward question frustrated customers new and old. Everyone thinks that as the Jazz experts we should know how to size our products. Finally, after some analysis and messing up countless whiteboards, we would like to present some sizing strategies and advice for the front-end applications in the Jazz platform: Rational Team Concert (RTC), Rational Requirements Composer (RRC)/Rational DOORS Next Generation (DNG) and Rational Quality Manager (RQM). These recommendations are based upon our product testing and analysis of customer deployments.

Standard disclaimer

A sizing estimate is an approximation of the hardware or software resources required to support an application set that is either currently implemented or new. The level of effort and scope of sizing degree of variability can range from small to very significant. Sizing does not constitute a performance guarantee. Customer results may vary, and IBM assumes no liability for actual results that differ from any estimates provided by IBM.

Where does this data come from?

The data in this sizing advice document comes from tests executed by the Rational Performance Engineering team and hardware and sizing models generated by IBM Global Techline, an internal pre-sales technical support organization. Further refinements of user patterns are derived from our analysis of many customer deployments. Specific references to the relevant data and test reports are cited below.

Basic principles

• The jazz products are wonderfully flexible. Whether you are new to Agile or a continuous integration junkie, the Jazz products will make planning, tracking, collaborating and delivering your project easier than ever before. They can be deployed in variety of configurations and they can be used in any number of ways. They can be used by large teams and small teams, co-located teams or widely distributed teams.
• What works for one customer might not work for another. Each organization has its own processes, rules, templates and habits. Each organization has valuable employees with varied skills and experiences. The workflow and datashape in use at one company will probably not meet the needs and demands of any other company. Each company has rules about how systems and hardware are configured and maintained, and one organization’s compliance may not be appropriate for another’s. Every organization has unique network characteristics, user authentication, VPNs and customized desktop configurations. Given all these and possibly other innate differences, the identical hardware deployed at one customer cannot be guaranteed to behave similarly to the exact deployment at another customer.
• It is normal for a customer’s workload to change during a project lifecycle. As end-users discover their optimal ways to work with the flexible Jazz tools, it’s not uncommon that the way they interact with the Jazz products changes over time. It isn’t just that more requirements are created at the beginning of a project, or that more defects are sometimes logged towards the end, it’s that as teams practice Agile and get used to iterative development, they work more collaboratively and leverage the strengths of the Jazz platform. For example, in some organizations, the number of work items in plans decrease, however work items increase their parent-child relationships.

Recommended topologies

We have outlined recommended deployment topologies for the CLM and SSE products on the jazz.net Deployment wiki. Customers implementing the Rational solution for Collaborative Lifecycle Management (CLM) or the Rational solution for Systems and Software Engineering (SSE) often ask how they should deploy the tools so that they perform well, run robustly, and can evolve without restriction. To simplify the wide range of choices, the Standard Topologies article series outline several standard topologies which are the expected and most frequently chosen deployment patterns.

Minimum requirements and practical limitations

We suggest an ideal minimum of 4 logical processors and 8 GB RAM on a 64-bit machine. The absolute minimum is 4 GB of RAM on a 64-bit machine, but for an enterprise deployment we prefer to suggest the ideal minimum. Also see System Requirements for Collaborative Lifecycle Management 4.0.5 and 4.0.6 and System Requirements for Systems and Software Engineering 4.0.5 and 4.0.6.

You will notice that the published test reports utilize hardware (cores and RAM) which are sized generously. A well designed deployment will be able to handle peak loads and unexpected usage patterns. We customarily size environments so that they may be able to accommodate bursts of up to double the amount of expected normal load. However, we do not recommending presuming an environment can handle such extreme loads for more than 4% of their standard operational time.

A deployment’s performance can be influenced by many things

The performance and behavior of a CLM or SSE deployment can be influenced by many factors or dimensions. These dimensions fall into two categories: application and product related (functional dimensions) and environment or topology related (non-functional dimensions). CLM and SSE’s flexibility and performance can be undermined by unintentional functional and non-functional design choices.

It is easy to design an ideal topology which presumes zero-second latency, unlimited bandwidth and ever-increasing storage capacity, but such capabilities are impossible in real deployments where there are known and possibly unexpected constraints and limits. It is important to be mindful of the gamut of possible constraints for the various dimensions of a CLM and SSE environment.

This table collects some of the factors which may influence performance. This table is by no means comprehensive. There may be other factors not listed here. A factor listed here may have different effect in different environments.

Given assumptions and caveats

• All hardware and operating systems are 64-bit.
• Representative product workloads (examples of how teams actually use the tools and what sort of load and transactions may occur) are described within the Rational Performance Engineering datasheets accessible here.
• For these examples, each application (Rational Team Concert, Rational Requirements Composer/Rational DOORS Next Generation, and Rational Quality Manager) are hosted on a dedicated server and connected to a separate Jazz Team Server (JTS). This document does not address how the Jazz Team Server (JTS) scales. JTS scaling information can be found here.
• Sizing estimates are provided for an approximate range of concurrent or active users. Decreasing the range of estimated concurrent users does not equate to a corresponding decrease in estimated resources. In other words, if an estimate provides a range of 300 to 500 concurrent users, it cannot be assumed that reducing by half the number of cores and RAM will meet the needs of half the amount of estimated users. In fact, we would recommend the same hardware configuration whether estimating for 150 to 250 users or 300 to 500 concurrent users.
• Similarly, increasing the range of estimated concurrent users does not equate to a corresponding increase in estimated resources. In other words, if an estimate provides a range of 300 to 500 concurrent users, it cannot be assumed that doubling the amount of cores and RAM will meet the needs of twice as much concurrent users.
• We cannot estimate the size of database repositories. The size of the database is governed by many factors: The organization’s customized workload, the organization’s customized datashape, and the work habits of the people in the organization. We suggest estimating a generous starting size for your database repositories, but more importantly, we suggest monitoring the growth of the repositories. Your database will grow based upon how it is used in your organization, and understanding your trends and growth patterns is the best way to anticipate database sizing needs. It is best to consider an enterprise storage solution which supports a high IOPS throughput since IO directly impacts performance. In addition mature storage solutions have the ability to size volumes as needed and optimize storage needs with real-time data deduplication.
• RAM consumption should never be more than half of the total available RAM; Maximum JVM heap size should not exceed half of the server’s allocated RAM.
• You may note that the data presented here may be different from data previously presented or even data that may be presented in the future. This sizing advice does not replace or supersede any other advice. Sizing estimates are derived from actual test data and based upon assumptions and for the most accurate sizing advice for your situation you need to work with IBM to understand your unique requirements.
• This document considers single applications connected to a single JTS server. Multiple application servers can be connected to a single JTS, and sizing advice for such configurations will be considered in a future publication. Considerations and strategies for when multiple application servers should be used will be discussed in a future publication.

Registered and concurrent user definitions

When we talk about users, we generally talk about registered or licensed users and a subset of concurrent or active users. Registered users refer to the number of users who are licensed and permitted to use a system. A company of 1200 employees may have 800 registered users. Concurrent users are the number of users logged in and actually using the system at any given time. A company of 1200 employees may have 800 registered users, but only 200 are logged in and actively working at any given time. We call these 200 users concurrent and active regardless of the intensity of how they may be working.

The number of concurrent users can often be estimated from the number of registered users. In an organization located on one continent or spanning just a few time zones, the number of concurrent users can be estimated to be between 20% and 25% of the total number of registered users. In an organization which spans the globe where registered users work in many different time zones, the number of concurrent users can be estimated to be between 10% and 15% of the total number of registered users.

For now we discuss server load in terms of users. This can be potentially misleading as different organizations’ users will work differently. We plan to add more precise transaction-rate and workload-rate data to our reports and sizing estimates, but for now, we specify server load in terms of users.

Rational Team Concert (RTC): Examples of different RTC usage patterns

Of all the Jazz products, Rational Team Concert (RTC) permits the widest variety of end-user and automated workflows. When estimating a RTC workload it is important to consider what percentage of the anticipated usage will come from work item users, SCM users and build agents or any combination of the three. Description of representative RTC workload we use in our sizing estimates can be found here.

Rational Team Concert (RTC) has three major workload types which can be used concurrently:

• Work item
• Software configuration management (SCM)
• Build agent

Our representative workloads combine these major workload types:

1. Work item only workload
2. Combined work item and SCM workload
3. Combined workitem, SCM and build agent workload

Rational Team Concert (RTC): A: Work item only workload

Rational Team Concert (RTC): B: Combined work item and SCM workload

Rational Team Concert (RTC): C: Combined work item, SCM and build agent workload

Rational Requirements Composer (RRC) / Rational Doors Next Generation (RDNG)

Description of representative RRC workload we use in our sizing estimates can be found here.

Rational Quality Manager (RQM)

Description of representative RQM workload we use in our sizing estimates can be found here.

Lifecycle Query Engine (LQE)

What is a sizing estimate for anyway? What are the next steps?

A sizing estimate attempts to forecast the number of users which a prescribed hardware configuration may comfortably support with a maximum response time at a defined interaction rate and a typical datashape. A sizing document is often created before a product is deployed in order to estimate the anticipated hardware needs for a deployment. Sometimes the sizing document’s most immediate goal is to specify hardware for allocation or procurement. Very often the sizing document specifies a maximum or largest anticipated target number of users simply so that the corresponding hardware allocation will not be exceeded and the anticipated hardware is requested only once.

Every CLM sizing document is an estimate: The CLM products are flexible and can be deployed to meet customers’ needs; customer hardware and deployment environments are unique for each customer; customer workload and habits will be unique and can be expected to vary across the products’ deployment lifetime.

A sizing estimate represents the best-guess assessment at a given point in time. As an environment is uses and grows, it is natural that assumptions made to support the sizing estimate may change. After deploying the environment we strongly recommend monitoring the environment and re-evaluating the assumptions and conclusions made during the planning phase as part of a deployment’s maintenance strategy. It is typical that an environment will grow and mature differently than the sizing estimate may anticipate. A standard maintenance and monitoring plan should include revisiting the sizing estimate and appropriately adjusting and tuning your deployment at key milestones, if not at a yearly minimum.

References

IBM System x3550 M4, http://www-03.ibm.com/systems/x/hardware/rack/x3550m4/

Standard deployment topologies overview , Steven Beard, Grant Covell, Tim Feeney, David Chadwick, Vaughn Rokosz: https://jazz.net/wiki/bin/view/Deployment/StandardTopologiesOverview

Performance impact of sharing a Jazz Team Server, Joseph Stellrecht, Vaughn Rokosz; July 24, 2013: https://jazz.net/wiki/bin/view/Deployment/PerformanceImpactOfSharingAJazzTeamServer

Collaborative Lifecycle Management 2012 Sizing Report (Standard Topology E1), Vaughn Rokosz, Skye Bischoff, David Chadwick, and Tim Feeney; June 13, 2012: https://jazz.net/library/article/814/

Collaborative Lifecycle Management 2011 Sizing Guide, David Chadwick, Tim Feeney, Philippe Mulet, Anuradha Ramamoorthy, and Skye Bischoff; June 24, 2011: https://jazz.net/library/article/641

Rational Team Concert (RTC) 2.0 sizing guide, Paul W. Weiss, Scott Rich, Jean-Michel Lemieux; June 24, 2009: https://jazz.net/library/content/articles/rtc/2.0/enterprise-configuration/sizing-guide/index.html

System Requirements for Collaborative Lifecycle Management 4.0.5 and 4.0.6, Deborah Weil-O'Day: https://jazz.net/wiki/bin/view/Deployment/CLMSystemRequirements405406

System Requirements for Systems and Software Engineering 4.0.5 and 4.0.6, Cindy McKeen: https://jazz.net/wiki/bin/view/Deployment/SSESystemRequirements405

Questions and comments:

• What other sizing guidance would you like to see here?
• Are there situations and topologies which are not covered?

Warning: Can't find topic Deployment.CLMSizingStrategyComments