Rational Team Concert 4.x performance tuning guide for z/OS

Steven Zheng and Joseph Lu, IBM
Last updated: October 22, 2012

Performance tuning is the process of changing configurable parameter values to maximize the capacity and performance of your server workloads. This document lists the parameters that were modified to improve performance during our benchmark tests. These parameter settings might not be appropriate for your environment. As a general rule, modify one parameter at a time so you can determine the impact of that parameter change. The values should not be taken as recommendations for all environments. The best values for your workload and environment might vary.

General tuning tips

Confirm you have adequate hardware that is tuned appropriately.
Confirm your server has sufficient physical memory and disk space. Changes in tuning parameters almost always affect memory allocation, storage allocation, or both.
Tune your network interface card for sufficient transmit and receive buffers.
Always install the latest cumulative fix packs.
Tune your TCP/IP network to avoid packet fragmentation. Choose a packet size that is the largest value the route between your server and the TCP/IP host can handle without fragmentation.
Confirm your operating systems and application servers are at the latest supported levels. Generally, each new release brings improved performance.
Cache static data as close as possible to the end user.

z/OS

z/OS includes many subsystems that provide reliability, availability, serviceability, and other qualities of service. The WebSphere Application Server uses these subsystems to improve performance quality and efficiency.

As a Java application, UNIX System Services (USS) runs the WebSphere Application Server, and the USS Hierarchical File System (HFS) holds the server configuration files, applications, and so forth. Websphere Application Server for z/OS also uses z/OS services such as LE (storage and message control), CS (TCP/IP communications), workload management (WLM), resource recovery services (RRS), and global resource serialization (GRS). Tune these services for optimal performance of Rational Team Concert running on the WebSphere Application Server on z/OS.

The WebSphere Application Server for z/OS runs in two types of address spaces: A Control Region (CR), and one or more servant regions. The CR receives the browser request and queues it to the WLM. WLM load balances requests from its queue across servant regions based on configured application policies. The Rational Team Concert server runs in the WebSphere Application Server servant region.

The WebSphere Application Server for System z is a UNIX System Services application. It runs in the UNIX kernel under z/OS. You must configure your WebSphere Application Server to indicate to the USS how many processes (address spaces), threads (tasks), sockets, and users it is expected to handle. You can set UNIX kernel configuration parameters in the BPXPRMxx parmlib member.

Tune the following kernel parameters:

MAXTHREADTASKS: Defines the maximum number of active threads for a single process. Set between 1000 and 5000.
MAXTHREADS: Defines the concurrent total active threads (running, queued, and exited) that a single process can have. MAXTHREADS should always be equal to or greater than MAXTHREADTASKS. Set MAXTHREADS to twice the value of MAXTHREADTASKS.
MAXFILEPROC: Defines the highest possible number of combined open file descriptors (file directories, sockets, and pipe) for a single USS process. Set MAXFILEPROC sufficiently high, such as 60,000 or higher. Resources are not used for these files until they are allocated, so there is no disadvantage to setting it high. For V1R6 and earlier, the limit is 64K. For z/OS V1R7 and later, the limit for MAXFILEPROC is 128K. MAXFILEPROC was set at 65535 for these benchmarks.
MAXSOCKETS: Defines the maximum number of sockets. Set MAXSOCKETS to twice the value of MAXFILEPROC + 500. MAXSOCKETS is set in the NETWORK statement in the BPXPRMxx member. MAXSOCKETS is a USS limitation, TCP/IP has no limits on the number of open sockets. There are no storage concerns when specifying large values for MAXSOCKETS, so set it sufficiently high, such as 120,000.
IPCSHMMPAGES: Defines the maximum pages for shared memory segments. Set IPCSHMMPAGES to 256.
MAXPMMAPAREA: Specifies the maximum number of data space pages that can be allocated for memory mapping of z/OS UNIX files. Storage is not allocated until memory mappings are active. For MAXMMAPAREA, you can set a system-wide limit in BPXPRMxx and then set higher limits for individual processes. Use the RACF ADDUSER or ALTUSER command to specify the MMAPAREAMAX limit on a per-process basis.

For example:
```
ALTUSER userid OMVS(MMAPAREAMAX(nnnn))
```
The SHMEMMAX and MEMLIMIT parameters enable installations to manage the 64-bit space more effectively. Set MAXMMAPAREA to 100000000.

If you are running more than one server or additional applications, check the following kernel parameters to ensure they are adequate:

MAXPROCSYS: Defines the number of concurrent active USS processes allowed within the system. Make sure this value is high enough to support the maximum number of processes at peak usage.
MAXPROCUSER: Defines the maximum number of concurrent active processes for a single USS user. Ensure the value of MAXPROCUSER is sufficient for the user with the most processes. To negate the effect of this parameter, set the value to be equal to or greater than MAXPROCSYS.
MAXUIDS: Specifies the number of concurrent unique user IDs (UIDs) that can use kernel services. Every user needs a unique UID, so ensure this value is high enough to support the maximum number of concurrent users.

WebSphere Application Server for z/OS

Run Websphere Application Server in 64-bit mode, because in 31-bit mode the maximum heap most installations would be able to specify would be within the 768M – 900M range, tending toward the low end.

You can use the WebSphere Application Server administrative console to adjust the following settings and values, which can enhance the performance of the Rational Team Concert server:

Workload profile
Number of servant regions
Java heap sizes

Workload profile

On z/OS, setting the workload profile to LONGWAIT is important for application processing that involves sending or receiving information across a network. The LONGWAIT setting allows the servant region to have a pool of 40 threads to handle concurrent requests. The default pool size is based on the number of CPs.

To set the workload profile, from the WebSphere Application Server administrative console navigate to:
Servers -> Application Servers -> servername -> ORB Service -> Advanced Settings -> Workload Profile
Set value to LONGWAIT

Number of servant regions

On z/OS, the Rational Team Concert server runs in the Websphere Application Server for z/OS Servant Regions. By default, there is only one servant to handle all concurrent requests. The Rational Team Concert server does not support running in more than one servant region.

To set the number of servant regions, from the WebSphere Application Server administrative console navigate to:
Servers -> Application Servers -> servername -> Server Instance -> Server Instance -> Maximum Number of Instances
Set the value to 1.

Java heap sizes

Java heap parameters influence performance and the behavior of garbage collection. Larger heap sizes take longer to fill and longer to collect garbage. With large heap sizes, applications run for a longer time before garbage is collected, and it takes longer to perform garbage collection. Make sure you monitor garbage collection and adjust your heap sizes so that no more than 5 percent of the total execution time is spent on garbage collection.

To monitor garbage collection, use the JVM verbose garbage collection option, which you enable by adding –verbose:gc to the JVM arguments. When you use the –verbose:gc option, you can convert the date and time of each garbage collection cycle to standard output. From the output, you can calculate how much of the total execution time is spent on garbage collection by dividing the time between garbage collection cycles by the time it took to collect the garbage. The quotient indicates how much time is spent collecting garbage during a given garbage collection interval.

With z/OS, both the controller and the servant region use a JVM. Usually, you do not need to tune the JVM in the controller. The following information applies to the JVM in the servant: There are two JVM parameters that specify how much memory is allocated for the JVM heap. The settings of these two parameters affect the performance of the Rational TeamConcert server. Make sure you monitor its usage if there is a substantial change in load. Heap size in the Servant region is installation dependent. As a starting point, the two parameters are as follows:

Initial heap size: This parameter specifies how much memory is allocated for the heap when the JVM starts. When you properly tune this parameter, you reduce garbage collection overhead, which improves server response time and throughput. On z/OS, the default setting for this option is too low, which results in a high number of insignificant garbage collections. Note that this parameter is workload-specific. Set the parameter to at least 512 MB, although server performance is best when you set the initial heap size to 2 GB.
To modify the initial heap size parameter, navigate to:
Servers -> Application Servers -> Process Definitions -> JVM -> Initial Heap Size
Set the value to between 512 MB and 2 GB as appropriate for your installation.
Maximum heap size: This parameter specifies the maximum heap size the Java interpreter can use for dynamically allocated objects and arrays. Just like with the initial heap size, when you properly tune this parameter, you reduce garbage collection overhead, which improves server response time and throughput. On z/OS, the default setting for this option is too low, which results in a high number of insignificant garbage collections. Note that this setting is workload-specific. Set the parameter to at least 2 GB, and for larger installations, to at least 4 GB.
To modify the maximum heap size parameter, navigate to:
Servers -> Application Servers -> servername -> Process Definitions -> JVM -> Max Heap Size
Set the value to at least 2 GB or an appropriate size for your installation.

DB2 on z/OS

The following parameters are optional to make the Rational Team Concert server on z/OS work correctly and perform efficiently.

Set RRULOCK=YES: The RRULOCK subsystem parameter specifies whether DB2 should use U (UPDATE) locks or S (SHARE) locks. Set this parameter to YES so that the server can acquire U locks during repeatable read (RR) or read stability (RS) operations.
For details about the RRULOCK parameter, see U LOCK FOR RR/RS field (RRULOCK subsystem parameter).
Set OPTHINTS=YES: The OPTHINTS subsystem parameter specifies whether you want to pass information to DB2 that might impact certain query access paths. Set this parameter to YES so that the server can use the best access path.
For details about the OPTHINTS parameter, see OPTIMIZATION HINTS field (OPTHINTS subsystem parameter).

DB2 on Linux for System z

State-selection fetching optimization can improve large database performance. To take advantage of this feature, you must modify some DB2_OPTPROFILE parameter settings. Perform the following commands to modify the DB2_OPTPROFILE parameter:

  db2set DB2_OPTPROFILE=YES  db2stop  db2start

Note: After the command runs, stop and restart the database.

Make sure all the commands succeed. To verify, run the command:

db2set DB2_OPTPROFILE

Rational Build Agent

Use the following Java options to maximize build performance:

-Xquickstart: -Xquickstart reduces Java startup time.
-Xshareclasses: -Xshareclasses uses the shared class cache to reduce startup time.

To indicate the Xquickstart and Xshareclasses options, complete the following steps:

Modify the bfagent startup shell script to indicate these options:

FA_JAVA_OPTIONS="-Xquickstart -Xscmx32M -Xshareclasses:name=fileagent"

Modify the bfagent startup shell script to indicate these options:

FA_JAVA_OPTIONS="-Xquickstart -Xscmx32M -Xshareclasses:name=fileagent"

Modify the Ant with Enterprise Extensions build definition by adding the following commands to the Java VM arguments on the Ant with Enterprise Extensions tab:
```
-Xscmx32M -Xshareclasses:name=antz
```
Xquickstart is the default.

Rational Team Concert has the option to publish log files only when an error occurrs, which reduces the time it takes to publish log files. To enable this function, select “Publish only when an error occurred” on the Job Output Publishing tab in the Ant wth Enterprise Extensions build definition.

For additional information on tuning Java on z/OS, see Hints and Tips for Java on z/OS.

Mass import tool: zimport

Use the following Java options to maximize zimport performance:

-Xquickstart: -Xquickstart reduces Java startup time. Add -Xquickstart to the zimport startup shell script ($INSTALL_PATH/usr/lpp/jazz/v4.0.1/scmtools/eclipse).
-Xshareclasses: -Xshareclasses uses the shared class cache to reduce startup time. Add -Xscmx32M -Xshareclasses:name=zimport to the zimport shell script ($INSTALL_PATH/usr/lpp/jazz/v4.0.1/scmtools/eclipse).

Note: Depending on which version you are using, your installation path might vary from the example. The first time you run the zimport command, you must set up the shared class cache. After you initialize the shared class cache, startup takes less time. The cache is cleared after the initial program load (IPL).

For additional information on tuning Java on z/OS, see Hints and Tips for Java on z/OS.

Using zAAP, zIIP, and IFL

Specialty engines are processors that let you expand the ways you use a mainframe computer for new workloads. IBM System z specialty engines can run independently or together.

To avoid having Workload Manager (WLM) determine if a zAAP or zIIP needs help and thus run specialty engine-eligible workload on a general purpose central processor (CP), set IFAHONORPRIORITY=NO in IEAOPTxx.

System z Application Assist Processor (zAAP)

The IBM System z Application Assist Processors (zAAPs) are available on all IBM zEnterprise, IBM System z10, IBM System z9, IBM eServer zSeries 990 (z990), and IBM eServer zSeries 890 (z890) systems. The zAAP specialty engine offers an affordable environment for new web-based applications and SOA-based technologies like Java and XML.

For more information, see System z Application Assist Processor (zAAP).

System z Integrated Information Processor (zIIP)

The IBM System z Integrated Information Processor (zIIP) is available on all IBM zEnterprise (zEnterprise), System z10, and System z9 servers. It is designed to increase general computing capacity and to lower overall total cost of computing for select data and transaction processing workloads for business intelligence (BI), ERP and CRM, and select network encryption workloads on the mainframe.

For more information, see System z Integrated Information Processor (zIIP).

Integrated Facility for Linux (IFL)

The Integrated Facility for Linux (IFL) is a processor dedicated to Linux workloads on IBM System z servers. The affordable IFL processor lets you add processing capacity exclusively for Linux workloads.

For more information, see Integrated Facility for Linux (IFL) .

For more information

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Feedback

Was this information helpful?

Yes

No 1 person rated this as helpful.