Rational Quality Manager best practices and lessons learned
This page may describe a usage or software configuration that is may not supported by IBM. This article was written by Tata Consultancy Services, and provides insight on our experience with best practices, and lessons learned while using Rational Quality Manager 4.0 (RQM).
Using test artifact categories
- When using test artifacts categories, we can organize our test artifacts in a systematic way, and can be used for filtering of test artifacts.
- These categories also helps in generating reports and dashboards.
- They help define artifacts in a hierarchical manner.
Simple Illustration:
Suppose your test case belongs to Test Plan “Sample”, in module “Add”. In this case we can easily define Test Case categories called module “Add”.
Scenario: You want reports of “Execution Status by Machine using Weight (Live)” in Test Plan and Test Case category called “Module” with the value “Add”.
Output: We can select different category values as parameters while generating reports. See the screen shot below, for category called ‘module’ having value called ‘Add’.
Define Categories
Choose parameters and run report
Measure test execution progress
Using weight to measure your test execution progress effectively:
Measurement of test execution progress is a very important aspect of the testing process. Here is an example given on how to use ‘weight’ to measure test execution progress effectively.
Illustration of scenarios:
Using default Weight (default is 100):
Suppose you have 2 test cases; One named “Test Case1″, and the other”Test Case2”, which having following details:
- TestCase1 has only 2 steps which consumes 5 minutes for its execution.
- TestCase2 has 10 steps which consumes more than 15 minutes for its execution.
(weight completed/total weight)*100 i.e., (100/200)*100=50%
Using user defined Weight:
On defining weight based on how much time they take, or the complexity of the execution of test cases:
TestCase1 has a weight 20, and TestCase2 has a weight 80. If you were to measure the progress now, it would be only approximately 20% completed because you have to execute TestCase2, which will take more time for its execution. Hence, your testing has completed only 20% rather than the 50% it would using the default Weight (i.e., (20/100)*100=20%).
Define weight
Execution progress using weight
One Test Script per Test Case
RQM provides the flexibility of attaching many test scripts to test cases for execution. However, at run time we can select any one of test script to run. Its strongly recommended to use one test script per test case. We can use more than one test script per test case only in the following scenario:
Scenario: Use more than one test script if you are running your test case in different platforms like Linux, or Windows. For example, your automation scripts may vary depending upon operating system and underlying hardware and software. In this scenario you can choose test scripts at execution time, which you plan to run in respective platforms.
Entry and Exit Criteria of Test Plan
The test plan basically talks about planning of your test effort and execution. The test plan can be configured to meet the needs of your team.
Test plans also provide two sections called exit and entry criteria. These two criteria help to deliver a quality product and plan your deliverables accordingly. Before you start your test plan execution, you can evaluate your entry criteria, whether its successful or unsuccessful, and decide upon whether you can go for further testing or not. Similarly you can also evaluate after your test execution is completed using exit criteria. A sample screen shot is provided showing entry criteria:
Leveraging Snapshots and Duplicate to achieve Versioning
Every phase of the SDLC process needs to maintain a baseline of all artifacts. In other words we call it configuration management. Currently RQM does not provide configuration management, however we can use some of the existing features of RQM to maintain baselines of various test artifacts. We can use 2 options:
- Using Snapshots
- Using Duplicate option
Option 1:Snapshots
Snapshots serves as reference details of previous versions of artifacts, and cannot replace with current version. The user can always go and see what was there in a previous version of the artifact, and he can always revert back to previous version by editing current artifact manually.
Snapshots:
Snapshots are non-editable, and cannot replace the current version with baselines. Instead, it serves as a reference of previous version.
Option2: Using Duplicate
In the previous section, we have seen how to take a baseline of artifacts using Snapshots. In this section, we will see how to use the Duplicate option provided within RQM for maintaining baselines of test artifacts.
We can use the Duplicate option provided by RQM, for creating new copies of the artifact. You can treat previous ones as a baseline, and the new copied one as current version. We can then start working on this new artifact. We can create references to the copied artifact by adding its RQM generated ID into some custom attribute fields, say Parent ID, which show where this new artifact was generated from.
In both the cases, using Snapshots or Duplicate, if you want to take a baseline of associated artifacts like test scripts, test cases, we need to do separate copy actions on those linked artifacts and link them to the newly created artifact according to your business needs.
Here the only advantage of using Duplicate over Snapshots is that Duplicate creates a new copy, which we can edit it. Snapshot is a read-only artifact, and cannot be modified. The screen shot below shows the use of Duplicate in RQM.
Script re-use using keywords
Using Keywords for test script authoring
It is often that in our manual test script steps, we may use similar test steps across different test scripts, and may have to write the same steps again and again. The solution provided by RQM is called “Keywords”.
Keywords are nothing but reference points to execution steps, and can be re-used. Using these Keywords in several scripts helps to avoid re-writing the same script/steps in script. When you create test script step, you need to just drag and drop your step to keywords section of the test script, as shown in the following screen shot. If required, a user can edit these keywords, so that edited keyword will be reflected in all other test steps. This saves you time and effort for both creating and editing test steps across multiple test scripts.
Illustration example:
The following “Login” Script step1 is used across different scripts. The following sequence of steps shows how to create a keyword using the drag and drop method, and how to use them in other scripts. To use keyword in other scripts, drag from the keyword view to manual steps.
Using Test Data and Execution Variables
TestData is simply a set of records written in spread sheet, and can be used data in our manual test scripts. For example, the same script is run for a different set of test data, sometimes called data driven testing.
Execution Variables are nothing but variables, whose values keep changing from one script to another. These variables can be passed across different scripts. For example, in a login script, the login name can be stored in a variable and can be used in another script to find out login details.
Import and Export of test artifacts
The following are a few guidelines for RQM exporting using spreadsheets:
In many projects, you may come across a scenario where you’re managing test artifacts in spreadsheets, and now you want migrate it to a test management tool like RQM. In this scenario, RQM provides a utility called RQMExcelWordImporter, a Microsoft Word/Excel plugin. For more details on how to download, and install RQMExcelWordImporter, refer to the following jazz.net article.
The following points are only guidelines, which worked well for us. They need not be followed in the same way, and also would depend on the config file that you write. You can find samples in the installation directory of the ImportAndExport utility under “Samples” folder.
- Do not add/delete heading row(s). Do not repeat heading rows in Microsoft Excel sheet.
- A test case can have more than one script linked to it. Linking of test cases to test scripts is handled by RQM automatically during the export from Microsoft Excel to RQM.
- All the test scripts to be linked to a test case should be in the same sheet.
- You don’t need to repeat test case information for every script in the sheet.
- Leave a blank row to separate one test script from other.
- Repeated upload of same Excel sheets to the same project in RQM may overwrite the existing test artifacts by the same name.
- Try to keep the test case name and test script name unique across all the test cases and test scripts.
Deployment
It is very important to consider the strategy and method for the deployment of any tool in our environment. We can treat this as a kind of prerequisite before the deployment of any tool.
Its suggested that we use a distributed deployment called “Enterprise Topology”, putting each one of these products (RQM, Rational Requirements Composer [RRC] and Rational Team Concert [RTC]) on different servers, and different physical machine. This will allow us to decrease the load on individual machine, thereby increasing performance. We can also take clustering into consideration when we deploy these products for their high availability. The selection of a deployment solution depends on the needs of the organization. For more information on clustering, deployment and the selection of a topology, refer to the following links:
- https://jazz.net/library/article/823
- http://publib.boulder.ibm.com/infocenter/rqmhelp/v1r0m0/index.jsp?topic=/com.ibm.rational.test.qm.install.doc/topics/t_instl_config_rqm_oracle.html
A sample screen shot for distributed deployment (Enterprise Topology) is shown below.
Lessons Learned
This section describes about findings and resolutions that we found during installation of RQM 4.0 setup in Linux environment, oracle and WAS 8.0.
Findings and Resolutions
Finding 1: Cannot invoke Installation Manager from windows using Putty
Resolution: One of the reason could be, might not have set your browser and display Values. Make sure that your browser and display variables are set properly.
Finding 2: Cannot install RQM using web-installer
Resolution: One of the reason could be there may be some cases that your system may be firewall protected hence will not be able to access the internet and web-installer needs internet to be connected. The resolution to this issue is, you can download installation manager repositories and use offline installation method.
Finding 3:JTS Applications are opening very slow with WAS 8.0 and Oracle 11g
Resolution: The reason could be that you may not have properly defined all of your environment variables correctly using WAS 8.0. Please see table below. The value of environment variable ORACLE_JDBC_DRIVER_FILE depends on version of Oracle database you are using, here version is 11g.
Environment Variable | Value |
|---|---|
| JAZZ_HOME | file:///JTSInstalltionDir/IBM/JazzTeamServer/server/conf |
| java.awt.headless | True |
| org.eclipse.emf.ecore.plugin.EcorePlugin.doNotLoadResourcesPlugin | True |
| ORACLE_JDBC_DRIVER_FILE | /JTSInstallationDir/IBM/JazzTeamServer/server/oracle/ojdbc6.jar |
| log4j.configuration | file:///JTSInstallationDir/IBM/JazzTeamServer/server/conf/startup_log4j.properties |
For more details how to set up JTS, refer the following link:
http://publib.boulder.ibm.com/infocenter/rqmhelp/v2r0/index.jsp?topic=/com.ibm.rational.test.qm.install.doc/topics/t_run_server_setup_wiz.htmlAnother reason why this may occur would be that the heap size in WAS could be less. Please make sure that you define minimum of 2048 MB for optimal performance as shown below.
Finding 4: Getting following error while setting up oracle database “Cannot load JDBC driver”
Resolution: Make sure that you copy the ojdbc6.jar in the following folder <JazzInstallDir>/server/oracle. For more details visit following link:
Finding 5: Not able to register the /qm or /ccm or /rm application
Resolution: One of the reason can be, you may be using same database table space for different applications. Please use the different database space for all the applications like ccm, qm etc. So that one application cannot use same database space as other uses it. Update the required parameters in teamserver.properties. If it does not work, try stopping the WAS server and run the ./repotools-qm.sh –createTables and start the WAS server.
Finding 6: Cannot login into RQM using user ID’s created in RQM 4.0 with tomcat server
Resolution: One of the reason could be sometimes your tomcat-user.xml may get corrupt because of some environmental issues. Please make sure that your tomcat-user.xml is not empty, if it is empty you may try to get the copy from another similar server and paste it in the following location <JTSinstallation dir>/server/tomcat/conf and restart the server.
Finding 7: Not able to run the JTS setup and getting following error: “Error 500: java.lang.NoClassDefFoundError: sun.awt.X11GraphicsEnvironment (initialization failure)”
Resolution: Please make sure that you define following custom properties java.awt.headless= true org.eclipse.emf.ecore.plugin.EcorePlugin.doNotLoadResourcesPlugin=true in WAS 7.0 “True” whereas not in WAS 8.0, here it should be “true” only.
For more information
- Jazz CLM 2012 Clustering
- Rational Quality Manager
- Integrating RQM using Command-line Adapter
- IBM Rational Help
About the author
Chandrahasa Reddy is an IBM Rational Product Specialist for the Tata Consultancy Services Ltd.(TCSL) within the Technology Excellence Group (TEG) group. He has over 3 years of experience in various Rational products, mainly on testing tools like RFT, RPT, AppScan and various other Jazz tools like RQM, RRC and RTC, as well as research in industry best practices. He can be contacted at chandrahasa.reddyd@tcs.com.
Copyright © 2012 IBM Corporation