How to get the number of test steps inside a RQM project
I couldn't find a way to get the total number of test steps in any RQM project. I can get the number of test cases, number of test scripts etc, but not the number of total steps which is important to understand the duration and complexity at the project level.
We also have the goal of proving the RQM advantage (vs QC) to show using RQM we can reduce the redundant steps due to the re-use opportunity in RQM via test scripts which can be associated with multiple test cases.
After importing 10000s of test steps into RQM from QC, we need a way to record the number of steps before removing redundant steps and after to show the % improvement we have made. For this we need a mechanism to get the number of test steps (at the project level, filterable for the state of the test script artifact).
After importing 10000s of test steps into RQM from QC, we need a way to record the number of steps before removing redundant steps and after to show the % improvement we have made. For this we need a mechanism to get the number of test steps (at the project level, filterable for the state of the test script artifact).
Someone in the forum suggested REST APIs for similar needs. Not sure which parameter to send ti the calls. Any help is appreciated.
7 answers
Hello Suraj,
1. I feel that calculating the complexity of RQM project with number of test steps in a script , will not be a standard approach ( at least with the customers whom i had seen). Since in some cases this can mislead you ( Ex: If a tester write a manual TS in 400 steps while the same functionality can be tested by another tester who can write 20 steps, so certainly human factor will come into place) .
I would recommend to consider the following options to understand the complexity of the RQM project using Weight / TER. In every TC / TS , you have the notion of weight , this can be handy. Based on the common discussion with you team, You can come up with a stand of consider weight as below
2. Reducing Redundant steps / Reuse :
3. To show the improvement before and after importing 10000 TS:
Utilization of Assets
These reports will help you to measure the productivity of your team in RQM...
Hope this information helps you...
1. I feel that calculating the complexity of RQM project with number of test steps in a script , will not be a standard approach ( at least with the customers whom i had seen). Since in some cases this can mislead you ( Ex: If a tester write a manual TS in 400 steps while the same functionality can be tested by another tester who can write 20 steps, so certainly human factor will come into place) .
I would recommend to consider the following options to understand the complexity of the RQM project using Weight / TER. In every TC / TS , you have the notion of weight , this can be handy. Based on the common discussion with you team, You can come up with a stand of consider weight as below
- Time taken for execution of the test script
- Complexity of the test script Ex: Happy scenario - 100 , while negative scenario -20
- Number of functionality tested in this script
- OR any other innovative idea
2. Reducing Redundant steps / Reuse :
- Pls make use of the keyword functionality that is available in RQM, will help you to define certain reusable steps as keywords and then you can reuse them
- In RRDI , you also have an out of the box report, that will give you the calculation on how many test scripts / cases had been reused.
-
3. To show the improvement before and after importing 10000 TS:
- I would recommend use the key words, use the above reports to measure your productivity.
- If you also had imported requirements from QC to RRC, then you can also measure the productivity based on the requirements linked to testcases.
-
There is also an out of the box report in RRDI based on RQM data
Utilization of Assets
These reports will help you to measure the productivity of your team in RQM...
Hope this information helps you...
Hello Mani,
Thanks for providing the insights of the RRDI. I agree that # of test steps is not an accurate measurement of complexity in all cases. But in cases where as you mentioned 20 steps were written in legacy scripts where 10 steps might be enough to test the functionality, unless we modify the steps the tester need to execute those 20 steps and record the results, objective evidences etc. This will count towards the total time taken to finish the testing. Of course after the initial execution of those legacy scripts, the scripts improvement will be done to make it as efficient as possible, but we still need to execute them in first time as these are legacy scripts. We are planning to use the keywords, definitely for new tests and eventually for the legacy scripts.
We don't have RRDI implemented due to various reasons. I was looking for some solution based on REST API or OSLC methods to get the # of test steps in an RQM project in the meantime.
Any help is appreciated.
Suraj
If you are going to use the REST API you would have to obtain the Steps via the Test Scripts
To obtain the XML of a Test Script you can use the following guide
GET
https://<PUBLIC URI>/qm/service/com.ibm.rqm.integration.service.IIntegrationService/resources/<QM Project Area>/testscript/urn:com.ibm.rqm:testscript:<ID>
…a working example would look like this
GET
https://clm401:9443/qm/service/com.ibm.rqm.integration.service.IIntegrationService/resources/LPA+project+1+%28Quality+Management%29/testscript/urn:com.ibm.rqm:testscript:30
This will return XML of the test script in which the steps are present (Be sure to include the request header ‘Accept’ with the value txt/xml or application/xml)
Given you are trying to obtain the steps on a project wide basis you would need to:
1) Create a routine (based on the request above) that would scroll through all of the test script ID’s in your project; obtaining the xml for each
2) Create an XML parser that would analyze each response content for the step tag(s) and increment a counter that would keep track of every step encountered
Keep in mind, there may be alternative ways to accomplish this; this one way that came to mind
Thanks Mani and Ari for the suggestions. Here is how I addressed this problem. This may not be the very efficient way, but worked for me without using RRDI. Thought of sharing this in case someone has the similar needs. I hope RQM team will add a column to show the # of test steps in the Browse->Test Scripts UI view.
1. While inside the RQM project, Navigate to Browse->Test Scripts to view the test scripts listing.
2. Export the list to the CSV file which includes the URL to the corresponding XML file.
3. Use the RQMUrlUtility to download the corresponding xml file into a folder on your hard disk. You can write batch file or any other scripts to read the CSV file and send the parameter to the RQMUrlUtility in the following format.
java -jar RQMUrlUtility.jar -command GET -user <USER NAME> -password <PASSWORD> -url <URL TO THE XML AS READ FROM THE CSV FILE> -filepath <PATH WHERE DOWNLOADED XML FILES SHOULD BE STORED>
4. Once all the xml files are downloaded to the hard disk, use the notepad+ tool to find the # of instances of word "stepIndex". I used "Find in Files" options of the notepad+.
Note: Instead of notepad+, one can use the grep function or similar one to get the # of occurrences of the word stepIndex
5. Each step inside the RQM test script has the tag stepIndex. So, # of occurrences of stepIndex inside the files in the downloaded XML folder = # of test steps in the project.
