cases against using multiple test scripts per test case
Hi,
We are looking for additional concerns for using test cases with multiple test scripts vs. the documented best practice of a 1:1 test case/test script ratio (see https://jazz.net/library/article/1238 and http://sleroyblog.wordpress.com/2013/05/22/a-rqm-usage-antipattern-multiplying-nonequiv-test-scripts-associated-to-a-test-case/) One of our concerns is possible consequences in RRDI traceability reports, specifically requirements traceability. With current data model you cannot report on test script steps. So for requirement traceability reports you will have requirements associated to the test cases. I do not think there is an OOTB report that will trace a requirement to a test script, e.g to see if all requirement are covered or what scripts must be run to test a requirement. So I cannot personally determine a problem that can be introduced by using multiple test scripts per test cases but we are hoping for additional insight from the community. Thanks, Brett |
2 answers
Hi Brett,
Along with Requirement Traceability report, what will happen to the TCER reports for multiple Test scripts When you execute a test case with 2 test script and pass one and fail other, the most recent result will be considered as a overall execution result. Infact there is an open Enhancement request for this. The more you read about this topic in Forums, it feels IBM wants users to use 1-1 mapping. The main question is why you want to have multiple Test Scripts in a TC. Check this Comments There are many reasons to have multiple test scripts for a single test case. When a test case is generic and you need to have several variations that can't easily be covered by test environments and it doesn't make sense to have multiple test cases with exactly the same test design. For example, let's say you are doing bad path testing and you have the case where you pull a fibre channel cable from a storage system. You pull the cable and wait 30 seconds to plug it back in and the system is supposed to recover. Next you do the same thing and wait say 2 minutes. You want the procedure in the test script to call this out and generate 2 separate results. It doesn't make sense to have to different test cases for this variation but 2 test scripts do make sense.
Another quite obvious case is you have a manual test script and you eventually are able to automate it - you would have a manual test script and a automated script attached to the same test case. You might want to use the manual at times for validation and automated for normal execution.
|
Hi Brett,
answering the 2nd part of your question: the traceability from requirement to test script(s) is available OOTB in RQM since 4.0. You could check this article for reference. Wrt. RRDI ability to providing an equivalent granularity... I would 1st recheck for OOTB reports... and if it's confirmed as non available, you may want to check QM & RM data models and dictionaries. Regards, Stéphane Comments This only works for test script steps and not individual test scripts which is sorely lacking in the product. |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.