Assessing test automation coverage for a feature suite
My team is investing heavily in functional test automation for our product. Given how we've modeled our feature set, we have approximately 100 features in the product. Some of these features are covered by functional test automation, others are tested manually. We are working hard to get to 100% automated functional test coverage.
Each feature test case is modeled as a "test case" artifact in RQM. The automated function tests are captured as RSpec test scripts in RTC SCM.
Given this starting configuration, what would you recommend that we do, so that we can easily use the combination of RQM and RTC to understand our test automation coverage - both percent covered and concrete information about whether a particular test case has coverage or not?
One answer
Hello Bill,
I would use the following approach.
1. Define for the TC a template which holds an enum for "automated=true|false" to control and filter which of your testcases are automated. You can use queries to find out which TC's are automated and which are not.
2. If you create a new TC the default value of the attribute would be false.
In the TC Design I would make a draft description how to automate the TC
3. Once you decide to automate the TC you create a RTC WI as quality task directly in the TC, the task is linked to the TC and the implementer can see the Test Case Design for input
4. Somebody will get the task creates the TestScript, and will also have a link to the ChangeSet for the TestScript when using RTC SCM and sets the attribute automated to true of the TC
TC->WI RTC Task->TS RTC SCM ChangeSets (Traceability)
5. Depending on your type of Testscript you need an adapter to import/export the Testscript to RQM for execution and in a second step in order to link it to the Test Script Section of your testcase
TC-TC TS Section ->Link to imported Testscript
Coverage can be measured in multiple ways:
TC with no TScript
TC which are automated
TC which are automated with results and RTC Defects
and many other dimensions like environment, milestones etc.
You could also trigger your tests from an RTC Build in RQM.
So there are many ways which should be discussed based on your project requirements, my scenario is only one path you could implement with RQM and RTC.
Best Regards,
René
I would use the following approach.
1. Define for the TC a template which holds an enum for "automated=true|false" to control and filter which of your testcases are automated. You can use queries to find out which TC's are automated and which are not.
2. If you create a new TC the default value of the attribute would be false.
In the TC Design I would make a draft description how to automate the TC
3. Once you decide to automate the TC you create a RTC WI as quality task directly in the TC, the task is linked to the TC and the implementer can see the Test Case Design for input
4. Somebody will get the task creates the TestScript, and will also have a link to the ChangeSet for the TestScript when using RTC SCM and sets the attribute automated to true of the TC
TC->WI RTC Task->TS RTC SCM ChangeSets (Traceability)
5. Depending on your type of Testscript you need an adapter to import/export the Testscript to RQM for execution and in a second step in order to link it to the Test Script Section of your testcase
TC-TC TS Section ->Link to imported Testscript
Coverage can be measured in multiple ways:
TC with no TScript
TC which are automated
TC which are automated with results and RTC Defects
and many other dimensions like environment, milestones etc.
You could also trigger your tests from an RTC Build in RQM.
So there are many ways which should be discussed based on your project requirements, my scenario is only one path you could implement with RQM and RTC.
Best Regards,
René