Unable to execute a suite if a Test case has more than one Test script
Hi,
We are currently using RQM 3.0.1 and we are setting up a RQM execution process within the team for our releases and whilst doing that we have bumped into a potential issue because of which we are not sure if its an issue with RQM or the way we are approaching the execution.
We are setting up our execution environment in such a way that we are creating Test Suites under the Test Plan and we are adding Test Cases into the Test Suite that we create. Each of the Test cases have corresponding Test Scripts linked to them. We create a Test Suite Execution Record and when we execute the same, we get to run the Test Cases (inturn the Test Scripts) within the suite in a sequential way without any issues and we are able to see the end result showing the correct status.
But if we have a situation when there are more than one Test Script within a Test Case then, when we execute the Test Suite Execution Record, then RQM prompts us to run only one of the many Test Scripts which are there within a given Test Case ( by default the first one ) and the others gets skipped.
To give a clear indication
TestSuite
-TestCase01
--TestScript01
--TestScript02
--TestScript03
-TestCase02
--TestScript04
--TestScript05
--TestScript06
Now when i execute the above suite, only the TestScript01 under TestCase01 and TestScript04 under TestCase02 gets executed. There is no provision to run the TestScript02 or TestScript03 from TestCase01 once the TestScript01 completes execution, it directly jumps into TestScript04 of TestCase02.
***Workaround***
Currently we have a workaround using which we are able to execute the other test scripts from the suite ie
If we add multiple instances of TestCase01 and then go ahead with editing the record and selecting TestScript02 and TestScript03 in the 2nd and 3rd instance and then save and execute, we are able to proceed with the execution covering all the Test scripts. But apparently this has a issue that it gives us a wrong count of the total test cases planned and executed as we are adding TestCase multiple times. Eg in the above case instead of telling 2 Test Cases passed it shows that 6 Test Cases passed.
***
So wanted to check if this is actually an issue in RQM and if it has been addressed in any of the latest releases or if the approach what we are taking is wrong or otherwise.
Cheers,
Shiv
We are currently using RQM 3.0.1 and we are setting up a RQM execution process within the team for our releases and whilst doing that we have bumped into a potential issue because of which we are not sure if its an issue with RQM or the way we are approaching the execution.
We are setting up our execution environment in such a way that we are creating Test Suites under the Test Plan and we are adding Test Cases into the Test Suite that we create. Each of the Test cases have corresponding Test Scripts linked to them. We create a Test Suite Execution Record and when we execute the same, we get to run the Test Cases (inturn the Test Scripts) within the suite in a sequential way without any issues and we are able to see the end result showing the correct status.
But if we have a situation when there are more than one Test Script within a Test Case then, when we execute the Test Suite Execution Record, then RQM prompts us to run only one of the many Test Scripts which are there within a given Test Case ( by default the first one ) and the others gets skipped.
To give a clear indication
TestSuite
-TestCase01
--TestScript01
--TestScript02
--TestScript03
-TestCase02
--TestScript04
--TestScript05
--TestScript06
Now when i execute the above suite, only the TestScript01 under TestCase01 and TestScript04 under TestCase02 gets executed. There is no provision to run the TestScript02 or TestScript03 from TestCase01 once the TestScript01 completes execution, it directly jumps into TestScript04 of TestCase02.
***Workaround***
Currently we have a workaround using which we are able to execute the other test scripts from the suite ie
If we add multiple instances of TestCase01 and then go ahead with editing the record and selecting TestScript02 and TestScript03 in the 2nd and 3rd instance and then save and execute, we are able to proceed with the execution covering all the Test scripts. But apparently this has a issue that it gives us a wrong count of the total test cases planned and executed as we are adding TestCase multiple times. Eg in the above case instead of telling 2 Test Cases passed it shows that 6 Test Cases passed.
***
So wanted to check if this is actually an issue in RQM and if it has been addressed in any of the latest releases or if the approach what we are taking is wrong or otherwise.
Cheers,
Shiv
4 answers
Hi Shiva,
RQM does not entertain multiple Test script in a Testcase to be executed all, but allows you to have different implementation of the same testcase in different situation like Test Environment based, In case all scripts are not doing what the objective of the test case is, then you should split each script to it's own testcase. In case all scripts tests same testcase but address different platform then you should use more then one Step in suite for same testcase and specify different test environment for each step. Based on Test environment chosen, you can select appropriate script.
Let me know if I can help you further.
RQM does not entertain multiple Test script in a Testcase to be executed all, but allows you to have different implementation of the same testcase in different situation like Test Environment based, In case all scripts are not doing what the objective of the test case is, then you should split each script to it's own testcase. In case all scripts tests same testcase but address different platform then you should use more then one Step in suite for same testcase and specify different test environment for each step. Based on Test environment chosen, you can select appropriate script.
Let me know if I can help you further.
Hi Pramod,
Thanks for your reply it really makes things much clearer.
But on the downside of things we have started facing another problem with the above setup.
Currently we have a Test suite which has about 100 test cases and each test case having a test script each. When i start to execute this test suite ( using Test suite execution record ) and complete say about 10 test cases on a given day and when i come back to this suite and again try to run this suite ( again using the Test Suite execution record ) am not able to jump to the 11th test case but instead the RQM shows me to start with the 1st test case. Apparently the icon next to the test case does indicate that its pass but without "re-executing" it i cant skip the test case to move on till 11th test case.
Secondly, another problem that we see is that everytime we are clicking on run for the Test Suite Execution Record , there is a new entry created under "Execution > View Test Suite Results" when we do a group by Test Plan which apparently looks like an issue because we expect to see only one Test Suite Result for a given run / test suite execution record combination.
Please advice if these two queries have to be raised as a part of a separate thread.
Thanks,
Shiv
Thanks for your reply it really makes things much clearer.
But on the downside of things we have started facing another problem with the above setup.
Currently we have a Test suite which has about 100 test cases and each test case having a test script each. When i start to execute this test suite ( using Test suite execution record ) and complete say about 10 test cases on a given day and when i come back to this suite and again try to run this suite ( again using the Test Suite execution record ) am not able to jump to the 11th test case but instead the RQM shows me to start with the 1st test case. Apparently the icon next to the test case does indicate that its pass but without "re-executing" it i cant skip the test case to move on till 11th test case.
Secondly, another problem that we see is that everytime we are clicking on run for the Test Suite Execution Record , there is a new entry created under "Execution > View Test Suite Results" when we do a group by Test Plan which apparently looks like an issue because we expect to see only one Test Suite Result for a given run / test suite execution record combination.
Please advice if these two queries have to be raised as a part of a separate thread.
Thanks,
Shiv
Hi Shiva,
Why are you executing the Test Suite Execution Record again and again on each day. Once you start the execution, all steps should be completed to complete the suite execution. Even you leave office for the day, you can go back to the Inprogress Suite Execution console from
1. Execution Console -> Click on Suite Progress column
2. From Test Suite Execution Record view. If the current result shows in progress, just click on that to open in progress execution
3. From Test Suite Execution Record section in Test Suite. If the current result shows in progress, just click on that to open in progress execution
Each time you run Test suite, a new instance of execution is started and that is mutually exclusive from other execution started. RQM does not prohibit you from running same TSER or TCSER multiple time by different users.
You see multiple Suite reasult for the same reason, each one representing an instance of execution you started
Why are you executing the Test Suite Execution Record again and again on each day. Once you start the execution, all steps should be completed to complete the suite execution. Even you leave office for the day, you can go back to the Inprogress Suite Execution console from
1. Execution Console -> Click on Suite Progress column
2. From Test Suite Execution Record view. If the current result shows in progress, just click on that to open in progress execution
3. From Test Suite Execution Record section in Test Suite. If the current result shows in progress, just click on that to open in progress execution
Each time you run Test suite, a new instance of execution is started and that is mutually exclusive from other execution started. RQM does not prohibit you from running same TSER or TCSER multiple time by different users.
You see multiple Suite reasult for the same reason, each one representing an instance of execution you started
Hi Shiva,
Why are you executing the Test Suite Execution Record again and again on each day. Once you start the execution, all steps should be completed to complete the suite execution. Even you leave office for the day, you can go back to the Inprogress Suite Execution console from
1. Execution Console -> Click on Suite Progress column
2. From Test Suite Execution Record view. If the current result shows in progress, just click on that to open in progress execution
3. From Test Suite Execution Record section in Test Suite. If the current result shows in progress, just click on that to open in progress execution
Each time you run Test suite, a new instance of execution is started and that is mutually exclusive from other execution started. RQM does not prohibit you from running same TSER or TCSER multiple time by different users.
You see multiple Suite result for the same reason, each one representing an instance of execution you started
Why are you executing the Test Suite Execution Record again and again on each day. Once you start the execution, all steps should be completed to complete the suite execution. Even you leave office for the day, you can go back to the Inprogress Suite Execution console from
1. Execution Console -> Click on Suite Progress column
2. From Test Suite Execution Record view. If the current result shows in progress, just click on that to open in progress execution
3. From Test Suite Execution Record section in Test Suite. If the current result shows in progress, just click on that to open in progress execution
Each time you run Test suite, a new instance of execution is started and that is mutually exclusive from other execution started. RQM does not prohibit you from running same TSER or TCSER multiple time by different users.
You see multiple Suite result for the same reason, each one representing an instance of execution you started
Comments
Hao Wan
JAZZ DEVELOPER Oct 28 '12, 9:46 p.m.Hi Shivakumar, I just retag the question, so RQM team can take a look at the problem.