Issue in Test Result of Test Case with Test Scripts having test data with multiple rows (iterations).
We have the below 3 issues in the Test Case Result of Test Cases with Test Scripts having Test Data with >100 rows.
a. There are 10 Steps in the Test Script and 200 Test Data rows(iteration of test for the test script).
In the Test Case Result, we see that it shows as 1 to x of 2000 steps, i.e., 10 steps x 200 test iterations).
But, the number of steps in the Test Script is only 10. What is the reason ? Shouldn't it show 1 to x of 10 steps
b. In the above cases, there are 2000 step results but not all are shown in the GUI.
c. Some of the steps have failed and some are deferred, but the overall Actual Result is Passed. What is this result based on in case of multiple data rows and steps ?
|
One answer
|
The answers to the queries are inline:
a. In the Test Case Result, we see that it shows as 1 to x of 2000 steps, i.e., 10 steps x 200 test iterations).
But, the number of steps in the Test Script is only 10. What is the reason ? Shouldn't it show 1 to x of 10 steps
>> We are using the Data Pool with 200 rows. So taking into account there are 10 steps in the test script, each steps is iterated 200 times. Hence there are 200 unique results for each step totalling to 2000 unique step results. Hence, this is expected.
b. All the test step results are not shown in GUI when view Test Case Result. >> In my environment, I had to scroll a while for the last iterations' step results to be seen. This is similar to the way the requirements load when a requirement collection has many requirements. A portion of the Test Step results(for example 100) are loaded first and then as we scroll more steps (like the next 100) are loaded. Considering the number of steps and in-effect the amount of data to be loaded, if the test case result was to load all the step results when it was opened, it would take a considerable amount of time leading to more problems.
c. Some of the steps have failed and some are deferred, but the overall Actual Result is Passed. What is this result based on in case of multiple data rows and steps ? >> In this case, the overall Actual Result of the test case should be failed. You may have to review the history to check if the result was manually updated by a user. In that case, to restrict the users from editing the Actual Result manually, remove the Edit Actual Result permission for the user role/s. https://www.ibm.com/support/knowledgecenter/SSYMRC_6.0.6.1/com.ibm.jazz.platform.doc/topics/c_qm_permissions_reference.html
|
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.