How is the latest test result determined? What parameters / conditions are really used?

Glenn Henderson (6●4●5) | asked Jun 25 '15, 2:57 p.m.

I'm pulling data through REST and made some assumptions about what the latest execution result should be, but they differ from what RQM is reporting on the GUI.

I have 3 test results for the same test case

ID start end status

69000 June 1st, June 5th passed

70000 May 1st, May 5th passed

20000 Jan 1st, July 1st failed

Which is the latest per RQM Rules?

69000 because it's the last started?

70000 because it's the largest ID (last entered test result)?

20000 because it ended the last?

My assumption had been that the last test to end (20000) would have been considered the "latest" test result but that seems to be incorrect for RQM. Is there any documentation that indicates what it is per RQM logic?

2 answers

Most liked answers ↑|Newest answers|Oldest answers

permanent link

Don Yang (7.6k●21●105●137) | answered Jun 26 '15, 12:46 a.m.
edited Jun 26 '15, 3:17 a.m.

You may want to see the comment 12 in Workitem:
https://jazz.net/jazz02/web/projects/Rational%20Quality%20Manager#action=com.ibm.team.workitem.viewWorkItem&id=101947

Basically each testcase can have different TCERs, if TCER has different combination of environment + iteration, then they are considered as different groups. Each group(TCER) will have latest result(execution end time stamp) override
its own previous group. In Test Case view, if you see the last result icon of each test case: the result with the highest severity across all these groups of results would take precedence. The order of severity is(high to low)
• Blocked
• Error
• Failed
• Deferred
• Partially Blocked
• Inconclusive
• Incomplete
• Permanently Failed
• Passed
• Not Run
• In Progress
• Paused

From above, you can see the icon shows last result is Failed, but after failed, there are two other groups execution results: inconclusive and Passed in my test, since Failed has higher severity in the 3 groups, the icon will show Failed even though it is not the latest execution result for the Test Case.

If my answer helps you, please consider accepting it. Thanks

Comments

Glenn Henderson commented Jun 26 '15, 10:12 a.m. | edited Jun 26 '15, 10:17 a.m.

Thanks for your response. However, I'm getting different results where passed is being given a higher order than Inconclusive.

permanent link

Glenn Henderson (6●4●5) | answered Jun 26 '15, 8:48 a.m.
edited Jun 26 '15, 8:56 a.m.

Appreciate the response, but I'm seeing that Passed is taking a higher priority than Inconclusive in my case below. The Environment and Iteration are all the same. Your response indicates that inclusive is higher in the selection order. So the latest TCER should be inclusive, but instead it's Passed.

Viewing the Test Results with some more details

I rarely use the GUI, only involved in the back end processing of results through REST. Still trying to figure out why 814403 is marked as current (latest), my assumption would have been 812080.

Comments

Glenn Henderson commented Jun 26 '15, 8:50 a.m. | edited Jun 26 '15, 8:50 a.m.

Sorry, but the images I embedded didn't save to the message and I don't have the ability to upload them.

Don Yang commented Jun 26 '15, 9:41 p.m.

if the environment, iteration are the same(same TCER), then the latest result will override the previous one, the severity order won't apply here. This is expected because the TCER, it is normal that user ran it failed and next time it is passed, we expect the latest passed result should be the one for that TCER. Only when there are different TCER for the same testcase, the latest result for each TCER will have severity order applied when we view the result for that test case.
Since there is no images in your comments, I am not sure what is 814403 and 812080 and what you mean marked as current(where is showing in the GUI etc). but the principle I explained above applies, we have confirmed with the RQM dev team on that.

It's all about the answers!

Related questions

How is the latest test result determined? What parameters / conditions are really used?

2 answers

Your answer