It's all about the answers!

Ask a question

How to best clean results from automated tests


Michael Fox (61643) | asked May 08 '13, 10:57 a.m.
JAZZ DEVELOPER
retagged May 08 '13, 2:02 p.m. by Lisa Caten (342913)

We're starting the process of implementing RQM. We have many hundreds of test scripts that have been written over a long period of time, and one of our concerns is that they don't always return accurate pass/fail results.  We get many false failures.  We've dealt with this by manually reviewing test log files and deciding what failures are real and which we can ignore. We want to do better with RQM, but cleaning up all our tests will take time.

If we execute these test cases automatically and the results are loaded back into RQM, then any reports will not be accurate because of the original bad data. Therefore, is there a way to efficiently review the test results in RQM and modify the results accordingly?  Any suggestions on how to handle this situation are very welcome.

thanks

One answer



permanent link
Lisa Caten (342913) | answered May 08 '13, 2:02 p.m.
FORUM MODERATOR / JAZZ DEVELOPER
If you know which test case results you want to change (from false failure to passed), you can browse Test Case Results, select the results that you want to change, click the Change icon in the Status column heading, select Passed, and then click OK. See Updating multiple artifacts.


Comments
Michael Fox commented May 09 '13, 9:42 a.m.
JAZZ DEVELOPER

The ability to update multiple results from one screen will be much more efficient then having to go to each result individually.  Since we'll likely have several people doing this daily after each nightly regression run, efficiency in doing this for hundreds of test results is essential.

thanks


Bob Gallagher commented May 09 '13, 11:47 a.m.

Michael does this existing bulk change feature satisfy your request? Or are you looking more for presenting more information while viewing the table of results in order to make the decision on whether to change the result to passed?  For example would having a column in the table where each cell would contain a link to the generated log file help? And would it be better still that when you hover over a link you get a popup to view the log file content?


Michael Fox commented May 14 '13, 8:47 a.m.
JAZZ DEVELOPER

I don't have a specific solution in mind. We're currently planning our RQM implementation so I can't really comment yet on how well this helps.  We know we're going to face this issue and I was looking for a discussion on how to best deal with the problem. I'm assuming that we aren't the only QA team with this behavior in legacy tests.  Having links to the log file is a great idea.  Generally, anything that would increase our efficiency reviewing the logs would be greatly appreciated.

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.