It's all about the answers!

Ask a question

How to best clean results from automated tests

Michael Fox (61643) | asked May 08 '13, 10:57 a.m.
retagged May 08 '13, 2:02 p.m. by Lisa Caten (342913)

We're starting the process of implementing RQM. We have many hundreds of test scripts that have been written over a long period of time, and one of our concerns is that they don't always return accurate pass/fail results.  We get many false failures.  We've dealt with this by manually reviewing test log files and deciding what failures are real and which we can ignore. We want to do better with RQM, but cleaning up all our tests will take time.

If we execute these test cases automatically and the results are loaded back into RQM, then any reports will not be accurate because of the original bad data. Therefore, is there a way to efficiently review the test results in RQM and modify the results accordingly?  Any suggestions on how to handle this situation are very welcome.


One answer

permanent link
Lisa Caten (342913) | answered May 08 '13, 2:02 p.m.
If you know which test case results you want to change (from false failure to passed), you can browse Test Case Results, select the results that you want to change, click the Change icon in the Status column heading, select Passed, and then click OK. See Updating multiple artifacts.

Michael Fox commented May 09 '13, 9:42 a.m.

The ability to update multiple results from one screen will be much more efficient then having to go to each result individually.  Since we'll likely have several people doing this daily after each nightly regression run, efficiency in doing this for hundreds of test results is essential.


Bob Gallagher commented May 09 '13, 11:47 a.m.

Michael does this existing bulk change feature satisfy your request? Or are you looking more for presenting more information while viewing the table of results in order to make the decision on whether to change the result to passed?  For example would having a column in the table where each cell would contain a link to the generated log file help? And would it be better still that when you hover over a link you get a popup to view the log file content?

Michael Fox commented May 14 '13, 8:47 a.m.

I don't have a specific solution in mind. We're currently planning our RQM implementation so I can't really comment yet on how well this helps.  We know we're going to face this issue and I was looking for a discussion on how to best deal with the problem. I'm assuming that we aren't the only QA team with this behavior in legacy tests.  Having links to the log file is a great idea.  Generally, anything that would increase our efficiency reviewing the logs would be greatly appreciated.

Your answer

Register or to post your answer.