How to find the test cases which fail most often in RQM

Eva Jensen (6●2●2) | asked Apr 08 '13, 5:43 a.m.

Hi,

To explain my question: In my team we are going to use RQM for automated test which runs each night.

Some test cases are not passing each time. They are unstable and need to be redesigned.

I want to use a report i RQM to identify these test cases. How do I do that?

I have say... 500 test cases with results 356 days a year.

I do not have the time to look in to all results for each test to find the ones that some times fail....

It could be a graph shown the number of fails for each test cases.....most will show 0.

Or just a list .....showing the count of fail or each test cases.....ordered by the highest count.

Does somthing like that exist in RQM or can it be created?

2 answers

Most liked answers ↑|Newest answers|Oldest answers

permanent link

Stephane Leroy (1.4k●1●4●9) | answered Apr 08 '13, 10:32 a.m.
JAZZ DEVELOPER

Hi Eva,

first I would check whether RQM OOTB reports cover your needs:
the "Execution Status by..." reports could be good candidate to start with.
(e.g. the "Execution Status by Owner using TCER Count (Live)" report may fit. Note that you could also customize it such that it only display failed/inconclusive/etc. executions results).

Since you mention this activity for "tracking the unstable test cases" occurs on a daily basis, I'd suggest you to add the identified report to your dashboard. Every morning, you could drill down to the failing TCERs and associated test cases from there.

If the OOTB reports are not enough, you'd like to envision the use of RRDI.

Regards,
Stéphane

Comments

Eva Jensen commented Apr 08 '13, 1:19 p.m.

Hi, I have tried to simulate some test results and use the suggested report. It seems to be that the report only pays attention to the latest test result. The intention is to look back on all old test results.

The result I am looking for might on the form:

Test case name x failed 34 times

Test case name y failed 32 times

Test case name z failed 15 times

Test case name p failed 5 times

Test case name g failed 0 times

for a defined period.....

This data might be collect over a period of 14 days to 6 month.

Perhaps this requires a specific way to organizes test cases and use TER's ?

permanent link

Stephane Leroy (1.4k●1●4●9) | answered Apr 09 '13, 6:01 a.m.
JAZZ DEVELOPER

Well. I see some options:

1) try the "Summary > Tester Report using TCER Count (Live)" report
   You will get information on Passed/Error/... status splitted by Owner
   and from any cell of the tab (defining an owner + a status), you could drill down to the associated "TCER Listing (Live)"

2) try the "Summary > Execution by Test Schedule (Live)"
   Note that you'd need to adapt to information it displays as Passed/Error/... are reflecting associated weights (and not the total numbers of times a TCER was run)

If not enough or if requiring more specific reporting , you'd probably want to figure out the capabilities provided of RRDI. Here are some interesting links :
1) the CLM 2011 Reporting workshop
2) the CLM 4.0.1 Reporting workshop
2) this blog post providing a mind map around Reporting for CLM

Regards,
Stéphane

It's all about the answers!

Related questions

How to find the test cases which fail most often in RQM

2 answers

Your answer