It's all about the answers!

Ask a question

TestCase by TestSuite Insight report - help -


Gonzalo Soto (961159) | asked Sep 22 '10, 9:00 p.m.
Hi!
I am working on a report called TestCase by TestSuite, that pulls data from RQM.

The structure of the report is:
TestPlan > TestSuites > TestCases > 1 Execution Result (latest one).

Up to TestCases everything goes right. Then I join every TC with their Execution Result (the latest one).

The problem is that a TC may belong to different TestSuites. So we may find the same TC within many TestSuites.
And this is a problem because when I run one of those TC creating a new Execution Result, all the TCs (with the same name but in different TestSuites) get updated.
I can't figure out how to make every TC unique, so a single Execution Result updates a single TC.

I also tried to join every TC with its Execution Record (ExecutionWorkItem), but I found out that I may find the same ExecutionRecord for the same TC in different TestSuites, as you may see in the screenshot:
http://img580.imageshack.us/img580/3237/tcbyts.th.jpg

So, in summary, I need to differentiate every TC, so when the same TC is ran within different TestSuites, the ExecutionResult updates just ONE TC.

Any ideas?

I was wondering if I can change the approach and go through the Test Suite Result Log...

Thanks in advance!

6 answers



permanent link
derry davis (23222016) | answered Sep 23 '10, 8:57 a.m.
Hi!
The problem is that a TC may belong to different TestSuites. So we may find the same TC within many TestSuites.
And this is a problem because when I run one of those TC creating a new Execution Result, all the TCs (with the same name but in different TestSuites) get updated.
I can't figure out how to make every TC unique, so a single Execution Result updates a single TC.


Hi Gonzalo,
It sounds like this is issue with RQM not necessarily Insight. I'm specifically referring to the bold above. If this is happening in RQM, I'm not sure there's anything you can do in Insight to resolve it. Garbage in, garbage out.
Actually, I would expect this behavior. If you have the same test case assigned to multiple test suites any time you update the record it should show up in all locations.
-Derry

permanent link
Gonzalo Soto (961159) | answered Sep 23 '10, 7:26 p.m.
Hi!
The problem is that a TC may belong to different TestSuites. So we may find the same TC within many TestSuites.
And this is a problem because when I run one of those TC creating a new Execution Result, all the TCs (with the same name but in different TestSuites) get updated.
I can't figure out how to make every TC unique, so a single Execution Result updates a single TC.


Hi Gonzalo,
It sounds like this is issue with RQM not necessarily Insight. I'm specifically referring to the bold above. If this is happening in RQM, I'm not sure there's anything you can do in Insight to resolve it. Garbage in, garbage out.
Actually, I would expect this behavior. If you have the same test case assigned to multiple test suites any time you update the record it should show up in all locations.
-Derry
Hi Derry, thanks for the quick answer.
I was able to associate the TC with its TestSuiteLog, and then join them all with the TestSuite. That way, every time an ExecResult is updates, it goes to its current TestSuiteLog and then to the related TC.
Take a look at the screenshot, I ran 2 TC from different TestSuites and they were updated correctly.
http://img713.imageshack.us/img713/4080/tcbytsv2.th.jpg

Now, the issue I am having is how to bring only the latest results. In the screenshot, you may notice that I bring all execution results. I just want the latest ones.
For that I tried the maximum* function over the updated field, and then filter maximum = updated date.

* maximum ( for ).

As you can see, the maximum function is scoped just for TCs. This works ok if TC names are not repeated.
BUT in my case, I could have more that 1 TC with the same name but ran on different TestSuites (see screenshot above). This makes the maximum function evaluate 2 TCs ran from different TestSuites and take the latest one...
I need to come with a function that evaluates the latest within TC and TSlog.

Or may be sort executions within TCs, and just bring the 1st one? Is that possible?

Any ideas?

Thanks in advance!

permanent link
derry davis (23222016) | answered Sep 23 '10, 10:29 p.m.
Funny thing, we ran into a problem with our execution results, test cases, and scripts today that is nearly identical to this one. It has to do with the way TER are displaying for every test script related to a test case that has a TER on it. Working on fixing it now.

permanent link
Gonzalo Soto (961159) | answered Sep 24 '10, 2:06 p.m.
Funny thing, we ran into a problem with our execution results, test cases, and scripts today that is nearly identical to this one. It has to do with the way TER are displaying for every test script related to a test case that has a TER on it. Working on fixing it now.


:lol: Please give me a heads up if you find a solution.

Meanwhile, what I did to solve my issue was to join TC+TCSuite and make it a Data Item to be used as a unique key that identifies one TestCase.

So, regarding the maximum function, I used this way:

http://img521.imageshack.us/img521/9308/screenshot833.th.png

As you can see in the screenshot, instead of using "for " I used "for ". That way, every TC is unique, and I am able to update its execution result correctly.

Can you think of any collateral damage?

Thanks for your time!

permanent link
derry davis (23222016) | answered Sep 27 '10, 11:14 a.m.
Funny thing, we ran into a problem with our execution results, test cases, and scripts today that is nearly identical to this one. It has to do with the way TER are displaying for every test script related to a test case that has a TER on it. Working on fixing it now.


:lol: Please give me a heads up if you find a solution.

Meanwhile, what I did to solve my issue was to join TC+TCSuite and make it a Data Item to be used as a unique key that identifies one TestCase.

So, regarding the maximum function, I used this way:

http://img521.imageshack.us/img521/9308/screenshot833.th.png

As you can see in the screenshot, instead of using "for " I used "for ". That way, every TC is unique, and I am able to update its execution result correctly.

Can you think of any collateral damage?

Thanks for your time!

Hi Gonzalo,
Well I've found out the RQM data model is more complex than I thought at first. As an FYI, there are 'test execution records' and 'test execution results'. These are 2 different objects and so far I have only found data for the results. As for the answer to our reporting issue, I used the join functionality in report builder and queries for test execution results, test cases, and test scripts and joined on the test case id. It seems to work but now I have the addition of the execution records.

Your solution seems reasonable. It seems like there might be a performance hit eventually, depending on the # of records. As work continues with this report, I'll keep it in mind.

-Derry

permanent link
Gonzalo Soto (961159) | answered Sep 27 '10, 5:22 p.m.
Funny thing, we ran into a problem with our execution results, test cases, and scripts today that is nearly identical to this one. It has to do with the way TER are displaying for every test script related to a test case that has a TER on it. Working on fixing it now.


:lol: Please give me a heads up if you find a solution.

Meanwhile, what I did to solve my issue was to join TC+TCSuite and make it a Data Item to be used as a unique key that identifies one TestCase.

So, regarding the maximum function, I used this way:

http://img521.imageshack.us/img521/9308/screenshot833.th.png

As you can see in the screenshot, instead of using "for " I used "for ". That way, every TC is unique, and I am able to update its execution result correctly.

Can you think of any collateral damage?

Thanks for your time!

Hi Gonzalo,
Well I've found out the RQM data model is more complex than I thought at first. As an FYI, there are 'test execution records' and 'test execution results'. These are 2 different objects and so far I have only found data for the results. As for the answer to our reporting issue, I used the join functionality in report builder and queries for test execution results, test cases, and test scripts and joined on the test case id. It seems to work but now I have the addition of the execution records.

Your solution seems reasonable. It seems like there might be a performance hit eventually, depending on the # of records. As work continues with this report, I'll keep it in mind.

-Derry

Hi Derry.
Yes, they are 2 different objects. You may find 'Test Execution Record' data as 'ExecutionWorkItem' on Insight.

In my case, when a TC is ran from different TSuites, the ExecutionWorkItem id seems to be the same. So the Execution Result is updating TCs even from multiple TSuites... and that's wrong. That's the issue I am challenging right now. I think I should go for the TestSuiteLog (Test Suite Execution Result in RQM), that way, I could update TCs with the correct results...

See the screenshot, where a TC is ran from multiple TS, and then the result is duplicated among the the TC with the same name:

http://img824.imageshack.us/img824/8345/screenshot835.th.png

You can tell because the have the exact same EndTime.

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.