What is the recommended practice for storing multi-level test results in RQM?
We have a test harness that runs suites of tests. Suite definitions are stored in source control, because they can vary from release to release. Within a test, there can be multiple test cases (similar to JUnit). (Note that this use of the term "test case" has a different meaning than in RQM.)
We would like to do the follow:
We would like to do the follow:
-
Invoke the harness with a named suite from within RQM (presumably using a TER?) (We can do this today.)
- When the test completes, import the test results for the full hierarchy of suite/test/test case
-
Have the test results sent to the warehouse, for analysis and reporting
- Run reports on the various levels (summary of suite or test or test case results)
-
Be able to navigate from a suite failure to the corresponding test(s) that failed and then to the corresponding test cases that failed
We have an adaptor working that will invoke the harness and import test results. However, we are only getting a single test result at the moment (for the overall suite) and not for the underlying test and test case details.
Comments
paul chu
JAZZ DEVELOPER May 31 '13, 3:59 p.m.I am a UI developer and from the UI, the Result Details section of a test suite execution result are the execution results of the individual test cases within the test suite. You should be able to identify which test case(s) failed within the failed test suite. What exactly is the "test cases" within a test if they are not the QM test case artifacts?
Dick Wallace
May 31 '13, 4:24 p.m.I don't think we can use RQM suites. As I described, our suites are defined in our test harness, outside of RQM. (Unless RQM lets me define a suite without assigning any tests to it and have those associations created when results are loaded.) Our test harness "tests" are what RQM calls "test cases". Our test harness "test cases" are sub-tests within a test (either from JUnit, GTest, or similar.) They are name-scoped to the test.
Deepa Doraswamy
JAZZ DEVELOPER Jun 03 '13, 7:23 a.m.Which adapter are you using and what is the version of RQM?
Thanks,
Deepa
Dick Wallace
Jun 03 '13, 7:44 a.m.We wrote our own adapter. RQM version is 4.0.1.