Creating Multiple Test Case Execution Records with REST API for a single Test Case

Matthew Brazeau

(21●5●5) Oct 01 '14, 2:49 p.m.

The RQM interface can generate multiple test case execution records for a single test case given a preset iteration and the presence of multiple test environments.

Given a new iteration and knowledge of the available test environments in an RQM project, how could one PUT multiple test case execution records into a single test case with the RqmApi?

I currently crash my JazzClient application when attempting to create multiple test case execution records for a single test case, and I cannot set the test environment or iteration (something I would like to be able to do).

Cheers.

0 votes

4 answers

6,196 views

0 votes

4 answers

Permanent link

Stephane Leroy

JAZZ DEVELOPER (1.4k●1●4●9) edited

Oct 03 '14, 3:31 a.m.

Hi Matthew,

I guess you would need to iterate in your client app.
But since you mention a crash, I'd advise you to reproduce with an attempt to create the smallest amount of TCERs.
Ideally, you would want to reduce this number to 1 only.

From there, you could debug / check whether your POST/PUT URL, headers and the Test Phase, Test Env, Test Plan that you feed your new TCERs are really valid. In case of incorrect configuration, the RQM would normally provide you some useful message.

References:
- the core one is RQM Reportable REST API: https://jazz.net/wiki/bin/view/Main/RqmApi
- for examples of URLs, you could refer to the following blog post: http://sleroyblog.wordpress.com/2013/04/09/querying-rqm-40-through-oslc-and-rest-api/

Regards,
Stéphane

0 votes

Permanent link

Matthew Brazeau

(21●5●5) Oct 02 '14, 8:23 p.m.

Hey Stephane,

I need to add multiple TCERs to a single test case with the RqmApi over the Rqm Reportable REST API (not currently using OSLC). Is that possible? I can only successfully add one TCER per test case. In my PUT request, I specify these parameters in my application (which get translated into a properly formatted xml for RQM):

create_test_case_execution_record( self, tc_record_name, desired_external_id, testcase_external_id, testplan_external_id, rqm_configuration_ext_id = None, tcer_categories = [] ):

So what differs between PUT requests for a TCER is desired_external_id and tcer_categories; I suspect that RQM does not see these differences as a valid reason to create two TCERs for the particular test case (referenced by testcase_external_id), and so RQM does not allow my second PUT request to be accepted.

Again, RQM seems to only allow multiple TCERs for a single test case if their test environment and iteration are unique, but these parameters are not accessible over the RqmApi for the RQM Reportable REST API (so generating multiple TCERs must be done in the RQM interface, which is what we are trying to instead automate in our application).

Best,

-Matt

0 votes

Comments

Stephane Leroy

JAZZ DEVELOPER Oct 03 '14, 3:42 a.m.

Hi Matt,
I believe your suspicion is right. A TCER is defined by a limited number of characteristics (test plan, test phase, test environment...). The RQM tool would not let you create TCERs that are considered redundant with a pre-existing one.
What's your use case for varying the category of a full defined (i.e. with test plan, etc.) Test Case Execution Record ?
Regards,
Stéphane

Permanent link

Matthew Brazeau

(21●5●5) Oct 03 '14, 10:42 a.m.

Hi Stephane,

A test case captures a unique behavior of our product. This behavior is to be observed in multiple test environments, and for each new firmware iteration. A firmware build is captured in a Test Case Execution Result since an iteration consists of one or many builds (so then a TCER contains one or many TCEResults; nothing unique here). This goes well with RQM's behavior of marking a TCER status as that of the most recently added/modified TCEResult since that then represents the status of our latest build within an iteration.

We have defined a test case execution record, for any test case, as having a unique test environment and iteration (as does RQM). Many test environments and iterations exist for many test cases. Generating sets of test case execution records, for a given test case in a given test plan, can be do so manually in the RQM interface (Test Plan -> Test Cases -> Action Menu -> Generate Execution Record -> then chose iteration & Generate Test Environments... bingo, you now have multiple TCERs for a single test case).

Our automation effort requires us to be able to generate multiple TCERs for a single test case on the go with iteration and test environment settable (not possible with the RQM Reportable REST API).

Use case example; a result has been made available and an existing test case does not have a matching TCER with correct iteration and test environment. Since there are many results and many test cases, it would not be feasible to stop/flag the uploading process, notify a human to go add the new iteration or test environment in the RQM interface (admin privileges needed to do this) and then, for each test case within a test plan, go and generate all possible combinations of iteration and test environment. In fact, there is no guarantee that every test case will be performed in every test environment for every iteration, so generating TCERs from the RQM interface in every combination of iteration and test environment before a result can be added could yield TCERs that never have a result uploaded.

Again there will be many test cases, test environments, and iterations so many TCERs will need to be generated as results are made available. We are not using RQM to generate test coverage for us (I suspect that is why RQM can generate TCERs for combinations of iterations and test environments) since an internal tool does this for us already. Instead, RQM is our tool for reporting what happened so we need to be able to specify test environment and iteration while creating a TCER, for a single test case, just before we create a result for said TCER.

Feel free to ask for any clarification(s),

-Matt

0 votes

Comments

Matthew Brazeau

Oct 03 '14, 10:46 a.m.

In this use case, there exists one test plan.

Stephane Leroy

JAZZ DEVELOPER Oct 06 '14, 12:33 p.m.

Hi Matthew,

first, let me share with you that I don't have a definitive answer for your particular use case.
That being said:
- I got that you're NOT using RQM for planning your test execution effort (through preliminary creation of TCERs) but only for reporting.
- but AFAIK, this doesn't imply that you would need some "duplicated" TCERs (i.e. TCER with exact same test env, test phase, etc.) in RQM.

Instead, I wonder if you have checked the option of having your internal tool to directly/indirectly feed RQM with the missing TCERs "on-the-fly" ? By "on-the-fly", I think of a process triggered in a periodic basis that would update RQM with NEW TCERs. Note that it should run BEFORE any related test execution result(s) is/are reported in RQM,
That way, you would not reach a situation where a test result is to be reported in RQM but was left orphan (in terms of associated TCER) like in your current situation.

My 2 cts.

- Stéphane

Matthew Brazeau

Oct 06 '14, 12:40 p.m.

Hey Stephane,

That plan makes sense and is what we wanted to do, but it is not possible to feed RQM multiple TCERs for a single test case. This is because each TCER for a single test case must have a unique test environment and iteration, but these are not configurable over the RQM Reportable REST API. Do you have a solution?

-Matt

Matthew Brazeau

Oct 06 '14, 12:45 p.m.

To be clear, try to PUT two TCERs into a single test case with whichever configurations you want over the RQM Reportable REST API. I have observed the second TCER to be rejected since RQM views it as a redundant artifact (again, since one cannot specify test environment and iteration during the PUT). Perhaps I have overlook something?

Stephane Leroy

JAZZ DEVELOPER Oct 06 '14, 12:55 p.m.

If you PUT the exact same TCERs definition: yes, indeed it will be rejected (nominal behavior). But if you change whatever part of n-uple (test Plan, test phase, test environment, etc.) so that's it's new to RQM, it should pass.
If you should re-use the same n-uple / TCER, then do not try to create it again, just reference the existing one when adding a new test result.

Maybe it's me who misunderstand your need ?...

Permanent link

Matthew Brazeau

(21●5●5) Oct 06 '14, 2:12 p.m.

We are almost on the same page;

Is it possible to set Test Environment and Iteration for a Test Case Execution Record, in general, from the Rqm Reportable REST API via a PUT request when creating a Test Case Execution Record? (Note that a GET request on a Test Case Execution Record does not report Test Environment or Iteration).

I am trying to add multiple TCER's to a single test case with constant test plan and test phase attributes. Test Environment and Iteration are the only attributes that will be different between the single test cases multiple TCER's (besides their respective external ID's)

0 votes

Comments

Stephane Leroy

JAZZ DEVELOPER Oct 07 '14, 5:04 a.m.

Hi Matthew,

hummh... anything wrong with your REST calls ?

A GET request on a TCER does provide Test Environment & Test Phase information. Let me provide what I get on a 5.0 (would be similar in 4.x):

You should be able to POST/PUT new combinations of n-uples / TCERs
Regards,
Stéphane

1 vote

Your answer

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers

Answers and Comments

Question details

usage

× 12,014

rational-quality-manager

× 10,964

Question asked: Oct 01 '14, 2:49 p.m.

Question was seen: 6,196 times

Last updated: Oct 07 '14, 7:23 p.m.

Creating Multiple Test Case Execution Records with REST API for a single Test Case

4 answers

6,196 views

0 votes

4 answers

Comments

Comments

Comments

Your answer

Follow this question

Question details

Related questions