Using RQM to track results from existing automation env
I am leading a project to attempt to get RQM into product for my team, initially JUST for tracking and reporting test results.
What I mean by this is that our existing system has a large mature automation environment for scheduling test execution to our test machines. It currently reports results in the form of XML into a bespoke test tracking tool. This tool simply allows us to tell it what tests exist, and what platforms they are supposed to run against, which defines the expected target counts of tests per platform. It receives the results from the automation environment that tell it which platform the result are for, and gives a result per test. This tool then provides various reporting capabilities. Whilst I understand that RQM is capable of completely replacing our entire system, taking on the role of scheduling test execution etc. I need to demonstrate that we can adopt it in a number of discrete achievable chunks that are not 'throw away everything in one go and replace it' So the problem... RQM seems hard wired to expect to be the thing that triggers test runs, and it gathers results as responses to what it kicked off. The REST api that I am looking to use to insert information does not support POST against executionrecords. so I can only update existing ones, or create a new one if I 'know' a new id to give it. Having played a little with this, I am also struggling to make these tie up properly, such that I can manually identify an ID that isn't currently used, and upload a result for it such that it shows up under executionresults. However none of the links from it work, I obviously need to do something else to make it link up properly. So that's a very long winded way of asking, is there an accepted way in which I can use RQM to plan my tests, but leave all the scheduling and exection to our existing system. Allowing it to interact with RQM in such a way as to tell it about the results and have everything work? |
10 answers
Hi, Daniel. Seems like you and I are in the same boat. I'm also trying to find a clean solution using the REST API (or any other alternative) that would allow me to dynamically create / update execution records from an existing automation framework.
I'll be watching this thread for sure. |
Hello,
Until the RQM test execution framework is made public, the REST API is the way to go, and even might still be the way to go when it is public, depending on your explicit needs. Daniel, there is no when you say Execution Record, I assume you refer to Test Execution Record (which is executionworkitem in the REST API, for legacy reasons). That does support POST/PUT. Execution Result does not support POST, but does support PUT. Any reason you're compelled to use POST instead of PUT? There's an important distinction that you both need to make: the REST API that is used by the RQM server to communicate with test adapters and facilitate remote execution, compared with the general REST API used to manage all of the artifacts in the RQM system. It sounds like you'd be best served by the latter, doing all of your planning and environment configuration in RQM, running/coordinating the tests externally in your existing system, and PUTing the results of these tests to the RQM server. Hope that helps a little. Regards, John |
Hello, Hi John, I did mean execution result, and the reason I *thought* I needed POST was so that RQM would create a new entry with a unique ID, rather than expect me to specify one. Since I would have no way of determining a valid and unused ID. If you do a PUT for an execution result with an ID that already exists it simply gets overwritten and you can't tell that it's happened. However, that said I've now discovered that if I do a POST and use a URL of format: https://hostname:9443/jazz/secure/service/com.ibm.rqm.integration.service.IIntegrationService/resources/DemoProject/executionresult/ORderSingleCD_ER.xml eg with the name of an xml file at the end of the URL, then RQM does generate me a new executionresult with a unique ID. This means I'm good to go I think, the next challenge is how to make sure I identify the correct TER to link an execution result with to ensure that when a test runs in my automation on an AIX box, that it relates it's result to a TER for an AIX box. |
the reason I *thought* I needed POST was so that RQM would create a new entry with a unique ID, rather than expect me to specify one. Since I would have no way of determining a valid and unused ID. If you do a PUT for an execution result with an ID that already exists it simply gets overwritten and you can't tell that it's happened. Your observation is correct. There are two types of IDs in RQM, the "internal" ID (of form urn:com.ibm.rqm:<type>:<id>) is generated by RQM and the "external id" which is an ID specified by some other source. When using the REST API you should never try to create a new item and manually assign it an id that looks like an internal id (for the very reasons that you pointed out). You should always give it your own external id. The external id is useful for migrating resources over from a different system that might have already had its own ID system in place. If on the other hand you don't care about the ID that gets assigned to a new item, I'd suggest using something like the current time's timestamp as the id. This will ensure that your ids are unique. Hope that clears things up. Mike |
Mike, thanks for clarifying the reasons behind my observations. Whilst I would say that is now clear to me, I don't think it is clear in the documentation for the REST API I think there are a few people like myself looking to interact with RQM in this way and it is not clear in the doc how to do so. Which is a shame since it does allow for an important usecase. |
|
I agree its a little cumbersome. I'll see what I can do about get more clear language into the doc.
Mike |
I agree its a little cumbersome. I'll see what I can do about get more clear language into the doc. I think the more I understand the REST api the more I see we will have difficulty trying to achieve what we need. We have an automation infastructure which controls scheduling and running of our tests (50k+) accross our environments (30+OSs) and they currently report results to a bespoke tracking system as TESTCASE NAME with result, and build, os name etc To use RQM we need to be able to generate an executionresult which ties up with a test execution record. Hoever there seems to be no way for the machine to derive the TER to use unless we create it via the REST api in the first place and give it some name that we can look up. The same applies for the test case. It seems that if we create it via the REST api then we can give it an identifier (such as test case name) which we can then use from an automation machine to look up the right record. However if the test case is created by RQM we can't find figure out the URN that RQM gave it. Is there any way that we can use the REST api to report executionresults and figure out the TER that it associates with? without having to create all entities externally and maintain them in our own external DB(which rather defeats the object) |
|
Further to my above post, having spoken to other teams in a similar situation I've found they have all been forced down the same path I am. Which is a much smaller step of adoption than we would like. I have raised an enhancement request explaining the implications and what I believe could be done to allow a sensible migration path.
https://jazz.net/jazz02/web/projects/Rational%20Quality%20Manager#action=com.ibm.team.workitem.viewWorkItem&id=21266 |
Hi,
I'm trying to configure de rqm.xml data configuration file to integrate RQM with INSIGHT. I fill all the fields in the Resources/RQM node and i get this error when I test connection: CRREE2871E Values of the fields are: - Name: RQM - Description: - URL: https://localhost:9444/jazz/secure/service/com.ibm.rqm.integration.service.IIntegrationService/resources/Quality+Manager - URL Type: Generic XML - Authentication Type: Form - User name: $user - Password: $pass User and password are correct because with this user and password I correctly login to the application. The port is 9444 because I change the default that is 9445. Some additional information: if I put the url in the browser I get http error 501. Any Ideas? |
On 4/26/2010 9:07 PM, jalberti wrote:
Hi, end of your URL to test connection. If it passes then remove it before doing the final save - this is a known problem and is documented somewhere but I don't remember where. |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.