build integration with RTC & test schedules
Hi,
We recently managed to get RQM to retrieve build notifications from our RTC repository however the build entries in RQM do not contain any useful details that could give some context to those builds record. All we get is a name and time stamp, and nothing else (i.e properties of the build, link to published artifacts, logs...). It is a problem because i was hoping to use the notifications as triggers to launch test suites. We have many build definitions running against many SCM streams each with different test requirements so if i cannot access any details about a particular build then i do not know - if a test run even make sense for that particular build - where to retrieve the deliverables to test - what test suites to run ... Is this the expected behavior? If it is is there a way to retrieve more? thanks you, david |
9 answers
Hi David,
- if a test run even make sense for that particular build You can use the status of the build to indicate whether tests should be run. I.e., don't run your tests if the build fails. Execution Schedules can be triggered by only a Successful build, or by any build. - where to retrieve the deliverables to test We had decided that this varies too much from deployment to deployment and is non-deterministic. Can you work with your RE team to establish where the binaries exist? Typically, they'll be stored on a central server in a predefined folder structure, with build name / id being used to identify a folder for a given build. - what test suites to run I'm curious, how would you expect a build result to indicate what tests to run? Currently there is no way to retrieve additional information beyond what is displayed, outside of going to RTC to retrieve this information. What else would you like to see? Thanks, John Hi, |
|
Thanks for the reply john,
i was hoping to be able to trigger the schedules based on other custom criteria. Even for a given component on a given stream not all its builds are meant to trigger functional tests (the tests we plan to manage via RQM). For example developers can trigger quick integration builds on demand based on their personal workspace, i such cases they usually want to skip fvt tests and would do so by passing a specific property to their build. So if RQM could have access to at least the start-up properties passed to the RTC builds, then the schedule could possibly be set to do a simple check property check. Typically, they'll be stored on a central server in a predefined folder structure, with build name / id being used to identify a folder for a given build. That's also our approach, except that the full path to the actual storage folder is determined by a few more properties than just the name/time-stamp of the build. Again those properties are passed to the builds themselves at start-up. I'm curious, how would you expect a build result to indicate what tests to run? Again, via the start-up properties of the RTC build. Currently there is no way to retrieve additional information beyond what is displayed, outside of going to RTC to retrieve this information. What else would you like to see? Well, the main thing would be the start-up properties passed to the RTC builds as they give the actual context of the build entries once created in RQM. A secondary but convenient add-on would be to get a hyperlink to the actual build record in RTC. Thanks again for the clarifications, david. |
|
John,
Typically, they'll be stored on a central server in a predefined folder structure, with build name / id being used to identify a folder for a given build. Actually, how is the "build name / id" passed to the automations contained in the schedule? I only ask because my current understanding is that any provisioning required for the tests in a given schedule has to be taken care of by some purpose built automation. Such automation would need some basic information to bootstrap itself. thanks, David. |
|
Hi David,
I definitely agree with most of the points you raise. We wanted to provide more automatic data passing in RQM 2.0 but ran out of time. In-house, we have an external service (Java cmd line program) that gets the most recent build from our central server. We can trigger that automation off of a build completion and deploy the most recent. What we wanted to do in RQM2.0 is map build data to available automation params. So I could say that the value of "buildName" from the Build should be substituted for automation param "foo" at run time. That did not make it so we'll be getting to that in a follow-on release. Regards, John John, Typically, they'll be stored on a central server in a predefined folder structure, with build name / id being used to identify a folder for a given build. Actually, how is the "build name / id" passed to the automations contained in the schedule? I only ask because my current understanding is that any provisioning required for the tests in a given schedule has to be taken care of by some purpose built automation. Such automation would need some basic information to bootstrap itself. thanks, David. |
What we wanted to do in RQM2.0 is map build data to available automation params. So I could say that the value of "buildName" from the Build should be substituted for automation param "foo" at run time. yes, a good ol' string replacement. That would do nicely :-) That did not make it so we'll be getting to that in a follow-on release. Do you have a related task/request in your backlog that i could add my voice to? i searched briefly but didn't find anything. If not I'll create one. thanks again for your time, david. |
|
David,
Please do submit your request. It holds more sway if it comes from a customer instead of from development. It was part of the original work, slipped, and no additional WI was created to backfill what did not make it. Thanks, John What we wanted to do in RQM2.0 is map build data to available automation params. So I could say that the value of "buildName" from the Build should be substituted for automation param "foo" at run time. yes, a good ol' string replacement. That would do nicely :-) That did not make it so we'll be getting to that in a follow-on release. Do you have a related task/request in your backlog that i could add my voice to? i searched briefly but didn't find anything. If not I'll create one. thanks again for your time, david. |
|
Enhancement 21670 created:https://jazz.net/jazz02/web/projects/Rational%20Quality%20Manager#action=com.ibm.team.workitem.viewWorkItem&id=21670
cheers, david. |
Hi,
I've set up an two RQM execution schedules which will kick off a test suite.. (a) when we get a good build (b) at a certain time neither seem to result in the test suite being run. Is there some process, agent or setting that needs to adjusted before this can work? Also, finding why something didn't happen is difficult :-) Any hints on how to debug this ? thanks in advance John |
|
Hi John,
Does the Suite contain manual tests or purely automated? If you go to: Execution->View Execution Schedules then within your schedule, click "History" Do you see anything listed under the history? It should include in-progress and completed execution schedules there. Clicking the link under "Actual Result" will take you to the details of that run in particular, letting you know what has happened (or not happened). Let me know what you find through that mechanism. Regards, John Hi, |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.