How to Auto-Create a Defect in RTC when Automated Test Case Fails ?
Hello,
I would like to automatically create Defects in RTC when automated Test Cases Fail, creating so a complete End-to-End automated-integrated process. Is there a way to do that integrating a Test Automation tool with RTC?
Actually I’m using Robot Framework and JMeter to do Test-Automation, but I could use any appropriate tool (Open Source) if it’s possible to eventually integrate it with RTC on my Client environment.
I’m running Front-End Test automation scripts written with Selenium Library keywords, and also automated back-end Testing (batches) everything orchestrated with JMeter, and I wonder if I can create something with Test Automation tools (maybe using RTC work items API) that automatically Log a Bug in RTC (I have Clients License) when any automated Test (front-end or back-end) fails.
Website Front-End application that I’m testing is available on internet (access with no VPN), and RTC is installed on Client’s network.
Really thank you in advance for all your kind help!
Accepted answer
You actually don't necessarily want to do this - there may be many reason why a test script fails for a test case. Many times it's an application configuration error, or a data error, or a network error, or the test script needs to be updated when the AUT changes, or the requirements change, the app changes but the script isn't up to date.
This is why they have the ability to mark a test result with an actual result, so that you can assess the result and determine if it's actually a defect in the AUT, or a problem with the test in some way. Then you can mark the result accurately, and either log a defect (if you do it from the results it will auto fill an awful lot of the info) or log a task to change the test environment or the script.
Comments
Hello Davyd, thanks a lot for answer.
The Client asked me to do a direct RTC integration with Robot Framework (or any other Test Automation tool, but Open Source) to automatically log a Defect when an automated Test Case fails at a precise step of the script, regardless what cause is.
It would be enough that the Test Automation tool could perform an automated Defect creation with Summary like the Test Case step ID that fails and attach a screenshot taken from the Robot F.
Any idea or suggestion please? Is that kind of integration supported by RTC?
Thank you
OK - so is the Test Case you're referring to in ETM or is it in a completely separate tool?
By far the best way to integrate any sort of automated testing tool is through the ETM Test Automation Adaptor API. In this approach, ETM drives the automated testing tool and directs it to execute whatever scripts it uses, and then the tool reports the results back into ETM, where it's analysed.
There are many Automation Adaptors already built by IBM and third parties, and one of those may already work for you out of the box.
IBM supplies one for any tool that can execute tests based on a command line script or call, and also one for Selenium Web Driver test scripts. I've just looked up the Robot User Guide and it has a command line interface, so that would allow you to integrate it straight away:
1 vote
It's also important to understand that many tools confuse the language around testing - what Robot is actually executing is a Test Script, not a Test case.
A Test Case states a requirement for testing, which is then implemented by a manual or automated script.
I know it's small but it's really important when you're talking about what you want to integrate to where - you're actually integrating an automated test script engine into the tools at a script level , not a test case level
1 vote
Davyd that was really Helpful !! Thank you
Yes i need to Automate Test Scripts from Robot Framework (a bunch of keywords) that when executed (a web portal application) and failed at some steps, the Robot runs a call to RTC (EWM) opening a Bug in automatic (could be just a Summary coming from the Failed Test Log...)
Actually my Test Cases are still not written in RQM (ETM), i've just received an .xls with a Test Cases list from my Client...that's all.
Hi again,
This is where many people end up taking a backwards approach to automated testing. It really should be the test management tool driving the automated engine, not the other way around. ETM does this really really well if you follow that approach and, from the Robot Framework documentation, it will also work really well as an automated testing tool if you drive it from ETM.
You can import the spreadsheet into ETM to create Test Cases, then implement those as scripts in Robot and register them in ETM as Command Line Adaptor executed test scripts. The Command Line Adaptor will help here because it can send back a directory listing of script names.
You then link the scripts to the test cases, and execute the test cases in ETM. ETM will call Robot and tell it to execute the correct script, and get the result and log back into ETM as the Test Result. You can then browse all the results and create a defect for the ones that are really bugs, the defect will be loaded with the script information automatically
The actual integration part of this will take you no more than half a day. If you tried to do this the other way round, you will end up taking several weeks, and you will lose a huge opportunity for really good reporting and metrics that will be automatically created for you
showing 5 of 6
show 1 more comments
One other answer
EWM supports these APIs: The EWM SDK, OSLC CM.
You can create ETM integrations using the automation adapter API https://jazz.net/wiki/bin/view/Main/RQMAdapterAPI . You should be able to use the EWM APIs to create work items. A good entry point would be the work item command line https://github.com/jazz-community/work-item-command-line .