Automated Test Deployment with the Rational solution for Collaborative Lifecycle Management - Part 1
Marc van Lint, IBM The Netherlands B.V.
Last updated: January 31, 2012
Build basis: Rational solution for Collaboritve Lifecyle Management 3.0 and Rational Functional Tester 8.2
Things have changed in the Rational solution for Collaborative
Lifecycle Management (CLM). The new architecture and new licenses of the
release 3.0 software provides an even tighter integration across the
disciplines. Teams can benefit from these changes.
This article describes how the test community can benefit from the development features provided by Rational Team Concert (RTC). Next to the planning and defect management, we now can utilize other features of RTC like code management and the build engine. This becomes especially important when customers go beyond the level of self contained scripts and maximize reuse by:
- Using the power of object oriented programming
- Maximize code reuse (e.g. helper classes or supporting software etc)
- Share changing datapools
A successful test depends on the deployed application and
the deployed test. This approach gives full control over the deployment
of test scripts. The development is done by the team sharing resources
and code. Deployment of test ware is done via automation in a fully
The chosen approach is described with Rational Functional Tester (RFT), but it's equally valid for Rational Performance Tester (RPT), Rational Tester for SOA Quality or other Eclipse based products.
The product RTC is very well received by developers. Now with the new releases and licensing testers can benefit tremendously from the full scope of the collaborative lifecycle management products of IBM Rational. Testers can utilize the whole scope of Rational Requirements Composer (RRC) to refine requirements and define their own test requirements. They can utilize RTC for the tasks definitions and the distribution and planning of the work. These benefits have significant impact on organizations, to name a few:
- Integration of development and test processes
A fully controlled development and test-developement process, with a high level of automation and automated deployment.
- Alignment of planning and execution of development and
Code developement and test developement are coupled from the start of the process. This results in 'just in time' development of test ware.
- High level of quality in the test scripts
All development tools are now available for test development. This includes Junit tests or code checkers. This enables the highest quality of testware eliminating test failure due to errourenes code in the tests.
- Improved predictability
Due to the high level of automation and thus control on the process the obtained work is of an expected quality. This will result in a high level of predictability of the test outcome.
This list is not complete, but hopefully it's good enough to read both documents.
The approach is especially interesting in combination with the
ICTL Framework of IBM. The framework results in a minimizing of
maintenance utilizing a Java-framework. When this is maintained the
describes solution will 'generate' new high quality test execution
More on the ICTL Framework can be found on IBM's developerWorks (ICTL Framework)
A solid base must be available before implementing this vision. This base should consist at least the following building blocks of defect management, source control and test automation. The organisation should have experience with all three area's before considering this journey. Otherwise this is an option in the future to evolve to.
We are in a situation where we already utilize the Rational-CLM product line. Requirements and specifications are captured with RRC, planning and development is done in RTC and test management with RQM.
To improve regression testing we want to leverage automated testing. Here fore a team of testers/developers is created. They will develop the automated tests. An important input for this development is the already defined manual test, which is already successfully use in the regression tests.
Multiple members are involved in creating the (automated) test scripts concurrently. Still the federated set of test-scripts and code are off-loaded to agents. The automated tests are initiated from the web-clients of RQM.
The envisioned used environment might look like the following picture:
Figure 1: The envisioned environment for test delivery.
The automated tests are be executed on the agents. The tests are started interactively in the RQM Gui but also automated by creating a RQM test schedule.
Figure 2: The two streams of code to the testing environment. All enabled by Rational Jazz
There will be two streams of code to the Test Environment:
- Code stream - the traditional (java) code delivery of the application
- Test stream - the (code) files needed for automated testing
Including Build component - all files needed for correct running of a test build
The Rational CLM Usage for testing
If an organization is interested in creating high quality software products in an efficient manner they need more than just automated testing. Only part of the process is test execution, that is what's optimized by automation. IBM Rational provides a platform to capitalize on benefits though all testing activities, including:
Utilizing Rational CLM the team collaboration will improve. While working, information is shared automatically. Plans are adapted automatically, feedback loops are instant. All improving speed and quality of communication.
Again, not only test automation, but also process automation. Automatically getting informed that code is available, triggering events. Test plans which are updated with latest execution results. Automatically reserve test machines and install software before a test starts. All are options. The compilation of status-documents can be automated with the Rational portfolio.
This article is a prime example of that.
Rational CLM gives insight in what's done across the teams. The vast amount of data is turned into personal information on dashboards. Reports are generated, jumpstarting management communication. With business intelligence technology improvements can be defined.
These area's are major improvements for (independant) developement organisations across all disciplines in all fases of the delivery process.
The Process Cycles
The following cycles are defined. These cycles are constantly repeated when possible using Rational automation.
Figure 3: The 5 cycles of test development.
A change will 'flow' along these steps to be implemented in the testing production. A change will be incorporated in a next step as soon as it's delivered by the previous one. A cycle will work on the materials available when executed.
- Cycle 1: Local Development
Local creation and validation of the test script.
- Cycle 2: Personal Build
Run a personal build on the server to validate compilation.
- Cycle 3: Integration Build - Automated
A automated serverbuild creating the deliverables.
- Cycle 4: Tag the Build - Automated
Mark specific builds ready for usage in the test environment.
- Cycle 5: Deploy Last Build - Automated
Deploy builds on the RFT-agents.
The deployed tests are executed automatically using a RQM Test Schedule.
Cycle 1: Local Development
At this step multiple test members are working concurrently on the same project. Some are defining and refining test plans, other test cases and other team members are executing tests.
- What to do?
A team member pulls a task to create or improve an automated test. From the team-plan he knows the priority. From the task in Rational he knows the context. All related information is available like description, discussions, related artifacts, open defects etc
This information is vailable from the well know Interactive Development Environment (IDE), the daily workbench of a developer.
Figure 4: Task view in the Eclipse IDE.
The test contains pieces of code defined by other team members, like here the validation routine to check for the price of a CD. While he develops, he accepts (or rejects) incomming changes of work done by other collegues. Conflicts are directly found and can be corrected.
Incoming changes of collegues are directly visible:
Figure 5: Incoming changes.
He validate the correctness by running the code against a test application locally.
More information can be found on:
- RTC - Features overview
Cycle 2: Personal Build
This step is targeted to compile the personal code with the
settings of the server, but with a copy of the local workspace.
If this build is successful, one can deliver the code to the build with a high level of confidence not breaking the integration build.
- Check the code in
By checking in the code, he 'saves' the code to the RTC server-side repository.
- Run a personal build
Validate the compilation of the script in the context of all other changes. At this stage one can implement Junit validations to obtain an even higher quality and increase confidence of success.
- Deliver the code
The code and changes are delivered to the stream.
Cycle 3: Integration Build - Automated
The integration build compiles all test scripts and their dependancies. What's done an how it's done depends highly on the implementation of the build process (see Part 2). In this example compilation is done and a package is created. This package can be distributed to the RFT Agents.
When an integration build is initiated the team is informed in
various ways. Clearly the dashboards can reveal overview and details of
build status. An other way is the toaster message appearing at the
bottom of the screen. This is only available when a RTC Client is
Figure 6: The friendly toaster message.
An other nice option of Jazz is that it can start a build on a scheduled time. So it's possible to start such a build a night, every night, or every hour. It will only do an actual build when changes are delivered.
Again JUnit tests can be implemented to target integration tests.
Cycle 4: Tag the Build - Automated
The tag ready_for_use is used in this example to identify
a build which can be used for regression testing.
In the example this tag is added to a build when a zip file is delivered. The build can have more extensive checks to do the validation of the build.
Often these toll-gate descisions are done manually. Based up on human judgement a person can add manually this tag.
Figure 7: The list of builds, some of them are marked with ready_for_use.
Cycle 5: Deploy - Automated
The deployment of the code is rather simple. The repository
workspace of the integration build is copied to the shared location.
This shared location is used by RQM and RFT as the 'source' location of
scripts to be executed.
In the sample script a zip-file is created and maintained in the RTC build log-files. In case of issues one can always recreate an old state of the testing environment.
Changes in test management
In essence there are no changes in how test management and manual testing is done. This is complemented by this test automation. When tests are implemented in a automated way, test management can schedule these automated tests instead of the manual implementation.
There are several new steps to execute for the RQM users.
- Importing the automated scripts
- Attach the automated scripts to the test cases
- Run or schedule tests
(Optionally) Create Test Schedule
Within RQM one can create a schedule to start tests at a certain moment in time. Make note that these tests are updated in an organic way and automatically deployed.
What did we create?
A new architecture is created. A brand new way of creating automated tests. We can leverage this for complete automated test, framework development, (automated) keyword testing or any combination. It enables automated test development in a concurrent fasion.
By using the Rational-CLM solution we are working on the right area's of value using work items and planning. Utilizing the source management and build management we can work concurrently on our automated tests.
But it's more. Is a start for a next level of productivity. I see things like:
- Generation of status documents
- Corrective actions based on metrics
- Productivity analysis
- Better estimations based on historic behavior
- Introduction of tests on the test-code.
There is more to explore.
Test teams are part of a developement lifecycle. Even independant test team do not mean that they are isolated individuals! Communication is essential. This is supported by the Rational Jazz platform. Work is defined, prioritized, planned and executed. A daily routine for agile teams but also for test teams. Test teams can fully utilize the benefits of Jazz.
When teams are distributed and working together to create testware advanced process support is needed. This is provided by IBM Rational Jazz products. Not only test management with Rational Quality Manager, but every aspect of test software delivery including creation of automatic builds.
It provides an unmatched level of automation for software and test software development. It provides a platform to think about next level of productivity.
For more information
- Automated Test Deployment with the Rational solution for Collaborative Lifecycle Management - Part 2
- Table mapping of licenses (Scroll down).
About the author
Marc van Lint started in 1986 at IBM as a intern. He was
active for 18 years in the Product Lifecyle Management branch of IBM. In
2005 he continued his career at IBM Rational. His prime focus was to
achief Quality in the software delivery process at the various
customers. He presents his visions to customers and help them to
implement their improvements. He contribues frequently to the community
by publishing and maintaining his own website. He's co-author of the IBM
Press book "Software Test Engineering with IBM Rational Functional
Tester - The Definitive Resource".
He can be contacted at firstname.lastname@example.org.
Copyright © 2012 IBM Corporation