In my last post, I described how we use IBM UrbanCode Deploy to quickly and easily deploy server applications produced by each build. In this post I will describe how we are using those deployed applications to improve our automated testing and reduce the time it takes to validate the contents of a build.
Quality is important in our software and we do a lot of testing to make sure the code behaves as expected. In our builds we run unit tests to verify that blocks of code perform as expected. We also run tests to make sure that different blocks integrate and play nicely together. These tests do a great job of making sure the pieces can do their part, but we also need a way to ensure that the system as a whole, the functionality customers actually use, works as designed. As software has become more distributed, the environments in which they are deployed have grown in complexity.
We have a build that produces two servers which provide different but interrelated services. In the “real” world, these servers are installed and configured so that they are aware of each other. Various third-party applications connect to one or the other of the servers (but usually not both) and can interact by posting and getting requests. While this interaction is occurring, the two servers communicate and work together. The process of validating this scenario is basically:
- Install server 1 on one machine
- Install server 2 on a different machine
- Configure server 2 to talk with server 1
- Install a third-party application on yet another machine
- Configure third-party application to talk with server 2
- Run tests with third-party application
- Verify that all interactions with server 2 are correct
- Verify that server 1 is in the expected state
When done manually, this is a time consuming and error prone task even when the tests themselves are automated. This may be acceptable for infrequent releases where a test team runs through the steps to verify “blessed” builds, but it simply doesn’t scale to the continuous delivery paradigm. Typically we run daily “real” builds, but sometimes dozens of personal builds are run by developers over the course of a global work day. We use IBM UrbanCode Deploy to automatically set up the environment so that we can run our tests as part of every build. The test results are reported back to the build result in Rational Team Concert, so when the build is done we know that a “good” build means not only did the compilation and packaging succeed, but also the functionality has been validated at unit, integration, and system levels.
Each build sets up an environment to do the testing that looks something like this:
My Rapid build deployment using IBM UrbanCode Deploy post describes how we are able to deploy the servers from the build process–that gets server 1 and server 2 deployed in the cloud. Since UrbanCode Deploy deployed both servers, it knows where they are and has a step that configures things so that the servers know about each other. The third-party app is easily brought up on the build machine and is told how to find server 2. The tests are then invoked and run against server 2 and, after verifying that works correctly, it is also possible for additional tests to validate the state of server 1.
Along with speeding up the testing process, we have observed several other benefits of this approach:
- Better test coverage because tests run against real server in a realistic environment.
- Build setup and running of tests is much simpler—just point the tests to the server and run them.
- Fewer differences between build and developer test execution.
- Developers can debug tests against the same server used by the build (or deploy a server with an identical configuration).
- Less overhead on build machines because servers are deployed on separate machines.
- Tests can be run against multiple, complex environments to provide better coverage.
I was only introduced to UrbanCode Deploy earlier this year, but I have been very pleased with how easy it was to integrate it into our process and provide value quickly. Like most processes, I expect that there will be numerous iterative changes and improvements to what we have in place, and I expect that UrbanCode Deploy usage will increase as we work to support continuous delivery and improve our automated test coverage.
Nathan Bak
Software Engineer, Master Inventor
IBM Rational
Loading...
We are working with a project that deals with Db2, Datastage and Reporting. Can we use UCD for automatic deployment in the area?
So far I haven’t encountered a scenario where UCD can’t be used for automatic deployment. We use use UCD for deploying fairly complex topologies with eight or more different machines involved.
Hi, were you able to make UCVD work for the Datastage and Reporting related deployment?
I am looking for the deployment in BI area involving Informatica, Cognos, Datastage.
Nathan, I’ve trying to create the environment mentioned above by you and need your help understanding how to get the test results reflect in RTC’s build result. This is what I have done so far:
1. Create an ANT build definition
2. Source is fetched, WAR is generated
3. The WAR is deployed in UCD using Post-Deploy option
What do I need to do from here? Secondly, I’ve notice another thing is that if the deployment in UCD fails the build result in RTC reflects as successful then indicating post-deploy failure. Why is that?
Thanks.
Hi Aamir. Once your WAR is deployed, you can start running tests. I think the reason that the the build result is still green even when the post-build deploy fails is because the post-build step only requests the deploy and does not wait or watch to see what happens. If that functionality is important to you, I would encourage you to create an enhancement request against RTC build.