In my last post, I described how we use IBM UrbanCode Deploy to quickly and easily deploy server applications produced by each build.  In this post I will describe how we are using those deployed applications to improve our automated testing and reduce the time it takes to validate the contents of a build.

Quality is important in our software and we do a lot of testing to make sure the code behaves as expected.  In our builds we run unit tests to verify that blocks of code perform as expected.  We also run tests to make sure that different blocks integrate and play nicely together.  These tests do a great job of making sure the pieces can do their part, but we also need a way to ensure that the system as a whole, the functionality customers actually use, works as designed.  As software has become more distributed, the environments in which they are deployed have grown in complexity.

We have a build that produces two servers which provide different but interrelated services.  In the “real” world, these servers are installed and configured so that they are aware of each other.  Various third-party applications connect to one or the other of the servers (but usually not both) and can interact by posting and getting requests.  While this interaction is occurring, the two servers communicate and work together.  The process of validating this scenario is basically:

1. Install server 1 on one machine
2. Install server 2 on a different machine
3. Configure server 2 to talk with server 1
4. Install a third-party application on yet another machine
5. Configure third-party application to talk with server 2
6. Run tests with third-party application
7. Verify that all interactions with server 2 are correct
8. Verify that server 1 is in the expected state

When done manually, this is a time consuming and error prone task even when the tests themselves are automated.  This may be acceptable for infrequent releases where a test team runs through the steps to verify “blessed” builds, but it simply doesn’t scale to the continuous delivery paradigm.  Typically we run daily “real” builds, but sometimes dozens of personal builds are run by developers over the course of a global work day.  We use IBM UrbanCode Deploy to automatically set up the environment so that we can run our tests as part of every build.  The test results are reported back to the build result in Rational Team Concert, so when the build is done we know that a “good” build means not only did the compilation and packaging succeed, but also the functionality has been validated at unit, integration, and system levels.

Each build sets up an environment to do the testing that looks something like this:

My Rapid build deployment using IBM UrbanCode Deploy post describes how we are able to deploy the servers from the build process–that gets server 1 and server 2 deployed in the cloud.  Since UrbanCode Deploy deployed both servers, it knows where they are and has a step that configures things so that the servers know about each other.  The third-party app is easily brought up on the build machine and is told how to find server 2.  The tests are then invoked and run against server 2 and, after verifying that works correctly, it is also possible for additional tests to validate the state of server 1.

Along with speeding up the testing process, we have observed several other benefits of this approach:

• Better test coverage because tests run against real server in a realistic environment.
• Build setup and running of tests is much simpler—just point the tests to the server and run them.
• Fewer differences between build and developer test execution.
• Developers can debug tests against the same server used by the build (or deploy a server with an identical configuration).
• Less overhead on build machines because servers are deployed on separate machines.
• Tests can be run against multiple, complex environments to provide better coverage.

I was only introduced to UrbanCode Deploy earlier this year, but I have been very pleased with how easy it was to integrate it into our process and provide value quickly.  Like most processes, I expect that there will be numerous iterative changes and improvements to what we have in place, and I expect that UrbanCode Deploy usage will increase as we work to support continuous delivery and improve our automated test coverage.

Nathan Bak
Software Engineer, Master Inventor
IBM Rational