In our pipeline, we use Rational Quality Manager (RQM) to run a lot of tests using dynamically created RQM adapters. While building the products, we also compile and package the tests to be run. After the tests and products to test are built, we create and start several RQM adapters to run the tests. Here’s how we create the adapters:

1. We create a new VM in the cloud—this takes 5–10 minutes (we are requesting about 100 VMs at the same time so it takes longer than it otherwise would).
2. We then install the test dependencies such as the test framework and the specific versions of web browsers to be used.
3. Next we install the correct version of the packaged tests (we want to use the tests aligned with the software under test).
4. Finally we install and run the RQM adapter (in our case the JUnit Selenium adapter).

It takes 10–20 minutes for the process to run and we invoke it many times in parallel.  When it is done, we have about 15 new VMs each with an RQM adapter listening and ready.  We then start the tests and, after the tests have been completed, we throw away the VMs.

Enter Docker

We modified the build of our tests so that along with compiling and packaging, the build also publishes a Docker image and pushes it to an internal registry. The Dockerfile does a pretty good job of showing what goes into the Docker image:

FROM debian:8.0
Maintainer Nathan Bak

# Install Java, utilities to extract archives, and display packages
RUN     apt-get update && apt-get install -y bzip2 openjdk-7-jre unzip vim wget x11vnc xvfb xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic && rm -rf /var/lib/apt/lists/*

# Install RQM Adapter
ENV        RQM_VERSION 6.0.0.0-RC1-RQMI20150511_1003
RUN        wget some.server.ibm.com:8125/rqm/RQM-Extras-JUnitSeleniumAdapter_$RQM_VERSION.zip \
 && mkdir /rqm && unzip RQM-Extras-JUnitSeleniumAdapter_$RQM_VERSION.zip -d /rqm \
 && rm  RQM-Extras-JUnitSeleniumAdapter_$RQM_VERSION.zip && unzip /rqm/adapters/RQMJUnitSeleniumAdapter.zip -d /rqm \
 && rm /rqm/adapters/RQMJUnitSeleniumAdapter.zip && rmdir /rqm/adapters && chmod 755 /rqm/start.sh

# Install Firefox
ENV        FIREFOX_VERSION 31.4.0esr
RUN        wget http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/$FIREFOX_VERSION/linux-x86_64/en-US/firefox-$FIREFOX_VERSION.tar.bz2 \
 && tar -jxvf firefox-$FIREFOX_VERSION.tar.bz2 -C / && rm firefox-$FIREFOX_VERSION.tar.bz2

# Install Selenium
ENV        SELENIUM_VERSION 2.45
ENV        SELENIUM_FULL_VERSION 2.45.0
RUN        wget http://selenium-release.storage.googleapis.com/$SELENIUM_VERSION/selenium-java-$SELENIUM_FULL_VERSION.zip \
 && unzip selenium-java-$SELENIUM_FULL_VERSION.zip -d / && mv /selenium-$SELENIUM_FULL_VERSION /selenium \
 && rm selenium-java-$SELENIUM_FULL_VERSION.zip && echo "" >> /rqm/selenium.properties \
 && echo com.ibm.rqm.selenium.seleniumClasspath=/selenium:/selenium/libs >> /rqm/selenium.properties

# Display properties
#          GEOMETRY SCREEN_WIDTH x SCREEN_HEIGHT x SCREEN_DEPTH
ENV        GEOMETRY 1200x900x24
ENV        DISPLAY :0
EXPOSE     5900

# Install run script
COPY          /run.sh /rqm/
RUN           chmod 755 /rqm/run.sh
ENTRYPOINT    [ "/rqm/run.sh" ]

# Install CSPF Test Asset
RUN        mkdir /media/ibm
COPY       /TestAssets/ /media/ibm/TestAssets
RUN        echo "" >> /media/ibm/TestAssets/bootstrap.properties \
 && echo browserPath=/firefox/firefox >> /media/ibm/TestAssets/bootstrap.properties \
 && echo "" >> /rqm/selenium.properties \
 && echo com.ibm.rqm.junit.classpath=/media/ibm/TestAssets/org.junit_4.8.1.v4_8_1_v20100427-1100:/media/ibm/TestAssets >> /rqm/selenium.properties

The run.sh script used as the entry point simply creates a display and VNC session and then starts up the adapter. Here are the contents of the script:

#! /bin/sh
cd /rqm
Xvfb $DISPLAY -screen 0 $GEOMETRY &
x11vnc -display $DISPLAY &
cd /rqm
./start.sh -repository $RQM_URL -user $RQM_USER -password $RQM_PASS -adapterName `hostname -f` -projectArea "$RQM_PROJECT"

Thanks to the magic of Docker layers and the fact that typically only the packaged tests change, building and pushing the image typically adds only a couple of minutes to the build—sometimes a bit more when the version of the RQM adapter or Selenium changes, but since this happens in parallel with the product code build, the extra time doesn’t matter.

As soon as we want to run the tests, the process of creating a new adapter now looks like this:

Since everything we need was packaged up in the Docker image, we simply start as many containers as we need adapters—this takes less than a minute on average.  After the tests run, cleanup is simply removing the container.

People are always impressed with the fact that using Docker allowed us to go from 10–20 minutes to about a minute, but I believe there are other benefits surpassing the performance improvement:

• Better reliability—running a container has fewer points of failure than creating a VM and running setup automation.
• Fewer resources needed—instead of creating a VM for each RQM adapter, we can spin up multiple Docker containers on the same machine.
• Empowered developers—the test developers have full control over the the environment in which the tests run. Changing the Firefox or Selenium version is as simple as changing a line in the Dockerfile.

Making the transition to using Docker to run our RQM adapters didn’t take too much effort and we quickly began to benefit from that effort. In future posts I’ll go into more detail about how the builds automatically create Docker images and how we designed our internal Docker registry.

Nathan Bak
DevOps Adoption Architect, Master Inventor
IBM Rational