Jazz-based servers allow you to record response files to be used when calling repotools setup from the command line.  The set of properties needed to go through setup on the command line successfully aren’t particularly easy to figure out, as they aren’t in product documentation.  Customers could use an example that shows them how to use the record response file function of the repotools command line utility to divine what properties to use, so they can put a jazz-based server in a Docker container and have express setup run automagically.  We hope to help others understand the challenges we faced when using jazz-based applications with Docker, so their learning curve is hopefully shorter.

Some acronyms we’ll use…

• RTC – Rational Team Concert
• CLM – Collaborative Lifecycle Management
• IoT CE (formerly known as SSE) Solution – Internet of Things Continuous Engineering, aka CE

Prerequisites:

1. Any Jazz-based product that has repotools_jts.sh (This could be full CLM, CE, or some subset… for our purposes, we will use CE 6.0.2)
2. A place for your automation to go to for the recorded response file.  We have a location in RTC source control where we store this.
3. Some practical knowledge of Docker, preferably enough to understand volumes, images, containers, and the way Docker handles networking.  See references below or at Docker.
4. Consider the scope of the automation you’re trying to accomplish.  In our implementation, an Urbancode Deploy application process initiates the building and running of Docker images/containers, copies a custom application war that runs server side with the JTS, copies the actual CE zip into the Docker container, and some other things.  Our automation basically stops at the point where you have setup run, a non-default admin user, but no Client Access Licenses (CALs) applied or projects created yet.  We hope to automate more in the future, but this gets us past a lot of the manual drudgery of setting up a CE Docker container with our custom app inside.  This article intentionally leaves out all of the Urbancode Deploy steps, and instead focuses on the Docker and repotools pieces of the puzzle.

Tools:

Docker: for creating a portable, reproducible environment in which we can run jazz-based products with a minimal amount of fuss.

Rational Team Concert (RTC): for builds and source control

Goals:

Learn how to create a response file for your jazz-based product install and deal with some of the potential snags along the way.

Assemble a Docker, jazz-based image that has had setup run on it and is as close to ready as possible to be used as a test environment.  Some of the notable pieces that help us accomplish this are listed below.

• A data volume for the jazz-based product. In our example, CE.  We will map the data volume to /opt/IBM/CE602
• We need a Dockerfile that copies files, sets permissions (say, on java or keytool), environment variables, and runs CE via supervisord, so we’ll need to configure that, too.
• Understand how to record a response file for repotools setup and how this can be run against the container via the cmd line on a Docker host.

Steps:

We’re going to satisfy both the high level goals above; first, by creating a Dockerfile and making sure we can build and run a container that has our jazz-based server running via supervisord; second, by learning how to work with the running container to generate our response file (if necessary) and run repotools_jts setup against it.  We’ll also talk a little about how this can fit into a continuous deployment story.

•  Generate the response file for repotools to use; this can be done on any machine or in the Docker image we build later.  For example, if CE 6.0.2 is installed in /opt/IBM/CE602, you will cd to the server folder there and run repotools_jts.sh setup responseFile=<your file>.  You will need to answer all questions, even if the default matches what you want, don’t simply hit enter.  In that case, just copy/paste the default and hit enter, otherwise the response may not be recorded.  If you get stuck while trying to record your responses because the data warehouse steps won’t validate, you can enter one of the arrow keys and hit enter, which will cause it to ignore validation, at least on my machine. 

• The following command(s) can be used to record the response file from within an interactive bash session in the running CE container.  Getting the generated file out of the container, as well as what you can do with it, is also described.  If you’re not using a Docker container to run repotools, ignore the Docker only steps (1 and 6) and do 2-5 on the command line.
  1. docker exec -it ce-<version> /bin/bash
  2. From the container:  cd /opt/IBM/CE602/server
  3. Make sure the properties file doesn’t already exist, then from the container:  /opt/IBM/CE602/server/repotools-jts.sh -setup responseFile=/path/to/data/repotools_setup.properties
  4. From the container, run through all the setup steps.  The response file will only be generated if you finish, so if you get stuck, see above.
  5. From the container, assuming you have the file generated, check it to make sure it isn’t empty.  The formatting isn’t the greatest; I like to organize mine after generation.
  6. Get out of the container and do docker inspect <data volume>.  The output will have a path in var you can go to and grab the file.  At this point, I put the file under source control in RTC, so it can be used by automation later.
• The following command is a one liner that will run repotool_jts setup within a running container, ce-<version>, using a properties file that we put in /usr/src/app.  This is useful is you don’t want an interactive bash session, like described above.  One reason why you might want to do this would be if you had an UrbanCode Deploy process that you wanted to use for automating the setup of a Jazz server test environment.

docker exec -i ce-<version> sh -c “cd /opt/IBM/CE602/server/; /opt/IBM/CE602/server/repotools-jts.sh -setup parametersFile=/usr/src/app/repotools_setup.properties”

• Here’s a link to a sample recorded properties file.  Please note that this is for convenience.  If you are somehow able to take this file and feed it through a different (most likely newer) version of CE, consider it a gift from the computer gods.  I know it works on CE 6.0.2, but the only way to know for sure is to record on the version you’re deploying to.
• Once setup has finished and you have had a chance to look at the file that gets generated, put it in source control, or some other place your automation will look for it.  As mentioned earlier, we throw ours into a Rational Team Concert source repository.
• Test that the response file will work by cleaning up your test machine and putting CE back on cleanly.  Usually I just build a fresh Docker container and rerun repotools with parametersFile instead of responseFile.  Another facet of this to consider is that this can be triggered every time a build happens, so when I commit a change to my properties file in source control, a build will occur, and usually I can see my Jazz server ready for customizations, about 20 minutes after releasing the change.
  1. Create and customize a Dockerfile for the container, when building… here’s a sample Dockerfile.  In the example below, you can see that we massage some of the permissions in our java installation, set some environment variables, define some ports to expose, copy a superivisord config, and invoke supervisord when the container starts.  For our purposes, we throw an IBM JDK into /usr/src/app, but you can set your container up with whatever java you prefer.  Also note that this image is not slimmed down and optimized for space.  I’m basing the image below on debian jessie, mostly because I like to pare down my images after I’ve worked with them awhile and have something more general purpose at the beginning.  You can see this in the fact that I’ve installed vim, as I frequently need to edit files in a running container during development.

     FROM debian:jessie
     RUN apt-get update -qq \
     && apt-get -yq install openssh-client \
     && apt-get --no-install-recommends install -qq wget > /dev/null \
     && apt-get install -yq --no-install-recommends pwgen ca-certificates \
     && apt-get -yq install unzip supervisor perl vim\
     && apt-get clean \
     && rm -rf /var/lib/apt/lists/*
     
     COPY . /usr/src/app
     RUN chmod +x /usr/src/app/ibm-java71/bin/java
     RUN chmod +x /usr/src/app/ibm-java71/jre/bin/java
     RUN chmod +x /usr/src/app/ibm-java71/jre/bin/keytool
     
     ENV JAVA_HOME /usr/src/app/ibm-java71
     ENV PATH $JAVA_HOME/bin:$PATH
     
     RUN mkdir -p /info
     COPY Dockerfile /info/
     
     EXPOSE 9080
     EXPOSE 9443
     COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
     CMD ["/usr/bin/supervisord", "-n"]

  2. Create a data volume:  docker create -v /sandboxdata –name data-dv-<version> httpd:2.4.  Again, note the arbitrary usage of the httpd 2.4 image.  This can be done with smaller base images.  This data volume is where we will put all the non-CE files we need, like our supervisord.conf, among other things.
  3. Create a data volume for CE (or the jazz-based product name): docker create -v /opt/IBM/CE602 –name ce-dv-<version> httpd:2.4.  The same comment above about image size applies here.  This data volume is exclusively for us to drop our jazz server (in this case, CE 6.0.2) into.
  4. Fill the data volume created above with the contents of a directory.  Note: “ibmjava7”, used below, is the name of a docker image on my host. You can use whatever image you like here. Before running this command, also make sure you’re in the folder you want everything copied from.    When you’re sitting in the right directory, run the following command: tar cf – . | docker run –rm -i –volumes-from data-dv<version> ibmjava7 tar xvf – -C /data
  5. Similarly, fill the CE data volume with the contents of the unzipped CE directory:  tar cf – . | docker run –rm -i –volumes-from ce-dv-<version> ibmjava7 tar xvf – -C /opt/IBM/CE602
  6. Build the docker container, by running this command in the folder with the Dockerfile you wrote.  docker build -t ce:<version> ./
  7. Running the container: docker run -d –volumes-from data-dv-<version> –volumes-from ce-dv-<version>  -p 9080:9080 -p 9443:9443 –name ce-<version> ce:<version>
     • ​When it’s time to restart the container for the server, you can do a docker ps -a and use the ID for your container in a subsequent docker restart <id> command. 
  8. If you encounter issues with setup on the command line…  Problems can happen in any piece of software;  running jazz setup via the command line is no exception to that rule.  Here are some things to consider.
     • Sometimes simply rerunning the repotools command is enough to get back on track.
     • One thing I do is leave ADMIN enabled, then disable it later, after setup has had a chance to run.  This is because if you let it, it will be disabled automatically by setup, but later, if you have a setup error and have to rerun, the ADMIN account won’t work, and you will have to add adminUserId and adminPassword to the repotools_jts command if you want to proceed.
     • While recording, sometimes the applications have issues during warehouse db setup.  Setup on the command line seems to do this in a slightly different order than the setup on the web.  For example, you might find that entering the default derby info for some product results in it telling you that it has a validation problem.  In order to proceed, sometimes I have to push one of the arrow keys and hit enter.
     • The option “includeLifecycleProjectStep” should be usable, but we’ve had issues with it and currently do not set it.

References:

• https://jazz.net/wiki/pub/Main/ScriptableSetupForConsumers/response_properties.txt – sample response properties currently on wiki