Jazz Library Getting Started with Jazz Code Coverage
Author name

Getting Started with Jazz Code Coverage

Summary

The Code Coverage component provides an easy and efficient way to measure the test coverage while your projects evolve. A tight integration into the Jazz workbench together with an easy-to-use reporting mechanism help any developer to quickly get test coverage results without spending too much time with the code coverage tooling. With the optional build step to produce test coverage results during a Jazz build, the Jazz Code Coverage tooling is even easier to use, and test coverage is measured automatically in Jazz builds without any user intervention nor configuration on the Jazz client.


Introduction

The Jazz Code Coverage component integrates code coverage measuring into the team’s build process and provides an easy-to-use interface to get code coverage results, both on the server (while producing builds of your code) and on the client (while launching a Java application). This document is divided into three parts: Part one explains how to set up a Jazz build to produce test coverage results, part two discusses how test coverage results can be obtained from locally running JUnit tests, and part three explains how code coverage results are best browsed in the Jazz user interface.

Prerequisites

We strongly recommend to first read the following documents: This document assumes that a Jazz server has been set up and you have access to it. Furthermore, it is assumed that you have successfully set up a Jazz build using an example build script from the above-mentioned article.

To be able to view coverage reports on the Jazz client, you need to install the JazzCodeCoverage-IXXXXXXXX-XXXX.zip download available from the Jazz Project Incubators list.

Part 1: Configuring a Jazz build to produce test coverage results

Getting test coverage results from a Jazz build is straight-forward. However, depending on the environment on the build server and already existing build scripts, there is no generic way to augment a build script to produce Jazz code coverage results. This is the reason why the Jazz Code Coverage tooling only offers a set of customizable ANT tasks that can be used to enhance existing ANT build scripts to perform test coverage measurements.

The process of computing test coverage results in a Jazz build happens in five steps:

  • compiling: the source code is compiled to a set of output directories.

  • instrumenting: the output from the compile step (Java .class files or JAR files) is instrumented with probes to collect coverage information when executed later. This step can be controlled by coverage filters, a mechanism to define the scope when collecting coverage information or generating coverage reports. This produces a set of meta-data files which are kept for coverage report generation.

  • testing: the build drives the tests based on the instrumented code, while collecting coverage information. The build scripts passed various additional arguments to the virtual machine running the tests, such as the boot class path entry to the coverage engine runtime and paths arguments defining the locations where to store coverage information after the tests terminate.

  • reporting: a coverage report is generated based on the collected meta-data and coverage data.

  • publishing: the created coverage report is published to the Jazz build result.

For each one of the steps outlined above, Jazz offers a corresponding ANT task if the functionality is not already provided by ANT itself.

The Jazz Code Coverage tooling comes with the following set of ANT tasks to implement the instrumenting and reporting step:

  <project>  	<taskdef name="instr" classname="com.vladium.emma.instr.instrTask" />  	<taskdef name="report" classname="com.vladium.emma.report.reportTask" />  	<taskdef name="coverage" classname="com.ibm.team.coverage.common.coverageTask" />  	<taskdef name="analysis" classname="com.ibm.team.coverage.common.analysisTask" />  </project>  
These ANT tasks are provided by the following JARs:
  • emma.jar
  • emma_ant.jar
  • com.ibm.team.coverage.common_0.6.0.IXXXXXXXX-XXXX.jar
Please refer to the Documentation section for further information about those ANT tasks. The file CoverageToolkitTaskDefs.xml in the Jazz build toolkit directory provides ready-to-use task definitions to be included into your build scripts.

Moreover, the Jazz Code Coverage tooling contributes an example build script which is based on a simple compile and test scenario, similar to the compile-and-test example build script in the examples subdirectory of the Jazz build toolkit directory. You can use the compile-and-test-coverage example build script as the basis to create your own Jazz build script, or use parts of the script to integrate test coverage into your existing ANT build script.

The compile-and-test-coverage example build script is basically an adoption of the compile-and-test build script with appropriate calls to the ANT tasks provided by the Jazz Code Coverage tooling. It features four new targets, init, instrumentCode, createReports and publishReports, plus some additional properties to control various aspects of the code coverage report generation.

Existing build scripts may use those targets and insert them at the appropriate locations, according to the five steps necessary to run tests using code coverage measurements. Please make sure that init is called before other code coverage ANT targets are invoked.

Part 2: Launching Applications to measure code coverage

Launching a Java application such as a JUnit test is easy. To find out how well your code is covered, you just need to follow three steps:

Instrument Projects

  • Go to the project properties of your project (Alt+Enter), browse to the Coverage tab, and enable static code instrumentation.

    Coverage Property Page

  • Alternatively, if you would like to instrument multiple projects at once, you might also use the Run > Coverage > Instrument Projects... action.

Run Applications

Run one or more Java applications to drive the code you would like to measure. The Jazz Coverage tooling not only works with JUnit tests, it works with all Java-based launches supported by Eclipse as well!

Obtain Results

There are two ways how to obtain coverage results from a launched application: The label decorations in Java-based views such as the Package Explorer and the Outline view, or a customized coverage report based on an analysis over several Java projects and application launches.

  • Observe the coverage decorations which are shown in the Package Explorer or the Outline view after every application launch:

    Coverage Decorations in the Package Explorer

  • Perform a coverage analysis to produce a customized coverage report: Choose Run > Coverage > Run Analysis... from the workbench menu and select the Java projects and application launches you would like to have included in your report:

    Run Coverage Analysis Dialog

  • Press Run and browse the created coverage report:

    Browsing the Coverage Report

    On the second page of the coverage report viewer, you will find some overall statistics about the coverage report:

    Coverage Report Statistics

The coverage report viewer offers rich sorting and grouping functionality. Java elements can be grouped by Jazz component (if available), Java package or Java type, and each column can be sorted to quickly spot the candidates for creating new test cases or improving existing ones. If you open the associated source files, Jazz will ask you to update any coverage information which is out-of-date.

Part 3: Browsing code coverage results

Once you have created a test coverage report or you are browsing a Jazz build result, you can start exploring the coverage report and see which parts of your code base need improvements with respect to test coverage:

Browsing Coverage Reports

Use the actions Show in Java Editor and Show in Package Explorer to navigate to the source code in your local workspace. If you are currently browsing a build result with code coverage information, Jazz will ask if the coverage information has not been downloaded or the locally available coverage information is out-of-date. In this case, you can let Jazz automatically update your local Eclipse workspace.

Downloading Results

If you have a Jazz build result with associated coverage information, you may want to start digging into your code to see how the test coverage of some classes could be improved. Fortunately, there is no need to run all tests again locally to get this information for the local Eclipse workspace.

  • In the Coverage tab of a build result, choose the action Download to Workspace.

This action “projects” the coverage information of the build result onto your local Eclipse workspace, giving you the feeling as if all tests of the build have been run locally.

  • Open the Package Explorer and browse the Java projects which have been compiled for the current build result. Note how the Java elements are annotated with coverage information from the build result.

  • Open a source file from the Package Explorer which is annotated with coverage information. Note how the Java Editor displays a ruler on the left side with line coverage information, which is constantly updated while you edit the source file.

The presentation of the coverage information in the Package Explorer and the Java Editor may be configured on the preference page Java > Coverage.

  • Once you are done exploring the coverage of your code in the local workspace, you may choose the action Run > Coverage > Clean Decorations... to get rid of all coverage information in your local Eclipse workspace.

Coverage Filters

Coverage filters are a means to limit the information overhead normally produced by a full coverage report covering several projects. They provide an easy way to specify which Java elements should participate in code coverage via inclusion/exclusion filters.

To specify coverage filters on a Java project, go to the project preferences of your project (Alt+Enter), browse to the Coverage tab, and edit the coverage filters.


Coverage Filters

On the Coverage Filter dialog, you can specify the scope and type of the filter. Coverage filters may be specified in the popular syntax known from the Open Type (Ctrl+Shirt+T) command.


Coverage Filter Dialog

The syntax follows the familiar *,?-wildcard class name patterns and has the following format:

  • [package.]type, the optional package name in dot notation, followed by the type-qualified name of the type
  • * matches any string
  • ? matches any character

Active Launch

The concept of an active launch is used in the Jazz user interface to specify the data source for coverage information such as the Package Explorer decorations and the line-level coverage decorations in the Java editor.

Every time a launch involving an instrumented project has terminated, the Jazz coverage tooling records the code coverage measured by that particular launch, and the decorations in Package Explorer and Java editor are updated accordingly.

There are however situations in the code-test-fix cycle where you would like to have this information stable, meaning that it does not change every time a launch has terminated. This is the reason for an active launch, which can be configured to always represent the most recently terminated one, or a particular launch from the launch history which does not necessarily have to be the most recent one.


Active Launch and Launch History

Keep the checkbox Use most recently launched as active launch checked if you would like to always have the most recent coverage information displayed. Otherwise, uncheck the checkbox, select the desired launch, and press Set active.

Examples

Build script examples with optional test coverage support can be found in the /jazz/buildsystem/buildtoolkit/examples folder of TeamConcert-IXXXXXXXX-XXXX.zip. A build script that enables coverage for the JUnit Sample project is shown here:

Documentation

Detailed documentation of the Jazz Ant tasks can be found at /jazz/buildsystem/buildtoolkit/task-javadoc/index.html in TeamConcert-IXXXXXXXX-XXXX.zip.

Sat, 01 Dec 2007