If there’s one question that we ask ourselves frequently in software development, it’s “how are we doing?”. All stakeholders, from first-line developers to project managers, from executives to customers, have a keen interest in the progress. The development process, especially on a larger team, produces a lot of information about work items, source control, builds and planning which, to the trained eye, conveys crucial tidbits about the trends, successes and failures of the team or the entire project. Acting on this information, making appropriate course corrections where necessary, can help ensure the future success of a project – the challenging bit is actually distilling this wisdom from the avalanche of data. Here’s where the Jazz Team Reports component, included in Rational Team Concert, can help.

The Team Reports component consists of two parts, which I’ll mention briefly to give you a feel for how we collect and present the data you see in reports. The first of these, the data warehouse, is a storage facility for aggregated, historical data. Automated snapshot tasks collect information periodically from the Jazz repository about various artifacts, including work items, change-sets and build results, and store it in the data warehouse. By doing this, we have access to historical information that wouldn’t be possible by simply querying the repository. The second part consists of the reporting engine, which visualizes a wide range of reports in the web UI, in the dashboard, and in the rich client, using information from the data warehouse as a data source.

With Team Concert beta 3 upon us, let’s use some of these reports to try to understand our progress since beta 2.

In the several milestones between beta 2 and beta 3, we closed more than 9000 work items. During that time, the backlog of planned open items seemed to range from about 1000 to close to 2000, although it converged nicely towards the end of beta 3 development:

The previous chart is only part of the story, however, because it only shows those work items which are planned for one of the iterations. The defect backlog report gives us a better look at all of the open defects, whether they are planned for one of these iterations or not. This is important because even unplanned defects represent real problems and real work. This chart shows generally steady fixing, followed by upward spikes in the backlog. The upward spikes correspond with, respectively, our deployment weeks for M5D1, M5, M6 and RC1, when we test extensively. It’s also important to point out that our efforts to decrease the backlog in RC1 far exceeded any progress we had made during M5 or M6. However, the chart still reflects that there are more than 4000 open defects, so we still have work to do:

The rate of incoming work items, with each bar representing one day, reflects a few things about our regular rhythm. First, we saw a lull in the second half of December as many developers took vacation. Second, it shows an expected cycle each week with little work taking place on the weekends, and the majority of work items reported near the middle of the week, when our builds are usually ready for testing. Last, we see noticeable spikes in logged work items during the weeks corresponding with the end of each of our iterations (the four largest spikes are, chronologically, during M5D1, M5, M6 and RC1 deployment weeks). These patterns are reassuring because it shows work in the expected places and no surprises:

The activity in our Weekly Integration stream bears a bit of explanation. Until M6, continuous integration took place in a separate stream, and the weekly integration truly was “weekly” – that is, on Monday each team delivered that week’s contribution to the weekly stream, followed by patches and fixes as needed. Starting with M6, however, the Jazz development team decided that it would be better in the end game to do continuous integration only. At that point we started using the Weekly Integration stream for continuous integration, and deprecated the use of the other stream. These trends are brought out in the chart below – in the first half, we see big spikes of activity on Mondays followed by a lull; in the second half, we see more even activity in the weekly stream throughout the week.

There are many other reports we use frequently to track such things as our build progress, the size trends of our repository and data warehouse, and work item trends related to planning estimates. For those ones we refer to frequently, the integration into the dashboard is a great feature which gives us at-a-glance access to the information we need to make informed decisions about our project:

You can see these reports, and many more, on the reports page on Jazz.net, which displays information about our own self-hosting. This data shows the progress of the Jazz Project gleaned from our own instance of Rational Team Concert. You can also use all of these reports on your own project. See the Getting Started page for Reports for more information.

—

James Moody
Jazz Reports Team Lead