From Techworld.com – How not to deploy an application over a WAN, “Although the medical appointment application was advertised as being client/server based, it wasn’t. In particular, the application was designed in such a way that the data in each field was sent character by character over the network. Hence, this was another example of an application that runs well over a LAN but presents performance problems over a WAN.”
We really wanted to avoid this with Rational Team Concert so we built a utility called the Jazz metronome to keep us honest. We wanted to make it easy to debug the client and find out when the server was being contacted (i.e. round-trips) and how long each server request took, so that we could easily see how chatty the client was being and fix the problem areas.
The Jazz metronome was previously part of the Source Control developer tools plug-in and wasn’t part of the Rational Team Concert client. But it turned out to be so useful that starting in the next milestone build following Beta 2 (M5, planned for late February), we’ll be including the tool in the milestone downloads so that our customers and developers can easily use it. You can enable the Jazz metronome from the Team > Jazz Source Control preference page which will enable a trim widget that shows the global round-trip count, then you can open the detail window from the trim widget to get service by service breakdown as shown in the video at the end of this entry.
The information displayed by the metronome tool is fairly low-level, it tracks which service and methods are called, the cumulative elapsed time in each request, and the slowest time. What’s also cool is that you can reset the counters and run something from the UI and see how chatty a particular feature is. With the service class grouping you can generally differentiate between work items, builds, process, or scm by looking at the service names.
We’re interested in your feedback and potential defects. To help you understand the information here are some usage hints:
- If it ain’t broke… if everything is running just fine then you can skip the rest of this blog entry and you never have to use the tool.
- However, if you are stuck in a hotel with spotty connectivity and find the client sluggish, open the metronome window and take a peek at what is happening.
- Turn off all background updating jobs in Team Central. Most of the query sections have a Resfresh option in their drop-down menu, while the build section has a configuration parameter for how often to contact the server. This will both help isolate who is being chatty and help reduce the round-trips while your connection is slow.
- Open the metronome tool and re-run the action which you find is running too slow. Take note of the service calls. Since there are several factors that affect WAN performance, for example amount of data being sent or number of round-trips, latency, it’s best to simply take notes and open a bug report.
- If you can, include information in the bug report about your connection characteristics, ping time to the server, your connection speed, and don’t forget to copy and paste the tree shown in the metronome window directly into the bug report. This can be done by selecting the rows in the tree and clicking the Copy action in the bottom right.
Reminder: The Jazz metronome tool is only available in Rational Team Concert milestone builds after Beta 2.
There are some service calls which are shared between all components. ITeamContentService is used to fetch and upload all content to and from the server. This method will be chatty when you load a repository workspace or upload work item attachements, so it will be more difficult to blame a particular component for being chatty. In many cases it’s ok to hit the server several times since we pipeline some calls and re-use the underlying http connection. This is why it’s best to interpret the information based on slowness of some operations and not only based on the round-trips in daily usage.
Please, open metronome and start measuring away when things feel slow over a WAN. If the numbers look fishy, open a bug report (don’t forget to look for duplicates first)! There are probably some embarrassingly chatty actions lurking out there, but we’d rather know about them sooner rather than later so that we can fix them. Let’s all work to make Rational Team Concert WAN friendly.
(Update for Team Concert 1.0 – June 2008)
We’ve added another feature to the metronome tool which helps measure your connection speed to a Jazz server. This can help decide how responsive the client will be given the bandwidth and latency between the client and the server.
—
Jean-Michel Lemieux
Jazz Source Control Team
This is a short 1:47 minute video showing the tool in action:
I love your blog, but every morning when I check my RSS feeds I’m disappointed to find that nothing new has been posted. Please pick up the pace. Jazz rocks!
Thanks John, there are more on the way soon. Stay tuned…
Nice utility. If this is still supported only with RTC, any plans of extending it to RQM?
Regarding comment 3 I don’t think so because the Eclipse client will be more or less completely be removed with CLM 2012 revision 4.0.1 from RQM. Metronome is part of the Eclipse client. What might be of interest in this content is “JazzMon – Seeing what your server is up to” this can be found under https://jazz.net/library/article/822.
Can this data be collected on the server and summarized to help find bottleneck locations/times? There is a similar feature for the web client dashboard. We’re getting load errors: Error while writing to path/file : failed to read entire content and have to restart. We also tried pathping, tracert, traceroute command line tools, and visual traceroute website. Increasing the RTC time out (default 8 minutes) didn’t help. It fails and has to be manually restarted and is just as slow as CCRC7. When will a new version of the RTC client be out? iFix008 has RTC 5.0.2 using Eclipse 4.2 we’re using Eclipse 4.4.2 Luna EE with JDK8 OSGi for development.