What is the best way to be confident our baselines are correct with JazzMon
We support between 20 - 30 RTC projects within IBM. Some of the projects have over 500 users with SCM and others are smaller projects with 50 users only using Planning.
I've been working with a subset of those projects in JazzMon and have recorded some baselines over a 24 hour period. My concern now is how can I be fairly confident that these baselines represent a "healthy" repository?
For example, apt.internal.common.rcp.llterationPlanService.fetchPlannedWorkItems has a baseline of 2.5s on one of our projects. How can I reasonably tell this is in a good range and not supposed to be under 1s in a good environment?
I understand it depends a lot on the environment, # of users, etc. but wondering if there could be a good "range" we should expect for these web services? I see in some documents like the Server Tuning doc that there are some estimates suggested for a few, but curious about the others.
I've been working with a subset of those projects in JazzMon and have recorded some baselines over a 24 hour period. My concern now is how can I be fairly confident that these baselines represent a "healthy" repository?
For example, apt.internal.common.rcp.llterationPlanService.fetchPlannedWorkItems has a baseline of 2.5s on one of our projects. How can I reasonably tell this is in a good range and not supposed to be under 1s in a good environment?
I understand it depends a lot on the environment, # of users, etc. but wondering if there could be a good "range" we should expect for these web services? I see in some documents like the Server Tuning doc that there are some estimates suggested for a few, but curious about the others.
Accepted answer
At this time we don't have absolute references to what is "healthy" for any given web service on a server. The use of reference baselines like snapshots of jazz.net is a starting point to provide some comparison to an actively used and closely watched example server but the numbers can change day by day with shifting usage patterns. Seeing what stands out most dramatically can suggest where the pain points are.
For example, drilling in on something like fetchPlannedWorkitems or other Plan web services may show much larger volumes of data in the web service results (bytesSent) that somewhat explains longer response times. If users consider the overall response time for plans too slow it may be worth looking at why their plans return dramatically larger results. Sometimes, very large plans reflect local practices that include entire backlogs within plans, something that might be reconsidered if the performance tradeoff is unacceptable.
For example, drilling in on something like fetchPlannedWorkitems or other Plan web services may show much larger volumes of data in the web service results (bytesSent) that somewhat explains longer response times. If users consider the overall response time for plans too slow it may be worth looking at why their plans return dramatically larger results. Sometimes, very large plans reflect local practices that include entire backlogs within plans, something that might be reconsidered if the performance tradeoff is unacceptable.
2 other answers
and we also see affects on avg response time of large files on the IVersionedContentGet web service.. 14 seconds of file data even at wan speeds is large. (normal around 50ms) and really messes up the rest of the stats.
Comments