It's all about the answers!

Ask a question

RTC 2.0 performance


Mike Rolls (12631813) | asked Aug 25 '09, 4:24 a.m.
We recently upgraded to RTC v2.0 and have found a significant degradation in performance.
We have noticed that in the jazz.log there are numerous reports of:
2009-08-24 10:00:38,182 ERROR t.jazz.ajax.service.internal.util.RootServicesUtil - Error while fetching and parsing the root services document

Has anyone encountered similar errors and have any suggestions as to whether we have mis-configured our service.

Thanks

4 answers



permanent link
Mike Rolls (12631813) | answered Aug 25 '09, 4:27 a.m.
Additional info, from the tomcat log:
SEVERE: Servlet.service() for servlet equinoxbridgeservlet threw exception
java.lang.IllegalStateException
at org.apache.catalina.connector.ResponseFacade.sendError(ResponseFacade.java:419)
at net.jazz.ajax.internal.javascript.WebBundleServlet.writeBundleResource(WebBundleServlet.java:328)
at net.jazz.ajax.internal.javascript.WebBundleServlet.service(WebBundleServlet.java:106)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:67)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:121)
at com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:54)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:810)

permanent link
Jean-Michel Lemieux (2.5k11) | answered Aug 26 '09, 8:57 a.m.
JAZZ DEVELOPER
What DB are you using? Can you post the raw fetch times as shown in the
counters we maintain on the server, you can find those at

https://YOURSERVER:9443/jazz/service/com.ibm.team.repository.service.internal.counters.ICounterContentService/

Also, what DB and application server are you running? What operations do
you find are slower?

We haven't found a noticeable performance problem in 2.0.

Jean-Michel

On 8/25/2009 4:38 AM, adcasupp wrote:
We recently upgraded to RTC v2.0 and have found a significant
degradation in performance.
We have noticed that in the jazz.log there are numerous reports of:
2009-08-24 10:00:38,182 ERROR
t.jazz.ajax.service.internal.util.RootServicesUtil - Error while
fetching and parsing the root services document

Has anyone encountered similar errors and have any suggestions as to
whether we have mis-configured our service.

Thanks

permanent link
Mike Rolls (12631813) | answered Aug 26 '09, 1:44 p.m.
Jean-Michel

The DB server is an Oracle 10.2.0.1 instance running on the same windows 2003 (64bit) server as the jazz instance (not using WAS just jazz / apache).
General usability is slow, do you have an idea what is causing the ERROR t.jazz.ajax.service.internal.util.RootServicesUtil - Error while fetching and parsing the root services document
errors in the jazz.log.

Is it a mis-configuration on my part?


Counter detail:
com.ibm.team.repository.service.internal.counters.contentservice.CounterContentServicecaching
counter hits misses
valmaxmin valmaxmin
feed entry cache 4184180 1261260
totals 4184180 1261260

Query Result Fetching
counter Rows Fetched Later Rows In First Page Rows Never Fetched Rows Not
Yet Fetched
cntavgmaxmindevtot cntavgmaxmindevtot cntavgmaxmindevtot
cntavgmaxmindevtot
Data Queries 162525250400 1,92522300155,700 1,230357022544,208
6435702272,272
Item Queries 66222601501,454 8,68311000415,969 417209669 009900
Text Queries 000000 000000 000000 000000
totals 82162600 1,854 10,60812300 21,669 1,23417700 44,277 6412700 2,272

resource usage
counter usage (pct)
cntavgmaxmindevtot
JDBC connection pool 388620.083500.3873241
RDB mediator pool 387970.083400.3863226
totals 776590.08350 6467

asynchronous tasks
counter elapsed time (seconds) queued count
cntavgmaxmindevtot valmaxmin
BuildResultPrunerTask 140.0310.1560.0150.0370.438 010
BuildSchedulerTask 2130.010.2500.0232.062 010
ItemCleanupTask 120.0120.03100.010.14 010
LicenseSnapshotTask 31.5944.5160.0622.5324.781 010
LogChangeEventAppenderTask 3360.0110.2500.0223.601 010
OutgoingSyncTask 4200000 010
ProcessChangeEvents 5120.0090.14100.0144.594 010
com.ibm.team.scm.monitorScrub 25600.0032.26500.0457.324 010
com.ibm.team.workitem.service.save.postnotification
160.1540.62400.1592.467 010
notification.mail 210.321.6090.0160.4026.716 010
updateLinksTask 30.0050.01600.0090.016 010
versionedContentCleanup 70.1050.1560.0940.0230.736 010
totals 37390.1884.5160.094 32.875 010

reporting
counter workitems.MicroBurndown
valmaxmin
report display 220
totals 220

web service
counter 1. elapsed time (seconds) 2. bytes sent to client 3. bytes
received from client
cntavgmaxmindevtot cntavgmaxmindevtot cntavgmaxmindevtot
com.ibm.team.apt.common.resource.IResourcePlanningService.findWorkLocation()
40.020.0310.0150.0080.078 457557557502300 475175175103004
com.ibm.team.build.internal.common.ITeamBuildService.getBuildResultRecordsForBuildResults()
10.110.110.1100.11 137113711371103711 18838838830883
com.ibm.team.dashboard.common.internal.service.IDashboardImageService.GET()
210.0030.06300.0140.063 000000 000000
com.ibm.team.dashboard.common.internal.service.IDashboardRestService.getDashboard()
40.0550.18800.0890.219 000000 000000
com.ibm.team.dashboard.common.internal.service.IDashboardRestService.getDefaultDashboardHref()
40.0550.1410.0160.0590.22 000000 000000
com.ibm.team.dashboard.common.internal.service.IDashboardRestService.getLeftNavInfo()
40.1060.1880.0470.060.424 000000 000000
com.ibm.team.dashboard.common.internal.service.IDashboardRestService.getViewletDefinitions()
190.1760.87500.3483.342 000000 000000
com.ibm.team.dashboard.viewlets.service.internal.members.IMemberInfoService.getMemberInfo()
20.0860.1090.0630.0330.172 000000 000000
com.ibm.team.dashboard.viewlets.service.internal.members.IMemberPhotoService.GET()
200000 000000 000000
com.ibm.team.dashboard.viewlets.service.internal.teams.ITeamsService.getTeamsForProjectArea()
20.0160.03100.0220.031 000000 000000
com.ibm.team.filesystem.common.IFileContentService.PUT()
500.7072.87500.77235.326 000000 506120.18101878202835.544306009
com.ibm.team.filesystem.common.IFilesystemService.compareWorkspaces()
30.8282.3440.0471.3132.485 313112107913689.3563933 318651865186505595
com.ibm.team.filesystem.common.IFilesystemService.getComponentStateSummaries()
80.0020.01600.0060.016 853753753704296 812131213121309704
com.ibm.team.filesystem.common.IFilesystemService.getFileTreeByVersionable()
20.070.0930.0470.0330.14 215031557144976.3683006 221472147214704294
com.ibm.team.filesystem.common.IFilesystemService.interpretChanges()
20.2260.2340.2190.0110.453 21642.520621223593.2633285
218951957183387.6813790
com.ibm.team.filesystem.common.workitems.internal.IFilesystemWorkItemService.createLink()
10.1090.1090.10900.109 18908908900890 116451645164501645
com.ibm.team.links.common.internal.IRemoteLinkService.findAuditableLinksByEndpoint()
330.040.90600.1561.313 331073.7581609356378.10335434 33887887887029271
com.ibm.team.process.internal.common.service.IProcessRestService.getAllProjectAreas()
130.0190.0320.0150.0070.251 000000 000000
com.ibm.team.process.internal.common.service.IProcessRestService.getProcessAreaHierarchies()
60.0110.01600.0080.064 000000 000000
com.ibm.team.process.internal.common.service.IProcessService.findProcessAreas()
10.0150.0150.01500.015 144404440444004440 116791679167901679
com.ibm.team.process.internal.common.service.IProcessService.getCurrentIterationsInfo()
100000 18348348340834 17817817810781
com.ibm.team.reports.common.internal.service.IReportRestService.postRenderQuery()
26.71910.7972.6415.76713.438 000000 000000
com.ibm.team.reports.service.internal.IReportsContentService.get() 200000
22591388812941834.2355182 000000
com.ibm.team.repository.common.ILicenseAdminService.getServerLicenseType()
30.0050.01600.0090.016 3669.66768264521.3622009 345945945901377
com.ibm.team.repository.common.ILicenseService.checkLicenses() 100000
13433433430343 19139139130913
com.ibm.team.repository.common.internal.IFeedService.GET()
4010.07814.17200.70831.368 000000 000000
com.ibm.team.repository.common.internal.IItemRestService.GET()
160.1850.8590.0620.2122.953 000000 000000
com.ibm.team.repository.common.internal.IRepositoryRemoteService.describe()
400000 41260126312582.165040 446046046001840
com.ibm.team.repository.common.internal.IRepositoryRemoteService.fetchDynamicPackages()
20.0860.1570.0160.10.173 211421142114202284 264364364301286
com.ibm.team.repository.common.internal.IRepositoryRemoteService.fetchHandlesByLocation()
130.0060.01600.0080.078 1347858335560.6856214 13781.6151136669141.46710161
com.ibm.team.repository.common.internal.IRepositoryRemoteService.fetchOrRefreshItems()
2410.0361.42200.1458.762 2413714.56791614988987.195895209
2411904.216341008533461.795458916
com.ibm.team.repository.common.internal.IRepositoryRemoteService.fetchStates()
100000 16416416410641 111191119111901119
com.ibm.team.repository.common.internal.IRepositoryRestService.GET()
1000000 000000 000000
com.ibm.team.repository.common.service.IPermissionService.assertPermission()
170.0010.01500.0040.015 171247.1181308320239.00221201 17616616616010472
com.ibm.team.repository.common.service.IQueryService.queryDataInContext()
110.0210.04700.0160.235 111772.90928534781239.39819502
113014.81837342416687.9933163
com.ibm.team.repository.common.service.IQueryService.queryItems() 100000
15635635630563 110441044104401044
com.ibm.team.repository.common.service.IQueryService.queryItemsInContext()
90.1610.70300.2251.452 9961.5561517526376.9528654
916627.55627997236813304.154149648
com.ibm.team.repository.common.transport.IDirectWritingContentService.get()
27.43812.9681.9077.82114.875 21826281826281826280365256 000000
com.ibm.team.repository.service.IItemRenderService.GET()
200.0160.17200.0380.312 000000 000000
com.ibm.team.repository.service.internal.IAdminRestService.getAllRoles()
7100.01600.0030.032 000000 000000
com.ibm.team.repository.service.internal.IAdminRestService.getContributorByUUID()
710.0010.01600.0040.062 000000 000000
com.ibm.team.repository.service.internal.IAdminRestService.getContributors()
630.0220.17200.031.376 000000 000000
com.ibm.team.repository.service.internal.IAdminRestService.getUserRegistryInfo()
7600.01600.0020.031 000000 000000
com.ibm.team.repository.service.internal.IAdminRestService.postContributor()
290.0430.1720.0160.0321.251 000000 000000
com.ibm.team.repository.service.internal.ILicenseAdminRestService.getAssignedLicenses()
700.0470.21900.043.316 000000 000000
com.ibm.team.repository.service.internal.ILicenseAdminRestService.getLicenseInfo()
1500.0780.2350.0160.04411.715 000000 000000
com.ibm.team.repository.service.internal.ILicenseAdminRestService.getUsedLicenseInfo()
370.0020.01600.0050.076 000000 000000
com.ibm.team.repository.service.internal.ILicenseAdminRestService.postLicenseInfo()
150.2940.5470.0150.1934.405 000000 000000
com.ibm.team.repository.service.internal.ILicenseCheckerRestService.getCheckLicense()
570.0010.01600.0040.047 000000 000000
com.ibm.team.repository.service.internal.IProductRegistryRestService.getAllProductInfo()
2100000 000000 000000
com.ibm.team.repository.service.internal.IServerConfigurationRestService.getServiceConfiguration()
50.0060.01600.0090.032 000000 000000
com.ibm.team.repository.service.internal.IServerConfigurationRestService.postConfigurationUpdates()
30.0940.15600.0830.281 000000 000000
com.ibm.team.repository.service.internal.IServerConfigurationRestService.postConfigurationValidation()
1360.0330.45400.0864.472 000000 000000
com.ibm.team.repository.service.internal.IServerStatusRestService.getComponents()
210.0210.15600.0310.439 000000 000000
com.ibm.team.repository.service.internal.IServerStatusRestService.getComponentsWithServiceConfig()
20.0620.0780.0470.0220.125 000000 000000
com.ibm.team.repository.service.internal.IServerStatusRestService.getDatabaseStatus()
210.0040.01600.0070.092 000000 000000
com.ibm.team.repository.service.internal.IServerStatusRestService.getServerInfo()
210.0010.01600.0030.016 000000 000000
com.ibm.team.repository.service.internal.IServerStatusRestService.postStatusForServices()
56700.01600.0030.235 000000 000000
com.ibm.team.repository.service.internal.counters.ICounterContentService.GET()
000000 000000 000000
com.ibm.team.repository.service.internal.discovery.IFriendsAdminRestService.getFriends()
100000 000000 000000
com.ibm.team.repository.service.internal.discovery.IRootDiscoveryService.GET()
40.0080.01600.0090.031 000000 000000
com.ibm.team.repository.service.internal.license.IFloatingLicenseService.GET()
7560.0080.15600.0166.408 000000 000000
com.ibm.team.repository.service.internal.license.ILicenseUploadService.POST()
50.0970.1720.0620.0440.484 000000 000000
com.ibm.team.repository.service.internal.oauth.IOAuthRestService.getConsumers()
10.0160.0160.01600.016 000000 000000
com.ibm.team.repository.service.internal.oauth.IOAuthRestService.getProvisionalRegistrations()
100000 000000 000000
com.ibm.team.repository.service.internal.webuiInitializer.IWebUIInitializerRestService.GET()
150.3031.0150.1560.2164.543 000000 000000
com.ibm.team.scm.common.IScmService.batchCommit()
70.3530.7970.0160.2552.469 71946.85724881845238.73613628
710870.2861496722105120.63376092
com.ibm.team.scm.common.IScmService.configurationBatchFetch()
40.5242.0470.0151.0162.094 4692.56936920.5772770 419031903190307612
com.ibm.team.scm.common.IScmService.createChangeSet()
10.2190.2190.21900.219 115371537153701537 123512351235102351
com.ibm.team.scm.common.IScmService.fetchStates() 10.0160.0160.01600.016
112751275127501275 116491649164901649
com.ibm.team.scm.common.IScmService.getChangeSetLinkSummaries()
110.0880.78100.2310.967 11584.27377641887.0866427
111521.72719111319188.58416739
com.ibm.team.scm.common.IScmService.getContributorState()
10.1250.1250.12500.125 14434434430443 112941294129401294
com.ibm.team.scm.common.IScmService.refreshWorkspaces()
151.0822.9840.0931.12416.235 152132.73329851471571.49331991
157543.61687912196927.885113154
com.ibm.team.scm.common.IScmService.setComment() 10.1090.1090.10900.109
19099099090909 114861486148601486
com.ibm.team.scm.common.IScmService.workspaceFindVersionablesByName()
30.3540.7340.1090.3341.062 324822482248207446 314861486148604458
com.ibm.team.scm.common.IVersionedContentService.GET()
13108.449606.6960.016184.7071409.842
1315876056.46213242803918436401919.954206388734 000000
com.ibm.team.scm.common.internal.IScmQueryService.findLocks()
10.9850.9850.98500.985 13863863860386 113601360136001360
com.ibm.team.scm.common.internal.IScmQueryService.findWorkspaces()
40.0620.1720.0150.0740.25 4534.5605371111.3242138 415041504150406016
com.ibm.team.workitem.common.internal.IQueryRepositoryService.fetchExportDescriptors()
20.0080.01500.0110.015 24974974970994 24714714710942
com.ibm.team.workitem.common.internal.IQueryRepositoryService.fetchMore()
1000000 102371.946597801176.11723719 10921.69229210.5169216
com.ibm.team.workitem.common.internal.IQueryRepositoryService.fetchQueryResults()
40.1210.2810.0310.1120.483 488094284247.73520 44029.7547192914784.2716119
com.ibm.team.workitem.common.internal.IQueryRepositoryService.findPersonalQueries()
100000 18598598590859 112661266126601266
com.ibm.team.workitem.common.internal.IWorkItemRepositoryService.computeHistory()
200.3980.8290.2180.1477.97 204144.8620429421174.11282896 20723723723014460
com.ibm.team.workitem.common.internal.IWorkItemRepositoryService.fetchNewer()
990.0030.01600.0060.3 99334334334033066 99970.3031236885151.07196060
com.ibm.team.workitem.common.internal.IWorkItemRepositoryService.findAvailableWorkItemTypes()
50.0310.0470.0150.0110.156 545345345302265 597497497404870
com.ibm.team.workitem.common.internal.IWorkItemRepositoryService.save()
110.310.4690.0940.1193.406 116078.909814142071292.67866868
1120486.8183006683506550.753225355
com.ibm.team.workitem.common.internal.model.IImageContentService.get()
120.0060.01600.0080.078 12388.83358555163.2434666 000000
com.ibm.team.workitem.common.internal.rest.IQueryEditorRestService.getAttributeTree()
20.070.0930.0470.0330.14 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryEditorRestService.getEditableQuery()
70.0050.01600.0080.032 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryEditorRestService.getPresentation()
300000 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryEditorRestService.postSaveQuery()
10.0780.0780.07800.078 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryRestService.getHistogram()
20.0550.0630.0470.0110.11 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryRestService.getQueries()
200.1511.50.0460.3223.015 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryRestService.getQueriesForOwner()
40.0160.03100.0130.062 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryRestService.getQuery()
150.0070.03100.010.109 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryRestService.getResults()
60.0370.15700.0590.22 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryRestService.getScopedQueries()
30.0670.0930.0310.0320.202 000000 000000
com.ibm.team.workitem.common.internal.rest.IQueryRestService.postGetResultSet()
300.1251.4690.0150.323.762 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getAllValues()
30.020.0310.0150.0090.061 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getContributorMailConfigDTO()
10.0160.0160.01600.016 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getEditorPresentation()
30.0310.0630.0150.0270.094 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getLinkTypes()
30.0050.01600.0090.016 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getQuickInformationPresentation()
30.0050.01500.0090.015 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getTime()
300000 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getValues()
190.010.03100.0090.188 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getWorkItemDTO2()
110.1170.3440.0150.0931.283 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.getWorkItemEditableProperties()
100.1170.250.0780.0511.172 000000 000000
com.ibm.team.workitem.common.internal.rest.IWorkItemRestService.postWorkItem2()
20.3520.50.2030.210.703 000000 000000
com.ibm.team.workitem.service.internal.rest.IAttachmentRestService.POST()
30.6821.6090.0630.8182.047 000000 000000
totals 35381.227606.6960.015 1618.099 589147885.4661324280390 208070764
6101011.555341000 1637093

Query Result Caching
counter Bytes Cached Now Bytes Ever Cached
cntavgmaxmindevtot cntavgmaxmindevtot
Data Queries 645,55515,0552,1382,712355,520
1,9254,06315,05503,7977,822,708
Item Queries 113,78750,29813,787013,787 8,68367136,86201,784583,881
Text Queries 000000 000000
totals 656,44750,2980 369,307 10,6081,377136,8620 8,406,589

Query Processing
counter Duration Parameter Count Total Rows
cntavgmaxmindevtot cntavgmaxmindevtot cntavgmaxmindevtot
Data Queries 1,92542,2660589,074 1,925137073,265 1,9252723003052,580
Item Queries 8,683190601214,077 8,6831310213,437 8,683273801017,492
Text Queries 000000 000000 000000
totals 10,60822,2660 23,151 10,6081370 16,702 10,608107380 70,072

Query Summary
counter Miss
valmaxmin
Page 000
totals 000

Query Result Sharing
counter Duplicate Time (in ms) Total Processed
valmaxmin cntavgmaxmindevtot valmaxmin
Query Result Sharing 61204612040 10,6080160095 70072700720
totals 61204612040 10,6080160 95 70072700720


Thanks
Mike

permanent link
Micky Holmes (61) | answered Jul 26 '10, 2:20 p.m.
Let's say a change set has been suspended and the user that suspended it has left the project. Can the resume a suspended change set?

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.