RTC performance

Hi,

RTC 2.0.0.2 iFix2
JTS on Windows
Tomcat

RTC performance is degrading. Web performance is pathetic. On the RTC web client, even opening a WI takes hours. Any pointer on how to improve the performance?

Thanks!

Can you reboot the server? Running out of space or memory on the server perhaps?

Also - is your database server very close (on same network segment)? For some reason (and we heard of the same problems on other apps - nothing to do with RTC) - if your database server is far away from the app server, webUI performance can be poor.

anthony

Hi,

Thanks for the response. The DB server and the JTS are both on RHEL (Linux) and close by.
Running a query from within the web client is pretty fast and when I click on any WI from the result set it takes hours to open that WI. IS it the problem with the Oracle DB (indexing ....)?

Thanks!

If you go to the 'Active Services' page in the Admin web UI, it can tell you which services have active requests, and their current stack trace. You could use this to narrow down whether the operation is blocked in Jazz server code, or waiting on the DB.

The URL for a local repo is:
https://localhost:9443/jazz/admin#action=com.ibm.team.repository.admin.activeServices

It also would be good to know whether you see the same degradation when using the Eclipse UI.

- It works perfectly fine from Eclipse UI. So it means then that there should be no issues with the Oracle DB or the server hosting it or the Tomcat or the server hosting it.

- another observation is that it doesn't happen for all the work item types and happens only for certain work item types. For example, tasks WIs can be seen from the web UI but defects WIs fails to open.

- Below is the message output in the the 'Active Services' page in the Admin web UI when a WI is failing to open on the web UI.

com.ibm.team.workitem.service.internal.rest.WorkItemRestService.getWorkItemEditableProperties
by sunitad, 00:01:49:576 running time
org.apache.xerces.impl.XMLEntityScanner.scanContent(Unknown Source)
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanContent(Unknown Source)
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)
org.eclipse.emf.ecore.xmi.impl.XMLLoadImpl.load(XMLLoadImpl.java:181)
com.ibm.team.repository.common.internal.marshal.util.WebServicesXMLLoadImpl.load(WebServicesXMLLoadImpl.java:76)
org.eclipse.emf.ecore.xmi.impl.XMLResourceImpl.doLoad(XMLResourceImpl.java:180)
com.ibm.team.repository.common.internal.marshal.impl.EObjectMarshaller.demarshalInputStreamToObject(EObjectMarshaller.java:501)
com.ibm.team.repository.service.internal.dataaccess.DocumentMediatorService$5.processRow(DocumentMediatorService.java:910)
com.ibm.team.repository.service.internal.dataaccess.FetchRows.execute(FetchRows.java:254)
com.ibm.team.repository.service.internal.dataaccess.FetchRows.execute(FetchRows.java:200)
com.ibm.team.repository.service.internal.dataaccess.DocumentMediatorService.doFetchItemValues(DocumentMediatorService.java:941)
com.ibm.team.repository.service.internal.dataaccess.DocumentMediatorService.doReadItemValuesByBlocks(DocumentMediatorService.java:814)
com.ibm.team.repository.service.internal.dataaccess.DocumentMediatorService.doReadItemValuesByBlocks(DocumentMediatorService.java:784)
com.ibm.team.repository.service.internal.dataaccess.DocumentMediatorService.fetchMultipleByKeyId(DocumentMediatorService.java:581)
com.ibm.team.repository.service.internal.dataaccess.GenericDocumentMediatorService.fetchItems(GenericDocumentMediatorService.java:129)
com.ibm.team.repository.service.internal.rdb.TransactionalCurrentStateCache$CurrentCacheHelper.genericHandleFetch(TransactionalCurrentStateCache.java:141)
com.ibm.team.repository.service.internal.rdb.AbstractTransactionalItemCache$AbstractCacheHelper.computeValues(AbstractTransactionalItemCache.java:622)
com.ibm.team.repository.service.internal.rdb.AbstractTransactionalItemCache$AbstractCacheHelper.computeValues(AbstractTransactionalItemCache.java:1)
com.ibm.team.repository.service.internal.rdb.TransactionalCache$ComputeMultipleAndCache.compute(TransactionalCache.java:1956)
com.ibm.team.repository.service.internal.rdb.TransactionalCache.get(TransactionalCache.java:421)
com.ibm.team.repository.service.internal.rdb.AbstractTransactionalItemCache.innerGet(AbstractTransactionalItemCache.java:271)
com.ibm.team.repository.service.internal.rdb.AbstractTransactionalItemCache.get(AbstractTransactionalItemCache.java:243)
com.ibm.team.repository.service.internal.RdbRepositoryDataMediator.doFetchItems(RdbRepositoryDataMediator.java:1907)
com.ibm.team.repository.service.internal.RdbRepositoryDataMediator.fetchItems(RdbRepositoryDataMediator.java:1881)
com.ibm.team.repository.service.internal.RepositoryItemService.doFetchItems(RepositoryItemService.java:797)
com.ibm.team.repository.service.internal.RepositoryItemService.access$0(RepositoryItemService.java:777)
com.ibm.team.repository.service.internal.RepositoryItemService$7.run(RepositoryItemService.java:727)
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$Transaction.run(RepositoryDatabase.java:466)
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$1.run(RepositoryDatabase.java:292)
com.ibm.team.repository.service.internal.rdb.ConnectionPoolService.withCurrentConnection(ConnectionPoolService.java:331)
sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:618)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
$Proxy21.withCurrentConnection(Unknown Source)
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runTransaction(RepositoryDatabase.java:288)
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runInTransaction(RepositoryDatabase.java:244)
com.ibm.team.repository.service.internal.TransactionService.runInTransaction(TransactionService.java:79)
sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:618)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
$Proxy43.runInTransaction(Unknown Source)
com.ibm.team.repository.service.internal.RepositoryItemService.fetchItemsPermissionAware(RepositoryItemService.java:725)
com.ibm.team.repository.service.internal.RepositoryItemService.fetchItems(RepositoryItemService.java:666)
sun.reflect.GeneratedMethodAccessor116.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:618)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
$Proxy73.fetchItems(Unknown Source)
com.ibm.team.repository.service.internal.ComponentRepositoryItemService.fetchItems(ComponentRepositoryItemService.java:504)
com.ibm.team.workitem.service.internal.AuditableServer.resolveAuditables(AuditableServer.java:116)
com.ibm.team.workitem.service.internal.AuditableServer.findAuditables(AuditableServer.java:169)
com.ibm.team.workitem.common.internal.model.SetAttributeType.getValueSet(SetAttributeType.java:42)
com.ibm.team.workitem.common.internal.attributeValueProviders.FallbackProvider.getValueSet(FallbackProvider.java:35)
com.ibm.team.workitem.common.internal.model.impl.AttributeImpl.getValueSet(AttributeImpl.java:868)
com.ibm.team.workitem.service.internal.rest.WorkItemRestService.getValues(WorkItemRestService.java:1057)
com.ibm.team.workitem.service.internal.rest.WorkItemRestService.getAllValues(WorkItemRestService.java:964)
com.ibm.team.workitem.service.internal.rest.WorkItemRestService.getWorkItemEditableProperties(WorkItemRestService.java:909)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:618)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
$Proxy193.getWorkItemEditableProperties(Unknown Source)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:618)
com.ibm.team.repository.servlet.AbstractTeamServerServlet.doModelledRestService(AbstractTeamServerServlet.java:571)
com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest2(AbstractTeamServerServlet.java:1752)
com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest(AbstractTeamServerServlet.java:1623)
com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1536)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90)
org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)
org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:75)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:121)
com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:54)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
java.lang.Thread.run(Thread.java:811)