What would cause high CPU for RRC 4.0.5?
![]()
About 1 month after we upgraded our whole environment to 4.0.5 (from 4.0.1), we have been seeing almost 100% CPU usage on our RRC server. The CPU is being used by the RRC java process. Immediately after a restart of the application, CPU is low, but a couple hours afterwards the CPU goes and stays high. Checking the rm.log, we can only find one message around the time of the issue starting (see below). (we have seen the same message shortly after our recent restarts)
It would not appear to be something a user is doing, as the CPU stays consistently high even when there are no Licenses being used by the application. Memory usage on the system seems to be just over 50%. Any idea when may be causing this? How we can figure it out and fix it? 2014-04-05 01:22:46,719 [ RM-ClusterSafeInitializers-3] ERROR .recentfeeds.internal.EntryFactory$PopulateInitial - Exception occurred while getting the recent resources details com.ibm.rdm.fronting.server.exception.InternalServerErrorException: Propagated exception; original message [Read timed out] at com.ibm.rdm.fronting.server.exception.RRSException.newException(RRSException.java:220) at com.ibm.rdm.fronting.server.exception.RRSException.newException(RRSException.java:737) at com.ibm.rdm.fronting.server.exception.RRSException.capture(RRSException.java:741) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalServiceExchange.checkError(InternalServiceExchange.java:53) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalServiceExchange.getResponse(InternalServiceExchange.java:37) at com.ibm.rdm.fronting.server.core.jpi.internal.CoreServiceRequest.executeServiceRequest(CoreServiceRequest.java:63) at com.ibm.rdm.fronting.server.core.spi.BaseServiceRequest.execute(BaseServiceRequest.java:117) at com.ibm.rdm.fronting.server.services.query.internal.QueryServiceUtil.doQuerySparql(QueryServiceUtil.java:2009) at com.ibm.rdm.fronting.server.services.query.internal.QueryServiceUtil.querySparql(QueryServiceUtil.java:1830) at com.ibm.rdm.fronting.server.services.recentfeeds.internal.EntryFactory$PopulateInitial.run(EntryFactory.java:108) at com.ibm.rdm.fronting.server.services.recentfeeds.internal.ClusterSafeInitializer$InternalInit.call(ClusterSafeInitializer.java:48) at com.ibm.rdm.fronting.server.concurrent.RRSConcurrency$ForwardingListeningExecutorServiceExtension$1.call(RRSConcurrency.java:106) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:908) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:931) at java.lang.Thread.run(Thread.java:773) Caused by: com.ibm.rdm.fronting.server.exception.RRSException: Read timed out at com.ibm.rdm.fronting.server.core.jpi.internal.AbstractCoreService.sendHttpRequest(AbstractCoreService.java:115) at com.ibm.rdm.fronting.server.core.jpi.internal.AbstractCoreService.sendRequest(AbstractCoreService.java:81) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.invokeHttpRequest(InternalCoreServices.java:860) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleCoreBulkables(InternalCoreServices.java:790) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.handleCoreRequests(InternalCoreServices.java:726) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:310) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:242) at com.ibm.rdm.fronting.server.core.jpi.internal.InternalCoreServices.execute(InternalCoreServices.java:234) at com.ibm.rdm.fronting.server.core.jpi.internal.CoreServiceRequest.executeServiceRequest(CoreServiceRequest.java:61) ... 11 more Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:140) at com.ibm.jsse2.b.a(b.java:197) at com.ibm.jsse2.b.a(b.java:242) at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:750) at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:484) at com.ibm.jsse2.f.read(f.java:49) at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149) at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:110) at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264) at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252) at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281) at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247) at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219) at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) at java.lang.reflect.Method.invoke(Method.java:611) at com.ibm.rdm.fronting.server.core.jpi.internal.http.StatsConnectionManager$RequestLineInterceptor.invoke(StatsConnectionManager.java:286) at com.ibm.rdm.fronting.server.core.jpi.internal.http.$Proxy29.receiveResponseHeader(Unknown Source) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464) at com.ibm.rdm.fronting.server.core.jpi.internal.http.RRCJazzHttpClient$1.execute(RRCJazzHttpClient.java:223) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820) at com.ibm.team.jfs.app.http.client.JazzHttpClient.execute(JazzHttpClient.java:288) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:927) at com.ibm.rdm.fronting.server.core.jpi.internal.AbstractCoreService.sendHttpRequest(AbstractCoreService.java:105) ... 19 more Request --Not Provided-- Response --Not Provided-- |
One answer
![]()
I suggest you open a PMR to troubleshoot this issue as it probably requires debugging the environment.
|
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.
Comments
I agree with Hakan. My best guess would be the ETL queries which will likely run right after the server is started. Indexing is another area I would suggest investigating. But again, best to open the PMR and let IBM Support help identify the cause.
Thanks Both - I submitted a PMR this afternoon and am working with L2 to track this down.