Java process utilizing 100% CPU in RTC 3.0.1.2
Running virtual guest OS Linux with 4 gb of ram, 4 cores, and enough disk space
I don't know for how long time this has been going on, but i have noticed that the Java process of
My Htop output looks like this.
I'm not that smart when it comes to administering the server, but i cannot see anything that resembles a looping faliure or the likes in the logs.
Are there anyone who's got a clue on what could be the cause?
-Sofus Albertsen
3 answers
Comments
com.ibm.team.repository.service.internal.ServerStatusRestService.getActiveServiceInfo
With this as the stacktrace i think:
java.lang.Thread.getStackTraceImpl(Native Method) java.lang.Thread.getStackTrace(Thread.java:1147) com.ibm.team.repository.service.internal.ServerStatusRestService.renderThreadStack(ServerStatusRestService.java:314) com.ibm.team.repository.service.internal.ServerStatusRestService.getActiveServiceInfo(ServerStatusRestService.java:304) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:618) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370) org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
There's more but im not allowed to post that much. Hope i helps
@clementliu: I think this would be better as added as a comment to the original question. These comments of yours are posted as "answers" which is a bit confusing if someone is looking for answers but instead see your comments.
@bkchulin, :) thanks. But looks like I can't upload screen shot to a comment so that's why sometimes I had to start a new conversation
1 vote
@clementliu: ah, good point. I've posted the question https://jazz.net/forum/questions/89178/is-it-possible-to-add-a-screenshot-when-posting-a-forum-comment hoping that if it is not possible, then it may be made possible.
Comments
00:00:00:000 running time so no, that's not it. if i do a ps -wwfC java i get the following: UID PID PPID C STIME TTY TIME CMD root 1529 1 99 11:18 ? 05:49:27 /opt/IBM/JazzTeamServer/server/jre/bin/java -Djava.util.logging.config.file=/opt/IBM/JazzTeamServer/server/tomcat/conf/logging.properties -Djazz.connector.sslProtocol=SSL_TLS -Djazz.connector.algorithm=IbmX509 -Djava.awt.headless=true -DSQLSERVER_JDBC=/opt/IBM/JazzTeamServer/server/sqlserver -DSQLSERVER_JDBC_DRIVER_FILE= -DORACLE_JDBC=/opt/IBM/JazzTeamServer/server/oracle -DORACLE_JDBC_DRIVER_FILE= -DDB2I_JDBC=/opt/IBM/JazzTeamServer/server/db2i -DDB2Z_JDBC=/opt/IBM/JazzTeamServer/server/db2z -Dorg.eclipse.emf.ecore.plugin.EcorePlugin.doNotLoadResourcesPlugin=true -DJAZZ_HOME=file:///opt/IBM/JazzTeamServer/server/conf -Dcom.ibm.team.repository.tempDir=/tmp -Xmx1200M -Dlog4j.configuration=file:///opt/IBM/JazzTeamServer/server/conf/startup_log4j.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/opt/IBM/JazzTeamServer/server/tomcat/common/endorsed -classpath /opt/IBM/JazzTeamServer/server/tomcat/bin/bootstrap.jar
And these 3 lines for the end of the process: -Dcatalina.base=/opt/IBM/JazzTeamServer/server/tomcat -Dcatalina.home=/opt/IBM/JazzTeamServer/server/tomcat -Djava.io.tmpdir=/opt/IBM/JazzTeamServer/server/tomcat/temp org.apache.catalina.startup.Bootstrap start
Comments
https://dl.dropbox.com/u/697024/Screenshot%20from%202012-09-26%2016%3A41%3A19.png
Hrm... getActiveServiceInfo is the only one? Then doesn't look like RTC server is busy with anything... wait... seems like you're looking at the JTS admin console... not the RTC admin console...
Can you actually take a screen shot of your RTC admin console?
The url should be:
https://hostname:port/ccm/admin
or
https://hostname:port/jazz/admin
Darn, i apologize for that one: Seems there acctually is another thread running
https://dl.dropbox.com/u/697024/rtc/Screenshot%20from%202012-09-26%2016%3A53%3A26.png https://dl.dropbox.com/u/697024/rtc/Screenshot%20from%202012-09-26%2016%3A53%3A33.png https://dl.dropbox.com/u/697024/rtc/Screenshot%20from%202012-09-26%2016%3A53%3A41.png
That thread is actually OK, only running for 7 secs and I bet it will disappear soon. Hrm... it doesn't look either JTS or RTC is busy... From your screen shot of Htop, I am wondering why there are so many Java processes running at the same time and each grabbing 34% of Memory??? I would suggest to submit a PMR to IBM support for them to dig into...Sorry...not a great help..
Could this be a garbage collection issue?
@Clement, I will try that, but thanks for your help anyway! @Ralph, i have tried to restart the server, and right after the initial startup, the process is running. if i refresh the active services, the com.ibm.team.repository.service.internal.QueryService.queryItems seems to be running exactly 14 seconds, and the starts all over again. I suspect somekind of deadlock?
I have created a PMR here.
Hope it helps.
Hi Sofus,
Were you able to fix the issue? What was the root cause and the resolution?