It's all about the answers!

Ask a question
QuestionsTagsUsersBadges
Follow this question
Question details

×42,790
×10,908
×5,999
×3,952

question asked: May 01 '13, 8:28 a.m.
question was seen: 5,677 times
last updated: May 10 '13, 6:09 a.m.

Related questions

FAQ - How this Forum works

My CLM applications are crashing again and again after every 6-7 hours.Have to restart the jts services to make applications up and running

0
pratibha gupta (15●1●11●15) | asked May 01 '13, 8:28 a.m.
the applications RTC, RRC and RQM are crashing after every 6-7 hours and need to restart the jts services again and again. We are using Apache tomcat as Application server
Comments
Rajat Singh commented May 01 '13, 9:07 a.m. | edited May 07 '13, 10:01 a.m.

Hi Pratibha


Could you describe more on this issue?
When you say crash, does it mean the tool pauses or the processes get killed?
Is there any specific pattern on the occurrence of the issue, apart from time frame of 6-7 hours (any specific task that leads to the crash).
What is the configuration of the machine on which CLM applications are installed?
Whats the OS?

Best Regards
Rajat

Matt Lennon commented May 01 '13, 9:44 a.m. | edited May 09 '13, 2:36 a.m.
JAZZ DEVELOPER

Hi Pratibha.

You should also check your CLM server logs
    <clm-server-install-dir/server/logs/{ccm,jts,qm}.log
and Tomcat logs for log entries corresponding to the time of the crashes.

What version of CLM are you running?

    -Matt

pratibha gupta commented May 02 '13, 1:29 a.m.

We are not able to log into the applications and have to restart the jts server . We are using RHEL 5.0 OS

pratibha gupta commented May 02 '13, 1:30 a.m.

I am using CLM 3.0.1.

Karl Weinert commented May 02 '13, 8:53 a.m. | edited May 09 '13, 2:35 a.m.
JAZZ DEVELOPER

How much memory do you have on your system?

Also check for javacore files somewhere in the path leading up to the directory where you are running java from.
I have not found a consistent place where they get created but here's a couple places to check:
/opt/IBM/JazzTeamServer/server
/opt/IBM/JazzTeamServer/server/jre/bin
They may even be in the tomcat directory structure.
/opt/IBM/JazzTeamServer/server/tomcat
/opt/IBM/JazzTeamServer/server/tomcat/bin
Or just search /opt/IBM/JazzTeamServer for "javacore*.txt"

If you find some that correspond to the time of the crashes the first 10 lines or so should provide more details on what caused the javacores to be created. Feel free to paste them in.




pratibha gupta commented May 07 '13, 9:00 a.m.

I checked on all the paths mentioned by you but could not find javacore*.txt files. Please let me know if there is any other path

1
Karl Weinert commented May 07 '13, 9:23 a.m.
JAZZ DEVELOPER

Hi Pratibha,

This is probably getting a little complex to troubleshoot via the forum.
It would be good to know how much memory your system has and how much of it is dedicated to the jvm

I would suggest opening a PMR but if you can't I would start by collecting WAIT tool output run for a period of time leading up to the time the system hangs. Start by running it every 15-30 minutes for an hour or two before the problem usually occurs.
This will output quite a bit of performance data that might help in figuring out what is going on.
You can download and read about the tool at https://wait.ibm.com/ 
I generally start looking at the javacores created by the wait tool with "IBM Thread and Monitor Dump Analyzer for Java" found here:
https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=2245aa39-fa5c-4475-b891-14c205f7333c

pratibha gupta commented May 07 '13, 9:26 a.m.

I got javacore*.txt but not able to understand these file. Plz check

0SECTION       ENVINFO subcomponent dump routine
NULL           =================================
1CIJAVAVERSION J2RE 5.0 IBM J9 2.3 Linux amd64-64 build j9vmxa6423ifx-20100510
1CIVMVERSION   VM build 20100509_57823_LHdSMr
1CIJITVERSION  JIT enabled - 20091016_1845ifx7_r8
1CIRUNNINGAS   Running as a standalone JVM
1CICMDLINE     /opt/ibm/JazzTeamServer/server/jre/bin/java -Djava.util.logging.config.file=/opt/ibm/JazzTeamServer/server/tomcat/conf/logging.properties -Djazz.connector.sslProtocol=SSL_TLS -Djazz.connector.algorithm=IbmX509 -Djava.awt.headless=true -DSQLSERVER_JDBC=/opt/ibm/JazzTeamServer/server/sqlserver -DSQLSERVER_JDBC_DRIVER_FILE= -DORACLE_JDBC=/opt/ibm/JazzTeamServer/server/oracle -DORACLE_JDBC_DRIVER_FILE=/opt/ibm/JazzTeamServer/server/oracle/ojdbc5.jar -DDB2I_JDBC=/opt/ibm/JazzTeamServer/server/db2i -DDB2Z_JDBC=/opt/ibm/JazzTeamServer/server/db2z -Dorg.eclipse.emf.ecore.plugin.EcorePlugin.doNotLoad

showing 5 of 8 show 3 more comments

Accepted answer

0
permanent link
Krzysztof Kaźmierczyk (7.4k●3●75●103) | answered May 10 '13, 6:09 a.m.
OK. The problem has been fixed. The possible root cause was high server load by multiple Tomcat processes running. Killing the old processes fixed the issue.
pratibha gupta selected this answer as the correct answer

2 other answers

Most liked answers ↑|Newest answers|Oldest answers

2
permanent link
Karl Weinert (2.0k●5●27●36) | answered May 07 '13, 10:20 a.m.
JAZZ DEVELOPER
Hi Pratibha,

I found your PMR and the javacores
It looks like you have 3 java processes running each using about 5 gigs of memory.
That is cutting it tight since it looks like your system has 16 gigs total memory.

One thing that might help the hangs would be setting the XMS value in the server.startup file

I am a bit concerned about having all three applications set to use 4 gigs of memory each as we don't want you to run out of system memory.
A resolution might require a bit of trial and error to get it right (unless you want to add system memory)
I would consider setting the QM and RM applications to use less, maybe set them to 3 gigs each and leave CCM at 4 gigs. Provided that CCM is the application that is under the heaviest use.
You would want to keep an eye on general performance after making that change.

Setting the XMS value:
It should be set to the same value as the XMX setting.
server.startup  will be in the following location:
/opt/IBM/JazzTeamServer/server
Currently you have:
set JAVA_OPTS=%JAVA_OPTS% -Xmx4000M

You will want to add this line after that one:
set JAVA_OPTS=%JAVA_OPTS% -Xms4000M

i.e.
set JAVA_OPTS=%JAVA_OPTS% -Xmx4000M
set JAVA_OPTS=%JAVA_OPTS% -Xms4000M
It will require a restart (or you can just wait till you are forced to restart)

One other thing I noticed in the PMR is that you restarted the database at some point and had trouble with RTC afterwards. If you restart the database it is advisable to restart RTC as well because there are usually stale connections to the database that need to be cleaned up.



0
permanent link
Luiz Almeida (230●1●55●89) | answered May 01 '13, 4:18 p.m.
 Pratibha, 
If you are using Firefox, maybe this can be the problem. In my case, Firefox are crashing due a problem with memory.


Comments
pratibha gupta commented May 02 '13, 1:30 a.m.

We are using internet explorer.

Your answer

Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.