Error during JBE build connecting to repository
Hi,
Every so often we get the following error in out build log for one of our JBE build engines: CRRTC3524W: Repository connection failed: CRJAZ0099I HTTP error "Too many open files" accessing https://alpine.rtp.raleigh.ibm.com:9443/jazz/service/com.ibm.team.repository.common.internal.IRepositoryRemoteService. Does the error "Too many open files" apply to the JBE build engine process, or the RTC repository? I have been restarting the JBE build engine process which gets around it for a short time, but it keeps coming back, is this the right solution, can anyone suggest how I might get around this? JBE is running on linux. Alasdair |
8 answers
You may be running into a known problem with the IBM JRE, whereby it leaks pipe handles over time. Are you running RTC 1.0 or 2.0? The 2.0 RTC Eclipse client includes a patched JDK with a fix for this leak. Try running with the -vm argument pointing to the java executable:
jbe -vm {path_to_2.0_client}/jazz/client/eclipse/jdk/java -repository ... For more details, see: https://jazz.net/wiki/bin/view/Main/BuildFAQ#WhichJDK |
To determine whether this is the problem, try using lsof, e.g.
lsof -p {jbe_process-id} | grep pipe | wc If all is well, the first number should be relatively small, i.e. less than 50, even if actively processing a build. |
I'm using RTC 1.0. Having run the command you suggest I have 260 and 136 lines, which is much higher than 50, and no build is currently running.
I recently updating to using a Java 6 JVM, for running JBE, and I think that is when I started seeing the problem. If this is a JDK problem then perhaps I just need to upgrade to a more recent service pack. Thanks Alasdair |
The fix for the leak isn't in a service release yet. The JDK included in the RTC 2.0 Eclipse client is a patched version of the IBM 1.5.0 SR9 JDK, with the fix for the pipe leak defect, plus a few other fixes for other JIT and class library defects.
I highly recommend using this JDK to run the build engine. |
The fix for the leak isn't in a service release yet. The JDK included in the RTC 2.0 Eclipse client is a patched version of the IBM 1.5.0 SR9 JDK, with the fix for the pipe leak defect, plus a few other fixes for other JIT and class library defects. I need to use a Java 6 JDK for our build engine. I cannot use Java 5. I have upgraded to a later version of Java 6 JDK and so far I have not hit the problem again. I will keep an eye on it and see how things go. Thanks Alasdair |
Which Java 6 JDK are you using, Sun or IBM's? The Sun one does not have the pipe leak problem, but as far as I know, the current IBM still does.
|
Which Java 6 JDK are you using, Sun or IBM's? The Sun one does not have the pipe leak problem, but as far as I know, the current IBM still does. I'm using the IBM one (SR5), I haven't seen the problem so far, but who knows. |
We got the same error:
CRRTC3524W: Repository connection failed: CRJAZ0099I HTTP error "Too many open files" accessing URL "https://intranet.secservizi.sec:443/jazz/service/com.ibm.team.repository.common.internal.IRepositoryRemoteService.
Even our JBE process was running on a Linux host with Java 6:
java version "1.6.0_21"
The lsof command showed a thousand open pipes:
$ lsof -p PID | grep pipe | wc -l 2246A restart of the process solved the issue. We are on RTC v2.0.0.2 iFix6. |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.