It's all about the answers!

Ask a question

Error during JBE build connecting to repository


Alasdair Nottingham (611610) | asked Jun 25 '09, 8:31 a.m.
Hi,

Every so often we get the following error in out build log for one of our JBE build engines:

CRRTC3524W: Repository connection failed: CRJAZ0099I HTTP error "Too many open files" accessing https://alpine.rtp.raleigh.ibm.com:9443/jazz/service/com.ibm.team.repository.common.internal.IRepositoryRemoteService.


Does the error "Too many open files" apply to the JBE build engine process, or the RTC repository? I have been restarting the JBE build engine process which gets around it for a short time, but it keeps coming back, is this the right solution, can anyone suggest how I might get around this? JBE is running on linux.

Alasdair

8 answers



permanent link
Nick Edgar (6.5k711) | answered Jun 30 '09, 3:59 p.m.
JAZZ DEVELOPER
You may be running into a known problem with the IBM JRE, whereby it leaks pipe handles over time. Are you running RTC 1.0 or 2.0? The 2.0 RTC Eclipse client includes a patched JDK with a fix for this leak. Try running with the -vm argument pointing to the java executable:

jbe -vm {path_to_2.0_client}/jazz/client/eclipse/jdk/java -repository ...

For more details, see:
https://jazz.net/wiki/bin/view/Main/BuildFAQ#WhichJDK

permanent link
Nick Edgar (6.5k711) | answered Jun 30 '09, 4:06 p.m.
JAZZ DEVELOPER
To determine whether this is the problem, try using lsof, e.g.
lsof -p {jbe_process-id} | grep pipe | wc

If all is well, the first number should be relatively small, i.e. less than 50, even if actively processing a build.

permanent link
Alasdair Nottingham (611610) | answered Jun 30 '09, 11:28 p.m.
I'm using RTC 1.0. Having run the command you suggest I have 260 and 136 lines, which is much higher than 50, and no build is currently running.

I recently updating to using a Java 6 JVM, for running JBE, and I think that is when I started seeing the problem. If this is a JDK problem then perhaps I just need to upgrade to a more recent service pack.

Thanks
Alasdair

permanent link
Nick Edgar (6.5k711) | answered Jul 06 '09, 11:57 a.m.
JAZZ DEVELOPER
The fix for the leak isn't in a service release yet. The JDK included in the RTC 2.0 Eclipse client is a patched version of the IBM 1.5.0 SR9 JDK, with the fix for the pipe leak defect, plus a few other fixes for other JIT and class library defects.

I highly recommend using this JDK to run the build engine.

permanent link
Alasdair Nottingham (611610) | answered Jul 07 '09, 4:15 a.m.
The fix for the leak isn't in a service release yet. The JDK included in the RTC 2.0 Eclipse client is a patched version of the IBM 1.5.0 SR9 JDK, with the fix for the pipe leak defect, plus a few other fixes for other JIT and class library defects.

I highly recommend using this JDK to run the build engine.


I need to use a Java 6 JDK for our build engine. I cannot use Java 5. I have upgraded to a later version of Java 6 JDK and so far I have not hit the problem again. I will keep an eye on it and see how things go.

Thanks
Alasdair

permanent link
Nick Edgar (6.5k711) | answered Jul 07 '09, 3:24 p.m.
JAZZ DEVELOPER
Which Java 6 JDK are you using, Sun or IBM's? The Sun one does not have the pipe leak problem, but as far as I know, the current IBM still does.

permanent link
Alasdair Nottingham (611610) | answered Jul 08 '09, 4:57 a.m.
Which Java 6 JDK are you using, Sun or IBM's? The Sun one does not have the pipe leak problem, but as far as I know, the current IBM still does.


I'm using the IBM one (SR5), I haven't seen the problem so far, but who knows.

permanent link
SEC Servizi (97123860) | answered Nov 13 '14, 10:51 a.m.
edited Nov 13 '14, 10:53 a.m.
We got the same error:
CRRTC3524W: Repository connection failed: CRJAZ0099I HTTP error "Too many open files" accessing URL "https://intranet.secservizi.sec:443/jazz/service/com.ibm.team.repository.common.internal.IRepositoryRemoteService.
	
	
	
Even our JBE process was running on a Linux host with Java 6:
java version "1.6.0_21"
Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
Java HotSpot(TM) Server VM (build 17.0-b16, mixed mode)
The lsof command showed a thousand open pipes:
$ lsof -p PID | grep pipe | wc -l
2246
A restart of the process solved the issue. We are on RTC v2.0.0.2 iFix6.

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.