It's all about the answers!

Ask a question

Urgent database and client issues in 2.0


Sterling Bates (2311612) | asked Aug 20 '09, 11:57 a.m.
We're having two issues that are potentially related shortly after our upgrade.

First, the 2.0 client will disconnect every so often (i.e. every ten to fifteen minutes or so) for many of the people on our team. We looked in the .metadata/.log file, but didn't see anything obviously related to a cause. Most of the developers are working through a firewall, but the setup was the same with our 1.0.1 installation and we had no issues like this.

Second, our DB2 server (not the same machine as the Jazz server) immediately floods with CPU-consuming threads when people log on. About half an hour ago I was forced to reboot the database server because it was completely non-responsive, and there were easily fifteen "db2sysc" instances all running at very high CPU rates. This is on an 8-way box so we have CPU to spare.

After bringing up the database and recycling the Jazz server, I had users log in to the system and immediately saw this:


4166 db2inst1 25 0 456m 33m 24m R 96.2 0.4 2:06.59 db2sysc
4149 db2inst1 25 0 456m 31m 22m R 93.6 0.4 2:07.70 db2sysc
4172 db2inst1 20 0 457m 32m 23m R 92.9 0.4 1:46.90 db2sysc
4161 db2inst1 25 0 457m 36m 27m R 91.6 0.5 1:05.89 db2sysc
4147 db2inst1 18 0 457m 36m 27m R 76.7 0.5 2:00.44 db2sysc
4160 db2inst1 18 0 456m 29m 21m D 75.3 0.4 2:17.81 db2sysc
4140 db2inst1 15 0 452m 18m 11m D 1.3 0.2 0:01.07 db2sysc
4176 db2inst1 15 0 456m 35m 26m S 1.3 0.4 0:01.76 db2sysc
4171 db2inst1 15 0 456m 33m 24m S 1.0 0.4 0:00.97 db2sysc
4175 db2inst1 15 0 488m 43m 31m S 1.0 0.5 0:01.95 db2sysc
4085 db2inst1 16 0 452m 11m 4068 S 0.7 0.1 0:00.50 db2sysc
4146 db2inst1 15 0 457m 37m 27m S 0.3 0.5 0:01.71 db2sysc
4168 db2inst1 15 0 456m 33m 24m S 0.3 0.4 0:00.87 db2sysc


I bolded the CPU utilization. These numbers never go down, and their growth could be related to the clients dropping and reconnecting all day.

Whether they're related or not, we're unable to get through even a half day of development like this.

2 answers



permanent link
Sterling Bates (2311612) | answered Aug 20 '09, 11:59 a.m.
When a developer gets logged off, the only message we see in .log file that *might* be related is the one below, but it could be a symptom of having been disconnected rather than being related to the disconnection itself:

!ENTRY org.eclipse.core.net 1 0 2009-08-20 11:35:33.383
!MESSAGE System property http.nonProxyHosts has been set to local|*.local|169.254/16|*.169.254/16 by an external source. This value will be overwritten using the values from the preferences
!ENTRY com.ibm.team.process.rcp.ui 4 0 2009-08-20 11:35:52.787
!MESSAGE Error logged from Process RCP UI:
!STACK 0
com.ibm.team.repository.common.InternalRepositoryException: Database connection is null. Check database status and availiablity.
at com.ibm.team.repository.service.internal.rdb.JDBCConnectionPool.checkConnection(JDBCConnectionPool.java:125)
at com.ibm.team.repository.service.internal.rdb.JDBCConnectionPool.getConnection(JDBCConnectionPool.java:97)
at com.ibm.team.repository.service.internal.rdb.ConnectionPoolService.withCurrentConnection(ConnectionPoolService.java:315)
at sun.reflect.GeneratedMethodAccessor60.invoke(null)

permanent link
Sterling Bates (2311612) | answered Aug 25 '09, 12:44 p.m.
For posterity, a defect was filed here:

https://jazz.net/jazz/web/projects/Rational%20Team%20Concert#action=com.ibm.team.workitem.viewWorkItem&id=90813

It looks like we got things under control by creating some indexes and dropping the number of total connections used by Jazz. (We're using DB2 by the way, in case I didn't mention that before.)

Your answer


Register or to post your answer.