Database timeout or deadlock on IBM i.
We've just completed at setup of RTC on IBM i (everything including WAS is running on IBM i).
Everything seems to work fine, we can log in and then try to create the first project. Deployment of templates worked fine. Project is created and the initialization begins.
After some time (about 1 min) the operation aborts and we see the following in the ccm.log:
2015-12-17 16:15:58,515 [WebContainer : 35 @@ 16:14 <unauthenticated> <LQE/1.0@10.200.62.65> /ccm/process-trs2] ERROR com.ibm.team.process - CRJAZ1318E The server could not connect to the database. Try the operation again.
CRJAZ0253I A deadlock or timeout occurred without automatic rollback.
SQL: select t1.ITEM_ID, t1.STATE_ID, t1.JZ_DISCRIMINATOR from JAZZCCMDB.PRCSSPRCSS t1 where t1.JZ_DISCRIMINATOR = 103 and ((t1.MODIFIED >= {ts '2015-12-10 16:14:54.584'}) and ((t1.MODIFIED <= ?) or (t1.MODIFIED >= ?))) order by t1.MODIFIED desc
SQL Exception #1
SQL Message: [SQL0913] Row or object PRCSSPRCSS in JAZZCCMDB type *FILE in use.
SQL State: 57033
Error Code: -913
com.ibm.team.repository.common.RetryableDatabaseException: CRJAZ1318E The server could not connect to the database. Try the operation again.
CRJAZ0253I A deadlock or timeout occurred without automatic rollback.
SQL: select t1.ITEM_ID, t1.STATE_ID, t1.JZ_DISCRIMINATOR from JAZZCCMDB.PRCSSPRCSS t1 where t1.JZ_DISCRIMINATOR = 103 and ((t1.MODIFIED >= {ts '2015-12-10 16:14:54.584'}) and ((t1.MODIFIED <= ?) or (t1.MODIFIED >= ?))) order by t1.MODIFIED desc
SQL Exception #1
SQL Message: [SQL0913] Row or object PRCSSPRCSS in JAZZCCMDB type *FILE in use.
SQL State: 57033
Error Code: -913
Could this be because the database (Library or *FILE) on the IBM i is running very slowly and then aborts?
The performance measurement reports 1300ms which is on the low side I must admit.
Could we maybe increase some timeout on the DB connection or on the IBM i file object itself?
Please, any comments are greatly appreciated!
/Morten.
One answer
Hi Morten,
I have not encountered this symptom before. We found a couple of articles that may lead to a solution:
http://www-01.ibm.com/support/docview.wss?uid=swg21190797
http://www-01.ibm.com/support/docview.wss?uid=nas8N1016772
If changing the wait times on file JAZZCCMDB/PRCSSPRCSS does not help, then consider opening a PMR against IBM i DB2 or JAZZ foundation. Ensure all PTF's are applied and provide version information (RTC, i/OS, WAS, ...).
I have not encountered this symptom before. We found a couple of articles that may lead to a solution:
http://www-01.ibm.com/support/docview.wss?uid=swg21190797
http://www-01.ibm.com/support/docview.wss?uid=nas8N1016772
If changing the wait times on file JAZZCCMDB/PRCSSPRCSS does not help, then consider opening a PMR against IBM i DB2 or JAZZ foundation. Ensure all PTF's are applied and provide version information (RTC, i/OS, WAS, ...).
Comments
Melissa Weaver
Dec 17 '15, 1:01 p.m.Did you check and see what job had a lock on the file
PRCSSPRCSS in JAZZCCMDB type *FILE?Morten Madsen
Dec 17 '15, 2:55 p.m.Sorry, I don't have access to the host right now. But yes, we checked it, and it looked like the normal jobs spawned by RTC. When we shut down the RTC WAS profile, all jobs where closed. So it seems to me, it is not a deadlock, just a slow database that exceeds the timout threshold.