Upgrade stuck on running post addTables for "com.ibm.team.scm" operation
Georg Kellner (840●3●78●109)
| asked Jun 05 '14, 12:04 p.m.
edited Sep 23 '14, 5:46 a.m. by Krzysztof Kaźmierczyk (7.4k●3●74●103)
Hi fellows,
I'm testing the upgrade 4.0.6 -> 5.0.0. Environment OS: w2k8 R2 DB: Oracle 11 Jazz: 4.0.6 with WAS 8.0, testsystem Jazz and DB on separate servers The job running post addTables for "com.ibm.team.scm" is running since three hours. The CCM database is around 500 MB, containing several project areas, some of them used for SCM tests. Both servers have a little processor usage of 8 - 10% and the network(1 GB/s) shows 0,5% usage. I'm wondering about the time of three hours plus, regarding this small database. What is the job doing? Does it a read/write for all db entries? All the steps before were well done, no error messages and very fast. greetings georg.
showing 5 of 11
show 6 more comments
|
Accepted answer
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.
Comments
The addTables will add any new tables to the SCM portion of the CCM DB which is required after the upgrade. Normally I would suggest that this may be an issue with performance of the servers or latency, but if your processor and network usage are showing so little there may be something else going on. Have you checked the repotools-ccm_addTables log located in your server directory? This may tell you some more info and if the upgrade is hung.
Also - I would double check you are using a support DB and JDBC driver, since this may also affect the addTables. The system requirements are located here for 5.0: https://jazz.net/wiki/bin/view/Deployment/CLMSystemRequirements
There was a significant change in the way SCM stores data in the 5.0 release. The migration for this may take longer than previous upgrades.
Did you run the online migration command before running addTables?
Are there estimates of this based on various database size / content ? We have SCM databases from a few Gb to 500Gb (that last one with likely a mountain of SCM data). I'd hate to be caught needing to schedule a multi-day outage to upgrade 1 repository....
The database size isn't quite comparable here because it's the number of item states to update that determines the time. When we migrated our database, the test runs on 490Gb with 2.6 million items and 28.8 million item states took about 10 hours of online migration followed by 3 hours of offline migration. Online migration used a priority of 100 and numOfStatesPerRun of 100.
I did an offline update, because it is a small test environment, so I didn't thought about system downtime or so.
Well, this might be the hint:
2014-06-05 14:37:15,512 CRJAZ1364I The connection to the following database was successful:
Db Product Name: Oracle
Db Product Version: Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
11.2.0.2 is supported.
Let's see, what support says.
Hi Tim,
I have another customer having the same issue on Oracle. Here is how the log finishes:
2014-08-27 18:08:52,161 Running post addTables for 'com.ibm.team.scm'...
And nothing happens through the night. How can we further troubleshot this issue? This is also offline migration.
They should contact support.
If they contact the support, then I will contact you in the internal escalation then :)
Anyway let me create the escalation for that and work internally.
It is likely a long upgrade as mentioned previously due to significant changes to the SCM implementation. It is converting all your data to allow SCM to implement new features such as tracking who added change sets to a stream/workspace.