It's all about the answers!

Ask a question

Upgrade stuck on running post addTables for "" operation

Georg Kellner (840377108) | asked Jun 05 '14, 12:04 p.m.
edited Sep 23 '14, 5:46 a.m. by Krzysztof Kaźmierczyk (7.4k369102)
Hi fellows,
I'm testing the upgrade 4.0.6 -> 5.0.0.
OS: w2k8 R2
DB: Oracle 11
Jazz: 4.0.6 with WAS 8.0, testsystem
Jazz and DB on separate servers

The job running post addTables for "" is running since three hours.
The CCM database is around 500 MB, containing several project areas, some of them used for SCM tests.
Both servers have a little processor usage of 8 - 10% and the network(1 GB/s) shows 0,5% usage.

I'm wondering about the time of three hours plus, regarding this small database.

What is the job doing?
Does it a read/write for all db entries?
All the steps before were well done, no error messages and very fast.

greetings georg.

Stephanie Bagot commented Jun 06 '14, 9:43 a.m. | edited Sep 23 '14, 5:46 a.m.

The addTables will add any new tables to the SCM portion of the CCM DB which is required after the upgrade. Normally I would suggest that this may be an issue with performance of the servers or latency, but if your processor and network usage are showing so little there may be something else going on. Have you checked the repotools-ccm_addTables log located in your server directory? This may tell you some more info and if the upgrade is hung.

Also - I would double check you are using a support DB and JDBC driver, since this may also affect the addTables. The system requirements are located here for 5.0:

Tim Mok commented Jun 06 '14, 11:14 a.m. | edited Aug 29 '14, 10:27 a.m.

There was a significant change in the way SCM stores data in the 5.0 release. The migration for this may take longer than previous upgrades.

Did you run the online migration command before running addTables?

Kevin Ramer commented Jun 06 '14, 2:34 p.m.

Are there estimates of this based on various database size / content ?  We have SCM databases from a few Gb to 500Gb (that last one with likely a mountain of SCM data).  I'd hate to be caught needing to schedule a multi-day outage to upgrade 1 repository....

Tim Mok commented Jun 06 '14, 4:18 p.m.

The database size isn't quite comparable here because it's the number of item states to update that determines the time. When we migrated our database, the test runs on 490Gb with 2.6 million items and 28.8 million item states took about 10 hours of online migration followed by 3 hours of offline migration. Online migration used a priority of 100 and numOfStatesPerRun of 100.

Georg Kellner commented Jun 10 '14, 6:45 a.m.

I did an offline update, because it is a small test environment, so I didn't thought about system downtime or so.

Georg Kellner commented Jun 10 '14, 7:12 a.m.

Well, this might be the hint:

2014-06-05 14:37:15,512 CRJAZ1364I The connection to the following database was successful:
 Db Product Name: Oracle
  Db Product Version: Oracle Database 11g Enterprise Edition Release - 64bit Production is supported.
Let's see, what support says.

Krzysztof Kaźmierczyk commented Aug 29 '14, 10:26 a.m.

Hi Tim,
I have another customer having the same issue on Oracle. Here is how the log finishes:

2014-08-27 18:08:52,161 Running post addTables for ''...

And nothing happens through the night. How can we further troubleshot this issue? This is also offline migration.

Tim Mok commented Aug 29 '14, 10:56 a.m.

They should contact support.

Krzysztof Kaźmierczyk commented Aug 29 '14, 3:43 p.m.

If they contact the support, then I will contact you in the internal escalation then :)
Anyway let me create the escalation for that and work internally.

naveen kumar commented Sep 20 '14, 5:40 a.m. | edited Sep 23 '14, 5:46 a.m.
 We are getting same error,please could you let us know if you found any solution for taking long time to execute following command while upgrade.
upgrade/ccm/ -oldApplicationHome /opt/IBM/JazzTeamServer/server/conf

Tim Mok commented Sep 22 '14, 9:30 a.m.

It is likely a long upgrade as mentioned previously due to significant changes to the SCM implementation. It is converting all your data to allow SCM to implement new features such as tracking who added change sets to a stream/workspace.

If you think it is taking an abnormal amount of time to upgrade, please contact support.

showing 5 of 11 show 6 more comments

Accepted answer

permanent link
Krzysztof Kaźmierczyk (7.4k369102) | answered Sep 23 '14, 5:45 a.m.
Hi Georg,
Your issue has been identified as a defect in RTC. See this bug for more details:
Georg Kellner selected this answer as the correct answer

Georg Kellner commented Sep 23 '14, 7:16 a.m.

Does this defect mean, we should avoid upgrades to 5.0 as far as this defect isn't solved as we are using Oracle as db provider?

Krzysztof Kaźmierczyk commented Sep 30 '14, 8:16 a.m.

Yes, that is correct.

Your answer

Register or to post your answer.