It's all about the answers!

Ask a question

DOORS Upgrade is taking too long


Rafael Hernandez (3222434) | asked Sep 06 '16, 8:16 p.m.
 Good day everyone,

We are planning to upgrade CLM from 5.0.2 to 6.0.2 and we are working on upgrade DOORS on a test server.
 
We executed EXEC sp_updatestats  And started the upgrade process but is taking a lot of time (6 days) and continues.

I wonder know if we can do anything to accelerate the upgrade process and i want to know if the process would be faster on our production server.

Our test enviroment:
-Multiple server topology
-JTS, CCM, RM, DM, DCC,RS,QM

-DOORS test enviroment:
4 cores
6 gb RAM
3 GB assigned to repotools script

DB test enviroment:
8 cores
16 GB RAM

Our Production enviroment:

-Multiple server topology
-JTS, CCM, RM, DM, DCC,RS,QM

-DOORS production enviroment:
8 cores
64 gb RAM

DB production enviroment:
40 cores
512 GB RAM

RM DATABASE SIZE: 42 GB
NUMBER OR ARTIFACTS: 46650

Thank you!



3 answers



permanent link
Rafael Hernandez (3222434) | answered Sep 12 '16, 10:46 a.m.
 The process continues after 11 days, any idea?

Comments
Lewis Tsao commented Sep 12 '16, 11:04 a.m.

Never figured which JVM actually gets used when running repotools
But may need to check the heap and nursery size to make them big enough.

  1. (Supplement to) documentation suggests at least 8GB for heap and 1/3 heap for nursery with an absolute minimum of 4GB for heap. I am surprised your test installation runs at all.
  2. DNG (RM) requires a fair amount of system memory(extra to heap) for cache

So any test environment will require at least 16GB system memory with 8GB for each CLM component.

If putting in multiple components in one JVM suggest 16GB heap and minimum 24-32GB RAM.


Rafael Hernandez commented Sep 12 '16, 11:19 a.m.

 Thanks Lewis,


I will try to get more memory for my server; i should stop the upgrade process, increase the RAM memory and execute it again?, im not sure what i have to do if i stop the upgrading process


Kevin Ramer commented Sep 12 '16, 11:47 a.m.

Has the performance of the database been taken into account ? 


Rafael Hernandez commented Sep 12 '16, 11:53 a.m.

 Hey Kevin


The database server CPU use is 2-4%, the Memory use is 92-95% 14.8/16 gb

Should i increase the database memory too?

Thx


Rafael Hernandez commented Sep 13 '16, 5:09 p.m.

 I already changed the available RAM memory for both VMs to 32 gb, edited the repotools-rm to:


set VMARGS=%VMARGS% -Xmx24576M
set VMARGS=%VMARGS% -Xms24576M
set VMARGS=%VMARGS% -Xmn7447M
set VMARGS=%VMARGS% -Xgcpolicy:gencon

The upgrading process is allocating about 10% of total memory for the rm server and 95% for the DB server

It's been upgrading for about a day and it cant finish

Any idea? i cant wait for 1 or 2 days in a productive upgrading


permanent link
Paul Ellis (1.3k613) | answered Sep 14 '16, 11:23 a.m.
JAZZ DEVELOPER
edited Sep 14 '16, 11:27 a.m.
Hi Rafael,

You appear to be doing everything correctly, as Lewis has written above.  For future reference you can see:
Upgrade Must Read Cheklist
6.0 Migration Update Database Statistics

I am not an SQL server database admin, but sp_updatestats seems to be different to update statistics. 
The problem seems to be with the DB server right now, so the heap and nursery is not the concern.  Are your DBAs able to shed light on this? 

On a server with 512GB RAM, are you able to get more?  If this were Oracle or DB2 I'd expect to see anything up to 64GB RAM used for an intense operation.  If would have thought 32GB might be enough, but maybe not.

In the 6.0 Upgrade datasheet then 64GB RAM was in use for a similar data size to yours.  Usually though, the update statistics prior to and post upgrade on these tables

Repository.ITEM_STATES
Repository.ITEM_CURRENTS
Resource.Resource
VVCMODEL.VERSION

VVCModel does exist in 5.x but it's empty - we have seen before symptoms where the table is getting built and doing things but DB isn't keeping up with the new data so it may need to be ran DURING upgrade.

Also ensure you have the latest ifix in place for 6.0.2 -> https://jazz.net/downloads/rational-doors-next-generation/releases/6.0.2/CLM_602_iFix004.zip

Finally be aware that post upgrade you may need this: http://www-01.ibm.com/support/docview.wss?uid=swg21975746

Alas, once you start an upgrade you would need to invoke your rollback procedure here to the start of the upgrade.

Kind regards,
Paul


Comments
Rafael Hernandez commented Sep 14 '16, 1:54 p.m.

 Thank you Paul, i will review all the information and try again, i


Rafael Hernandez commented Sep 15 '16, 11:07 a.m.

 I just increased the DB server ram memory up to 64 gb and the problem persist, its been 24 hours and the upgrading process doesnt end.


The upgrading process stucks on a project area, this is the information on log file:

By the way, the RM server memory is stable on 5.7GB of 32 GB and the database server is stable on 29.5GB of 64 GB

The network type is gigabit

I think the best is to open a PMR

Thank you!


permanent link
Paul Ellis (1.3k613) | answered Sep 15 '16, 12:27 p.m.
JAZZ DEVELOPER
Hi again,

If you are able to I would.  It sounds like it got stuck somehow, so we would need to troubleshoot.

I am sorry we could not resolve it here.

Kind regards,
Paul


Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.