EditAttachPrintable
r21 - 2017-02-27 - 13:28:50 - PaulEllisYou are here: TWiki >  Deployment Web > DeploymentInstallingUpgradingAndMigrating > DOORSToDNGMigrationAndIntegration > DOORSToDNGMigrationSizingGuide

Rational DOORS 9 to Rational DOORS Next Generation migration sizing guide new.png

Authors: PaulEllis, ErikCraig, PaulClunie
Build basis: Rational DOORS 9.6.1.3+, Rational DOORS Next Generation 6.0.1, 6.0.2, 6.0.3

The purpose of this page is to collate information from other sections of the deployment wiki and aggregate those with new guidance for DOORS 9 migration packages and Rational DOORS Next Generation(RDNG) sizing. There are important new updates since the RDNG 6.0.2 release. This document was updated on May 4th, 2016 to reflect these.

DOORS 9 does not publish sizing recommendations for module and project size in the same way as DOORS Next Generation's sizing guides. However, it is imperative when evaluating a migration from DOORS 9 to DOORS Next Generation that your DOORS 9 data is analyzed and modified accordingly before attempting imports into RDNG.

In order to understand the context of your migration you should avail of the Migration Metrics utility built into DOORS 9.6.1.3 and later. Also see the online help topic for Migrating data to Rational DOORS Next Generation

Also ensure that you are comfortable with the detailed guidance advice from the Migrate data from Rational DOORS to Rational DOORS Next Generation developerWorks article before proceeding.

Once you have an understanding of your current data shape within DOORS 9 you can more easily map this to how you wish to structure your modules and projects within RDNG.

Sizing recommendations for Rational DOORS Next Generation

The most helpful information when evaluating whether your data will migrate easily across to RDNG is to compare the output from the Migration Metrics tool and correlate module and project sizings with our published advice on Rational DOORS Next Generation Sizing in 6.x.

The key metrics from these guides in relation to migration are below. In Rational Requirements Composer Project Sizing Recommendations we explicitly stated our recommendations, but that document was written for Rational Requirements Composer 4.x . Since then our recommendations have been more implicit via the performance datasheets and guidelines. The sections below once again try to simplify the detailed information within the datasheets for rule-of-thumb guidance.

Recommended folder, module, project and instance sizes

RDNG 6.0.x (Windows)

RDNG 6.0 / 6.0.1

RDNG 6.0.2+ (mapped I/O)

Total Number of Folders / Modules

10,000*

10,000*

10,000*

Artifacts per Project

85,000 - 150,000

85,000 - 150,000

85,000 - 150,000

Artifacts per instance

2 - 2.5 million

2 - 2.5 million

4 - 5 million

Memory mapped I/O since DNG 6.0.2

There is an important change to the way in which DNG references the data in the Jena indices since the 6.0.2 release. This change is by default with Linux, but those that run their DNG system on Windows require manual steps to enact these changes. Setting Memory Mapped I/O on Windows 64 bits systems documents how to do this, but also the subsequent sections reflect on the decisions and considerations that need to accompany turning this on.

Why turn it on? As the table shows, the scalability increases when the indices are read outside of the Java Virtual Machine heap space. The performance is also improved when heavier workloads are in operation.

Additional Guidance:

10,000 (10k) artifacts per module and folder
This helps to improve readability. As a best practice start with creating a folder structure in the project and avoid adding too many artifacts directly in the root folder. There are other artifact types that need consideration, number of links within project and cross project that could increase as well (based on the above project, module and folder size limits).
It must be noted that attempts to import beyond 10k will require a different set of resources to the standard implementation. It will be increasingly difficult to work effectively with a module above this size, even if the import is successful. See below for further details on considerations for larger modules.

85,000 - 150,000 artifacts per project
Artifacts per project should generally not exceed 85,000 and these include folders, modules, module artifacts and collections. In case there are a high number of images, tables which in turn increase size of artifacts, consider a lower artifact number per project.
If the installation is appropriately sized then these maximum numbers could potentially increase to 100k -150k. However, as general guidance the 85k project sizing is a better target for planning where possible.

2 - 5 million artifacts per instance/repository
A comfortable overall size of a repository depends on many different aspects of both the data, hardware and number of users concurrently using the system. In the Recommended hardware settings below you can see how you would need to scale your hardware for each load. Further, more detailed guidance is given in the performance report for RDNG 6.x.

  • The guidance though for Rational DOORS Next Generation 6.0 and 6.0.1 on Linux is that you will start to see negative performance once you exceed 2 - 2.5 million artifacts.
  • The guidance though for Rational DOORS Next Generation 6.0.2 and 6.0.3 on Windows, by default, is that you will start to see negative performance once you exceed 2 - 2.5 million artifacts per instance/repository


There are a couple of important changes within RDNG 6.0.2. The aforementioned Jena change will assist both Windows and Linux, but more crucially, it was possible to switch the way Jena indices are managed by the operating system in 6.0.2 which affords the maximum size of the instance to approximately double from RDNG 6.0.1.

  • The guidance though for Rational DOORS Next Generation 6.0.2 with memory mapped I/O is that you will start to see negative performance once you exceed 4 - 5 million artifacts.

Recommended hardware settings

The simplest approach for sizing is to match your pattern with this diagram which demonstrates hardware scalability.
VAUGHN.JPG

The Tested Hardware section of the Sizing and Tuning Guide for Rational DOORS Next Generation version 6.0.2 shows the hardware recommended for the data shapes we tested against.

The System Requirements no longer quote a minimum specifications for CPU and RAM, preferring to use the workload definitions within the Sizing guides. Smaller systems should extrapolate down. If DNG is one of the elements of your CLM solution, then please also consider our Proof Of Concept sizing for CLM 6.x.

Please note that this information is specific to 6.0.3 and that changes to the maximum number or artifacts in the database, with all other variables are subject to change in future releases.

Sizing limitations for the size of the migration package created in DOORS 9

In Rational DOORS Next Generation(RDNG) 6.0.1, there are recommended limits on the size of the migration package which are dependent on the machine resources of the importing RDNG server.

As for numbers of DOORS objects in a single migration package, we have some benchmark figures for a realistic set of modules of varying sizes and object hierarchies. This provides a good guide to the maximum amount of data that can be handled in a single migration import.

The machine specification in use for the export of the DOORS 9 and import into RDNG 6.0.1. Based on the figures within the section Recommended hardware settings above, :

  • Windows Server 2012
  • 32GB RAM
  • 2 x vCPU
  • Java heap values set at: Xmx 16G, Xms 16G, Xmn 4G

The following example was generated using DOORS v9.6.1.4 and RDNG v6.0.1:

NEW: The table below has been expanded to include figures from importing the same package using RDNG v6.0.2. Note the size of the indices is reduced and the time to import is significantly improved. The 6.0.1 numbers have been kept for comparison and evaluation of whether upgrading to the latest release would benefit your size of implementation.

Total number of objects / Modules

Number of Links

Number of OLEs and Size(MB)

Size of package / uncompressed

Duration of DOORS 9 Export

Duration of DNG Import 6.0.1

Duration of DNG Import 6.0.2

RDNG Indices Size 6.0.1

RDNG Indices Size 6.0.2

11114 / 21

8000

800 / 11MB

14.2MB / 136MB

4 mins, 6 secs

67 mins

52 mins

733 MB

<733 MB

26676 / 43

2000

1920 / 26.4MB

35MB / 397MB

11 mins, 40 secs

3 hours, 16 mins

2 hours, 21 mins

2.16GB

1.67GB

60018 / 106

50,000

4320 / 59.5MB

78.3MB / 809MB

30 mins, 31 secs

10 hours, 4 mins

5 hours, 32 mins

4.22GB

3.95GB

120,036 / 212

100,000

8640 / 118.9MB

156MB / 1.58GB

1 hour, 22 mins

25 hours, 9 mins

12 hours, 27 mins

8.01GB

7.8GB


The table shows that as the size of the imports grow, the hardware specification becomes increasingly inadequate. The final time of 25 hours in RDNG 6.0.1 would be greatly improved with additional resources. There are considerable improvements with 6.0.2, but even the 15 hour figure could be drastically improved with more resources.

Notes:

  • Using the specification above, files above approximately 2Gb uncompressed typically fail to upload into RDNG.
  • Larger ReqIF/migiz packages will require larger Java heap sizes and therefore memory.
  • There will be significant disk activity during an import for the application server and the database server


Considerations before exporting DOORS data to RDNG

The main guidance for approaching this migration is comprehensively discussed within Migrate data from Rational DOORS to Rational DOORS Next Generation developerWorks article.

If after assessing which data sets need to be exported you are facing sizes in excess of those discussed in this article then you should strongly consider performing the data manipulation prior to exporting from DOORS and well before attempting an import. Considerations for larger imports above explains the additional resources required to accept a large package, so postponing the manipulation to post-export is nonsensical.

The following procedure is how you would potentially split DOORS 9.x modules into smaller components, prior to migration. The example below is for a 50k object module within DOORS 9.

  1. Run the DXL from technote How to keep links when restoring a DOORS module archive to get all the links in to attributes in the master module
  2. Move artifacts to several/new target modules (Tools -> Functions -> Copy Object), perhaps 5 modules would be ideal so that each module has 10k artifacts.
  3. Use "Link by Attribute" to re-establish the incoming and outgoing links to/from each module

This information is provided as an example method for reconfiguring data without losing links, which would usually make this action prohibitive. Please contact DOORS Support for further assistance with implementing the technote, although also please note the Disclaimer.


Considerations for larger imports.

What is considered a large import can be subjective, but in terms of this article it is any migration which attempts over 10k artifacts in a single migration or ReqIF package. The recommended limit of 10k artifacts per module are still respected, but as with the examples above, there are multiple modules destined for a project of approximately 100k artifacts.

  • Increase the log file of the database server. Empirically, our expertise resides with DB2, therefore the following settings are recommended for change in order to unblock the bottleneck of the database server.

   db2 connect to RM6
   db2 update db cfg for RM6 using LOGPRIMARY 120
   db2 update db cfg for RM6 using LOGFILSIZ 16384


  • Ensure the hard disks in use are optimal. Solid State Disks(SSD) are recommended for the RM indices location and the database server's log file area. Standard 10k rpm as disk speed is generally insufficient for the read/write access required on very heavily used directories. Use hdparm –tT to track disk speed. The Timing cached reads per second are crucial.

  • More memory. As the packages grow larger, as will the amount of RAM required.
  • Java Heap space available. Consider increasing your Xmx, Xms and Xmn relative to your package size. You should actively monitor your Java garbage collection and allocate more resources accordingly.
  • CPU. The machine used for the above numbers is significantly below that recommended for larger data sets, as extrapolated from relative size prescribed within the published advice on Rational DOORS Next Generation Sizing in 6.x.
  • Run regular updates on the database statistics after each large migration. See, Keeping database statistics up-to-date, for instructions for your database server.
  • Finally, note that long running sessions can fall foul of web server timeouts. Please note Tuning Jazz systems to support performance testing for guidance if a single operation import exceeds 6 hours.

Related topics: Calculating your requirements metrics, Collaborative Lifecycle Management performance report: RDNG 6.0, Sizing and tuning guide for Rational DOORS Next Generation 6.0

External links: Migrate data from Rational DOORS to Rational DOORS Next Generation

Additional contributors: KouroshManeshni, BenSilverman, TimHards, VaughnRokosz, KnutRadloff, PaulYoung

Topic attachments
I Attachment Action Size Date Who Comment
Jpgjpg VAUGHN.JPG manage 57.1 K 2016-04-08 - 14:06 PaulEllis Scalability and Sizing
Edit | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r36 | r23 < r22 < r21 < r20 | More topic actions...
 
This site is powered by the TWiki collaboration platformCopyright © by IBM and non-IBM contributing authors. All material on this collaboration platform is the property of the contributing authors.
Contributions are governed by our Terms of Use. Please read the following disclaimer.