Using the ClearCase Importer to Import ClearCase History

Maneesh Mehra, Cunxia Sun, Carol Yutkowitz
IBM Rational Software Group
Last updated: September 6, 2012

1 Introduction

The ClearCase importer allows ClearCase users import VOB data with some limited history into Rational Team Concert. ClearCase UCM users can import all or a pre-determined set of baselines created in a UCM stream into Rational Team Concert, and base ClearCase users can choose to import a pre-determined set of label types applied to interesting versions in the VOB.

Prerequisites: The ClearCase Importer uses the same user interface as the ClearCase Synchronizer and is based on its familiar semantics of importing synchronized streams, defining synchronization roots and running sync jobs periodically. However, unlike the synchronizer, which synchronizes changes in both directions (between Jazz and ClearCase VOBs), the importer only synchronizes in one direction, updating the Jazz repository with changes from ClearCase. It does not update the ClearCase repository with changes made in Jazz. Since the ClearCase Importer shares a lot of its implementation with the ClearCase Synchronizer, this article assumes that the user is familiar with its concepts and has already set up Rational Team Concert for importing ClearCase data and has configured a synchronization host. If you are not already familiar with these concepts, please see [1]. This article also assumes that Jazz Team Server repository is configured and an RTC Client can connect to it using a valid user account. In addition, this article assumes that the reader is familiar with ClearCase UCM and base ClearCase concepts. Instructions in this article are in reference to setting up a host running the Windows operating system [2]. Finally, the example in this article was created using the Eclipse Way process specification, which is no longer bundled with Rational Team Concert, but available for download on jazz.net. Articles supporting this use case for other process specifications will be available in the future. It should be relatively straightforward to adapt the example to whatever process you have deployed.

2 Setting up the importer for UCM history

The setup for importing UCM stream data is much the same as that for the ClearCase Synchronizer. The UI allows users to indicate that they would like to import ClearCase VOB data with history as captured in the baselines present on the UCM stream. Users can import:

1. All baselines in the UCM stream; OR
2. A subset of baselines in the UCM stream. Baselines can be pre-selected by applying a ClearCase attribute type to them. Later when the user selects a synchronization root, they can specify the name of this attribute type. Only those baselines which have this attribute applied to them will have their content imported.

2.1 Import a UCM stream into Rational Team Concert

1. Choose a method for importing the UCM stream. There are three ways that UCM streams can be imported into Rational Team Concert. It does not matter which way is chosen. For the benefit of the user, this article lists all the methods:
   • Method 1: In Rational Team Concert, choose File->Import->ClearCase Stream or Branch (under Jazz Source Control). Click Next.

     

   • Method 2: In Rational Team Concert’s ClearCase Synchronized Streams view, click on hyperlink titled Import ClearCase data into Jazz. If this view is not visible, choose Window->Show view->Other->Jazz Source Control->ClearCase Synchronized Streams [3].

     

   • Method 3: In Rational Team Concert’s ClearCase Synchronized Streams view, click on the Import ClearCase data into Jazz icon.

     

2. Enter the name of the UCM stream for which history is to be imported.
3. Click the Import with history from ClearCase check box to indicate that you want to import history. If this box is not checked, the synchronizer rather than the importer will be used.

   

4. Click Next.
5. Enter a UNC path to a valid view storage area. Optionally, enter additional information for the storage directory.

   

6. Click Next.
7. Choose a Team Area.

   

8. Click Next.
9. Enter values for the synchronization user ID and path to the ccctools folder of the ClearCase Synchronizer installation area.

   

10. Choose other default values. Click Finish.
11. The wizard creates a synchronized stream in Rational Team Concert and starts the synchronization process in the background. Once this is complete, bring up the Select Files wizard. Notice that the ClearCase Synchronized Streams view is populated with information about the UCM stream data just entered in the wizard.

    

2.2 Select synchronization roots and baseline filter

The Select Files wizard started by the import stream wizard allows users to select the synchronization roots (the set of files and folders to be synchronized). It also allows users to specify whether they would like to import all baselines for the component the sync roots belong to, or some subset of the baselines as indicated by a ClearCase attribute applied to them. For UCM imports, it is highly recommended that users select a folder that is visible in all baselines as a synchronization root [4].

2.2.1 Import All Baselines

1. Select the root of a UCM component from which to import baselines.
2. Choose the default option to import all baselines.

   

3. Click Finish.

   The wizard sends an import request to the Jazz repository that is serviced by the synchronization process running on the synchronization host. Depending on the size of the component, number of baselines of the component in the stream and the amount of historical data available in these baselines, it may take from few minutes to a few hours (or even days) before all UCM data is imported into Rational Team Concert.

4. Wait for the synchronization request to be completed.

2.2.2 Import A Subset of Baselines

To select a subset of baselines from the list of all baselines for a given component in the UCM stream, users first need to create a ClearCase attribute type and then apply it to the baselines they would like to import. The attribute type must be created in a VOB that is in the component’s administrative hierarchy. The attribute’s vtype must be string or integer and its value should be any valid string or integer (A suggested value is true).

1. Select the root of a UCM component whose baselines will imported.
2. Choose the option to specify the baselines that have an attribute.
3. Enter the name of the attribute type.

   

4. Click Finish. Similar to Section 2.2.1, the wizard sends an import request to the Jazz repository that is serviced by the synchronization process running on the synchronization host. Depending on the size of the component, number of baselines of the component in the stream that have the attribute applied to them and the amount of historical data available in these baselines, it may take from few minutes to a few hours (or even days) before interesting UCM data is imported into Rational Team Concert.
5. Wait for the synchronization request to be completed.

2.3 Review imported history data

1. Once the synchronization request is completed by the synchronization process, the newly imported component shows up as an incoming change in the Pending Changes view on the merge workspace created by the import stream wizard.

   

2. Right-click the component and select Show->History.

   

3. Double click any history item in the History view to look at the details of the change in the Change Explorer view. This view shows the change applied to the Jazz repository for this component in the synchronized stream.

   

4. Right-click the history item in the History view and choose Open Work Items. Notice the Work Item editor view open for this work item. The work item contains useful information about the UCM activities imported in this change set. In addition, it describes the UCM baseline that contained these activities for back reference into ClearCase.

   

3 Setting up the importer for base ClearCase history

The setup for importing base ClearCase history is much the same as the setup for synchronization. The UI allows users to indicate that they would like to import ClearCase VOB data with history as captured in labeled versions. It also provides a mechanism for users to tag which label types should be imported from the list of all available label types in the VOB’s administrative hierarchy. Unlike the import of UCM history which contains a UI option to specify import of all baselines, for base ClearCase label types must always be pre-selected by applying a ClearCase attribute type to them. Later when the user selects a synchronization root, they must specify the name of this attribute type. Only those label types which have this attribute applied to them will have their content imported. The Base ClearCase History Importer also imposes additional requirements on the vtype and the value of the attribute type:

• The attribute vtype must be string or integer; AND
• The attribute value applied to label types must be an incrementally increasing positive integer (for example, “1”, “2”, “3” etc.) [5]

When importing base ClearCase history, attribute values signify the order of processing of the label types. The importer sorts all available label types based on their attribute values and starts processing the type with the lowest value. For this reason, the lowest value must be applied to the “backstop” label type that will be used for the creation of the synchronized stream. (“Backstop” just means the earliest label to be imported, and is the label specified on the first page of the import wizard.) All other label types to be imported must have values in the increasing numerical order as shown above. Finally, all the interesting label types must be applied fully to the file-system hierarchy in the VOB (starting from the VOB root) and must be locked prior to import into Rational Team Concert by the Base ClearCase History Importer.

NOTE: Unlike the base ClearCase Synchronizer, the base ClearCase Importer does not need a branch type at the time of creation of the Synchronized stream, because the specified branch type is used only to update the ClearCase repository with changes made in the Jazz repository, which the importer does not do.

3.1 Import base ClearCase branch into Rational Team Concert

1. Follow one of the 3 methods as listed in Section 2.1 to bring up the import stream wizard.
2. Select Import from an existing Base ClearCase Branch.
3. Enter the name of the backstop label type for the synchronized stream.
4. Click the Import with history from ClearCase check box to indicate that you want to do an import. Not selecting this box will assume use of the synchronizer.
5. Read the informative message on why the branch type is not needed for import:

   

6. Click Next.

   

7. Enter a UNC path to a valid view storage area. Optionally, enter additional information for the storage directory.

   

8. Click Next.
9. Choose a Team Area.

   

10. Click Next.
11. Enter values for the synchronization user ID and path to the ccctools folder of the ClearCase Synchronizer installation area.

    

12. Choose other default values. Click Finish.
13. Wait for the wizard to create a synchronized stream in Rational Team Concert, start the synchronization process in the background and finally bring up the Select Files wizard. Notice that the ClearCase Synchronized Streams view is populated with information about the base ClearCase synchronized stream data just entered on the wizard.

    

3.2 Select synchronization roots and label type filter

The Select Files wizard started by the import stream wizard allows users to select the roots to be imported (called “synchronization roots.” It also allows users to specify the attribute type used to tag the subset of label types to be imported. For base ClearCase imports, it is highly recommended that users select a as a synchronization root a folder that is visible in all label types to be imported [6].

3.2.1 Import A Subset of Label Types

To select a subset of label types from the list of all label types in a given VOB, users need to first create a ClearCase attribute type and then apply it to the label types they would like to import. The attribute type must be created in a VOB that is in the VOB’s administrative hierarchy.

1. Select the root of a VOB from which to import labeled versions.
2. Enter the ClearCase attribute type.

   

3. Click Finish. The wizard sends an import request to the Jazz repository that is serviced by the synchronization process running on the synchronization host. Depending on the size of the VOB, number of label types that have the attribute applied to them and the amount of historical data tagged by these label types, it may take from few minutes to a few hours (or even days) before all requested base ClearCase data is imported into Rational Team Concert.
4. Wait for the synchronization request to be completed.

3.3 Review imported history data

1. After the synchronization request is completed by the synchronization process, the newly imported component appears as an incoming change in the Pending Changes view on the merge workspace created by the import stream wizard.

   

2. Right click on the component and select Show->History.

   

3. Double click on any history item in the History view to look at the details of the change in the Change Explorer view. This view shows the change applied to the Jazz repository for this component in the synchronized stream.

   

4. Right-click the history item in the History view and choose Open Work Items. The Work Item editor opens for this work item. The work item contains useful information about the Base ClearCase label type imported as a result of bringing over this change for a back reference into ClearCase.

   

Appendix 1: Performance Data in Rational Team Concert 4.0

This section provides some idea of the times to expect to import files and folder using the importer. This is by no means a benchmark, but hopefully it will give some idea of what to expect when importing data using the importer. Your performance will also vary with the speed of the server host, database type, and synchronization host that you use. Additional data will be provided in the future as available.

The tests were run in the following environment, unless otherwise marked:

• Jazz server (4.0; Repository DB: DB2; tomcat): Windows Server 2008, 64-bit Operating System; CPU 3.66 GHz (4 processors); RAM 8 GB
• Sync host Linux machine: RedHat Linux server 5.4; x86_64; CPU: 3 GHz; RAM 3 GB
• ClearCase 8.0.1
• ClearCase data from ClearCase production VOB at IBM

Import of UCM baseline history (large data set)

Tests on importing all baselines (321) and a subset of baselines ( 2 of 321) were performed on the Linux sync host. To import a subset of baselines, an attribute type was first created in the UCM VOB; e.g.:

  cleartool mkattype -global -nc -shared -vtype string BaselineHistoryImporter  

The attribute type was then applied to a subset of baselines to mark them for importing; for example, on Windows this would look like:

  cleartool mkattr -nc BaselineHistoryImporter "True" baseline:Int1.C1.BL6@pvob  cleartool mkattr -nc BaselineHistoryImporter "True" baseline:Int1.C1.BL7@pvob  ...  

Import of all baselines of a component

  Total sync time:   6 hours 38 minutes 49 seconds  Number of baselines imported:   321  Number of files created:  3818  Number of files updated:   4083  Number of folders created:   672  Number of folders updated:   102  Number of symbolic link created: 432     Total changes written to RTC (file/folder/symlink created/updated): 9107    Average time for an update/create (file or folder):    2.63 seconds     Average time to import one baseline:   1.24 minutes 

Import of 2 baselines of a component

  Total sync time:   59 minutes 33 seconds  Number of baselines imported:   2  Number of files created:   3735  Number of files updated:   123  Number of folders created:  670  Number of folders updated:    32  Number of symbolic links created:   417    Total changes written to RTC (file/folder/symlink created/updated): 4977    Average time for an update/create (file or folder):   0.72 seconds    Average time to import a baseline:  30 minutes

Import of base ClearCase history (small data set)

The VOB was prepared for import by creating a string-valued attribute type:

  cleartool mkattype -global -shared -nc -vtype string basecc.importer  

and then applying instances of the attribute to the labels desired for import, indicating the order in which the labels should be supported in the value of the attribute:

  cleartool mkattr -nc basecc.importer "1" lbtype:BL1@basevob  cleartool mkattr -nc basecc.importer "2" lbtype:BL2@basevob ...  

and then finally, locking the label type:

  cleartool lock lbtype:BL1@basevob  

Import of base ClearCase data

  Number of lbtypes imported:   49  Total sync time:  39 minutes 49 seconds     Number of files created:   97  Number of files updated:   93  Number of folders created:   17  Number of folders updated:   1  Number of symbolic link created: 17    Total changes written to RTC (file/folder/symlink created/updated):  225    Average time for an update/create (file or folder):   10.6 seconds  Average time to import one label type:   0.82  minutes  

Appendix 2: Performance Data in Rational Team Concert 3.0.1

The tests were run in the following environment, unless otherwise marked:

• Jazz server (3.0.1; Repository DB: DB2; tomcat): Windows Server 2008, 64-bit Operating System; CPU 3.66 GHz (4 processors); RAM 8 GB
• Sync host Linux machine: RedHat Linux server 5.4; x86_64; CPU: 3 GHz; RAM 3 GB
• ClearCase 7.1.2
• ClearCase data from ClearCase production VOB at IBM

Import of UCM baseline history (large data set)

Tests on importing all baselines (316) and a subset of baselines ( 2 of 316) were performed on the Linux sync host. To import a subset of baselines, an attribute type was first created in the UCM VOB; e.g.:

  cleartool mkattype -global -nc -shared -vtype string BaselineHistoryImporter  

The attribute type was then applied to a subset of baselines to mark them for importing; for example, on Windows this would look like:

  cleartool mkattr -nc BaselineHistoryImporter "True" baseline:Int1.C1.BL6@pvob  cleartool mkattr -nc BaselineHistoryImporter "True" baseline:Int1.C1.BL7@pvob  ...  

Import of all baselines of a component

  Total sync time:   12 hours 47 minutes 2 seconds  Number of baselines imported:   316  Number of files created:  3818  Number of files updated:   4083  Number of folders created:   672  Number of folders updated:   102  Number of symbolic link created: 432     Total changes written to RTC (file/folder/symlink created/updated): 9107    Average time for an update/create (file or folder):    4.92 seconds     Average time to import one baseline:   2.43 minutes 

Import of 2 baselines of a component

  Total sync time:   2 hours 5 minutes  Number of baselines imported:   2  Number of files created:   3735  Number of files updated:   123  Number of folders created:  670  Number of folders updated:    32  Number of symbolic links created:   417    Total changes written to RTC (file/folder/symlink created/updated): 4977    Average time for an update/create (file or folder):   1.79 seconds    Average time to import a baseline:  74 minutes

Import of base ClearCase history (small data set)

Import of base ClearCase data

  Number of lbtypes imported:   49  Total sync time:   1 hours 37 minutes     Number of files created:   97  Number of files updated:   93  Number of folders created:   17  Number of folders updated:   1  Number of symbolic link created: 17    Total changes written to RTC (file/folder/symlink created/updated):  225    Average time for an update/create (file or folder):   25.9 seconds  Average time to import one label type:   1.98 minutes  

Appendix 3: Performance Data in Rational Team Concert 2.0

The tests were run in the following environment, unless otherwise marked:

• Local Jazz server (Rational Team Concert 2.0)
• Sync host on the same windows XP machine as Jazz server (CPU 3.00GHz , RAM 3.75GB)
• Jazz repository DB: Derby
• ClearCase 7.1
• ClearCase data from ClearCase production VOB at IBM

Import of UCM baseline history (large data set)

Tests on importing all baselines (222) and subsets of baselines (30 of 222 and 2 of 222) were performed.

Import of all baselines of a component

  Total sync time:   20 hours 3 minutes (DB2, RTC 2.0.0.1)  Total sync time:   32 hours 5 minutes (Derby, RTC 2.0)  Number of baselines imported:   222  Number of files created:  3818  Number of files updated:   4083  Number of folders created:   672  Number of folders updated:   83  Number of symbolic link created: 432     Total changes written to RTC (file/folder/symlink created/updated): 9088    Average time for an update/create:    7.94 seconds (DB2, RTC 2.0.0.1)  Average time for an update/create:   12.7 seconds  (Derby, RTC 2.0)    Average time to import one baseline:   5.42 minutes (DB2, RTC 2.0.0.1)  Average time to import one baseline:   8.67 minutes (Derby, RTC 2.0)  

Import of 30 baselines of a component

  Total Sync Time:   7 hours 50 minutes  Number of Baselines Imported:   30  Number of files created:   4276  Number of files updated:   2161  Number of folders created:   694  Number of folders updated:   43    Total changes written to RTC (file/folder/symlink created/updated): 7174    Average time for an update/create (file or folder):   3.93 seconds  Average time to import a baseline:   15.66 minutes  

Import of 2 baselines of a component

  Total sync time:   3 hours 20 minutes  Number of baselines imported:   2  Number of files created:   3735  Number of files updated:   123  Number of folders created:  174  Number of folders updated:    508  Number of symbolic links created:   52    Total changes written to RTC (file/folder/symlink created/updated): 4592    Average time for an update/create (file or folder):   2.61 seconds  Average time to import a baseline:  100 minutes  

Import of base ClearCase history (small data set)

  Number of lbtypes imported:   49  Total sync time:   1 hours 50 minutes     Number of files created:   97  Number of files updated:   93  Number of folders created:   17  Number of folders updated:   1  Number of symbolic link created: 17    Total changes written to RTC (file/folder/symlink created/updated):  225    Average time for an update/create (file or folder):   29.3 seconds  Average time to import one label type:   2.04 minutes  

[1]For additional information, refer Section Configuring and using the ClearCase Connector for Jazz source control (under Managing change and releases) at Rational Team Concert InfoCenter at http://www-1.ibm.com/support/docview.wss?rs=3488&uid=swg27012781.[top]

[2]These instructions can be applied to a UNIX environment wherever appropriate.[top]

[3]This method is applicable only when no synchronized streams are present in the currently selected Team Area.[top]

[4]Since the UCM History Importer attempts to roll forward the history of the stream starting from the first available baseline of the component in the stream, if a folder that is not visible to all baselines is selected, the importer algorithm will remove it as a synchronization root when it becomes invisible. When this happens, no changes from subsequent baselines of the component will be brought into Rational Team Concert. For small UCM components, users can select the component root as the synchronization root for the UCM History Importer.[top]

[5]For base ClearCase, the importer sorts all label types based on the applied attribute’s value. This makes the implementation independent of the creation date of the label type. This requirement also helps determine the order of processing of the label types and is useful in cases when label types are created in a certain order but not applied in the same order.[top]

[6]Since the base ClearCase importer attempts to roll forward the history of the VOB starting from the first label type (according to the attribute numbering), if a folder that is not visible in all labels is selected to be imported, the importer algorithm will remove it as a synchronization root when it becomes invisible. When this happens, no changes from subsequent label types will be brought into Rational Team Concert. For small VOBs, user can select the VOB root as the synchronization root for a base ClearCase importer.[top]

---

Copyright © 2011, 2012 IBM Corporation