Repotools Verify command - additional guidance

Authors: PaulEllis, ZeeshanChoudhry
Build basis: None.

The 6.0.5 release of Collaborative Lifecycle Management seeks to expand the published knowledge to advise on when to run the repository tools command to verify the integrity of a database.

The command "repotools -verify" can be used to verify your database as part of a weekly/monthly maintenance schedule, or prior to upgrade or other major deployment activities. This command allows logical verification of objects, as well as some mid-level concepts such as context checking.

Use the repotools verify command to verify the integrity of a database. There are different levels for use with this command, using what level; and document the error messages so you can contextualise the seriousness of the warning/errors.

Purpose
The verify command is used for contents verification in the database with verification levels set between number one and ten.

What level should I use?

Each component uses these levels with different verbose checks so its hard to be definitive which level would do what for Builds or SCM component for example. Level 5 should be the base to start with if verification is needed.

Level one is the minimum verification level. Level one for example, verifies if the item can be fetched by simple fetch call at a very top level. If some components show major problems, then you can start with a higher level to get more verbose output.

Level ten is the maximum verification level. When the level is set to ten, it looks for the particular content of the component in all database tables and verifies their validity. Only run level 10, if instructed to do so by IBM Support. This level will produce very verbose output and take some time to complete.
It is expected that you run this command as a scheduled task on at least a monthly basis. It is especially recommended that you run this command prior to an upgrade to ensure that any output is verified by IBM Support. Therefore, it is also recommended to perform the command with plenty of time to allow analysis and resolution of any more serious errors (so not 4 hours prior to upgrade!).

Types of errors

This page will summarise those errors that could be returned from the tool. Those errors that are non-serious and do not currently have a technote elsewhere are explicitly listed. There are some errors which were documented prior to this page, which will reference the published technote.

1) CRRTC3004E errors

Known examples of this error are:

• CRRTC3004E: Build component item of type BuildResult with item ID .......
• CRRTC3004E: Build component item of type BuildEngine with item ID .....
• CRRTC3004E: Build component item of type BuildRequest with item ID....

Answer: These 3 errors are largely harmless and do not impact the upgrade. The missing BuildEngineActivity item should not cause any problems. When the last contact time for the engine is updated, this item will get recreated if needed. If, however, you do see related errors in the server log, the workaround would be to delete and recreate the build engine.

2) ...was found in the ITEM_CURRENTS table, but was not found in the Item query table

An example of this error would be:

The Item with ItemID "_abCDEfg7HijKlMnWto3lwg" and StateId "_xyz-E8j7EeaQuJlWto3lwg" was found in the ITEM_CURRENTS table, but was not found in the Item query table.

Answer: This error can be ignored. Having this in the Items Current and not in the Query table is not going to cause any issue.

3) ...was found in the ITEM_CURRENTS table, but was not found in the Item query table

An example of this error would be:

The Item with ItemID "_VjyEYckdEeaQuJlWto3lwg" and StateId "_Vr-XEskdEeaQuJlWto3lwg" was not found in the ITEM_STATES table, but was found in the item query table.

Answer: This error can be ignored. Having this item in the States table and not in query table is not going to effect anything.

4) ...was found in the ITEM_CURRENTS table, but the same Item had a different StateId,...

An example of this error would be:

The Item with ItemID "_3Ac5BCdeFgHIjkLm7nO6pQ" and StateId "_-vUbAMclEdeQfJlWgh3ijk" was found in the ITEM_CURRENTS table, but the same Item had a different StateId, "_Y64-zMyxEwvQuJlWts3rqp", in the Item query table

Answer: These sort of error would need Online Verify to check what ItemType this is. In the online verify those items are pointing to Build Results, Build Engines or Build requests. Build engine activity and are harmless for upgrade problems.
However, these can be cleaned up if needed. Send the output from repotools -verify; and Online -verify, to IBM Support via Service Request (SR) if you wish to proceed with removing the cause of these errors.

5) CRJAZ1150E The repository was not verified errors

Known examples of this error are:

• CRJAZ1150E The repository was not verified: Some problems were found while checking the consistency of the Item tables.
• CRJAZ1150E The repository was not verified: Some problems were found in the tables related to the "com.ibm.team.enterprise.scd#ScanResult" Item type.
• CRJAZ1150E The repository was not verified: There were some problems found in the Item query table for Item type "com.ibm.team.enterprise.scd#ScanResult".
• CRJAZ1150E The repository was not verified: There were some problems found in the Item query table for Item type "com.ibm.team.enterprise.scd#ScanResult".
• CRJAZ1150E The repository was not verified: Some problems were found in the tables related to the "com.ibm.team.enterprise.scd#ScanRequest" Item type.
• CRJAZ1150E The repository was not verified: There were some problems found in the Item query table for Item type "com.ibm.team.enterprise.scd#ScanRequest".
• CRJAZ1150E The repository was not verified: Some problems were found in the tables related to the "com.ibm.team.links#AuditableLink" Item type.

Answer: Online verify would problem better to check if its consistency problem or some thing structural is not good enough. We would need to bring in the Enterprise Extensions team to help resolve issues with these tables, so please create a Problem Maintenance Record (PMR) via Service Request (SR) if you encounter similar issues.

6) CRJAZ1150E The repository was not verified...CRRTC5132E errors

Known examples of this error are:

Verifying items present in the "com.ibm.team.scm" component.
CRJAZ1150E The repository was not verified:         SCM Verification failures
CRJAZ1150E The repository was not verified:         CRRTC5132E The _ktNhguc_EeSUf5igoHwgaQ component entry has an invalid historic
 entry index.
CRJAZ1150E The repository was not verified:         CRRTC5132E The _4HP4lMmtEeWqiLLRAUIq4g component entry has an invalid historic
 entry index.

Answer: The mentioned error indicating historical index missing for some SCM item can be safely ignored. We have encountered these after migrations, caused by history state references not handled in a clean way, but they tend not to cause any issues. Run Online verify to check for DB consistency issues.

7) CRJAZ1150E The repository was not verified:...CRRTC5141E

A known example of this error are:

CRJAZ1150E The repository was not verified: CRRTC5141E The following item does not have a persisted query record: FileItem[_3zRTkKUuEd2m-sD01I2KJg]

Answer: The mentioned error can be safely ignored.

8) CRJAZ1150E The repository was not verified: CRRTC5113E

A known example of this error are: CRJAZ1150E The repository was not verified: CRRTC5113E There is a claim for the C4Ejryb1vBt7oAf6_vuwYS4pARgGikuIKn79RDVmyDY content hash, which does not exist in the repository.

Answer: The mentioned error can be safely ignored.

9) CRJAZ1150E The repository was not verified: CRJAZ0215E

CRJAZ1150E The repository was not verified: CRJAZ0215E The following record was not found in the database: com.ibm.team.apt.internal.common.nucleus.impl.IterationPlanRecordHandleImpl@32e06670 (stateId: [UUID _HaKTwckcEeaQuJlWto3lwg], itemId: [UUID _lzAVwPv_EeOPPKTEK6lM1Q], origin: , immutable: )

Answer: These sort of errors would need Online Verify to check what the underlying problem is. Send the output from repotools -verify; and Online -verify, to IBM Support via Service Request (SR) if you wish to proceed with removing the cause of these errors.

If you are seeing this error message in the Rational Team Concert web client, when viewing a work item's results, see: Displaying a work item results in "CRJAZ0215E the following record cannot be found" error in RTC Web client.

10)CRJAZ2221E The repotools command could not be verified or run

Answer: This comment can be found at the end of the verify log. It signifies that there were some errors during verification and at the end you get this error reported. If by any chance there is an exception to run the command , you would also see that error.

11) CRJAZ1150E The repository was not verified: CRJAZ6043E

CRJAZ1150E The repository was not verified: CRJAZ6043E The area cannot be created because its name includes one or more of these invalid characters: & < > " ' /

Answer: See technote, Attempts to upgrade the IBM Rational Team Concert (RTC) results in the error CRJAZ6043E., to resolve this problem prior to upgrading.

Related topics: Online Verify Tool, Repository tools command to verify the integrity of a database

External links:

• IBM

Additional contributors: ElisabethCarbone-Hodel, SusanWu, MichaelAfshar