Recovering Collaborative Lifecyle Management application data for production restore
In this article we are going to present how to recover Collaborative Lifecycle Management application data in case of any disaster like any accidental deletion of records. One example is an accidental deletion of Rational Quality Manager Test Execution Records (TER) and recovering those on the production server. The purpose of this article is to provide more details about building a temporary server with the required backup containing the deleted items considering the limitations of maintaining production server hostname and public URI of the application. This article limits the explanation to building the temporary server with recovered data and restoring it with and without production server downtime. Restoration can be done using respective application utility tools.
The production server
For the production server used in this example, Rational Team Concert server (RTC) was deployed using WepSphere Application Server on an AIX server (rtcser1.in.ibm.com) and IBM DB2 server on a different AIX server (rqmdb2.in.ibm.com). Similarly, Rational Quality Manager (RQM) along with Rational Requirements Composer (RRC) was deployed using rqmser1.in.ibm.com and rqmdb2.in.ibm.com with scalable hardware and storage resources. The setup was similar to a Software as a Service (SAAS) cloud service infrastruture with multiple projects using both RTC and RQM. A few RQM projects were integrated with RTC and RRC. In this scenario, one of the RQM project’s complete Test Execution Records (more than 3000 records) was unfortunately deleted by overlooking the filtering option. It had to be quickly restored on the production server to avoid a critical impact to the project. The RQM server was configured using a single DB2 database instance with JTS, QM and DW as database names. Restoring with the latest backup of QM database would impact other RQM projects data integrity on the production server. The following sections explain the methods which can help to restore the deleted Test Execution Records back to the production server with and without taking downtime.
Traditional Restore method – With production downtime
Build a temporary RQM and DB2 server for recovering the deleted RQM Test Execution Records. While copying the deleted Test Execution Records from the latest backup using RQM Copy Utility, the production RQM service should be brought down to avoid data integrity issues for the remaining projects. Due to Public URI restrictions, only the production server with its public URI can successfully run jts/setup. Hence, production server downtime becomes mandatory. The following figure explains the restore steps in detail.
Restoration using temporary Domain Name Server (DNS) to avoid downtime
This method helps to avoid downtime to the production RQM server. Also, the recovered Test Execution Records (TERs) from the latest database backup are copied from a temporary restore RQM server to the production RQM server’s respective project area directly. To achieve this, deploy a temporary Domain Name Server (DNS) using a Linux box and create two DNS records for rqmser1.in.ibm and jazzclm.in.ibm.com. Make sure you use two different IPs to avoid conflicts with the production server. Install AIX on the restore server and configure the /etc/resolve.conf with the nameserver as the Linux server IP. Configure the two IPs with one IP resolving to hostname rqmser1.in.ibm.com and another to jazzclm.in.ibm.com. Make appropriate entries in the /etc/hosts file. Install WebSphere Application server (WAS) and create a WAS profile to configure RQM as per the production server using the same JazzTeamServer. Configure the teamserver.properties file of jts and qm to point to the restored DB2 server. Run jts/setup on the Linux server using the supported web browser. Although you use the public URI similar to production server while accessing it on this Linux server, it points to the restore RQM server. Ensure the JTS and QM databases are configured to point to the restore database server and successfully Test the connection to complete the setup. Make sure you are able to access RQM project areas without any issues and the list of Test Execution Records from the respective RQM project area. The following figure demonstrates the setup.
A proper configuration in the /etc/hosts files can help to avoid setting up a separate Domain Name server (DNS) server. On Linux and Unix, the file path is /etc/hosts and on Windows it is C:WINDOWSsystem32driversetchosts.
The hosts file should contain entries as below on restore RQM WAS server, restore RQM DB2 server and on a Linux or Windows machine to run the jts/setup. 9.184.167.137 rqmser1.in.ibm.com rqmser1 9.184.167.79 jazzclm.in.ibm.com jazzclm 9.184.167.139 swestest5.in.ibm.com swestest5
RQM utility and Restoration command
Please refer to the information section of the RQM utility for more details on how to use the tool. Using RQMCopyUtility, the deleted Test Execution Records were able to get copied from the restore RQM server to the production RQM server. Before executing the RQMCopyUtility command, make sure the computer is pointing to the actual DNS server. It should not point to the temporary DNS server, and the /etc/hosts file should not have any entries as shown in the previous section. Use the following command as a reference. Here 9.184.167.79 (Temporary restore RQM server) is the source and jazzclm.in.ibm.com (production RQM server) is the destination.
java -jar RQMCopyUtility.jar -s="https://9.184.167.79:9444/jazz/service/com.ibm.rqm.integration.servic e.IIntegrationService/resources/Project_Area_Name" -us=Username -pws=Password -d="https://jazzclm.in.ibm.com:9444 /jazz/service/com.ibm.rqm.integration.service.IIntegrationService/resources/Project_Area_name" -ud=Userid -pwd=Password -a=executionresult,executionworkitem -is=testphase:urn:com.ibm.rqm:testphase:121, testphase:urn:com .ibm.rqm:testphase:126,testphase:urn:com.ibm.rqm:testphase:220,testphase:urn:com.ibm.rqm:testphase:172,testphase:urn: com.ibm.rqm:testphase:222,testphase:urn:com.ibm.rqm:testphase:223,testphase:urn:com.ibm.rqm:testphase:224,testphase:urn: com.ibm.rqm:testphase:225,testphase:urn:com.ibm.rqm:testphase:226 -l=importlog7.txt -f -pl=progress7.txt
Known Issues and Limitations
1. While running jts/setup using the Public URI from restore RQM server, could not proceed from Requirements Management setup. Since the RQM project areas were able to access without any issues and also the deleted Test Execution Records were able to access from the respective project area it was ignored.
2. While copying the TERs using RQMCopy Utility, it ignored the existing testcases, test plans, test environments of the TER and duplicated those entries. Hence, we have to manually remove the older entries which were duplicated.
3. After RQM copy the links to defects in test cases and in TER history were missing.
4. Suppose a test plan had test phases / schedules / milestones in it. If any testphases in the test plan are deleted, those entries can be still read through REST APIs. While copying, the REST API tries to read them and gets a 404 error making the copying to fail. To over come this manually search those entries and instruct the copy utility to ignore them using “-is” option. This will not happen with any other RQM artifact such as test case, TER etc.
For more information
- RQMAPI (RQMAPI link from Jazz.net wiki)
- RQM Copy Utility (RQM Copy Utility link from Jazz.net wiki)
- Basics of cloud computing (Navigating the IBM cloud link from developerWorks technical library)
About the authors
Murali Dhandapani is an IBM Certified IT Specialist in Systems Management. He spent 10 years in various facets of IT and has gained experience in Linux, Unix, virtualization, high-availability solutions and Rational Source control management tools. He can be contacted at muralindia@in.ibm.com.
Balasubramanian Manivasagam is a Software Engineering Services Manager at IBM India Software Labs. He has 14 years of experience in Unix (Solaris, AIX), Windows, Storage, Network and Rational ClearCase. He can be contacted at bmanivas@in.ibm.com.
Madan Kumar Shanmugam is a Senior Staff Software Engineer. He has 8 years of experience and works in Rational Client Support. He works mainly on Rational Team Concert, Synergy and BuildForge. He can be contacted at madakuma@in.ibm.com.
Copyright © 2012 IBM Corporation