Reducing the size of the Rational Team Concert repository database
Abstract
This article explains how to reduce the disk space of the Rational Team Concert (RTC) repository database (DB2). RTC offers to delete the contents of source control artifacts and work item attachments. However, in the case of DB2, it is not directly reflected in the size of database. To reduce the size of the database, the database administrator needs to not only run the database reorganization, but also needs to set the high water mark value to a lower one. This article explains the basic feature available in RTC for deleting the contents in both source control and work items, and basic theory of database size reclaim.
Introduction
Rational Team Concert is a tool set to provide a source control system, work item management, software build management, and work planning features. Rational Team Concert works as a web system to provide these features, and stores artifacts into repositories such as the IBM DB2 Universal Database. RTC’s source control is primarily designed to store source code, but there is no restriction regarding which contents are managed in the source control system. For example, some projects might need to store design documents such as .doc, .xls or .ppt files. These files tend to be large in size and the growth rate of the repository size might be faster than expected. As a result, there may be potential issues such as a lack of sufficient disk space, or reduced performance in the repository. Some artifacts stored in RTC may need to be deleted to prevent potential issues. There may be different use cases in which the user may want to delete content.Some examples include:
- The storage area of the repository will not have enough disk space, and expanding disk capacity can not be done immediately. The administrator may want to delete unnecessary artifacts so that disk space can be reclaimed until hardware can be added to expand disk capacity.
- A user wants to maintain the latest attachment to the work item so that the attachments do not confuse developers. Or a user, by mistake, attached a file which included confidential information.
- There is a contract in an outsourcing situation that some content must be deleted before the contract ends. For example, test data might be borrowed during the contract period, and this data must be deleted.
Likewise, work item attachments can also be deleted from the client for Eclipse IDE. Note, the “Remove” operation is different from the “Delete” operation. For example, even if you just remove the attachment, the attachment itself is stored in the repository.
When users delete content, they may assume that disk space has been reclaimed. However, in the case of DB2, these delete operations do not directly reflect the size of repository database. This is because the storage used to store artifact is Database Managed Space (DMS). Unlike System Management Space (SMS), DMS does not reclaim free space, but instead marks it as Free Pages. The sample storage is shown in list 1:
> db2 connect CCM
db2 list tablespaces
..
Tablespace ID = 4
Name = CONTENTS
Type = Database managed space
Contents = All permanent data. Large table space.
State = 0x0000
Detailed explanation:
Normal
Total pages = 250000
Useable pages = 249952
Used pages = 53632
Free pages = 196320
High water mark (pages) = 53632
Page size (bytes) = 16384
Extent size (pages) = 32
Prefetch size (pages) = 32
Number of containers = 1
...
List 1 shows that CONTENTS table space type is database managed space. This table space is used to store the artifacts of Rational Team Concert. Note that in this example, Used pages and High water mark (pages) are equal in value. The high water mark refers to the page number of the first page in the extent following the last allocated extent. The high water mark is not an indicator of the number of used pages because some of the extents below the high-water mark might have been freed as a result of deleting data. In this case, even though there might be free pages below it, the high water mark remains as the highest allocated page in the table space.
For example, if you delete the contents of table space, it may decrease the Used pages, however the high water mark value remains the same or even increases. This has implications of database table space reorganization. Even though we issue the command “db2 reorg”, the size of the table space remains the same due to this high water mark.
The DB2 reference manual offer several ways to “Reclaimable storage”. In this article, we verified one way of using “Lowering high water mark method”.
Scenario
The scenario we have evaluated was as follows:- User adds large files into the RTC repository several times.
- User deletes SCM artifacts by using the “scm delete” command.
- Reclaim the table space size by reducing high water mark.
First of all, we recorded some initial status. Figure 1 shows the initial status before artifacts are checked-in and delivered to the stream.
Figure 1 – Original table information before file check-in.
The “Used pages” currently sits at 1, and available disk space was 47%. We added about 750 MByte binary files to the repository. Figure 2 shows the table space information after the SCM check-in and deliver operation.
Figure 2 – Table space size after “scm check-in and deliver”.
The table page size has increased and available disk space become 50%.
We then deleted the file contents. The “scm delete” command was used to delete the contents as follows:
scm delete state-content --force -r <repository url=""> <file uuid=""> <state uuid="">
e.g.
scm delete state-content --force -r https://9.188.255.195:9443/ccm/ _lngT4ISsEeOB4d7ZOucnhA _Gh5RVIStEeOVvb8c-QPOAw
</state></file></repository>
Then the “db2 reorg” command was issued. Figure 3 shows the table and disk space information before the “db2 reorg” and Figure 4 shows the table information after the “db2 reorg” command.
Figure 3 – table information before “db2 reorg”.
Figure 4 – table information after “db2 reorg”.
Figure 3 indicated that the “Used pages” and “High water mark (pages)” values are equal. The image on the right shows the disk space. Figure 4 shows that “Used pages” was lowered, but “High water mark (pages)” have increased. Due to this, the disk size remained the same, at 50%.
Then we issued the command to lower the high water mark, as follows:
db2 alter tablespace contentts lower high water mark
Figure 5 shows the table space information and disk size information. Note that “High water mark (pages)” has been reduced, but the disk size is still the same.
Figure 5 – table information and disk space information after lowering high water mark.
To actually reduce the size of the table space, we need to run “db2 later_tablespace”, as follows:
db2 alter tablespace contentts reduce (FILE '<container><container name>' <size to reduce> <size to="" reduce="">)
</size></container>
Note that we can not reduce the total page size so that it is smaller than the “High water mark” setting. Considering that we have 866MB of free space, we specified 866MB. The size of free space was calculated according this formula:
<size to reduce> <size to="" reduce=""> = (page size) * (free pages) = 16K * 55424 / 1024 = 866MB.
</size>
Figure 6 shows that table information and disk space information.
Figure 6 – table space and disk space information after db2 alter_tablespace command.
As you can see, not only have “Free pages” been reduced, but also “Total pages” are reduced. As a result, disk space was reclaimed, and we now have 44% usage.
Summary
In this article, the author and technical contributors explained about basic theory and scenario to reduce the disk size of the Rational Team Concert repository database. This method has been verified, though we do recommend testing the procedure in a test environment prior to applying it to a production database. We also recommend that you work with the IBM technical team or client support team to fully understand the DB2 table space reclaim method. We need to choose the best option case by case.
For more information
- DB2 inforcenter – Table space and storage management / Reclaimable storage
- Rational Team Concert – delete state-content
- Rational Team Concert – Deleting Content From Source Control in Rational Team Concert
About the author and technical contributors
Takehiko Amano works as a member Rational Emerging Technology Team. Current interest area is JazzHub and DevOps, and emerging technology from IBM.
Shun Haraguchi works as a member Rational Technical Support Team. Current interest area is SDP, DevOps, and RIA.
Yasuyuki Kubota is a member of Rational Technical Support Team and in charge of the PoC of UrbanCode Products in Japan. Current interest area is DevOps and Mobile.
Copyright © 2014 IBM Corporation
Yes