Will archiving unused project areas make my Insight Data Manager ETL for RTC faster?
One answer
Most of the REST calls used by the RTC ETL has a parameter that excludes data from archived projects, meaning the REST API will not return them.
See the following extract from ri_jdbc.log and the projectAreaArchived=false parameter.
This means the REST API will not return data from archived projects area, reducing the number of records that the ETL has to process.
There are a few exceptions where data is fetched from all project areas like "WORKITEM_CCM"."Link" "WORKITEM_CCM"."DeletedWorkItem" "WORKITEM_CCM"."DeletedLink" "WORKITEM_CCM"."WorkItemTypeMapping" that I think is because when a project area is archived links and deletions still needs to be processed.
As an additional benefit a project area that is archived will have no changes, meaning the delta ETL will never have to process any data from it in the future.
This is achieved with the modifiedsince parameter.
In conclusion, archiving unused project areas can be beneficial to the ETL execution.
Note: It's of course difficult to estimate the magnitude of the change in execution time.
See the following extract from ri_jdbc.log and the projectAreaArchived=false parameter.
12/11/2015 16:10:09,279 INFO main com.ibm.rational.drivers.jdbc.xml.RDSStatement : CRRRE1223I: Preparing the query => SELECT "uniqueId", "modified", "itemId", "archived", "name", "description", "contextId", "href", "qualifiedName", "projectAreaItemId", "id", "DATASOURCE_ID" FROM "WORKITEM_CCM"."Category" WHERE modifiedsince='2015-12-11 15:53:47' AND projectAreaArchived=false
This means the REST API will not return data from archived projects area, reducing the number of records that the ETL has to process.
There are a few exceptions where data is fetched from all project areas like "WORKITEM_CCM"."Link" "WORKITEM_CCM"."DeletedWorkItem" "WORKITEM_CCM"."DeletedLink" "WORKITEM_CCM"."WorkItemTypeMapping" that I think is because when a project area is archived links and deletions still needs to be processed.
As an additional benefit a project area that is archived will have no changes, meaning the delta ETL will never have to process any data from it in the future.
This is achieved with the modifiedsince parameter.
In conclusion, archiving unused project areas can be beneficial to the ETL execution.
Note: It's of course difficult to estimate the magnitude of the change in execution time.