Keep using Data Mart until Data Warehouse ETLs finish (RTC 3.0.1)
Hi,
We are going to migrate our RTC 3.0 to 3.0.1 version. During the time the Data Warehouse ETLs are running some reports will show inaccurate data.
I have two questions:
- Given that ETLs may take several days to run for the first time, is there any way to keep using Data Mart until ETLs finish?
- How much will be the impact in the RTC performance?
I would appreciate any help you may give me.
Regards,
Accepted answer
Leonardo,
Here is a set of steps I once provided to a customer who wanted to do the same thing (continue to have access to the BIRT reports while the transfer is taking place). Since you have already created a DW and may also know how to run jts/setup to configure the use of the new DW, you can skip that information.
Switching to a DW from a Datamart
Here is a set of steps I once provided to a customer who wanted to do the same thing (continue to have access to the BIRT reports while the transfer is taking place). Since you have already created a DW and may also know how to run jts/setup to configure the use of the new DW, you can skip that information.
Switching to a DW from a Datamart
Switching from a 2.x style datamart to the 3.0.1 data warehouse involves at a high-level the following steps. There is a way to continue to have access to the BIRT reports while the transfer is taking place. I have added that to the steps below.
- Creation of the Data Warehouse DB. http://publib.boulder.ibm.com/infocenter/clmhelp/v3r0m1/topic/com.ibm.jazz.install.doc/topics/t_s_server_installation_setup_sql.html
- Run Jts/setup and point to the DW. JTS setup takes care of updating your configuration to use the DW and the JAVA ETL jobs. It will create tables in the DW db and it will take you through the CCM piece of the DW configuration.
- IMPORTANT: Before running the ETLs, go the ccm/admin -> Reports page -> Data Warehouse Connection and set the "Reporting Data Source" property to: Local
- Click Save. Check some CCM reports. If they are not blank, then the setting is working as expected.
- Let the ETL jobs run as scheduled by default at midnight. Do not start them manually as they will need to run in a precise order. This process will transfer the data from the 2.x style datamart to the empty tables in the new DW. Reference: https://jazz.net/wiki/bin/view/Main/CALMReportingMigration
- Check the DW ETL job completion status from the JTS/admin and CCM/admin pages.
- When the ETLs finish running and are green, go the ccm/admin -> Reports page -> Data Warehouse Connection and set the "Reporting Data Source" property to: Automatic
8. The BIRT reports will start using the data warehouse after Step 7 is completed.
3 other answers
The RTC will continue to use DataMart until the ETLs have run once successfully. After that RTC switches to warehouse automatically. You can continue the use the DataMart during this operation (with inaccurate data).
Also, as you rightly pointed out, the first run of the ETLs will be slow. However, subsequent fetches being delta, they will be much faster.
The impact of the RTC performance will depend on the RTC user load on database and the total number of WorkItems (which will define the ETL load). If the number of WIs is very high and there is a busy traffic on RTC servers as well, then it is recommended to have a separate database server for the warehouse.
Thanks for your answer Ankur.
"The RTC will continue to use DataMart until the ETLs have run once successfully". But, how do you run ETLs for the first time? I think that you have to do it manually. And to do this you need to select "Data Warehouse..." in ccm/admin-->Reports-->Data Collections Jobs. And when you choose that option, RTC uses Data Warehouse instead of Data Mart.
So, my question is... is it possible to run Data Warehouse ETLs while using the Data Mart?.
Regards,