It's all about the answers!

Ask a question

Is there a way to run a full ETL load automatically?


Andre Gusmao (802640) | asked Feb 12 '15, 12:05 p.m.
During a few months the results from the partial loads were not being monitored, and it turns out that some daily jobs kept failing for months until someone noticed it.
We identified and fixed the issues, and now the daily jobs are finishing successfully on a daily basis. 
However, we noticed some gaps in the DW database because some reports containing records created during the period the jobs were failing are incomplete. 
I understand I need to run a full ETL load to fix this, since the documentation says the full load job rebuilds all the data warehouse tables from scratch.
My question is: is there a way I can start the full load so that all jobs will be executed automatically? Or do I have to run them one by one manually, obeying this sequence:
  • Jazz Team Server/Common and Repository
  • Rational® Team Concert/Common and Repository
  • Rational Quality Manager/Common and Repository
  • Rational Team Concert/APT and SCM
  • Rational Team Concert/Work Items and Build
  • Rational Quality Manager/Work Items
  • Rational Quality Manager/Quality Management
  • Jazz Team Server/Requirements
  • Jazz Team Server/Star
It is not clear to me if the option "Run all data warehouse collection jobs for all applications" starts a partial load or a full load.

Thanks!

Comments
Andre Gusmao commented Feb 12 '15, 12:08 p.m.

 I forgot to mention that I was hoping the upgrade to 5.0.2 could solve this, since there are some migration jobs that will run before the actual partial load jobs. But it didn't change anything.

Accepted answer


permanent link
Francesco Chiossi (5.7k11119) | answered Feb 13 '15, 4:57 a.m.
Hello Andre,

if you are using the Default CLM data collection jobs the full load can only be triggered manually job by job.
The execution order is the one you mentioned.

DCC and Data Magager ETL offers a possibility for run the full load for all applications as a batch.

Best Regards,

Francesco Chiossi

Andre Gusmao selected this answer as the correct answer

Comments
Andre Gusmao commented Feb 13 '15, 9:36 a.m.

Thanks Francesco! This is yet another reason to move to the new DCC ASAP.

One other answer



permanent link
Jackie Albert (1.6k14947) | answered Feb 12 '15, 1:22 p.m.
Andre,
Are you using the CLM data collection jobs (from the RTC server Reports tab),  the Data Manager ETL's, or the Data Collection Component (DCC) to populate your data warehouse?  It's not entirely clear from your explanation. 

Comments
Andre Gusmao commented Feb 12 '15, 1:58 p.m.

Hi Jackie,

sorry, I forgot to mention that we are still using the Default CLM data collection
(Java ETL in CLM). The plan is to move to the new DCC after we finish configuring and validating it, but in the meantime we need to figure out a solution for these gaps in the DW database.

Thanks!

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.