Jazz Forum Welcome to the Jazz Community Forum Connect and collaborate with IBM Engineering experts and users

Is there a way to run a full ETL load automatically?

Questions
Tags
Users
Badges
Andre Gusmao
(80●26●40) Feb 12 '15, 12:05 p.m.
During a few months the results from the partial loads were not being monitored, and it turns out that some daily jobs kept failing for months until someone noticed it.
We identified and fixed the issues, and now the daily jobs are finishing successfully on a daily basis. 
However, we noticed some gaps in the DW database because some reports containing records created during the period the jobs were failing are incomplete. 
I understand I need to run a full ETL load to fix this, since the documentation says the full load job rebuilds all the data warehouse tables from scratch.
My question is: is there a way I can start the full load so that all jobs will be executed automatically? Or do I have to run them one by one manually, obeying this sequence:
  • Jazz Team Server/Common and Repository
  • Rational® Team Concert/Common and Repository
  • Rational Quality Manager/Common and Repository
  • Rational Team Concert/APT and SCM
  • Rational Team Concert/Work Items and Build
  • Rational Quality Manager/Work Items
  • Rational Quality Manager/Quality Management
  • Jazz Team Server/Requirements
  • Jazz Team Server/Star
It is not clear to me if the option "Run all data warehouse collection jobs for all applications" starts a partial load or a full load.

Thanks!

0 votes

Comments
Andre Gusmao
Feb 12 '15, 12:08 p.m.

 I forgot to mention that I was hoping the upgrade to 5.0.2 could solve this, since there are some migration jobs that will run before the actual partial load jobs. But it didn't change anything.

2 answers
4,518 views
0 votes

Accepted answer

Permanent link
Francesco Chiossi
(5.7k●1●11●19) Feb 13 '15, 4:57 a.m.
Hello Andre,

if you are using the Default CLM data collection jobs the full load can only be triggered manually job by job.
The execution order is the one you mentioned.

DCC and Data Magager ETL offers a possibility for run the full load for all applications as a batch.

Best Regards,

Francesco Chiossi

Andre Gusmao selected this answer as the correct answer

2 votes

Comments
Andre Gusmao
Feb 13 '15, 9:36 a.m.

Thanks Francesco! This is yet another reason to move to the new DCC ASAP.

One other answer

Permanent link
Jackie Albert
(1.6k●1●49●47) Feb 12 '15, 1:22 p.m.
Andre,
Are you using the CLM data collection jobs (from the RTC server Reports tab),  the Data Manager ETL's, or the Data Collection Component (DCC) to populate your data warehouse?  It's not entirely clear from your explanation. 

0 votes

Comments
Andre Gusmao
Feb 12 '15, 1:58 p.m.

Hi Jackie,

sorry, I forgot to mention that we are still using the Default CLM data collection
(Java ETL in CLM). The plan is to move to the new DCC after we finish configuring and validating it, but in the meantime we need to figure out a solution for these gaps in the DW database.

Thanks!

Your answer

Register or log in to post your answer.

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Ask a question
Guidelines
FAQ
Search context
Follow this question

By Email: 

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers
Answers and Comments
Question details
× 403

Question asked: Feb 12 '15, 12:05 p.m.

Question was seen: 4,518 times

Last updated: Feb 13 '15, 9:36 a.m.

Confirmation Cancel Confirm