It's all about the answers!

Ask a question

DCC version 5.0.2 Data Collection Jobs involving History do full loads after every execution even though Delta load since previous run is selected


Matthew Owenby (23811) | asked Aug 21 '15, 2:02 p.m.
edited Aug 21 '15, 2:35 p.m.
DCC version 5.0.2 Data Collection Jobs involving History appear to be performing full loads after every execution even though Delta load since previous run is selected.  Is this by design?  A good example is Build History.

Comments
Kevin Ramer commented Aug 21 '15, 3:02 p.m.

Do you see "modifiedsince=1969...something" or modifiedsince=1899...something" when viewing the log ( i.e. beginning of time )?  It will be in the first dozen lines or so.  Do those jobs show Success ?

The History jobs aren't enabled on the 6 DCC I configured in the last few weeks, there's no documentation about what they are/do so I've left them off.... 


Kevin Ramer commented Aug 21 '15, 3:15 p.m.

In the 5.0.2 Knowledge Center I came across this :

Note: The trending historical data that the Star ETL collects is not a delta-only data collection job and it can consume a large amount of disk space over time. For the best performance results, do not run the jobs more than once a day.

[ Rational Reporting / Adminstering / Managing data collection .... ]


Matthew Owenby commented Aug 21 '15, 4:32 p.m.

Kevin,
You are correct with them being disabled by default.  I enabled all the jobs and they are showing success.   I don't however see the modifiedsince in the logs.  This isn't the Star ETL job it's any of the History jobs contained in the Data-Mart Data Collection area of the DCC Web interface.

Accepted answer


permanent link
Kevin Ramer (4.5k8184200) | answered Aug 25 '15, 8:37 a.m.
See the Tip:  under the Data-Mart Collection section in the Knowlege Center.
[ Rational Reporting / Reporting architecture overview / Data Collection with DCC ]

Limit scheduling the Data-mart data collection jobs to run as infrequently as possible, for example at most once a day. Processing Data-mart data collections can take a significant amount of time because Data-mart jobs are not delta jobs and typically insert a lot of data in the data warehouse.

Matthew Owenby selected this answer as the correct answer

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.