It's all about the answers!

Ask a question

DCC version 5.0.2 Data Collection Jobs involving History do full loads after every execution even though Delta load since previous run is selected


Matthew Owenby (23311) | asked Aug 21 '15, 2:02 p.m.
edited Aug 21 '15, 2:35 p.m.
DCC version 5.0.2 Data Collection Jobs involving History appear to be performing full loads after every execution even though Delta load since previous run is selected.  Is this by design?  A good example is Build History.

Comments
Kevin Ramer commented Aug 21 '15, 3:02 p.m.

Do you see "modifiedsince=1969...something" or modifiedsince=1899...something" when viewing the log ( i.e. beginning of time )?  It will be in the first dozen lines or so.  Do those jobs show Success ?

The History jobs aren't enabled on the 6 DCC I configured in the last few weeks, there's no documentation about what they are/do so I've left them off.... 


Kevin Ramer commented Aug 21 '15, 3:15 p.m.

In the 5.0.2 Knowledge Center I came across this :

Note: The trending historical data that the Star ETL collects is not a delta-only data collection job and it can consume a large amount of disk space over time. For the best performance results, do not run the jobs more than once a day.

[ Rational Reporting / Adminstering / Managing data collection .... ]


Matthew Owenby commented Aug 21 '15, 4:32 p.m.

Kevin,
You are correct with them being disabled by default.  I enabled all the jobs and they are showing success.   I don't however see the modifiedsince in the logs.  This isn't the Star ETL job it's any of the History jobs contained in the Data-Mart Data Collection area of the DCC Web interface.

Accepted answer


permanent link
Kevin Ramer (4.3k6156174) | answered Aug 25 '15, 8:37 a.m.
See the Tip:  under the Data-Mart Collection section in the Knowlege Center.
[ Rational Reporting / Reporting architecture overview / Data Collection with DCC ]

Limit scheduling the Data-mart data collection jobs to run as infrequently as possible, for example at most once a day. Processing Data-mart data collections can take a significant amount of time because Data-mart jobs are not delta jobs and typically insert a lot of data in the data warehouse.

Matthew Owenby selected this answer as the correct answer

Your answer


Register or to post your answer.