DCC version 5.0.2 Data Collection Jobs involving History do full loads after every execution even though Delta load since previous run is selected
Accepted answer
See the Tip: under the Data-Mart Collection section in the Knowlege Center.
[ Rational Reporting / Reporting architecture overview / Data Collection with DCC ]
[ Rational Reporting / Reporting architecture overview / Data Collection with DCC ]
Limit scheduling the Data-mart data collection jobs to run as infrequently as possible, for example at most once a day. Processing Data-mart data collections can take a significant amount of time because Data-mart jobs are not delta jobs and typically insert a lot of data in the data warehouse.
Comments
Kevin Ramer
Aug 21 '15, 3:02 p.m.Do you see "modifiedsince=1969...something" or modifiedsince=1899...something" when viewing the log ( i.e. beginning of time )? It will be in the first dozen lines or so. Do those jobs show Success ?
The History jobs aren't enabled on the 6 DCC I configured in the last few weeks, there's no documentation about what they are/do so I've left them off....
Kevin Ramer
Aug 21 '15, 3:15 p.m.In the 5.0.2 Knowledge Center I came across this :
Note: The trending historical data that the Star ETL collects is not a delta-only data collection job and it can consume a large amount of disk space over time. For the best performance results, do not run the jobs more than once a day.
[ Rational Reporting / Adminstering / Managing data collection .... ]
Matthew Owenby
Aug 21 '15, 4:32 p.m.Kevin,
You are correct with them being disabled by default. I enabled all the jobs and they are showing success. I don't however see the modifiedsince in the logs. This isn't the Star ETL job it's any of the History jobs contained in the Data-Mart Data Collection area of the DCC Web interface.