DCC version 5.0.2 Data Collection Jobs involving History do full loads after every execution even though Delta load since previous run is selected
DCC version 5.0.2 Data Collection Jobs involving History appear to be performing full loads after every execution even though Delta load since previous run is selected. Is this by design? A good example is Build History.
|
Accepted answer
See the Tip: under the Data-Mart Collection section in the Knowlege Center.
[ Rational Reporting / Reporting architecture overview / Data Collection with DCC ] Limit scheduling the Data-mart data collection jobs to run as infrequently as possible, for example at most once a day. Processing Data-mart data collections can take a significant amount of time because Data-mart jobs are not delta jobs and typically insert a lot of data in the data warehouse. Matthew Owenby selected this answer as the correct answer
|
Comments
Do you see "modifiedsince=1969...something" or modifiedsince=1899...something" when viewing the log ( i.e. beginning of time )? It will be in the first dozen lines or so. Do those jobs show Success ?
The History jobs aren't enabled on the 6 DCC I configured in the last few weeks, there's no documentation about what they are/do so I've left them off....
In the 5.0.2 Knowledge Center I came across this :
Note: The trending historical data that the Star ETL collects is not a delta-only data collection job and it can consume a large amount of disk space over time. For the best performance results, do not run the jobs more than once a day.
[ Rational Reporting / Adminstering / Managing data collection .... ]
Kevin,
You are correct with them being disabled by default. I enabled all the jobs and they are showing success. I don't however see the modifiedsince in the logs. This isn't the Star ETL job it's any of the History jobs contained in the Data-Mart Data Collection area of the DCC Web interface.