How long should a full data warehouse run take? Is 25 hours for 20,000 requirements reasonable?
issue: Time it takes for a full data warehouse build. Currently to run a full data warehouse, for 20,000 requirements it is taking 25 hours. What is best practice? Is 25 hour reasonable?
3 questions:
What is the threshold for an ingestion?
What do we need to fix the current situation ? Reduce the number of requirements per run?
What do we need to fix the situation long term?
3 questions:
What is the threshold for an ingestion?
What do we need to fix the current situation ? Reduce the number of requirements per run?
What do we need to fix the situation long term?
One answer
Hello Sue,
if by "full" you mean a full data warehouse ETL (reloading all the data as opposed to just the changes since last load) this is an operation that in normal circumstances is only done one time.
The time the ETL takes to run is depending on a large number of factors (version of CLM, hardware resources, which tool you are using, age of the repository, custom attributes, ....) so it's difficult to estimate if the time is reasonable or not and to provide recommendations.
In general archiving unused projects can help with the performances of all the ETL methods.
Best Regards,
Francesco Chiossi
if by "full" you mean a full data warehouse ETL (reloading all the data as opposed to just the changes since last load) this is an operation that in normal circumstances is only done one time.
The time the ETL takes to run is depending on a large number of factors (version of CLM, hardware resources, which tool you are using, age of the repository, custom attributes, ....) so it's difficult to estimate if the time is reasonable or not and to provide recommendations.
In general archiving unused projects can help with the performances of all the ETL methods.
Best Regards,
Francesco Chiossi
Comments
Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Feb 04 '16, 8:34 a.m.If you ask these kinds of questions, it is always advisable to provide information about