Is it possible to disable change events temporarily?
We want to adopt RTC as tool for defect tracking, and to enable seamless operation, we want to import stuff from our current tracking system into RTC. Now, our current database has grown over years, and so the import has to deal with approx. 500.000 work items and millions of comments.
The duration of the import job is approaching 72 hours, which conflicts with the target to do the cutover on a weekend, minimizing impact on the user community.
Now I wonder whether we can squeeze out a couple of hours by having RTC not generate change events for the import-related transactions. Does anyone know if generation of change events can be turned off somehow?
Also, if you have good tips how to speed up a mass import of the mentioned scale, I would be very happy to read about that!
Any help greatly appreciated!
Markus
The duration of the import job is approaching 72 hours, which conflicts with the target to do the cutover on a weekend, minimizing impact on the user community.
Now I wonder whether we can squeeze out a couple of hours by having RTC not generate change events for the import-related transactions. Does anyone know if generation of change events can be turned off somehow?
Also, if you have good tips how to speed up a mass import of the mentioned scale, I would be very happy to read about that!
Any help greatly appreciated!
Markus
3 answers
Hi Markus.
I don't believe you can disable change events.
Consider breaking up the import job into chunks and importing each chunk in parallel in a separate RTC Client session, possibly on separate client machines. I haven't tried this, but it should work as long as there are no interdependencies between the work items you're importing in the different sessions. You'll want to monitor the load on your server and not overload it.
-Matt
I don't believe you can disable change events.
Consider breaking up the import job into chunks and importing each chunk in parallel in a separate RTC Client session, possibly on separate client machines. I haven't tried this, but it should work as long as there are no interdependencies between the work items you're importing in the different sessions. You'll want to monitor the load on your server and not overload it.
-Matt
Comments
Hi Matt,
thanks for your quick response. We already use multi-threading, but from within a single JVM. This approach is faster than running single-threaded, but as you point out the bottleneck seems to be the database system. That's why I wondered if we could get rid of some database load, and I don't think we'd need change events for imported data.
Thanks for your help,
Markus
What version of RTC and where are you running the import from? Collocation for the purpose of the import has helped some, but it really depends on the setup, system and network capabilities as to whether the tradeoffs in performance would work for this case. There are a lot of basic checks for optimizations/interference that may pop up something that can yield better results as well. From a system perspective, check firewalls/scanning tools to make sure you aren't getting any slowdown/interference on the operations, memory/cpu/disk/temp space isn't being used up, check the logging and activity on the DB server to make sure it isn't throttling, check the network transfer speed to make sure it is where you expect, etc.