loading large components
I noticed today that a 10MB/2.000 files upload (sharing project) between a
1.0.1.1 client and a server over an MPLS cloud was about 60x faster (less than 1 minute) than a 150MB/20.000 files upload between the same sites. Something doesn't seem to scale linearly in RTC. Can anyone reproduce this? Any comments? Thanks, Ton van Velzen |
4 answers
While we do not have a hard limit restricting the size of a share
operation in Jazz Source Control, the larger the share you do, the more rows get written in the database (potentially 7N for the N files/folders), which adds contention to the system, a very long running transaction, and potentially cause other problems like: - content gets cleaned up before it is claimed in the checkin - transaction log or buffer pools may unexpectedly become exceeded That being said, the better hardware/disk that you use, the higher that limit is. We are in the process of figuring out the means of determining that threshold for the deployment. Once we have that data, we can restructure our client applications to chunk large shares into chunks that perform best for the system. Thanks JohnC SCM Server Ton van Velzen wrote: I noticed today that a 10MB/2.000 files upload (sharing project) between |
While we do not have a hard limit restricting the size of a share I noticed today that a 10MB/2.000 files upload (sharing project) between So - it sounds like there might be some tuning on the database - buffer pools, etc that we could think about to cover this kind of situation. I suppose the good news is that sharing a project (that initial upload with that many files) is a relatively rare event in the lifecycle of a real project - it just happens to be the one you see first :-) anthony |
Thanks for the reply. It would be good to be able to follow this work. Under
what workitem is this work being tracked? When is this client restructuring expected to become available? Will it be part of the 2.0 release of RTC? Thanks, Ton. johnc wrote: While we do not have a hard limit restricting the size of a share |
Agreed that the upload of large data sets is the first thing you see. I don't
mean to be pessimistic, but downloading large data sets not only takes a noticeable amount of time when you do it frequently (for multiplatform tests, e.g.), but large data sets invite large/complex change sets too. I suspect that updating rows is not less expensive than inserting rows. So, largeness will not go away once you have loaded the big chunks up. Once you got it, largeness is there to stay. The notion of large components is a reality for many customers. They use RTC in a way that RTC ideally is able to be tuned to. Large chunk usage patterns may be quite different from the way other customers will use it. performance tuning is a very good idea for large or atypical installations. kesterto wrote: johncwrote: So - it sounds like there might be some tuning on the database - buffer pools, etc that we could think about to cover this kind of situation. I suppose the good news is that sharing a project (that initial upload with that many files) is a relatively rare event in the lifecycle of a real project - it just happens to be the one you see first :-) anthony |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.