Slow loading speed when using loadrule file.

Marian Marinov

(11●1●1) edited

Jul 05 '17, 2:33 p.m.

Hello,

We have huge problems with the speed of loading a workspace when using a loadrule file. If we are not using loadrule file the load speed is normal, but when we try with the loadrule it takes about 10minutes(sometimes more) only at one step "Checking load completion conditions".

Can you help me find out what could be the reason of such big delay. After this delay of aobut 10minutes the speed of loading is the same as without loadrule file(about 5 minutes at most).

Thank you.

0 votes

Comments

Thomas Kirstätter

Jul 03 '17, 4:47 p.m.

can you give some more hints on the size of the WS? number of components? overall size on disc? Do you use hierarchical components? RTC Version?

Marian Marinov

Jul 04 '17, 9:40 a.m.

The RTC Version we are using is 6.0.2. The components in the stream are about 200. The size of the whole stream is about 11GB. We have several loadrule files some of them are getting about 100mb others about 230mb. The tests I'm making are with loadrule that gets about 100mb(2200 files). We are using hierarchical components only for structuring the components, but those are not used within the loadrule files. The loadrule files are getting the files directly from the leaf components. Can you tell me, how the hierarchical components influence the loading time?

David Lafreniere

FORUM MODERATOR / JAZZ DEVELOPER Jul 05 '17, 2:41 p.m.

Hierarchical components should not impact the load time; although it would be nice to confirm this. Can you test this by flattening the entire workspace in a repository workspace and then try loading? When done you can discard the component hierarchy changes so that they are not accidentally delivered to the stream. Also, can you describe exactly how you would manually load without the load rule? And is there any chance you can attach/post a snippet of how the load rule looks? I'm wondering if when you manually load, you select a root folder or component for example, so that it loads everything into one 'share root' folder, whereas when the load rule is used, it loads every sibling file/folder as a separate 'share root'.

Marian Marinov

Jul 22 '17, 1:51 p.m.

I can confirm that the problem is not coming from the hierarchical components. About the loadrule files I think there is a misunderstanding. We are using loadrule files. When I said we were not using hierarchical components within the loadrule files I meant that we are not loading parent components and expecting automatically loading of all the subcomponents. Basically what we do within the loadrule files is: downloading whole folders from a component to a specific path within the build structure: <parentLoadRule> <component name="RTC_CMP_NAME_1"/>  <parentFolder repositoryPath="/CMP_FOLDER_TO_DOWNLOAD"/>  <sandboxRelativePath includeRepositoryPath="true" pathPrefix="/Path_in_the_build_structure"/> </parentLoadRule> or downloading specific files from component to specific places within the build structure <itemLoadRule> <component name="RTC_CMP_NAME_2"/>  <item repositoryPath="/Path_to_File_name_within_the_component"/>  <sandboxRelativePath includeRepositoryPath="true" pathPrefix="/Place_within_the_build_structure_where_the_file_should_go"/> </itemLoadRule>

Marian Marinov

Jul 06 '17, 11:20 a.m.

David do you have information what exactly "Checking load completion conditions" means? This step is really short when not using loadrule file and it takes ages if we use loadrule files. Maybe we could somehow modify the loadrule file to make it faster. In addition the loadrule file is about 3000lines long together with all the comments.

David Lafreniere

FORUM MODERATOR / JAZZ DEVELOPER Jul 06 '17, 2:29 p.m.

"Checking load completion conditions" is an Eclipse job that the scm code runs before fetching the actual content; it evaluates the load request (ex: to confirm there is no overlap in terms of existing sandboxes or shares on disk), I would say the length of time this takes would be proportional to the number of shares ('parent' files or folders being loaded). That is why I was trying to get exact details on how you load (to make sure we are comparing apples to apples in a sense). Ex: In a hypothetical example, if 'folderA' had 50 child files and 50 child empty folders, and you used a <parentLoadRule> to load folderA, this ends up creating 100 'share roots' in the sandbox, whereas if you went to the Load Wizard and selected folderA to be loaded, it would end up with 1 share root in the sandbox (folderA itself). This 'might' be related to your situation, but you would have to figure this out.

David Lafreniere

FORUM MODERATOR / JAZZ DEVELOPER Jul 06 '17, 2:36 p.m.

Also, could you try getting Metronome data when loading during the two different cases. Here is how to turn that on: https://jazz.net/help-dev/clm/index.jsp?topic=%2Fcom.ibm.team.concert.doc%2Ftopics%2Ft_metronome.html When ready open it and open the "Services" tab and hit the "Reset" button, then quickly load without load rules, then copy the data. Then unload/clean-up, hit "Reset" again (which is very important to do) and load using load rules. This might indicate that one case is doing more server hits than the other.

Marian Marinov

Jul 07 '17, 11:21 a.m.

Using Metronome I have found that the ping to the JAZZ server is about 200ms and it seems that this causes the issue and the time for scm command to execute is the biggest within the statistics, because there are a lot of requests to the server(the checks you mentioned) caused by the loadrule file. For JAZZ server normally we are using a mirror to the real server to have faster speeds. The irony is that when connected back to the original server the ping was 50ms and the loading speed is much faster now. I found out that even if the upload and download speeds from the mirror are twice bigger(or more) than the original server the ping to the server plays bigger role in our loading speed. I would like to ask is this normal? Normally the ping to the mirror server if I make a ping from my PC from command prompt is 1ms, but the ping from Metronome says different thing 200ms and even more. Could it be a problem within the configuration of the mirror? I would expect that the mirror will be faster in any way.

showing 5 of 8 show 3 more comments

1 answer

7,743 views

0 votes

One answer

Permanent link

anna roy

(11●1) edited

Oct 24 '19, 3:03 a.m.

0 votes

Your answer

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers

Answers and Comments

Question details

rational-team-concert

× 43,080

troubleshooting

× 6,082

scm

× 1,210

source-control

× 890

performance

× 65

load

× 57

Question asked: Jul 03 '17, 4:58 a.m.

Question was seen: 7,743 times

Last updated: Oct 24 '19, 3:03 a.m.

Slow loading speed when using loadrule file.

Comments

1 answer

7,743 views

0 votes

One answer

Your answer

Follow this question

Question details

Related questions