It's all about the answers!

Ask a question
QuestionsTagsUsersBadges
Guidelines | FAQ
Follow this question
Question details

×42,877
×6,036
×1,201
×889
×65
×57

question asked: Jul 03 '17, 4:58 a.m.
question was seen: 4,332 times
last updated: Oct 24 '19, 3:03 a.m.

Related questions

Slow loading speed when using loadrule file.

0
Marian Marinov (11●1●1) | asked Jul 03 '17, 4:58 a.m.
edited Jul 05 '17, 2:33 p.m. by David Lafreniere (4.8k●7)

 Hello,

   
       We have huge problems with the speed of loading a workspace when using a loadrule file. If we are not using loadrule file the load speed is normal, but when we try with the loadrule it takes about 10minutes(sometimes more) only at one step "Checking load completion conditions".
        Can you help me find out what could be the reason of such big delay. After this delay of aobut 10minutes the speed of loading is the same as without loadrule file(about 5 minutes at most).

Thank you.

Comments
Thomas Kirstätter commented Jul 03 '17, 4:47 p.m.

can you give some more hints on the size of the WS? number of components? overall size on disc? Do you use hierarchical components? RTC Version?


Marian Marinov commented Jul 04 '17, 9:40 a.m.

 The RTC Version we are using is 6.0.2. The components in the stream are about 200. The size of the whole stream is about 11GB. We have several loadrule files some of them are getting about 100mb others about 230mb. The tests I'm making are with loadrule that gets about 100mb(2200 files). 

   We are using hierarchical components only for structuring the components, but those are not used within the loadrule files. The loadrule files are getting the files directly from the leaf components. Can you tell me, how the hierarchical components influence the loading time?

David Lafreniere commented Jul 05 '17, 2:41 p.m.
FORUM MODERATOR / JAZZ DEVELOPER

Hierarchical components should not impact the load time; although it would be nice to confirm this. Can you test this by flattening the entire workspace in a repository workspace and then try loading? When done you can discard the component hierarchy changes so that they are not accidentally delivered to the stream.

Also, can you describe exactly how you would manually load without the load rule? And is there any chance you can attach/post a snippet of how the load rule looks? I'm wondering if when you manually load, you select a root folder or component for example, so that it loads everything into one 'share root' folder, whereas when the load rule is used, it loads every sibling file/folder as a separate 'share root'.

Marian Marinov commented Jul 06 '17, 7:59 a.m. | edited Jul 22 '17, 1:51 p.m.
 I can confirm that the problem is not coming from the hierarchical components. 
About the loadrule files I think there is a misunderstanding. We are using loadrule files. When I said we were not using hierarchical components within the loadrule files I meant that we are not loading parent components and expecting automatically loading of all the subcomponents.
Basically what we do within the loadrule files is:

downloading whole folders from a component to a specific path within the build structure:

   <parentLoadRule>
        <component name="RTC_CMP_NAME_1"/>
        <!-- <component itemId="_bjINMTimEeeXlpNLUsJSzA" /> -->
        <parentFolder repositoryPath="/CMP_FOLDER_TO_DOWNLOAD"/>
        <!-- <parentFolder itemType="folder" itemId="_VADhmPsfEeW-3fzLE168VA" /> -->
        <sandboxRelativePath includeRepositoryPath="true" pathPrefix="/Path_in_the_build_structure"/>
    </parentLoadRule>

or downloading specific files from component to specific places within the build structure

    <itemLoadRule>
        <component name="RTC_CMP_NAME_2"/>
        <!-- <component itemId="_mQAX4DisEeeXlpNLUsJSzA" /> -->
        <item repositoryPath="/Path_to_File_name_within_the_component"/>
        <!-- <item itemType="file" itemId="_kMq9EvscEeW-3fzLE168VA" /> -->
        <sandboxRelativePath includeRepositoryPath="true" pathPrefix="/Place_within_the_build_structure_where_the_file_should_go"/>
    </itemLoadRule>
    
Marian Marinov commented Jul 06 '17, 11:20 a.m.

David do you have information what exactly "Checking load completion conditions" means? This step is really short when not using loadrule file and it takes ages if we use loadrule files. Maybe we could somehow modify the loadrule file to make it faster. In addition the loadrule file is about 3000lines long together with all the comments.

David Lafreniere commented Jul 06 '17, 2:29 p.m.
FORUM MODERATOR / JAZZ DEVELOPER

"Checking load completion conditions" is an Eclipse job that the scm code runs before fetching the actual content; it evaluates the load request (ex: to confirm there is no overlap in terms of existing sandboxes or shares on disk), I would say the length of time this takes would be proportional to the number of shares ('parent' files or folders being loaded). That is why I was trying to get exact details on how you load (to make sure we are comparing apples to apples in a sense). Ex: In a hypothetical example, if 'folderA' had 50 child files and 50 child empty folders, and you used a
<parentLoadRule> to load folderA, this ends up creating 100 'share roots' in the sandbox, whereas if you went to the Load Wizard and selected folderA to be loaded, it would end up with 1 share root in the sandbox (folderA itself). This 'might' be related to your situation, but you would have to figure this out.

David Lafreniere commented Jul 06 '17, 2:36 p.m.
FORUM MODERATOR / JAZZ DEVELOPER

Also, could you try getting Metronome data when loading during the two different cases.

Here is how to turn that on:
https://jazz.net/help-dev/clm/index.jsp?topic=%2Fcom.ibm.team.concert.doc%2Ftopics%2Ft_metronome.html

When ready open it and open the "Services" tab and hit the "Reset" button, then quickly load without load rules, then copy the data. Then unload/clean-up, hit "Reset" again (which is very important to do) and load using load rules. This might indicate that one case is doing more server hits than the other.

Marian Marinov commented Jul 07 '17, 11:21 a.m.

Using Metronome I have found that the ping to the JAZZ server is about 200ms and it seems that this causes the issue and the time for scm command to execute is the biggest within the statistics, because there are a lot of requests to the server(the checks you mentioned) caused by the loadrule file.

For JAZZ server normally we are using a mirror to the real server to have faster speeds. The irony is that when connected back to the original server the ping was 50ms and the loading speed is much faster now.
I found out that even if the upload and download speeds from the mirror are twice bigger(or more) than the original server the ping to the server plays bigger role in our loading speed.
I would like to ask is this normal? Normally the ping to the mirror server if I make a ping from my PC from command prompt is 1ms, but the ping from Metronome says different thing 200ms and even more. Could it be a problem within the configuration of the mirror? I would expect that the mirror will be faster in any way.

showing 5 of 8 show 3 more comments

One answer

Most liked answers ↑|Newest answers|Oldest answers

0
permanent link
anna roy (11●1) | answered Oct 24 '19, 3:02 a.m.
edited Oct 24 '19, 3:03 a.m.

  <style type="text/css"> &amp;amp;amp;amp;amp;amp;amp;amp;lt;!--td {border: 1px solid #ccc;}br {mso-data-placement:same-cell;}--&amp;amp;amp;amp;amp;amp;amp;amp;gt; </style> I don't see a reason for hierarchical components impacting the load time. However, it would be better to identify their impact. I would suggest you to test this by flattening the entire workspace in a repository workspace and try loading it again. After you are done, you can discard the component hierarchy changes to ensure that they are not accidentally delivered straight to the stream. Also, consider Load Testing of the Service. 



Your answer

Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.