Building by Components
We have started setting up Jazz Build Engines to use with our projects. We are synchronizing the code with our existing Clearcase infrastructure using the CC Connector.
One stream in particular has ~20 components. They are synced correctly from CC into RTC. They are organized to be maintained separately, but ultimately each depend on one another for building the complete release. Initially, there was confusion with the loading of the components into a Workspace, where the load options would drop the Now, we're attempting to configure the It appears the issue found with the loading of components into a workspace is replicated here in the Jazz Build Engine, where all the components contents are dumped into the one folder (the "Load Directory"). This was resolved with respect to the Package Explorer view, but I don't see the same options presented for the Build Definition. To simplify this, I chose only 1 component to attempt to load, using the "Select Components to Exclude" dialog. When I request a build with this configuration, I would assume the JBE would attempt to sync the one component checked and execute the command line build command. Instead, I get errors generated in the log, complaining about collisions (see log below). The colliding artifacts do exist in other components, but when I have excluded all the other components, Is it expected that I create a separate Build Definition for each component? Is there no way to have the components load within a component level directory (instead of the contents of each component dumped into the "Load Directory", colliding with the contents of other components)? 2010-11-29 20:10:27 running on host: ina**** |
11 answers
I'm not the expert on this but I'll take a stab at it. I'm also using RTC 3 so your results may vary :)
It sounds like you have the following: Component A: Contains a root directory called foo\somefiles... Component B: Contains a root directory called foo\bar\somefiles... You have placed both A and B into a workspace and now wish to build it. In an attempt to simplify the matter, you went and told the build definition "Exclude component B." Yet, the error/warnings you are receiving is that while loading the workspace it is throwing collision errors even though you excluded component B from the build. Is that correct? You are correct that everything is going to be placed in the load directory. Whatever is present in the component (right click on the component and do show repository files) is exactly what is going to be loaded in the load directory. Should your components have overlapping directory structures (foo), the tool will flag it as a warning and fail. I'm able to reproduce the warnings you received (and the build fails). What's surprising is that even by excluding a component it is still flagging it as a warning. If this really is the case, this should be an enhancement request to not scan excluded components. I am not able to reproduce this problem in RTC 3 (Component A and B in the same workspace and specified exclude component B -- works fine) so perhaps this issue was fixed? Regardless, it doesn't solve your end goal of having a single build for all your components. To solve your dilemma I'd strongly suggest refactoring the code and/or component strategy. This is the cleanest and easiest to support long term. Components are meant to be separated and loosely coupled so that you can apply baselines to different versions. Either check it all in as one component (the foo component), or decouple your code into separate foo and bar components. Are foo and bar ever at different versions? If not, then a single foo component works just fine. That might not be the answer you wanted to hear so let's try solving it the less idealistic way. (In RTC 3) there is a "Create folders for components" checkbox in the Jazz Source Control portion of the build definition. Simply check that box and your individual components will be loaded in to <Load>\<Component>\foo\bar, etc. Then just modify your build script to do any combination/moving of directories after they are loaded. I still wouldn't recommend it because many times developers just don't know how things are going to be combined until it's too late... Hope that helps, or let me know where I got off track :) |
Yes, that is correct. Just to simplify it even more... Component A: Contains in root a file named "example.txt". Component B: Contains in root a file named "example.txt'. What's surprising is that even by excluding a component it is still flagging it as a warning. If this really is the case, this should be an enhancement request to not scan excluded components. I am not able to reproduce this problem in RTC 3 (Component A and B in the same workspace and specified exclude component B -- works fine) so perhaps this issue was fixed? Unfortunately upgrading isn't an option here at this time. I'm running on 2.0.0.2. I expected the behavior of your 3.0 test to be what I encountered in 2.0- unless I'm missing something- I consider it a bug. To solve your dilemma I'd strongly suggest refactoring the code and/or component strategy. This is the cleanest and easiest to support long term. Components are meant to be separated and loosely coupled so that you can apply baselines to different versions. Either check it all in as one component (the foo component), or decouple your code into separate foo and bar components. Are foo and bar ever at different versions? If not, then a single foo component works just fine. Not sure I understand the intent, Benjamin. These components are separated and loosely coupled. They each can be baselined separately and built separately. I don't see how that has a concern on the JBE use cases mentioned. To me this point isn't an issue and works as expected outside of the JBE. Also notice that we are using legacy code, synchronized with Clearcase. So this structure (and component folder hierarchy) is critical to the use of existing build scripts and other processes in place already. That might not be the answer you wanted to hear so let's try solving it the less idealistic way. (In RTC 3) there is a "Create folders for components" checkbox in the Jazz Source Control portion of the build definition. Simply check that box and your individual components will be loaded in to <Load>\<Component>\foo\bar, etc. I don't see that option in RTC 2. I'm guessing it's a new feature. Searching the Work Items in RTC, I found Story 44004 and Enhancement 104619, both in backlog. I'm hoping someone with explicit knowledge of this feature can chime in validating this is new to 3.0 or a feature I'm simply not seeing in 2.0. |
|
Makes sense! A work around could include calling the build toolkit from your build script to check out source code rather than using the Jazz Source Code in the build definition. Not pretty, but possible.
I fired up RTC 2 and confirmed the feature to check out source code into a folder named by the component does not exist. I have forwarded this thread to Nick Edgar to see if he can help give some ideas. |
The ability to have separate folders for components when loading the build workspace is only in 3.0, unfortunately. As a workaround in 2.x, you can:
- configure it to load only one component (as you've done); this component should contain the build script - the build script can use the SCM CLI to load the other components wherever it wishes In this way, JBE is still responsible for doing the main accept/fetch, but will only fetch the first component. The rest get loaded using the SCM CLI. I suspect that the problem you're encountering with conflicts is due to existing content on disk. Try checking the "delete before load" option in the build definition, or manually delete the load directory. For more on using the SCM CLI for build scenarios, see: http://jazz.net/library/article/195/ As for the work items mentioned, #44004 covers further enhancements to load rules, and #104619 is a dup of the work done in 3.0 in #111335 (I've marked it as such). |
Verified this is not the case and that this appears to be a pretty big bug with 2.0. Assigned the build definition to a brand new build engine. Immediately failed, giving me the collision error mentioned before (no existing content there, and the "Delete directory before loading" checked) With this error, doesn't appear I can do anything command line, because the initial component isn't loaded to execute the build and pull in the other components as you've suggested. Any other possible workarounds to at least get one component onto the JBE? |
|
Sorry to hear it's still not working. I've asked the SCM team to look into this further. The only other workaround I can think of is to create a separate build workspace with a script that uses the SCM CLI to do the fetches from the main build workspace. It would probably need to drive the accept from the stream as well.
|
Can you confirm that you have the excluded components configured appropriately. I would expect that in the dialog you have all the components selected except for the one you want to load.
Sorry to hear it's still not working. I've asked the SCM team to look into this further. The only other workaround I can think of is to create a separate build workspace with a script that uses the SCM CLI to do the fetches from the main build workspace. It would probably need to drive the accept from the stream as well. |
Can you confirm that you have the excluded components configured appropriately. I would expect that in the dialog you have all the components selected except for the one you want to load. I have 16 components in the workspace. When I open the Jazz Source Control tab in the Build Definition, I see in the Components to exclude box: "15 components selected". However, when I click select and open, I see two components unchecked. After re-selecting the unchecked component, it syncs as expected. Sorry for that and thanks for your assistance. |
|
Did you have specify any load rules? For example in other components other than the one you want loaded, or that contain items with the same name to be loaded.
Can you confirm that you have the excluded components configured appropriately. I would expect that in the dialog you have all the components selected except for the one you want to load. Sorry to hear it's still not working. I've asked the SCM team to look into this further. The only other workaround I can think of is to create a separate build workspace with a script that uses the SCM CLI to do the fetches from the main build workspace. It would probably need to drive the accept from the stream as well. |
|
ahh, I think I know what happened, in case this is a test case that isn't currently tested.
Originally, the stream had two components of the same name- one synced correctly and one that wasn't. I had only loaded one component into the workspace and when I configured the build components, I excluded the component. At a later point, I removed one of the components from the stream and re-loaded. I'm assuming/guessing the originally configured build definition still had the reference to the originally selected 15, however the component that remained (and eventually loaded) was never excluded and therefore unchecked? Not sure-- just trying to understand why the text would read 15 excluded and still have one unselected. |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.