Managing automatic deliver upstream/down on successful build
Hi,
Our larger team currently has an integration stream, and several team streams under that. The team streams build, and when successful, we have someone who has to manually push those changes up to the integration stream, and pull the latest changesets down to the team stream. We want to automate all of this, so that on successful builds, the flows happen automatically. I read this article, that shows how to setup such flows, and it is a very good starting point: https://jazz.net/library/article/649 But, one thing it does not cover is what will happen in the case of a merge failure or when automerges are "iffy" I'd say that we probably have to manually merge 10% to 15% of the time. If the post successful build option is to deliver the snapshot changesets upstream, what will happen if there is a merge failure on that deliver? How can we manage who will take action when that occurs? Also, if anyone has any pointers to a specific cookbook or more exact example of setting up delivery / accept to/from integration streams in an automatic fashion, that would be great. Thanks, Greg |
10 answers
The deliver will replace everything in the target with what was in the build. This works best when your integration stream consists of components that receive changes from individual streams. It sounds like your teams may deliver to the same components and this causes conflicts. This wouldn't work well with post-build deliver since any new changes in the integration stream would be replaced.
It would be better if your teams had their own components. You wouldn't have the problem of merging. Plus, everything that is delivered would have been blessed by a good build. If you're merging then you don't know that the integration stream will be in a good state after delivery. |
The deliver will replace everything in the target with what was in the build. This works best when your integration stream consists of components that receive changes from individual streams. It sounds like your teams may deliver to the same components and this causes conflicts. This wouldn't work well with post-build deliver since any new changes in the integration stream would be replaced. That's quite unfortunate. I can not imagine any situation where that (overwriting changes in integration stream, and not doing a merge) would be acceptable. Also, I do not see much value in the solution that "each team should have their own component" -- If the teams' code were that segmented, we wouldn't have to use multiple team streams. |
Ralph Schoon (63.4k●3●36●46)
| answered Apr 05 '12, 12:44 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
As far as I know the idea of the post build deliver is to deliver to a stream that is only consumed. One use case is to deliver automatically with a gate, e.g. build succeeds to a test stream. This ensures the testers have always a stream that builds for testing. They don't change the code and only use the result.
Some articles: https://jazz.net/library/article/649 https://jazz.net/library/article/599 SCM systems can not prevent conflicts, except with pessimistic locking, which has its own problems. Also pessimistic locking typically would not cover conflicting changes on different streams/branches either. Only isolating work in components and a careful integration can. However, the more frequent integration occurs, the less likely conflicts occur. So you have a team that works on the same things work against one stream. They can integrate constantly and prevent conflicts from happening. You have streams where integrators from each team flows and picks up and integrates changes frequently. The more frequent the less likely you have conflicts. |
Also, I do not see much value in the solution that "each team should have their own component" -- If the teams' code were that segmented, we wouldn't have to use multiple team streams. One benefit is team streams contain only the components that they depend on. Less content would be loaded. But maybe your project can't be structured that way because teams are tightly integrated. Perhaps there are some components in your integration stream that only accept from one source (ie. only one team delivers to that component)? You could use post-build deliver on those components. |
Hi Ralph, Yes, this is exactly what we would like to achieve -- that the automatic integration flows often and flows quickly, so that there will be fewer conflicts. However, the result when there are conflicts can not be to wipe out changes and overwrite code: That is unacceptable: It should be to ask for intervention. Lets say that I was able to implement a system where code was merging up and down often enough that only 1 out of 20 builds, there was a conflict. In the current design, I can't implement such a system, because that 1 out of 20 times, a disaster occurs and changes are overwritten. But since I can not implement a system that incorporates human responses, I now have to do everything manually. I have to have someone on call, all the time, as an code monitor and integrator to push buttons all 20 times because there's no way to setup an automated system. We currently have people that have to be around and available, pushing buttons and watching paint dry almost 24 hours a day, 7 days a week. The vast majority of the time, There are no conflicts, so they are just selecting changesets and pressing "accept" and "deliver" If we could automate it, they might only have to respond to a merge conflict once a week, via text or email. Cheers, Greg |
|
Ralph Schoon (63.4k●3●36●46)
| answered Apr 05 '12, 2:23 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Hi Greg,
we have such capabilities for synchronization e.g. with CC (I am not sure we have a notification). I am not sure we have it for post build. I have not used the feature, so you could try what happens. 8-) I think it is a reasonable request to have a check box to alert someone in case there is a conflict. So please don't hesitate to create an enhancement request. |
|
I'm not sure that post-build deliver is intended for your scenario. The deliver leaves the target component in the same state as the build because it performs a component replacement. To merge in with changes that already exist in the stream defeats the purpose of the post-build deliver. The changes merged in invalidates the changes that come in from a good build because your code could have changed. You'd have to rebuild it all anyway to validate the merged changes.
Check out this current enhancement request: Add a regular (i.e. "merge" rather than "replace") deliver option for the post-build deliver follow-up action Comments Hi Tim,
|
To merge in with changes that already exist in the stream defeats the purpose of the post-build deliver. The changes merged in invalidates the changes that come in from a good build because your code could have changed. You'd have to rebuild it all anyway to validate the merged changes. This is correct -- we want each team's build to be green, and only their "tested" changes to be merged up to the integration stream. We do know that we need to then re-run the build in the integration stream, to know that build works at that level. We want only changesets that have tested green in the integration builds to be accepted down into the team streams. We are doing this all today, just manually instead of automated. We try to stick only to changesets in a "good build" snapshot. But when you are doing everything by hand, mistakes happen. Maybe the post build step is not the right way -- so I'm actually open to any kind of solution. Given what I've described here -- is there some other way to achieve this automation? |
Geoffrey Clemm (30.1k●3●30●35)
| answered Apr 06 '12, 10:00 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Have you tried invoking the appropriate scm commands from your build script? I.e. do what work item 194461 requests, but coding it yourself.
Cheers, Geoff To merge in with changes that already exist in the stream defeats the purpose of the post-build deliver. The changes merged in invalidates the changes that come in from a good build because your code could have changed. You'd have to rebuild it all anyway to validate the merged changes. This is correct -- we want each team's build to be green, and only their "tested" changes to be merged up to the integration stream. We do know that we need to then re-run the build in the integration stream, to know that build works at that level. We want only changesets that have tested green in the integration builds to be accepted down into the team streams. We are doing this all today, just manually instead of automated. We try to stick only to changesets in a "good build" snapshot. But when you are doing everything by hand, mistakes happen. Maybe the post build step is not the right way -- so I'm actually open to any kind of solution. Given what I've described here -- is there some other way to achieve this automation? |
|
The other option is decoupling your teams so that they aren't modifying code in the same files. If there are some common files, they can go into a common component. Then you can use post-build deliver on components that is only modified by one team.
This may or may not be feasible depending on how much time you want to spend on refactoring. |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.