How do concurrent builds work with a single repository workspace?
The Jazz Source Control page of the build definition provides for a single repository workspace to be specified, but consider two builds of one build def running concurrently; the repo workspace is loaded into two load directory sandboxes. The end of my build process checks in and delivers the changes to the stream then takes a snapshot.
How will this work if two builds are trying to check in and deliver at the same time?
This is in the context of CLM 4.0.7 on Windows Server 2008 R2.
Accepted answer
I've already have this problem, and then I tried to avoid it. It works if the file checked-in in the two builds are different. Otherwise the second deliver will get an error because it needs a merge (which is the case if you're trying to check-in generated artifacts, as I suppose), unless you're managing it with your custom check-in-and-delivery script/extension.
Comments
If you want to check in and deliver generated files in a build, you will always have the potential of conflicts. I think. In this case I would go a step further and not use the standard tab in the build, or only to load the build scripts. I would then use the CLI to create a unique repository workspace, load that, run the build and use the CLI to check in and deliver. You would have to deal with conflicts in any case.
As I said, this can only happen if you check in during builds which I personally would consider being a bad thing. There might be reasons why people want it, but it is a bit unclean from my perspective. Generated stuff should be generated during the build. Checking in object files and exe files and the like in builds is usually also frowned upon and I would consider other files generated in builds as in the same category.
2 other answers
The Jazz Build Engine does not check in or deliver any changes from the build repository workspace.
Quite the opposite, the Jazz Source Control options are set up in a way that the repository workspace can accept the changes from the stream in public builds. If set up this way this essentially means that the JBE loads what is in the stream to the build workspace and builds that.
The snapshot is taken after accepting - not after any delivery, since delivery never happens.
For private builds the JBE does not even have write permissions, so here it loads exactly the content of the workspace and builds this. Any created content would only be local to the load location and never show up in the repository workspace itself. The JBE couldn't check anything in, even if it tried to, due to the lack of permissions.
If you create automation and additional scripting that checks in generated files from the build, I don't know what would happen. As far as I can tell, the repository workspace, once loaded is never checking consistency. This would happen in custom build scripts e.g. using the CLI in which case you might run into errors if you check into the repository workspace in two places.
I am not a fan of checking in artifacts that are generated during a build. To me, these are generated and should not be in source control anyway.
Comments
Hi Ralph, I agree with you about generated artifacts but unfortunately I've customers that want to version artifacts instead of storing it in some artifacts repository. But there are some case on which I had to do this operation: for example using maven with release plugin in order to increase the version or remove the SNAPSHOT tag. I still don't like it, as I prefer other way (for example avoid maven release plugin) but if it's the way that developers use then a check-in after build is needed.
I know, sometimes you don't have a choice, I am not blaming you here 8).
But that does not make it being a better idea, unfortunately. It is also pretty irrelevant what SCM or build tool you use in this case. The problem is pretty much the same in any combination.
I have valid reasons for storing generated artefacts in RTC, and I understand the point about such stuff being derived and therefore re-derivable but there are certain advantages of being able to tie source and object togther with a snapshot, and other advantages of being able to drive automated deployment from a snapshot on a release stream. For us, 'some artefacts repository' for deployable artefacts is RTC and the advantages of this approach outweigh the disadvantages.
Comments
RTC is designed as a source control management and change management tool. The requirements for this kind of tools are very different from the ones for an artifact repository that mainly deals with binaries. SCM needs to store deltas, requires merging and is optimized for that. Artifact repositories are optimized for binaries and other different purposes. You can try to use RTC as a binary artifact repository to some extent.
This is like different types of cars, say racing car and truck. A truck is a bad racing car and vice versa. You could also use each one as a means to go on vacation with the family, but it might not be a god fit either.
You need SCM, but you also might need an artifact repository for binaries.
Anyway, consider looking int the thoughts in https://rsjazz.wordpress.com/2015/10/28/build-artifacts-publishing-and-automated-build-output-management-using-the-plain-java-client-libraries/ and the related posts.