Will JBE support concurrent build requests?
After playing around a little with the JBE, it looks like it serializes
incoming build requests. Are there plans to allow it to handle concurrent build requests? Thanks, Brian |
10 answers
We are not currently planning to support this for 1.0.
If this is something you would like to see, please create a work item and we will consider it. --- Ryan Manwiller Jazz Team Build |
As Geoff said, one approach to achieve handling multiple requests on the
same build machine is to just run as many instances of jbe as you'd like. You'll probably want to start them up in different working directories. --- Ryan Manwiller Jazz Team Build |
Geoffrey Clemm (30.1k●3●30●35)
| answered Jan 24 '08, 5:48 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
I believe the idea is that you can fire up additional JBE's, if you
wanted multiple jobs to be running in parallel. Cheers, Geoff Brian Gillan wrote: After playing around a little with the JBE, it looks like it serializes |
Ryan Manwiller wrote:
As Geoff said, one approach to achieve handling multiple requests on the I'll have to discuss with our team. I'm in a centralized build group and we currently support 100's of builds, so I'm not sure starting a separate JBE for each would be practical. I think we can get by now as we'll only be supporting a few builds initially. After further investigation, I'll open up a request if it looks like well need this. Thanks, Brian |
(I hope Ryan won't come over and start yelling at me for saying this...)
Brian Gillan wrote: I'm in a centralized build group and If you have 100's of builds to manage, I suggest that you look at some tool other than the Jazz Build Engine for managing them. JBE is not terribly sophisticated in its ability to schedule builds or distribute them among machines. Some other tools (such as Rational Build Forge) are much better at that. It is still possible to send build results to Jazz even if the build wasn't started via JBE. That's how our team works. Most builds are started from Build Forge, and results are sent to our Jazz repository so developers can the results in their IDE. -- David Olsen IBM Rational |
David is correct. Let me clarify a little.
JBE is intended as a simple entry level build engine that can be substituted for as your needs demand. The rest of the Jazz Build framework is designed for this (to allow any build engine). Your builds can still participate by publishing status and results to the Jazz repository, and the team can still benefit from the awareness, control, and linking of build artifacts in Team Concert. JBE is a small piece of the Jazz Build environment. JBE isn't involved in the scheduling or management of builds. It's a client program that grabs a request from the Jazz repository and spawns a process for the build. Many other build engines/tools can replace this function, while adding much more power and features, such as Build Forge. |
OK. I was looking at the JBE to perhaps serve as an entrypoint into our
existing build environment or perhaps serve as a broker between our build environment and a Jazz/RTC repository. In our build environment we have a machine called a "BuildMaster" which is the coordinator for a given build. Oversimplifying, it interacts with the SCM and possibly DCT system for reporting, getting the source to be processed, then dispatches "builds" to "BuildEngines" for each platform/OS that needs to be built. Once all dispatched builds have been completed, the BuildMaster handles post-processing work. Is there some high level documentation available for the Jazz Build Framework and where JBE fits in, other than the tutorial/overview type information available on jazz.net? Thanks, Brian |
So far, there are 3 documents on jazz.net, which you may have already seen.
Overview: https://jazz.net/learn/LearnItem.jsp?href=content/docs/platform-overview/team-build-overview.html Getting started: https://jazz.net/learn/LearnItem.jsp?href=content/docs/build-setup/index.html Tutorial: https://jazz.net/learn/LearnItem.jsp?href=content/docs/client-tour/using-build.html Regarding where JBE fits in. It fills the role of a simple build loop that you can run on build machines in an environment where you are not trying to do anything fancy like parallel/distributed building. Its similar to CruiseControl in its simplicity and purpose. In the environment you describe, JBE probably doesn't have a place. You already have a sophisticated build infrastructure in place. Instead, you integrate with jazz at the build toolkit level, using Ant to call the toolkit tasks to communicate between your build environment and the jazz repository. There are tasks for getting the next build request (so you can still use jazz scheduling and requesting), notifying jazz of build start and complete, jazz scm, publishing results, and updating the build activity in the jazz repository. Ideally, you could do all this from a command line tool or REST api, but for now you have to callout through ant. By telling the jazz repository what your builds are doing, the team gets the benefit of first class build awareness in the IDE and linking with the other jazz artifacts. If you look at the example in /jazz/buildsystem/buildtoolkit/examples/standalone/build.xml, you'll see that even a plain ant script can drive a build from start to complete, no JBE necessary. We're working on better docs for the ant tasks. For now, you can use the javadoc at /jazz/buildsystem/buildtoolkit/task-javadoc/index.html. It sounds like your BuildMaster is where you might want to do some of this communication with jazz. However, some detailed activity updates could also be done from your BuildEngines as well. Regardless, you need to make Jazz aware of the entity that is polling for request and processing requests (start/complete build). This entity is known as the Build Engine in jazz, and you must declare it by creating a Build Engine in the Jazz repository. In your case, this build engine is representing your BuildMaster. The Build Engine declared in Jazz is just an abstraction for whatever thing is out there doing builds. In some cases, its a simple JBE. In other cases its a powerful build system like Build Forge that is farming out steps to hundreds of build machines. Finally, Jazz Build Definitions will be associated with your Build Engine declared in Jazz. Users can request those builds and schedules for those builds can be set. I will think about getting another document or article up regarding how to integrate your existing setup with Jazz Build. |
Ryan Manwiller wrote:
already have a sophisticated build infrastructure in place. Instead, you Our builds are currently all driven by scripts (ksh and Perl). I'm new to Ant, but I presume that rather than having Ant drive the overall build process, we could create a series of ant scripts for each interaction we need to do with the Jazz repository and invoke them from our existing build scripts.
Our BuildMaster currently only has very high level information such as "build started on platform x", "build complete on platform x". The problem we'll have with interacting with Jazz from our BuildEngines, is some of them are running ancient OSes, and may not support Ant/Java needed. Probably not really an issue though, since we provide details for each platform via a web interface.
- We create a "monitor" process on our BuildMaster that essentially loops waiting for build requests to come initiated from a Jazz client. When a request comes in, we simply start a new build process asynchronously and go back to waiting for the next request. We could also allow build requests to be submitted via our existing web interface. - We create a series of ant stubs for various interactions with the Jazz repository, and for "Jazz" builds, our existing build scripts will call out to these stubs at various steps in the process. Does that sounds like a workable approach? Thanks, Brian |
Yes...that sounds workable. You'll need to make sure that either the process kicking off the async builds or your ant call outs in the build script always completes the builds or they'll end up in an in-progress state forever. We have quite a bit of logic baked into our build engine to handle this that you may want to look at.
Don Weinand Jazz Team Build Developer |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.