It's all about the answers!

Ask a question

Will JBE support concurrent build requests?


Brian Gillan (3215330) | asked Jan 24 '08, 5:28 p.m.
After playing around a little with the JBE, it looks like it serializes
incoming build requests. Are there plans to allow it to handle
concurrent build requests?

Thanks, Brian

10 answers



permanent link
Ryan Manwiller (1.3k1) | answered Jan 24 '08, 5:38 p.m.
JAZZ DEVELOPER
We are not currently planning to support this for 1.0.

If this is something you would like to see, please create a work item and we
will consider it.

---
Ryan Manwiller
Jazz Team Build

permanent link
Ryan Manwiller (1.3k1) | answered Jan 24 '08, 5:48 p.m.
JAZZ DEVELOPER
As Geoff said, one approach to achieve handling multiple requests on the
same build machine is to just run as many instances of jbe as you'd like.
You'll probably want to start them up in different working directories.

---
Ryan Manwiller
Jazz Team Build

permanent link
Geoffrey Clemm (30.1k33035) | answered Jan 24 '08, 5:48 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
I believe the idea is that you can fire up additional JBE's, if you
wanted multiple jobs to be running in parallel.

Cheers,
Geoff

Brian Gillan wrote:
After playing around a little with the JBE, it looks like it serializes
incoming build requests. Are there plans to allow it to handle
concurrent build requests?

Thanks, Brian

permanent link
Brian Gillan (3215330) | answered Jan 25 '08, 8:28 a.m.
Ryan Manwiller wrote:
As Geoff said, one approach to achieve handling multiple requests on the
same build machine is to just run as many instances of jbe as you'd
like. You'll probably want to start them up in different working
directories.

---
Ryan Manwiller
Jazz Team Build

I'll have to discuss with our team. I'm in a centralized build group and
we currently support 100's of builds, so I'm not sure starting a
separate JBE for each would be practical. I think we can get by now as
we'll only be supporting a few builds initially. After further
investigation, I'll open up a request if it looks like well need this.

Thanks, Brian

permanent link
David Olsen (5237) | answered Jan 25 '08, 7:08 p.m.
JAZZ DEVELOPER
(I hope Ryan won't come over and start yelling at me for saying this...)

Brian Gillan wrote:
I'm in a centralized build group and
we currently support 100's of builds, so I'm not sure starting a
separate JBE for each would be practical.

If you have 100's of builds to manage, I suggest that you look at some
tool other than the Jazz Build Engine for managing them. JBE is not
terribly sophisticated in its ability to schedule builds or distribute
them among machines. Some other tools (such as Rational Build Forge)
are much better at that. It is still possible to send build results to
Jazz even if the build wasn't started via JBE.

That's how our team works. Most builds are started from Build Forge,
and results are sent to our Jazz repository so developers can the
results in their IDE.

--
David Olsen
IBM Rational

permanent link
Ryan Manwiller (1.3k1) | answered Jan 25 '08, 7:58 p.m.
JAZZ DEVELOPER
David is correct. Let me clarify a little.

JBE is intended as a simple entry level build engine that can be substituted
for as your needs demand. The rest of the Jazz Build framework is designed
for this (to allow any build engine). Your builds can still participate by
publishing status and results to the Jazz repository, and the team can still
benefit from the awareness, control, and linking of build artifacts in Team
Concert.

JBE is a small piece of the Jazz Build environment. JBE isn't involved in
the scheduling or management of builds. It's a client program that grabs a
request from the Jazz repository and spawns a process for the build. Many
other build engines/tools can replace this function, while adding much more
power and features, such as Build Forge.

permanent link
Brian Gillan (3215330) | answered Jan 28 '08, 3:48 p.m.
OK. I was looking at the JBE to perhaps serve as an entrypoint into our
existing build environment or perhaps serve as a broker between our
build environment and a Jazz/RTC repository. In our build environment we
have a machine called a "BuildMaster" which is the coordinator for a
given build. Oversimplifying, it interacts with the SCM and possibly DCT
system for reporting, getting the source to be processed, then
dispatches "builds" to "BuildEngines" for each platform/OS that needs to
be built. Once all dispatched builds have been completed, the
BuildMaster handles post-processing work.

Is there some high level documentation available for the Jazz Build
Framework and where JBE fits in, other than the tutorial/overview type
information available on jazz.net?

Thanks, Brian

permanent link
Ryan Manwiller (1.3k1) | answered Jan 29 '08, 5:08 p.m.
JAZZ DEVELOPER
So far, there are 3 documents on jazz.net, which you may have already seen.

Overview:
https://jazz.net/learn/LearnItem.jsp?href=content/docs/platform-overview/team-build-overview.html

Getting started:
https://jazz.net/learn/LearnItem.jsp?href=content/docs/build-setup/index.html

Tutorial:
https://jazz.net/learn/LearnItem.jsp?href=content/docs/client-tour/using-build.html

Regarding where JBE fits in. It fills the role of a simple build loop that
you can run on build machines in an environment where you are not trying to
do anything fancy like parallel/distributed building. Its similar to
CruiseControl in its simplicity and purpose.

In the environment you describe, JBE probably doesn't have a place. You
already have a sophisticated build infrastructure in place. Instead, you
integrate with jazz at the build toolkit level, using Ant to call the
toolkit tasks to communicate between your build environment and the jazz
repository. There are tasks for getting the next build request (so you can
still use jazz scheduling and requesting), notifying jazz of build start and
complete, jazz scm, publishing results, and updating the build activity in
the jazz repository. Ideally, you could do all this from a command line tool
or REST api, but for now you have to callout through ant. By telling the
jazz repository what your builds are doing, the team gets the benefit of
first class build awareness in the IDE and linking with the other jazz
artifacts.

If you look at the example in
/jazz/buildsystem/buildtoolkit/examples/standalone/build.xml, you'll see
that even a plain ant script can drive a build from start to complete, no
JBE necessary. We're working on better docs for the ant tasks. For now, you
can use the javadoc at
/jazz/buildsystem/buildtoolkit/task-javadoc/index.html.

It sounds like your BuildMaster is where you might want to do some of this
communication with jazz. However, some detailed activity updates could also
be done from your BuildEngines as well. Regardless, you need to make Jazz
aware of the entity that is polling for request and processing requests
(start/complete build). This entity is known as the Build Engine in jazz,
and you must declare it by creating a Build Engine in the Jazz repository.
In your case, this build engine is representing your BuildMaster. The Build
Engine declared in Jazz is just an abstraction for whatever thing is out
there doing builds. In some cases, its a simple JBE. In other cases its a
powerful build system like Build Forge that is farming out steps to hundreds
of build machines.

Finally, Jazz Build Definitions will be associated with your Build Engine
declared in Jazz. Users can request those builds and schedules for those
builds can be set.

I will think about getting another document or article up regarding how to
integrate your existing setup with Jazz Build.

permanent link
Brian Gillan (3215330) | answered Feb 01 '08, 4:28 p.m.
Ryan Manwiller wrote:

already have a sophisticated build infrastructure in place. Instead, you
integrate with jazz at the build toolkit level, using Ant to call the
toolkit tasks to communicate between your build environment and the jazz
repository. There are tasks for getting the next build request (so you
can still use jazz scheduling and requesting), notifying jazz of build
start and complete, jazz scm, publishing results, and updating the build
activity in the jazz repository. Ideally, you could do all this from a
command line tool or REST api, but for now you have to callout through
ant. By telling the jazz repository what your builds are doing, the team
gets the benefit of first class build awareness in the IDE and linking
with the other jazz artifacts.

Our builds are currently all driven by scripts (ksh and Perl). I'm new
to Ant, but I presume that rather than having Ant drive the overall
build process, we could create a series of ant scripts for each
interaction we need to do with the Jazz repository and invoke them from
our existing build scripts.

It sounds like your BuildMaster is where you might want to do some of
this communication with jazz. However, some detailed activity updates
could also be done from your BuildEngines as well. Regardless, you need

Our BuildMaster currently only has very high level information such as
"build started on platform x", "build complete on platform x". The
problem we'll have with interacting with Jazz from our BuildEngines, is
some of them are running ancient OSes, and may not support Ant/Java
needed. Probably not really an issue though, since we provide details
for each platform via a web interface.


I will think about getting another document or article up regarding how
to integrate your existing setup with Jazz Build.

So here's what I envision at this point:

- We create a "monitor" process on our BuildMaster that essentially
loops waiting for build requests to come initiated from a Jazz client.
When a request comes in, we simply start a new build process
asynchronously and go back to waiting for the next request. We could
also allow build requests to be submitted via our existing web interface.
- We create a series of ant stubs for various interactions with the Jazz
repository, and for "Jazz" builds, our existing build scripts will call
out to these stubs at various steps in the process.

Does that sounds like a workable approach?

Thanks, Brian

permanent link
Don Weinand (7851) | answered Feb 01 '08, 4:38 p.m.
JAZZ DEVELOPER
Yes...that sounds workable. You'll need to make sure that either the process kicking off the async builds or your ant call outs in the build script always completes the builds or they'll end up in an in-progress state forever. We have quite a bit of logic baked into our build engine to handle this that you may want to look at.

Don Weinand
Jazz Team Build Developer

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.