Jazz Forum Welcome to the Jazz Community Forum Connect and collaborate with IBM Engineering experts and users

setting priority of multiple build engines

Questions
Tags
Users
Badges
Rob Leach
(35●4●8) Aug 04 '14, 9:35 p.m.
 Hi,
We have multiple build definitions and multiple build engines. Is there a way to prioritize build engines so that you can influence which order build engines are picked by any given build definition.

Here is the scenarios; for simplicity let's say we have three build definitions and two build engines. All three of the build definitions are configure to have both build engines as their "supporting build engines". But with the list of two build engines, let's say we always want jbe1 to be picked first if it is not busy, and then jbe2 pick as a second choice.

Obviously our actual setup is much more complex with higher numbers of build definitions and build engines. But the principle is still the goal. Set the priority of supporting build engines so we can influence the order in which build engines are picked.

The build engine selection process appears to start before the Jazz server has started executing any code (ant scripts) that I have control over, so I don't think I can control that selection programmatically. So is there a property we can set that will provide that functionality. Currently you can set the following property in your build engine definition: requestFilter. If you set this property to personalBuild=false then you can enforce no personal builds on the respective build engine (so you influenced some level of control). The ideal would be something similar to this functionality to influence build engine priority.

I have found a couple of questions and an article that at first looked promising, but did not lead anywhere meaningful. This question had an answer that suggested the REST API, but that would be more work that seems appropriate for the simplicity of the functionality. This question isn't relevant for our needs. And then this article doesn't deal with our question.

Cheers,
-Rob

0 votes

Comments
Rob Leach
Aug 04 '14, 9:51 p.m.
 Oh and I should add a quick note. I'm sure a build setup using Jenkins could possibly solve this problem, but we are far out from migrating to Jenkins anytime soon; so that is not an option for us at this moment. 

3 answers
7,601 views
0 votes

Accepted answer

Permanent link
Nick Edgar
JAZZ DEVELOPER (6.5k●7●11) edited
Aug 06 '14, 12:54 p.m.
As Ralph said, priority-based assignment of builds to engines is not currently supported out-of-the-box. The only property the requestFilter mechanism supports currently is the one for personalBuild.
The best you can do now is adjust the -sleepTime arg to JBE to have the preferred instances polling more frequently, increasing the chance that they'll pick up a request first. But this is non-deterministic.

It would be relatively straightforward to write a script to pull requests off the queue, then invoke JBE as a one-off for that particular build, on the next available preferred machine, e.g. via ssh.  If you're interested in exploring that, I'd be happy to help.
Rob Leach selected this answer as the correct answer

0 votes

Comments
Rob Leach
Aug 06 '14, 1:19 p.m.
You know what Nick, that isn't too bad of an option (adjusting the sleepTime). But as you point out, it would definitely not be an exacting solution. If a working model is implemented it is probably one that would make me cringe each time I remember how it works. Your second solution seems like it would just be easier to implement an agent and figure out how to provide the missing functionality/features that the JBE provides out of the box. As a customer it simply seems like a waste of my time when I believe the tool should provide that functionality/ability to me (so that I can use it where and as needed).

Nick Edgar
JAZZ DEVELOPER Aug 06 '14, 3:24 p.m.
I understand and appreciate your point about "it should just work out of the box".  The problem is that, architecturally, JBE works as a client of the RTC server, and is independent of other running JBE instances, so there's no central coordination over which builds run where. You can partition build definitions across different engines in the build farm, but there's still no preferential matching to different engines (or other capabilities of the build machines, e.g. OS / compilers / other tools). Enterprise grade build systems like Build Forge support that, and we support integration with Build Forge, but are not likely to add similar features to RTC Build, especially given how commoditized the whole space has become with Jenkins (which we also integrate with). How many build engines are you dealing with? Is -sleepTime tweaking an option? If not, would you want to explore some kind of ordered engine list approach? 

2 other answers

Permanent link
Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER (64.6k●3●36●49) edited
Aug 05 '14, 4:39 a.m.
Rob,

as the answer to the first question suggests, there is nothing so far. The JBE is supposed to be a simple engine and not sophisticated. It works on build definitions. The first build engine available will poll the queues it supports and will grab the first pending build request it finds. You could try to explore the REST or java API's if you are able to force a build engine to do what you want. I am not sure that will work. 

Some links I have for API's: https://jazz.net/library/article/1229 , https://jazz.net/library/article/807 and all the stuff on my blog (there is no build related post however).

0 votes

Comments
Rob Leach
Aug 05 '14, 10:30 a.m.
Sure I agree (and I've read that in other articles), the JBE is supposed to be a simple engine. But it is also the richest of all the "engine" options out there for RTC builds. None of the other "engine" options offer the same level of rich functionality out of the box. Depending on how you setup your environment in terms of code flow (making use of a green build stream--having scheduled builds based on developer activity just to name a few), you lose all of that functionality when considering other build engines. Point in case if you setup your build environment to make use of a green build stream which automatically delivers change sets to an integration stream after a successful build, the JBE gives you that through a very simple mechanism. If I consider the build agent for a build engine as apposed to the JBE then you lose that; you're required to set that up for yourself via other means. The end result for me is a net loss. I have too much work already on my plate to now have to re-implement what we've already implemented using the JBE.

Permanent link
SEC Servizi
(971●2●38●60) edited
Jan 08 '19, 9:46 a.m.

We got a similar goal: we have different JBEs running on different hosts all supporting the same build definition. One of these JBEs runs on a "special" host and we have to force build definition to use this "special" JBE according to some build request properties.

Currently you can set the following property in your build engine definition: requestFilter. If you set this property to personalBuild=false then you can enforce no personal builds on the respective build engine (so you influenced some level of control). 
Based on this statement, we could set property requestFilter="personalBuild=true" on the "special" JBE and request a personal build to force picking up the "special" JBE. Correct?
Thanks in advance.
Cheers.

0 votes

Comments
SEC Servizi
Jan 08 '19, 9:43 a.m.
According to Build FAQ, we could even activate/deactivate all JBEs but the special one when we have to request this special build: You can prevent a given engine from picking up new builds by unchecking the 'Active' checkbox in the build engine editor; this does not affect any in-progress builds.

Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Jan 08 '19, 9:58 a.m.
It escapes me why you want to do all of that complex stuff if you could simply create a new build definition from the existing one and have the "special" build engine exclusively running on that build. 

SEC Servizi
Jan 08 '19, 10:07 a.m.
Ops, we forgotten to explicit the prerequisite that we can't create a new build definition.

Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Jan 09 '19, 4:33 a.m.
This makes even less sense, why would you not be able to create a new build definition? Get that fixed. 

SEC Servizi
Jan 10 '19, 3:46 a.m.
Well, there's nothing to get fixed: no new build definitions is a prerequisite. Cheers.

Ralph Schoon
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER Jan 10 '19, 12:29 p.m.
Well, good luck then. Cheers. 

SEC Servizi
Jan 11 '19, 6:19 a.m.
The build engine selection process appears to start before the Jazz server has started executing any code (ant scripts) that I have control over, so I don't think I can control that selection programmatically. Since we are launching build requests via ITeamBuildRequestService server API, we found that method #requestAndStartBuild accepts a build engine as input parameter. Unluckily this method does not accept other input parameters as method #requestBuild does but creating the correct param for method #requestBuild2 should do the trick... Cheers.

showing 5 of 7 show 2 more comments

Your answer

Register or log in to post your answer.

Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.

Ask a question
Guidelines
FAQ
Search context
Follow this question

By Email: 

Once you sign in you will be able to subscribe for any updates here.

By RSS:

Answers
Answers and Comments
Question details
× 6,181
× 563

Question asked: Aug 04 '14, 9:35 p.m.

Question was seen: 7,601 times

Last updated: Jan 11 '19, 6:19 a.m.

Confirmation Cancel Confirm