setting priority of multiple build engines
Hi,
We have multiple build definitions and multiple build engines. Is there a way to prioritize build engines so that you can influence which order build engines are picked by any given build definition.
Here is the scenarios; for simplicity let's say we have three build definitions and two build engines. All three of the build definitions are configure to have both build engines as their "supporting build engines". But with the list of two build engines, let's say we always want jbe1 to be picked first if it is not busy, and then jbe2 pick as a second choice.
Obviously our actual setup is much more complex with higher numbers of build definitions and build engines. But the principle is still the goal. Set the priority of supporting build engines so we can influence the order in which build engines are picked.
I have found a couple of questions and an article that at first looked promising, but did not lead anywhere meaningful. This question had an answer that suggested the REST API, but that would be more work that seems appropriate for the simplicity of the functionality. This question isn't relevant for our needs. And then this article doesn't deal with our question.
Cheers,
-Rob
|
Accepted answer
Nick Edgar (6.5k●7●11)
| answered Aug 06 '14, 12:52 p.m.
JAZZ DEVELOPER edited Aug 06 '14, 12:54 p.m.
As Ralph said, priority-based assignment of builds to engines is not currently supported out-of-the-box. The only property the requestFilter mechanism supports currently is the one for personalBuild.
The best you can do now is adjust the -sleepTime arg to JBE to have the preferred instances polling more frequently, increasing the chance that they'll pick up a request first. But this is non-deterministic.
It would be relatively straightforward to write a script to pull requests off the queue, then invoke JBE as a one-off for that particular build, on the next available preferred machine, e.g. via ssh. If you're interested in exploring that, I'd be happy to help.
Rob Leach selected this answer as the correct answer
Comments You know what Nick, that isn't too bad of an option (adjusting the sleepTime). But as you point out, it would definitely not be an exacting solution. If a working model is implemented it is probably one that would make me cringe each time I remember how it works.
Your second solution seems like it would just be easier to implement an agent and figure out how to provide the missing functionality/features that the JBE provides out of the box. As a customer it simply seems like a waste of my time when I believe the tool should provide that functionality/ability to me (so that I can use it where and as needed).
I understand and appreciate your point about "it should just work out of the box". The problem is that, architecturally, JBE works as a client of the RTC server, and is independent of other running JBE instances, so there's no central coordination over which builds run where. You can partition build definitions across different engines in the build farm, but there's still no preferential matching to different engines (or other capabilities of the build machines, e.g. OS / compilers / other tools). Enterprise grade build systems like Build Forge support that, and we support integration with Build Forge, but are not likely to add similar features to RTC Build, especially given how commoditized the whole space has become with Jenkins (which we also integrate with).
How many build engines are you dealing with? Is -sleepTime tweaking an option? If not, would you want to explore some kind of ordered engine list approach?
|
2 other answers
Ralph Schoon (61.8k●3●36●43)
| answered Aug 05 '14, 4:38 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER edited Aug 05 '14, 4:39 a.m.
Rob,
as the answer to the first question suggests, there is nothing so far. The JBE is supposed to be a simple engine and not sophisticated. It works on build definitions. The first build engine available will poll the queues it supports and will grab the first pending build request it finds. You could try to explore the REST or java API's if you are able to force a build engine to do what you want. I am not sure that will work. Some links I have for API's: https://jazz.net/library/article/1229 , https://jazz.net/library/article/807 and all the stuff on my blog (there is no build related post however). Comments Sure I agree (and I've read that in other articles), the JBE is supposed to be a simple engine. But it is also the richest of all the "engine" options out there for RTC builds. None of the other "engine" options offer the same level of rich functionality out of the box. Depending on how you setup your environment in terms of code flow (making use of a green build stream--having scheduled builds based on developer activity just to name a few), you lose all of that functionality when considering other build engines. Point in case if you setup your build environment to make use of a green build stream which automatically delivers change sets to an integration stream after a successful build, the JBE gives you that through a very simple mechanism. If I consider the build agent for a build engine as apposed to the JBE then you lose that; you're required to set that up for yourself via other means. The end result for me is a net loss. I have too much work already on my plate to now have to re-implement what we've already implemented using the JBE. |
We got a similar goal: we have different JBEs running on different hosts all supporting the same build definition. One of these JBEs runs on a "special" host and we have to force build definition to use this "special" JBE according to some build request properties. Currently you can set the following property in your build engine definition: requestFilter. If you set this property to personalBuild=false then you can enforce no personal builds on the respective build engine (so you influenced some level of control).Based on this statement, we could set property requestFilter="personalBuild=true" on the "special" JBE and request a personal build to force picking up the "special" JBE. Correct?
Thanks in advance.
Cheers.
Comments According to Build FAQ, we could even activate/deactivate all JBEs but the special one when we have to request this special build: You can prevent a given engine from picking up new builds by unchecking the 'Active' checkbox in the build engine editor; this does not affect any in-progress builds. It escapes me why you want to do all of that complex stuff if you could simply create a new build definition from the existing one and have the "special" build engine exclusively running on that build. Ops, we forgotten to explicit the prerequisite that we can't create a new build definition. This makes even less sense, why would you not be able to create a new build definition? Get that fixed. Well, there's nothing to get fixed: no new build definitions is a prerequisite. Cheers.
Ralph Schoon
commented Jan 10 '19, 12:29 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Well, good luck then. Cheers. The build engine selection process appears to start before the Jazz server has started executing any code (ant scripts) that I have control over, so I don't think I can control that selection programmatically.
Since we are launching build requests via ITeamBuildRequestService server API, we found that method #requestAndStartBuild accepts a build engine as input parameter. Unluckily this method does not accept other input parameters as method #requestBuild does but creating the correct param for method #requestBuild2 should do the trick...
Cheers.
showing 5 of 7
show 2 more comments
|
Comments
Oh and I should add a quick note. I'm sure a build setup using Jenkins could possibly solve this problem, but we are far out from migrating to Jenkins anytime soon; so that is not an option for us at this moment.