Run multiple personal builds of same build definition in parallel on one jenkins
Is it possible to run multiple personal builds of the same build definition in parallel on one jenkins with multiple slaves?
Usecase: Multiple developer are working on the same stream. Before they deliver their changes they would like to trigger a personal build. In a huge project the build takes approx. 1 h. If every single developer is requestion a personal build with the same build definition there will be a large queue on jenkins side. That means, they have to wait a long time until their build is finished. Is there any way to run these jobs in parallel without creating a single build definition for each developer on the same jenkins with multiple slaves? Thanks for any input and advice. |
Accepted answer
Jenkins allows for parallel execution of builds for a Job.
Job configuration page has a check box, "Execute concurrent builds if necessary". Also, in the master node configuration set the "# of executors" field to more than 1. Once these two are done, parallel job execution is enabled. This should work for triggering personal builds from RTC build definition as well. To confirm I tried the mentioned scenario with no slaves, but was able to successfully trigger, execute and complete multiple personal builds triggered by different users from the same build definition. Each of the builds had their own workspace, with workspace name being appended with "@<number>". This is explained in context help of the "Execute concurrent builds if necessary". Also they warn about using this setting with custom workspace, in which case the workspace might be overwritten. Christian Giesa selected this answer as the correct answer
|
One other answer
Ralph Schoon (63.5k●3●36●46)
| answered Aug 03 '16, 2:04 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER edited Aug 03 '16, 2:06 a.m.
There is no free lunch. If a build takes an hour, running 4 in parallel will take (more than) 4 hours.
The only option you have is to add more build engines and likely more build hardware. There are ways to do that in Jenkins. Not my specialty though. You can probably run more than one build process/engine on one server without over-committing it, but there will be limits. If you have multiple build engines, each build engine will pick up the next pending request if it finishes the current request. Also consider a caching proxy between the repository and the build engines. This might be also of interest: https://rsjazz.wordpress.com/2013/11/22/seperating-personal-from-other-builds/ Comments Thanks, Ralph. The option I was looking for in the jenkins job configuration is "Execute concurrent builds if necessary". This was disabled by default in our configuration. Now the builds are running in parallel on the slaves and do not block the other ones.
Christian, please be aware that Jenkins also supports master slave configurations. Please be aware that there can be racing conditions if you run the same build with the same repository workspace in parallel and accept changes into this. You can create such conditions by specific automation, It should not happen in your scenario, but you should be aware.
Thanks for the information. As I understood when running a personal build the user will specify his own repository workspace, but when running for example an integration build always the same repository given in the build definition will be taken. Due to that it would be good to create two different jenkins jobs and build definitions one for personal builds and one for the integration builds. For personal builds I would allow "execute concurrent builds if necessary" for the other one I will avoid running this job in parallel. This would avoid conflicts or inconsistencies in the repository workspace of the integration build job. Am I right? The personal builds are not an issue, as the repository workspace is just loaded.
|
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.