It's all about the answers!

Ask a question

Why do parallel job steps still poll for an available server?


John Bobinyec (683432) | asked May 08 '14, 3:48 p.m.
I have a project that has several parallel job steps.  Only two can run at a time.  This has been set by the maximum number of jobs that can run on the server.  All jobs run on the same server.  So, while two of the jobs are running, the rest of them are apparently polling to see when the server becomes available.  Why?  The Management Console machine should already know that the maximum number of job steps are running.  It should also know that when one of these jobs finishes, then the server will become available again.

Also, while these job steps are polling, their run clock is ticking.  It shows all of the accumulated wait time and makes it look like the job took an inordinately long time to run, whereas it was mostly just waiting to get started.

Build Forge 8.0.0.1.

Thanks,
John Bobinyec

Accepted answer


permanent link
Ryan Ruscett (1.0k413) | answered May 08 '14, 4:10 p.m.
 Hello,

The start time of a threaded step depends on the availability of the server. If the step cannot be started, the system waits and tries again. You can't control which step starts first. This is because the design is for the console to say, oh threaded step, let's run the next one. There has to be another one right? Kicks off the next step. Thread enabled, let's kick off another step. Oh wait, server limit has been reached. Let's just sit and wait. 

Let say for example you had 40 threaded steps. Now, these steps could be running on 40 different servers or 10. It's easier for the console to have a step reach out to it's desired server than it is to pool the potentially endless amounts of threaded steps on however many servers, log that and use that dynamically changing data to run a query against something so that the job can then kick off. Plus, I could see a race condition causing issues here too. 

If they run on the same server with max jobs 2. I would set the max threaded steps to 2 instead of the server max jobs. Just an option.

I hope this helps and doesn't confuse you more. 
Thanks!
John Bobinyec selected this answer as the correct answer

Comments
John Bobinyec commented May 08 '14, 7:36 p.m.

Thanks.  The max threaded steps sounds like what I need.  How do I do that?  The project has a property called Max Threads.  It's set to Unlimited and I can't change it.  When the mouse hovers over it, it turns into a circle with a slash through it.

jb

2 other answers



permanent link
Ryan Ruscett (1.0k413) | answered May 09 '14, 9:48 a.m.
Hey John,

That is a very good point. If you can't manage the threads in that fashion you are forced to use the Max Jobs on the server, which brings us back around full circle to that step after the threaded steps is sitting and waiting for an opening all while the click is ticking and extending the job total time. This may or may not have been considered when deciding to not make this feature work for the java engine. All I could suggest is creating an RFE for MJC Thread Throttling or something along those lines.

Thanks!

Comments
Ryan Ruscett commented May 09 '14, 9:52 a.m. | edited May 23 '14, 11:18 a.m.

There is actually a push to add this flow control feature back into the product. Hang on for a bit longer. Do not quote me, but I do believe this feature is coming back and will be available to the either java or pearl engines in an upcoming release.

Thanks!


permanent link
Ryan Ruscett (1.0k413) | answered May 09 '14, 8:03 a.m.
aaahhh Yeah that is a defect at the moment. I thought it was fixed in 8.0.0.1 but I guess not. Not to worry, it will be fixed in the next release I imagine. Don't quote me but you are not the only one running into this issue. 

Thanks for using the forums and participating.

Comments
John Bobinyec commented May 09 '14, 8:41 a.m.

I just checked the bug reports.  The Max Threads property no longer works because it is deprecated in the MJC.  The solution seems to be that it will no longer be displayed on the panel.

Got any other suggestions?  What's the proper way to regulate how many threads run on a particular server?  Max Jobs on the server?

Thanks,
John Bobinyec

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.