It's all about the answers!

Ask a question
QuestionsTagsUsersBadges
Follow this question
Question details

×5,999
×1,192

question asked: Aug 20 '13, 9:59 a.m.
question was seen: 6,782 times
last updated: Sep 10 '13, 6:30 p.m.

Related questions

FAQ - How this Forum works

Who shut down Build Forge??

0
Jozef Vandenmooter (163●3●29●33) | asked Aug 20 '13, 9:59 a.m.
edited Aug 26 '13, 2:41 p.m.

Hi guys,


Build Forge was down yesterday morning. When I checked the catalna log, I saw it lost communication with its database at one point. This has happened before (we have network issues). However, I also see the following in the log (that I'd never seen before), about an hour after the communication problem occurred:


Aug 19, 2013 9:26:12 AM org.apache.catalina.core.StandardServer await

INFO: A valid shutdown command was received via the shutdown port. Stopping the Server instance.

Aug 19, 2013 9:26:12 AM org.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler ["http-bio-8080"]


Everybody swears they did not stop the engine. Is it possible Build Forge sent this message to itself and shut itself down? Does it do that if it realizes its database connection is down?

 

Thanks,

 

Jozef

Accepted answer

1
permanent link
Jason Kelly (41●1) | answered Aug 21 '13, 3:44 p.m.
 Build Forge has no way of shutting itself down like this.  The only way for this to happen is that someone or something told it to shut down.  Build Forge does not restart when it drops a connection to the database.   Is it possible that an automated process told it to shut down or maybe a system reboot?  
Jozef Vandenmooter selected this answer as the correct answer
Comments
Jozef Vandenmooter commented Aug 21 '13, 4:37 p.m.

Thanks Jason. That server has not been rebooted in weeks, so that can't be it. Now that I know it wasn't BF I'll be looking elsewhere for the reason.

Jozef

Jozef Vandenmooter commented Aug 26 '13, 2:49 p.m.

Jason,

It actually happened again. Build Forge lost communication with its database and 40 minutes later Apache shut itself down.

Aug 22, 2013 9:23:51 AM com.buildforge.services.server.event.RTCEventReaper run
SEVERE: Exception during RTCEvent reaping
Throwable occurred: com.buildforge.services.common.db.DBException: {SQLException during retrieval of connection from pool}
...
Aug 22, 2013 10:03:37 AM org.apache.catalina.core.StandardServer await
INFO: A valid shutdown command was received via the shutdown port. Stopping the Server instance.

I had to manually restart the engine to get Build Forge operational again.

The previous time there was also 40 minutes between BF losing network connection and turning itself off. I went through the Windows Security event logs and there is no record of anyone logging in around those times (each logon/logoff is recorded in that log).

I would really like to understand this behavior.

Thanks,

Jozef

1
Joshua Keefe commented Sep 10 '13, 6:30 p.m.

Build Forge is a database centric product, and any absence of the database connection will prevent Build Forge from functioning properly.

If the database connection is lose, the expected behavior is the following.

  • Apache/PHP loses the database connection, and will re-attempt to connect periodically until successful.
  • Tomcat rbf-services loses the database connection, and will re-attempt to connect periodically until successful.
  • Build Forge engine process will immediately shutdown and die.
Given that the Build Forge engine process dies once the database connection is lost, it will be required to restart Build Forge completely to bring the console back up.

So, we could look into why Tomcat was shutdown 40 minutes after the database connection was lost, but it would be moot to the fact that the Build Forge engine was already gone and a full restart of Build Forge will already be required.

One other answer

Most liked answers ↓|Newest answers|Oldest answers

0
permanent link
Ryan Ruscett (1.0k●4●13) | answered Aug 27 '13, 10:11 a.m.
 Is it possible that there is a windows dependency that was created to shut down Build Forge services when it loses connectivity to the Database. I could see this being a potential good practice, especially with redundancy. 
Comments
Ryan Ruscett commented Aug 27 '13, 10:22 a.m.

 Two things can happen when the database loses connectivity.


1. When the Database loses connectivity. The engine shuts down immediately. PHP and APACHE can not communicate with the Engine.
2. Tomcat can not communicate.

There was an IBM JDK defect regarding the number of server tests that are done. When you have a high amount of invalid servers where server test can not go through. Apache has been known to receive a signal much like the one you noted above in your error. Except, just Apache shuts down. Tomcat is still left up. Is this what you are seeing?

For further diagnostics of this issue, it may prove beneficial to open a PMR. Thanks!

Your answer

Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.