We've installed the Build Forge integration into our RTC 2.0.0.2 server. Our Build Forge server is at 7.1.2.2.

Though the setup works, it is rather unstable. The buildforge_service.log is periodically reporting the same exception, but are not sure how to resolve. Here are the issues we are seeing:

* Only a subset of builds requested from RTC start

* Of that subset, only a portion of those builds complete

* Things are somewhat more reliable when starting builds in the Build Forge console. The majority end up with a completed build result in RTC. However, every once in a while, the build result is not completed in RTC (even though the build completed in Build Forge).

In terms of users we have:

1. The Build Forge connector services (on the RTC side) are using a user with a Build System Client Access License. It is not the default RTC ADMIN user.
2. This is the same user that we use for our JBE build engines.
3. We created a user in Build Forge that has permissions to start the desired Build Forge project.
4. This is the user entered in the RTC build definition. For purposes of discussion it is called bfuser.

We are getting exceptions in the buildforge_service.log (produced by the RTC server). Quite frequently we see an exception followed by a build loop attempt. Then, it repeats. One puzzling thing is the log lists RunAs User is: ADMIN. This is puzzling because as noted in #1 above, we aren't using ADMIN in the configuration. If anyone has any hints about the exception loop below, it would be most appreciated!

EventPollerRunnable, Thu Jan 05 15:49:22 CST 2012: com.buildforge.services.client.dbo.RTCEvent.findAllCreatedAfter(RTCEvent.java:85)
BuildForgeEventPollerRunnable, 15:49:22: com.ibm.rational.buildforge.team.service.BuildForgeEventPollerRunnable.getRTCEvents(BuildForgeEventPollerRunnable.java:920)
BuildForgeEventPollerRunnable, 15:49:22: com.ibm.rational.buildforge.team.service.BuildForgeEventPollerRunnable.run(BuildForgeEventPollerRunnable.java:179)
BuildForgeEventPollerRunnable, 15:49:22: com.ibm.team.repository.service.internal.permission.ImpersonationService.runAs(ImpersonationService.java:44)
BuildForgeEventPollerRunnable, 15:49:22: sun.reflect.GeneratedMethodAccessor194.invoke(Unknown Source)
BuildForgeEventPollerRunnable, 15:49:22: sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
BuildForgeEventPollerRunnable, 15:49:22: java.lang.reflect.Method.invoke(Method.java:618)
BuildForgeEventPollerRunnable, 15:49:22: org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
BuildForgeEventPollerRunnable, 15:49:22: org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
BuildForgeEventPollerRunnable, 15:49:22: org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
BuildForgeEventPollerRunnable, 15:49:22: $Proxy80.runAs(Unknown Source)
BuildForgeEventPollerRunnable, 15:49:22: com.ibm.rational.buildforge.team.service.BuildForgeEventPollerScheduledTask.runTask(BuildForgeEventPollerScheduledTask.java:45)
BuildForgeEventPollerRunnable, 15:49:22: com.ibm.team.repository.service.async.AbstractAutoScheduledTask.executeTask(AbstractAutoScheduledTask.java:88)
BuildForgeEventPollerRunnable, 15:49:22: sun.reflect.GeneratedMethodAccessor187.invoke(Unknown Source)
BuildForgeEventPollerRunnable, 15:49:22: sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
BuildForgeEventPollerRunnable, 15:49:22: java.lang.reflect.Method.invoke(Method.java:618)
BuildForgeEventPollerRunnable, 15:49:22: org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
BuildForgeEventPollerRunnable, 15:49:22: org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
BuildForgeEventPollerRunnable, 15:49:22: org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
BuildForgeEventPollerRunnable, 15:49:22: $Proxy234.executeTask(Unknown Source)
BuildForgeEventPollerRunnable, 15:49:22: com.ibm.team.repository.service.internal.scheduler.AsynchronousTaskRunner.runTask(AsynchronousTaskRunner.java:118)
BuildForgeEventPollerRunnable, 15:49:22: com.ibm.team.repository.service.internal.scheduler.AsynchronousTaskRunner.run(AsynchronousTaskRunner.java:92)
BuildForgeEventPollerRunnable, 15:49:22: java.lang.Thread.run(Thread.java:811)
BuildForgeDataManager, 15:49:22: Removing connection from pool and disconnecting from BuildForge. CredKey: my.server.com:3966:bfuser
BuildForgeDataManager, 15:49:23: Connection disconnected: com.buildforge.services.client.api.APIClientConnection@35a535a5
BuildForgeDataManager, 15:49:23: Connection created: com.buildforge.services.client.api.APIClientConnection@2c3c2c3c
BuildForgeDataManager, 15:49:23: Authed with user/pass.
BuildForgeEventPollerRunnable, 15:49:23: RTC events found: []
BuildForgeEventPollerRunnable, 15:49:23: getBFEventsAndSort started.
BuildForgeEventPollerRunnable, 15:49:23: Sorted list of events.
BuildForgeEventPollerRunnable, 15:49:23: Setting time last checked: 0, for definition ID: _hYiRMDcbEeGLm_CVbjoNfg and server key: my.server.com:3966
BuildForgeEventPollerRunnable, 15:49:23: getBFEventsAndSort completed.
BuildForgeEventPollerRunnable, 15:49:23: []
BuildForgeBuildLoopRunnable, 15:49:37: Starting build loop run task, buildResultsMap: {Build Forge - Hello:9f2a70d10c571000d80d47fb2a5a2a5a:my.server.com:3966=com.ibm.team.build.internal.common.model.impl.BuildResultImpl@238f238f (stateId: , itemId: , origin: <unset>, immutable: true) (contextId: , modified: 2012-01-05 15:42:02.34, workingCopy: false) (mergePredecessor: null, workingCopyPredecessor: null, workingCopyMergePredecessor: null, predecessor: null) (buildStatus: OK, buildState: IN_PROGRESS, label: , buildTimeTaken: -1, buildStartTime: 1325799722249, summary: , ignoreWarnings: true, tags: , deleteAllowed: true, personalBuild: false), Build Forge - Hello:9f296f4d0c571000eb0b47fb2ae52ae5:my.server.com:3966=com.ibm.team.build.internal.common.model.impl.BuildResultImpl@2cc22cc2 (stateId: , itemId: , origin: <unset>, immutable: true) (contextId: , modified: 2012-01-05 15:42:00.794, workingCopy: false) (mergePredecessor: null, workingCopyPredecessor: null, workingCopyMergePredecessor: null, predecessor: null) (buildStatus: OK, buildState: IN_PROGRESS, label: , buildTimeTaken: -1, buildStartTime: 1325799720513, summary: , ignoreWarnings: true, tags: , deleteAllowed: true, personalBuild: false)}
BuildForgeBuildLoopRunnable, 15:49:43: response: com.ibm.team.process.internal.common.advice.impl.ItemsResponseImpl@14071407 (internalId: <unset>)
BuildForgeEventPollerRunnable, 15:49:43: RunAs User is: ADMIN
BuildForgeEventPollerRunnable, 15:49:43: Starting build loop run task, buildResultsMap: {Build Forge - Hello:9f2a70d10c571000d80d47fb2a5a2a5a:my.server.com:3966=com.ibm.team.build.internal.common.model.impl.BuildResultImpl@238f238f (stateId: , itemId: , origin: <unset>, immutable: true) (contextId: , modified: 2012-01-05 15:42:02.34, workingCopy: false) (mergePredecessor: null, workingCopyPredecessor: null, workingCopyMergePredecessor: null, predecessor: null) (buildStatus: OK, buildState: IN_PROGRESS, label: , buildTimeTaken: -1, buildStartTime: 1325799722249, summary: , ignoreWarnings: true, tags: , deleteAllowed: true, personalBuild: false), Build Forge - Hello:9f296f4d0c571000eb0b47fb2ae52ae5:my.server.com:3966=com.ibm.team.build.internal.common.model.impl.BuildResultImpl@2cc22cc2 (stateId: , itemId: , origin: <unset>, immutable: true) (contextId: , modified: 2012-01-05 15:42:00.794, workingCopy: false) (mergePredecessor: null, workingCopyPredecessor: null, workingCopyMergePredecessor: null, predecessor: null) (buildStatus: OK, buildState: IN_PROGRESS, label: , buildTimeTaken: -1, buildStartTime: 1325799720513, summary: , ignoreWarnings: true, tags: , deleteAllowed: true, personalBuild: false)}
BuildForgeEventPollerRunnable, 15:49:43: jobActivitiesTable: {Build Forge - Hello:9f2a70d10c571000d80d47fb2a5a2a5a:my.server.com:3966=com.ibm.team.build.internal.common.model.impl.BuildActivityIdImpl@69856985 (internalId: <unset>) (id: _GujdoTfmEeGGDMYrL0CYwQ), Build Forge - Hello:9f296f4d0c571000eb0b47fb2ae52ae5:my.server.com:3966=com.ibm.team.build.internal.common.model.impl.BuildActivityIdImpl@79a179a1 (internalId: <unset>) (id: _GhtUcDfmEeGGDMYrL0CYwQ)}
BuildForgeEventPollerRunnable, 15:49:43: stepActivitiesTable: {}
BuildForgeEventPollerRunnable, 15:49:43: serverEventCreatedTimeTable: {_hYiRMDcbEeGLm_CVbjoNfg:my.server.com:3966=1325799254000}
BuildForgeEventPollerRunnable, 15:49:43: Adding engine to list: com.ibm.team.build.internal.common.model.impl.BuildEngineImpl@c2f0c2f (stateId: , itemId: , origin: <unset>, immutable: true) (contextId: , modified: 2012-01-05 14:56:28.353, workingCopy: <unset>) (mergePredecessor: null, workingCopyPredecessor: <unset>, workingCopyMergePredecessor: <unset>, predecessor: ) (supportsCancellation: false, engineContactInterval: 0, useTeamScheduler: false, id: RationalBuildForgeConnector, active: true)
BuildForgeEventPollerRunnable, 15:49:43: Build definition name: Build Forge - Hello
BuildForgeEventPollerRunnable, 15:49:43: Processing this build definition: com.ibm.team.build.internal.common.model.impl.BuildDefinitionImpl@49bd49bd (stateId: , itemId: , origin: <unset>, immutable: true) (contextId: , modified: 2012-01-05 15:23:40.358, workingCopy: <unset>) (mergePredecessor: null, workingCopyPredecessor: <unset>, workingCopyMergePredecessor: <unset>, predecessor: ) (id: Build Forge - Hello, description: , ignoreWarnings: true)
BuildForgeEventPollerRunnable, 15:49:43: Project data: project name: Hello, uuid: 4bac3d2f0c571000926647fb2b3c2b3c
BuildForgeEventPollerRunnable, 15:49:43: Connection data: userid: bfuser, hostname: my.server.com, port: 3966
BuildForgeEventPollerRunnable, 15:49:43: Connection data: getAllLogs: false, getLogsNotPassedOrSkipped: true, getCustomBomData: true, numFirstLogs: 5, numLastLogs: 5, numFirstLogsEnabled: true, numLastLogsEnabled: true
BuildForgeDataManager, 15:49:43: Connection returned from pool for key: my.server.com:3966:bfuser = com.buildforge.services.client.api.APIClientConnection@2c3c2c3c
BuildForgeEventPollerRunnable, 15:49:43: Time last checked: 1325799254000, for definition ID: _hYiRMDcbEeGLm_CVbjoNfg and server key: my.server.com:3966
same exception stack trace as earlier...things continue

Is it rock-solid stable under 3.0.1? Or, will there still be periodic instability? Will I need to periodically restart RTC and/or Build Forge to keep it stable?

I setup a local RTC 3.0.1 server and, so far, the Build Forge integration is rock solid. I can submit a bunch of builds from RTC. They all get picked up, a job starts and completes in Build Forge, then the build request is completed in RTC. Perfect. I was happy to see that it picked up more than one build request instead of processing them serially. With lots of code deliveries, we'll need the ability for more than one build to be running at once.

Upgrading to 3.0.1 may be an option for us; we'll have to see. I also opened a support ticket with IBM Support to get their take on the RTC 2.0.0.2 Build Forge integration instability.