It's all about the answers!

Ask a question

Failed CC Synchronization Import into RTC


Roderick Thomas (661179) | asked Dec 07 '10, 10:44 p.m.
Our SCM team has been testing and evaluating the CC Synchronizer for a while now. We have a stream in CC that is comprised of ~15 components and have been able to successfully import that stream with the components into a test project with relative ease.

We're attempting the same import with a real project and have had a number of issues. Specifically, the synchronization was kicked off and in the current activity section, it was reported to be listing the directories in clearcase. It ran for hours and eventually the process had to be killed manually. This component has consistently failed since. We're going to attempt to load the component (which contains ~4k files and ~1k folders) by pieces.

The command running at the time of failure is: cleartool ls -recurse -short <path>

Are there concerns with loading the component in the proposed piecemeal way?

Is there anyway to troubleshoot issues with respect to loading? It's not a scalable solution to have our SCM team try to import existing components piecemeal.

While in the middle of importing, there doesn't appear to be a way to cancel/abort the import. Is there a command we're simply missing on the context menu? Are there ramifications for killing the import process manually (since there doesn't appear to be another option through the tool)?

4 answers



permanent link
Geoffrey Clemm (30.1k33035) | answered Dec 08 '10, 11:08 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Comments below.

On 12/7/2010 10:53 PM, roderick.thomas wrote:
Our SCM team has been testing and evaluating the CC Synchronizer for a
while now. We have a stream in CC that is comprised of ~15 components
and have been able to successfully import that stream with the
components into a test project with relative ease.

So far, so good (:-).

We're attempting the same import with a real project and have had a
number of issues. Specifically, the synchronization was kicked off
and in the current activity section, it was reported to be listing
the directories in clearcase. It ran for hours and eventually the
process had to be killed manually. This component has consistently
failed since. We're going to attempt to load the component (which
contains ~4k files and ~1k folders) by pieces.

The command running at the time of failure is: cleartool ls -recurse
-short<path

The first thing to do in a case like this is to run the cleartool
command directly from the command line. Note that the synchronizer is
reporting the exact command it is running through cleartool ... not some
internal variant. So when you run that command, it is doing precisely
what the synchronizer is doing.

Are there concerns with loading the component in the proposed
piecemeal way?

No problem with doing that (other than the user annoyance of doing so).
Also, note that you can specify a whole set of roots in one request ...
that way, if it makes it pasts the first piece, it will automatically
try the next one. And if a piece fails, it will automatically try the
next one. And if one piece hangs (more comments below), you can kill
that process, and your next sync request will start up at the one that
didn't succeed.

Is there anyway to troubleshoot issues with respect to loading? It's
not a scalable solution to have our SCM team try to import existing
components piecemeal.

The simplest approach is to see what cleartool command is taking an
unexpected amount of time. The "activities" reported back to the sync
result give you a rough idea of how the operations are taking (Rational
support can tell you how to adjust the level of operation tracing that
occurs).

While in the middle of importing, there doesn't appear to be a way to
cancel/abort the import. Is there a command we're simply missing on
the context menu? Are there ramifications for killing the import
process manually (since there doesn't appear to be another option
through the tool)?

The only way to kill it at the moment is to kill the operating system
process that is performing the sync (on the sync host). You usually
have to "unlock" that sync stream (using the "unlock" command in the
sync editor or the sync properties page. Then you can just request the
sync, and it will re-try any of the sync roots that weren't successfully
imported on the previous request.

Cheers,
Geoff

permanent link
Roderick Thomas (661179) | answered Dec 09 '10, 10:34 a.m.
So we tried the piecemeal approach and ran into further issues: Synchronization failed: Deferred operation failed. For instance, we're seeing stack traces like below.

To your point about running the command via the cmd prompt, the team has done that and it completes as expected. Is there nothing else happening that could impact the result (e.g., timeout properties)?


javax.wvcm.WvcmException: Deferred operation failed
at com.ibm.team.connector.scm.client.JzProvider.wrapException(JzProvider.java:2145)
at com.ibm.team.connector.scm.client.JzRepoWorkspace.doUpdate(JzRepoWorkspace.java:1126)
at com.ibm.team.connector.scm.client.JzRepoWorkspace.doUpdate(JzRepoWorkspace.java:1046)
at com.ibm.team.connector.scm.client.JzRepoWorkspace.doUpdate(JzRepoWorkspace.java:976)
at com.ibm.rational.wvcm.ri.srvc.SrvcResource.doUpdate(SrvcResource.java:1808)
at com.ibm.rational.wvcm.ri.impl.WvcmServiceConverter.doUpdate(WvcmServiceConverter.java:1187)
at com.ibm.rational.wvcm.ri.impl.WorkspaceImpl.doUpdate(WorkspaceImpl.java:181)
at com.ibm.rational.wvcm.interop.InteropStream.internalSync(InteropStream.java:2569)
at com.ibm.rational.wvcm.interop.InteropStream.doOperation(InteropStream.java:3941)
at com.ibm.team.connector.scm.operations.ConnectorOps.synchronizeStream(ConnectorOps.java:209)
at com.ibm.team.connector.scm.cli.AbstractCcConnectorSubCommand.synchronizeStream(AbstractCcConnectorSubCommand.java:184)
at com.ibm.team.connector.scm.cli.SyncSubcommand.synchronizeStream(SyncSubcommand.java:112)
at com.ibm.team.connector.scm.cli.SyncSubcommand.run(SyncSubcommand.java:74)
at com.ibm.team.filesystem.cli.core.internal.SubcommandLauncher.run(Unknown Source)
at com.ibm.team.filesystem.cli.core.internal.SubcommandLauncher.doStart(Unknown Source)
at com.ibm.team.filesystem.cli.core.internal.SubcommandLauncher.run(Unknown Source)
at com.ibm.team.filesystem.cli.core.internal.Application.start(Unknown Source)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(Unknown Source)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(Unknown Source)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(Unknown Source)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(Unknown Source)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at org.eclipse.equinox.launcher.Main.invokeFramework(Unknown Source)
at org.eclipse.equinox.launcher.Main.basicRun(Unknown Source)
at org.eclipse.equinox.launcher.Main.run(Unknown Source)
Caused by: com.ibm.team.repository.common.InternalRepositoryException: Deferred operation failed
at com.ibm.team.repository.service.internal.RdbRepositoryDataMediator.preCommitNotify(RdbRepositoryDataMediator.java:1561)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$WriteExclusiveTransaction.completeTransaction(RepositoryDatabase.java:666)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$Transaction.finishTransaction(RepositoryDatabase.java:500)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$1.run(RepositoryDatabase.java:293)
at com.ibm.team.repository.service.internal.rdb.ConnectionPoolService.withCurrentConnection(ConnectionPoolService.java:331)
at sun.reflect.GeneratedMethodAccessor98.invoke(null)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy22.withCurrentConnection(null)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runTransaction(RepositoryDatabase.java:288)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runInTransaction(RepositoryDatabase.java:244)
at com.ibm.team.repository.service.internal.TransactionService.runInTransaction(TransactionService.java:79)
at sun.reflect.GeneratedMethodAccessor114.invoke(null)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy44.runInTransaction(null)
at com.ibm.team.scm.service.internal.AbstractScmService.runInTransaction(AbstractScmService.java:310)
at com.ibm.team.scm.service.internal.AbstractScmService.runInTransaction(AbstractScmService.java:389)
at com.ibm.team.scm.service.internal.ScmService.updateComponents(ScmService.java:3562)
at sun.reflect.GeneratedMethodAccessor565.invoke(null)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy274.updateComponents(null)
at sun.reflect.GeneratedMethodAccessor565.invoke(null)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleMethod(AbstractTeamServerServlet.java:1170)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.executeMethod(AbstractTeamServerServlet.java:926)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.doPost(AbstractTeamServerServlet.java:728)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest2(AbstractTeamServerServlet.java:1773)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest(AbstractTeamServerServlet.java:1642)
at com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1555)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:75)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:121)
at com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:54)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:873)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:811)
at com.ibm.team.repository.common.internal.marshal.util.MarshallerUtil.decodeExceptions(MarshallerUtil.java:326)
at com.ibm.team.repository.common.internal.marshal.util.MarshallerUtil.decodeExceptions(MarshallerUtil.java:296)
at com.ibm.team.repository.common.internal.marshal.util.MarshallerUtil.decodeFault(MarshallerUtil.java:261)
at com.ibm.team.repository.transport.client.RemoteTeamService.constructExceptionFromFault(RemoteTeamService.java:613)
at com.ibm.team.repository.transport.client.RemoteTeamService.executeMethod(RemoteTeamService.java:483)
at com.ibm.team.repository.transport.client.RemoteTeamService.invoke(RemoteTeamService.java:201)
at com.ibm.team.repository.transport.client.ServiceInvocationHandler.invoke(ServiceInvocationHandler.java:43)
at $Proxy17.updateComponents(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at com.ibm.team.repository.client.internal.ServiceInterfaceProxy.invokeServiceCall(ServiceInterfaceProxy.java:163)
at com.ibm.team.repository.client.internal.ServiceInterfaceProxy.invoke(ServiceInterfaceProxy.java:87)
at $Proxy17.updateComponents(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at com.ibm.team.scm.client.internal.ScmServiceInterfaceProxy$2.run(ScmServiceInterfaceProxy.java:164)
at com.ibm.team.repository.client.internal.TeamRepository$3.run(TeamRepository.java:1211)
at com.ibm.team.repository.common.transport.CancelableCaller.call(CancelableCaller.java:79)
at com.ibm.team.repository.client.internal.TeamRepository.callCancelableService(TeamRepository.java:1204)
at com.ibm.team.scm.client.internal.ScmClientLibraryContext.callCancelableService(ScmClientLibraryContext.java:70)
at com.ibm.team.scm.client.internal.ScmServiceInterfaceProxy.invokeCancelableService(ScmServiceInterfaceProxy.java:161)
at com.ibm.team.scm.client.internal.ScmServiceInterfaceProxy.invoke(ScmServiceInterfaceProxy.java:91)
at $Proxy17.updateComponents(Unknown Source)
at com.ibm.team.scm.client.internal.FlowNodeConnection.replaceComponents(FlowNodeConnection.java:686)
at com.ibm.team.scm.client.internal.FlowNodeConnection.replaceComponent(FlowNodeConnection.java:569)
at com.ibm.team.connector.scm.client.JzRepoWorkspace.doUpdate(JzRepoWorkspace.java:1103)
... 27 more
Caused by: com.ibm.team.repository.common.TeamRepositoryException: deferred save failed
at com.ibm.team.repository.service.internal.RdbRepositoryDataMediator.saveItemInTxn(RdbRepositoryDataMediator.java:490)
at com.ibm.team.repository.service.internal.RepositoryItemService.doSaveItemInTxn(RepositoryItemService.java:1191)
at com.ibm.team.repository.service.internal.RepositoryItemService.access$11(RepositoryItemService.java:1184)
at com.ibm.team.repository.service.internal.RepositoryItemService$14.run(RepositoryItemService.java:1165)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runTransaction(RepositoryDatabase.java:320)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runInTransaction(RepositoryDatabase.java:244)
at com.ibm.team.repository.service.internal.TransactionService.runInTransaction(TransactionService.java:79)
at sun.reflect.GeneratedMethodAccessor114.invoke(null)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy44.runInTransaction(null)
at com.ibm.team.repository.service.internal.RepositoryItemService.saveItemInTxn(RepositoryItemService.java:1163)
at sun.reflect.GeneratedMethodAccessor173.invoke(null)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy74.saveItemInTxn(null)
at com.ibm.team.repository.service.internal.ComponentRepositoryItemService.saveItemInTxn(ComponentRepositoryItemService.java:557)
at com.ibm.team.scm.service.internal.CacheItemServiceWrapper.saveItemInTxn(CacheItemServiceWrapper.java:233)
at com.ibm.team.repository.service.internal.RepositoryTransactionContext.save(RepositoryTransactionContext.java:703)
at com.ibm.team.scm.service.internal.ScmTransactionContext.save(ScmTransactionContext.java:1080)
at com.ibm.team.scm.service.internal.AbstractScmService$2.run(AbstractScmService.java:412)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$Transaction.run(RepositoryDatabase.java:466)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$1.run(RepositoryDatabase.java:292)
... 119 more
Caused by: com.ibm.team.repository.common.InternalRepositoryException: CRJAZ0447I SQL statement execution failed.Vendor specific exception
SQL: INSERT INTO RTC.SCM_HISTORIC_BASIS (STATE_ID, ITEM_ID, CONTEXT_ID, MODIFIED, MODIFIED_BY_ITEM_ID, BASIS_ITEM_ID, INDEX_COL, WORKSPACE_ITEM_ID, COMPONENT_ITEM_ID, LATEST_REMOVED, REPLACE) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
SQL Exception #1
SQL Message: ORA-01654: unable to extend index RTC.SCM_HISTORIC_BASIS_WS_CRI by 8 in tablespace RTC_DATA

SQL State: 72000
Error Code: 1654
Class: com.ibm.team.repository.service.internal.dataaccess.write2.Row
Item Handle: com.ibm.team.scm.common.internal.impl.HistoricBasisImpl@3dcb3dcb (stateId: , itemId: , origin: <unset>, immutable: true) (contextId: , modified: 2010-12-09 01:39:10.039, workingCopy: false) (predecessor: <unset>) (deliveryDate: 2010-12-09 01:39:09.781, index: 47, latestRemoved: false, replace: true)
at com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.createIRE(BatchingRowReceiver.java:417)
at com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.createIRE(BatchingRowReceiver.java:426)
at com.ibm.team.repository.service.internal.dataaccess.write2.Row.handleExecuteException(Row.java:96)
at com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.simulatedBatch(BatchingRowReceiver.java:396)
at com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.executeBatch(BatchingRowReceiver.java:269)
at com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.execute(BatchingRowReceiver.java:196)
at com.ibm.team.repository.service.internal.dataaccess.IRowReceiver$IRowReceiverLocator.executePendingChanges(IRowReceiver.java:140)
at com.ibm.team.repository.service.internal.dataaccess.IRowReceiver$IRowReceiverLocator.executePendingChanges(IRowReceiver.java:118)
at com.ibm.team.repository.service.internal.RdbRepositoryDataMediator.preCommitNotify(RdbRepositoryDataMediator.java:1534)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$WriteExclusiveTransaction.completeTransaction(RepositoryDatabase.java:666)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$Transaction.finishTransaction(RepositoryDatabase.java:500)
at com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$1.run(RepositoryDatabase.java:293)
... 119 more
Caused by: java.sql.SQLException: ORA-01654: unable to extend index RTC.SCM_HISTORIC_BASIS_WS_CRI by 8 in tablespace RTC_DATA

at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:205)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1008)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3449)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3530)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1062)
at com.ibm.team.repository.service.internal.db.jdbcwrappers.stat.PreparedStatementStatWrapper.executeUpdate(PreparedStatementStatWrapper.java:62)
at com.ibm.team.repository.service.internal.db.jdbcwrappers.errlog.PreparedStatementErrLogWrapper.executeUpdate(PreparedStatementErrLogWrapper.java:86)
at com.ibm.team.repository.service.internal.db.jdbcwrappers.leak.PreparedStatementLeakWrapper.executeUpdate(PreparedStatementLeakWrapper.java:58)
at com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.simulatedBatch(BatchingRowReceiver.java:388)
... 127 more

permanent link
Geoffrey Clemm (30.1k33035) | answered Dec 09 '10, 10:08 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Hi Roderick,

It looks like you have some significant problems with your RTC
configuration (I noticed that you reported other strange RTC errors
unrelated to the synchronizer). When you work with support on your
other RTC error, you'll want to mention this stack trace to them as
well, since that might help them track down what is wrong, in
particular, the "deferred operation" is just the checkin of a set of
changes.

SQL Exception #1
SQL Message: ORA-01654: unable to extend index
RTC.SCM_HISTORIC_BASIS_WS_CRI by 8 in tablespace RTC_DATA

I'm not a database person, but this might indicate either a disk space
problem, or a permissions problem.

Cheers,
Geoff

On 12/9/2010 10:38 AM, roderick.thomas wrote:
So we tried the piecemeal approach and ran into further issues:
Synchronization failed: Deferred operation failed. For instance,
we're seeing stack traces like below.

To your point about running the command via the cmd prompt, the team
has done that and it completes as expected. Is there nothing else
happening that could impact the result (e.g., timeout properties)?


javax.wvcm.WvcmException: Deferred operation failed
at
com.ibm.team.connector.scm.client.JzProvider.wrapException(JzProvider.java:2145)
at
com.ibm.team.connector.scm.client.JzRepoWorkspace.doUpdate(JzRepoWorkspace.java:1126)
at
com.ibm.team.connector.scm.client.JzRepoWorkspace.doUpdate(JzRepoWorkspace.java:1046)
at
com.ibm.team.connector.scm.client.JzRepoWorkspace.doUpdate(JzRepoWorkspace.java:976)
at
com.ibm.rational.wvcm.ri.srvc.SrvcResource.doUpdate(SrvcResource.java:1808)
at
com.ibm.rational.wvcm.ri.impl.WvcmServiceConverter.doUpdate(WvcmServiceConverter.java:1187)
at
com.ibm.rational.wvcm.ri.impl.WorkspaceImpl.doUpdate(WorkspaceImpl.java:181)
at
com.ibm.rational.wvcm.interop.InteropStream.internalSync(InteropStream.java:2569)
at
com.ibm.rational.wvcm.interop.InteropStream.doOperation(InteropStream.java:3941)
at
com.ibm.team.connector.scm.operations.ConnectorOps.synchronizeStream(ConnectorOps.java:209)
at
com.ibm.team.connector.scm.cli.AbstractCcConnectorSubCommand.synchronizeStream(AbstractCcConnectorSubCommand.java:184)
at
com.ibm.team.connector.scm.cli.SyncSubcommand.synchronizeStream(SyncSubcommand.java:112)
at
com.ibm.team.connector.scm.cli.SyncSubcommand.run(SyncSubcommand.java:74)
at
com.ibm.team.filesystem.cli.core.internal.SubcommandLauncher.run(Unknown
Source)
at
com.ibm.team.filesystem.cli.core.internal.SubcommandLauncher.doStart(Unknown
Source)
at
com.ibm.team.filesystem.cli.core.internal.SubcommandLauncher.run(Unknown
Source)
at
com.ibm.team.filesystem.cli.core.internal.Application.start(Unknown
Source)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(Unknown
Source)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(Unknown
Source)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(Unknown
Source)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(Unknown
Source)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(Unknown
Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at org.eclipse.equinox.launcher.Main.invokeFramework(Unknown
Source)
at org.eclipse.equinox.launcher.Main.basicRun(Unknown Source)
at org.eclipse.equinox.launcher.Main.run(Unknown Source)
Caused by:
com.ibm.team.repository.common.InternalRepositoryException: Deferred
operation failed
at
com.ibm.team.repository.service.internal.RdbRepositoryDataMediator.preCommitNotify(RdbRepositoryDataMediator.java:1561)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$WriteExclusiveTransaction.completeTransaction(RepositoryDatabase.java:666)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$Transaction.finishTransaction(RepositoryDatabase.java:500)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$1.run(RepositoryDatabase.java:293)
at
com.ibm.team.repository.service.internal.rdb.ConnectionPoolService.withCurrentConnection(ConnectionPoolService.java:331)
at sun.reflect.GeneratedMethodAccessor98.invoke(null)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy22.withCurrentConnection(null)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runTransaction(RepositoryDatabase.java:288)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runInTransaction(RepositoryDatabase.java:244)
at
com.ibm.team.repository.service.internal.TransactionService.runInTransaction(TransactionService.java:79)
at sun.reflect.GeneratedMethodAccessor114.invoke(null)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy44.runInTransaction(null)
at
com.ibm.team.scm.service.internal.AbstractScmService.runInTransaction(AbstractScmService.java:310)
at
com.ibm.team.scm.service.internal.AbstractScmService.runInTransaction(AbstractScmService.java:389)
at
com.ibm.team.scm.service.internal.ScmService.updateComponents(ScmService.java:3562)
at sun.reflect.GeneratedMethodAccessor565.invoke(null)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy274.updateComponents(null)
at sun.reflect.GeneratedMethodAccessor565.invoke(null)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at
com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleMethod(AbstractTeamServerServlet.java:1170)
at
com.ibm.team.repository.servlet.AbstractTeamServerServlet.executeMethod(AbstractTeamServerServlet.java:926)
at
com.ibm.team.repository.servlet.AbstractTeamServerServlet.doPost(AbstractTeamServerServlet.java:728)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at
com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest2(AbstractTeamServerServlet.java:1773)
at
com.ibm.team.repository.servlet.AbstractTeamServerServlet.handleRequest(AbstractTeamServerServlet.java:1642)
at
com.ibm.team.repository.servlet.AbstractTeamServerServlet.service(AbstractTeamServerServlet.java:1555)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at
org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
at
org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)
at
org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:75)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at
org.eclipse.equinox.servletbridge.BridgeServlet.service(BridgeServlet.java:121)
at
com.ibm.team.repository.server.servletbridge.JazzServlet.service(JazzServlet.java:54)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:525)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:873)
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:811)
at
com.ibm.team.repository.common.internal.marshal.util.MarshallerUtil.decodeExceptions(MarshallerUtil.java:326)
at
com.ibm.team.repository.common.internal.marshal.util.MarshallerUtil.decodeExceptions(MarshallerUtil.java:296)
at
com.ibm.team.repository.common.internal.marshal.util.MarshallerUtil.decodeFault(MarshallerUtil.java:261)
at
com.ibm.team.repository.transport.client.RemoteTeamService.constructExceptionFromFault(RemoteTeamService.java:613)
at
com.ibm.team.repository.transport.client.RemoteTeamService.executeMethod(RemoteTeamService.java:483)
at
com.ibm.team.repository.transport.client.RemoteTeamService.invoke(RemoteTeamService.java:201)
at
com.ibm.team.repository.transport.client.ServiceInvocationHandler.invoke(ServiceInvocationHandler.java:43)
at $Proxy17.updateComponents(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at
com.ibm.team.repository.client.internal.ServiceInterfaceProxy.invokeServiceCall(ServiceInterfaceProxy.java:163)
at
com.ibm.team.repository.client.internal.ServiceInterfaceProxy.invoke(ServiceInterfaceProxy.java:87)
at $Proxy17.updateComponents(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at
com.ibm.team.scm.client.internal.ScmServiceInterfaceProxy$2.run(ScmServiceInterfaceProxy.java:164)
at
com.ibm.team.repository.client.internal.TeamRepository$3.run(TeamRepository.java:1211)
at
com.ibm.team.repository.common.transport.CancelableCaller.call(CancelableCaller.java:79)
at
com.ibm.team.repository.client.internal.TeamRepository.callCancelableService(TeamRepository.java:1204)
at
com.ibm.team.scm.client.internal.ScmClientLibraryContext.callCancelableService(ScmClientLibraryContext.java:70)
at
com.ibm.team.scm.client.internal.ScmServiceInterfaceProxy.invokeCancelableService(ScmServiceInterfaceProxy.java:161)
at
com.ibm.team.scm.client.internal.ScmServiceInterfaceProxy.invoke(ScmServiceInterfaceProxy.java:91)
at $Proxy17.updateComponents(Unknown Source)
at
com.ibm.team.scm.client.internal.FlowNodeConnection.replaceComponents(FlowNodeConnection.java:686)
at
com.ibm.team.scm.client.internal.FlowNodeConnection.replaceComponent(FlowNodeConnection.java:569)
at
com.ibm.team.connector.scm.client.JzRepoWorkspace.doUpdate(JzRepoWorkspace.java:1103)
... 27 more
Caused by: com.ibm.team.repository.common.TeamRepositoryException:
deferred save failed
at
com.ibm.team.repository.service.internal.RdbRepositoryDataMediator.saveItemInTxn(RdbRepositoryDataMediator.java:490)
at
com.ibm.team.repository.service.internal.RepositoryItemService.doSaveItemInTxn(RepositoryItemService.java:1191)
at
com.ibm.team.repository.service.internal.RepositoryItemService.access$11(RepositoryItemService.java:1184)
at
com.ibm.team.repository.service.internal.RepositoryItemService$14.run(RepositoryItemService.java:1165)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runTransaction(RepositoryDatabase.java:320)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase.runInTransaction(RepositoryDatabase.java:244)
at
com.ibm.team.repository.service.internal.TransactionService.runInTransaction(TransactionService.java:79)
at sun.reflect.GeneratedMethodAccessor114.invoke(null)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy44.runInTransaction(null)
at
com.ibm.team.repository.service.internal.RepositoryItemService.saveItemInTxn(RepositoryItemService.java:1163)
at sun.reflect.GeneratedMethodAccessor173.invoke(null)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.invoke(ExportProxyServiceRecord.java:370)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord.access$0(ExportProxyServiceRecord.java:356)
at
org.eclipse.soda.sat.core.internal.record.ExportProxyServiceRecord$ExportedServiceInvocationHandler.invoke(ExportProxyServiceRecord.java:56)
at $Proxy74.saveItemInTxn(null)
at
com.ibm.team.repository.service.internal.ComponentRepositoryItemService.saveItemInTxn(ComponentRepositoryItemService.java:557)
at
com.ibm.team.scm.service.internal.CacheItemServiceWrapper.saveItemInTxn(CacheItemServiceWrapper.java:233)
at
com.ibm.team.repository.service.internal.RepositoryTransactionContext.save(RepositoryTransactionContext.java:703)
at
com.ibm.team.scm.service.internal.ScmTransactionContext.save(ScmTransactionContext.java:1080)
at
com.ibm.team.scm.service.internal.AbstractScmService$2.run(AbstractScmService.java:412)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$Transaction.run(RepositoryDatabase.java:466)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$1.run(RepositoryDatabase.java:292)
... 119 more
Caused by:
com.ibm.team.repository.common.InternalRepositoryException:
CRJAZ0447I SQL statement execution failed.Vendor specific exception
SQL: INSERT INTO RTC.SCM_HISTORIC_BASIS (STATE_ID, ITEM_ID,
CONTEXT_ID, MODIFIED, MODIFIED_BY_ITEM_ID, BASIS_ITEM_ID, INDEX_COL,
WORKSPACE_ITEM_ID, COMPONENT_ITEM_ID, LATEST_REMOVED, REPLACE) VALUES
(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
SQL Exception #1
SQL Message: ORA-01654: unable to extend index
RTC.SCM_HISTORIC_BASIS_WS_CRI by 8 in tablespace RTC_DATA

SQL State: 72000
Error Code: 1654
Class:
com.ibm.team.repository.service.internal.dataaccess.write2.Row
Item Handle:
com.ibm.team.scm.common.internal.impl.HistoricBasisImpl@3dcb3dcb
(stateId: , itemId: , origin:<unset>, immutable: true)
(contextId: , modified: 2010-12-09
01:39:10.039, workingCopy: false) (predecessor:<unset>)
(deliveryDate: 2010-12-09 01:39:09.781, index: 47, latestRemoved:
false, replace: true)
at
com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.createIRE(BatchingRowReceiver.java:417)
at
com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.createIRE(BatchingRowReceiver.java:426)
at
com.ibm.team.repository.service.internal.dataaccess.write2.Row.handleExecuteException(Row.java:96)
at
com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.simulatedBatch(BatchingRowReceiver.java:396)
at
com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.executeBatch(BatchingRowReceiver.java:269)
at
com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.execute(BatchingRowReceiver.java:196)
at
com.ibm.team.repository.service.internal.dataaccess.IRowReceiver$IRowReceiverLocator.executePendingChanges(IRowReceiver.java:140)
at
com.ibm.team.repository.service.internal.dataaccess.IRowReceiver$IRowReceiverLocator.executePendingChanges(IRowReceiver.java:118)
at
com.ibm.team.repository.service.internal.RdbRepositoryDataMediator.preCommitNotify(RdbRepositoryDataMediator.java:1534)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$WriteExclusiveTransaction.completeTransaction(RepositoryDatabase.java:666)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$Transaction.finishTransaction(RepositoryDatabase.java:500)
at
com.ibm.team.repository.service.internal.rdb.RepositoryDatabase$1.run(RepositoryDatabase.java:293)
... 119 more
Caused by: java.sql.SQLException: ORA-01654: unable to extend index
RTC.SCM_HISTORIC_BASIS_WS_CRI by 8 in tablespace RTC_DATA

at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at
oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:205)
at
oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1008)
at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)
at
oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3449)
at
oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3530)
at
oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1062)
at
com.ibm.team.repository.service.internal.db.jdbcwrappers.stat.PreparedStatementStatWrapper.executeUpdate(PreparedStatementStatWrapper.java:62)
at
com.ibm.team.repository.service.internal.db.jdbcwrappers.errlog.PreparedStatementErrLogWrapper.executeUpdate(PreparedStatementErrLogWrapper.java:86)
at
com.ibm.team.repository.service.internal.db.jdbcwrappers.leak.PreparedStatementLeakWrapper.executeUpdate(PreparedStatementLeakWrapper.java:58)
at
com.ibm.team.repository.service.internal.dataaccess.BatchingRowReceiver.simulatedBatch(BatchingRowReceiver.java:388)
... 127 more


permanent link
Roderick Thomas (661179) | answered Dec 10 '10, 11:22 a.m.

I'm not a database person, but this might indicate either a disk space
problem, or a permissions problem.

Cheers,
Geoff



And the root cause was attributed to a DB space allocation issue. Thanks.

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.