Loading RTC repository workspace is failing for files randomly everytime
Dear Experts,
We are using CLM RTC 5.0.2. When we load the repository workspace, it fails everytime with the error message,
Status ERROR: com.ibm.team.filesystem.client code=0 There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync. null children=[Status ERROR: com.ibm.team.filesystem.client code=0 Failed to download
The load is successful on subsequent reloads and it fails for random files on each load that we perform. The major issue is, it is breaking the build process and the builds are failing where we have no option to reload and build.
Please find the builds logs pasted below.
04:00:35 There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync.
04:00:35 Failed to download Deleted <load file path>
04:00:35 Failed to download Deleted <load file path>
04:00:35 Failed to download Deleted <load file path>
04:00:35
04:00:35 com.ibm.team.build.internal.scm.SourceControlUtility$2: Status ERROR: com.ibm.team.filesystem.client code=0 There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync. null children=[Status ERROR: com.ibm.team.filesystem.client code=0 Failed to download Deleted <load file path>
r com.ibm.team.filesystem.client.FileSystemException: Error while writing to /SFS_NGS_AUX/Deployment/Simulation/SP/SPDataAcquisition/XYCalibrationHTOblique2013-10-25-08-23-57_verify_4th.spr : Expected content hash of hIQ6JHIjkn_h3ptouL_1SRLu9JPeCqHfBRFtOqt8rkE but got 3E3eu0qlGlGZLaE_gW0BsTdD32ltBtRGur5i09NbmsA Status ERROR: com.ibm.team.filesystem.client code=0 Failed to downloadDeleted <load file path>l.pdf com.ibm.team.filesystem.client.FileSystemException: Error while writing to /SFS_NGS_AUX/Deployment/Install/KLA/NGSHelp/NGS_User_Manual.pdf : Expected content hash of KK9Xy7KZ31xUX2ovaIoB906t4gFdnjkERlAadMuY3oI but got yBnG65_4equOIf1OgCRzASMIdwkBrt1LurUaLnwWFpg Status ERROR: com.ibm.team.filesystem.client code=0 Failed to download /SFS_NGS_AUX/Deployment/Redistribute/x64/3rd_Party/Matlab/archives/parallel_gpu_libs_win64_1407828389.enc com.ibm.team.filesystem.client.FileSystemException: Error while writing to /SFS_NGS_AUX/Deployment/Redistribute/x64/3rd_Party/Matlab/archives/parallel_gpu_libs_win64_1407828389.enc : Expected content hash of xx8ADmWV_GyQZ0gxyX6bzKSqYLAcSZlPtoHFXMzYTo8 but got NTvY_VX7qTNtUa6w4TY35oW2fPTjKRc5414XVTWHXaU]
04:00:35 at com.ibm.team.build.internal.scm.SourceControlUtility.updateFileCopyArea(SourceControlUtility.java:671)
04:00:35 at com.ibm.team.build.internal.hjplugin.rtc.RepositoryConnection.checkout(RepositoryConnection.java:436)
04:00:35 at com.ibm.team.build.internal.hjplugin.rtc.RTCFacade.checkout(RTCFacade.java:390)
04:00:35 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
04:00:35 at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
04:00:35 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
04:00:35 at java.lang.reflect.Method.invoke(Unknown Source)
04:00:35 at com.ibm.team.build.internal.hjplugin.RTCFacadeFactory$RTCFacadeWrapper.invoke(RTCFacadeFactory.java:115)
04:00:35 at com.ibm.team.build.internal.hjplugin.RTCCheckoutTask.invoke(RTCCheckoutTask.java:166)
04:00:35 at com.ibm.team.build.internal.hjplugin.RTCCheckoutTask.invoke(RTCCheckoutTask.java:32)
04:00:35 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2688)
04:00:35 at hudson.remoting.UserRequest.perform(UserRequest.java:121)
04:00:35 at hudson.remoting.UserRequest.perform(UserRequest.java:49)
04:00:35 at hudson.remoting.Request$2.run(Request.java:324)
04:00:35 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
04:00:35 at java.util.concurrent.FutureTask.run(Unknown Source)
04:00:35 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
04:00:35 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
04:00:35 at hudson.remoting.Engine$1$1.run(Engine.java:63)
04:00:35 at java.lang.Thread.run(Unknown Source)
04:00:35 Contains : 0 Failed to download Deleted <load file path>
04:00:35 com.ibm.team.filesystem.client.FileSystemException: Error while writing to Deleted <load file path>
: Expected content hash of hIQ6JHIjkn_h3ptouL_1SRLu9JPeCqHfBRFtOqt8rkE but got 3E3eu0qlGlGZLaE_gW0BsTdD32ltBtRGur5i09NbmsA
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$6.run(MergeLoadMutator.java:1809)
04:00:35 at com.ibm.team.filesystem.client.internal.SharingManager.doSilentChange(SharingManager.java:766)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator.storeFileContents(MergeLoadMutator.java:1765)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator.access$0(MergeLoadMutator.java:1762)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$DownloadHandler.downloadStreamAcquired(MergeLoadMutator.java:162)
04:00:35 at com.ibm.team.filesystem.client.FileDownloadHandler.downloadStreamAcquired(FileDownloadHandler.java:86)
04:00:35 at com.ibm.team.filesystem.client.FileDownloadHandler.downloadStreamAcquired(FileDownloadHandler.java:1)
04:00:35 at com.ibm.team.scm.client.content.BasicAsyncVersionedContentManagerSession.internalDoRetrieveContent(BasicAsyncVersionedContentManagerSession.java:117)
04:00:35 at com.ibm.team.scm.client.content.BasicAsyncVersionedContentManagerSession$1.run(BasicAsyncVersionedContentManagerSession.java:190)
04:00:35 at com.ibm.team.repository.client.util.ThreadPool$PoolJob.run(ThreadPool.java:134)
04:00:35 at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
04:00:35 Caused by: java.io.IOException: Expected content hash of hIQ6JHIjkn_h3ptouL_1SRLu9JPeCqHfBRFtOqt8rkE but got 3E3eu0qlGlGZLaE_gW0BsTdD32ltBtRGur5i09NbmsA
04:00:35 at com.ibm.team.scm.client.content.BasicVersionedContentManager$9.checkHash(BasicVersionedContentManager.java:1589)
04:00:35 at com.ibm.team.scm.client.content.BasicVersionedContentManager$9.read(BasicVersionedContentManager.java:1600)
04:00:35 at com.ibm.team.scm.client.internal.content.MonitoredInputStream.read(MonitoredInputStream.java:80)
04:00:35 at com.ibm.team.scm.client.content.ProgressTrackingInputStream.read(ProgressTrackingInputStream.java:100)
04:00:35 at com.ibm.team.internal.repository.rcp.streams.DigestComputingInputStream.read(DigestComputingInputStream.java:52)
04:00:35 at java.io.InputStream.read(Unknown Source)
04:00:35 at com.ibm.team.repository.client.internal.ContentManager.read(ContentManager.java:1028)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$6.run(MergeLoadMutator.java:1787)
04:00:35 ... 10 more
04:00:35 Contains : 1 Failed to download Deleted <load file path>
.pdf
04:00:35 com.ibm.team.filesystem.client.FileSystemException: Error while writing to Deleted <load file path>
_Manual.pdf : Expected content hash of KK9Xy7KZ31xUX2ovaIoB906t4gFdnjkERlAadMuY3oI but got yBnG65_4equOIf1OgCRzASMIdwkBrt1LurUaLnwWFpg
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$6.run(MergeLoadMutator.java:1809)
04:00:35 at com.ibm.team.filesystem.client.internal.SharingManager.doSilentChange(SharingManager.java:766)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator.storeFileContents(MergeLoadMutator.java:1765)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator.access$0(MergeLoadMutator.java:1762)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$DownloadHandler.downloadStreamAcquired(MergeLoadMutator.java:162)
04:00:35 at com.ibm.team.filesystem.client.FileDownloadHandler.downloadStreamAcquired(FileDownloadHandler.java:86)
04:00:35 at com.ibm.team.filesystem.client.FileDownloadHandler.downloadStreamAcquired(FileDownloadHandler.java:1)
04:00:35 at com.ibm.team.scm.client.content.BasicAsyncVersionedContentManagerSession.internalDoRetrieveContent(BasicAsyncVersionedContentManagerSession.java:117)
04:00:35 at com.ibm.team.scm.client.content.BasicAsyncVersionedContentManagerSession$1.run(BasicAsyncVersionedContentManagerSession.java:190)
04:00:35 at com.ibm.team.repository.client.util.ThreadPool$PoolJob.run(ThreadPool.java:134)
04:00:35 at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
04:00:35 Caused by: java.io.IOException: Expected content hash of KK9Xy7KZ31xUX2ovaIoB906t4gFdnjkERlAadMuY3oI but got yBnG65_4equOIf1OgCRzASMIdwkBrt1LurUaLnwWFpg
04:00:35 at com.ibm.team.scm.client.content.BasicVersionedContentManager$9.checkHash(BasicVersionedContentManager.java:1589)
04:00:35 at com.ibm.team.scm.client.content.BasicVersionedContentManager$9.read(BasicVersionedContentManager.java:1600)
04:00:35 at com.ibm.team.scm.client.internal.content.MonitoredInputStream.read(MonitoredInputStream.java:80)
04:00:35 at com.ibm.team.scm.client.content.ProgressTrackingInputStream.read(ProgressTrackingInputStream.java:100)
04:00:35 at com.ibm.team.internal.repository.rcp.streams.DigestComputingInputStream.read(DigestComputingInputStream.java:52)
04:00:35 at java.io.InputStream.read(Unknown Source)
04:00:35 at com.ibm.team.repository.client.internal.ContentManager.read(ContentManager.java:1028)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$6.run(MergeLoadMutator.java:1787)
04:00:35 ... 10 more
04:00:35 Contains : 2 Failed to download Deleted <load file path>
89.enc
04:00:35 com.ibm.team.filesystem.client.FileSystemException: Error while writing to Deleted <load file path>
4_1407828389.enc : Expected content hash of xx8ADmWV_GyQZ0gxyX6bzKSqYLAcSZlPtoHFXMzYTo8 but got NTvY_VX7qTNtUa6w4TY35oW2fPTjKRc5414XVTWHXaU
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$6.run(MergeLoadMutator.java:1809)
04:00:35 at com.ibm.team.filesystem.client.internal.SharingManager.doSilentChange(SharingManager.java:766)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator.storeFileContents(MergeLoadMutator.java:1765)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator.access$0(MergeLoadMutator.java:1762)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$DownloadHandler.downloadStreamAcquired(MergeLoadMutator.java:162)
04:00:35 at com.ibm.team.filesystem.client.FileDownloadHandler.downloadStreamAcquired(FileDownloadHandler.java:86)
04:00:35 at com.ibm.team.filesystem.client.FileDownloadHandler.downloadStreamAcquired(FileDownloadHandler.java:1)
04:00:35 at com.ibm.team.scm.client.content.BasicAsyncVersionedContentManagerSession.internalDoRetrieveContent(BasicAsyncVersionedContentManagerSession.java:117)
04:00:35 at com.ibm.team.scm.client.content.BasicAsyncVersionedContentManagerSession$1.run(BasicAsyncVersionedContentManagerSession.java:190)
04:00:35 at com.ibm.team.repository.client.util.ThreadPool$PoolJob.run(ThreadPool.java:134)
04:00:35 at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
04:00:35 Caused by: java.io.IOException: Expected content hash of xx8ADmWV_GyQZ0gxyX6bzKSqYLAcSZlPtoHFXMzYTo8 but got NTvY_VX7qTNtUa6w4TY35oW2fPTjKRc5414XVTWHXaU
04:00:35 at com.ibm.team.scm.client.content.BasicVersionedContentManager$9.checkHash(BasicVersionedContentManager.java:1589)
04:00:35 at com.ibm.team.scm.client.content.BasicVersionedContentManager$9.read(BasicVersionedContentManager.java:1600)
04:00:35 at com.ibm.team.scm.client.internal.content.MonitoredInputStream.read(MonitoredInputStream.java:80)
04:00:35 at com.ibm.team.scm.client.content.ProgressTrackingInputStream.read(ProgressTrackingInputStream.java:100)
04:00:35 at com.ibm.team.internal.repository.rcp.streams.DigestComputingInputStream.read(DigestComputingInputStream.java:52)
04:00:35 at java.io.InputStream.read(Unknown Source)
04:00:35 at com.ibm.team.repository.client.internal.ContentManager.read(ContentManager.java:1028)
04:00:35 at com.ibm.team.filesystem.client.internal.load.MergeLoadMutator$6.run(MergeLoadMutator.java:1787)
04:00:35 ... 10 more
04:00:35 FATAL: RTC : checkout failure: RTC : checkout failure: There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync.
04:00:35 There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync.
Deleted <load file path>
04:00:35
04:00:35 hudson.AbortException: RTC : checkout failure: There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync.
04:00:35 There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync.
04:00:35 Failed to download Deleted <load file path>
04:00:35 Failed to download /Deleted <load file path>
04:00:35 Failed to download /Deleted <load file path>
04:00:35
04:00:35 at com.ibm.team.build.internal.hjplugin.RTCCheckoutTask.invoke(RTCCheckoutTask.java:199)
04:00:35 at com.ibm.team.build.internal.hjplugin.RTCCheckoutTask.invoke(RTCCheckoutTask.java:32)
04:00:35 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2688)
04:00:35 at hudson.remoting.UserRequest.perform(UserRequest.java:121)
04:00:35 at hudson.remoting.UserRequest.perform(UserRequest.java:49)
04:00:35 at hudson.remoting.Request$2.run(Request.java:324)
04:00:35 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
04:00:35 at java.util.concurrent.FutureTask.run(Unknown Source)
04:00:35 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
04:00:35 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
04:00:35 at hudson.remoting.Engine$1$1.run(Engine.java:63)
04:00:35 at java.lang.Thread.run(Unknown Source)
04:00:35 at ......remote call to sfs-si7-7(Native Method)
04:00:35 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413)
04:00:35 at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
04:00:35 at hudson.remoting.Channel.call(Channel.java:778)
04:00:35 at hudson.FilePath.act(FilePath.java:978)
04:00:35 at hudson.FilePath.act(FilePath.java:967)
04:00:35 at com.ibm.team.build.internal.hjplugin.RTCScm.checkout(RTCScm.java:1082)
04:00:35 at hudson.model.AbstractProject.checkout(AbstractProject.java:1274)
04:00:35 at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:609)
04:00:35 at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
04:00:35 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:531)
04:00:35 at hudson.model.Run.execute(Run.java:1738)
04:00:35 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
04:00:35 at hudson.model.ResourceController.execute(ResourceController.java:98)
04:00:35 at hudson.model.Executor.run(Executor.java:381)
04:00:35 ERROR: RTC : checkout failure: RTC : checkout failure: There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync.
04:00:35 There were errors during download. It is recommended that components be reloaded. The reload will be incremental, loading only the missing items. You can further reduce the reload time by reloading just the projects out of sync.
Deleted <load file path>
04:00:35 [BFA] Scanning build for known causes...
04:00:35 [BFA] Found failure cause(s):
04:00:35 [BFA] (Network) Java Error from category Network
04:00:35 [BFA] (Network) Jazz Error from category Network
04:00:35 [BFA] Done. 0s
04:00:39 Started calculate disk usage of build
04:00:39 Finished Calculation of disk usage of build in 0 seconds
04:00:39 Started calculate disk usage of workspace
04:00:39 Finished Calculation of disk usage of workspace in 0 seconds
04:00:39 Email was triggered for: Always
04:00:39 Email was triggered for: Failure - Any
04:00:39 Sending email for trigger: Failure - Any
04:00:39 Finished: FAILURE
Thanks
Anbarasu Arivoli
3 answers
Hello Anbarasu,
Test the connectivity between the machines.
For instance, start with a trace route (tracert if windows) to check the route executed. Compare with the route from another machine, better one that does not have this hash codes mismatch.
Then, use ping with a big package between the machines, to verify if it is failing
ping xxxxxx -l 1000
or bigger
ping xxxx -l 65500
Ping from your client machine to the CCM server
Test the connectivity between the machines.
For instance, start with a trace route (tracert if windows) to check the route executed. Compare with the route from another machine, better one that does not have this hash codes mismatch.
Then, use ping with a big package between the machines, to verify if it is failing
ping xxxxxx -l 1000
or bigger
ping xxxx -l 65500
Ping from your client machine to the CCM server
Comments
Rakesh Dama
Sep 04 '17, 2:51 p.m.Hi folks,