com.ibm.team.build.common.TeamBuildException: CRRTC3505E
In our project we have 4 build engines (windows 7, vista, XP) running 3 different PDE builds.
Most of the time everything works like a charm. However, we occasionally run into the following error: com.ibm.team.build.common.TeamBuildException: CRRTC3505E: Unable to delete fetch destination This sometimes doesn't occur for several builds (maybe a day or so) but then for no apparent reason a build will suddenly fail with aforementioned error. I have logged on to the corresponding build engine and had a look at the fetch destination directory. Sometimes only a set of empty folders is left, another time 2 or 3 non-empty folders may still exist (the build engine has happily deleted the other 300 MB w/o complaining). I can delete whatever has been left over by the build engine and then re-request the same build and that will run w/o further errors. Thus, it doesn't seem like a rogue process still has an open file handle and is blocking the deletion. Any ideas what might be going on here? |
15 answers
On 2011/01/13 13:23, ubreu wrote:
I can delete whatever has been left over by the build My best guess is that a rogue process still has an open file handle and is blocking the deletion, but that the process goes away or at least releases the handle before you log in and look around. Unfortunately I don't have any good tips for figuring out whether or not that is true and, if so, what the rogue process is. -- David Olsen IBM Rational |
That would be my best guess too, but something still smells fishy.
Today I achieved this: A first build (the first of the day) on the build engine failed: 2011-01-19 08:59:42 Abrufziel "C:\build\belimo-core" wird vor dem Abruf gelscht... com.ibm.team.build.common.TeamBuildException: CRRTC3505E: Das Abrufziel "C:\build\belimo-core" kann nicht gelscht werden. at com.ibm.team.build.internal.engine.JazzScmPreBuildParticipant.preBuild(JazzScmPreBuildParticipant.java:218) at com.ibm.team.build.internal.engine.BuildLoop.invokePreBuildParticipants(BuildLoop.java:806) at com.ibm.team.build.internal.engine.BuildLoop$2.run(BuildLoop.java:622) at java.lang.Thread.run(Thread.java:662) I then re-requested the same build on the same build engine a couple of minutes later w/o logging on to the build engine and the build went through smoothly. 2011-01-19 09:04:22 Abrufziel "C:\build\belimo-core" wird vor dem Abruf gelscht... 2011-01-19 09:04:22 Dateien fr Abrufziel "C:\build\belimo-core" werden abgerufen... |
However, we occasionally run into the following error: This is probably not the case, but do you have two build engines running on the same machine and using the same fetch directory? If so, build A could be building from the directory, while build B attempts to delete the directory. |
com.ibm.team.build.common.TeamBuildException: CRRTC3505E: Unable to delete fetch destination I looked around real quick and found this forum topic with the same issue: http://jazz.net/forums/viewtopic.php?t=3841 Can you see if what they say there is the same for you? In the mean time I will attempt to replicate this issue... |
I've been able to reproduce the error twice since. I haven't seen a pattern as to which directories can't be deleted.
Once a single directory was left over (e.g. plugins/foo/bar/quux.jar) and everything else was deleted. The second time only a set of empty directories was left over. Some with leading dots, others with "normal" names and one @dot directory (we are using PDE build). |
com.ibm.team.build.common.TeamBuildException: CRRTC3505E: Unable to delete fetch destination I looked around real quick and found this forum topic with the same issue: http://jazz.net/forums/viewtopic.php?t=3841 Can you see if what they say there is the same for you? In the mean time I will attempt to replicate this issue... Unfortunately I have yet to see this behavior in my testing, nor do I know why this is happening. I can confirm however that in RTC 3.0, when checking the "Delete directory before loading" checkbox in the "Jazz Source Control" tab, this does delete all files/folders in the load directory (even if they have a "." in front) Since most users use their own build scripts to do their building, I can only assume that some process is still running and holding a lock on the file (or not releasing it properly). I wonder if it possible customers are using other programs/processes/ant tasks to do the building (other than what is including in the build toolkit), and perhaps maybe they are not releasing a lock on the files? It's hard to tell with so many possibilities. |
Is there any chance that you are using symbolic links for any of your fetched files? I think there might be issues trying to delete symbolic links.
|
Is there any chance that you are using symbolic links for any of your fetched files? I think there might be issues trying to delete symbolic links. Sorry for my late response. We weren't using any symbolic links. The problem however, hasn't arisen any more for the past month. Maybe we were having issues with network latency (the build engines are VMs running on an ESX.) |
Hello!
I have the same problem with TeamBuildException: CRRTC3505E. I used MS process monitor from the sysinternal utility pack to reproduce the processes during an attemp to delete fetch destination. It appears to be a sharing violation as a result of a failing attemp to delete the .jazz folder and its content. Here is some of the log I was able to filter from the process monitor. Lots of processes are left out to avoid clutter. Maybe someone has an idea what to do.
3:45:49.5873030 PM jbe.exe 3280 SetDispositionInformationFile C:\rtcbuild-sandbox\.jazz5 NOT EMPTY Delete: True 3:45:49.5942497 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.components SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.5956236 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.components SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.5964013 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.descriptors.dat SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.5976761 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.descriptors.dat SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.5984555 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.inversedescriptors.dat SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.5997258 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.inversedescriptors.dat SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.6005041 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.inverseiteminfo.dat SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.6024543 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.inverseiteminfo.dat SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.6032086 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.jazzlock SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.6044571 PM jbe.exe 3280 CreateFile C:\rtcbuild-sandbox\.jazz5\.jazzlock SHARING VIOLATION Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a 3:45:49.6052692 PM jbe.exe 3280 SetDispositionInformationFile C:\rtcbuild-sandbox\.jazz5 NOT EMPTY Delete: True 3:45:49.6063059 PM jbe.exe 3280 SetDispositionInformationFile C:\rtcbuild-sandbox NOT EMPTY Delete: True |
Hi, Can anyone who sees this issue provide some additional information in the hopes that we can identify some pattern and narrow down the issue. Some questions could include: 1) What version of JBE are you using? 2) How is the build definition set-up; how are you starting the build? 3) Can you confirm that there are not two engines running on the same machine (using the same load directory) 4) Does this happen all the time for you? If not, how often? (can you think of anything different about the setup/build when this does happen?) 5) Is there any time where a next build would be able to delete the directory properly, or once it happens, do all future builds fail to delete the directory? 6) If a build fails to delete the load directory (with JBE and the build loop still running, but not building anything) are you able to manually delete the directory? This may indicate that something in JBE has locks on some files in the directory. If that doesn't work, can you kill JBE and then try to manually delete the file? 7) Are there any build scripts , or forked off programs/executable that do work on this load directory? 8) What files are left over when the build fails. Is it always the .jazz folder? or is it always some other folders / files , or does it appear completely random? 9) What is the operating system, and where/how is the load directory specified ("relative" , or are you using an "absolute" path?). Please try to add additional information if I missed a question that you feel might help. |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.