It's all about the answers!

Ask a question

How to prevent RTC from clearing jenkins workspace prior to a build


Brett Waldo (1101421) | asked Feb 24 '21, 11:41 p.m.
edited Feb 25 '21, 8:11 a.m.

 Trying to switch to using more declarative pipeline jobs in jenkins. Hit an issue where it seems for every build, the workspace is deleted upon job starting (during the fetch stage).


This appears to work just fine when the jenkins job is configured with a pipeline script that is copy/pasted into the jenkins UI.  However, when I try to load the pipeline script from SCM, using the load from Repository workspace option, the workspace is deleted completely prior to the fetch.

Note, checking the lightweight checkout has no effect.

There are two fetch stages, the first fetches on master from the repository workspace the jenkins file, and the jenkinsfile has a build definition that forces a fetch on the jenkins node.  After the first fetch the files on the node are still there. it appears during the second fetch stage the workspace is first deleted.  

The option to "Delete Before Load" is NOT checked in the build definition.  As stated above, the exact same jenkins file works when directly copied into the jenkins UI. Only when loading from SCM do I hit this case. I've only tried loading from workspace, not loading from build definition or stream yet.

Of course, the jenkins file also does not have any steps to run cleanWs or anything as well.

It appears the directory is actually deleted, not just some sort of a deletion of generated files, as if I have a shell open in the workspace directory, I get the stale handle issue.

Jenkins 2.263.4
Team Concert Jenkins Plugin 2.2.0
RTC v6.0.6.1

and


Comments
Brett Waldo commented Feb 24 '21, 11:45 p.m.

weird deja-vu, but i asked almost this exact question a few years back.  (i don't remember and it didn't show up on google).  But that time, the issue was my configuration, I don't believe that is the case here because I just switch back and forth between the configurations for it to work/not work.



Brett Waldo commented Feb 24 '21, 11:56 p.m.

 Some more details, I found loading the pipeline script from a Stream versus from a workspace avoids this bug. I have yet to determine if there is any drawback to using the stream (e.g. not sure how polling for changes work etc)


Brett Waldo commented Feb 25 '21, 2:19 a.m.

For those wondering, using the stream I don't think is a decent option. The workspace has scoped components but the stream doesn't so you end up with extra builds for unrelated components and dirty change logs. 


Ralph Schoon commented Feb 25 '21, 2:43 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

If you think there is an issue, please open a support ticket or open a defect here: https://jazz.net/jazz/web/projects/Rational%20Team%20Concert#action=com.ibm.team.workitem.viewWelcome


Brett Waldo commented Feb 25 '21, 7:49 a.m.

 @rschoon , sorry, I wasn't clear in the description. Version 2.2 is the version of the plugin, which is most recent version as of 2/24/2021.  RTC version is 6.0.6.1 


I am aware of many of those articles, but difficult to filter through them as there are a lot with old incorrect information. The one you linked appears fairly current, but does not seem to even touch on using jenkins declarative pipeline syntax, which from what I can gather is the "new" recommended approach and seems to be most likely the problem with this bug.

I tried to file a bug with the plugin, but couldn't figure out how actually.


Ralph Schoon commented Feb 26 '21, 2:07 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER

I asked someone more competent than mI to have a look here. No promises. 


1
Lakshmi Narasimhan T V commented Feb 26 '21, 10:13 a.m. | edited Feb 26 '21, 10:26 a.m.
JAZZ DEVELOPER
Hi Brett,

In both scripted or declarative pipeline, the syntax for checkout doesn't change and steps are usually unaffected. However, some known issues with SCM in general have been documented here.

Have you added

agent {
options {skipDefaultCheckout(true)}
}   
            
to prevent a checkout from repository workspace in the agent? I think this additional checkout is causing some files to be unloaded which once again are loaded from the build definition.



Brett Waldo commented Feb 26 '21, 11:23 a.m. | edited Feb 26 '21, 11:23 a.m.

Hello Lakshmi, 

I had read that page before couldn't really grasp what it was trying to say. Rereading it now I see the big issue is for large jobs with continuous commits, there is a race condition that a commit happened between loading the pipeline and loading the workspace. Interesting, hadn't thought of that but probably not a use case on my smaller program. 

But I did try it anyway and it does work!  I don't really see why this was only a bug when loading from workspace and not from a stream or a build definition.  Seems like the behavior should be the same regardless.

That said, it works so I really appreciate your help. I had gone through the trouble of creating 10s of build definitions which are such a bear and using the workspace is so much easier to maintain.

Thanks!

showing 5 of 8 show 3 more comments

Accepted answer


permanent link
Brett Waldo (1101421) | answered Feb 26 '21, 11:26 a.m.

 Lakshmi's suggestion to use 


agent {
    
options {skipDefaultCheckout(true)}
} 
    

as discussed here is a good enough workaround for me.  


Ralph Schoon selected this answer as the correct answer

Comments
Brett Waldo commented Mar 12 '21, 3:22 p.m.

Looks like however in doing this RTC no longer associates any builds to the workitems, e.g. "Included In Builds" does not have any links created anymore when using this approach. I think the only viable solution is just to create 100s of individual build definitions, which is annoying from a maintenance point of view but should be doable. 


Lakshmi Narasimhan T V commented Mar 13 '21, 12:56 a.m. | edited Mar 13 '21, 1:07 a.m.
JAZZ DEVELOPER
Hi
There is an option in the plugin for "Repository Workspace" job configuration to add Jenkins build links to work items.  Please see https://plugins.jenkins.io/teamconcert/#configuring-for-repository-workspace. Note that "Included in builds" link gets created only for Build definition based builds. For other job configurations, the Jenkins build URL is added as a "Related Artifact" link.

Also, if you are using the same repository workspace in both "Pipeline script from SCM" and the build definition's SCM configuration in the JenkinsFile, accept will happen in the first checkout. The second checkout will already have all the changes and hence you will not see "Included in builds" links for the build result in the work item.  You can use different repository workspaces in the first and second checkout.

Lakshmi Narasimhan T V commented Mar 13 '21, 1:03 a.m. | edited Mar 13 '21, 1:06 a.m.
JAZZ DEVELOPER

If you still need the traceability from the build result to work items, you can manually add "included in build" links using the workItemPublisher ant task.

One other answer



permanent link
Ralph Schoon (60.9k33643) | answered Feb 25 '21, 2:38 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
edited Feb 25 '21, 2:39 a.m.

 RTC 2.2 is no longer supported. The Jenkins plugin for that is likely no longer supported either. Or are you referring to the RTC Jenkins Plugin?


For supported and current versions of RTC, are you aware of: https://jazz.net/library/article/92827 and articles in general?


Comments
Brett Waldo commented Feb 26 '21, 11:27 a.m.

I meant the plugin version was 2.2 not the RTC version. My mistake 

Your answer


Register or to post your answer.