Assuming that we could make the properties work (through our own rules about defining & using properties for use in fields that aren't exported), how do I get the properties file to be a member of the snapshot for this build? Seems like we have a chicken-egg problem: we need the properties file to be part of the snapshot but the properties file isn't created until after the snapshot is created & associated with the build.

Instead of trying to add the properties file to the snapshot, why don't you use an ant task to publish the properties file back (or any other build customization files) to the build result. This way for any given build result, you can see the exact properties/customizations that went into that build.

Jeff,

to my knowledge they are not.

However it is possible in the ANT part of the build description allows to publish the properties of the build. This includes information such as parameters. Since it is a text file it could potentially be versioned. I haven't played with such an scenario yet.

There is a show history on the Build Definition context menu in the Eclipse client but I am not able to figure ut what to do with the histories

Ralph

Are build definitions and build engines capable of being version controlled?

For example, when RTC runs a build it creates a snapshot to enable recreating that build at some point in the future but the configuration of the build definition and build engine also participate in determining the output of the build, especially if your build is parametrized.

In my experimentation I haven't seen a way to record the configuration of a build definition or build engine to aid in future build recreates.

Hi,

not sure you can get to the properties in a commandline build. I'd suggest to investigate. I'd also suggest you create a work item for the wish to version the build definition and additional data with the build, sounds like a reasonable request.

However, you could prompt data into a text file and version that file with the lscm tool in a batch. That might be a way to go for now.
I think it is also possible to echo the environment with an ant task in a file.

Ralph

I can see the history on the build definitions & the build engines, which is to say that if I select "Show History" from the context menu the history view will be populated showing the dates of changes & who made them but I can't seem to do anything at all with the historical versions that are listed: can't view them, can't compare them, nothing.

The properties idea is interesting but I see two problems with it:

1. I don't think that the properties by themselves are enough. We're currently using a command-line build definition & I don't see any properties for the fields on the "Command Line" tab.
   
2. Assuming that we could make the properties work (through our own rules about defining & using properties for use in fields that aren't exported), how do I get the properties file to be a member of the snapshot for this build? Seems like we have a chicken-egg problem: we need the properties file to be part of the snapshot but the properties file isn't created until after the snapshot is created & associated with the build.
   

Jeff,

to my knowledge they are not.

However it is possible in the ANT part of the build description allows to publish the properties of the build. This includes information such as parameters. Since it is a text file it could potentially be versioned. I haven't played with such an scenario yet.

There is a show history on the Build Definition context menu in the Eclipse client but I am not able to figure ut what to do with the histories

Ralph

Are build definitions and build engines capable of being version controlled?

For example, when RTC runs a build it creates a snapshot to enable recreating that build at some point in the future but the configuration of the build definition and build engine also participate in determining the output of the build, especially if your build is parametrized.

In my experimentation I haven't seen a way to record the configuration of a build definition or build engine to aid in future build recreates.

We could do that. The difficulty here is that all of this property stuff is a workaround & it all requires some manual intervention.

1. we're assuming that all configurable fields in a build definition are solely going to reference properties and nothing else; that's doable obviously but it's cumbersome and there's no way to enforce it outside of a business process/control
   
2. even if you only use properties there's no information about the relative order of the properties used in any particular field so even with properties you still can't be entirely sure that you can faithfully reproduce a historical build
   
3. You're still going to have to manually restore the properties if you need to rebuild a historical build
   

Hi Jeff thanks for pointing out these subtle issues.

If you right click on a build result node in the "Builds" view you will see the following two context menu items:
1. Request Build...
2. Request Rebuild...

From my understanding, the semantics between those two actions are to either:
1. use the latest 'state' of the Build Definition, or
2. use the 'snapshot' Build Definition instance as when the build was first created.

Most Build Definitions are customized using 'Build Properties'... and so what we currently do for #2 (Request Rebuild) is copy over all build properties. However it might make sense to use the configuration properties from the snapshot build definition instance as well. Note: some 'fields' in the Build Definition are considered "Build Properties" (Ex: In the "Jazz Source Control" tab: the build work space --> team.scm.workspaceUUID or the load directory --> team.scm.fetchDestination", etc.). while other 'fields' in the Build Definition are considered "Configuration Properties" (Ex: The "Ant" tab: "build file", "build targets", "ant arguments", etc.)

So a semi-work around would be to use build properties for all fields (configuration properties), and so you never change these fields/configuration properties over time, but only change the build properties. This way when ever you "Request Rebuild..." it will use the "build properties" as defined at that point in time, and so that should ensure we have the same build configuration from a Build Definition point of view.

A lot of our build customization (other than say specific ant scripts) involves using build using build properties, and so the current behavior has worked for us thus far. However you raised a good point and so I have raised a work item against this issue here:
https://jazzdev.torolab.ibm.com:9443/jazz/web/projects/Rational%20Team%20Concert#action=com.ibm.team.workitem.viewWorkItem&id=150988

The other subtle point about doing this is you have to remember that when you request a build or a rebuild (using the new or old build definition), the "Jazz Source Control" tab is specified to 'fetch' the source from a particular repository workspace. This means it fetches the 'latest' source. However if you wanted 'older' source, you can simply create a repository workspace and load from a snapshot (since builds usually create a snapshot and so there is a snapshot associated with a given build result). Now in the "Request Rebuild..." dialog you choose 'personal build' and select the repository workspace which you just crated from an older snapshot. Keep in mind these extra steps are not ideal, however I would assume it would be rare to re-build older source often. Usually if a 'build' is worth keeping you un-check "Deletion allowed" in the Build Result and keep it around for what ever build result/contribution data you may need.

Jeff probably can't see that link. The public one is: 150988: "Request Rebuild" action should use configuration properties from the "Build Definition Instance" and not just the build properties. I've cc'ed Jeff.

Jeff, which build system do you use currently? How does it handle the SCM aspects of a rebuild? That is, if you rebuild an older build, does it use the SCM snapshot (tag, whatever) at that point in time, or the current state, or give you the choice? Is that support built into the build system, or just the way you've set up your scripts?

Our current build system can rebuild any historical version of the code, with zero manual intervention, period. My product group is in the process of evaluating RTC for a possible migration and this is one of the major sticking points that we are encountering. We require 100% confidence that we can faithfully reproduce any deliverable sent to a customer and I just can't see how we can do that with Jazz build.

Since this forum discussion later turned into an email discussion, I am posting a rough blob of the 'outcome' so that other forum readers see what is happening in regards to being able to accurately recreate historical builds.
The 3 issues related to this (and the existing "Request ReBuild..." action) are as follows:



1. "...the current state of the build definition is used..."

It is true that in a sense that we do use the 'current' state of the build definition. 'Build properties' that are entered by customers are currently re-used, however the behind the scene 'configuration properties' are not re-used.

This issue was made into the following work item:
Defect 150988: "Request Rebuild" action should use configuration properties from the "Build Definition Instance" and not just the build properties
https://jazz.net/jazz/resource/itemName/com.ibm.team.workitem.WorkItem/150988

Short-term work around: Turn all required configuration properties into build properties defined in the "Properties" tab of the build definition (as discussed above)


2. "... the current state of the build engine is used..."

A good point, however this might be hard to address. There are a couple situations where things may become difficult. A common setup for users is to have several "Supporting Build Engines" defined in the "Overview" page of a build definition. This means that when a build is requested, any of the 'n' build engines (or different physical machines) could pick up the request. It is possible for multiple 'build engines' to be run on the same physical machine, however others often create a different 'build engine' for each physical build machine. Sometimes users put unique build properties on the build engine (such as specific library\tool paths) which correspond to that physical machine (could be running different operating systems as well). And so there are some apparent difficulties of ensuring that a "Request Rebuild" will re-use the same build engine properties. Examples include:
-There is no guarantee that the physical machine or build engine actually exists in the future.
-If the same physical machine running the same build engine still exists, it's possible that hardware/software upgrades required some build property updates on that build engine (and the old build property values would just not work anymore).
These are examples where in some situations things may break down; however I am not saying that this is not something we may want to look into, since clearly if there has been no machine/build engine changes since the build was first run, then there should be no reason it cannot be run again.

This issue was made into the following work item:
Enhancement 151475: Allow "Request Rebuild..." action to possibly allow the reuse of the same build engine and build properties defined in that build engine.
https://jazz.net/jazz/resource/itemName/com.ibm.team.workitem.WorkItem/151475

Short-term work around: Define the build properties in the build definition instead of the build engine. This is arguably a better place for platform-neutral properties since they can be used on multiple build engines/machines, whereas properties defined in an engine are only used on the platform/machine which starts jbe with a given build engine.

Also see this link for some info about build properties/build engines: https://jazz.net/wiki/bin/view/Main/BuildFAQ#Can_multiple_Jazz_Build_Engine_J


3. "... build using the historical scm snapshot..."

It is true, the current work around of selecting a "personal build" to build from a special workspace is a tedious one.

Issues:
When we first make a build, the build definition says pull the source from 'this' repository workspace, and so if we were to go back in time and re-do just that we would still be pulling source from that repository workspace. This is the current behavior of "Request Rebuild". However, it gets difficult because the proposed changes would now have us say, "use the same build definition as before, but ignore the part that says 'pull the source from this repository workspace', and instead pull the source from a particular snapshot in time. The difficulty with that is in the Jazz world we first need a repository workspace with that snapshot in order to do any Jazz SCM fetching. Although theoretically everything is still doable at this point since a snapshot could be associated with the build result at question. The other difficulty is that it is very much possible that a given build result does not have any snapshot associated with it. For example, if a build definition does not have the optional "Jazz Source Control" tab selected, no snapshot will automatically be created and associated with the build result. (users may fetch source using either the SCM CLI (cmd line), or ant tasks, or it's possible that their source control system does not exist in Jazz, etc.). Personal builds also do not create a snapshot (even if the build definition they are using has the "Jazz Source Control" tab selected).

This issue was made into the following work item:
Enhancement 151476: Allow "Request Rebuild..." action to possibly build from the same 'snapshot' associated with a build result.
https://jazz.net/jazz/resource/itemName/com.ibm.team.workitem.WorkItem/151476

Short-term work around: Do as mentioned in above and request that "personal build".

------

I think part of the issue with these problems is that the Jazz build tools are meant to be a light weight driver for other build technologies, and so having complete control and ensuring 100% replication of a previous build is problematic. One advantage though is that our tools allow customers to use their own build system and still post back rich build results (using ant tasks found in our build toolkit or jbe).

So if all these work-arounds are a big set back for any customer in an obtaining 100% historical build replication, perhaps one could investigate using their current build system and integrating it with Rational Team Concert.

A good example of another continuous integration build system that integrates with Rational Team Concert is detailed in the following article "Using the Hudson build integration system with Rational Team Concert"
http://jazz.net/library/article/350

Here are some other misc. links related to this:

Publishing compile results:
https://jazz.net/wiki/bin/view/Main/BuildFAQ#How_do_I_publish_Java_compilatio

Publishing test results:
https://jazz.net/wiki/bin/view/Main/BuildFAQ#How_can_I_publish_test_results


Hope this helps,