Source code control non- java/c, help!?!
My head is about to explode.. I have this 'product' that builds multiple runtimes from the same source, on 5 different platforms. and requires 2 different delivery packages.
It takes too long to work thru all the processes to make changes, do the builds, make the deliverables, do the tests.. so I want to put this all together into a system.. source mgmt, change mgmt, auto builds, test and packaging.
so I setup a jenkins server, with distributed slaves, all running on virtual machines.. windows, 3 flavors of linux( 32 and 64bit), solaris, aix, and hpux.
I defined builds and can invoke them.. so the back part works.. (I don't do packaging build yet)..
I want to put the build results (binaries) into artifactory to deliver to the packager, and anyone else that wants to take the parts and test, or repackage.
so far, so good..
not its time to address the source code mgmt side..
I develop in Eclipse. but have hand built shell script build scripts.
again,this is not java. but perl.
so, now I want to put this into source control.. and I'm stuck.. who come first? the chicken or the egg?
the sample doc is create a workspace, and deliver to the stream. wait, who created the stream, how does the code/parts get INTO the workspace.. I have something already,, isn't that a baseline? how does that get into the system..
I see share a 'project'.. but right now I am pretty monolithic and I need to fix that..
I have the code, test scripts, the delivery packaging scripts, the build scripts, some
supporting files (images) and special text files.. I want to put these is separate areas in the source system (is this component?).
then I also have the perl runtime, and the special compiler runtime I use to DO the build.
I suppose I would put those in Artifactory, and deploy to the build system as part of the build definition..
(but binaries in the source system are bad news)
scm load with get the files OUT of the source system.. what puts them in THE FIRST time?
It takes too long to work thru all the processes to make changes, do the builds, make the deliverables, do the tests.. so I want to put this all together into a system.. source mgmt, change mgmt, auto builds, test and packaging.
so I setup a jenkins server, with distributed slaves, all running on virtual machines.. windows, 3 flavors of linux( 32 and 64bit), solaris, aix, and hpux.
I defined builds and can invoke them.. so the back part works.. (I don't do packaging build yet)..
I want to put the build results (binaries) into artifactory to deliver to the packager, and anyone else that wants to take the parts and test, or repackage.
so far, so good..
not its time to address the source code mgmt side..
I develop in Eclipse. but have hand built shell script build scripts.
again,this is not java. but perl.
so, now I want to put this into source control.. and I'm stuck.. who come first? the chicken or the egg?
the sample doc is create a workspace, and deliver to the stream. wait, who created the stream, how does the code/parts get INTO the workspace.. I have something already,, isn't that a baseline? how does that get into the system..
I see share a 'project'.. but right now I am pretty monolithic and I need to fix that..
I have the code, test scripts, the delivery packaging scripts, the build scripts, some
supporting files (images) and special text files.. I want to put these is separate areas in the source system (is this component?).
then I also have the perl runtime, and the special compiler runtime I use to DO the build.
I suppose I would put those in Artifactory, and deploy to the build system as part of the build definition..
(but binaries in the source system are bad news)
scm load with get the files OUT of the source system.. what puts them in THE FIRST time?
6 answers
(This is from memory. The order might not be completely accurate, since I don't do this very often.)
Organize all the source files in Eclipse projects as you see fit. It is nice if you get everything organized correctly from the start. But it is easy enough to move things around later, so don't fret about it.
Create any SCM components that you might need. The components will be empty at first.
Create a new stream, adding your components to it.
Create a new repository workspace from the stream. (Right click on the stream and select New Workspace.) By doing it from the stream, the repository workspace will have the same components as the stream. But it will be empty, so there won't be anything to load into your Eclipse workspace yet.
Right click on a project in your Eclipse workspace and select Team > Share Project. That will take you through a wizard and put that project under source control in your repository workspace. Once you have shared all the Eclipse projects that you want to share, you can deliver your changes from the repository workspace to the stream.
If you are unsure about this, or if you want to practice before doing it for real, you should try it out in a test RTC repository first. Once you are done practicing, you can right click on the Eclipse project and select Team > Disconnect. That will disconnect the Eclipse project from the repository without deleting its contents, so can then share it again with the real RTC repository.
I do not recommend storing the Perl runtime or the special compiler runtime in RTC source control. Artifactory seems like a better choice. But there are a lot of things that affect that decision, such as their size, how often they change, and how they get installed on the build machines. So I can't give you a firm answer on that one.
Comments
David, we store things like the ECJ compiler jars in SCM in the MTM sample. I think it is OK to do so if they don't change often and it makes building easier. If you don't want to pay the price of loading these tools every time, one trick I have see was have a build that loads the tools and the regular build does only load the source.
so, the approach is
components
stream
workspace - this is a personal?
thanks.. missing data still , how do I get my physical files related to the components? the wizard doesn't seem to understand folders, or .txt or .pl files..
also, how do I delete a component definition? I want to do something different
and on the binaries.. just be warned.. the OOtB config with Websphere will fall over and die if the binaries consume the websphere heap.. we had a team store the apache source kit and the oracle db install kit in the source tree and reload it on every clean build.. kills websphere faster than u can imagine.
components
stream
workspace - this is a personal?
thanks.. missing data still , how do I get my physical files related to the components? the wizard doesn't seem to understand folders, or .txt or .pl files..
also, how do I delete a component definition? I want to do something different
and on the binaries.. just be warned.. the OOtB config with Websphere will fall over and die if the binaries consume the websphere heap.. we had a team store the apache source kit and the oracle db install kit in the source tree and reload it on every clean build.. kills websphere faster than u can imagine.
Sam,
if you want to share files in the eclipse client, you must have the file in an Eclipse project. Each component can hold the files for several Eclipse projects. In the Eclipse client you can only share Eclipse projects. The Eclipse project can be just a general project. It does not have to do anything, except the .project file needs to be there.
If you have a folder structure like:
arbitraryRoot
Folder 1
Folder2
Folder3
Folder4
You can have several options. 1st you can create just one eclipse project and copy the whole tree into it (with the project being the folder that contains the data of arbitraryRoot and all sub folders. This would not allow componentization.
If arbitraryRoot does not contain any data, you can create Eclipse projects for all the folders (with the folder name as project name) and do the same for each folder and share each folder. This allows to split into components.
There are more capabilities in the SCM that could be interesting. See https://jazz.net/wiki/bin/view/Main/SourceControlMain and for loading https://jazz.net/wiki/bin/view/Main/SCMContentLoading .
As far as I understand with load as and component rules you can do a lot more magic to load things in the way you want it, but I haven't collected a lot of experience with it.
If you have a massive amount of binaries, don't put them into SCM, set up a caching proxy in front of your build farm, or just load the binaries once with a different build definition and be safe to assume they are stored somewhere stable relative to the build folder, so that you can access them on every build machine.
if you want to share files in the eclipse client, you must have the file in an Eclipse project. Each component can hold the files for several Eclipse projects. In the Eclipse client you can only share Eclipse projects. The Eclipse project can be just a general project. It does not have to do anything, except the .project file needs to be there.
If you have a folder structure like:
arbitraryRoot
Folder 1
Folder2
Folder3
Folder4
You can have several options. 1st you can create just one eclipse project and copy the whole tree into it (with the project being the folder that contains the data of arbitraryRoot and all sub folders. This would not allow componentization.
If arbitraryRoot does not contain any data, you can create Eclipse projects for all the folders (with the folder name as project name) and do the same for each folder and share each folder. This allows to split into components.
There are more capabilities in the SCM that could be interesting. See https://jazz.net/wiki/bin/view/Main/SourceControlMain and for loading https://jazz.net/wiki/bin/view/Main/SCMContentLoading .
As far as I understand with load as and component rules you can do a lot more magic to load things in the way you want it, but I haven't collected a lot of experience with it.
If you have a massive amount of binaries, don't put them into SCM, set up a caching proxy in front of your build farm, or just load the binaries once with a different build definition and be safe to assume they are stored somewhere stable relative to the build folder, so that you can access them on every build machine.