Options for converting a very large existing environment (SVN & CVS) to RTC
Andrei Litvin (23●1●2●9) | asked Apr 03 '13, 8:53 a.m.
We are looking at performing a large migration from existing source control (mostly SVN and CVS ... and we can automate CVS to SVN as an intermediate step) to Rational Team Concert. However the size of our environment is giving me trouble, since I cannot see any way of automating things. In general, our environment looks like this:
- about 100 SVN respositories (with a format labeled as "uncommon" by the SVN2RTC conversion documentation)
- about 80 CVS repositories
- 20-40 repositories in a mix of other formats (Source Safe, Mercurial, Git)
- about half of our repositories are large: 10-15 years of history, with 10+ branches out of which 5-6 are actually maintenance branches that get developed on (or at least fixes get backported on them and we get nightly builds on them)
And I need to convert all of the above to RTC. I would consider the "mix of other formats" on a case-by-case basis, however I would expect to have SVN and CVS converted in an uniform matter. The CVS conversion it seems should be done via an intermediate cvs2svn conversion, to preserve branches and tags.
In any case, I tried to gather as much information about this as possible. I installed RTC in a test environment on a VM (with DB2 and everything, due to the sheer size of our repositories), and gathered also the following on-line documentation:
https://jazz.net/forum/questions/61525/svn-and-cvs-migration-to-rtc - asks about CSV/SVN Migration on windows, however it has some good links. Specifically:
- Main migration description: https://jazz.net/library/article/650
- Migration planning guide: https://jazz.net/library/article/70
- Concept comparison between SVN and RTC: https://jazz.net/library/article/639
- Easing into Jazz Source Control: https://jazz.net/library/article/539
Supported import layouts:
Is command-line supported (with the answer being: NO, not supported):
Points that are giving me trouble:
a) "Repository organizations where the trunk folder itself maps to an Eclipse project are uncommon and are not supported by the importer:"
We have about 100 of these "uncommon" projects, and our tried-and-tested cvs-to-svn scripts would convert any cvs repository to this format, resulting in 200 of the uncommon layouts. My best solution so far was to update the svn dump by either using something like https://github.com/jwiegley/svndumptool or script one myself based on the documentation at http://svn.apache.org/repos/asf/subversion/trunk/notes/dump-load-format.txt
b) "the SVN importer only imports a single branch at a time"
We have maintenance requirements for about 5-6 branches for most of the projects, and in general I think nobody would like to lose history or have "new code on RTC, old code in svn/cvs/something else". This makes it so that not only do we have 100+ projects, but each project will need 6-12 branches imported as well.
c) No scripting support, only eclipse GUI.
GUI-based, manual conversion seems highly impractical due to the sheer volume of data. Humans make mistakes when repeating the same task over and over again.
What would be some viable options for us doing the conversion?
I am looking at importing 100s of branches it seems, with quite a few of them being in the 2-5 hour import range according to the planning guide. I considered scripting eclipse, however http://wiki.eclipse.org/E4/Scripting shows no progress, and even if Eclipse Monkey or some of it's forks work, this looks like an awful hack. Is there anything I am missing?
Tim Mok (6.4k●3●8) | answered Apr 04 '13, 1:23 p.m.
You could write your own importer by extending the import framework and you would have to do it for each of the other repo formats (git, Mercurial, etc.) if you can't convert them to SVN. The documentation for this is sparse (https://jazz.net/wiki/bin/view/Main/SCMImportFromOther) and this would be quite a bit of work if your SVN repo structure deviates from standard (ie. trunk, branches, tags). However, it would give you better control on making the import automated and you can handle the specific way you use SVN.