Why is there a new SCM?
I have installed and used Team Concert and am very impressed with the functionality of the product, but I have to admit I have concerns over some of the capabilities of the server component. Specifically, I'm wondering why we feel the need to create yet another new source code management (SCM) system within TeamConcert/Jazz when there are already many useful systems in use today. Wouldn't it have been easier to utilise CVS, Subversion or ClearCase; since these are either open or already IBM owned.
Whilst I can appreciate that it's likely that there is significant involvement from the ClearCase development team (or at least I'd really hope there was) it makes me really nervous to think that such a good product (i.e. Team Concert/Jazz) is going to be based on something so immature when, in theory, it doesn't have to. It also is concerning that connectivity to any existing deployments of CVS or ClearCase within the enterprise will not treat these tools as first class source repositories; instead treating them as a 2nd class synchronisation point from the TeamConcert/Jazz server repository. This syncronisation provides additional merge/conflict complexity, especially in environments where the existing SCM installation has conflicting changes applied to it (e.g. when the support teams fix the production release using plain old RAD and the changes are merged across to the development branch). There are now potential conflicts at two points (i.e. the merge within the SCM and the syncronisation of Team Server with the SCM) instead of one. Although I've read the articles in the "Learn about Jazz" section and specifically the "Jazz Platform Technical Overview" article and a few of the wiki articles relating to the connectors and interop, and can see how the product may have got itself into a position where it's become essential that additional metadata is stored for the source artefacts, I'm still not convinced that it's a great move. Therefore, I'd be interested to understand any justification of decisions, or indeed correction/clarification of my mis-interpretations. I guess the same discussion could also apply to the concept of work items and ClearQuest. Any information you can provide would be very gratefully received. |
30 answers
Les,
Apologies that an answer was not forthcoming sooner. I'll try to answer your question. The Jazz vision is fundamentally about integration across the phases of the lifecycle. We believe that this kind of end-to-end integration leads to what we call emergent value - or value greater than the sum of the parts. What kind of value do we envision? Better control over your project, better understanding of your project and its state of "health", automation of previously manual tasks, quick-fix automated assistance because the tools understand what you are doing, less programmer time spent on "rote" administrative tasks that add little value to what you are building, process integration that ensures the process defined by the team is actually followed, tracability of artifacts and artifact relationships that allows reasoning about how changes to one artifact affect related others, tracking and auditability of everything that happens to the state of the repository and your code, reduced development costs, ... the list goes on and on. Studies done into collaborative development environments for software show us that collaborative software development is fundamentally artifact based, and change management/versioning is a core capability that acts to anchor all of the other pieces. A limitation of even the best of today's state of the art SCM, or work tracking, or other lifecycle tools, is that they were not designed from the ground up for this kind of integration. While they often do a good job of addressing particular phases of the software lifecycle, they arent extensible in a general way, and dont allow navigation and tracability between all related artifacts *across* the phases of the lifecycle. With Jazz, we set out to build a fully integrated (yet architecturally componentized), collaborative development environment with the integration built in from day 1, because we don't believe you can fully achieve this level of integration "after the fact". Yet at the same time we recognized that we needed to also integrate with established "backbone" SCM systems like ClearCase, and work with other popular SCM's like Subversion. We are therefore persuing two paths simultaneously: First, we are building a new, modern SCM technology designed from the ground up to support a highly integrated collaborative development environment. We have assembled some of the top technical talent in IBM Rational with a diverse set of skills ranging from agile and open source systems to formal enterprise systems. We have ClearCase developers and experts on the team - some of the top SCM professionals in the industry who understand what works and what doesn't, and the pros and cons of virtually every SCM approach on the market today. This SCM is built on a new general purpose, extensible, repository system that runs on a standard J2EE application server and relational database and leverages modern internet standards. It works on Tomcat/Derby on the low end, and is designed to leverage high end middleware like WebSphere and DB2 (or similar products from other companies) for maximum scalability and performance. Because this SCM is built on the general purpose Jazz repository, we have full integration across all repository artifacts. Second, we are implementing two integration approaches to interoperate with other existing SCM systems: One is a connector mechanism that allows Jazz to integrate with and complement backbone SCM systems using a replication approach. The other is a looser integration mechanism that allows you to use the non-SCM parts of Jazz to complement an existing SCM, albeit with a reduced level of integration and some limits on the tracability, artifact reasoning, and process control. As you might expect, we focused on interconnection with ClearCase as our first priority, and you will see the combination of ClearCase and Jazz continue to evolve towards a more integrated, "best-of-both-worlds" environment. Over the next months you'll also see us beginning work on other integrations. We believe this approach gives us the flexibility to both deliver the maximum value of integration, while also complementing and integrating with other SCM systems in ways that give customers a great deal of flexibility. But as we learn more we will adjust over time. Our vision has not been fully proven yet, but we believe we are on a good path. The exciting thing is that when we started the Jazz project we predicted emergent value through integration, but now we are actually seeing it both in our own use of Jazz for development, and in feedback from others. Even more exciting is that as we learn more we are recognizing more and more opportunities that were inconceivable with today's state of the art looser integrations. And there are several University research teams pushing the limits even further to see how far they can go with Jazz. We believe we are only beginning to understand all the possibilities that this kind of integration enables. Dave Thomson IBM Distinguished Engineer Jazz Project Director |
|
Dave,
Many thanks for taking the time to answer my query. It's very pleasing to note you confirm that there are ClearCase developers and experts within the team developing the new SCM. This kind of knowledge re-use is absolutely essential in ensuring that the new product has a chance at success. I believe that failures in the SCM after release would be very costly since it would potentially put at jeopardy the trust and integrity of the whole of the Jazz platform dependent upon it. It's also reassuring that you are committed to delivering integration between Jazz and existing SCM products. Whilst I can understand that ClearCase would be a priority, it's my view that integration with SVN and CVS would be a key requirement for the 1.0 release of TeamConcert. This would allow the close collaborative working to be effective within the Jazz based team, whilst also allowing deliveries to existing corporate repositories. Do you know if this is a realistic expectation? Finally, I feel that key messages such as the response you have given, are important and should be easily accessible. It's likely that others will also want to understand the rationale behind this, and other decisions. Would it be appropriate to setup an area on Jazz.net (e.g. within the wiki or within the "Jazz Platform Technical Overview" article) to capture this information to help prevent similar queries in the future? regards, Les |
|
Yes, although we haven't finalized our plans, SVN integration is likely for 1.0. At present we are discussing doing the looser integration mechanism first. We'll be updating the 1.0 plan on the website in the next few weeks to reflect SVN integration plans for 1.0. That said, I would be interested in knowing more about the use case that is of primary interest to you, so we can factor this into our thinking.
I agree completely that we need to do a better job of making this kind of information more accessible. We are looking at a few options... one possibility is a FAQ format. Thanks again for initiating this discussion. I think it is an important one and I appreciate your patience. Dave |
At the risk of opening up, what appears to be, a hot topic ....
I understand the point that the Jazz SCM 'integrates' fully with the rest of the Jazz, presumably because its SCM carries important metadata along with the source payload, and that this level of integration can't be otherwise achieved. CVS can't do this in any generalized way, but CC can (there's a connector for that) but as I understand it, Subversion probably also can. i.e. SVN can carry arbitrary metadata with it's source file payload. Is there agreement that SVN could, in principle (and ignoring performance issues), be used as a full-function Jazz SCM (like CC) if a suitable connector was developed ? Dave |
Geoffrey Clemm (30.1k●3●30●35)
| answered Nov 02 '07, 3:17 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
Hi David,
To be clear, the ClearCase Connector synchronizes SCM data between ClearCase and the Jazz SCM system (bidirectionally and incrementally). So the way you get tight integration between ClearCase and TeamConcert is not by connecting ClearCase directly to the other components (such as Workitems) in TeamConcert (I would call that a "bridge"), but rather by connecting ClearCase to Jazz SCM (what I would call a "connector"). So you get maximal integration of SCM with the other TeamConcert components by using Jazz SCM, but you get the "next best thing" by using a connector between your favorite SCM system (e.g., ClearCase) and Jazz SCM (and use Jazz SCM in your daily activities). In release 1.0 of TeamConcert, we'll be providing a "connector" to ClearCase, and a "Workitem bridge" to SVN. The SCM connector technology could be used to provide an SCM connector to SVN, but that currently is not in the release 1.0 plan. Cheers, Geoff davidward@us.ibm.com wrote:
|
|
Hi again Geoff
Is there any published info about the planned SVN workitem bridge (and any other SCM bridges that are planned)?? Cheers Dave Geoffrey Clemm wrote: Hi David, |
|
Geoffrey Clemm (30.1k●3●30●35)
| answered Dec 01 '07, 8:58 a.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
You can investigate (and comment on) all of the ongoing bridge work, by
querying for workitems in the category of "Source Control/Import" (a benefit of "open commercial development" :-). For example, the SVN work is in the category of "Source Control/Import/SVN" (both "import" and "bridge" work currently appears in the "import" category). The other importer is for CVS, in "Source Control/Import/CVS". David Ward wrote: Hi again Geoff |
David,
A wiki page has been created that describes the state of the SVN support in the Jazz M4 (and beta2) drops: https://jazz.net/wiki/bin/view/Main/SCMSubversion Feel free to provide any feedback by logging work items against the "Source Control/Import/Subversion" category. Michael Geoffrey Clemm wrote: You can investigate (and comment on) all of the ongoing bridge work, by |
Hi Michael,
I suspect this wiki page (along with the message of SVN-Jazz interop) will prove very helpful in moving Jazz into the customer space. Speaking as an SVN advocate working at several CVS shops, it's a hard sell to get anyone to simply drop the source control with which they feel comfortable and adopt a new one. I think the graded approach of offering both integration and migration paths between Jazz and SVN will help ease the concerns. This wiki page will be very popular as we move this into the field. - Paul Michael Valenta wrote: David, |
|
Michael
Very interesting wiki page. Thanks for letting me know. Michael Valenta wrote: David, |
Your answer
Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.