It's all about the answers!

Ask a question

Why is there a new SCM?


Les Jones (12611) | asked Sep 13 '07, 12:47 p.m.
I have installed and used Team Concert and am very impressed with the functionality of the product, but I have to admit I have concerns over some of the capabilities of the server component. Specifically, I'm wondering why we feel the need to create yet another new source code management (SCM) system within TeamConcert/Jazz when there are already many useful systems in use today. Wouldn't it have been easier to utilise CVS, Subversion or ClearCase; since these are either open or already IBM owned.

Whilst I can appreciate that it's likely that there is significant involvement from the ClearCase development team (or at least I'd really hope there was) it makes me really nervous to think that such a good product (i.e. Team Concert/Jazz) is going to be based on something so immature when, in theory, it doesn't have to.

It also is concerning that connectivity to any existing deployments of CVS or ClearCase within the enterprise will not treat these tools as first class source repositories; instead treating them as a 2nd class synchronisation point from the TeamConcert/Jazz server repository. This syncronisation provides additional merge/conflict complexity, especially in environments where the existing SCM installation has conflicting changes applied to it (e.g. when the support teams fix the production release using plain old RAD and the changes are merged across to the development branch). There are now potential conflicts at two points (i.e. the merge within the SCM and the syncronisation of Team Server with the SCM) instead of one.

Although I've read the articles in the "Learn about Jazz" section and specifically the "Jazz Platform Technical Overview" article and a few of the wiki articles relating to the connectors and interop, and can see how the product may have got itself into a position where it's become essential that additional metadata is stored for the source artefacts, I'm still not convinced that it's a great move. Therefore, I'd be interested to understand any justification of decisions, or indeed correction/clarification of my mis-interpretations. I guess the same discussion could also apply to the concept of work items and ClearQuest.

Any information you can provide would be very gratefully received.

30 answers



permanent link
Brian Nelson (361) | answered Sep 18 '07, 9:54 a.m.
(N.B. Apologies if this post sounds negative or attacking; I've tried
re-phrasing a couple of times but each time it sounds even worse when
I read it back - I'm just being inquisitive and trying to get the
heart of the matter, so please try to take what I'm saying positively
:-) )

Understood. :-)

Are you suggesting that ClearCase (CVS/SVN/et.al) cannot perform as
required?
Are you suggesting that the implementation of the proxy mechanism
un-does any scalability improvements (such as request aggregation)
that the underlying SCM would otherwise be able to take advantage
of?
Do you have a specific target network topology that is driving these
decisions?

As I see it, the team server should be linked to the external SCM via
a very fast, low latency LAN connection. In many circumstances one
could assume the servers to be in the same rack or possibly even
co-located on the same server. It sounds to me though, that you're
assuming the team server to external SCM link to be slow and chatty
and hence something to be avoided where possible; yet it's this very
link that most developers will be using day in/day out right now.

While co-locating may work for some groups, it certainly won't work for
everyone. In these
days of remote access, I'm actually doubtful it would even work for a
majority, most likely
a slim minority. That being the case, at some point you're going to have
network hops, and
even if they're local (LAN), it's still going to slow things down a lot. I
doubt customers would
embrace a system where we told them they HAD to put a new SCM system on the
same
server as their existing SCM system. And BTW, with ClearCase at least, the
big customers
tend to have more than one server (VOB servers, view servers, registry
server, etc), so it's
quite impossible to co-locate everything together anyway. Those servers are
very busy as it
is, putting another SCM system on the same server would eat up more already
scarce
resources.


All that said, this still doesn't explain why a new SCM is being
written from scratch, it would merely explain why TC/Jazz has an
embedded SCM.


I wasn't attempting to address your core question, merely one aspect that
you brought up.
I'd suggest reading the Jazz docs, although it's not hard to understand why
IBM would want
an SCM system that is feature-rich like ClearCase and fast like CVS -- and
to get that you'd
probably need to start from scratch.


Brian

permanent link
Les Jones (12611) | answered Sep 19 '07, 6:45 a.m.
Brian Nelson wrote:
"While co-locating may work for some groups, it certainly won't work for everyone. ..."
Totally agree; I can see how it could be interpreted that that's what I mean, but it isn't. Co-location is certainly an option for many teams, but certainly not all.

I guess my point really was that teams have an existing SCM (well - I hope they do!) that is running in an existing environment that they are happy with. (Environment includes server hardware, deployment topology, network, etc.) Putting Jazz into that environment could be a co-located installation (if the server has sufficent cpu/memory/disk/etc.) or could be on a separate server. The real difficulty I'm having is in understanding that step, i.e. the justification, that requires the additional SCM; and on top of that, why an existing package (CVS/CC/SVN) - perhaps with a few extensions - would not be acceptable.

permanent link
Les Jones (12611) | answered Sep 24 '07, 10:08 a.m.
Whilst I'm grateful for the time taken to provide some responses, it's unfortunate that the core question hasn't been answered. Therefore, in the hope of getting a straight answer, I'll re-articulate the question.

I think it is essential that, as an industry, we do not continually re-invent the wheel; instead focussing our time and effort onto those things that are going to deliver real value to our clients. Utilising existing technologies and packages, where appropriate, allows us to focus our efforts appropriately.

It is important to note that the Jazz platform does not write a new web server nor try to implement a new relational database (for a simple install we get Tomcat and Derby, for a more scalable solution we take a little more time and effort and migrate to WebSphere and DB2). Also, to support the server host for the components, Equinox has been used, giving a stable underpinning into which the components can be deployed, Eclipse has been used as the basis for the Team Concert client, and I'm sure there are many other examples.

Why then has the Jazz team chosen to move away from this philosophy for the SCM, where there are already existing and mature products that could have been used. I believe it is essential that we prevent an unnecessary proliferation of such tools, especially a new immature SCM component, unless the relevant due diligence has been undertaken to balance the risks and costs against the benefits, showing that the latter out-weighs the former.

As an absolute minimum I would have thought that a simple comparison (e.g. spreadsheet or HTML table in the wiki) would be capable of showing the thought processes that had been undertaken and the decisions made, and that this could have been provided quickly and with little fuss. I am disheartened that something like this has not been forthcoming.

Although I appreciate the new SCM will be getting widespread testing during this open beta, that, in itself, would not be a good enough answer and detracts time and effort away from the core essence of what we should be testing; i.e. the ability for Team Concert/Jazz to provide a platform to improve productivity and reduce the risks of delivering projects distributed in multiple locations.

Therefore, is it possible that someone involved in the decision making process please post an answer that provides the clear justification of why this decision was made, what other options were considered and why they were deemed to be less appropriate.

Many thanks.

permanent link
Aaron Cohen (8207851) | answered Sep 24 '07, 10:36 a.m.
JAZZ DEVELOPER
I personally like the fact that Jazz is including a new SCM. Right now, no matter which SCM/Defect solution you use, your data ends up distributed all over the place. Your Source Control Files are in one database, you defects are in another database, and linking them all together becomes a job unto itself. Granted I am only an end-user of Jazz and not involved in any decision making, but the fact that jazz is essentially a common repository for scm/defects/etc...should be reason enough for the new SCM.

permanent link
Les Jones (12611) | answered Sep 24 '07, 10:47 a.m.
Aaron, thanks for your perspective. I'm certainly not anti a single repository and I do appreciate that others will have a different perspective from myself.

As you may be able to guess though, I disagree with you that this would be a good enough reason in its own right; however, it certainly could be deemed to be one of the benefits of a new bespoke SCM - also appreciating that this advantage is an operational benefit, not an end-user benefit.

permanent link
Aaron Cohen (8207851) | answered Sep 24 '07, 11:05 a.m.
JAZZ DEVELOPER
I agree with the "if it ain't broke, don't fix it" approach, but in many ways the current SCMs out there are indeed broken (in my opinion) from the standpoint of a small team. Now I am not trying to start a flame war, but for small teams that need the strong integration between workitems/defects/enhancements/source control with next to no administration work, there really isn't much out there. Administrating ClearCase and Clearquest is almost a full-time job unto iteslef and open source tools like gforge/bugzilla/cvs/subversion/mylyn simply do not have a rich enough feature set to meet all of the needs. Granted they come close, but are not quite enough.

So far I am able to administer 30+ people on my Jazz server. Granted I am running into problems here and there but they are nothing compared to the Administration headaches associated with a Clearcase/ClearQuest deployment I made for 5 people.

The Jazz solution should make small development teams nimble. As I pointed out before, there are lightweight scm solutions out there but its the ability to tightly integrate them in one repository that makes the key difference.

permanent link
Les Jones (12611) | answered Sep 24 '07, 11:14 a.m.
Aaron, totally agree that we don't want a flame war; would be an inappropriate use of this forum. So I'm not going to comment on my views of CC/CQ/Bugzilla/CVS/SVN/etal...

All I want is the decision makers to publicise their decision making process. At the end of the day, to enable me (and I'm sure others like me) to deploy Team Concert and Jazz into a 'real' development environment, I am going to have to justify things such as the new SCM. Without this information, to introduce something new is going to be a difficult uphill task and ultimately could prevent the use of an excellent tool within our teams. I feel that would be a great shame.

permanent link
Pete Franklin (6) | answered Sep 24 '07, 11:43 a.m.
A unified repository is something I've been asking Rational for for some years now - everything, be it a requirement, a defect report, change request, database create script, class in a diagram or piece of code should just be a object in the repository.

If the new SCM is supposed to deliver this, it may be a justifiable design decision, but as Les has pointed out it does not come without risk. Given that this risk is likely to affect take-up of the platform, Les' original question still seems reasonable, and I'm starting to wonder why no-one is answering it.

permanent link
David Robinson (2111) | answered Sep 25 '07, 2:46 a.m.
From the original post:
"Whilst I can appreciate that it's likely that there is significant involvement from the ClearCase development team (or at least I'd really hope there was) it makes me really nervous to think that such a good product (i.e. Team Concert/Jazz) is going to be based on something so immature when, in theory, it doesn't have to."


Just to add my 2 cents to focusing this discussion, I think that Les (correct me if I am wrong) is really looking for assurance the the Jazz SCM is not as immature as it might appear.

There are 2 important aspects to designing a new product (I know I'm simplifying here :o ) - the WHAT functionality to include and the HOW to implement it.

IBM has access to a rich experience with SCM design with its ClearCase personell and customers want to know that we have drawn on that experience, learning from both things that are really good about ClearCase and things that could have been done better. Certainly there are people in ClearCase who with its current functional design, which has evolved over about 16 years, would like to redesign the implementation of that from the ground up! instead of moving forward on the legacy code base.

Similarly there is a rich experience within IBM of using other SCM tools like CVS.

Jazz Source Control Overview
does not give reference to this history.

So I think that Les is looking for the Jazz team to "declare their SCM heritage" rather than just rely on the hope "that it's likely that there is significant involvement from the ClearCase development team". In that case the Jazz SCM turns out to be not really immature at all.

permanent link
Les Jones (12611) | answered Sep 25 '07, 3:02 a.m.
David, thanks for your input.

Your interpretation is on the right lines but not spot on. What you've highlighted is a possible (and very important) key risk mitigator that could have been used to help justify the decision. Again, on its own, unlikely to be a sufficient argument, but one of the aspects that would help move the team away from an existing package and towards the 'bespoke build' decision.

Your answer


Register or to post your answer.


Dashboards and work items are no longer publicly available, so some links may be invalid. We now provide similar information through other means. Learn more here.