It's all about the answers!

Ask a question

Why is there a new SCM?


Les Jones (12611) | asked Sep 13 '07, 12:47 p.m.
I have installed and used Team Concert and am very impressed with the functionality of the product, but I have to admit I have concerns over some of the capabilities of the server component. Specifically, I'm wondering why we feel the need to create yet another new source code management (SCM) system within TeamConcert/Jazz when there are already many useful systems in use today. Wouldn't it have been easier to utilise CVS, Subversion or ClearCase; since these are either open or already IBM owned.

Whilst I can appreciate that it's likely that there is significant involvement from the ClearCase development team (or at least I'd really hope there was) it makes me really nervous to think that such a good product (i.e. Team Concert/Jazz) is going to be based on something so immature when, in theory, it doesn't have to.

It also is concerning that connectivity to any existing deployments of CVS or ClearCase within the enterprise will not treat these tools as first class source repositories; instead treating them as a 2nd class synchronisation point from the TeamConcert/Jazz server repository. This syncronisation provides additional merge/conflict complexity, especially in environments where the existing SCM installation has conflicting changes applied to it (e.g. when the support teams fix the production release using plain old RAD and the changes are merged across to the development branch). There are now potential conflicts at two points (i.e. the merge within the SCM and the syncronisation of Team Server with the SCM) instead of one.

Although I've read the articles in the "Learn about Jazz" section and specifically the "Jazz Platform Technical Overview" article and a few of the wiki articles relating to the connectors and interop, and can see how the product may have got itself into a position where it's become essential that additional metadata is stored for the source artefacts, I'm still not convinced that it's a great move. Therefore, I'd be interested to understand any justification of decisions, or indeed correction/clarification of my mis-interpretations. I guess the same discussion could also apply to the concept of work items and ClearQuest.

Any information you can provide would be very gratefully received.

30 answers



permanent link
Les Jones (12611) | answered Sep 14 '07, 2:25 a.m.
As a point of clarification of why I've asked this question here and now; although I had used Team Concert prior to this week, I'd not fully appreciated some of the implications of the server component until a Jazz presentation at the Rational Software Developer Conference here in the UK this week.

At that presentation I asked this question (and a couple of others that I have also raised in this forum) and the members of the Jazz team present were unable to give me an answer. They suggested that I raise the question here so that I would get the definitive answer from those 'in the know'.

permanent link
Christophe Elek (2.9k12921) | answered Sep 14 '07, 4:57 a.m.
JAZZ DEVELOPER
leslie.jones@capgemini-dot-com.no-spam.invalid (lesojones) wrote in
news:fcd9l0$dcl$1@localhost.localdomain:

They suggested that I
raise the question here so that I would get the definitive answer
from those 'in the know'.


Hello,
We'll try to get you an answer on this one :) Without getting into a
flamming war :)

--
Christophe Elek
Serviceability Architect
IBM Software Group - Rational

permanent link
Les Jones (12611) | answered Sep 14 '07, 9:06 a.m.
Many thanks; I'm not after a war, I can appreciate that opinions on this may vary greatl. I'm interested in the decisions and justifications of the Jazz team. COTS package vs. bespoke build always needs to have a cost vs benefit or risk vs. reward type assesment.

The thing is, I know it's a question I am definitely going to be asked myself and something that I will need to justify to enable TeamConcert/Jazz to be used for real.

permanent link
Brian Nelson (361) | answered Sep 14 '07, 10:06 a.m.
Hello,

Have you checked out the SCM Connector that we're building for Jazz? More
information
can be found on the Jazz wiki:

http://jazz.torolab.ibm.com/wiki/bin/view/Main/ConnectorOverview

In a nutshell, the Connector will allow you to use all the nifty features of
Jazz on the client, while
retaining all your data in ClearCase. New projects can try out Jazz, while
existing installations
can keep using ClearCase, all the while data is sync'ed in both directions
so everyone sees
everyone else's code!

Hope this helps,


Brian

"lesojones" <leslie.jones@capgemini-dot-com.no-spam.invalid> wrote in
message news:fcbpjp$pb1$1@localhost.localdomain...
I have installed and used Team Concert and am very impressed with the
functionality of the product, but I have to admit I have concerns
over some of the capabilities of the server component. Specifically,
I'm wondering why we feel the need to create yet another new source
code management (SCM) system within TeamConcert/Jazz when there are
already many useful systems in use today. Wouldn't it have been
easier to utilise CVS, Subversion or ClearCase; since these are
either open or already IBM owned.

Whilst I can appreciate that it's likely that there is significant
involvement from the ClearCase development team (or at least I'd
really hope there was) it makes me really nervous to think that such
a good product (i.e. Team Concert/Jazz) is going to be based on
something so immature when, in theory, it doesn't have to.

It also is concerning that connectivity to any existing deployments of
CVS or ClearCase within the enterprise will not treat these tools as
first class source repositories; instead treating them as a 2nd class
synchronisation point from the TeamConcert/Jazz server repository.
This syncronisation provides additional merge/conflict complexity,
especially in environments where the existing SCM installation has
conflicting changes applied to it (e.g. when the support teams fix
the production release using plain old RAD and the changes are merged
across to the development branch). There are now potential conflicts
at two points (i.e. the merge within the SCM and the syncronisation
of Team Server with the SCM) instead of one.

Although I've read the articles in the "Learn about Jazz"
section and specifically the "Jazz Platform Technical
Overview" article and a few of the wiki articles relating to the
connectors and interop, and can see how the product may have got
itself into a position where it's become essential that additional
metadata is stored for the source artefacts, I'm still not convinced
that it's a great move. Therefore, I'd be interested to understand
any justification of decisions, or indeed correction/clarification of
my mis-interpretations. I guess the same discussion could also apply
to the concept of work items and ClearQuest.

Any information you can provide would be very gratefully received.

permanent link
Les Jones (12611) | answered Sep 14 '07, 11:18 a.m.
Brian Nelson wrote:
Have you checked out the SCM Connector that we're building for Jazz? More information can be found on the Jazz wiki:
http://jazz.torolab.ibm.com/wiki/bin/view/Main/ConnectorOverview


Thanks; I'd read the article but haven't downloaded anything. (For info, non-IBM-ers can find the article at https://jazz.net/wiki/bin/view/Main/ConnectorOverview).

Unfortunately, for me, the article muddies the water a little - perhaps I'm a little 'hard of thinking' ;-)

On the one hand it states that :
"the Jazz interoperation platform provides data proxies that capture the key data and metadata from the repositories being used, allowing the Jazz interoperation mechanisms easy and efficient access to this data."
which for me implies that the data only resides in the external repository and that the internal Jazz SCM repos is not used for that source artefact, yet the article then goes on to discuss replication and synchronisation, implying exactly the opposite.

The key for me is that if I'm using CVS or ClearCase (or hopefully SVN soon?) as my enterprise SCM, if the code also resides within an internal Jazz SCM (within the Derby/DB2 database) then it's that decision I'm wanting to hear more about. If, on the other hand, the internal SCM is not used to store code, and that the source is always accessed directly from the real SCM (CVS/CC/etc.) with the data proxies associating the relevant metadata to make Jazz work, then there's less of an issue.

Obviously, I appreciate I could be completely mis-interpreting what's going on, but from what I was told at the UK RSDC, when you have a CC repository, the source versions that are relevant to TC/Jazz are held twice and are synchronised as required.

permanent link
Brian Nelson (361) | answered Sep 14 '07, 3:04 p.m.
The key for me is that if I'm using CVS or ClearCase (or hopefully SVN
soon?) as my enterprise SCM, if the code also resides within an
internal Jazz SCM (within the Derby/DB2 database) then it's that
decision I'm wanting to hear more about. If, on the other hand, the
internal SCM is not used to store code, and that the source is always
accessed directly from the real SCM (CVS/CC/etc.) with the data
proxies associating the relevant metadata to make Jazz work, then
there's less of an issue.

After some thought, it's probably pretty obvious that to have the proxies --
the go-betweens
for the respective SCM systems in play -- talk to the other side anytime a
user wants to
do something with a file that the performance would quickly degrade to the
point that it
most likely wouldn't be usable. The only way I can see to get Jazz speed
with ClearCase
data, for example, is to have that data in Jazz.

This way, the user only takes the hit when they synchronize the two sides;
at all other times
both sides get whatever performance is usual for that system.


Brian

permanent link
Geoffrey Clemm (29.7k23035) | answered Sep 14 '07, 9:52 p.m.
FORUM ADMINISTRATOR / FORUM MODERATOR / JAZZ DEVELOPER
lesojones wrote:

... from what I was told at the UK RSDC, when you have a CC
repository, the source versions that are relevant to TC/Jazz are held
twice and are synchronised as required.

To provide a bit more detail, what is synchronized are two streams (one
in TeamConcert, and one in ClearCase). The idea here is that in an SCM
system, a stream (or branch) is the basis of sharing between team
members. So what the connector mechanism does for you is to allow each
team to decide whether they want to work in TeamConcert or in ClearCase
(or both, if you're feeling ambidextrous :-). Changes you deliver to a
ClearCase stream are of course available (via rebase) to all team
members using ClearCase, but are also available (via accept) to all
other team members using TeamConcert.

The fact that information is being copied around in the background
should be of no more concern to you than that information is being
copied around for you during nightly disk backup or multisite replication.

Cheers,
Geoff

permanent link
Les Jones (12611) | answered Sep 17 '07, 3:59 a.m.
Brian Nelson wrote:
"After some thought, it's probably pretty obvious that to have the proxies -- the go-betweens for the respective SCM systems in play -- talk to the other side anytime a user wants to do something with a file that the performance would quickly degrade to the point that it most likely wouldn't be usable. The only way I can see to get Jazz speed with ClearCase data, for example, is to have that data in Jazz.
This way, the user only takes the hit when they synchronize the two sides; at all other times both sides get whatever performance is usual for that system. "

(N.B. Apologies if this post sounds negative or attacking; I've tried re-phrasing a couple of times but each time it sounds even worse when I read it back - I'm just being inquisitive and trying to get the heart of the matter, so please try to take what I'm saying positively :-) )

Thanks for the information Brian, but I'm not quite sure what it is you're implying here.

Are you suggesting that ClearCase (CVS/SVN/et.al) cannot perform as required?
Are you suggesting that the implementation of the proxy mechanism un-does any scalability improvements (such as request aggregation) that the underlying SCM would otherwise be able to take advantage of?
Do you have a specific target network topology that is driving these decisions?

As I see it, the team server should be linked to the external SCM via a very fast, low latency LAN connection. In many circumstances one could assume the servers to be in the same rack or possibly even co-located on the same server. It sounds to me though, that you're assuming the team server to external SCM link to be slow and chatty and hence something to be avoided where possible; yet it's this very link that most developers will be using day in/day out right now.

All that said, this still doesn't explain why a new SCM is being written from scratch, it would merely explain why TC/Jazz has an embedded SCM.

permanent link
Les Jones (12611) | answered Sep 17 '07, 4:09 a.m.
gmclemm wrote:
"The fact that information is being copied around in the background should be of no more concern to you than that information is being copied around for you during nightly disk backup or multisite replication."

Unfortunately, it is of great concern. Ops have been doing nightly backups for decades and it is an incredibly well understood activity, so not really a like-for-like comparison.

The better example is CC multi-site. Whilst this is very well known to us now, this knowledge has been gained over several years of executing projects with CC-MS. This was initially very painful but has paid dividends in the long run by having key individuals understanding the details of the CC-MS 'black box'. The point here is that any new 'black box' will not work as expected in it's first release; it will need a bedding in period. By having a new bespoke SCM at the heart of TC/Jazz, a substantial risk has been added that could have been mitigated via the use of existing technology.

I do appreciate that the cost of this additional risk may be offset by measurable benefits in implementing the new SCM; it's this concrete argument I am yet to hear.

permanent link
Les Jones (12611) | answered Sep 17 '07, 4:14 a.m.
gmclemm wrote:
"To provide a bit more detail, what is synchronized are two streams (one in TeamConcert, and one in ClearCase). The idea here is that in an SCM system, a stream (or branch) is the basis of sharing between team members. So what the connector mechanism does for you is to allow each team to decide whether they want to work in TeamConcert or in ClearCase (or both, if you're feeling ambidextrous :-). Changes you deliver to a ClearCase stream are of course available (via rebase) to all team members using ClearCase, but are also available (via accept) to all other team members using TeamConcert."

Geoff, thanks for this detail. In this style of working this is obviously a very good practice and something I'd expect many of us to be doing. However, since there will be some who try to work in a more chaotic way, good practice documentation is always going to be essential to guide those 'renegades' ;-)

Your answer


Register or to post your answer.