Jazz Source Control: Design Objectives
We started with a rather simple objective of wanting to build a source control tool that made working with others easier and more collaborative. Sounds simple enough, but our experience was that collaboration was often obfuscated by source control systems that leak their implementation details to their users. Instead it should be the other way around: collaborative concepts should be first class. Tasks such as reviewing changes, switching tasks, exchanging changes with others, requiring code reviews during the end game, and setting up enterprise level promotion hierarchies should not require a doctorate in branching or the duct taping of 5 different tools together.
When we started development, the buzz around distributed version control was increasing. But what we saw in distributed version control was a set of users asking for easier collaboration but not necessarily the physical side effect of having everyone with their own entire copy of the repository on their personal machines. To us, this was a good reinforcement that users crave more flexible collaboration support, but we viewed distributed as a deployment pattern and not a means to an end. I had always hoped that if our source control model was flexible enough and built on a solid foundation of simple collaboration concepts, distributed would just be an option you had in how you worked with others but you didn’t “have” to be distributed to benefit from these collaboration features. Consequently, we started development not by making Jazz Source Control distributed but instead worked on the collaboration model and how to bring this collaboration model to the user and push on the ALM integrations with the other lifecycle domains such as build and work items.
Before outlining the main design goals of Jazz Source Control, let’s talk a bit about the overall product called Rational Team Concert. Although we have built a source control system included in Rational Team Concert, we recognize the popularity and breadth of usage of other tools. Since Team Concert is more than just about source control, we allow the use of other versioning tools such as Git, ClearCase, or Subversion with the planning and tracking features and scrum/agile development features of Team Concert. If you aren’t in the market for a new source control tool, but you like all the tracking and planning features in Team Concert, you can bring both worlds together.
The remainder of this article introduces the top design goals with examples. Note that we haven’t finished implementing all the aspects of the vision, but as you’ll discover, there are many aspects of each design goal which have now been implemented. During the next releases we are working on expanding the support in each one of these areas.
Goal 1 : Give all users access to the version control tool
Separate the right to use of version control with the gesture of sharing your changes with your team. This means that you should be able to collaborate with peers, backup changes, rollback, or show history, independently from when and how you make those changes visible to my team. The desired outcome was to provide a large portion of the benefits of a distributed “mindset” in a central repository, for customers that can’t allow distributed or don’t want to.
This is a common theme with distributed tools: you don’t need commit rights to use a version control tool on a code base that you want to work with. We’ve found that many of our customers didn’t want the overhead of allowing every developer to have a repository on their laptops, but they wanted to support a similar model as a distributed one: allowing users to use version control even when they don’t have commit rights or don’t want to be forced to share their work with the team just to back it up. It’s also not practical with some of the code base sizes that we deal with to force all users to have full history replicas just to get this flexibility.
To this end, we’ve added server side workspaces, called repository workspaces. They allow users have a private area on the server in which to make their changes, check-in, experiment, and backup. They control this area, and decide when to push their changes (deliver), or pull changes (accept). Users can also accept changes from other users directly or from work items, bypassing a stream entirely. In the figure below users Alize and Gabriel have repository workpaces and they flow their changes to the Hyperbola stream. The unit of change is a change set, which provides atomic grouping of changes.
Figure: Overview of the basic source control model
Goal 2 : Stop talking about branching, and just do it!
There is a saying that “people talk more about ‘it’ than they do ‘it'”. Well, this can also be said about branching; people talk about it, plan it, but just do not do it enough. The problem is that branch functionality has often been implemented and presented to the user in ways that make branching seem scary. Instead of leaking implementation details about branching, we wanted to experiment with taking the verbs people use in their day to day conversations, and turn those verbs into gestures that you can do in the version control system. I’ve banned the word branch from our UIs in exchange for finding user terms and gestures for tasks that users had traditionally had to use a branch for.
Of course making it easy to have isolation (e.g branching) when needed also means that conflict resolution has to be made a lot easier. For this reason we’ve tried to improve conflict handling in three main areas:
- Store conflicts with the repository workspace so that you can resolve conflicts at your own pace. You could even give your repository to someone else to continue with the merge. Basically, avoiding the merge process of being transient of something that goes away when you accidentally close your session.
- Structural conflicts have to be handled in a first class way, such as moves and renames.
- Allow previews of conflict situations so that you can decide to backout or rebase before accepting changes that would cause merging.
Branching all the time without knowing it
Everything is a change set in Jazz Source Control. It’s the unit of work, and we allow operations on these change sets that feel natural such as suspending and resuming them. If you have something else to do, just suspend the changes and we will track them for you — at any time you can resume them. You can suspend as many times as you want. Of course, conflicts can occur because parallel work is happening, but conflict handling across move, renames, edits is supported and tooled in a first class way.
This model supports several levels of parallel development as a first class concept. You have the following tools available:
- Repository workspaces: Provide constant isolation. You don’t have to make your changes visible to the team just to back up or use the repository features. It also allows accepting from team members’ repository workspaces or delivering to your other repository workspaces.
- Suspend and Resume: Provide task level isolation for personal work and a very easy mechanism to put changes on the side.
- Work Item links: Provide light-weight task-level isolation for personal or team work. Work on a feature, attach to a work item, and discard from your workspace. You or someone else continues the work by accepting the change sets back into their repository workspace.
- Streams: Provide team or feature isolation.
Goal 3 : It’s never too simple
Even with all these cools features, such as repository workspaces, conflict handling, suspend, and resume, there is a class of users that just don’t want to deal with this perceived complexity. Open source tools really dropped the ball on these users, but I really like the workflows that they make you consider. First, don’t make me merge, ever, unless I opt into it. Secondly, don’t make me deal with the distinction of having user level sandboxes and an extra set of operations to share my changes with the team. This is the classic single line of development with pessimistic locking pattern, which although some may consider un-collaborative, it has its usefulness.
We starting supporting some of these simplified workflows in 2.0.0.2. Without modifying the underlying change model, we allow users to edit files directly in a stream without needing to load or create a file area. You can lock and unlock files and make changes in a matter of seconds. This makes it very useful for making quick changes and allowing non-developers without an IDE to do the same.
Figure: Editing a file directly in the web. The file is locked and a change set is created, but no repository workspace is required
For locking, because we share a lot of the same properties as Git (such as highly distributed changes), it’s hard to define what exactly gets locked when there are many repository workspaces, streams, and possibly clones of the history on other machines. We opted for a simple model that allows users to lock files along a set of streams. They decide which stream hierarchy they deliver to, and lock there only. This means others can continue to fork those files at their own leisure, but if they deliver to the same stream they will get warned and be blocked from delivering until the lock is released.
Figure: Locking a file in a stream or along a stream hierarchy.
The next steps which we haven’t tooled just yet, but for which we have backend support, is allowing developers in their IDEs to work directly off a stream in a traditional “pessimistic” model. We would like to enhance this traditional model and allow users to still attach a change set to a work item or find out about what files other have locked. Then they can transition into the other workflows on demand.
Goal 4 : Enable distributed workflows
Sharing changes shouldn’t be constrained by the physical location of the repository workspaces or streams. Allowing distributed source control is about enabling the design goals discussed previously with the same user interface and gestures as in the single repository model. Users could opt in to distributed based on their corporate policies and collaboration needs with other groups or departments.
This also brings up another requirement for corporate-wide repositories: we didn’t want to force users to clone an entire repository for distributed, but instead just the components they need. Jazz Source Control allows partitioning into self-contained groups of changes we called components. In a repository with thousands of components, you can choose to take a small subset onto another server.
There are also additional considerations for a distributed model in ALM tools. For example, linking between artifacts, such as change sets or baselines and work items, is important. Once a change set has flowed to a new repository, the process rules that govern the integration of these change sets into streams require cross repository artifact linking.
Figure: New repository workspaces can be created in other repositories
Figure: The Pending Changes view showing a flow target to another repository
Conclusion
We believe that our model enables you to benefit from version control and adopt your usage based on your skill level and requirements for central versus distributed configurations. In addition, the tooling of source control in the context of the other roles in a software project will have significant productivity gains over the more narrowly scoped source control tools, which design integrations as an after thought.
There are great articles that explain the ideas behind other distributed source control tools, such as the one from Better Explained and another from Joel Spolsky. After you’ve read our design guide and read what others have to say about designs of version control tools, the key differentiators to keep in mind are:
- Our model allows everyone to have a sandbox (e.g. repository workspaces) with the advantages of a central server and without having to replicate an entire repository history to do so.
- Our model makes branching easy with new ways of sharing changes via peer-to-peer exchange between sandboxes and work items.
- Our model makes some of the basics such as backing out changes, fixing goofs, and merging a lot simpler. For example, in Joel’s tutorial on Mercurial there are too many steps to perform daily tasks. An example is our support for suspending and resuming, or backing out changes by moving into other change sets or rebasing.
- Our model takes version control and enhances its use by rich integration into the other ALM tools such as build and work items.
- Our model scales using components instead of more individual repositories.
- Our model considers locking as something that still makes sense for binary files, it should still be supported in modern source control systems.
- Our model keeps things simple and supports users who may not need as much flexibility with easy ways of collaborating.
- Overall reading guide for Team Concert
- Jazz Source Control Tutorial
- Understanding Conflicts
- Merging by example
- Multi-stream development
- Patterns for read-protecting source code
- How scalable are repository workspaces?
- Finding your changes – using releases, builds, and snapshots
- Managing vendor and third-party code
- Advanced source control workflows
- Flow changes cross repositories with Rational Team Concert
Copyright © 2011, 2012 IBM Corporation