From silos to agility: Rethinking enterprise software delivery with the Rational solution for Collaborative Lifecycle Management
Table of contents
Discordant enterprise software development
The crescendo of Jazz
Composing and delivering on multiple systems
Teams with different tunes
Harmonizing with Jazz Collaborative Lifecycle Management (CLM)
The chords of Jazz CLM
Jazz CLM sets the tone from business to production
When Rational Requirements Composer, Team Concert and Quality Manager take the stage
Benefits for enterprise software delivery
Discordant enterprise software development
As enterprises adapt to evolving marketplaces, the evolution of business applications has had an ancillary effect of giving rise to disconnected or loosely linked teams. More often teams are organized around run times, roles and technology, data, or architecture, but not around the people who make up the team. The development of business applications has never been easy, but projects should not fail because of a lack of visible progress or a vast array of technology and development styles that are too expansive to contain.
Leveraging the work done by Grady Booch and Alan Brown on Collaborative Development Environments1 as the next step to Integrated Development Environments (IDEs) combined with the deep experience from the Eclipse development which was one of the first project distributed on a large scale but delivering on time, IBM started the IBM Jazz project to merge evolving business requirements with the latest technological advances in social networking. In 2006, a small group of experienced IBM Power and IBM System z® tool developers formed an initial work group. The group’s goal was to explore ways in which Jazz technology could add value to IBM enterprise tools and provide a framework for organizing teams around people, artifacts and processes. The first results were soon presented and became what was then named Rational Team Concert for System z and Rational Team Concert for Power systems, giving two dedicated flavors to the Rational Team Concert product, based on Jazz technology. As the work on integration and scaling up continued, all runtime flavors and the teams building them were combined to create a complete Application Lifecycle Management tool to accelerate the delivery of enterprise software across multiple platforms.
The crescendo of Jazz
Business application development began on the mainframe. Developers built applications while sharing a common system where they could directly edit, compile and deploy code. With the common system as the center for business application development, project development workflows became the first collaborative development environment.
Soon, new sets of development tools based on new languages emerged to help create PC applications. While developers’ productivity increased, so did system complexity, since developers had to import and export artifacts between tools, hindering teamwork and collaboration.
In early 2000, a new form of integrated development environment (IDE) changed all that: the multi–language IDE. An extensible platform for building development tools, Eclipse led the multi–language IDE pack as an open source project available to the entire development community. The broadband boom helped Eclipse gain popularity through the Internet and increased two–way collaboration using downloads and uploads. Soon, a number of new collaborative tools proliferated— including instant messaging, Web conferencing, file sharing, wikis and blogs–making teamwork easier and accelerating the software delivery cycle. These tools also pushed new development trends and changed how the IT industry and developers work today. Development teams that used to work at a single location are now collaborating as smaller teams that are geographically distributed around the globe. This trend also hit the Jazz development group, which is composed of small functional teams, distributed all around the world, spanning all continents.
The Jazz development group has carefully monitored and lived through these shifts, making the most of the new business orientations. In response, IBM is building an environment that can meet the needs of the individual, lift the constraints on team productivity and provide a complete view on what is going on for all phases and aspects of the software lifecycle delivery. With the Jazz Collaborative Lifecycle Management platform, stakeholders as well as IT participants can leverage teamwork and interactive concepts to define requirements, design, test and deliver products within a pervasive collaborative development environment to speed reactivity and provide business agility.
Composing and delivering on multiple systems
Although many people are unfamiliar with mainframe and midrange systems, we all rely on them every day2. Do you know what’s happening behind the scenes when you withdraw cash from an ATM, buy gasoline for your vehicle, or await your monthly paycheck? A large majority of these transactions are computing without interruption on IBM Power and System z servers all over the world, every day.
Over the years, developers have added more applications and more power to these systems to accommodate growing business demands. Such additions have resulted in a huge variety of artifacts that were originally written to run on the first computers but today coexist with the latest technologies–such as Java Platform, Enterprise Edition (Java EE) applications, asynchronous JavaScript and XML (AJAX) assets and Web 2.0 artifacts. A close examination of artifacts on a given system can show us how complex dependencies have been built between software which spans the entire IT history of a company.
The lifetime of an application running on an IBM Power or System z system can be between 10 and 40 years. Your monthly payroll system may have been built on programs that were written before you were even born. This can present real maintenance challenges in terms of finding the history behind an application and bringing new employees into the workforce.
If we look at the payroll example—the application that may have been coded before you were born—it is likely that the requirements for your locale have evolved since your payroll application was first coded, compiled and deployed by your company. It is also likely that your company experienced structural or IT infrastructural changes. All the changes that occurred over time inevitably had an impact on the application’s original source code, which means your company ended up with thousands of interrelated resources that all needed to be managed simultaneously. This endeavor is monumental in itself.
But to make everyone’s work even more complex, the applications we have to work with have evolved over time. Business analysts had to juggle with constantly changing requirements, which in turn have an impact on architects and developers who have to consolidate the links to existing or as yet non—existent architecture or pieces of code. And even when developers are using the same language, they encounter a plethora of code organizations in which current and former developers have kept their artifacts according to need or design. The huge variety makes it difficult to find artifacts, identify the right relations and maintain them.
Another problem developers have to face is a variety of data organizations and file systems. When partitioning large projects into smaller components according to people, geography or development platforms, project managers have to coordinate the work with teams and team members. If all of the problems mentioned above exist in a small development context, they also exist in an enterprise development context—but they’re bigger.
Teams with different tunes
When it comes to building and running business applications, there is always the challenge of matching the right people to the right work. In some cases there is a division between people with business knowledge and people with technical expertise. For development, further divisions frequently occur around the system structure.
Figure 1 Siloed teams
Although companies vary dramatically according to size, culture, history and other criteria, IBM has identified four primary organizational tendencies that affect the way development teams are organized (or disorganized, in some cases).
Organizing around run time
Despite the emergence of new ways to limit friction between run times —like using model–driven architecture, platform–agnostic languages like Java, or business–oriented languages like Enterprise Generation Language (EGL)–application developers continue to be grouped according to the systems they build on. Education and experience naturally link people together, but creating teams based on these commonalities can make it difficult to adapt to emerging technologies.
Organizing around roles and technology
Whether the role is business analysis, development, quality assurance, building and deployment, or a combination of all four, the tools, languages and platforms that have to be mastered can be very different and are often handled by separate teams, sometimes even in different locations. This leads to a lot of paperwork and uncertainty to carry on the various workflows between the participants.
Organizing around data
In cases where data has evolved over time—such as from a traditional System z file system Partitioned Data Set to an advanced relational database—the way to access the data, and even the access routines, is usually handled by another individual or team that is strictly dedicated to data with very few or no links to the business.
Organizing around architecture
Stacks of applications, using different technologies and running on different platforms, need to be constantly verified and maintained. To manage verification and maintenance, teams are often organized around architecture and their modernization approaches.
Harmonizing with Jazz Collaborative Lifecycle Management (CLM)
IBM Power and System z platforms are extremely powerful and can efficiently and reliably support large and diverse development organizations. Nonetheless, these platforms can present challenges. Jazz technology addresses these challenges from within the tools used by team members involved in enterprise application lifecycles.
The intent of this paper is not to give an in–depth view of Jazz technology itself, but rather to reveal how this technology can help overcome the challenges of enterprise application development. Because software delivery is complex and difficult, successful enterprise application development requires that the people, the artifacts and the processes work together, in harmony.
Figure 2 Connecting analysts, architects, developers and testers
With the technology provided by Jazz CLM, all groups mentioned previously are now able to coordinate their work at all stages of the software delivery lifecycle and face the many challenges enterprises have to deliver; deliver on time, deliver quality and deliver what really makes sense to the business. By offering an extensible framework that dynamically integrates and synchronizes people, artifacts and processes, Jazz CLM can create cohesion and facilitate improved and accelerated software development projects.
In developing the Jazz platform and integrated products, we first investigated the entire enterprise development stack in the IBM Rational Software Delivery Platform to determine how this technology could add value. This investigation involved digging inside some of IBM’s most important development tools, such as IBM Rational Business Developer, IBM Rational Application Developer, IBM Rational Developer for System z and IBM Rational Developer for IBM Power software. We also investigated tools used by a wider range of developers.
Next, the team considered the actors themselves, whether they are business analysts, project managers, developers, testers or managing deployed applications. A new technology would need to help maintain individual developers’ productivity while still supporting their team roles and their collaboration with dispersed team members. It would also have to support the way all participants in the software delivery lifecycle work today but still provide a framework that would allow them to share, adapt and enhance their unique processes over time.
During recent years, the Jazz development group, as it expanded, lived and explored some of the current challenges faced by development teams and sought to solve them. To understand some of the challenges, consider again our monthly payroll system example.
The simplified system, presented in Figure 3, contains three main components: one to deal with updating employee absences, another to calculate actual payroll, and the third to handle legal declarations. Let’s say that the first component—the one that updates employee absences—was recently developed using a Java EE Web application. This application runs on distributed servers that access back–end services created in EGL with data stored in an IBM DB2® relational database that employees can access to edit and display their absences. Absence–related data and inputs from other sources are then used by the payroll calculation process that is run on the System z platform and launched monthly by a Job Control Language (JCL) batch program that invokes COBOL code. At the end of this process, paychecks are printed and some payroll information is collected and provided to a midrange IBM Power system application in charge of collecting tax declarations.
Figure 2 Simplified payroll system
The chords of Jazz CLM
With that development context in mind, consider what happens in a typical day for the team in charge. After receiving a request from one of the application business analysts to meet new legal requirements and therefore change an absence–related code, the development lead or architect has to first identify the affected parts of the system and the different teams responsible for them. After a few verbal or written exchanges with its originator to clearly understand the requirements, he or she can then use, in best cases an impact analysis tool like IBM Rational Asset Analyzer (RAA) software to discover the potential effects. Then, according to the results, he or she informs the teams in charge of the affected components that those assets should be updated, built, tested and deployed to production by a given date.
This kind of complicated structure for delivering software is vulnerable to a number of errors, most of them human–related. On top of that, the coordination of the work being done can be another source of problems. Even assuming there is only one team working on the Web application with a team leader who will distribute the work, the task of deleting the absence–related code from the Web application can still involve a large number of team members. And to make their collaboration even more difficult, team members might use different development tools, different bug tracking systems and different code repositories. This lack of standardization and coordinated planning among the teams requires that all of the participants be extra vigilant and follow the different aspects of the work carefully.
After successfully changing the absence–related code from the Web application, team members must update the rest of the payroll system to align with the new business requirement. There are no direct dependences between these two components, so both operations could be done concurrently, but does anyone on the team know about the changes? Again, lack of transparency means knowledge of changes is uncertain.
Changes can continue in the same erratic way for a long time until someone tries to build and test the newly updated payroll system. This is when problems that were lying dormant in the system might be discovered. Many types of problems could arise: from a record-length definition incompatibility between a COBOL program and a JCL calling the program, to any other integration problem within the entire system.
Jazz CLM sets the tone from business to production
Enterprise applications have to handle many different artifacts, and the tools provided to developers must operate efficiently and transparently. Handling enterprise development styles means that Jazz technology– based products like IBM Rational Team Concert software can store traditional artifacts (such as COBOL, RPG, PL/I and High Level Assembler (HLASM) programs, and JCL and Control Language (CL) batch scripts) as well as more recent artifacts (such as EGL programs; HTML, JavaServer Pages (JSP) or JavaServer Faces (JSF) Web pages; Web Services Definition Language (WSDL) or any Java EE artifacts) in one place and share them between teams and team members. RTC can handle all of these formats. And Rational Team Concert can store these artifacts in a way that is independent of the development tool being used.
This aspect of enterprise development also helps build applications. Rational Team Concert enables developers to build applications that execute on multiple platforms, including IBM Power or System z platforms, independent of where the Rational Team Concert server resides.
In the IBM Power and System z contexts, IT people have established elaborate, consolidated processes over the years to support software configuration management (SCM). To accommodate these processes and extend the capabilities of the team, Rational Team Concert features integrated work items, developer builds and SCM tools that think and work in unison. The SCM component of the Rational Team Concert Developer for IBM Enterprise Platforms allows artifacts to be developed locally in the development environment client using IBM Rational Developer for System z or Power software, or it allows artifacts to be developed remotely on the enterprise file system using the client for management only. The SCM component can also be used during all phases of the artifact lifecycle, including editing, testing, building and debugging in a cross-platform development context.
Providing the base technology and infrastructure to define processaware operations, Jazz technology can be easily extended to fulfill the needs of an enterprise development environment. New components can be added to RTC deployments to provide enterprise–specific work items, such as tasks, defects, enhancements and plan items. Each can have specific processes defined for working with these work item types. Enterprise stakeholders can create work items using the technological entry forms and templates. Consolidated information and reports enable users to get a more transparent view of complex IT systems and their development projects. They can also have a clear view on where the various projects stand, whether being a maintenance project or a new development, identify the potential risks and ensure quality and traceability aspects.
As enterprise development organizations create artifacts, they generate large volumes of documents detailing which artifacts are being promoted, which are being deployed and which are being removed from production. These documents casually flow between teams, but their information is critical, particularly when it needs to be used to address security or legal compliance regulations. Having a clear understanding of the current state of any of these documents is usually like looking for a pin in a haystack as no integrated approach is provided to coordinate team plans, activities or artifacts.
The solution offered by Jazz CLM is to provide such an integrated view to all the participants of the development lifecycle, from business requirements elicitation to driving the creation and testing of applications, through process automation, traceability and reporting.
When Rational Requirements Composer, Team Concert and Quality Manager take the stage
Let’s look back at our payroll example again. The work could be simplified by using a new set of tools based on Jazz technology and designed for enterprise development. First, the development lead starts with the requirement for removing an absence–related code. In spite of the fact that it is a simple requirement, there could be any number of reasons, such as for legal compliance, why the company would need to carefully track its elicitation and it would be inappropriate or even dangerous to make the request through an informal, untracked communication method such as e–mail or instant messaging.Instead, using Rational Requirements Composer, the requirement details can be fully explained in the context of the request. The request details can include related documents provided by business experts along with the deadline for when the new system needs to be operating in production. Integrated planning can then ease the process of tracking the status of this requirement throughout the complete cycle of development and ensure any required artifacts to complete the work do not get stuck in areas of the overall IT group, including those assigned to external contractors.
When the requirement is ready to be passed along to development for implementation, a number of strategies can come into play; direct tools integration such as what is provided between Requirements Composer and Team Concert or light coupling when an intermediate tool is needed to model architecture with Rational Software Architect for example. Whatever path is chosen, all tool participants collaborate and link the development artifacts that the development teams rely on at all stages of the development lifecycle. This is done using the Open Lifecycle Collaboration Services for which the Jazz CLM solution provides the structure.
Conducting the development implementation for all enterprise languages and platforms using Rational Team Concert Developer for IBM Enterprise Platforms, the development lead creates one or more work items directly within its client that could involve enterprise specific tooling like Rational Developer for zEnterprise, like in Figure 4 or just a simple Web browser.
This convenient way to interact with the Jazz CLM–based products and get the information the users need to drive their work is provided in the environment they are most comfortable and productive with, whether being browser–based, based on Eclipse or Visual Studio, a command line, programming interfaces or even an ISPF client. This helps keep people “on task” and “engaged” in their work.
Figure 4 Rational Developer for zEnterprise based on RTC
While using the same tool, the development lead asks another team member to provide more details about the payroll system components that would be affected by the changes. The development lead then reassigns the associated work items to the teams responsible for the various updates.
Via enterprise development tools based on Rational Team Concert, email notifications or RSS feeds inform the teams of their newly assigned work items and of updates to existing items.
The developer or team leader in charge of EGL code learns of the new task and reviews the team workload displayed on the team central view. He or she can then reassign a work item to another developer. By visualizing and understanding the entire context of the assigned task, the EGL developer can quickly start working on it or, according to what has already been assigned in the integrated planning, or might choose to place the task in the backlog or refer it to a later iteration. Upon reading the task description, the developer has access to references–presented through hyperlinks—to the EGL program, to related work items, which may or may not be blocked by this and, as an example, to records that need to be updated.
Using these hyperlinks to load and open the EGL editor on the program, the developer can now begin work. From this perspective, the EGL developer is in a familiar environment where content assistance and other code productivity tools can help him or her to complete the task. And after clearly documenting the EGL code update performed in the task editor, the developer changes the work item status to resolved to reflect its completion.
Now that it is associated to the task, the EGL code update waits to be delivered to the other members of the EGL team. It is presented in the pending changes view until the developer requests delivery.
During delivery of the pending update, the process mechanism in RTC applicable to the EGL development team at this stage of the project is invoked automatically. The rules and controls defined are processed against the code being delivered. Through the automated adviser, the control settings can warn the developer that one of the programs is incorrectly named and should be changed. This can save the development team from having to later correct the naming and repeat steps.
The newly updated system is about to be integrated into one cohesive unit; it will then be compiled and tested before being deployed to the requested systems on which the components will run. To simplify this highly complex process, the construction of a heterogeneous and cross–platform system is directed by a single conductor, the Jazz CLM solution using multiple groups of instruments.
Scheduled or manually requested, the integration build performed by Rational Team Concert in our payroll example takes all the updated component code from a single stream, provides a coherent functional map of the entire payroll system, and lists the programming languages used (COBOL, EGL and JSP). The affected applications are recompiled and unit tested, and, before being deployed to IBM Power systems, a System z or a distributed system, the build results are automatically delivered to the subscribed contributors. During this operation, snapshots of the current states are made for later use, such as for code maintenance or recovery.
At this stage of the development cycle, when all teams involved have successfully delivered and built all the parts that need to be updated to fulfill the requirement, the multiple components need to be submitted for verification. Although testing came earlier on in development, there is one important stage in the cycle when this is done on a complete update, after promoting all pieces of code and deploying the new system to QA (Quality Assurance) to monitor, evaluate and validate it before replacing the version currently running in production. Rational Quality Manager offers the ability to model verification plans and test execution.
Tracking the execution of tests among a single team is relatively easy, but when it comes to coordinating tests within a complex enterprise system using multiple runtimes and based on what is provided by various and heterogeneous teams, things quickly start to become difficult and part of the system can easily fall off the radar. This is when the linking of tests to requirements comes into play in order to reduce the risk of failing to identify all the tests to be conducted to ensure the quality of the system before it can replace the one in production. These links go both ways as they can also serve the business analyst to identify which requirements are invalidated in the system because of failing tests. This transparency between QA, business analysis and code development can open the door to new development processes, like temporarily removing business functions or features based on the exact code updates being faulty, simply by looking at the failing tests and related artifacts.
The various combinations of the links provided between the products are an example of using the Open Services for Lifecycle Collaboration (OSLC) interfaces in the Jazz CLM solution, as shown in Figure 5. These links and the combination of information can help the teams find numerous new ways to quickly react to change and provide more business agility.
Figure 5 Linked Lifecycle Data (OSLC)
OSLC is a community, opened to industry participants and set of specifications for linked lifecycle data. The goal of this community effort is to define open, public descriptions of resources, formats and RESTful services and interfaces for sharing the things that teams rely on, like change requests, test cases, defects, requirements and models to help product and software delivery teams by making it easier to use lifecycle tools in combination.
For all the teams involved in this update, the work is almost finished. The development lead and also the business analyst originator of this development, who subscribed to the different work items involved in this project, are notified of every change and can at any moment reassess and reorganize work. They can also create custom queries to watch project progress. These can be displayed on user–specific dashboards. Jazz CLM solution using Rational Requirements Composer, Team Concert and Quality Manager, through an Eclipse client or a browser–based interface, provides a set of tools that raise the visibility of the overall project—from simple notifications to highly specialized and mixed data dashboards and reports. The most utilized feature is the iteration plan, where all contributors can see the exact status of the work for an entire project based on live data of the work being currently done.
Benefits for enterprise software delivery
By leveraging the Jazz CLM solution with Rational Requirements Composer, Team Concert and Quality Manager combined inside the enterprise space, all development participants can collaborate; share and exchange data with their colleagues to ease the software delivery process; control what they deliver, get early feedback on what they plan to deliver, ensure quality of the business system they’re building and facilitate reaction to change.
The Jazz CLM solution can help consolidate multiplatform development, including IBM Power and System z technology as well as distributed platforms, by improving IT governance, reducing development and maintenance costs, and enhancing the quality of delivered software. Moreover, Rational Requirements Composer, Team Concert and Quality Manager with IBM Power and System z features can be used in combination out of the box, tuned to fit a given enterprise development style, governance and IT infrastructure without breaking current operating procedures but helping teams to deliver high quality enterprise software.
By using the Jazz CLM solution to manage and share all development artifacts in a common and transparent way between teams, team members and stakeholders, it is possible to more effectively leverage the strength and scalability of midrange and mainframe development architecture, regardless of staff location, size, or time zone. These platforms offer robust quality of service to help make work more effective with multidisciplinary teams and business partners.
As Jazz technology–based integrated products like RTC, RRC, and RQM are deployed for enterprise environments, application development in large organizations has the potential to become streamlined and transparent, and software delivery can be made more predictable than ever before.
For more information
To learn more about IBM Rational Requirements Composer, Rational Team Concert and Rational Quality Manager software, contact your IBM representative or IBM Business Partner, or visit: ibm.com
To learn more about IBM Jazz technology, visit: https://jazz.net
To learn more about IBM Enterprise Modernization solutions, visit: http://www-01.ibm.com/software/rational/solutions/em/
1 A.W. Brown and G. Booch, Collaborative Development Environments, Chapter 1 Advances in Computers Vol. 53, Accademic Press, June 2003.
2 60-80% of corporate data currently resides on mainframes — Estimates provided by the Meta Group.
About the author
Jean–Yves Rigolet is a software engineer in the IBM Software Group, working in the Rational Division. With 20 years of application and development tools shipping experience in a variety of programming environments and platforms, including the IBM Telecom Web Services Toolkit, VisualAge Pacbase and the VisualAge Generator products, he is currently based in the IBM France Software Laboratory in Paris, where he is Scrum Master on Rational Team Concert. Jean–Yves is also the author of various articles and papers on enterprise development and agile, which have led him to present the Enterprise Modernization collaborative strategy and agile adoption at various conferences and events worldwide.
Prior to joining Rational, Jean–Yves spent several months launching a Workplace and Eclipse practice as part of the Software Group Lab Services initiative and helped customers by building a workshop which he facilitated around Europe.
Prior to IBM, Jean–Yves acquired solid experience on enterprise applications during the first few years of his career through building business applications for CGI Informatique (acquired by IBM when he joined the labs).
© Copyright 2011 IBM Corporation
Yes