Since the release of CLM 5.0, CLM (now ELM) has used an approach to deliver maintenance to customers via a regular, light weight, and cumulative iFix.

The ELM iFix addresses a couple major pain points with maintenance for ELM customers:

• high cost of adoption of full maintenance releases,
• manual and error prone installation of test fixes,
• difficulty tracking and managing fixes running on the system,
• difficulty with rollback, and
• prone to regressions when upgrading to a newer ELM main release.

As a result of this approach to maintenance, the ELM iFix Release team has been able to bring high quality, frequent, easy to adopt iFixes to our customers. Consequently, the development cost for implementing, testing, and delivering APAR fixes has been dramatically reduced:

• from minimal 3 to 6 months for full maintenance release cycles, down to maximum 5 weeks.
• Acceptance test has been reduced from about 2 weeks to daily by the test automation pipelines.
• PSIRT security fixes were reduced from 3 or 6 months via full maintenance releases, down to within 5 weeks.
• Release Engineering team is supporting iFix streams with about 1/3 of the cost of supporting streams to produce full releases.

This blog focuses on highlighting the transformations for both support and development teams as well as discussing the benefits that customers will see from this new maintenance of ELM products.

Continuous Collaboration and Feedback with Support

The support organization plays a vital role in the maintenance of any software solution. Transparency of the iFix review process as well as up to date plans and status are very important to the support organization. The ELM iFix Release team uses several methods to track such transparency:

1. Automatic planning of all Sev1/Sev2 APARs to be fixed for the release in which it was Found In (if possible) and ported forward. Sev3/Sev4 APARs will be investigated for the next main release as governed by our quality metrics for that release.
2. Maintenance Items to track all expected APAR deliveries. Governance, tracking and collaboration available at your fingertips.
3. Plan Items are used for current and future iFix plans.
4. Internal dashboard for up-to-date status and estimations.
5. Fix Central is the one stop shop for the most current iFixes. Anyone can search, download, and communicate with customers about all the high quality cumulative iFixes available.
   Note that the most current iFixes are also available for download from jazz.net.
6. Each iFix comes with a readme.txt that clearly lists all APAR fixes included in the iFix; with APAR IDs, description of the defect, and link to the Maintenance Item that implemented the fix.

Transformations Inside Development

1. Continuous deployment – the iFix patch deployment solution made continuous adoption of maintenance possible. Jazz Server has included a capability called Patch Service which is able to detect one single, small patch file and re-provision the entire server with the patch at server startup time. The availability of the Patch Service dramatically reduced the cost of adopting new patch files, thus making constant verification and testing possible.
2. Continuous patch file generation
   Continuous testing and delivery of an iFix would not be possible without the right content – a constantly available ELM patch file.  In order to provide this, the ELM iFix Release team utilizes an automation task as part of the build process. As a result, a patch file that contains all fixes delivered to the maintenance stream since GA is continuously produced from each nightly build.
3. User documentation automation
   The authoring of the majority of readme.txt content for each iFix is automated. Automation was introduced to generate a readme.txt file out of each nightly build. Accurate, continuous, and a huge saver of man hours.
4. Continuous testing
   The low cost of deployment and continuous patch file availability make it possible to leverage all of the automated tests from the automation pipeline with very few modifications. Adoption of automation pipeline for ELM maintenance dramatically reduced the high cost of manual testing, increased ROI from continuous testing, and provides the ability to inherit further test automation improvements.
5. Continuous shared test server update
   Even for manual testing, automation exists to continuously update shared test servers after each nightly build. New test automation on patch file content validation was also added as part of this pipeline.
6. PSIRT security fix – cheaper, quicker, better
   PSIRT security fix delivery changed to a much lower cost model via self-serve JRE and Liberty updates plus iFix if needed, instead of delivering through full maintenance releases. Test automation was leveraged again for PSIRT security fixes against maintenance as well.
   IBM Security Vulnerability Management

Customer Perspectives

What is the advantage of adopting iFixes? What does the ELM maintenance approach mean to me? What are the future considerations system administrators and process owners should have and what does IBM recommend?

These are questions we’ve heard from customers since we introduced the iFix Process for ELM maintenance. We hope this blog will help explain why the new ELM maintenance approach is the right one for you!

Deployment – a VERY EASY adoption

Prior to the introduction of the iFix Process, customers would have adopted APAR fixes in one of two ways. One way was to install a temporary hot fix in the form of one or more replacement plugin jars. This was a manual, error prone process carried out by the customer which included the risk of overwriting previous fixes as one or more files were directly replaced in the deployment directory. A second way was to install a new full release of the ELM products (for example, 7.0.2 SR1), which required a side-by-side installation and migration which can be expensive.

In comparison, the ELM iFix delivers all server-side APAR fixes in the form of a single patch file, with the following characteristics:

• One file to patch the entire server
• Minimal in size. For example, a server patch file containing hundreds of APAR fixes is typically 50-75MB
• Very simple adoption. Simply add one patch file to a pre-defined directory. Just follow instructions in readme.txt
• Uninstall is as simple as install (again, follow instructions in readme.txt).
• No modification needed to existing installation — that means, no touching any existing files
• Cumulative … which means you only need the most current iFix at any time, as it always includes all APAR fixes available since GA.

The following images illustrate differences in the deployments between a temporary hot fix, side-by-side install and the iFix.

Temporary hot fix – too many installation files are impacted. It’s also worth noting that the file names must remain the same in order to work. That is, you cannot visibly tell from the file name if it is the original GA file or one that has been patched or for what reasons it was patched.

Full replacement (side-by-side) release – can be expensive to deploy.

The iFix is a single patch file with all fixes for the entire server!

Improved Tracking of APARs

All APARs included in the iFix can be tracked easily. You know exactly what’s running on your system. Simply refer to the readme.txt that comes with each iFix for the list of APARs, with:

• APAR ID
• Work item used to deliver to the iFix stream
• URL of this work item, if you need more details

Upgrade Path Clarified

No more wondering about “what is the next safe release to upgrade to?” after adopting the iFix!

All APAR fixes provided via an iFix are forward ported to more recent releases, however there are cases where releases may have become less preferred or gone out of support. As an example, a fix for 6.0.6.1 would be forward ported to the 7.x releases such as 7.0.1 and 7.0.2 but may not have been provided for 7.0 due to having gone out of service.
There may also be a delay due to the timing of the release schedules involved. For example, a fix provided in a 7.0.2 iFix may have been too late to include in the 7.0.3 GA release, but it will be provided in the earliest possible 7.0.3 iFix and the 7.0.4 GA release.

So how can I be sure in which release any particular APAR fix is available?

The ELM iFix Process provides traceability through linking. An APAR will always include a link to the jazz.net work item which serves as the base reported Defect (confirmed by the APAR Number field in the work item). That base Defect will be used to deliver the fix to the next possible main release (ie, 7.0.3 – see the Planned For field once the Defect is Resolved/Closed). From that base Defect, there will be Tracks links to any maintenance items that were used to deliver that fix to prior releases via an iFix. Again, check the Planned For field of the maintenance item once it is Resolved/Closed to determine precisely which iFix first contained the fix. It is actually most common that fixes will be available for prior releases well ahead of the next GA release. And as stated earlier, from there the iFixes are cumulative.

Considerations and Recommendations

With all the above features of ELM Maintenance, we hope you consider the following to take the best advantage of what IBM has to offer:

• Adopt the latest ELM iFix available to get the best stability without picking up new functionality.
• Plan ahead for your next ELM upgrade. Take advantage of the upgrade path recommendation from iFix. The best mod release for you may be the next one coming up!
• Contact IBM Support for the latest iFix available for your ELM version or just check out the Downloads section on jazz.net.
• In general, for new functionality and maintenance, stay current with the latest mod release. ELM 7.0.2 SR1 is the current recommended release.

Additional Reference Material

Cases and APARs – Understanding the IBM Engineering Lifecycle Management Support Process

Your comments and feedback are greatly appreciated so please don’t hesitate to drop a line or two with your stories and experiences to make this interactive! We’d love to hear your experience as it relates to ELM maintenance.