r29 - 2017-07-05 - 10:28:19 - PaulEllisYou are here: TWiki >  Deployment Web > DeploymentInstallingUpgradingAndMigrating > UpgradePlanning

updated.pngUpgrade planning

Author: PaulEllis, BenjaminSilverman
Build basis: None. Release agnostic.

This article provides you with a list of actions you should be taking when planning an upgrade. It is intended to help you think through the upgrade process without getting lost in the execution details.

The following advice has been adapted from a presentation at InterConnect 2015, 2016 and 2017 and is intended to be release agnostic. However, where links are release specific and pertinent then the links are for the latest 6.0.x release.

The 3 core aspects to an upgrade

3coreaspects.JPG

Over the past number of years the IBM teams, that focus on Collaborative Lifecycle Management(CLM) upgrades, have realized that there are essentially 3 core elements to completing a successful upgrade. The technical aspects of an upgrade, whilst important, are certainly not the most critical.
This article details what lies behind the 3 core aspects of an upgrade, from planning through to execution.

Planning Planning.jpg

"What considerations should be in my plan?":

checklist1.JPG

People

There are essentially three sets of people who are involved during the upgrade lifecyle. The Users and the various stakeholders are the most obvious. However, as an upgrade is so much more than the technical steps on the day/weekend, utilizing project management techniques is strongly advised for larger deployments.
Also consider which IT teams need to be involved during the upgrade, who owns each of the systems, who owns each operation
  • application server admin
  • database admin
  • networking admin
  • don’t forget the training aspect

Documentation

The most important document is the Interactive Upgrade Guide (IUG), where you can generate instructions tailored to your configuration. The IUG asks that you specify your configuration, operating system, application(s) to be upgraded, data warehouse requirements, integrations, etc and uses this input to generate detailed instructions and an upgrade checklist specifically tailored for your configuration.
Also ensure you utilize:

The support portal and jazz.net sizings are generally used as a basis for sizing based on the typical requirements stated. Plan to have the available capability to expand and explore past the minimum requirements if possible. When sizing, consider your future expansion needs and most stressful usage scenarios to determine the needed capacity and topology.

Timings

Ensure that there is plenty of contingency within an upgrade plan. Add a time buffer to each major operation to allow for unforeseen problems.
Depending on your data, some of the upgrade processes can take quite a bit of time and if a problem is discovered, the upgrade may need to be restarted.

  • The best upgrade plans, will state the hand-off between operations and how these steps will be validated
  • Plan for your applications to be unavailable
  • Someone must manage the various parties and the overall progress
  • Ultimately, if the upgrade does not go to plan, there must be a focal point that determines if the upgrade is cancelled or whether there is sufficient contingency not to roll-back


checklist2.JPG

Features

New features help support your business case for upgrade. There are several pages on jazz.net that are designed to assist you with understanding the release you're looking to adopt.

  • What is new in CLM
  • The Overview tab in the product/solution downloads page provides a summary of the main features along with video demonstrations.
  • For further details, look at the New and Noteworthy section on the product downloads page
  • Release Notes states what was fixed in the release, enhancements, workarounds, current open issues, etc.

When reviewing features of the CLM application, ensure also that you look at the each of these pages for each of the applications you wish to upgrade
  • Verify that your hot fixes are in the release
  • Verify that all system requirements are met, required integrations are supported, and product release levels are compatible

Environments

Unless you are running as a managed service, or in the cloud, then Collaborative Lifecycle Management (CLM) is a software stack with many considerations. Where multiple components require changing, you need to understand in which order to execute these.

Upgradecomponents.JPG

  • Ensure any customizations/settings are brought forward, for example: Java Virtual Machine heap size changes

Approach

If there are multiple instances of CLM, or any particular application involved, then that invariably also means multiple instances of users, stakeholders and considerations to balance. This should dictate your approach, whether it be one colossal upgrade effort for all, or a more segmented piecemeal option. The interested parties may make this simpler for you by not being able to upgrade simultaneously due to conflicting release schedules etc. Embrace the approach most suited to you and your stakeholders.

Example scenario

In terms of selecting which component to upgrade first, see how an upgrade from CLM 4.0.3 to 5.0.2 could be approached

The operating systems below may be used to host CLM applications:

  Before After
CLM Version 4.0.3 5.0.2
Database version DB2 9.7 DB2 10.5
Application Server Websphere 7.0.0.35 Websphere 8.5.5
Operating System Redhat Enterprise Linux (RHEL) 5 Redhat Enterprise Linux (RHEL) 7

* Unsupported on CLM 5.0.2

The recommended order for the situation above would be:

  • Upgrade the CLM Operating System(s)
  • Upgrade the Application Server
  • Upgrade the Database Management System
  • Upgrade CLM

Review your topology

When planning an upgrade, review your topology to ensure it will meet your business needs. This is the time to consider changing it if necessary.
To simplify CLM deployment options, IBM has outlined several topologies which are the recommended and most frequently chosen deployment patterns seen in the field. They represent tried and true examples of how customers have successfully deployed the CLM solution and are used by our system testing organization to perform deep “customer simulation” testing which includes installation, upgrade, functional, performance, and robustness testing.

When the need to switch topologies, application servers, databases, or other resources is discovered, make sure to migrate only one system part at a time where possible to isolate changes to the system.

Verify that your hot fixes are in the release

To avoid regression, an important step in upgrade planning is to ensure that any hot fixes that have been deployed in your enterprise are fixed in the new release. Where there are gaps, contact IBM Technical Support who will help you get the necessary fixes.

You can get a list of hot fixes by clicking on about this application in the top right-hand side of the web UI. For Rational Requirements Composer releases prior to 5.0, you will need to maintain a separate inventory of hot fixes that have been applied.

Testing testing.jpg


Setting up a staging environment

Ideally, a staging environment is a test sandbox that includes a snapshot of production data isolated from the production environment.
It is imperative that you test the upgrade procedure prior to attempting it with your production system.
It is strongly recommended you do this with your production data, so that you can fully test links and artifacts.
See Topologies and mapping files for setting up a test staging environment for details.
Finally, the test environment should be ‘clean’ and not used for other evaluation or testing purposes unless the changes can be completely backed out.
The options, at a glance, for achieving a staging area with production data are:

  • Isolated subnet environment
  • Isolated Virtual environment
  • Use a local hosts file
  • SOCKs server
  • DNS server

Reasons for a staging environment

The main three good reasons to test an upgrade are:

3goodreasons.JPG

The planning process and documentation to be used during the upgrade depends upon the timings and bespoke steps ascertained during testing. You are also looking to:

  • Define a repeatable process
  • Realize any defects now
  • Run any performance tests
  • Experimentation; particularly for users to understand new features

You should be or become very familiar with the monitoring section of this wiki during testing, to be able to recognize performance patterns during upgrade. Knowing when and whether your system should be under stress could be the difference between a successful, or aborted upgrade.

Further considerations for testing

There are several techniques designed to ensure that the testing of the upgrade, through iterations, as well as the upgrade itself run more smoothly.

Although production data is preferred, there are two recommended scenarios ypu can use to validate your CLM upgrade:

There is also the historical project:

  • CLM Build Verification Test (BVT) - This test scenario, available on jazz.net, is based on a CD Classic application that provides a site to buy and rate classic music CDs.


Downtime Mitigation

Below are some of the strategies used by customers to mitigate downtime during an upgrade:

  • Collocation - Temporary moving the DBMS closer (or in some cases on) to the application server
  • Temporarily increasing system resources (CPU/RAM) and increasing the heap size for repotools
  • Keeping the machines use exclusive during the upgrade in the case other applications are running any consuming system resources
  • Cleaning up any unwanted data prior to the upgrade. For example, large sandbox projects or even instances of the application that have been migrated and no longer used
  • Re-indexing applications prior to upgrading to ensure index integrity (repotools- -reindex all). See repotools command reference for usage.


Utilizing experiences experiences.jpg


Rollback and recovery

The IUG provides guidance in the event you need to back out of an upgrade. Make sure you have a back out plan.
Built on top of the IUG, our client facing teams have developed good practice tips based on their experiences upgrading the CLM application suite, which is summarized below and

  • Restore points prevent abandoned upgrades

rollback.JPG

  • Document a full rollback procedure
  • Don’t panic – it’s a tested part of the procedure

As with any backup and recovery procedure, it is important that the back out plan is tested. It is likely that you will only need to return to a previous restore point, for example, restore to the point of having successfully upgraded the Jazz Team Server. You may need to coordinate a virtual machine rollback to the point of a database backup, so ensure these tasks are completed concurrently.

Upgrades are successful when...

We collated experiences of the successful upgrades, even among the most complex of scenarios. Upgrades are successful when...

  • Defined custom test cases for common usage scenarios
  • Detailed Use Cases define how to perform actions
  • Step-by-step plans for overall upgrade
  • Full detailed plan of the day/weekend upgrade, including who, when, how etc..

Trial runs of upgrade
  • Append newly learned information to plans and improve upgrade procedures
  • Provide better estimates of each action and more accurately predict downtime for end users
  • Catch problems before the day of upgrade

Reporting considerations

Over time, there has been updates to the recommended upgrade order when there exists a reporting server (either RRDI or Insight) in your deployment. It is recommended that you review the Version Compatibility Matrix for CLM, RRDI, and Insight prior to upgrading. Currently:

  • CLM should be upgraded before RRDI/Insight
  • It is not required to upgrade CLM/RRDI at the same time
  • Follow pre-upgrade tasks for upgrading RRDI (disabling SSL and backing up configuration)
  • If using Insight, skip the data warehouse upgrade during the CLM upgrade and run migrateDW first. Then run the repotools-jts –upgradeWarehouse
  • Like other applications, DCC should always be at the same level as the JTS

It is worth reiterating the benefits that were mainly introduced in 5.0 for consideration:

  • Enable the Jazz Reporting Service (JRS) (5.x)
  • Improve dashboard performance by replacing widgets with JRS reports when possible
  • Allow the average user to create lifecycle traceability reports
Enable the Data Collection Component (DCC) (5.0.x)
  • Disable Java ETL or Data Manager CLM ETL (Insight)
  • Delegating ETL processing to the DCC takes the strain of ETL processing away from the CLM systems. Great for globally distributed deployments.
  • Provides faster and more frequent ETL refresh for closer to real-time reporting data

Evaluate using Online Migration

As of 4.0.6, online migration and estimating is available for RQM(Not available for 5.x to 6.x upgrade, see Note below). As of 5.0, it is available for RTC as well. Offline migration is still the recommended path as it allows for easier recovery from migration errors than an online migration. Before implementing an online migration, determine whether offline or online migration is warranted for your repository by running the migration estimation command and then checking the results against the following test matrices:

RTC Online Migration Test Matrix RQM Online Migration Test Matrix

Repository tools command to evaluate the online migration process

*Note:* There are some important updates regarding Online Migration noted on the Upgrade Insider page.


Final checklist clipboard.jpg

As this article explains, the majority of work for an upgrade is prior to the actual event itself. The last appreciation of an upgrade is that it is usually performed by real people, over a weekend and/or late at night. Therefore, reducing the manual steps and ensuring all eventualities are documented will relieve pressure on those executing the upgrade.

Here is a basic checklist for the upgrade event itself and a summary of this article:

  • Checked Staged communications
  • Checked Don’t deviate from the plan
  • Checked Ensure operator follows each step
  • Checked Abort and return another day
  • Checked Track each step against experience
  • Checked Monitor for deviance from expected
  • Checked Once complete run confirmation tests
  • Checked Scour logs to ensure error-free
  • Checked Inform users system is available


Contact us when you need help

There are several ways in which you can get help:

You can contact your Technical Sales Representative or, if you are an Accelerated Value Partner, then you can also reach out to your Accelerated Value Lead to review and answer any questions you have regarding the upgrade.

You can also post questions in the Jazz Forum on jazz.net. It is monitored by the jazz community and email notifications let you know when your question has been answered or has been commented upon. Before posting your question, please search the forum as it might already be answered.

Related topics: Deployment installing upgrading and migrating, Using the product information centers

External links:

Additional contributors: RosaNaranjo, RickMaludzinski, GeraldMitchell

Topic attachments
I Attachment Action Size Date Who Comment
JpgJPG 3coreaspects.JPG manage 63.8 K 2015-05-25 - 10:19 PaulEllis Title - 3 core aspects
JpgJPG 3goodreasons.JPG manage 20.6 K 2015-05-25 - 15:21 PaulEllis  
Jpgjpg Planning.jpg manage 24.5 K 2015-05-25 - 11:10 PaulEllis Planning
JpgJPG Upgradecomponents.JPG manage 73.8 K 2015-05-25 - 14:25 PaulEllis Components of an upgrade
JpgJPG checklist1.JPG manage 65.8 K 2015-05-25 - 12:37 PaulEllis  
JpgJPG checklist2.JPG manage 73.4 K 2015-05-25 - 12:32 PaulEllis features, envs, approach
Jpgjpg clipboard.jpg manage 14.2 K 2015-05-26 - 09:18 PaulEllis  
Jpgjpg experiences.jpg manage 30.4 K 2015-05-25 - 15:45 PaulEllis  
JpgJPG rollback.JPG manage 31.3 K 2015-05-26 - 08:25 PaulEllis Rollback
Jpgjpg testing.jpg manage 27.4 K 2015-05-25 - 15:00 PaulEllis  
Edit | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r29 < r28 < r27 < r26 < r25 | More topic actions
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Contributions are governed by our Terms of Use. Please read the following disclaimer.
Ideas, requests, problems regarding the Deployment wiki? Create a new task in the RTC Deployment wiki project