< Previous | Next >

Lesson 5: Define quality objectives and set entry and exit criteria

In this lesson, you will learn about quality objectives, how to define them, and how to add them to test plans to determine entry and exit criteria.

About this task

Quality objectives define the metrics for what constitutes a quality product. You can use quality objectives to define the overall quality goals for a release, as well as to define the required entry and exit criteria for testing. You can use these goals to determine the level of quality that must be achieved before your product is considered ready to be released.

Quality objectives often track things like the number of open defects and the number of blocked or failed execution records. Any quality objectives involving defects now take into account those defects that are managed in OSLC-linked Change and Configuration Management (CCM) providers such as Engineering Workflow Management and Rational ClearQuest.

Defining quality objectives

About this task

During the planning process, the test team defines quality objectives. First, the test manager examines the predefined quality objectives in the project properties and evaluates whether they are suitable for the test team. Later, test leads implement the objectives in individual test plans, where the test team can track whether or not each objective has been met.

To define new objectives, you can open the Managing Quality Objectives editor from the Project Properties page or from an individual test plan. Any edits that you make are available to all test plans in the project area.

In the following example, you log in as Tammy, the test manager for the JKE Banking project, and review the project-level quality objectives.

Procedure

  1. Log in as the test manager by typing tammy for both the user ID and password. If you are prompted with a project login window, select the JKE Banking (Quality Management) project. The quality dashboard opens. To modify and save project properties, a user must have the necessary permissions. This can be any one of the following:
    • A user granted permission to save and modify a project area in the process configuration.
    • A user who is an administrator of the project area.
    • A user who is granted either the JazzProjectAdmins or JazzAdmins repository group permissions.
  2. Click the Administration (Administration) icon in the top right portion of the banner and select Manage Project Properties. Then select Quality Objectives.
    The Manage Quality Objectives section of Project properties.

    Each quality objective includes a Name and Description, as well as a Condition and a Target.

  3. After reviewing the current project quality objectives, Tammy decides to modify the settings of some of the predefined quality objectives and also create some new, user-defined ones.

    For example, one of the predefined quality objectives states that the Percentage of Blocked Execution Records must be less than 10%. Tammy believes that the target should be less than 5%.

    1. In the Name column, find the row containing Percentage of Blocked Execution Records.
    2. In the Target column for that row, double-click the value 10.
    3. Lower the target value by typing 5; then press Enter.
  4. Now, Tammy adds a new quality objective.
    1. Click the Add Row icon (Add table row). A blank row is added to the end of the table.
    2. In the Name column of the new row, type Number of Customer-Submitted Defects Open; in Description, type Objective stating that all customer-submitted defects must be closed.
    3. In Condition, choose =, and in Target, type 0.
    Remember: Both the predefined and user-defined quality objectives can be used in any test plan within that project area. However, only predefined quality objectives that are known by the Quality Management application can be used in computations. User-defined quality objectives are informational only and do not contain computed values.
  5. Click Save.

Quality objectives and test plans

You use the Quality Objectives section of a test plan to list the quality objectives for a release.

Typically, quality objectives provide various measurements of quality for the overall release. You might list several performance objectives, such as the maximum allowable response time for certain functions or the minimally acceptable number of concurrent users. You might also list several objectives that pertain to usability or reliability.

Entry and exit criteria are also considered quality objectives. Entry criteria define the prerequisite items that must be achieved before testing can begin. You can use the Entry Criteria section to specify in table format the conditions required to begin testing, such as the minimum level of product and feature quality necessary before testing begins.

Exit criteria define the conditions that need to be met before the testing can be concluded. Use the Exit Criteria section to specify in table format the conditions to meet for a particular test cycle to be considered complete. For example, you might specify that testing is incomplete until 100% of the test cases have been run and that all of the most severe defects have been fixed. During the course of a test cycle, you can adjust the exit criteria.

Adding quality objectives to the test plan

About this task

A test lead, who is responsible for a particular test plan, defines the overall quality objectives for the test plan using the quality objectives defined by the test manager, and if necessary, creates additional quality objectives in the test plan.

Note: Any new quality objectives that are added in the test plan are available to all test plans in the project area.

For example, the test lead might want to add some overall quality objectives that have not been defined in the project properties, such as the number of concurrent users that the application under test must support or the maximum time allowed for the application under test to open.

When a quality objective is added to a test plan, the Condition and Target are merged together in the Expected column, as shown in the following figure:

Quality Objectives

The test lead sets the status to Not Started.

Procedure

Tammy is also acting as the test lead for the JKE Banking version 1.0.1 release. To utilize quality objectives in the JKE Banking V1.0.1 test plan, continue as Tammy and complete the following steps:

  1. Add a quality objective to the Quality Objectives section:
    1. Click the Quality Objectives section. Four quality objectives are listed, as previously defined for the System Verification test plan. Tammy determines that the existing quality objectives still apply to the new test plan, but she wants to add the new project-level quality objective as well.
    2. To add the new quality objective to the test plan, click the Add Quality Objectives icon (Add table row), select the Number of Customer-Submitted Defects Open objective, and click OK.
    3. Your new quality objective is included at the end of the list.
      Quality Objectives
    4. Save the test plan.
  2. Now Tammy opens the Entry Criteria section of the test plan to define the prerequisites that must be met before testing can begin.

    For example a System Verification Test team might want to require that all functional verification tests have been attempted and that 95% have been completed; a Functional Verification Test team might want to require that the user interface is frozen.

  3. Tammy opens the Exit Criteria section of the test plan to define the conditions that must be met before testing can be concluded.

    For example a System Verification Test team might want to require that all System Verification Tests have been attempted and that 95% have been completed.

Evaluate exit and entry criteria

Procedure

  1. As the development effort progresses, the test lead determines if the test entry criteria are being met.
    1. The test lead refreshes the view with the most recent values by clicking the Evaluate Quality Objectives icon (Evaluate quality objectives). This causes Quality Management to gather the actual data associated with the objective and measure it against the expected value.
    2. The test lead compares these values with the expected values, sets the Status, and comments on each quality objective.

      Entry Criteria

    3. The test lead meets with other team members to determine whether the entrance criteria have been met.

      The test lead may decide to stick with the original entry criteria estimates or to make adjustments.

  2. As the testing effort moves forward, the test lead determines whether the exit criteria are being met, following a similar process to that of the entry criteria.
  3. At the end of the testing effort, the team evaluates whether the overall quality objectives have been met.

Lesson checkpoint

In this lesson, you learned how to complete the following tasks:
  • Edit project-level quality objectives
  • Add quality objectives to a test plan
  • Determine and evaluate test entry and exit criteria
< Previous | Next >

video icon Video

Jazz.net channel
Software Education channel

learn icon Courses

IoT Academy
Skills Gateway

ask icon Community

Jazz.net
Jazz.net forums
Jazz.net library

support icon Support

IBM Support Community
Deployment wiki