Running test cases

If a test case execution record for a test case is not created yet, you can run the test case directly and associate the test execution results with a particular test plan.

About this task

When you run a test, the Quality Management application runs a test case execution record. If test case execution records for a test case that you want to run are created, you initiate the test run from a test case execution record. If you have not created test case execution records for a test case (and do not want to), you can initiate the test run from the test case. When you initiate a test run from a test case, you define the test case execution record in the Run Test Case dialog box by selecting a test plan, iteration, and test environment to associate with the test run.

Procedure

  1. From an open test case, click the menu arrow next to the Run icon Run and select Run to open the Run Test Case dialog box.
  2. Select values in the Run Test Case dialog box.
    1. In the Test Case Execution Records list, select an existing test case execution record, or accept the default option, New Test Case Execution Record, to create a new execution record. If you choose to create a new execution record, select values in the Test Plan, Iteration, Test Environment, and (if enabled) Channel fields.
      Remember: Channels are subsets of Test Environments. Listed channels depend on the selected Test Environment.
      Note: If you have an existing execution record that uses the values you specify for the test plan, test iteration, and test environment, the existing execution record is used and a new one is not created.
    2. In the Test Script list, select a test script.
    3. If you selected an automated test script, set the following values.
      1. In the Machine list, select a specific test machine on which to run the test.
      2. If you want to schedule the test case to run at a future time, select Schedule Execution and choose a date and time.
    4. To associate the results of this test run with a build record, click the Change Associated Build icon Change Associated Build and select a build record. The test case result from this test run is automatically linked to the build record that you choose.
    5. To record a result for this test without executing the script, select Create Result without Execution and then, in the Set Verdict of the Result aslist, select a value.
    6. To override the values of execution variables that are defined in your test case and test script, click Modify Execution Variable Value(s). Then, in the test case and test suite variables lists, type the value that you want to assign to each variable.
    7. To select records in test data that is associated with the test script, select Select Records from Test Data, click Next, and then select records that you want to use during the test run.

      You can adjust values in the Record Selection Criteria section to show a subset of the test data or clear the fields to show all records. Records that you select at run time override the record selection criteria that is set in the test case or test script.

  3. Click OK or Finish to run the test case.
    The script execution window opens and an arrow points to the first statement.
    Note: If an automated test script is associated with the test case, select the appropriate adapter to run the test, and then click OK. The script execution window opens, and the test runs automatically. At the end of the test run, the adapter attaches the tool-specific result to the Result Details section of the test case result.
  4. For each step, follow the instructions in the step and then and assign a verdict to the step.

    Possible verdicts include:

    • Pass Pass
    • Fail Fail
    • Inconclusive Inconclusive
    • Blocked Blocked
    • Error Error
    • PermFailed PermFailed
    • Deferred Deferred
    • Skipped Skipped
    Note: To see the options for Blocked, Error, PermFailed, and Deferred, click the arrow next to the Inconclusive icon Inconclusive.
    Tip: To assign one verdict to the current step and all subsequent steps, click the Apply All icon Apply all verdict, select the verdict that you want, and then click OK.
  5. Optional: To skip steps during test case execution:
    1. Select the step and click the Inconclusive icon (Inconclusive).
    2. From the list, select Skipped.
    Note: You cannot skip all the steps during manual test case execution.
    Skipped steps do not affect the overall verdict of the execution result. However, skipped steps are included in the weight distribution of execution results.
  6. Optional: Type the step results in the Actual Results field.

    You can also set a user preference to automatically populate an empty actual result when manual step execution passes. You can select one of these options: apply the expected result to the actual result, or apply a predefined message to the actual result. If you want an empty actual result when you run a manual step, you can enter a space so that the actual result appears empty but is not. The actual result is not automatically populated by the new preference. For more information about how to set this preference, see the Execution preference in User preferences for quality management.

    Note: When you use the Apply All option to assign a verdict to the current step and the subsequent step, the actual result for all the selected steps is not automatically populated. The actual result for all the selected steps remains empty.
  7. Optional: Add a comment to a statement:
    1. From the toolbar, click the Show/hide contents icon (Show/hide Contents). Make sure that the Show Attachments/Links box is checked.
    2. Click the Comment icon (Comment) in the column beside the active step to add, edit, and remove comments. The Comment icon is included with each statement that has a comment.
  8. Optional: Add an attachment to a statement. You could create a screen capture of the application under test and attach it to the statement. To attach a file to a statement:
    1. From the toolbar, click the Show/hide contents icon (Show/hide Contents). Make sure that the Show Attachments/Links box is checked.
    2. Click the Attachments icon (Attachments) in the column beside the active step to add and remove attachments. The Attachment icon and the attached file's name are included with each statement that has an attachment.
  9. Optional: Add an image directly to a result by clicking the Insert Image (Insert image) or Capture Screenshot icons.
  10. If you discover problems with the application under test, you can submit a defect or link the activate statement to an existing defect:
    1. From the toolbar, click the Show/hide contents icon (Show/hide Contents). Make sure that the Show Attachments/Links box is checked.
    2. To create a new defect, click the Create New Defect icon (Create new defect) in the column beside the active step. The Create New Defect form opens.
    3. To link a statement to an existing defect, click the Link to Existing Defect icon (Link to existing defect). Whether you create a new defect or link to an existing one, links to defects appear beside associated statements.
  11. When the test run is complete, click Show result to view the test case result.

    By default, a lowest severity state is selected based on the list of enabled states under Project Properties > Execution State. Severity of the execution state in ascending order as follows:

    • In Progress
    • Passed
    • PermFailed
    • Incomplete
    • Inconclusive
    • Partially Blocked
    • Deferred
    • Failed
    • Error
    • Blocked

    Paused state is never selected by default.

    Note: The overall verdict of the test case result follows the highest severity state among all the step verdicts.
  12. Update the test case result as required, and click Save to save your updates.

    For example, you can modify the Actual Result value, provide a detailed description in the Notes section, or modify the weight distribution in the Weight Distribution section. After you edit, be sure to click Save to save your changes.

    Note:

    To change the Actual Result value in a test case result, you must have the Save Test Case Result: Edit Actual Result permission.

    The Result Details section varies with the type of test script that you run. For manual test scripts, the Result Details section includes a bar chart of step results, followed by the script steps with results. You can add comments to the script steps in this section.

    To attach or remove a file from a step in the Result Details section, click Edit and then click the Attachments icon.

What to do next

To view the test case result later, in the main menu, click Execution > Browse > Test Case Results.

You can also rerun a test case from the test case results. This option is not available for executions without a script.


video icon Video

Jazz.net channel
Software Education channel

learn icon Courses

IoT Academy
Skills Gateway

ask icon Community

Jazz.net
Jazz.net forums
Jazz.net library

support icon Support

IBM Support Community
Deployment wiki