Running test suite execution records

You can run tests in several ways, including by running a test suite execution record.

About this task

You can run a test suite directly, or you can run a test suite execution record. When you run a test suite directly, the test suite runs in the local test environment. A default test suite execution record is generated, but the record does not include any planning or test environment information.

If your situation is more complex and you must support multiple test environments, you can generate test suite execution records to cover your planning and environment needs. You can designate specific test environments as test cells. For example, test cell Z might consist of servers A and B with a workstation that runs a specific adapter.

You create test suite execution records in the context of a test suite. Test suite execution records map test planning and environment information to the test suite. When you run a test suite execution record, you run a test suite in a specific software and hardware environment and associate the test with a specific test plan and test iteration.

You can use test suite execution records to run both manual and automated tests. If you created manual test scripts, you can add them to a test case in the suite. Then, you can associate test suite execution records with that suite. Similarly, if you refer to automated test scripts from Engineering Test Management, you can add them to a test case in the suite and associate test suite execution records with the suite. Typically, when you run the test suite execution record, you run the scripts that were added to the test cases in the associated test suite.

After testing commences, you can pause test suite runs from the Test suite execution console. You might pause test suite runs when the system that you are testing needs minor adjustments before testing can proceed.

When you pause a test suite run, any manual test runs that are part of the suite are also paused. If you pause a test while an automated test is running, that test is completed; however, automated tests that are not started do not run. In either case, the test suite is paused. You can resume paused test suites from the Test suite execution console, Test suite execution record list view, or Test suite execution record editor.

You can stop test suites from the Test suite execution console. Unfinished test cases in that suite are marked incomplete. Stopped test suites cannot be resumed.

Procedure

  1. Complete one of these steps:
    • Create a test suite execution record and associate it with a test suite, and then open the test suite.
    • Open a test suite and then generate multiple test suite execution records for the test suite.
  2. In an open test suite, open the Test Suite Execution Records section.
  3. Select the test suite execution record to run, and then click the Run icon (Run).
  4. Optional: From the Channels menu, select a channel. Whether a channel is available depends on how the test environment is configured.
  5. If the test suite contains test cases that have automated test scripts, you can select multiple test cases and assign the same environment or test cell to them. Select the check boxes that are next to the test cases, and then click the Select a machine to run this test (Select a machine to run this test) icon. To associate test cells with the test cases that compose a test suite, use the Test Cases menu for the test suite.
    Note: Most automated test adapters can run only one script at a time. If you select the same adapter for multiple test cases, the scripts that are on that adapter are run in a sequence. To run test scripts in parallel, you must select a different adapter for each script.
  6. Optional: Specify which test cases to exclude from the test suite run. If you do not exclude test cases, all of the test cases in the suite are included in the test suite run. To skip test cases, do one or both of these actions:
    • To skip any test cases that passed when the test suite execution record last ran, click the Exclude Passed Test Cases (Exclude Passed Test Cases) icon.
    • To skip specific test cases, select the test cases in the table, and click the Exclude from Execution (Exclude Execution) icon.
    The test cases that you selected are marked with an icon in the Exclude from Execution column and are not included in the test suite run. If you decide to include the test cases in the test suite run, select the test cases again, and click the Exclude from Execution (Exclude Execution) icon.
  7. To associate the results of this test suite run with a build record, next to the Build Record option, click the Change Associated Build icon (Change Associated Build) and select a build record. The execution result from this test run is automatically linked to the build record you select.
  8. To schedule the test suite execution record to run at a future time, select Schedule Execution and select a date and time.
  9. Select whether to run the test cases that are in the suite in parallel.
  10. If you selected the option to run the tests in a sequence, you might select more options:
    • If your test scripts use execution variables and you want to pass the values between scripts during the test suite run, select Pass execution variables between scripts.
    • To specify that the test suite must be stopped if a single test case does not pass, select Stop suite execution if any test does not pass. This option is useful if you are running a test case in a test suite that depends on the success of previous test cases.
  11. To record a result for this test without running the scripts, select Create Result without Execution, and select a result.
  12. To temporarily modify the values of execution variables that are defined in your test suite, click Modify Execution Variable Value(s).
  13. Open the Test suite execution console by clicking Run. If you are running tests in a sequence, the first test case in the sequence is shown in the console. If you are running tests in parallel, the automated test cases start to run and the manual test cases are listed in the console. If the first test case in the suite is an automated test, the test runs automatically.
  14. If the first test case in the suite is a manual test, click Start Manual Test. In the window that opens, follow the steps. When you are finished, click Close. If you are running tests in parallel and have manual test cases, you can return to the Test suite execution console at any time and start running another test case in the suite.
  15. Optional: You can combine the results of multiple executions of the same test suite record and create a consolidated record by using the roll-up operation.
    Note: You must have the Create Roll Up Result permission to roll up the results for a test suite execution record.

    For more information about how to roll up the result for a test suite execution record, see Rolling up results for test suite execution records.

  16. When a test case is completed, you can view the execution result for the test case by clicking the icon in the Result column.
  17. Update the execution results as needed, and be sure to save your updates. For example, you can provide a detailed description of the test results in the Result Details section.
    Note: The Results Details section of the test suite result displays a snapshot of the test case results when the associated test suite was executed. If any of the test case results are deleted, the entries remain in the Result Details section to indicate that the test cases were executed, even if there are no details on the deleted test case results.
  18. To submit a defect, in the Change Requests section, click the Create New Defect (Create new defect) or Link to Existing Defect (Link to existing defect) icons. In the defect form that opens, type information about the problem, and then save the page.
  19. Run the remaining test cases in the suite. If you want to finish running the test suite later, or if another tester is responsible for the next test case, close the page. You can return to the test suite execution later.
    If you close the Test suite execution console during a test suite run, you can return to it in any of these ways:
    • Click Execution > Browse Test Case Execution Records and then click the Last Result icon.
    • Click Execution > Execution Console and find the test case that must be run as part of the test suite. In the Test Suite Progress column, click the link.
    • Open a test plan and in the Test Suite Execution Records section, click the Last Result icon.

What to do next

If your organization requires a review of the test execution results, use the Formal Review section of each execution result to assign reviewers and approvers.

video icon Video

Jazz.net channel
Software Education channel

learn icon Courses

IoT Academy
Skills Gateway

ask icon Community

Jazz.net
Jazz.net forums
Jazz.net library

support icon Support

IBM Support Community
Deployment wiki