Configuring and running tests from TestArchitect Client

From a TestArchitect Client session, tests can be configured with the Execute Test dialog box. Options include selecting which tests to run, where to run them, how to handle results, and whether to generate additional diagnostic information during testing. You also have the option to run your test directly from the dialog box (online execution), or to generate a batch file allowing for a later test run (offline execution).
The Execute Test dialog box can be accessed by right-clicking any test module node in the TestArchitect explorer tree and then clicking Execute Test.
Fastpath: F9

The Execute Test dialog box is then presented.

Following are the fields of the Execute Test dialog box:
  1. Test Modules panel: lists the test module(s) to be executed.

    • If a test module is selected, it appears in this box.
    • If you select a test folder instead a test module, all test modules in the folder appear in this box.
    • You can expand each test module to specify which tests to execute:
      • Individual test cases.
      • The initial section of the test module starts before the first test case with the INITIAL action. This section is where you would typically perform initializing operations like launching the application under test and logging into it.
      • The final section of the test module starts after the final test case with the FINAL action. You can use the final section to perform some housekeeping before the conclusion of your test, such as logging out of the application.
    • Save as Test Suite...: Use this button to create a static test suite containing the group of test modules listed in the Test Module panel. (Learn more.)
  2. General tab:

    • Settings panel:
      • Result Name: Name used to uniquely identify this test run for future reference; this result name together with the name of the controller or device is used as the title on the test result page. For example, if the Result Name for the test running on controller is Action Based Testing (2013-04-04 13.54.11), the title for the test run page will be Action Based Testing (2013-04-04 13.54.11) -
      • Comment: (Optional) Allows you to add a comment that will be included in the report generated after the test is executed. The comment value will appear as a field of the result item in the Results subtree of theTestArchitect explorer tree, and can therefore be used to filter or sort your reports.
      • Build Number: (Optional) Use to specify the AUT build number. The build number is displayed in the test result report and can be used as a filter or sort criteria.
        Note: During execution, the built-in action assign result fieldcan be used to assign a new value to this field, which changes the build number.
      • Automation Tools: Click this button to set the automation tools (see Lesson #8: Using an automation harness). The following settings are available for automation tools:
        • Playback Tool: Select the automation playback tool.
        • Executable(s): Path to the automation playback tool executable file.
        • Script(s): Path to the automation script folder.
        • Command Line: Add any command line switches you may wish to include to control the third-party tool.
    • Controllers/Devices panel:
      • Displays the list of target controllers and/or devices upon which the test will execute. If there is exactly one controller/device selected, this panel will display its name; otherwise, the number of selected controllers/devices is displayed instead.
        • TestArchitect defaults to the list of controllers/devices specified for the previous execution session.
        • If, when a test starts, TestArchitect cannot access any controller or device on the list, a message box is displayed to indicate the problem.
      • Select Controllers/Devices: Click this button to designate which controller and device the test will execute on.
        • Lab Manager Server: (Display only) IP and port number of the Lab Manager Server to which the test controllers and devices are registered.
        • Controllers/Devices panel: Lists all available controllers and devices on which the test can be executed. The list consists of those controllers and devices that are either registered with the Lab Manager Server or have been manually added with the Add Controller button.
        • Controller Port Configuration: Use this panel to specify to TestArchitect the port number that the remote machine is using for its TestArchitectcontroller, if not using the default.
          • Host Name: (Display only) IP address of remote machine currently selected in the Controllers/Devices panel.
          • Port: Port number through which TestArchitect will attempt to communicate with the controller on the host specified in the Host Name field. If this is not the port on which the controller is known to be listening, change this value and then click Save .
      Note: To run tests on multiple controllers or devices simultaneously, see Multiple device execution for details.
    • Variation Specification panel:
      • Keyword: Keyword, or comma-delimited list of keywords, specifying the test variation to be executed, if any. (See Creating keyword variations.)
        Attention: If the test module selected for execution is a variation, this field is automatically filled in. If multiple test module variations are selected for execution and they do not all feature identical keyword sets, this field is not auto-filled.

      • AUT Version: Enter a value or click the Select Version button to specify a variation tailored to an AUT version or platform (See Creating linked variations.)
        Attention: If the test module selected for execution is a variation, this field is automatically filled in. If multiple test module variations are selected for execution and they do not all feature identical keyword sets, this field is not auto-filled.

      • Time Traveling: To opt for time traveling execution, which selects an historical "snapshot" of the test's project items for execution during the test run, select the check box and provide an appropriate timestamp. (See Time Traveling for details.)
    • Screenshot recording panel: Use this panel to enable and configure the capturing of screenshots of UI-Interacting actions. For details, see Capturing screenshots during test execution
  3. Advanced tab:

    • Export result(s) to HTML: export test results to HTML file automatically once the text execution is complete (see Exporting local test results to an HTML file for more details).
      • Include screenshots: retain all captured screenshots in the exported HTML test result.
        • Optimized resolution: (Default) Included screenshots' dimensions are optimized to save space in the exported HTML test results. Specifically, the screenshots are saved as thumbnail images.
        • Regular resolution: Original resolution of included screenshots is retained. Specifically, the screenshots are saved as full size images.
      • If the exported test result is a master result, that is, it is a test suite result, or it contains subresults, the Include all sub test results check box is available. With this option chosen, the master result and its subresults are all exported into HTML files.
    • Export result(s) to xUnit: export test results to a XML file automatically in xUnit-format for integrating into the continuous integration tools once the text execution is complete.
    • Export result(s) to XML detail: export test results to XML file automatically once the text execution is complete (see Exporting test results to XML for more details).
    • Upload result to Quality Center: automatically upload test results to Quality Center after the test terminates. Click browse button and select the Quality Center test set to upload test results to. See Automatically uploading TestArchitect test results to Quality Center for details.
    • Automatically add result(s) to repository: TestArchitect automatically stores the test results, based on predefined conditions, to a specific location on a repository once the text execution is complete (see Adding test results automatically for more details).
  4. Startup Settings tab: List of custom, user-defined settings to be created at test run startup, and/or built-in settings that are to be set to your choice of values at startup.

    • Startup Settings: Enable this check box to allow the listed built-in settings to be initialized with your values at startup (overriding the TestArchitect defaults), and the user-defined settings to be both created and initialized. (Learn more.)
  5. Execute: Click this button to start executing selected tests.
    Instead of executing the tests, you can use the following:
    • Compile Only: Prepare a test execution, but do not actually start the execution. Automation engineers can use this to test harness scripts in their own development environment.
    • Generate Batch File: Generate a batch file (*.BAT file in Windows) to execute the test in command line mode.
Tip: To quickly execute a test module you are working on in the editor, press F9. This starts the execution without invoking the Execute Test dialog box.