Running TestArchitect tests from Microsoft Test Manager

After creating a TA-TFS integration project in Visual Studio that contains test cases, you can plan your tests, execute test cases, and view test results in MTM.

Ensure that you have already taken the following steps:
Important: When running an automated test, the execution of every single TFS test case entails the execution of both the INITIAL and FINAL sections, if any. For example, the execution of test case #01 is preceded by the execution of the INITIAL section, and followed by the FINAL section; the same is true for test case #02, and each subsequent test case.
  1. Run MTM and connect it to the team project from TFS where your project resides.
  2. In MTM Testing Center, click Test on the ribbon. Then click the Run Tests tab. Double-click your TFS project in the project explorer tree.

    Your selected project appears in the MTM main window panel showing the test cases that are linked with TestArchitect test cases.
  3. In the MTM main window panel, right-click a test case that you want to run and choose Run. The selected test case must be of the automated type. (That is, the value of Automated must be Yes.)
    Note: If you want to run a test in a different environment or with different test settings, choose Run with options.
    The TestArchitect Execution dialog box appears.

  4. In the dialog box, enter the requested information, as described here.
    Remember:
    • To save all defined configurations, click the Save button in the TestArchitect Execution dialog box.
    • By default, the TestArchitect Execution dialog box only appears on the first run. If and when the configurations saved from the previous run are no longer valid, the dialog box reappears on the subsequent run.
  5. In the TestArchitect Execution dialog box, click Run to execute the tests.
    Important: If you are executing tests for the first time, you are prompted to enter your TestArchitect license server information. Enter the required information in the dialog box. If the specified license server is reached successfully, or the trial key is validated, the provided license information is stored. From then on, you will be able to execute tests without again being prompted for this information.

    CAUTION:
    MTM's automated tests settings include two timeout settings for imposing limits on automated test times. If these settings are enabled, you must ensure that they meet or exceed the maximum expected test times for your tests. These settings are available under MTM's Lab Center, under Test Settings > Timeouts:

    • The first setting, Abort a test run if the total time exceeds:, when enabled via the check box, attempts to enforce a ceiling on the total run time of your test run. Should you set a value here and your test run exceeds it, your test is not aborted or directly affected in any way. However, your test results will not be uploaded to TFS.
    • The setting Mark an individual test as failed if its execution time exceeds: is applied to each individual test of your test run. Hence, if it is used (which we advise against), you should ensure that it exceeds the maximum possible test time of your longest test of the test run set.
      CAUTION:
      If this setting is enabled and any one of your tests does exceed its value, it may have the effect of corrupting the transferred results of other tests in the set.

A test run panel appears showing the status of the running test. Initially, the Result Overview section displays the current state of the tests as pending while the test cases are run. After the selected test cases finish their runs, the Result Overview section is updated to show the status of the test runs.

Important: When running an automated test, the execution of every single TFS test case entails the execution of both the INITIAL and FINAL sections, if any. For example, the execution of test case #01 is preceded by the execution of the INITIAL section, and followed by the FINAL section; the same is true for test case #02, and each subsequent test case.

Additionally, the test run results are generated, and then uploaded into the Attachment sections as HTML attachment file(s).



Tip:
  • Right-click the attachment file and select Open to view the test results in a browser.
  • The layout of the generated test results when test cases are executed in MTM is similar to the layout displayed when the same test cases are executed in TestArchitect.