View Orchestration Results

View Orchestration Results Overview

Orchestrations in Functionize allow you to execute test cases across multiple projects. This is especially useful for end-to-end scenarios and system testing. An Orchestration Result is a status given after a Single or Group Orchestration is deployed. The results break down which individual tests failed or passed. This also shows you warnings or incomplete executions.

 

How To View Orchestration Results

  1. Log into the Functionize app
  2. In the top navigation menu, select the Orchestrations tabOrch.png


  3. Orchestrations List page will be displayed with the most recent Orchestration listed at the top by default, the Execution Details column displays a quick-view results summary of each Orchestrationorc listing.png


  4. To view more options or more detailed orchestration results, click the Actions menu in the far-right column for a specific Orchestration and select Orchestration Resultsorc result.png


Orchestration Results Report

The Orchestration Results report opens in a new browser tab which breaks down the execution details displaying all execution runs via a drop-down menu, basic results at a glance, results per browser, summary of each test in the orchestration, performance trends and action durations.orc result report.png

 

 

Download Results

A detailed CSV summary of all tests executed and high-level result details may be downloaded by clicking Export CSVorc export.png

 

Viewing More Details

  1. Failed tests will include details on the point of failure which can be viewed by clicking the arrow button on the left side of the test in the summary section


  2. Click the Details button for any test to open that test's execution Test Details page


  3. A new tab will open with a read-only Test Detail page for this executiontest detail page orc.png


  4. The Information button displays basic statistics of the test


  5. The Test Action menu contains options to download a PDF of the test or view performance reports


  6. Click View Results or View button next to any action for a step-by-step screenshot, read-only view in Slider view


  7. Actions may be submitted to our Train the AI feature in the Slider View
    See User Guide: Report to AI


Rerunning Individual Tests

Tests in an Orchestration can be rerun without executing the entire orchestration. Each test can be rerun irrespective of its Passed, Failed, or Warning status. However, currently, individual tests must be relaunched for reruns. Hover over the browser icon to select the option to rerun your test.

When a test case is rerun as part of an orchestration, the latest result will supersede older results for that browser and dataset (when applicable). Although both results will appear, integrations or notifications to test management tools, and results accessed via API, will only provide the most recent result for the browser and dataset combination.

 

Orchestration Results History

The top of the Orchestration Results page contains several menus to filter your view. From this page you can view a different Orchestration, view a different Date/Time, filter by Tags, and filter by Browser. Viewing the history of your orchestration is as simple as selecting a previous run from the Date/Time drop-down on the left. The page will refresh to show the newly selected Orchestration Results report.

 

Orchestration Performance Trends and Action Duration Reports

The Orchestration Results page provides some high-level performance trends and action duration graphs to provide insight into site under test performance metrics.  The following detail graphs are provided as the user scrolls down through the Orchestration Results page:

 

Performance Trend Line

Displays the different performance metrics (i.e. Page Loading Time, DOM Interactive, Visual Completion, etc.) in a line graph for the test cases used in the Orchestration.


Page Performance

Displays the different performance metrics (i.e. Page Loading Time, DOM Interactive, Visual Completion, etc.) in a bar graph for the test cases used in the Orchestration.


Performance Details

Select the specific Performance metric (i.e. Page Load, DNS, Visual Completion, etc.) to view in the graph.


Action Duration

Select the specific test case from the Orchestration to view each action's, within the test case, duration to execute.