1. Functionize
  2. User Guide
  3. Test Executions & Orchestrations
  4. Orchestrations

Create an Orchestration

Follow

Create an Orchestration Overview

Functionize enables users to execute multiple test cases from various projects, setting the test execution run order and creating better organization for test groups and types. These orchestrations can execute tens, hundreds, or even thousands of test cases globally, across any browser or mobile device. Users can create single or group orchestrations.

For every execution of an orchestration, users can:

  • Check the performance of the orchestration execution over time.
  • Check the stability of the test cases executed as part of an orchestration over time.

Key Features

  • Single Orchestration: Execute one bucket/group of tests in parallel or sequential mode, using filters such as projects, tags, and modules.
  • Group Orchestration: Set up multiple buckets with granular organization, such as executing one group sequentially and another in parallel.

Accessing Orchestrations

Click on Orchestrations on the header of the page.


Creating a Single Type Orchestration

Open Orchestration Tab:

    • Click on Create New Orchestration.

Details Tab:

    • Title: Name your orchestration.
    • Projects: Select projects to run in this orchestration.
    • Environment: Choose the environment for execution.
    • Folders: Select a folder to filter test cases.
    • Tags: Optionally filter test cases using tags.
    • Exclude Tag: Optionally exclude test cases using tags.
    • Test Cases: Select test cases based on previous filters.
    • Schedules: Choose a schedule (On-Demand, Hourly, Every 4 Hours, Every 12 Hours, Daily).
    • Select Browser: Select browsers for the orchestration (must be enabled at the individual test case level).
    • Orchestration Type: Choose between Background and TDM Run.

Notifications Tab:

    • Jenkins Integration:
      • Jenkins Crumb URL: Enter the URL.
      • Jenkins Callback URL: Enter the URL.
      • Jenkins Authentication: Enter username:password.
      • Jenkins Parameters: Enter parameters as key1:value1,key2:value2.
    • Slack Integration:
      • Web-hook URL: Enter the URL.
      • Channel: Enter the channel name.
    • FogBugz Integration:
      • API Key: Enter the key.
      • API URL: Enter the URL.
      • Project Number: Enter the number.
    • Sentry.io Integration:
      • Authorization Token: Enter the token.
      • Organization Slug: Enter the slug.
      • Project Slug: Enter the slug.
      • DSN: Enter the DSN.
    • GitLab Integration:
      • Private Token: Enter the token.
      • Project Id: Enter the ID.
    • Jira Integration:
      • Domain URL: Enter the URL.
      • Username: Enter the username.
      • API Token: Enter the token.
      • Issue Key: Enter the key.
    • Rocket Chat Integration:
      • Domain URL: Enter the URL.
      • Username: Enter the username.
      • Password: Enter the password.
      • Issue Key: Enter the key.
    • Webhooks Integration:
      • Extensions Post-Hook: Enter the URL.
    • MSTeams Integration:
      Web-hook URL: Enter the URL.

Microsoft Teams Workflow

The customer needs to create a workflow in their Microsoft Teams account. Upon creating a new workflow, they will receive a unique HTTP POST URL, which should be entered in the orchestration notification settings in Functionize.

Example Webhook URL: 

https://prod-54.westus.logic.azure.com:443/workflows/6fd0396b959443d788128136be704479/triggers/manual/paths/invoke?api-version=2016-06-01&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=x4rkE-_4EHpePwjdtcsosWhduNs32GoFLdv7eAs3ePw

Steps for Microsoft Teams Workflow

  1. They need to select "Body" in the content settings and enter the schema provided below into the "Schema" text box.
    JSON Schema: 
    {
    "type": "object",
    "properties": {
    "title": {
    "type": "string"
    },
    "start_date": {
    "type": "string"
    },
    "end_date": {
    "type": "string"
    },
    "reportUrl": {
    "type": "string"
    },
    "runId": {
    "type": "string"
    },
    "executionTime": {
    "type": "string"
    },
    "projects": {
    "type": "string"
    },
    "failed": {
    "type": "string"
    },
    "passed": {
    "type": "string"
    },
    "incomplete": {
    "type": "string"
    },
    "warning": {
    "type": "string"
    },
    "total_executed": {
    "type": "string"
    },
    "testcases": {
    "type": "string"
    },
    "status": {
    "type": "string"
    },
    "testcasePassedUrl": {
    "type": "string"
    },
    "testcaseFailedUrl": {
    "type": "string"
    },
    "testcaseWarningUrl": {
    "type": "string"
    }
    }
    }
  2. Now, they need to add a step to post a card to the channel. Here, they should select their team and channel settings, then enter the JSON provided below into the Adaptive Card text area.

    Adaptive Card Json: 
     {
    "type": "AdaptiveCard",
    "body": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "@{body('Parse_JSON')?['title']}"
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Start Time:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['start_date']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "End Time:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['end_date']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Total Execution Time:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['executionTime']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Run Id:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['runId']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Download Report:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['reportUrl']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Projects:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['projects']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Execution Results"
    }
    ],
    "width": "auto"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Failed:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['failed']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Warning:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['warning']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Passed:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['passed']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Incomplete:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['incomplete']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Total Executed:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['total_executed']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Status:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['status']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Projects:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['projects']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Test Cases:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['testcases']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Passed Test Details:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['testcasePassedUrl']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    },
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Failed Test Details:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['testcaseFailedUrl']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    },
    ]
    },
    {
    "type": "ColumnSet",
    "columns": [
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "size": "Medium",
    "weight": "Bolder",
    "text": "Warning Test Details:"
    }
    ],
    "width": "auto"
    },
    {
    "type": "Column",
    "items": [
    {
    "type": "TextBlock",
    "text": "@{body('Parse_JSON')?['testcaseWarningUrl']}",
    "wrap": true
    }
    ],
    "width": "stretch"
    }

    ]
    }
    ],
    "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
    "version": "1.5"
    }
  3. Once the flow is set up, we get the webhook URL and now the new webhook URL can be use in the orchestration settings and An alert will be sent to the channel when the orchestration run completes.


Advanced Tab (Optional):

        • Run Order: Select Parallel (default) or Sequential.
        • Orchestration Type: Choose Single.
        • Run without proxy: Run the Orchestration without Proxy.
        • Re-Run: Automatically re-run incomplete or failed test cases.
        • Email Alerts: Enter email addresses.
        • Email Note: Add a message.
        • Environment Management: Override URLs.
        • Proxy Setting: Enter proxy server details (IP).
        • Windows Resolution: Set resolution (Width and Height in pixels).
        • Alerts Delivery: Choose alert delivery method.
        • Webhook URL: Input webhook URL if applicable.
        • Submit Linked Tests to TCM: Select On Demand or Automatic submission.

Cookies Tab (Optional):

    • Click + Add new cookie and enter Parameter and Value details.

Headers Tab (Optional):

    • Click + Add New Header and enter Parameter and Value details.

Environment Configuration:

    • Select the Environment Configuration Value from the Dropdown

Integration Tab: 

    • Select the test plan and test suit for Azure integration.

Confirm Test Cases:

    • Click Confirm and then Submit.

Execute Orchestration:

    • Locate the orchestration in the list, use the Search field if necessary, and click the Start button.

    • In the Run Orchestration window, click Confirm.

Creating a Group Type Orchestration

  1. Open Orchestration Tab:
    • Click on Create New Orchestration.
  2. Details Tab:
    • Follow the same steps as for a Single Type Orchestration.
  3. Integrations Tab:
    • Add and configure relevant integrations as per the Single Type Orchestration.
  4. Advanced Tab:
    • Run Order: Select Parallel or Sequential.
      • Parallel ( Default ): Select this to run the tests simultaneously
      • Sequential: Select this to run the test sequentially
    • Orchestration Type: Choose Group.
    • Run without proxy: Run the Orchestration without Proxy.
    • Re-Run: Automatically re-run incomplete or failed test cases.
    • Email Alerts: Enter email addresses.
    • Email Note: Add a message.
    • Environment Management: Override URLs.
    • Proxy String: Enter proxy server details (IP).
    • Windows Resolution: Set resolution (Width and Height in pixels).
    • Alerts Delivery: Choose alert delivery method.
    • Webhook URL: Input webhook URL if applicable.
    • Submit Linked Tests to TCM: Select On Demand or Automatic submission.
  5. Click Confirm:
  6. Group Orchestration Test Cases:
    • Click Add New Group to create groups of test cases.
  7. Finalize Groups:
    • Select the Run Order for each group.
    • If selecting Sequential, a checkbox for Halt on Failure appears.
    • Organize groups with specific run orders, then click Submit.
  8. Execute Orchestration:
    • Locate the orchestration in the list, use the Search field if necessary, and click the Execute icon.
    • In the Run Orchestration window, click Confirm.

Creating an Orchestration Using Test Data Management

Please see User Guide: Test Data Management (TDM) - Orchestrations




 

Related articles