Xray Integration

Functionize and Xray Integration Overview

Functionize integrates with Xray to provide Xray users and teams with more visibility into automated test results and metrics. You can integrate Functionize with Xray, no matter how Xray is deployed within Jira: Cloud, Server, and Data Center.

  • Why integrate with a test management tool? Automation coverage and test results are more valuable when shared with a wide audience, especially in large organizations. Seeing Functionize test results in Xray allows you to report both manual and automated test results from a single place.
  • Why add Functionize to Xray? Functionize allows you to easily automate your manual Xray tests without code using Architect. Using the integration, you can easily access linked information from both tools and click on hyperlinks to jump back and forth between them easily.

Here is the high-level integration flow after setup:

  1. Create a manual Xray test that is ready to be automated
  2. Create the automated test in Functionize using Architect
  3. Link the automated Functionize test with the Xray manual test
  4. Create an orchestration in Functionize using the linked test(s)
  5. Submit the Functionize orchestration results into Xray, this can be done automatically depending on the integration settings
  6. See the Functionize orchestration results in Xray, which are not available for any existing Xray dashboard and management reports


Initial Xray Set up in Functionize

To set up the native integration, you can configure the project mapping in the Functionize Team's Integration settings. Administrative permissions are required in Functionize to configure the integration from Xray.

  1. Open the Team page, then the Integrations tab

  2. On the left, select Xray from the Test Case Management section
  3. Select Cloud or Server/Data Center
  4. Under Test Authentication/Connection, n the main panel, provide credentials details for the connection:
    • Enter the URL for the Jira project
    • Enter Jira Username
    • Enter Jira API Token - not user password
      Note: for Jira API token assistance, read here: Manage API tokens for your Atlassian account
    • Enter Xray Client ID
    • Enter Xray Client Secret
      Note: Xray Client details can be found in your Jira account by a Jira Admin by accessing Apps > Manage Apps > API Keys. For further assistance read here: Global Settings: API Keys
  5. Click Test Sign In and the status next to the button should update to Connection established with a green check mark when ready

  6. Scroll down to the Project Mappings section where the user will select the Functionize and Xray Projects to link together and select the appropriate Functionize Project Environment to be used - map the Functionize Test Statuses to the Xray Test Statuses

  7. Multiple projects may be added, however each must be mapped individually
  8. Click Save Mapping to save your connection details


Linking Test Cases

  1. Open the Functionize project linked with Xray and on the left panel, confirm the selected project environment previously mapped with the Xray project on the Teams > Integration page
  2. The list of tests on the Test Details page main panel will display the + Link Test

  3. Click the + Link Test to open the Xray Integration window

  4. Select the corresponding Xray manual test case and then click Link Test
  5. Once a test is mapped, the Xray manual test name will be listed in the place of + Link Test along with a pencil and xicon to edit or delete the link at any time

Sending Results to Xray

An orchestration is a grouping of tests, designed to run multiple tests in parallel or sequentially.  The testing results from a Functionize orchestration can then be sent to Xray so that these testing results can be included in dashboard and management QA reports.  To learn how to create an orchestration within Functionize, check out this guide: Create an Orchestration.

  1. Open the Orchestrations page, then click Create New Orchestration

  2. Enter the relevant information on the Details tab

  3. On the Advanced tab, select how the results of each Functionize orchestration run sent to Xray

  4. You may also select an Xray Test Plan to connect this Functionize orchestration, select Xray Test Plan Name from the Xray Test Plan drop-down menu

  5. Click Confirm
  6. Confirm the Functionize test cases to include and click Submit

  7. Click the Play/Re-Run button in the Execute column to start the Functionize orchestration

  8. Then click Confirm in the Re-run Orchestration confirmation modal

  9. Open the Actions menu and select Orchestration Results

  10. When the Orchestration has completed, this is when the user connects the Functionize orchestration to an Xray Test Plan
    Note: This is for when you have selected 'On Demand' for Submit Linked Tests to TCM in your Functionize orchestration Advanced settings.
    Note: This will update the connection for future runs of this Functionize orchestration. It will not connect the results of the executions run prior to the test plan being connected.

  11. Click Submit to Xray and a modal will open to show the name of the Xray Execution and a link to the execution will be visible next to the Xray logo
    Note: This is for when you have selected 'On Demand' for Submit Linked Tests to TCM in your Functionize orchestration Advanced settings.


Viewing Results in Xray

We have implemented a two-way linking feature between Functionize and Xray, enabling you to view test execution result details in both Xray and Functionize orchestrations. Additionally, you'll find links to each connected test in both Xray and Functionize. All test results from a single Orchestration will be consolidated into a single Test Execution issue in Xray, including results from cross-browser and multiple dataset testing. If any configuration of a test case fails (in any browser or dataset), the linked test case will be marked as a failure within that Xray Test Execution.

Please note that if an automated test in Functionize is included in the orchestration but is not linked to a corresponding manual test in Xray, the results table will provide information; however, it will not be included in the overall Text Execution status section of the Xray Test Execution.