Functionize integrates with Xray to provide Xray users and teams more visibility into automated test results. You can integrate Functionize with Xray, no matter how Xray is deployed with Jira: Cloud, Server, and Data Center.
Why integrate with a test management tool? Automation coverage and test results are more valuable when shared with a wide audience, especially in large organizations. Seeing Functionize test results in Xray allows you to report both manual and automated test results from a single place.
Why add Functionize to Xray? Functionize allows you to easily automate your manual Xray tests without code using Architect. Using the integration, you can easily see linked information from both tools and click on hyperlinks to easily jump back and forth between them.
Here is the high-level integration flow after setup:
- Xray manual test is created and ready to be automated.
- Create the test in Functionize using Architect.
- Link the automated Functionize test with the Xray manual test.
- Create an orchestration in Functionize using the linked test.
- Submit the results into Xray. (This can be done automatically depending on the integration settings.)
- See the orchestration results in Xray.
Initial Set up
- Open the Team Page, then the Integrations Tab.
- On the left, select Xray from the Test Case Management section.
- In the main panel, provide credentials for the connection. Select Cloud or Server/Data Center.
- Enter the URL for your Jira project, your Jira username, and your Jira API token (not your password). For Jira API token assistance, read here: Manage API tokens for your Atlassian account
- Enter your Xray Client ID and Client Secret. These can be found in your Jira account by a Jira Admin by accessing Apps > Manage Apps > API Keys For further assistance read here: Global Settings: API Keys
- Click Test Sign In and the status next to the button should update to Connection established with a green check mark when ready.
- Scroll down to the Project Mappings section. Here you will select the Functionize and Xray projects you wish to link together and select the appropriate testing environment to be used. Map the Functionize test statuses to the Xray test statuses as you would like them reported.
- Multiple projects may be added, however each must be mapped individually.
- Click Submit to save your connection details.
Linking Test Cases
- Open the Functionize project you have linked with Xray. On the left panel, confirm you have selected the testing environment previously mapped with your Xray project.
- The list of tests in the main panel will display the Xray logo + Link Test.
- Click Link Test to open the Xray Linking window.
- Select the corresponding Xray test case. > Click Link Test.
- Once a test is mapped, the Xray test name will be listed in the place of Link Test along with a pencil icon to edit the connection at any time.
Sending Results to Xray
- Open the Orchestrations page, then click Create New Orchestration.
- Enter the relevant information on the Details Tab.
- On the Advanced Tab, select how you'd like results of each Orchestration run sent to Xray.
- You may also select an Xray Test Plan to connect with this Orchestration; select it from the Xray Test Plan drop-down menu.
- Click Add New.
- Confirm the test cases to include and click Submit.
- Click the Play button in the Execute column to start the Orchestration, then click Run Orchestration in the confirmation modal.
- Open the Actions menu and select Orchestration Results.
- When the Orchestration has completed, click Submit to Xray and a modal will open to show the name of the Xray Execution and a link to the execution will be visible under the Submit to Xray button. NOTE: This is for when you have selected "On Demand" for Submit Linked Tests to TCM in your Orchestration creation.
- This is also an opportunity to connect the Orchestration to an Xray Test Plan if one was not selected during creation. NOTE: This will update the connection for future runs of this Orchestration. It will not connect the results of the executions run prior to the test plan being connected.
We've implemented two-way linking between Functionize and Xray which enables you to see execution result details in Xray and Functionize Orchestrations as well as links to each connected test in both Xray and Functionize. All test results from a single Orchestration will be added to a single Test Execution issue in Xray including cross browser and multiple dataset testing. If any configuration of a test case fails (any browser, any dataset), the linked test case will be marked as a failure within that Test Execution.
Please note that if a test in Functionize is included in the orchestration but is not linked to a corresponding test in Xray, the results table will provide information, but it will not be included in the Overall Execution Status section.