Functionize and Rally Integration Overview
Functionize integrates with Rally to provide users and teams with more visibility into automated test results. After the integration is set up, you can link a test from Functionize to the corresponding test case in Rally.
Why Integrate with a Test Management Tool?
Automation coverage and test results are more valuable when shared with a wide audience, especially in large organizations. Viewing Functionize test results in Rally allows you to report both manual and automated test results from a single place.
Why Add Functionize to Rally?
Functionize enables you to easily automate your manual Rally tests without code using Architect. With the integration, you can effortlessly access linked information from both tools and click on hyperlinks to navigate back and forth between them.
High-Level Integration Flow After Setup:
- Rally manual test is created and ready to be automated.
- Create the test in Functionize using Architect.
- Link the automated Functionize test with the Rally manual test.
- Create an orchestration in Functionize using the linked test.
- Execute the Functionize orchestration as part of a release test cycle.
- Submit the results into Rally, which can be done automatically depending on the integration settings in Functionize.
- View the orchestration results in Rally.
Initial Setup
-
Open the Team Page:
- Navigate to the Team page, then the Integrations tab.
-
Select Rally:
- On the left, select Rally from the Test Case Management section on the left side panel.
- In the main panel, provide credentials for the connection, enter the URL for your Rally project and your Rally API Key.
- Click Test Sign In and the status next to the button should update when ready.
- On the left, select Rally from the Test Case Management section on the left side panel.
-
Project Mappings:
- Scroll down to the Project Mappings section.
- Select the Functionize and Rally Workspaces, Projects, and Sub-projects you wish to link together.
- Choose the appropriate Project Environment to be used and map the Functionize Test Statuses to the Rally Test Statuses.
- First, select a Workspace.
- Then select a Project or a Sub-Project that is nested in the folder.
- Multiple projects may be added, but each must be mapped individually using the Add Project button.
- Click Save Mapping to save your integration connection details.
Linking Test Cases via Project Page
-
Open Functionize Project:
- Open the Functionize Project you have linked with Rally.
- On the left panel, confirm you have selected the Project Environment previously mapped with your Rally Project.
-
Link Test:
- The list of tests in the main panel will display the Rally logo + Link Test, click + Link Test.
- Select the corresponding Rally test case to link to and then click Link Test.
- Once a test is mapped, the Rally test name will be listed in the place of Link Test along with a pencil icon to edit the connection at any time.
- The list of tests in the main panel will display the Rally logo + Link Test, click + Link Test.
Linking Test Cases via Test Detail Page
- Open Test Detail Page:
- Open the Test Detail page for a test.
- The Rally logo + Link Test will be displayed to the right above the test actions. Click + Link Test.
- Link Test:
- Select the corresponding Rally test case to link to and click Link Test.
- Once a test is mapped, the Rally test name will be listed in the place of Link Test along with a pencil icon to edit the connection at any time.
Note: If you select a Rally test case which has already been linked to a Functionize test, you will receive a warning that the link will be overridden by the new connection.
- Select the corresponding Rally test case to link to and click Link Test.
Sending Results to Rally
-
Create Orchestration:
- Open the Orchestrations tab, then click + Create New Orchestration.
- Enter the relevant information on the Details tab.
- On the Advanced tab, select Submit Linked Test to TCM for how you would like results of each Orchestration run sent to Rally.
- Click Confirm.
- Confirm the test cases to include and click Submit.
- Open the Orchestrations tab, then click + Create New Orchestration.
-
Execute Orchestration:
- Execute the Orchestration either On Demand, on a Schedule, or via CI/CD Pipeline.
- Open the Orchestration Results, under the Actions column click the Results button.
-
Submit to Rally:
- If you selected 'On Demand' for Submit Linked Test to TCM during Orchestration creation, then on the Orchestration Results page, click the Submit to Rally button.
- A toast message will appear to confirm the results have been sent to Rally and a link to the execution will be visible under the Rally Fix Version field.
Note: If you select an 'Automatic Submission' during orchestration creation, the execution link will be visible when you open the Orchestration Results page with no button for manual submission.
- If you selected 'On Demand' for Submit Linked Test to TCM during Orchestration creation, then on the Orchestration Results page, click the Submit to Rally button.
Rally Execution
We've implemented two-way linking between Functionize and Rally, enabling you to see execution result details in both Rally and Functionize Orchestrations.
This includes links to each connected test in both Rally and Functionize. All test results from a single Orchestration will be added to a single Test Execution in Rally, encompassing cross-browser and multiple dataset testing. If any configuration of a test case fails (any browser, any dataset), the linked test case within Rally will be marked as a failure within that Rally Test Execution.
Note: If a test in Functionize is included in the orchestration but is not linked to a corresponding test in Rally, the results table will provide information, but it will not be included in the Overall Execution Status section.