TestRail Integration

Functionize and TestRail Integration Overview

Integrate Functionize test automation with TestRail to allow automated test results from Functionize to be visible in TestRail for reporting purposes. This native integration allows you to connect to TestRail Cloud. After the integration is setup, you can link a test from Functionize to the corresponding test case TestRail. Then, when the test is executed through an orchestration in Functionize, the orchestration results will populate into a TestRail test run. From both tools, you can see fields that are mapped, including the Status and ID. You also have a hyperlink to navigate easily between linked objects across the two tools.

Setting up the Integration

To set up the native integration, you can configure the project mapping in the Functionize Team's Integration settings. Administrative permissions are required in Functionize to configure the integration from TestRail.

Note: We currently only support TestRail Cloud and the project type: 'Single Repository for all cases' (see image below).

Configure TestRail

1. Click on Administration from the top right corner, select Site Settings from the right navigation panel and then select the API tab
   
   
   
2. Navigate from Administration >> Customizations >> Result Fields, click on the + Add Field to enter the new custom field for Functionize Result Link, will be used to insert a direct link from the TestRail test run back to the Functionize Test Result on every execution
   
   
   
3. In the Add Custom Field page, fill out the field details as described and shown in the image below:
   
   

• • Enter the Label of Functionize Result Link
  • Enter the System Name as functionize_test_result_link
  • Select Type as Url (Link)
  • The Description field is optional
  • Select the templates and projects appropriate for any project to link to Functionize

Configure Functionize 

Navigate to Integration Settings

1. As a Functionize Team Administrator, navigate to the Team tab
   
   
   
2. Click on the Integrations tab, select the TestRail radio button from the left menu underneath Test Case Management section
   
   
   

Establish the Connection

3. Under Test Authentication/Connection, enter the credentials for your TestRail Cloud instance:

• • URL
  • Username
  • Password
  • Timeout(secs)
  • Test Plan Check
  • Proxy URL and Port fields are only required if a proxy connection is needed, otherwise skip these fields

Note: These credentials must be for a TestRail Administrator since the integration will generate custom fields into TestRail once the connection is established.

4. Click the Test Sign In button to establish the connection
   
   

Map Projects and Statuses

5. Set up the Functionize project(s) under Projects Mappings section
   
   
   
6. Select the Functionize Project to link, along with the Project Environment
7. Select the TestRail Project from the dropdown menu
   
   
   
8. Map the values for Functionize Status to the TestRail Status values, Functionize's default status values are: FAILED, INCOMPLETE, WARNING, and PASSED and for the integration to work properly, these statuses should be mapped to status values in TestRail
   
   Note: Statuses can be customized for the TestRail status values within TestRail. See how in the section below.
   
   
9. Click Save Mappings
10. A confirmation message will display indicating the mapping was successful
    
    
    

Optional: To Customize the Available TestRail Statuses

1. Log into TestRail as an administrator
2. Click on the Administration tab on the far right
3. Click on Customizations from the right navigation panel
4. Click on the Statuses tab
   
   
   
5. Customize the Status Names as desired, in this example, we are naming the Unnamed 2 status to Warning
   
   
   
6. Confirm This status is active and then click Save Status
7. Now the new TestRail Status will be available in the dropdown menu for TestRail Statuses in the Project Mappings configuration, see below:
   
   
   

Using the Functionize and TestRail Integration

Functionize + TestRail Integration Flow Overview

Once you have completed setting up the integration, you can now use it. This integration is designed to make the best use of both tools. When manual tests from TestRail are ready to be automated, you can easily automate those tests using Architect. To learn how to create automated tests using Architect, check out this guide: Getting Started with Architect.

Once the automated test is created using Architect, you can link the Functionize test to the corresponding TestRail test case. Next, run the test using a Functionize orchestration, and the Functionize orchestration results will be populated in TestRail for viewing. Through this workflow, you will be able to navigate easily between both tools using hyperlinks and view the integrated data. See the flow below:

Linking Test Cases

1. Navigate to the linked project in Functionize, and then click on View Project
2. Notice that all tests in this linked project now display the TestRail logo and you can also see the ability to link each individual test to a TestRail test case, as indicated by the +Link Test hyperlink
   
   
   
3. Find the Functionize test to link to TestRail, then, click on the + Link Test hyperlink
   
4. In the Testrail Integration pop-up window, find the TestRail test case - either type in the TestRail test case name or choose from the folder tree list of test cases, select the test case using the radio button and click Link Test
   
   
   
5. A 'Test Case mapped successfully' confirmation will display and now the Functionize test case has been linked to the TestRail test case, as shown below:
   
   
   

Running an Orchestration

An orchestration is a grouping of tests, designed to run multiple tests in parallel or sequentially.   The testing results from a Functionize orchestration can then be sent to TestRail so that these testing results can be included in dashboard and management QA reports.  To learn how to create an orchestration within Functionize, check out this guide: Create an Orchestration.

Viewing Results in TestRail

Each time the Functionize orchestration is executed on the Functionize Cloud, it will create a new Test Run in TestRail. The Test Run will show all the test cases that were executed as part of that  Functionize orchestration.

1. In TestRail, navigate into your linked project and then click on Test Runs & Resultsfrom the top navigation bar
2. Click into the test run name from the Test Runs & Results page, by default, the test run will be created by the integration using this syntax:
   

   <Name of Functionize Orchestration>: <Date and Timestamp of Execution>

   
   
   
3. On the Test Run page, the execution results from Functionize orchestration can be seen
   Note: You do NOT need to be a Functionize user to view the details. If you click the Functionize link, you will see the Functionize orchestration and test results in read-only mode. This is particularly useful if you want to share the details with your developers or other non-Functionize users.
4. To view the details of the execution, click on the Functionize links (see image below):
   • Functionize Orchestration Link: Click on this link to view the orchestration result in Functionize
   • Functionize Result Link: Click on this link to view the specific test result in Functionize
     

Enable Functionize Result Link Column

1. To view the Functionize Result Link column for your TestRail tests, you can enable it by clicking into the Columns button:
   
   
   
2. In the Add Column window, select the Functionize Result Link All option and click the Update Columns button to save