Test Case Execution Overview
Functionize's Test Cloud allows testers to provision or de-provision virtual machines (VMs) as needed to run as many tests as their subscription allows, across as many browsers as required, without the need for manual infrastructure setup. In this tutorial, you will learn how to execute a test case (or multiple test cases) within the same project in Functionize.
A test case can be executed in a variety of ways:
-
By default, a test case runs in the background, and you are unable to view the live interaction with the browser
-
Live Execution allows users to see how the test is interacting with the browser while it is running on the VM in the Functionize Cloud
-
Live Debug allows users to interact with the test while it is running - during a Live Debug execution, the user can pause the test, set breakpoints (to run the test to a certain step), and make edits to the test
-
Run with Variables is for test cases that include variables - if the user chooses this option, a pop-up will appear before the test case begins executing, allowing the user to update or override the variable values and runs the test case in the background
-
Headless Run runs the test case in the background, enabling tests to run faster and with less infrastructure, this makes it easier to perform tests early in the development cycle and find and fix bugs more quickly
How to Execute a Test Case in Functionize
View Project and select the test case(s) to execute. It is possible to execute a single test case by Browser or multiple test cases within the same project through the Bulk Action drop-down on the Test Listing page.
To Execute a Single Test Case By Browser
- Hover over the browser to execute the test case on
- Click Run Test for that browser
To Execute Multiple Test Cases by Bulk Action within the same project
- Select the test case(s) to be executed by checking the checkboxes to the left of each test case
-
Go to Bulk Action dropdown
- Click Run Tests
For more information on Bulk Actions, click HERE.
To Execute Multiple Test Cases in multiple projects see >> Create an Orchestration.
Live Execution
Live Execution can be used to run a test by watching the execution live inside the VM, no interaction with the test execution is allowed with this execution method. This provides a real-time view into the test execution experience, so the user can see what is happening and get instant feedback.
Note: Live Executions run a little slower than the default, background runs.
How to Perform Live Execution
- Click on the name of the TC, to be redirected to the Test Details page
-
The different ways to execute a test are displayed when you click on the Execute Test button
- Click on the Live Execution option of the test case and then click on Watch Now button to watch the live execution of the test case on a newly provisioned VM in the Functionize Cloud
- Once the Live Execution begins, the VM where the test runs starts provisioning - this VM start up can take up to a few minutes
-
When the VM is ready, the test will show a step-by-step live execution of your test on the fresh VM provisioned in the Functionize Cloud
Live Debug
Live Debug can be used to interact with the test while it's running by setting breakpoints to pause the test and make edits to the test. This provides a real-time view into the test execution experience, so the user can see what is happening and get instant feedback, by pulling up Chrome Dev Tools to check for Console and/or Network errors.
Note: Live Debugs run a little slower than the default, background runs.
How to Perform Live Debug
Please see detailed Live Debug execution method.
Helpful Hint for Using Live Execution and Live Debug
For the quickest launch of Live Execution or Live Debug tests, let Functionize automatically select the first available Google Data Center region in which to run the virtual machine. Generally, it will be run in a US Google Data Center region, unless the user specifies a specific region under the Test Actions button > Test Settings > Advanced tab. We have the largest pool of resources available when that Test Settings Region field is left blank.
Updating Your Region
- On the Test Details page, click on Test Actions >> Test Settings
- In the Test Settings window, open the Advanced tab
- If the Region field displays 'Select Region', the test case is all set, the test case will automatically be set up to run in the region with the greatest capacity
- If the Region field displays a specific region, and the test is not dependent upon a specific location, open the drop-down menu and select the 'Select Region'
- Click on Save
Run with Variables
Run with Variable can be used with test cases that use variables. When selected, a pop-up window appears that allows the tester to update/override the variable value. This option will ultimately execute the test case in the background.
How to Perform Run with Variables
-
Click on the name of the TC, to be redirected to the Test Details page
-
The different ways to execute a test are displayed when you click on the Execute Test button
- Click on the Run with Variables option of the test case and then in the Run with Variables pop-up window, update/override the necessary Variable(s) and click the Run button
-
Once the Execution begins, the VM in the background, where the test run starts Provisioning and ultimately will change to Processing as the test case executes
Headless Run
Headless Run executes the test case in the background without any view into the live interaction with the browser. Since the test case runs in the background, it enables the tests to run faster and with less infrastructure, this makes it easier to perform tests recieive results more quickly.
How to Perform Headless Run
-
Click on the name of the TC, to be redirected to the Test Details page
-
The different ways to execute a test are displayed when you click on the Execute Test button
- Once the Execution begins, the VM in the background, where the test run starts Provisioning and ultimately will change to Processing as the test case executes