Fixing Failed Test Cases

Fixing Failed Test Cases Overview

Once a user has determined the reason for the problem through Debugging and Diagnosing, there are many methods for updating the failed test, including suggesting updates, smart screenshots, live debugging, action settings, test executors, selectors, etc.

1. SmartFix
   

   Functionize's ML will suggest ways to fix the test with a one-click solution. This can help update the test when the wrong element is selected, when it appears that you may have entered an incorrect password, or to update verifications.
   
   
   See User Guide: SmartFix
   See Video: SmartFix
   
   

2. Update Action Setting Flags
   

   Each action in a test has certain Settings, Flags, and Information associated with it. These can be accessed via the Test Details page or Slider View. A complete list of Action Types and the Settings available to each is linked below.
   
   There are times when pop-ups unexpectedly appear, and the user may desire an Optional Action. Alternatively, if a website has changed in one environment, there might be a need to Skip an action to prevent it from running.
   
   
   See User Guide: Action Settings
   See User Guide: Optional and Skipped Actions
   See User Guide: Action Types
   
   

3. Update the Executor
   

   If there is a click that didn't actually click, or an input that didn't input text as expected, the user will can change the Executor.
   
   
   See User Guide: Customizing Executors
   See Video: How to Update an Executor
   
   

4. Update the Selection Method
   

   Functionize's default ML selection is at least 99.9% accurate with element selection, but it does sometimes find the wrong element, instead of failing the action, especially when clicking on elements. This can mean we select the wrong element on occasion however the ability to overrule this is available via our Selectors Override feature.
   
   
   See User Guide: Customizing Tests with Selectors
   
   

5. Force Fail a Passing Test
   

   There are times when we've attempted to Self-Heal for the user, and it didn't perform as expected. However, if the test lacks a verification following the clicks that is unique to the new page or new area on the page, the test can yield a False Positive. In such cases, the test case likely displays a Self-Heal tag. When you encounter these tags, you can manually mark the test as failed. This ensures that the ML model does not mistakenly learn that a Click was correct when the test is re-executed.
   
   
   See User Guide: Force Fail a Passing Test Case
   See Video: Force Fail a Passing Test Case
   
   

6. Live Debug
   

   We recommend Live Debug as the most robust method to update a test case. Live Debug lets you interact with a test while it is running on Virtual Machines on the Functionize Cloud. This means you can quickly diagnose test failures or modify tests in a clean execution environment.
   
   
   See User Guide: Live Debug
   
   

7. Local Edit
   

   When performing a Local Edit, all of the ML data used to execute the test is recreated. Local edit allows you to use Architect on your machine to make changes in test flows, element selections, and verifications.
   
   
   See User Guide: Local Edit
   

8. Submit a Support Ticket
   

   If you are unable to resolve your test failure with above options, please reach out to the Functionize Customer Support team.
   
   See User Guide: Formatting and Submitting Support tickets