Fixing Failed Test Cases Overview
Once a user has determined the reason for the problem through Debugging and Diagnosing, there are many methods for updating the failed test, including suggesting updates, smart screenshots, live debugging, action settings, test executors, selectors, etc.
SmartFixFunctionize's ML will suggest ways to fix the test with a one-click solution. This can help update the test when the wrong element is selected, when it appears that you may have entered an incorrect password, or to update verifications.
See User Guide: SmartFix
See Video: SmartFix
Update Action Setting FlagsEach action in a test has certain Settings, Flags, and Information associated with it. These can be accessed via the Test Details page or Slider View. A complete list of Action Types and the Settings available to each is linked below.
There are times when pop-ups unexpectedly appear, and the user may desire an Optional Action. Alternatively, if a website has changed in one environment, there might be a need to Skip an action to prevent it from running.
See User Guide: Action Settings
See User Guide: Optional and Skipped Actions
See User Guide: Action Types
Update the ExecutorIf there is a click that didn't actually click, or an input that didn't input text as expected, the user will can change the Executor.
See User Guide: Customizing Executors
See Video: How to Update an Executor
Update the Selection MethodFunctionize's default ML selection is at least 99.9% accurate with element selection, but it does sometimes find the wrong element, instead of failing the action, especially when clicking on elements. This can mean we select the wrong element on occasion however the ability to overrule this is available via our Selectors Override feature.
See User Guide: Customizing Tests with Selectors
Force Fail a Passing TestThere are times when we've attempted to Self-Heal for the user, and it didn't perform as expected. However, if the test lacks a verification following the clicks that is unique to the new page or new area on the page, the test can yield a False Positive. In such cases, the test case likely displays a Self-Heal tag. When you encounter these tags, you can manually mark the test as failed. This ensures that the ML model does not mistakenly learn that a Click was correct when the test is re-executed.
See User Guide: Force Fail a Passing Test Case
See Video: Force Fail a Passing Test Case
Live DebugWe recommend Live Debug as the most robust method to update a test case. Live Debug lets you interact with a test while it is running on Virtual Machines on the Functionize Cloud. This means you can quickly diagnose test failures or modify tests in a clean execution environment.
See User Guide: Live Debug
Local EditWhen performing a Local Edit, all of the ML data used to execute the test is recreated. Local edit allows you to use Architect on your machine to make changes in test flows, element selections, and verifications.
See User Guide: Local Edit
Submit a Support TicketIf you are unable to resolve your test failure with above options, please reach out to the Functionize Customer Support team.
See User Guide: Formatting and Submitting Support tickets