Fixing Failed Test Cases

Fixing Failed Test Cases Overview

Once a user has determined the reason for the problem through Debugging and Diagnosing, there are many methods for updating the failed test, including suggesting updates, smart screenshots, live debugging, action settings, test executors, selectors, etc.

  1. SmartFix

    Functionize's ML will suggest ways to fix the test with a one-click solution. This can help update the test when the wrong element is selected, when it appears that you may have entered an incorrect password, or to update verifications.

    See User Guide: SmartFix
    See Video: SmartFix

  2. Update Action Setting Flags

    Each action in a test has certain Settings, Flags, and Information associated with it. These can be accessed via the Test Details page or Slider View. A complete list of Action Types and the Settings available to each is linked below.

    There are times when pop-ups unexpectedly appear, and the user may desire an Optional Action. Alternatively, if a website has changed in one environment, there might be a need to Skip an action to prevent it from running.

    See User Guide: Action Settings
    See User Guide: Optional and Skipped Actions

    See User Guide: Action Types

  3. Update the Executor

    If there is a click that didn't actually click, or an input that didn't input text as expected, the user will can change the Executor.

    See User Guide: Customizing Executors
    See Video: How to Update an Executor

  4. Update the Selection Method

    Functionize's default ML selection is at least 99.9% accurate with element selection, but it does sometimes find the wrong element, instead of failing the action, especially when clicking on elements. This can mean we select the wrong element on occasion however the ability to overrule this is available via our Selectors Override feature.

    See User Guide: Customizing Tests with Selectors

  5. Force Fail a Passing Test

    There are times when we've attempted to Self-Heal for the user, and it didn't perform as expected. However, if the test lacks a verification following the clicks that is unique to the new page or new area on the page, the test can yield a False Positive. In such cases, the test case likely displays a Self-Heal tag. When you encounter these tags, you can manually mark the test as failed. This ensures that the ML model does not mistakenly learn that a Click was correct when the test is re-executed.

    See User Guide: Force Fail a Passing Test Case
    See Video: Force Fail a Passing Test Case

  6. Live Debug

    We recommend Live Debug as the most robust method to update a test case. Live Debug lets you interact with a test while it is running on Virtual Machines on the Functionize Cloud. This means you can quickly diagnose test failures or modify tests in a clean execution environment.

    See User Guide: Live Debug

  7. Local Edit

    When performing a Local Edit, all of the ML data used to execute the test is recreated. Local edit allows you to use Architect on your machine to make changes in test flows, element selections, and verifications.

    See User Guide: Local Edit
  8. Submit a Support Ticket

    If you are unable to resolve your test failure with above options, please reach out to the Functionize Customer Support team.

    See User Guide: Formatting and Submitting Support tickets