Force Fail a Passing Test Case Overview

Self-healing helps the user save countless hours fixing broken test cases. However, in certain circumstances, this may result in a false positive or a test case that passed when it should have been considered a failed test case. In the context of testing, a false positive occurs when a test indicates success (pass) even though there is an actual defect or issue present. False positives should be corrected so that the machine learning model can learn and ensure that tests are passed or failed correctly when executed again in the future. This corrective action is called Force Fail, which gives the user the ability to manually fail a test that has passed.

Causes of Failed Test Cases

The most common reason for false positives is incorrect and/or missing verifications in a test case.

How to Force Fail a Test Case

Whether a missing verification is the culprit for your false positive or some other reason, a user has the ability to Force Fail a test case to intentionally mark a test as failed and ultimately correct the false positive.

Functionize provides two different options to Force Fail a test case:

Force Fail on the Test Detail Page

1. Open the Test Details page
2. Expand the action to be Force Failed in order to view the Settings tab
   
   
   
3. Click the Fail button
   
   
   
4. Click the Fail button in the Confirm model
5. The test action will change to Failed status with the Execution Error message "Failed manually by User on Date"
   
   
   
6. Click the Save button.

Force Fail on the Slider Page

1. Open the Test Details page
2. Click on the View Results button or click the View button to the right of any Action
   
   
   
3. Click on the gear icon for the specific step to fail
   
   
   
4. Click the Fail button
   
   
   
5. Click Fail in the Confirm model
   
   
   
6. Once the failure has been confirmed, the overall test is marked failed with an indication that the test was manually failed
   

For further instructions, please see a video example of Force Fail.

Note: When Force Fail is used, in certain situations, a SmartFix suggestion may be provided to recommend how the test can be optimized for future executions. As an example, alternate suggestions or elements may be offered to select on the page. Should that SmartFix suggestion be accepted (e.g. selecting a different element on the page), then in future executions, the test will check for that element instead. These SmartFix suggestions offer another peek into our machine learning capabilities that are continually looking for ways to optimize tests. Using the Force Fail feature and modifying the test case with proper verifications will allow the user to better run tests in the future.

Please see the following User Guide on SmartFix for more information.