The biggest pain point in test case maintenance is failed test cases. In fact, we did an entire webinar on it! Functionize eliminates the need for any scripted maintenance by encompassing two important features in our platform:
- A diagnosis that is assisted by Artificial Intelligence (AI).
- Actual fixing of the failure using AI/Machine learning.
Functionize accomplishes this in one of two ways:
- Self-healing tests using a set of rules.
- Root cause analysis (RCA) engine applying AI learning with a 1-click solution.
Self Healing Tests
Self-healing is completed in the cloud at the time of test execution - if there is a failed test case in Functionize that means the test case hasn't "self-healed" for one reason or another and human input is required. Typically human input is required if the site has changed significantly, the workflow has changed, but the test hasn't been updated, or there is a valid bug.
Note: Selenium scripting captures only one data point (XPath) for every element. Functionize identifies not just a single selector but 100's of data elements, which is the basis for machine learning. Functionize collects all of the data and analyzes it to find discrepancies and changing patterns over time for the purpose of rating, ranking and categorizing to result in the elimination of selector maintenance altogether.
Diagnosis and Maintenance
- Go to any Project
- When viewing a project, the Functional Tests tab will be the main page.
- Locate the failed test case after it has been executed. See the example below, highlighting all passed/failed test cases.
- Red = failed test case
- Green = passed test case
- Next to a failed test case, click the Actions button.
- Select Details
The test steps will be displayed. If the test was created with NLP or the test had text-based steps added while Architecting, that text along with the executed test steps will all be displayed.
Passed steps are highlighted in green, failed steps in red.
If a verification fails, the test will continue executing so there may be a series of passed steps, a failed step, and then additional passed steps. This is expected and allows maximum value to be extracted from test results.
Scroll down to the failed step. Click the screenshot on a test step a few steps prior to the failed step. This will offer a more inclusive context to the reason for the failure.
Using the blue arrows, click through to the failed step, looking for visual clues of the failure. These can include error messages, page not loading, or already registered accounts.
Stop at the failed test step screenshot. Use the information shown on and around the screenshot to diagnose the test case In the example shown below, Functionize is highlighting an execution error.
- On the left-hand side, information about the failed step is displayed.
- On the bottom, the specific operator and verification type is outlined in addition to what data was found on previous executions of this step, which in this case are the same.
Previous Successful Run (PSR)
Functionize also (and most importantly) offers the ability to view the previous successful run along with multiple screenshots per step, allowing the user to visually compare what passed with the failed execution. This can be accomplished across multiple steps and is accessible at the top right of any test step.
NOTE: Previous Successful Run is possibly the most important and useful diagnose tool offered because it allows the comparison of failed execution to a passed execution and to easily see the visual difference between the two.
In the screenshot below, P stands for Previous and C stands for Current. Toggling between the two will show the screenshots for the corresponding execution. Look at multiple steps leading up to the failed step to see when screenshots start to diverge.
Learn more about test diagnosis in this webinar.