Test case maintenance often comes down to quickly identifying why a test failed and applying the right fix. Functionize’s Diagnose Agent is built to accelerate this process. By running diagnostics, you can uncover the root cause of a failure, explore alternative solutions, and apply intelligent fixes to improve test stability.
Note: the option to use Diagnose Agent will only be displayed if the test execution has finished and has at least one failed step.
Accessing the Diagnostics Feature
- Navigate to the Projects tab and open the failed test case you want to diagnose.
- On the test page, locate the Diagnostics button.
- Click it to launch the Diagnose Agent.
Running Diagnostics
- After clicking Diagnostics, a panel labeled “Identifying Solutions…” will appear.
- The Diagnose Agent begins evaluating the failure context, analyzing each step in the test to identify potential issues.
- Data analyzed includes:
- Screenshots (action-level and full page)
- Execution context and configuration data
- Step-level metadata and comparisons
This allows the tool to pinpoint exactly where and why the test deviated from expected behavior.
Reviewing Diagnostic Results
Once analysis is complete, the Diagnostic Results window provides a detailed breakdown of findings:
- Failure step: The exact test action where the failure occurred.
- Expected vs Actual: Clear visibility into what was anticipated versus what happened.
- Root cause indicators: Mismatched data (e.g., text), unexpected UI changes, or timing issues.
- Suggested fixes: Next-step actions generated by the Diagnose Agent.
Diagnostic and Maintenance Options
The Diagnose Agent provides three paths to resolving issues:
1. Train the AI
Send structured feedback about the failure directly to Functionize’s AI.
- Click Train the AI after diagnostics.
- A dialog box opens prompting you to describe the issue. Examples include:
- Wrong element selected
- Page load or timing issue
- Environment-specific behavior
- Optionally, allow Functionize to re-run the test for support purposes.
- Click Submit to send details to the AI team, improving long-term accuracy.
2. Run Maintenance
Automates minor fixes based on diagnostic findings.
- After reviewing results, click Run Maintenance.
- A Maintenance in Progress window appears.
- Once complete, the Maintenance Results window shows suggested changes, each with:
- The identified issue
- How the suggested change addresses it
- Select the most appropriate solution, then click Apply Changes.
- A new test run will execute with the updates to confirm resolution.
- After running, you will get an option to accept the fix or discard it.
- Note: the changes will automatically be applied to the test once the resolution is confirmed and the fix is accepted.
3. Alternative Analysis
Explore different ways the failed step could be resolved.
- Click Alternative Analysis on the failed step.
- The AI generates alternative selectors, actions, or strategies.
- Review the recommendations and apply the most suitable option.
Best Practices for Using Diagnostics
- Verify first: Check that you’re using the correct test data and environment configuration before running diagnostics.
- Start broad, then refine: Use diagnostics first to quickly categorize the failure, then drill down into specific options.
- Combine with debugging tools: Pair diagnostics with screenshots, Slider View, or Live Debug for deeper validation.
- Validate after applying changes: Always re-run the test after applying maintenance or alternative analysis to ensure reliability.
- Train the AI when in doubt: The more feedback provided, the smarter the Functionize ML engine becomes over time.
Summary
The Diagnose Agent is your first stop when a test fails in Functionize. It provides:
- Root cause analysis (expected vs actual).
- Automated fixes through Run Maintenance.
- Alternative paths when the primary approach fails.
- Continuous AI improvement through Train the AI feedback.
By combining these capabilities, you can resolve failures faster, reduce manual effort, and improve the long-term resilience of your test suite.