Using Visual Verifications

Functionize leverages Computer Vision (CV) powered by machine learning to perform visual verifications of your applications. This ensures that UI changes are detected intelligently, reducing false negatives from trivial rendering differences (like pixel shifts between browsers).

A visual verification step consistently fails because a baseline image has not been established. This can occur even after the test case containing the step has been executed multiple times.

There are two ways to add visual verifications:

• Full Page Verification (Visual Page Check / VisPageChk)
• Element-Based Verification (Visual Check)

Shared Concepts

• Match Requirement %
  • Default: 92% per project.
  • Controls tolerance for pixel differences.
  • Can be adjusted globally (Project Settings > Advanced) or per action while creating or editing a test case.
• Baselines
  • First successful execution sets the baseline image.
  • Baselines are browser-specific (Chrome, Firefox, etc.) to avoid cross-browser mismatches.
  • Can be updated manually after any execution.
• Comparison Types
  • Baseline Comparison: Compare against saved baseline image.
  • Previous Step Comparison: Compare against the immediately prior step’s image.
• Conversion to Normal Verification
  • Visual verifications can be changed to normal verifications by unchecking the Visual Verification checkbox in the test step settings.

Full Page Verification (Visual Page Check)

Purpose: Verify an entire page screenshot, including off-viewport content.

• Types:
  • Baseline Verification → compares current screenshot with baseline.
  • Previous Step Verification → compares screenshot with a previous step’s capture.
• Adding a Full Page Visual Check
  • In Architect, select + → Advanced Actions → Visual Page Chk.
  • Set Visual Match Requirement % (default pulled from project).
  • Save → A VisPageChk Action is added to the test flow.
• Alternative Ways to Insert
  • Test Detail Page → Insert → Full Page Check Action.
  • Slider View → Quick Add → Insert → Full Page Check Action.
• Viewing Results
  • Access results via Slider View.
  • See Current vs Baseline images side by side.
  • Tools available:
    • Ignore Regions → Exclude defined areas from checks.
    • Modified Areas → Highlight changed regions.
    • Compare Regions → Narrow checks to specific regions.
    • Basic Pixel Comparison → Override ML for strict pixel matching.
• Limitations
  • Screenshots begin at current scroll position → use Scroll Action in Architect to reset to top.
  • Very large or complex pages may only capture partial screenshots. The same may be true in cases where some elements are larger than the viewport.
  • Currently supported only on desktop browsers.

Element-Based Verification (Visual Check)

Purpose: Verify a specific UI element (image, div, span, logo, etc.).

• Types:
  • Baseline Verification → Compares the element to the saved baseline.
  • Previous Step Verification → Compares the element against the same element in a previous step.
• Creating a Baseline Element Verification
  • Launch Architect.
  • Click the eye icon (Visual Verification).
  • Select the element (must fully fit in viewport).
  • In the Visual Check modal:
    • Choose Baseline Comparison (default).
    • Set Match Requirement %.
  • Save → The action appears with an eye icon in the test steps.
• Creating a Step Comparison Verification
  • Follow the same process as Baseline until modal appears.
  • Choose Step Comparison.
  • Select which previous step to compare against.
  • Adjust Match Requirement % if needed.
  • Save.
• Adding Visual Checks to Existing Verifies
  • In Architect, click the white dot next to a Verify Action.
  • Select Visual Check from the side panel.
  • Configure & Save.

Viewing Results

• Within Architected Test Case
  • Hover over the Visual Check tag to view:
    • Current run image (right)
    • Baseline image (left)
    • Actual vs Expected Match %
    • Match Requirement %
    • Baseline image for the browser
• Baseline Management
  • Click Edit Match Requirement → Update threshold for future runs.
  • Click Update Baseline → Replace with current image.
  • From Actions Log:
    • View all baseline tags.
    • Hover to preview baseline image and settings.

Default Settings

• Element Match Requirement %
  • Project Settings → Advanced Tab → Element Match Requirement Default %.
• Full Page Match Requirement %
  • Project Settings → Advanced Tab → Full Page Visual Check Default %.

Visual Verification Cheatsheet 

Why Use Visual Verifications?

• Ensure UI looks correct across browsers.
• Catch visual regressions that functional verifications can miss.
• Train ML baselines to understand intended designs, not trivial rendering differences.

When to Use

• Full Page Verification (Visual Page Check)
  • The whole page must remain consistent.
  • Good for layouts, dashboards, landing pages.
  • Catch visual regressions across the entire screen.
• Element-Based Verification (Visual Check)
  • Focus on a specific logo, button, banner, image, or container.
  • Best for branding elements (e.g., company logo), or UI components that must remain pixel-perfect.

Types of Visual Verifications

Best Practices

• Use Baselines per browser → avoids false negatives.
• Default Match Requirement % = 92% (can adjust per test or project).
• Place verifications early & often (fail fast).
• Use static & unique elements (avoid divs with multiple children).
• Update baselines only after confirming intended UI changes.

Quick How-To

Full Page Verification

1. Architect → + → Advanced Actions → Visual Page Chk
2. Set Match % → Save.
3. Review in Slider View (Baseline vs Current).

Element Verification

1. Architect → Eye icon on element.
2. Choose Baseline or Step Comparison.
3. Adjust Match % if needed → Save.

Viewing Results

• Hover Visual Check tag → See Baseline vs Current side by side.
• Adjust Match % or Update Baseline when appropriate.
• Manage defaults in Project Settings > Advanced Tab.

Limitations & Tips

• Full-page screenshots start from current scroll → use Scroll Action to reset.
• Extremely long/complex pages may capture partially.
• Currently desktop browsers only.
• Ignore Regions & Compare Regions help isolate relevant UI.

Rule of Thumb:

• Full Page = “Is the entire UI consistent?”
• Element = “Is this specific thing correct?”