Overview
A test step may fail due to an incorrect text validation, reporting that it found different text than what was expected. However, upon reviewing the step's screenshot, the text appears to be correct and matches the expected value. This creates a discrepancy between the failed step details and the visual evidence in the screenshot.
How It Works
This issue can occur when a test is executed using an outdated runtime version. The platform's text extraction mechanism, which reads the text from the application under test, is a separate process from the screenshot capture. An older runtime may contain a bug or an outdated text extraction engine that misinterprets the text on the page. While the screenshot correctly captures the visual state of the UI, the flawed text extraction process reports incorrect data to the validation engine, causing the step to fail.
Limitations
The primary solution is to ensure that tests are running on the latest runtime version. The latest versions include bug fixes and improvements to the text extraction engine, which resolves these types of discrepancies. Re-executing the test on the current runtime will typically lead to a successful pass, as the text will be read correctly.
Related Info
Always verify that your test environments are configured to use the latest available runtime to benefit from the most recent stability improvements and bug fixes.