Verifications are important in any test automation. Without proper verifications some would say a test isn’t even a test. The importance of verifications when testing with machine learning is even more critical.
Selectors are not used when testing with machine learning, which is a fundamental shift in how test automation executes. This advancement allows the automation to be more flexible and to understand incremental changes to the site under test, which drastically reduces maintenance.
What does this mean?
This means our platform will select the most likely element for interactions, like clicks and inputs, based on the state of the page. The automation is not tied to a selector. By design, if the desired element is not on the page or the page has changed significantly, the platform will select the next best element for a click or input.
An element will always be interacted with for clicks and inputs. This is an important fundamental difference and improvement over traditional automation. It also requires a mental change to the way we, as testers, think about automation in order to be successful.
If a test contains improper verifications and subsequently false passes, the platform will assume the desired flow was completed successfully. This, in turn, rewrites the machine learning data and reinforces training the system to select elements incorrectly.
Verifications are critical to provide a feedback loop to the platform to help it understand your desired goals and increase test stability. You are working with a teammate when testing with machine learning and like any partnership communication is essential. Verifications are that communication.
Best Practices for Using Verifications
Use verifications throughout test case creation and ensure the verifications produce the desired effect.
Here are a few examples:
- Verifications should always occur after a new page load (even if the URL stays the same).
- If the state of a page changes after a given set of actions, a verification should be applied.
- In almost all scenarios, the last action of a test case should be a verification.
- Verifications should be on elements that are static and unique.
- Verifications must be on elements that are generated after a set of actions has completed successfully and not on elements that were already present on the page.
- Verify only a single element at a time, not a div that contains multiple elements as this will limit the self healing properties of the test and make it harder to diagnose.
See also Verifications & the Elements Tab.