The Functionize Customer Success Team has identified the most common scenarios that cause a test to require remodeling and created a set of suggested solutions to assist the highest probability of test case modeling success.
Missing Elements
The element mentioned in the test plan was not found on the application.
Suggested Solution: Recheck the test plan and relevant elements before submitting the test plan to ensure accuracy. Is the element hidden in a drop-down on the page and requires a hover to see it?
Site Mentioned in the Test Plan has Changed
The site identified in the test plan has changed and no longer contains the element(s) which are needed to be verified. This occurs most commonly around new site releases and sites undergoing rapid development.
Suggested Solution: Anytime a URL changes, make sure the test plan data has been updated to reflect this.
Instructions are Vague
Written instructions are ambiguous or values not explicitly mentioned in a test plan.
Suggested Solution: AI is tremendously powerful at understanding instructions. Make sure that the test plan instructions are detailed and crystal clear. For example, if instructions are "Enter data in the search box", these instructions should be clearly stated to include exact data outlining the value type to be entered. See also: Effective Test Case Writing for Natural Language Processing (NLP).
Missing Steps
The test plan is missing essential steps. As a result, the Functionize AI is unable to proceed under the assumption. "Hovers" are notoriously the most commonly forgotten step!
Suggested Solution: Make sure every, single step in the test plan is listed. For example, if a test plan includes entering a user name and password to log in, make sure to also include "Click on Login button" after entering the credentials. This may seem obvious. However, one cannot assume.
Mismatched Values
The values which are to be verified as per the test plan do not match with what is shown on the page. An example would be if the test plan indicates a value of 1000 but the actual acceptable value could also be 1001.
Suggested Solution: Make steps with values optional, provide a range for the value, or edit the operators to "contains" rather than "equal to".
Interrupted Sequential Steps
The page or file expected to load hangs on the load, causing a time-lapse when attempting to perform an action and therefore the next action cannot happen. This results in interruption and failure.
Suggested Solution: Check internally with the team to ensure the application is performing within the project specifications.
Unidentifiable Special Characters
Some special characters are used in a test plan that Functionize can not verify.
Suggested Solution: Make sure all special characters used in a test plan are accepted by the UTF-8 Framework.
Wrong file uploads
A file is uploaded with an unacceptable file format for the application being tested, causing an error where modeling is unable to proceed.
Suggested Solution: Ensure the file to be uploaded conforms with the acceptable file formats for the application being tested.
404 Errors/Wrong URL
Either the site is unreachable or an incorrect URL is included in the test plan. This results in an error when the test steps are performed during execution where the test is redirected to a different or incorrect URL.
Suggested Solution: Ensure the URL included in the test plan steps is the correct URL.
Page Objects not being leveraged correctly
Page Objects are not being used correctly during the test case step creation.
Suggested Solution: Read Page Object Creation and Usage for a tutorial on how to create a page object.
Free Form Merging of Step Description and Data
Test step descriptions and test step data should be separate and not merged. Values should be defined within the test data column and not in the test steps themselves.
Suggested Solution: See our Best Practices document Effective Test Case Writing for Natural Language Processing (NLP) as well as Sample NLP Test Case.
Slow page loads
Functionize automation runs in the cloud instead of on the site’s network so the page load times are more realistic and often longer than what onsite employees may experience. Tests cannot execute at the cloud if the site under test is too slow to respond.
Suggested Solution: Please ensure the site is not experiencing any performance-related issues