Element Miner Overview

The Element Miner allows a user to drill down and find an element using specialized syntax. This proprietary, function-based language was developed by our Data Science team to leverage the power of our Machine Learning Engine.

One of the greatest benefits of using AI/ML is the millions of data points collected from an application. Element Miner digs into this abundance of data to correctly identify elements. Leverage the power of the Functionize ML Engine to find elements with greater accuracy than traditional selector-based approaches.

This means that a Functionize tests run with much higher accuracy than traditional selector-based approaches, which find elements based on a single or a handful of element attributes such as Xpath or CSS selectors.

The Element Miner takes element recognition to the next level as it allows a user to select elements based on a more granular set of parameters. Unlike selectors that use unique attributes (e.g., finding the element based on a specific ID), a user can use an attribute that has multiple matching elements. In this case, Functionize uses machine learning on top of the Element Miner to find the best matching element.

This allows you to be more granular with what you’re looking for, but handle complex scenarios like these:

• Finding an element based on its relative position in a group of similar elements, like looking for text that is near the element
• Finding an element based on the page structure, so based on how many siblings or parents an element has in the DOM

Note: Element Miner is a feature that is available to all users. It can be found on the Advanced tab of the Action details on the Test Detail page and Slider View. Try it out and give us feedback at support@functionize.com.

 

Element Miner Details

The Element Miner is used to constrain the element selection model using definitions and rules/commands. The idea is to have a tool that runs constraint commands that will reduce the search space of the model and increase accuracy.

The following are commands defined in the Element Miner model:

• Attribute

Applies attribute-value correlation; this matches a command to HTML attributes from elements in the DOM.

attribute(list_rules)

Where list_rules is given as a comma-separated list of attribute=value rules or attribute?value for the CONTAINS operator. Example:

attribute(CL=main-menu, TV="User name", role=menu)
attribute(TV?"User")

The CONTAINS operator ? is less strict as we return all nodes whose attribute matches the argument. For example attribute(TV?"app") will match both apple and whatsapp. Note: that the Attribute Value is case sensitive so it won’t match with whatsApp.

The string value can be given without quotes if it does not have spaces, and quotes are required if the value is a string with spaces.

The comma separation implies an AND operator. In the example above, we can translate that command as follows: we want an element with attribute CL=main-menu and TV="User name" and role=menu.

We can list attribute-value pairs using the OR operator |, for example:

attribute(CL=main-menu| TV="User name"| role=menu)

or list them using combinations:

attribute(CL=main-menu, TV="User name"| TV="name" | TV="email", role=menu, TN=DIV)

You can have several lines of code commands, for example:

attribute(CL=main-menu, role=menu)
attribute(TV="User name"| TV="name" | TV="email")
attribute(TN=DIV)

where each line groups a set of attribute-value rules and concatenates the groups using an implicit AND operator.

There is also a NOT EQUALS operator (!=) which can be used as follows:

attribute(TN!="DIV")

This will select all elements on the page that don't have the tag name 'div'.

Here is a list of abbreviations used by the Element Miner model. If an attribute has an abbreviation, that should be used rather than the full name.

 

• Context

Correlates elements based on some context. Contextual elements are elements outside of the element we want to select - usually nearby when looking at the screen visually and also nearby when considering the HTML tree structure.

context(context_list)

Where context_list lists a set of attribute-value context rules using the | (OR) separator. This constrains the search space to elements having this context only. For example:

context(value="password" | TV="login" | TV="username")

The above example would search for an element that has correlation with a different element with a value attribute equal to 'password' or text equal to 'login' or text equals to 'username.' In other words, our model looks at the previous history and finds the relationship between the selected element and the contextual element, and tries to find that same relationship in the current run.

A common use case for this is if there are several input fields with no differentiating attributes, and we want to make sure we are selecting the input field that corresponds to a specific label. We could use the label’s text as the context we want to guide the model to select the correct input field. So, for instance, we might use context("Address") to select an input field where we want to enter a user’s address.

As it is very common to match a context using a text value (TV), this TV=some_value can be simplified as:

context(value="password" | "login" | "username")

Which is equivalent to the first example. That is, if you only set a string value like 'login', it is equivalent to writing TV="login". This is a shortcut only for TV attribute-value rules.

Note: Context does not accept listing the attribute-values using the AND comma-separated operator as in the attribute command.

 

• Content

Correlates elements based on content data. Content usually refers to text values of the element we want to select, but in general, can refer to any attribute values of children/descendants of the element. Children/descendants refer to the HTML structure and are nested within the desired element.

content("Accept" | PH="Street address")
content(value="username")

If no attribute abbreviation is given, the model assumes that it should look for the text value. It works similarly to the context command, where the model will look at previous history and find the correlation between the selected element and the content, and then it will try to find a similar relationship in the current run to locate the same content.

When trying to match by text, the content command should generally be preferred over using attribute(TV=”xxx”), since content can perform a deeper search of the HTML structure, whereas attribute(TV…) will only look for the direct child of the selected element. 

 

Element Miner Examples

Each line of Element Miner code is a single sentence. So for example the following code contains several commands which are executed and combined using an AND operation.

attribute(NT="1")
attribute(PV="1")
attribute(CL="btn btn-ghost-snow btn-lg salesforce-sans-regular")
context("date")