Here are the highlights of this release:
- Jira integration: log defects into Jira and give full visibility into test execution results
- TestRail integration improvement: more granular settings to send results
- Functionize CLI: to connect to non-public sites or to integrate with CI tools
- Smart waits, Smart lazy loading: auto-detect network traffic and wait for pages or elements to finish loading for more resilient tests
- Improved page objects UX: better bulk updating of commonly used tests
- Performance improvements: for up 10x faster performance
Plus much more! For the full list of improvements, check out: Release Notes 5.0.
Jira is likely the most popular application lifecycle management tool. Most of our customers rely on Jira to track backlogs and defects in order to ensure that things get done as planned. As a result, testers and developers have to use Jira to collaborate when bugs are discovered and resolved.
In this release, we drastically improved our native integration with Jira. You can now integrate Functionize with your Jira projects regardless of whether you use Jira Cloud or Jira Data Center. You can set up the integration to submit defects linked to execution results. It doesn’t matter whether your company calls them bugs, defects, change requests, or any custom Jira issue type! After you execute Functionize tests, explore any potential failures. Then, submit defects from Functionize into Jira.
Once your defect is submitted, it is linked with the Functionize test result. Going forward, you have visibility on both sides, so that testers and developers alike are informed of the latest execution result as well as the defect status. This way, if developers want to view the details of the initial steps to reproduce the bug, they can click into Functionize. If testers want to know which bugs are ready for retesting, they can see the bug status in Functionize. This integration is a major milestone in boosting collaboration and keeping everyone on the same page for bug tracking and resolution.
TestRail Integration Improvement
One of the major benefits of using the TestRail integration is seeing Functionize orchestration results in TestRail. However, if you have tests in multiple orchestrations or if your orchestrations are run often, this can lead to a lot of results data in TestRail. Sometimes, too much data can be overwhelming and get in the way of focusing into the details that matter. To combat this, we now offer more granular control over your orchestration results. So, you now have more flexibility when you set up the TestRail integration.
Here are the options to send orchestration results:
- Always (all results): this is helpful if you want to maximize visibility in TestRail. If your orchestration is run once a week or less frequently, then each result will still be significant to see. If your orchestrations are run more frequently, this may be overkill.
- Only passed: this option may be helpful if your results are intended for positive evidence that your application is not broken.
- On demand (ad hoc): this option sends the least amount of data, since someone needs to manually click on the button to submit a result to TestRail. This option will be helpful if you need to push results sparingly. For example, if you want to verify failed tests first before sending the result to TestRail.
The Functionize CLI allows you to connect with external tools more easily. Although the possibilities are endless, we are especially excited about these two use cases:
Use the Functionize CLI to connect to non-public websites. This may be necessary if your application under test is only accessible with a special connection or runs locally. If so, the CLI supports a simple tunneling option to connect to a private site. This is much easier than setting up a white list for all our servers.
Use the Functionize CLI to integrate with CI/CD tools. Imagine if your CI/CD could execute your Functionize orchestrations and wait for the results before releasing new features. Well, with the Functionize CLI you can build efficient multi-stage pipelines from Jenkins with a simple one line command. The Functionize CLI enables Functionize to seamlessly fit into any DevOps toolchain.
Smart Waits, Smart Lazy Loading
Wouldn’t it be great if your tests could dynamically adapt to network conditions. Waiting for pages to load instead of incorrectly failing tests because of higher traffic? Brittle tests like these are painful to deal with one of the major culprits for a brittle test is the difference in behavior across environments. When a test runs on a test machine, it might behave very differently compared to when it’s run on a tester’s local machine. These differences lead to false positives, or tests that fail when they should actually pass.
Cookies, network configuration, network speed and other plugins or addons can all affect how an application behaves on a tester’s machine versus a customer’s machine or a clean test machine. Many times, tests fail simply because they don’t wait long enough for an action to occur on the test site or special code needs to be added in order to account for dynamic web pages.
We believe that AI powered tests should “just work” out of the box. Needing to custom-build everything for these extremely common scenarios is inefficient. For this reason, we are introducing two new intelligent testing features: Smart Waits and Smart Lazy Loading.
Smart Waits will analyze the network traffic on a page and dynamically wait for the page to load, reducing the incidence of failing tests. You can add a Smart Wait while you’re creating the test using Architect. This can also be done using Quick Add Action while editing an existing test.
Smart Lazy Loading knows to wait until an element is ready for interaction. This is useful if a test interacts with an element that isn’t loaded until the user scrolls down the page (think Netflix or a continuous news site). In this case, this minimizes the waiting required, keeping the tests running as fast as possible. This ultimately keeps the tests passing as well as running quickly.
Improved Page Objects UX
If you have hundreds or thousands of tests, it’s wise to use page objects to organize any steps that are commonly repeated across multiple tests. For example, let’s say that multiple tests require you to log into your application using a certain user profile and password schema. This is a common example that would benefit from using page objects. Page objects allow you to insert those login steps into any test that requires logging in. This makes it much faster to build tests without having to recreate those same steps.
In this new release, we’ve overhauled the user experience of this feature to make it easier to manage changes to page objects. You can now modify page objects directly from a test case. Once it’s changed, the change applies to all test cases that use this page object. This enhancement allows you to better manage common test steps in bulk without having to make changes in the individual tests.
Improved UX for Test Detail and Page Object Pages
The Test Detail page is jam packed with functionality. That is why we optimized this page to make it easier to review, edit, share, and run. The new toolbar streamlines how features are organized including:
- Different ways to execute tests: Background Run, Live Execution, or Live Debug (now available for everyone)
- Easier way to jump into results
- Test editing capabilities like Import Actions, Bulk Update, and Delete
- Switch into different environments
You can also find similar improvements on the Page Object page.
Rerun Failed Tests in an Orchestration
Orchestrations are great for executing a large suite of tests together. Sometimes, you may want to rerun specific tests that have failed. Let’s say that a bug was fixed and is ready for retesting. Or, perhaps you want to verify the test execution on a different browser. You can now rerun a failed test directly from the orchestration. This saves you considerable time since you don’t need to rerun the entire suite of tests.
We know how important speed is for optimal UX and efficiency. That is why we aim to optimize performance across the platform with every release. In this release, we have drastically improved performance across various pages and processes, with up to 10x improvement in certain areas. Hopefully, you will have a smoother, faster experience using the Functionize platform.