Model-based and AI-powered test automation: the best of both worlds

AI has garnered a lot of attention in the software testing space recently – from smart test execution and self-healing tests to neural network capabilities that enhance test resilience. According to IDC’s FutureScape, over 54% of executives say that AI solutions implemented in their business have already increased productivity. Furthermore, a whopping 83% of organizations say AI is a strategic priority for their business as of today. AI has made the impossible possible. Although some questions remain about which AI implementations will ultimately deliver the greatest business value, one thing is for sure – AI is already transforming test automation as we know it.

In case you missed the news, Tricentis launched Vision AI, an engine within Tosca 14 that uses advanced heuristics and convolutional neural network technology to deliver test automation for both legacy and modern apps, as well as build test cases from a simple UI mockup – irrespective of the underlying technology.

Despite the hype, there is still some confusion on what this all means and how it comes together. How exactly does the Vision AI engine work in Tosca? How does it integrate with model-based test automation and other technology-specific engines within Tosca? What does Vision AI uniquely offer and how do all the Tricentis AI-powered solutions come together in-one-piece?

Read on… we’ll answer everything you ever wanted to know about Tricentis Tosca’s AI-powered platform.

Tricentis Tosca, powered by Vision AI

Let’s be clear: Vision AI is not a standalone product. It’s an AI-powered engine within Tricentis Tosca version 14. The new Vision AI engine does not replace existing engines within Tosca – rather, it works in tandem with existing Tosca capabilities to extend end-to-end test automation across new use cases, technologies and platforms. This will allow you to automate more, start testing much earlier in each sprint, reduce the pain of testing across application changes, and extend test automation to complex modern and legacy technologies.

But how exactly does Vision AI relate to Tosca’s model-based test automation approach? As you may know, model-based test automation is the pinnacle of Tosca. It involves scanning the application under test, extracting its technical layer and creating a business readable automation model through a no-code approach. This renders test case creation and maintenance much faster and less error prone, leading to more efficient, robust and reusable testing.

Vision AI uses the same principle for designing a test. The only difference is that the Vision AI engine identifies elements on a user interface through its neural network technology, which we explain below. You can build Vision AI tests side-by-side with tests using other Tosca engines. Simply drag-and-drop Vision AI modules into the same test case that uses modules from other engines (like the HTML engine).

You can also amplify your end-to-end test automation by mixing Vision AI test cases with additional Tosca capabilities, for example:

• Risk-Based Testing. One of the main challenges we hear in testing today is that we are running hundreds, if not thousands of test scripts, which in effect, cover only about 20-40% of our business risk. How can we minimize our testing efforts and maximize our risk coverage? Tosca’s smart test case design provides you with a centralized area to design and maintain data-driven test scenarios. Through a built-in algorithm, Tosca autogenerates tests based on the different scenarios that you design according to business logic. With this approach, you can skyrocket your test resiliency as well as business risk coverage by testing the most critical requirements first.

• Test Data Management. The reality today is that agile teams work in multi-persona teams across distributed environments. This makes it challenging to maintain data when doing end-to-end testing that spans across business applications and architectures. Moreover, manually creating test data is a resource-intensive and laborious process that slows down testing and ultimately, the delivery of releases. With Tricentis Test Data Management, you can easily provision, store, manage and keep track of test data every time it is altered or consumed throughout your end-to-end tests.

• Service Virtualization. Lack of test environment availability is the second largest bottleneck to achieving high levels of test automation (World Quality Report, 2019). With Tricentis Orchestrated Service Virtualization, you can simulate test environments when they are not available or difficult to access. This is crucial for ensuring your integration tests execute continuously and reliably each time when systems are down.

These are just some capabilities of Tosca’s comprehensive test automation platform, and Vision AI tests can easily be integrated into those existing workflows and scenarios. Bottom line is that regardless of which engine or feature you use within Tosca, the Vision AI engine can augment the robustness of your existing tests through AI-powered capabilities.

No need for technical identifiers with Vision AI

One of the hidden costs of test automation frameworks is that tests break due to fragile object identification. Every change in the application impacts the underlying object properties. As a result, the identifier of an object will no longer be able to locate and steer that object. Teams end up spending a lot of time manually refactoring thousands of test scripts leading to high maintenance costs.

Vision AI opens a whole new dimension for test automation. Simply put, Vision AI does not care about the technical layer of an application. Rather, it acts just like a pair of human eyes by ‘seeing’ and recognizing elements on a page.

You may be thinking: “This is like AI-enabled visual testing.” Not exactly. Unlike visual testing, Vision AI does not look at object locators like ID, tag name, or XPath. Furthermore, Vision AI does not compare the visual differences between a previously-captured baseline image and the existing image when a test runs. Instead, Vision AI uses visual clues through deep learning technology to recognize objects and controls and steer them irrespective of the underlying technology layer.

In spite of their different use cases, both AI-enabled visual testing and Vision AI can enhance test case resilience, thereby reducing test maintenance effort and costs.

When would you use what? Tosca engines and Vision AI – it takes two more to tango

A general rule of thumb is that Vision AI can supplement engines within Tosca where automation might be difficult or impossible. We’ve broken this down into three main areas, though the use cases can be multiple.

Testing a remote or virtual application running on Citrix or VMware

The COVID-19 pandemic has shifted much of the world to remote work environments, leading to an even greater demand for virtual and remote desktop interfaces like Citrix and VMware. Creating test automation for a remotely-hosted application is not possible since you cannot reach the application’s underlying technical properties in order to identify and steer elements. This is exactly where Vision AI can assist. Based on its convolutional neural network technology, the test steers elements on a page by looking at the visual cues only, irrespective of the underlying technical layer. With the Vision AI engine in Tosca, you can build and run tests across any virtualized application environment, even on the hardest-to-reach interfaces.

Extreme shift left by building a test from a simple mockup

In a typical agile sprint, testers would normally have to wait before the UI was fully developed before UI test automation could begin. Companies that adopt agile and DevOps practices such as behavior-driven development (BDD) in their test automation scripts would still have to wait until the UI was ready in order to begin test automation. This slowed down the release of software. With Tricentis Vision AI, you can finally ‘shift left’ and start UI testing before the UI even exists. Vision AI can build a test from a simple mockup and run the same test on the fully developed user interface or application.

Testing modern and legacy applications

More and more enterprises are undergoing application modernization as part of their digital transformation journey. Unfortunately, test automation solutions for both legacy and modern technologies have been slow or unavailable, sometimes requiring heavy customization. Testers are left spending significant time manually refactoring tests for complex or hard-to-identify controls. With Vision AI, automation is based on visual cues making it able to run across multiple technologies, removing the pain of creating costly customizations which only slow down modernization efforts.

Vision AI puts the human tester at the centerpiece

Using Vision AI in Tosca makes automation easy and accessible to all. From technical users to business users, anyone can embed Vision AI into their test automation workflows. This will allow you to design smarter tests and amplify your test resiliency which does not require constant human manual intervention.

We’ve just outlined some of the candidates for Vision AI. Having said that, there’s still a ceiling on how much Vision AI can automate different scenarios. For instance, Vision AI cannot be used for non-UI testing (like API testing) since it does not interact with the underlying technical layer. Additionally, technology-specific engines in Tosca may be more stable for certain test automation scenarios due to their robust technical identification techniques.

Train Vision AI to constantly improve

Before you leave us, bear in mind: as new data arrives, Vision AI improves its ability to handle more complex test automation scenarios. You can be part of the process by training Vision AI to be smarter and adapt to a plethora of diverse applications, platforms, and technologies you use in your specific industry.

Are you interested in experiencing Vision AI for yourself? Please reach out to us here. There are humans behind these blogs who would be more than happy to help.