Redefining continuous testing to include continuous performance testing

The wall between functional testing and performance testing has been eroding bit by bit over the past couple of years. Now, the Tricentis acquisition of Neotys breaks down that wall altogether.

In a nutshell, this integration lets you convert Tosca functional test scripts into NeoLoad performance test scripts with just one click. No more reinventing the wheel or performance testing becoming a bottleneck at the 11th hour. Now you can accelerate performance testing cycles to match the faster clock speed of functional testing cycles. With the ability to easily incorporate performance testing into development sprints, now “go” really means “go.”

Recently the founders of Tricentis and Neotys — Wolfgang Platz and Thibaud Bussière, respectively — discussed why Neotys becoming part of the Tricentis family is such a big deal and what it means for enterprise Continuous Testing in their webinar Enterprise Continuous Performance Testing: Making the story complete.

NeoLoad completes the Tricentis story

Wolfgang often talks about how Tricentis automation and AI tackle enterprise application testing by answering three key questions:

• What do you test?
• Does it work?
• Does it scale?

Tricentis addresses the first two with LiveCompare and Tosca. “We have actually built a very nice handover between Tricentis LiveCompare and Tricentis Tosca that shows the test hits and test gaps that you want to close,” said Wolfgang. “However, when it came to a nice transition from ‘does it work?’ into ‘does it scale?’ — meaning from the functional testing side into the performance testing side — there was still a question mark. Resolving that question is happening now, and our answer is Tricentis NeoLoad.” This is where the ONE Test Definition comes into play.”

Introducing the ONE test definition

The ONE Test Definition approach is “really about leveraging the work of others. If somebody has already done the job for us, we should be able to use that! You’ve spent the effort to build the right functional test definition — we can reuse the script for performance testing,” Thibaud explained.

“Once you have all the key steps and transactions that matter defined in your functional test, we can convert them with one click into NeoLoad for a complete performance script. All your key transactions are retrieved, the time you have to wait between clicks, and of course you can modify it. But the story is really about this being automated. We save you time by converting this [functional] script into a brand-new NeoLoad [performance] script. And as the application keeps evolving and your functional test keeps evolving, you can update the performance script immediately. If you have fine-tuned or modified your functional script, NeoLoad retrieves those elements you have changed and updates the performance script,” said Thibaud.

It’s this automated script maintenance that’s such a force multiplier and separates Tricentis NeoLoad from the pack. NeoLoad updates only the parts of the Tosca functional script that have changed, instead of having to rescript the test from scratch the way you do with other performance testing solutions. Customers have stated that they see 80-90% time savings in script maintenance.

Wolfgang added: “Let me make one thing clear. This is not painting a nice picture of the future, which may or may not happen. This is already here. This is technology that our customers are already using.”

What does this mean for enterprise continuous testing?

If you’re like most enterprises, your release cycles are broken down into a series of (usually weekly or bi-weekly) sprints where development and unit testing happens in parallel. But traditionally performance testing has been deferred until all the unit components are good to go from a functional perspective. There has always been a mismatch between the speed of getting to “go” for functional testing and getting to “go” for performance testing.

“What our Tricentis Tosca customers do is run so-called smoke tests — meaning you check the most relevant capabilities in a small, comprehensive test portfolio. As soon as this is green, then you’re able to run additional sanity checks that go into more depth when it comes to business risk coverage. And given the automation we have with Tricentis Tosca, you can do this in a continuous fashion,” said Wolfgang. This is enterprise Continuous Testing that ensures a very well-tested application at the end of the development phase — from a functional standpoint.

Now the same high-speed, high-volume cadence is possible with performance testing. “Typically, performance testing was done late, very close to the release, once all the functional elements worked perfectly before you saw if it would scale. The problem with that is, you lose a lot of time because if something fails, you have to go all the way back to the beginning to revalidate everything. Performance testing becomes the ‘bad guy.’ We were slowing down the process of deploying the application,” added Thibaud.

What Tricentis NeoLoad does is Shift Left performance testing — into the smoke tests and sanity checks — to enhance confidence in the quality of releases. As Thibaud explained, “The idea is to identify the functional smoke tests that are relevant, from a performance point of view, that give you confidence in terms of risk coverage. Because in one click — almost automatically — you can convert your Tosca script into NeoLoad, you have the ability to keep pace and cover your bases. The benefit is that you have super-fast feedback loops, so you find your performance problem much earlier. Developers have this information right away.”

In other words, now we’re talking about enterprise Continuous Performance Testing.

Check out Wolfgang and Thibaud’s full discussion in the on-demand webinar Enterprise Continuous Performance Testing: Making the story complete.