En
Twinformatics
How Twinformatics uses NeoLoad + Tosca to minimize effort required to reduce performance business risk
reduction in test maintenance effort
default
increase in performance tests executed
default
increase in test coverage
default
Company overview
Twinformatics is an information technology company that develops and operates software solutions for the Vienna Insurance Group, Wiener Städtische Versicherung AG, and Donau Versicherung AG. From design and development to operation and maintenance, they manage services and solutions, end to end, for about 25,000 employees and millions of customers. Their areas of responsibility also include the operation and development of SAP-based core insurance systems, from existing systems and claims systems through debt collection and disbursement, to group-wide financial controlling and HR solutions.
With insurance applications and services increasingly customer-directed through web applications, system performance is a critical competitive business driver. The Twinformatics performance testing team consists of one internal test manager and three external consultants (Triscon) testing four major and eight minor releases each year, plus on average one hotfix per month optionally.
- Industry: Technology
- Location: Austria
- Products:
NeoLoad’s automated test updates reduce maintenance efforts from 20 days to four hours
In today’s ultra-competitive insurance industry, potential customers and policyholders have zero tolerance for slow or buggy applications. They expect a fast, frictionless Amazon-like end-to-end digital experience — on any device, anytime, anywhere. If they don’t get it, they’ll look elsewhere. After all, the competition is only a click away.
Markus Bonner oversees Release/Test Services at Twinformatics. The company started with testing software mainly with its internal users. There was limited risk since any performance issues were kept within the insurance company and not visible to customers.
Twinformatics then decided to open up its applications to customers and allow them to directly interact with the core systems through a web frontend. For example, they could sign up for an insurance policy.
“Performance testing is important to ensure that whatever we show our customers is really working and performing. Quality and performance is a big business issue now,” says Bonner.
Twinformatics started load testing with JMeter but found it required a lot of effort to create and maintain its test cases.
“The original plan was to run performance tests with every major release, four times a year. So we had to dedicate ~80 man-days per year just for maintenance. This was costing us something like €80,000, so it was obvious that we needed a solution that takes less maintenance effort,” says Bonner. “Then it was an easy business case to go with Tricentis NeoLoad.”
- Develop software faster but with improved performance and stability
- Create iterative load tests faster
- Reduce performance test maintenance effort
- Automate load tests for different platforms (web, SAP)
- Make meaningful results and reports available to all stakeholders
Wolfgang Gaida, Test Manager at Twinformatics, also saw issues with JMeter and explained that once Twinformatics opened up its software to its customers, “We started seeing performance and stability issues in production when there were, say, 100 or 200 concurrent users.
Gaida knew they needed a better solution. “I had experience load testing with NeoLoad for a few years, so I knew it would meet our requirements for iterative load testing for different applications and platforms with low maintenance effort. And it’s easier to record new test scenarios [than with JMeter].”
Specifically, while JMeter has a GUI with record/playback capabilities, testers still need some specialized coding expertise to further customize scripts. NeoLoad is different because everything about test design is drag-and-drop. Users can create even large, complex tests without the hassle of coding by hand. With its frameworks approach, NeoLoad also automates the time-consuming, painful task of correlating dynamic parameters and variables.
The biggest game-changer is NeoLoad’s automatic test update. Legacy script-based testing tools like JMeter or LoadRunner are susceptible to notoriously fragile tests. Whenever an application changes or somebody modifies something in the flow of the test scenario, the test script breaks and it takes forever to fix. But NeoLoad technology updates only the part of the test that has changed and re-uses the rest, keeping variable extractors, think times, SLAs, loops, JavaScript, validations, and more from the original design.
“With NeoLoad frameworks, the ‘rules’ change the parameters automatically and correctly for each test iteration. So the [update maintenance] effort is not as great as with JMeter. We have reduced the maintenance for each test set from 20 man-days to four to five hours. We’ve gone from executing four test cases a year to about 365 — a 90X increase. And we now cover 7X more systems,” adds Gaida.
Results:
- Release more performant and stable applications faster
- Reduced performance test maintenance from 20 man-days to four hours
- Convert Tosca functional tests into NeoLoad performance tests with one click
- Increased test coverage from 10-20% to 70-80%
- Went from testing one system to seven
- Able to provide business stakeholders with meaningful results and reports
Tosca + NeoLoad = reduced risk (at speed and scale)
What’s particularly noteworthy about this 90X boost in testing is not the overall number of tests, but the increased risk coverage. A common problem with traditional testing approaches is that they focus on counting the number of test cases run, rather than on measuring business risk. In other words, it doesn’t matter how many tests are run if teams are not testing what’s most important.
Twinformatics has been way ahead of the curve on tackling this challenge, having adopted Tosca and its risk-based test optimization approach some 16 years ago. Tosca’s combined smart test case design and requirements prioritization helps Twinformatics align testing activities with business priorities, focus software testing resources on the tasks that matter most to the business, and stop creating, maintaining, and running tests that don’t add value.
“Before Tosca, we had — depending on who you asked — something like 10-20% coverage, without really having any real idea. Now, with Tosca and our risk-based regression testing, we have a target of 70-80% coverage, with 70-80% of this coverage automated. And now we’re doing this too for load and performance tests,” explains Bonner.
One way Twinformatics is able to get the highest possible risk coverage with minimal effort is via the native Tosca-NeoLoad integration that enables you to convert functional tests into performance tests with one click. Essentially, NeoLoad takes functional tests and “flips” them into performance tests so that you don’t need to maintain two separate sets of tests for performance testing and functional testing. Just keep your functional tests up to date, and you’ve got exactly what you need.
“We don’t need to discover new scenarios for load and performance tests separately. We can use the Tosca test cases. It’s very easy; it’s only one click to define a proxy and then we play the Tosca test cases to have performance test cases recorded in NeoLoad. And if the framework parameters change, we automatically have an updated [performance] test case,” says Gaida.
This “single definition” capability dovetails perfectly with the Twinformatics risk-based testing approach. They have developed a process that ensures tight collaboration between business stakeholders and testers. “The business sees that we have coverage of business processes according to the business risks. We use Tosca requirements for business-risk prioritization, so it’s helpful when the business can see what we do and can ‘weight’ their processes. When we first started, every process was, of course, equally important. So we got together and asked, what is the impact if something fails? How often does this process run? And with that, we came up with a business-risk rating process — which is the basis for prioritizing test cases,” says Bonner.
Tricentis partner:
