Modernize testing in 8 steps

Why

Why Modernize?

Why should you invest in software testing? And if you’ve already invested heavily in your testing program, why should you invest in modernizing your current practices and tools? If these are questions you’ve heard or asked yourself, take a cue from your peers, Gartner, and the World Quality Report.

When you modernize your testing, you make it easier for IT staff and business stakeholders to collaborate in ways that lead to faster software delivery and better business outcomes.

For further background on what’s really important in software quality assurance (QA) and testing, look no further than the World Quality Report for 2020-21. Some points to note:

• Between 2018 and 2020, the percentage of respondents saying QA and testing contribute to business growth and outcomes from from 68 to 74 percent.
• “Detect software defects before going live” wasn’t the leading response—it came in second at 72 percent. In other words, driving business growth and outcomes is now seen as more important than finding bugs.

Additionally, for the first time, the 2020 survey included three additional possible responses:

• Quality at Speed: Speed up software releases with good quality (63%)
• Automate: Make QA and Testing a smarter automated process (62%)
• Enablement: Support everybody in the team to achieve higher quality (60%)

These last three responses are significant. Well over half of companies want to speed up software releases and are using QA and testing automation to get there.

Last but not least, six out of 10 are making software quality the responsibility of everybody on the team—not just one group. You can imagine the implication: empowering DevOps, producing business-readable tests, and ensuring everyone is capable of running tests to some degree.

Well over half of companies want to speed up software releases and are using QA and testing automation to get there. Six out of 10 are making software quality the responsibility of everybody on the team—not just one group.

If you’re ready to empower your own DevOps team, produce business-readable tests, and ensure everyone is capable of running tests to some degree, you should know that this isn’t a pipe dream—but it’s also not something that happens by accident.

We’ve broken down the journey into eight steps that can help you transform your organization into a modern testing organization.

 

Vision

Establish the baseline for your plan

Your vision of what your future pipeline of high-quality products will look like is what will guide you through the process of modernizing your testing organization. That’s why it’s essential to get it right. To that end:

• Take your time. Don’t write a vision just so you can say you have a vision. Approach the process of writing a vision as a chance to create your ideal world for QA and testing. Even if you end up having to make certain compromises due to manpower or budget, at least if you start off aiming high, you’ll end up in a much better place than you’re in now.
• Make it short, clear, and inspiring. You’ll need to communicate your vision to many other colleagues over the years to come. That’s why you should make sure your vision grabs the attention, inspires action, and is short enough for others to remember so that they can communicate it, too.
• Involve stakeholders. Don’t let the process of crafting a vision be the exclusive domain of two or three managers. If some of your stakeholders feel left out of the process of writing your vision, they’ll probably never buy into it. Encourage them to share their needs and goals so that you can not only incorporate these into your vision but also modernize your testing in ways that meet these needs.

Plan

Define your strategy

Once you’ve set your vision for modernizing testing, it’s time to define your strategy for making that vision a reality. Simply follow these steps:

 

Identify the right people

People are the key to your testing transformation. Before you begin, take the time to identify your most important players—the people who have bought into your vision and have the skills, experience, and leadership qualities to make your transformation happen.

Of course, some of the right people you identify may not yet have all the skills they’ll need to fulfill the roles you’ve envisioned for them. In this case, you’ll need to map out career paths for these people and plan how you’ll train and educate them on how to carry out your vision.

 

Define your test strategy

Your next step is to identify your continuous testing processes. If you’re going with an agile approach, you’ll need to align your agile testing cycle with the development methodologies you’re using, whether that includes SAFe, LeSS, Spotify, or Scrum.

You will also need to define testing responsibilities so that there’s no uncertainty about which tasks belong to developers within the team and which are the responsibility of other team members. Don’t forget to assign the tasks that involve multiple teams. Who will perform end-to-end testing across teams? This step will have a major impact on future decisions around roles, environment, and test data strategy.

Your test strategy should also involve making sure your testing organization has the right tools for every possible situation. At this stage, you’ll need to evaluate tools based on how well they support your most critical capabilities, match to the needs and skills of your technical roles, and ease of integration and use.

 

Choose your operating model

Next, you should consider how your testing organization will actually run. Tricentis has evaluated the testing organizations of 85 Fortune 500 companies. We’ve noticed that they tend to be structured based on one of three models, each of which comes with pros and cons.

Learn more about operating models for testing teams: centralized, decentralized and hybrid.

 

Calculate your business case

Finally, your test strategy should include ROI calculations. The most obvious areas of potential benefit are the increased speed to market, reduced risk, and greater cost efficiency that a modernized testing organization can drive. The numbers you generate here can be crucial for helping you communicate the importance of investing in testing.

 

Build

Assemble a team of motivated individuals

Here is where you begin to line up your team of people who not only have the talent to help you, but also the motivation and enthusiasm to get started right now. Follow these steps:

Launch a pilot program. 

Find your sweet spot for trying out your new approach. It could be a single application, product, project, or program. Take an iterative approach rather than attempting a “big bang” that will affect multiple projects across your organization. How do you select such a project? Evaluate the savings possible, resources available, and chances of achieving a quick win.

 

Build an Enterprise Enablement team

This core testing team will consist of Test Architects and Automation Engineers who will orchestrate all decision-making throughout your organization on testing tools and infrastructure. They will also gather and propagate best practices

 

Train your organization

Your Enterprise Enablement team will now educate individual teams on the testing strategy that was defined in the previous step. They will work to balance the need for agility against the need to establish continuous end-to-end testing.

 

Establish

Set up your testing infrastructure

In step four, your newly established Enterprise Enablement team will begin to take action towards setting up the processes and tools that will form the foundation of your end-to-end enterprise testing program. Here’s what this step entails:

 

Set up the testing infrastructure

Your EE team will now deploy applications on top of your existing testing infrastructure. This activity can include automation, virtualization, testing, and test data management tools. The EE team will also integrate these tools into your current development and deployment pipeline. Along the way, they should strive to automate as many tasks as possible.

 

Build a service catalog

The best way to make your enterprise testing program sustainable is to build a service catalog around your testing activities. Make sure your EE team documents all tasks so that they can become standard services that are available to any department. You can then market these services internally as packages.

 

Share best practices

Your enterprise testing program won’t succeed unless you set up a culture of learning around it. Share your learnings with other teams across your organization. Propagate your success stories. A good way to do this is by establishing communities of practice.

 

Scale

Empower teams to be autonomous

In step five, your EE team will begin to prove its worth in the way it enables individual teams to meet their own goals while staying within the overall objectives of your testing organization. This is where your EE team gradually helps scale testing excellence from one pilot team or project to your entire organization. Here’s what it takes:

 

Onboard new teams

Your EE team will provide individual teams with the guidelines and tools they need to execute your organizational testing strategy. They should prioritize the onboarding of teams based on the impact and value that will likely result from having each team adopt the testing strategy. The EE team can standardize onboarding by offering services from the services catalog.

 

Onboard new projects and applications

The EE team should use the same mindset when deciding which projects and applications it should onboard into the new testing organization. Which systems will have the most impact and deliver the greatest value? For example, if you’re implementing a new CRM solution and lack testing expertise in the department that’s implementing it, you could assign members of your EE team to the project on a short- or long-term basis. Or, you could create a “SWAT” team made up of automation specialists and test architects to work on the project.

 

Enable teams

The EE team must make sure teams know how to use the tools properly, work within the delivery pipeline, and interpret the vast amounts of data they’ll encounter during testing. EE should be ready to provide hands-on training as well as to establish a community of practices that will function on an ongoing basis to ensure that the team follows best practices and can provide training to new employees as they are hired.

 

Streamline quality

This is where your EE team will ensure that all teams are implementing your organizational testing strategy correctly. This step includes making sure your testing strategy is integrated into your organization.

 

You’ll know your testing program is maintaining an appropriate level of quality as it scales when you see greater risk coverage, lower maintenance, faster implementation, higher productivity and higher automation rates.

 

 

Measure

Deliver meaningful insights

You can’t keep your testing program on track unless you’re measuring its performance and using insights to guide your decisions as you seek to enhance it. The right metrics will help your organization understand the whole story of your testing program and apply their efforts to the areas that will deliver the most value. Your EE team should work to:

 

Understand the business risk of product defects and the impact of your testing strategy

When your organization can properly interpret the data coming from your testing tools, they can deliver the right feedback to the right stakeholders at the right time. They can ensure that updates won’t adversely affect the user experience. They can also get immediate insights on whether a release candidate has an acceptable level of risk.

 

Measure at the product, process, and organizational levels

Measuring at different levels will give you greater confidence that change isn’t just an illusion—it’s really happening. Most testing teams are used to looking closely at the current state of a product. What about the state of your teams? What roles do they have? Are they improving in terms of test coverage? How are they doing in automation? The Maturity Assessment is an exercise that EE teams should repeat frequently to measure progress throughout their processes and the entire organization.

 

Use contextual over counting metrics

Your teams can generally add the most value by prioritizing contextual metrics such as requirements coverage and user experience. They should focus on measuring the user experience across an end-to-end transaction, rather than simply looking for improvements in the speed and productivity of the testing team.

Innovate

Pay continuous attention to technical excellence

A testing organization that continues to innovate is a testing organization that will continue to improve. Every organization should integrate innovation into its lifecycle. There are three ways to do this:

 

Increase automation

One of the biggest obstacles to innovation is waste. Automation eliminates delays and accelerates innovation. It also alleviates the frustration and boredom that come with manual work, and frees up team members to spend more time adding value.

Create a culture of innovation

This step involves embracing an agile development model and regularly evaluating new technology that can help you bring your testing organization to new levels of excellence. It also encompasses providing platforms that encourage innovation. It can include putting on events such as hackathons, or gamifying your day-to-day development and testing activities. It must include sending the message to your teams that there’s no such thing as a bad idea. Every idea should be brought to the table and discussed, but in a way that’s focused on your organization’s business objectives.

Reflect and improve

It’s essential for the organization to understand that pursuing technical excellence isn’t just about teaching and enforcing standards—it’s also about listening and iterating. Teams across the organization have valuable perspectives on the software delivery pipeline. These perspectives can benefit everyone in the organization—from business users to developers and testers! The best way to gather these insights is in regularly scheduled sessions organized by the EE team. EE teams should use these insights to inspire their continuous innovation.

 

Share

Tell your story

After you’ve implemented your modernized testing program across your organization, your work is far from done. In addition to continuing to innovate, you must tell your story by marketing your work and sharing the results to help other teams replicate your success. Here’s how to approach the task:

 

Accelerate change management and buy-in

You can do much to get other departments on board with your success by putting it in terms that they will understand. Not everyone will appreciate the value of writing 5,000 test cases automatically. But when you mention that you accelerated the testing cycle by one week, saving 200 labor days and $100,000, they will be much more likely to take notice.

 

Show a clear path to success

Be generous in giving away your secrets. Show the clear, concrete steps you took to modernize testing. Present lighthouse projects that make it easy for other teams to copy you.

Build credibility with others

Once you’re recognized for your success, you’ll gain a reputation in your testing organization for being able to deliver. Other teams will turn to you for advice. Use these opportunities to drive best practices across your organization.

 

 

Conclusion

Focus on measurable, incremental improvements

We would like to encouraging a greater investment in, and focus on, modernizing testing. Once you’ve automated and enhanced testing for one product or one team, it’s time to focus on the next. And even when your entire organization is following testing best practices and embracing agile principles, there will still be untapped value to realize. Above all, strive for continuous innovation—because when an organization stops innovating, it will ultimately get left behind by smarter, more motivated competitors.