Defining your test strategy

Introduction

Accelerating your software delivery cycle is impossible unless you also accelerate and enhance your approach to software testing. In another article we described the eight steps of the journey to transform your organization into a modern testing organization.

 

This article will focus on defining your test strategy. You’ll learn about the importance of adopting agile practices. We’ll also walk you through the levels of testing activities, outline the components of a test strategy, explain what goes into a modern tooling strategy, and summarize what it will take to establish your strategy.

 

Defining your test strategy

A test strategy is an outline that describes the testing approach a company will take in its software development life cycle (SDLC). Because many of today’s teams follow the principles of agile development, they should also follow the principles of agile testing. To that end, adopting to agile testing is a key element of defining a test strategy:

1. Embrace agile testing
2. Understand the three levels of testing activities
3. Align the components of your test strategy
4. Create a modern tooling strategy
5. Establish your strategy

 

 

1

Embrace agile testing

Agile testing encompasses the same test types that testing teams have become accustomed to using in the testing cycle. The difference is that in agile testing, teams must perform these tests more often within a shorter period.

• Unit testing is usually performed by developers on the code, class, or function level.
• System testing validates the functionality of the system.
• System integration testing validates integrations between systems.
• End-to-end testing validates complete business processes.
• User acceptance testing gives the team feedback from business users.

Teams may also choose to perform non-functional testing, such as security testing or load and performance testing. Because agile development requires teams to execute each of these types of testing much more frequently, it will be ideal for teams to automate as much as possible.

 

Automation can not only help you meet the need for speed in agile testing but also help you deal with increasing complexity. Agile teams often must face:

• Disparate testing strategies. Teams working at different stages, on different processes, and on different applications will tend to use their own testing strategies. These strategies may conflict with those used by other teams.
• Siloed testing cycles. The testing strategy and resources used by one team are not accessible to other teams.
• Multiple teams working on one application. For these teams, collaboration and communication are of the utmost importance.

 

As you scale testing across your organization, challenges like these will result in more dependencies, less efficiency, and higher costs. If you are going to continue meeting your customers’ quality standards, you need to transform your testing practices now with a mind for simplicity and efficiency.

2

Understand the three levels of testing activities

 

We can begin to remove the complexity from scaling up our testing activities by understanding the three levels of testing activities that companies typically perform: team, program, and enterprise. These levels of testing would be simple enough to perform if companies had only one team and one application under development at a time. But your company probably has multiple teams working simultaneously on various programs—and as you expand your testing footprint, your testing activities will become more complex and require more effort. Keep in mind that, depending on the size of your company, certain activities may shift left or right between these levels.

 

Team level

At the Team level, teams perform unit and system testing. They execute API and contract tests, test microservices and databases, and validate your features and functions. Teams should perform this testing in an isolated environment. Some of the components can be virtualized, but everything is tailored to the needs of the team.

 

There are two central roles involved in testing at the Team level:

• Automation specialists execute the tests with the toolset provided and according to the defined testing strategy. These specialists build and execute the automated, standardized, and reusable tests, which will be integrated into the provided toolset.
• Test analysts perform risk analysis to constantly redefine the smoke test, regression and progression portfolio and exploratory tests.

 

At the Team level, automation specialists and test analysts should strive to:

• Deliver solutions that can be reused across teams
• Follow automation best practices
• Put the right environment in place
• Consider how virtualization can help
• Have test data ready
• Automate as many stages as possible
• Work to eliminate false positives
• Incorporate Exploratory Testing into the testing process

 

Program level

 

At the Program level, teams test the user interface, business processes, and the integrations between systems. Testing may include virtualized components.

 

There are three central roles involved in testing at the Program level:

• The Program Test Architect will define the test strategy and testing tool chain and will orchestrate tool decisions
• The Program Automation Engineer will implement and integrate the testing strategy and tools for continuous testing.
• The System Team will be assembling and executing automated tests across multiple teams or applications / line of business.

Enterprise level

Testing at the Enterprise level involves testing across multiple testing stages. One example would be testing customer transactions across different applications. Enterprise-level testing may also be called end-to-end testing. It incorporates functional and non-functional testing and may encompass load and performance testing. At this stage, the environment begins to closely resemble the production environment. Testing combines the artifacts of different teams into end-to-end tests handed over to a specialized team (system team) for further testing. On top of end-to-end testing, business users will perform user acceptance testing, which incorporates a business perspective into the testing results.

 

There are three central roles involved in testing at the Enterprise level:

• The Enterprise Test Architect will define the test strategy and testing tool chain and will orchestrate tool decisions on an enterprise level
• The Enterprise Automation Engineer will implement and integrate the testing strategy and tools for (enterprise) continuous testing
• The System Team will be assembling and executing automated tests across multiple teams or applications, line of business, and programs.

 

At the Enterprise level, testing team members should strive to:

• Orchestrate testing at the Enterprise level across all programs
• Cover the most-used and highest-risk end-to-end business processes
• Simulate end user behavior
• Define production tests

 

 

3

Align the components of your test strategy

 

Once you’ve clarified the three levels of testing activities your teams will perform, you can begin to define your testing practice.

 

Your testing practice form the core of your testing strategy. It is how you determine whether your software is ready for release. Your testing strategy is defined by these layers: people, process, and technology.

 

People

At the people layer, shared teams operate at different levels. Here, the system team will perform end-to-end tests using the artifacts delivered by other teams.

 

Answers to critical people layer questions

How can we modernize our test organization?

Define responsibilities. Give everyone clear instructions about their role in continuous testing.

 

How can we collaborate efficiently?

Maximize collaboration. Set clear expectations about what each team member can expect from others and what they will be expected to deliver.

 

How will we scale testing to other teams?

Enable end-to-end testing. Use the responsibilities and deliverables you’ve established to set a framework for end-to-end testing.

 

How can we all become more agile?

Address governance. Assign governance to a dedicated Enterprise Enablement team.

 

LEARN MORE ABOUT THE ENTERPRISE ENABLEMENT TEAM

 

Process

At the process layer, your teams will focus on establishing consistent, accurate measurement of your testing activities so that they can provide useful feedback to stakeholders. They will also develop your strategy in ways that help accelerate and prioritize testing.

 

Your team may seek answer to these questions

How can we unify our testing?

Increase traceability with a state-of-the-art test management tool.

 

How can we measure our transformation?

Unify reporting across teams, applications, tools, and business processes.

 

How can we reduce waste?

Unify testing tools to enable end-to-end collaboration.

 

How can we shorten our release or testing cycle?

Leverage synergies by removing duplicate systems, processes, and work.

 

You’ll also tighten your continuous testing strategy by:

• Carefully planning what you’re testing.
• Strategizing about how you’ll prepare the test data.
• Ensuring your test data is easily shared, accessed, and managed across teams and environments.
• Creating a test execution strategy that defines which tests to execute, when, where, and with what data.
• Making sure your test environments are available and stable for test execution.

 

Technology

At the technology layer, your teams will enhance functional testing by focusing on automated test case execution, service virtualization, user interface testing, API testing, and test data management. They’ll also evaluate non-functional testing such as performance testing, seeking to incorporate as much automation as possible.

 

Focus on using technology to enhance your testing

How can we increase release frequency?

With Automated test case execution, teams will reduce manual work and ensure the team gets speedy notifications when any issues are introduced into the application. By doing so, they can increase coverage and reduce risk.

 

How can we increase quality?

Performance testing automates the tests that verify your application’s speed, robustness, reliability, and correct sizing.

 

How can we test unreliable, evolving, or not yet completed applications?

With service virtualization, teams will shift testing left and increase the frequency of tests and deployments. They’ll democratize testing by creating services that can be shared by different teams throughout all stages of the testing cycle.

 

How can we simulate and test customer behavior?

User interface testing aims to improve the overall product experience for testers as well as business users and customers.

 

How can we reduce waste?

With good test data management, they’ll free up time for your testers and streamline manual, slow, and costly processes.

 

How can we optimize costs through operational excellence?

In API testing, teams will test the migration of microservice-based applications, APIs, and more.

 

 

4

Create a modern tooling strategy

 

If your organization is serious about achieving high-speed releases, you need to optimize and orchestrate quality across teams, projects, and applications. Once you’ve determined the components of your testing strategy, focus on building your tooling strategy. A modern tool strategy should be driven by specific objectives and be capable of fulfilling many different requirements.

 

These specific objectives should include:

• Scaling test automation to an enterprise level. Focus on integrating and scaling test automation into your entire software development life cycle as well as your test management processes.
• Enhancing collaboration across teams. Standardize alignment between your developers, testers, and business with collaboration and traceability from requirements, tests, and defect levels.
• Increasing speed to market. Enable your organization to approach QA strategically by testing early and often and getting to market faster.

 

This is where tooling comes into the picture. You can support these objectives by implementing the right tools. Look for solutions that are highly integrated, so that they will enable and foster collaboration among your teams. More specifically, your solutions should provide:

• An open platform that supports and orchestrates a broad range of open-source and commercial technologies.
• Traceability, so that you can trace your production deployments back to your business and functional requirements with DevOps ecosystem integrations.
• A full testing life cycle. If you find a single platform that supports numerous testing requirements, you’ll benefit from having a common information model that enables all users to collaborate on test cases, plans, data, and artifacts for use across all phases of testing.
• Real-time analytics. When you integrate reporting and data warehouse technology, you can get comprehensive visibility and insights for all your testing activities in one place—enabling you to make continuous improvements.

5

Establish your strategy

 

You’ve defined your testing strategy and tools. Now it’s time to implement your strategy. This is where your Enterprise Enablement team will get involved.

 

The Enterprise Enablement team includes architects and engineers from the Enterprise level who set teams’ testing strategies across the organization by:

• Orchestrating decision-making on testing tools.
• Representing QA in strategic discussions.
• Defining and managing operating models.
• Gathering and propagating best practices across teams.
• Enabling new teams.

6

Conclusion

 

At its essence, building your test strategy simply involves determining how many testing activities you want to perform, what your team is capable of, and what kinds of tools you need to implement to empower your teams. Throughout this process, you’ll strive to unify your teams, increase their collaboration, and remove the complexity from everything they do by standardizing tools and processes to the greatest possible extent.

 

Remember: the goal here isn’t initial perfection—it’s constant improvement.