Q2
Q2 drives quality DevOps ecosystem at speed with qTest
Company overview
In the world of retail banking, consumers expect more than just checking accounts and debit cards – they expect the right set of solutions to transform their financial lives. In 2004, Q2 founder Hank Seale set out to build stronger and more diverse communities by strengthening their financial institutions through technology. To achieve this, he developed a single platform that removes technical barriers by integrating with legacy financial institution technology. Today, one in every 10 digital banking customers in America are signing into Q2’s single platform to bank through their chosen financial institution.
Q2 has reinvested over 20% of their annual revenue (approximately $500 million in 2021) on R&D and built out an industry leading in-house usability testing facility focused on delivering the best possible experience. Q2’s product philosophy – “Ask. Understand. Deliver.” – means software updates must be delivered at the speed of customer anticipation and with the utmost quality. In order to meet that goal, Q2 releases new versions of its digital banking software multiple times per month for web, iOS, and Android.
“We are in the fintech industry – testing is critical,” says Jan Acosta, Vice President, Digital Banking. “We must have well documented tests that show very clear traceability on what we executed. In order to achieve the coverage and quality we need, a large emphasis is put on automation – we can’t do it all by hand and release at the cadence we want to.”
The team at Q2 has been moving to greater levels of automation. They leverage multiple automation tools and frameworks to ensure quality for each release, including Selenium, Appium, and homegrown frameworks based in Python. With so many tools and processes across teams, it was often difficult to obtain a clear picture of where a given release stood. The team needed a way to holistically trace issues back to specific tests and environments, and to communicate those issues back to the development teams. Leaders relied heavily on gut feelings about where the release readiness stood instead of being driven by data.
While some teams had access to various types of test case management tools, high levels of automation across a disparate, unintegrated toolset created bottlenecks and visibility issues. To address these issues and continue to scale their DevOps progress, leadership at Q2 determined a clear need for a more streamlined approach to test management.
- Industry: Financial Services
- Organization size: 1,001-5,000
- Location: United States
- Products:
Finding success: Visibility, traceability, and defining guardrails to consistent quality
Finding success: Visibility, traceability, and defining guardrails to consistent quality
Organization of projects, processes, and teams was an important first step. Projects in qTest are broken up by functionality and integrated with specific boards in Jira. Because of functionality overlap from one team to the next, test case sharing in qTest allows others to quickly locate and execute existing test cases – both manual and automated. This process reduces redundancy in test cases being leveraged by the multiple teams from one release to the next and makes test case maintenance across teams easier to facilitate.
“The biggest factor was setting standards,” says Acosta. “Being organized when you are dealing with a large team and several large projects helps create more effective management of resources.”
A top priority was implementing specific guardrails to ensure consistency and ease of collaboration across teams. One example was the naming convention for test assets. Because these fields are customizable in qTest, Q2’s test cases now follow a specific naming convention that matches Bitbucket and other repositories, so both testers and developers can easily identify what they need when test cases need to be addressed.
Custom fields and statuses have also helped to evolve the testing process for Q2. For example, in addition to custom fields indicating where the test is run (mobile, web or both), custom statuses are leveraged in order to triage failed test cases. Failed test cases trigger a review to determine the source of the failure.
The team realized failures were sometimes due to server downtime, mobile devices being offline, network issues, or other test environment issues not directly related to defects in the code. This visibility has allowed the testing team to work more closely with various departments within IT to optimize how and when they are executing test cases. This move has significantly reduced false positives caused by environment issues.
“qTest has given much greater confidence with the larger development organization and even more so with the delivery and support organization. I can now SHOW them what I did. It has allowed us to make data-driven decisions instead of gut feel decisions. Data-driven decisions take the emotion out of the conversation to determine how to best proceed forward.”
- “qTest is intuitive. It has all the things a great test case management system should have. As a leader, it has all the key things I wanted – to document and capture my test cases, a way to see my execution results, integrations and traceability between Jira to connect the dots, and to have my data accessible so we can report in a fashion that fits our needs."
