Testing with observability in mind: How NeoLoad + Datadog improve every release

Collaboration across IT is increasingly critical to having a smooth and effective software release cadence. As software complexity grows, it becomes more difficult for individual team members to have a thorough understanding of all the different elements that comprise a successful software application.

In particular, the elements of software operations are often not considerations for application development teams. They are focused on the definition of features, crafting of the code, and validation that it delivers on the feature correctly – not on how the software is planned to be managed operationally or how the return on value will be assessed. By bringing in insights from the production operations teams, application development can gain insight into these constraints while also providing visibility to the production teams on how application development is proceeding This accomplishes several goals.

First, the application development process suddenly gains access to a wealth of technological solutions – such as Datadog – that can enable significantly more effective collaboration and tracking of software components. Datadog specifically allows for integration with CI and SCM systems, providing mechanisms to allow for the tracking of code changes against software components, helping teams understand what has changed, who made the change, and how those changes were tested. Combined with Tricentis NeoLoad’s ability to bi-directionally sync data both into and out of Datadog, teams can view the performance characteristics and system resource usage data without any additional effort as part of the existing continuous quality effort they are already pursuing. This allows the Apdev teams to understand the impact of their work in a much more holistic way through visibility into Datadog data while NeoLoad drives a performance test against the software components.

Second, the production teams themselves gain visibility and insight into what changes are being made and, most importantly, how those changes are being validated in the context of running it in production. Performance engineering practices form an obvious on-ramp for engagement as part of this practice, allowing a natural way for the IT operations team members to gain visibility into the application development processes. More importantly, as the testing proceeds, they gain the opportunity to add their own insights into the quality process. For instance, commenting that while the tests were successful, there was a 40% increase in CPU consumption, and they have some concerns – but also to understand whether their monitoring profiles are correct or not.

What this means is that the Datadog community of users can see how things are being tested out – and if those tests do not go well, people will want to know why. If you look at the data you collected and still cannot determine what the underlying cause of a performance test failure is, then it is highly likely that in a production environment a similar failure will be difficult to triage and remediate, because you don’t have insight into the true cause of the issue. This opportunity to adjust your monitoring profiles to better observe the application components is a valuable way for IT operations teams to make their lives easier and more efficient for the downstream operation of the software itself.

These benefits can be quantified and measured. NeoLoad customers who have integrated with Datadog report up to 80% reductions in the time spent analyzing performance test results, while at the same time the number of production incidents related to performance experience a corresponding 70% to 90% drop, while also seeing remediation times for incidents shortening as well. If you’d like to learn more about how NeoLoad and Datadog can work together to help you deliver software with high quality that operates effectively and delivers on the value intended, visit us at Datadog Dash.