The Performance Advisory Council: Insights on performance engineering for 2024

The Tricentis-sponsored Performance Advisory Council (PAC) is an industry-agnostic event that brings together performance engineering experts to speculate on and discuss the future of the industry. Subject matter experts, consultants, and veteran practitioners explore industry trends and best practices in this annual event.

The most recent PAC, held in Orlando in December 2023, was a great success. I was joined by a group of experts in the field, and together we collectively had over a century of experience creating, executing, and analyzing performance tests. We discussed several emerging patterns and challenges that I believe will shape 2024, including shifting performance testing left, infrastructure as code, APM integration as those solutions continue to mature, CI integration, and more.

Read to learn more about who led each session and what was discussed! For more in-depth content, the PAC presentations, recordings, and transcripts are also available here.

Bryan Cole, Director Customer Engineering – NeoLoad, Tricentis (that’s me!)

I am a performance evangelist with over twenty years of experience working for vendors, developing performance tests for (and expertise in) literally hundreds of different software environments. I led a session with a frank conversation about the coming impact of AI on performance engineering. The conversation grew to include the coming impact on every facet of the software industry and beyond. Spoiler: we don’t know what the future is going to bring, but we discussed at length the first steps that are already being taken.

Daniel Murillo, Senior Product Marketing Manager, Tricentis
Raul DeLeon, Senior Professional Services Consultant, Tricentis

This session, titled “Talking Performance Engineering,” covered terms and terminology referenced by performance testing teams. Daniel and Raul explored how best to communicate in a common language to convey the value of results more effectively. This included a discussion on reporting and KPIs, different roles and use cases, and effective integration strategies.

Alan Gordon, Performance Engineer & Product Manager, Altersis Performance

Alan, who specializes in performance testing/engineering, APM solutions, and Agile and DevOps testing, led a session titled “Adventures in CPE: Lessons learned from 8 years of shifting left performance.” Alan discussed his own experiences as he transformed his organization, moving performance engineering from an infrequent task completed by specialists to one that’s consistent and repeatable. The lessons he learned, as well as the solutions to challenges he faced, were insightful to hear!

James Pully, Chief Performance Officer, QA Consultants

James, who’s been a voice in the industry for many years now, spoke on “Early performance engineering: How to ensure your sample set is large enough to measure performance.” James talked about the business justification of performance engineering. He also explored how approaching the exercise with a mindset of financial justification can make the entire process not only easy to fund and expand, but also build collaboration across teams and get everyone invested in a performance journey.

Scott Moore, head of Scott Moore Consulting, owner of the popular Software Engineering with Scott Moore YouTube channel and the SMC Journal podcast

In his session, “Using eBPF Metrics During a Load Test,” Scott discussed Extended Berkley Packet Filter (eBPF) metrics, and how as software systems march toward containerization, new insights into the underlying behavior of systems can be gleaned through implementation and collection of metrics at this layer.

Alex Podelko, Senior Performance Engineer, Amazon Web Services

Alex discussed how the implementation of a continuous performance testing practice requires not new software platforms, but a change in how you leverage existing platforms. He showcased the benefits of modifying your process flows and highlighted the outcomes you can receive at various stages of the application delivery process.

The PAC is a tremendously rewarding experience for me to run. It provides a wealth of knowledge for me to help structure my own understanding of the performance engineering space, and I am pleased that the content is available for anyone to consume and think about in 2024. I will be writing in-depth articles discussing the various aspects of each presentation and thoughts from the groups over the coming months, as my own understanding of the content deepens over time.

Don’t forget to check out the PAC presentations, recordings, and transcripts! I hope you learn useful strategies to implement that help you on your own performance engineering journey.