Top 5 metrics for streaming video performance

On-demand Internet streaming media has exploded over the past couple of years — and shows no sign of slowing down. According to Cisco, by 2022 82% of all global Internet traffic will come from streaming videos and downloads. Today more video content is uploaded in 30 days than what major US television networks have created in the past 30 years.

Virtually every media company has jumped on the online streaming video bandwagon. The notion of “cord-cutting” has become a phenomenon — which entails getting rid of your cable and substituting online video streaming subscription services. The plethora of direct-to-consumer platforms may well devolve into guerilla warfare: some will survive, and others will go by the wayside

As more and more people expect flawless on-demand videos — and they have more and more options — performance becomes a major differentiator. Providing reliable, resilient video streaming is a must-have for media and entertainment companies. A truly resilient live video stream delivers a smooth and consistent experience for viewers, without gaps, stalls, or silences.

What do you need to know about load testing video streaming performance? Here are the top 5 metrics to help you guarantee great video streaming performance for your consumers.

Top 5 metrics to measure

Here are the 5 metrics you should be tracking in order to make sure your video streaming is performing top notch.

1. Bit rate

Bit rate might be the most important metric to measure, as it helps you understand the quality of the video that you users are experiencing. A higher average bit rate means a higher quality image (for a given screen resolution). The bit rate indicates how many bits of video (or information in general) can be transmitted over a specific period of time. High-definition television is typically transmitted at a bit rate of 8-15 Mbps, while for Netflix it’s about 6 Mbps.

2. Buffer fill

The time spent filling in the buffer when the video is first started. This is important because you want to know how long your users are waiting before a video actually starts playing. If they are staring at that rotating circle for too long, they may leave before even a frame flashes on their retinas.

3. Lag length

After the buffer is initially filled, the video starts playing. At that point, if all goes well (meaning the download rate can keep up with the bit rate), the viewer will see their content in one smooth playback. However, this isn’t always the case — sometimes the buffer is drained, and without enough video to continue, playback halts. All the time spent waiting (including the initial buffer fill) is collectively called the lag length, and is an important metric for understanding the viewer’s experience. Ideally your lag length isn’t any greater than the initial buffer fill. If it is, you know that your consumer’s experience has been jarringly interrupted.

4. Play length

The amount of data actually streamed out of your datacenter — the actual seconds, minutes, and hours of content that have been consumed by your users. This is an important metric to understand for capacity and infrastructure planning, and also helps you estimate peak data volumes and the overall demand for streamed data.

5. Lag ratio

The lag ratio is easily calculated as waiting time over watching time. Because there is always an initial buffer period, you can’t ever quite get this ratio to zero. But you can get it to be pretty low. Think about it this way: the last time you watched a 2-hour movie, how many of those minutes were you staring at a “buffering…” message? How many would you tolerate before you turned it off? 3 minutes? That’s a 2.5% lag ratio.

Technologies that improve performance also complicate measurement

Clearly, consumers want video streaming services that are available (they can reach the videos they want), consistent (they don’t re-buffer), and high-quality (they look great) — probably in that order. All of this requires a high-functioning data center, high-throughput and resilient networks, and a consumer device (their TV or mobile phone) that’s beefy enough to handle the video along with anything else the user might be doing at the same time.

But you don’t always get all that, and so technologies like Adaptive Bitrate Streaming have been developed. Here, the server changes the quality of the video it is streaming when the viewer’s device can’t receive it or process it fast enough. So if you have ever noticed that a video’s quality goes down because your computer got busy with a virus scan, for example, that’s what’s happening. The bit rate is adapting — getting lowered — to make sure that the viewer’s video is not interrupted.

Adaptive bitrate streaming is built into technologies and products like MPEG-DASH, Adobe HTTP Dynamic Streaming, Apple HTTP Live Streaming, and Microsoft Smooth Streaming. By doing this, they help address a big concern your users have — the sudden interruption of a video with a “loading” message. However, if you are trying to load test your system and make sure your consumers are getting the highest-quality video and best possible experience as well, this technique actually complicates things.

Most load testing tools can’t accurately measure the metrics outlined above when adaptive bitrate streaming technologies are in place. So while your consumers aren’t experiencing playback interruptions, they may still be getting angry at fuzzified frames and muffled colors.

Load test your streaming video properly

The advanced technologies found in Tricentis NeoLoad make it an excellent load testing platform for streaming video. With NeoLoad you can:

• Accurately assess key streaming metrics, even with adaptive bitrate streaming.
• See how a realistic viewership impacts your performance to plan for peak viewing – going so far as simulating geographically dispersed viewers all over the world operating TVs, PCs, and mobile devices.
• Design the video streaming infrastructure you need to keep consumers watching the videos they want, on the devices they want, at the quality they deserve.

Don’t make your consumers mad

Customer satisfaction has always been of concern for consumer-facing companies, but social media has upped the ante significantly

Think about the customer experience when they run into poor streaming video performance and how much a brand can be damaged by poor video streaming. According to a survey conducted by UMass-Amherst, consumers give up on an online video if it doesn’t load in two seconds. A dissatisfied customer will tell between 9-15 people about their experience. Around 13% of dissatisfied customers tell more than 20 people. Just look at how angry Twitter got when Super Bowl ads started crashing.

Showtime, HBO Max, and Disney+ all had service outages ahead of season finales or live events. Back in March when Disney+ aired its finale for WandaVision, some 2,300 viewers reported outages to Down Detector almost as soon as the episode was released. Almost 50,000 viewers complained when HBO Max experienced problems with the Mare of Easttown finale. And Showtime had to refund about 20,000 subscribers who couldn’t watch the Floyd Mayweather-Logan Paul exhibition match.

No company wants to be the subject of a Twitterstorm or wind up on the fron pages for the wrong reason. Performance testing is important and not many tools can test streaming video performance well.

Check out NeoLoad. It will (1) make sure your video streaming performance is top notch and (2) prevent you from making your customers angry.

The post was originally published in 2014 and was most recently updated in July 2021.