The State of Mobile Video (September 2018)

For years, Opensignal has reported on the speed, availability and latency metrics of the world's mobile operators. But now we're taking our analysis of the mobile user experience one step further. Keeping with our mission of measuring what real people actually see on their mobile phones, we are examining the single-most important service on mobile networks today: video. Opensignal analyzed the consumer mobile video experience in 69 countries around the world, and the results were definitely surprising. Countries with the most sophisticated networks and the fastest speeds aren't necessarily those providing the highest-quality video-viewing experience. In the case of mobile video, faster isn't always better.

Highlights

Many countries provide a Very Good video experience

Eleven of the 69 countries we analyzed earned a Very Good rating on Opensignal's video experience scale, meaning mobile video loaded quickly and rarely stalled even at higher resolutions. But even among those elite nations there is still room for improvement. No country achieved the highest video experience rating of Excellent.

The fastest country doesn't offer the best mobile video quality

South Korea was by far the fastest of the 69 countries we analyzed in this report, but 15 other countries ranked higher in video experience. The best video experience we recorded was in the Czech Republic.

European countries top our mobile video chart

European countries tended to outperform the rest of the world in mobile video experience. Not only did an EU nation top our list, but of the 11 countries that earned a Very Good score, nine of them were in Europe.

The relationship between speed and video experience is complicated

Opensignal's analysis shows that video experience and connection speed are linked in countries where speeds are relatively slow, but once a country passes the 15 Mbps threshold in average overall download speed, the raw power of connections has little bearing on streaming video quality.

Overall Video Experience

This chart shows the video experience scores for each country. Opensignal's video experience metric is derived from an ITU-based approach for determining video quality. The metric calculation takes picture quality, video loading time and stall rate into account. We report video experience on a scale of 0-100, with scores falling into the following categories:

  • 75-100 Excellent
  • 65-75 Very Good
  • 55-65 Good
  • 40-55 Fair
  • 0-40 Poor
Overall Video Experience Comparison

Overall Speed

This chart shows the overall download speed that users in each country see over its data networks. We define overall speed as the average mobile data connection a user experiences based on both the speeds and availability of a country’s 3G and 4G networks.

Overall download speed measurements vary considerably from country to country depending on their particular stage of 3G and 4G development. For instance a country with fast LTE speeds but low 4G availability might have a much lower overall speed than a country with moderate LTE speeds but a very high level of 4G availability.

Overall Download Speed Comparison

The Full Spectrum of Mobile Video

This chart plots overall video experience against overall download speed for all of the countries covered in this report. Countries higher up and toward the right in the chart have both fast mobile broadband speeds and offer a better video experience, while those in the bottom left of the chart have slower speeds and offer a poorer video experience. As the graph indicates, speeds and video experience are loosely related, but it is possible to offer a good mobile video experience even without fast download speeds.

Comparing Overall Video Experience and Download Speed

The Global Picture

All of the countries examined in this report are shown on this interactive map, detailing the distribution of mobile networking capabilities across the world. Those countries that perform better in a particular metric are shaded darker, and you can select different metrics to view in the drop down menu. Countries not included in this report are shaded in grey. Our sample only included the countries for which we had enough test data to make a statistically meaningful analysis. As our user base grows, we’ll continue to add more countries to our reports.

Analysis

At Opensignal we have always prided ourselves on our ability to measure what the consumer experiences on mobile networks. We measure the real-world speeds consumers see on their smartphones, rather than the highly optimized connections typically reported. We track how often real users have access to 4G signals in the places they frequent, rather than using complicated models to predict coverage. We run our latency tests not to specialty test nodes, but to the servers from which consumers seek their content. Opensignal's philosophy has always been that the only metrics that matter are the ones that measure the true consumer experience. The rest are just ephemera.

Real-world speed and real-world availability are very useful for gauging the power and reach of an operator's 4G services, but speed and availability are still relatively abstract concepts. Consumers don't consume speed. They don't occupy themselves with availability. Instead, they're surfing the mobile internet, chatting on social media and streaming video. That's why we've decided to take another step forward in our approach. In addition to tracking the underlying metrics of the mobile networks, we've begun analyzing how the consumer experiences the apps and services they use on their mobile phones. We're kicking off this new forward-looking approach with an analysis of video experience. It seeks to answer a simple but enormously relevant question for modern mobile consumers: How good or bad does video render on my operator's network?

A first-of-its-kind measurement in the mobile industry, Opensignal's video experience metric is derived from an International Telecommunication Union (ITU)-based approach for measuring video quality. Our tests sample video at multiple resolutions accessed from multiple content providers, and they weigh three main criteria: the load time before the video begins playing, the stalling rate characterized by stops and stutters in the video playback, and the level of picture resolution. We measure video experience on a scale from 0 to 100 — the higher the score, the better the video experience. Furthermore, we divide those scores into ranges to determine their rating. A score that falls within 75-100 is Excellent, 65-75 is Very Good, 55-65 is Good, 40-55 is Fair and 0-40 is poor. An Excellent score is a big achievement indeed, meaning fast load times and practically non-existent stalling at all resolutions. As we descend down the ratings, loading times get longer and we encounter more stops and stutters in the video stream. At our worst rating, Poor, users don't have a good experience at all with very long waits for the video to start and frequent stalling even at the lowest resolutions. (For more details on our video experience metric see this blog post.)

For our analysis we examined 69 countries spread throughout the globe to see how they stacked up in video experience. We also ranked those same 69 countries in overall download speed and then plotted speed against video experience to see if we could define a relationship between the might of a country's mobile broadband networks and the quality of video customers see. We discovered that the relationship is a complicated one. Where mobile broadband connections are slow, speed has a big impact on video experience. But at faster levels, speed has little relation to the quality of video streaming. The countries with the fastest speed don't necessarily offer the best video experience.

In search of the perfect mobile video stream

As you can see from our overall video experience chart, no country falls into the Excellent (75-100) category for average video quality. An Excellent score would indicate that a country's operators are able to consistently support high-resolution video with fast loading times and nearly non-existent stalling, across their networks. 4G networks have come a long way since first introduced nine years ago, but the fact that no country in our list was able to meet the highest benchmark for video quality shows just how far mobile technologies need to progress.

Even the countries with the most sophisticated LTE networks had to settle for a Very Good (65-75) rating in our video measurements, and many countries renowned for their 4G services didn't even make that list. The Czech Republic topped our chart with an overall video experience score of 68.5, and only 10 other countries joined it among the video experience elite with a score of Very Good. Most of them were in central and northern Europe, while the United Arab Emirates and Singapore represented the Middle East and Asia. Few users in these countries would find fault with the video quality they received as a Very Good score generally means fast loading times and only occasional stalling. What primarily distinguishes a Very Good score from an Excellent score is less consistency in the video quality from different video sources and at different resolutions.

The vast majority of the 69 countries we examined fell into a relatively narrow range of scores between 40 and 65, earning them either Fair or Good ratings. So what does that mean exactly? It means for much of the world, the typical mobile video experience leaves something to be desired. Video load times are sluggish; stops and stutters mid-stream are common to varying degrees; and connections often have trouble coping with higher-resolution formats, especially for those countries in the Fair category. In general, European countries tended to rank higher than their counterparts in the Americas, while Asian and Middle Eastern countries are scattered throughout the rankings.

Finally, three countries, India, Iran and the Philippines, fell below the Fair threshold into Poor territory (0-40), where the typical consumer experience is characterized by frequent stalling during video playback and long loading times even for low-resolution video.

Why speed isn't everything

For the second part of our analysis, we calculated overall download speeds in the 69 countries we examined to compare against our video experience metric. Overall download speed takes into account both average 3G and 4G speeds and the availability of each type of network connection, and is intended to show the typical download speeds consumers experience on an everyday basis. As you can see from the chart, there is a large amount of variation in the overall speed scores of different countries — some pushed the 40 Mbps barrier, while others languished below 10 Mbps. But you'll also notice the rankings in overall speed are much different than the rankings for video experience, especially at the top end of the chart.

Things get really interesting when we plot overall speed against overall video experience in our Full Spectrum of Mobile Video chart. Here we clearly see there is some relationship between average speeds and video quality as our video experience scores generally improve the faster speeds get, but it's not an exact correlation. South Korea is the prime example. With an overall download speed of 45.6 Mbps, it was the fastest of the 69 countries by quite some margin. Yet South Korea was well short of the top mark in video experience. It didn't even make the cut of 11 countries in our Very Good ratings. Rather it landed in the Good tier, right alongside several countries with nowhere near the sheer mobile broadband might of the East Asian 4G powerhouse. Kuwait's average download speed was a mere 14.7 Mbps, but it was nearly level with South Korea in video experience. Clearly a good video experience is dependent on more than a lightning-fast mobile connection.

Where we see the biggest correlation between speed and video quality is among the countries with slower mobile connections. Nearly all of the countries that had Fair video ratings fell within a tight cluster below the 14 Mbps overall speed mark, and in this group a small increase in overall speed often coincided with a sharp climb up the video experience scale. At the lower end of the table, speed clearly is a big factor governing the quality of a video stream, and in those slower countries even a small boost in speed can have a big impact on video experience.

But right around the 15-Mbps threshold, our plot curve starts leveling off and the distribution of countries along that curve becomes far more random. That means the tight relationship between video experience and download speed becomes much more loose as average connection speeds climb higher. Beyond the 20-Mbps threshold, speed had relatively little say in the matter of how good video experience actually was in our measurements. Of the 11 countries that scored Very Good ratings, their overall download speeds ranged from 21 Mbps to 40 Mbps. The country with the best video experience score in this analysis, the Czech Republic, didn't even make the top 10 list in speed.

So if raw average throughput isn't determining the quality of experience in faster countries, then what is? Latency, which measures the response time of a network, has a big impact on our video experience metric, as the longer a device waits to hear back from the video server, the longer it takes for the video to load and begin playing. Consistency of connection speed is also an important factor. A super-fast connection isn't necessary to stream video over mobile networks, but a video needs relatively consistent throughput to avoid stalling. Thus a network that delivers a 50-Mbps connection one second and a 2 Mbps connection the following second is likely to provide an inferior overall video experience than a network that can maintain a constant 20 Mbps connection over a long duration of time. Maintaining consistent speeds isn't an easy thing for a mobile operator due to the nature of cellular networking. Unlike wireline broadband, where subscribers often have a dedicated line to the network, mobile networks operate through shared capacity. As the number of users and data demands change on any given cell, the mobile network is constantly redividing and redistributing its available capacity, which can create large fluctuations in individual connection speed from one moment to the next.

Video experience can also be heavily impacted by operator policy. Many operators globally use video optimization technologies to restrict the level of video resolution their customers can access on their phones. As our tests sample video at different resolutions, any downgrading of video quality — say from HD to SD — would have an impact on our scores.

The U.S. is a prime example of such policies at work. In our scatter plot chart, the U.S. is an outlier. It lands in the middle of the Fair range of video experience with a score of 46.8, but its average overall download speed of 16.5 Mbps suggests it should have scored much higher in video experience. Almost every other country with an overall speed of 16 Mbps or greater earned a Good video experience rating in our measurements. The explanation lies in the evolving nature of data plans in the United States. Unlimited plans are gaining huge ground, and to prevent their networks from becoming overloaded with video traffic, operators have put streaming restrictions on their different tiers of unlimited plans. For many users on unlimited plans in the U.S., the highest resolution video they can stream over a mobile connection is 480p.

So it stands to reason that if an operator lifts restrictions on video resolution, then its video experience scores would improve, right? Not necessarily. Depending on the type of video, a 720p stream can consume twice as much or more data than a 480p stream. And as video now accounts for the majority of all mobile internet traffic, a doubling of the gross tonnage of video consumption would have a major impact on any operator's network. More traffic leads to congestion, and congestion not only impacts overall speeds available to consumers but can also lead to inconsistent connections and poorer latencies — all of which have a bearing on video experience. Ironically, if U.S. operators were to lift video restrictions on their unlimited plans, it’s possible that the typical video experience in the U.S. might suffer more.

The future of mobile video

Speed has been the focal point of the mobile industry ever since the first LTE network came online nearly a decade ago. But speed is just an abstract measurement, not an indication of how consumers actually experience the internet or applications on their mobile phones. Speed certainly has a bearing on mobile video experience, but as this report shows, it's not the sole factor determining the quality of video we see in countries with sophisticated LTE services. We would argue that it's time for the industry to turn its attention away from raw speed and focus on the combination of factors that go into ensuring a good mobile internet experience. Video is the ideal place to start as it depends on so many elements: the availability of 4G connections, the latency of the network and the consistency of connections, as well as the individual data-shaping policies of operators.

Judged by the user experience rather than rote measurements, the mobile industry still has work to do when it comes to video. Countries with extremely powerful LTE networks in terms of download and upload speeds, availability and coverage, aren't always providing the best video experience. And no country we've analyzed has managed to achieve the highest level of video streaming quality. If operators want to boost the video experience they offer their wireless customers, building faster and faster LTE-Advanced networks isn't necessarily the answer.

It might well be the case that the best quality video experience may not come from a 4G network. 5G is often touted for its gigabit speeds, but more important for video are 5G's purported low latencies and high-density network architecture, which would respectively cut down on video load times and create the resilient connections necessary for consistent video playback. There's no question that 4G revolutionized mobile video, but in order for mobile video to reach its full potential, we may need to wait for 5G.

Our Methodology

Opensignal measures the real-world experience of consumers on mobile networks as they go about their daily lives. We collect 2 billion individual measurements every day from tens of millions of smartphones worldwide.

Our measurements are collected at all hours of the day, every day of the year, under conditions of normal usage, including inside buildings and outdoors, in cities and the countryside, and everywhere in between. By analyzing on-device measurements recorded in the places where subscribers actually live, work and travel, we report on mobile network service the way users truly experience it.

We continually adapt our methodology to best represent the changing experience of consumers on mobile networks and, therefore, comparisons of the results to past reports should be considered indicative only. For more information on how we collect and analyze our data, see our methodology page.

For this particular report, 87,011,812,678 datapoints were collected from 8,002,357 users during the period: May 14th - Aug 11th 2018.

Opensignal, Inc. All rights reserved.

Opensignal, Inc retains ownership of this report including all intellectual property rights, data, content, graphs & analysis. Reports produced by Opensignal, Inc may not be quoted, reproduced, distributed, published for any commercial purpose (including use in advertisements or other promotional content) without prior written consent.