Opensignal has long been at the forefront of innovation when it comes to measuring real-world network experience. From introducing Availability (the percentage of time users can connect to a given network technology), followed by Video Experience and Games Experience, our metrics enable us to dive deeper into the real-world mobile network experience. In this explainer, we’re looking to showcase yet another set of pioneering metrics that help users quantify the quality of their network connections: Core and Excellent Consistent Quality.
Why two metrics?
Our consistent quality metrics are looking to provide an answer that users truly care about when it comes to their network health: will my apps work when I need them on my mobile service? It’s evident that some applications require a lot more bandwidth than others. Video streaming or video conferencing places greater demand on your network connection than checking your emails or web browsing.
That’s why we created two separate measures to determine consistent quality: Core Consistent Quality and Excellent Consistent Quality. They’re both based on the same set of key performance indicators, what’s different are the thresholds associated with each for a satisfactory network experience.
How do we calculate Consistent Quality?
We identified a set of six key performance indicators: download speed, upload speed, latency, jitter, packet loss, and time to first byte. We then calculate what percentage of tests meet or exceed each threshold for Core and Excellent Consistent Quality.
Core Consistent Quality is designed to represent common but less-demanding use-cases than Excellent Consistent Quality, specifically standard-definition video streaming, web browsing, email use, and photo-sharing.
The Excellent Consistent Quality thresholds have been designed using the latest available network performance recommendations from services like Netflix, or Skype, as well as live video streaming on services like YouTube and streaming video games.
Both these metrics are represented as a percentage in our reports, translating to the percentage of user tests that have met the minimum recommended performance thresholds. Users can interpret these metrics to mean the proportion of the occasions when they go to use their devices for typical applications that the connection will support that usage.
Looking for some more details?
Download and upload speeds
Download and upload speeds remain an important factor in network experience. A download speed of 5 Mbps is sufficient to stream HD video from any of the most popular streaming services, while the Core Consistent Quality threshold of 1.5 Mbps is in line with streaming standard-definition video. Our Excellent Consistent Quality upload speed threshold of 1.5 Mbps means users can conduct a video chat, upload images or videos to Snapchat, or send emails with large attachments without noticeable delay.
Once the throughput thresholds have been reached, latency is arguably the biggest difference-maker in user experience. Our threshold of 50 ms is fast enough for online games like Fortnite, as well as voice and video calling. At faster download speeds, lower latency is also the most significant factor in web page load time. Using the 100 ms latency threshold for our Core quality measurement ensures a connection is still usable, although subscribers will notice a difference in performance.
Jitter is a measure of the variation in latency, and has a major impact on the quality of real-time services like voice and video calling. A low average latency is fine, but if the packet latency jumps between 25 ms and 75 ms regularly, users will see video artifacts and jumpiness, which degrades the experience. The excellent quality jitter threshold of less than 12 ms ensures that operators are achieving a consistently low latency, rather than just a low average.
Packet discard reflects the percentage of data packets that fail to arrive at the destination and in the correct order. It has the greatest impact on video calling, where missing packets result in jumps in audio and video, but at a high percentage, can also cause noticeable slowdowns in web browsing or data downloads. More demanding use cases are less tolerant of a high packet discard percentage, which explains the low threshold for packet discard on our Excellent quality measurement.
Time to first byte
Time to first byte represents the time taken for a connection to be established. Adding this KPI helps to represent the full experience of the user from the moment they attempt to connect to a service. This is because a slow initial connection time will make the download experience feel slower, even if the speed is fast once the connection is made. The time to first byte threshold is set in line with the required download speed for each tier of Consistent Quality.
Are the thresholds set in stone?
Absolutely not. As consumer habits change and applications continue to improve, the definition of what constitutes as a sufficient network connection will also continue to change. Opensignal continues to monitor the network requirements of popular applications, as well as engage with industry experts on the measurement of end-user mobile network experience. As a result, the thresholds used for Consistent Quality are subject to revision in order to provide the most accurate representation of mobile network experience.
Find out how operators in your country rank in Core and Excellent Consistent Quality in our latest market insights!
Opensignal Limited retains ownership of this insight including all intellectual property rights, data, content, graphs & analysis. Reports and insights produced by Opensignal Limited may not be quoted, reproduced, distributed, published for any commercial purpose (including use in advertisements or other promotional content) without prior written consent. Journalists are encouraged to quote information included in Opensignal reports and insights provided they include clear source attribution. For more information, contact [email protected].