Broadband Network Experience Metrics

Broadband Consistent Quality (BCQ) measures the network experience from the perspective of a single device, assuming that connectivity has first been established. BCQ specifically measures whether the network can consistently meet the needs of common tasks for a single device. It sets thresholds for performance metrics like download and upload throughputs at levels appropriate for individual rather than multiple device usage (unlike the broader sufficiency thresholds used in the Broadband Reliability metric).

BCQ uses six key performance indicators: download and upload throughputs, latency, jitter, packet loss, and time to first byte, setting thresholds appropriate for individual rather than multiple device usage. These components are evaluated against thresholds recommended for various demanding common applications used for a range of common tasks.

To calculate the metric value, the proportion of tests that pass the requirements of CQ is multiplied by the test success ratio, which is the proportion of completed tests to all tests conducted. Tests that pass indicate that activities such as video calling, uploading an image to social media, or using smart home applications will be possible without noticeable lag or slowdown.

The pre-determined thresholds for the component performance metrics are:

• Download throughput > 5 Mbps
• Upload throughput > 1.5 Mbps
• Latency < 50ms
• Jitter < 12ms
• Packet discard < 1%
• Time to first byte <0.8s

Opensignal’s adaptive video experience quantifies the quality of video streamed to mobile devices by measuring real-world video streams over an operator's network. The metric measures users’ adaptive video experience using a Mean Opinion Score (MOS) approach inspired by International Telecommunication Union (ITU) studies which have derived a relationship between technical parameters of adaptive bitrate video streaming and the perceived video experience as reported by real people.

The videos tested are streamed directly from the world’s largest video content providers and include a wide selection of resolutions that dynamically match the network conditions, available bandwidth and device performance. Resolutions range from 144p to 2160p, which is also called 4K or UHD (Ultra High Definition). The model calculates a MOS score on a 0 to 100 scale by evaluating a number of parameters, including: the time to start playing the video, the quality of the video, the time playing each resolution, and the time spent re-buffering.

Measured in Mbps, Opensignal's Download Speed represents the average download speed a user experiences over Wi-Fi across an operator’s broadband network from an application host (CDN).

Measured in Mbps, Opensignal's Upload Speed represents the average upload speed a user experiences on WiFi across an operator’s broadband network to an application host (CDN).

Opensignal's Broadband Reliability Experience measures the ability of a household to connect to the internet and to successfully complete 'uninterrupted' tasks across multiple devices, encompassing work and recreational activities.

Calculated on a scale of 100-1000 — with higher scores being better — Reliability consists of the following components:

• Connectivity - successfully connecting to the broadband network. The percentage of tests that are connected to all eligible tests indicating whether users are able to connect to the WiFi network.
• Completion - successfully completing the tasks (e.g. streaming video, sending/receiving texts, making video and voice calls). The percentage of tests that are complete indicating whether users are able to complete a task when they are able to connect to the network.
• Sufficiency - having sufficient connection in order to start and sustain a ‘good enough’/’uninterrupted’ session. The percentage of tests that pass all the required thresholds of sufficiency indicating whether users have a sufficient session when they are able to connect to the network on completed tasks.

The required thresholds for a successful Sufficiency test are:

• Download throughput > 25 Mbps
• Upload throughput > 3 Mbps
• Latency < 50ms
• Jitter < 12ms
• Time to first byte <0.8s

Users' perceptions of reliability are inherently subjective, but an Opensignal study of 6,000 broadband users has established the following rules of thumb for understanding Reliability scores:

• 600 and above - Users who rate their home Internet as highly reliable typically have experienced reliability scores 600 and above. That means the connection has either never dropped or dropped very infrequently and service has been sufficient for usual activities almost all the time.
• 400 to 600 - Users have a range of perspectives about these levels of reliability. Factors such as pricing, competition, brand or access technology perception may factor as much or more into users’ perception of reliability when experiencing scores in this range, as their actual experienced home internet reliability.
• Under 400 - Users who rate their home Internet as having poor reliability typically have experienced reliability scores 400 and below. That means the connection has dropped too frequently and service has been insufficient for usual activities too frequently.