Broadband Methodology Overview
Our Broadband Methodology Overview includes:
(click to jump to a section)
Methodology and definitions
Opensignal captures user measurements across all broadband network architectures, encompassing fibre (FTTx), xDSL, cable, fixed wireless access (FWA) via 4G and 5G, as well as satellite, and reports on a holistic evaluation of our users’ experience with the listed providers' internet services.
In our public reports, we only include the largest fixed broadband providers by market share in each area, determined by our user base size.
Tariff characteristics — for example, speed tiers or data caps — vary greatly by provider and the dispersion of the plan mix will affect the average experience result. Opensignal’s measurements capture users’ experience, regardless of the plan that they have purchased from their provider. This report analyzes the real-world situation across all users’ plans.
Consistent Quality
Consistent Quality measures if the network is sufficient to support common application requirements at a level that is ‘good enough’ for users to maintain (or complete) various typical tasks on their devices.
We combine different experience indicators such as download speed, upload speed, latency, jitter, packet discard, and time to first byte to calculate BCQ. These components are evaluated against thresholds recommended for various demanding common applications used for a range of common tasks.
To calculate the metric value, the proportion of tests that pass the requirements of BCQ is multiplied by the test success ratio, which is the proportion of completed tests to all tests conducted. Tests that pass indicate that activities such as video calling, uploading an image to social media, or using smart home applications will be possible without noticeable lag or slowdown.
The pre-determined thresholds for the component performance metrics are:
- Download Speed > 5 Mbps
- Upload Speed > 1.5 Mbps
- Latency < 50ms
- Jitter < 12ms
- Packet discard < 1%
- Time to first byte < 0.8s
Video Experience
Opensignal’s Video Experience quantifies the quality of video streamed to mobile end user devices by measuring real-world video streams over an operator's networks. The metric is based on an International Telecommunication Union (ITU) approach , built upon detailed studies which have derived a relationship between technical parameters, including picture quality, video loading time and stall rate, with the perceived video experience as reported by real people.
To calculate video experience, we are directly measuring video streams from end-user devices and using this ITU approach to quantify the overall video experience for each operator on a scale from 0 to 100. The videos tested include a mixture of resolutions — including Full HD (FHD) and 4K / Ultra HD (UHD) — and are streamed directly from the world’s largest video content providers.
The following scale can be used to relate the Video Experience scores to the actual experience our users received:
Excellent (78 or above)
Our users were, on average, able to stream video at 1080p or better with fast loading times and no stalling.
Very Good (68 or more but less than 78)
Our users were, on average, able to stream video at 1080p or better with satisfactory loading times and little stalling.
Good (58 or more but less than 68)
Our users were, on average, able to stream video at 720p or better with satisfactory loading times and little stalling.
Fair (48 or more but less than 58)
Our users were, on average, able to stream video at 720p or better with satisfactory loading times and substantial stalling.
Poor (Under 48)
Our users, on average, encountered very high loading times or high levels of stalling or were only able to stream the video at resolutions below 720p.
Download Speed (average)
Measured in Mbps, Opensignal's Download Speed represents the average download speed a user experiences over Wi-Fi across an operator’s broadband network from an application host (CDN).
Download Speed (peak)
Measured in Mbps, Opensignal's Download Speed (peak) represents the peak download speed a user experiences across an operator’s broadband network from an application host (CDN). Peak speed is represented by the 98th percentile of all speeds tests.
Upload Speed (average)
Measured in Mbps, Opensignal's Upload Speed represents the average upload speed a user experiences on WiFi across an operator’s broadband network to an application host (CDN).
Upload Speed (peak)
Measured in Mbps, Opensignal's Upload Speed (peak) represents the peak upload speed a user experiences across an operator’s broadband network to an application host (CDN). Peak speed is represented by the 98th percentile of all speed tests.
Success Rate
Opensignal’s Success Rate measures the average proportion of successfully completed tests relative to all tests executed on an operator’s broadband network.
Reporting
Per Device Values: Per device values are combined using a simple average to yield the Opensignal metrics that are found in our reports and analysis. For each of our reports, we show details of the active user base we have sampled from to calculate the results. The sample size will differ for each operator, as users can only measure their own network providers’ performance.
Confidence Intervals: We provide an upper and lower estimate of confidence interval per operator, calculated using recognized standard techniques based on the sample size of measurements. Confidence intervals provide information on the margins of error or the precision in the metric calculations. They represent the range in which the true value is very likely to be, taking into account the entire range of data measurements.
Statistically significant results: Whenever the confidence intervals for two or more operators overlap in a particular metric, the result is a statistical tie. This is because one operator’s apparent lead may not hold true once measurement uncertainty is taken into account. For this reason, awards could have multiple winners in our reports.
Standardized geographical boundaries: A common practice in reports from other sources is to “cherry-pick” geographic boundaries or time periods to be able to make an observation about a specific operator. For example, highlighting performance for a particular area of a city, or over a particular time period. We do not do this and only report on standardized geographical boundaries (where available) and over the entire period covered by the measurements. Our reporting timetable is under our control and not released to operators in advance. This ensures that reports represent the consistent experience of the majority of users.