OpenSignal Comment Time: You Ask, We Answer

We recently got a comment on Facebook from one of our users about the results of our most recent USA report. Russell asked many great questions about our methodology and our business model.  We thought other users might be asking the same things, so we've answered your questions in this blogpost.

How we get our data

The measurements on which we based our latest USA report were collected from more than 237,000 consumer smartphones around the country. Not only do we collect and analyze measurements from an array of smartphones, we collect them 24 hours a day (versus at only one point in time) in all the locations where people live work and travel, even inside buildings (which not all collection methods can do) — in bustling metros, remote countrysides and everywhere in between.

We collect 2 billion individual measurements every day from tens of millions of smartphones globally. We can confidently say that we analyze consumers’ real-world mobile network experience at the largest scale and frequency in the wireless industry. Most importantly, our analysis is completely independent. We do not partner with any operator to build our universe of testers. They are all independent, regular folks interested in the truth about the service they receive on their mobile network. Nor do we run tests when it’s most convenient for an operator. We test all networks globally, every single day. In this way, we are able to offer a completely transparent and independent view of network experience as reported by mobile subscribers themselves.

  What we claim… and what we don’t

We are very careful never to claim that an operator is the overall “best" or the “worst”. These are subjective terms: different aspects of network experience matter more to different people and other factors such as pricing and customer service matter, too. Some users might prefer an operator with the highest speed while others might be interested in a reliable connection. Avid consumers of video content might care more about latency so they aren’t left long waiting for videos to start. We report what the measurements show and leave it to consumers to use our data to inform their choices.

Using third-party data to backup network performance claims is a common practice in the mobile industry (and others). Both T-Mobile and Verizon use third-party data to substantiate network performance claims. It’s up to consumers to decide which third-party data they find most credible and useful. It’s also true that operators offer different plan features and service levels to suit all kinds of subscriber needs. Consumers are definitely well advised to check the service agreement details carefully when selecting a plan.

Our business model

Behind all of the independent data we publish to consumers and the industry we do, of course, have a commercial model to support it. We have a range of products and services that provide more granular insight into the mobile network experience of consumers globally, and we are proud that these insights are used across the industry by mobile operators, telecoms regulators, equipment manufacturers and analysts. Why do they work with us?  We like to believe it’s because they share our belief that measuring how the network performs directly through users’ eyes is key to building better wireless networks.

We are, in fact, proud to be a champion of change in the wireless industry.  As (data) scientists, we  know observing real-world subjects yields far more accurate results than laboratory testing, which is why we are so passionate about using measurements from actual users, versus from predictive models, as the basis for our analysis. Championing a new user-centric approach to measuring network experience will inevitably invite detractors vested in the status quo, but it will not deter us from our commitment to independently report the truth in mobile network experience.