Benchmarking Malaysia’s 4G mobile network experience before the JENDELA initiative

Recently, the Government of Malaysia announced a multi-billion dollar national digital infrastructure plan, Jalinan Digital Negara (JENDELA), to improve the country’s digital infrastructure and connectivity. As a part of this approach, the initiative will work to strengthen existing 4G networks and establish a solid foundation for 5G.

The Malaysian telecom regulator, MCMC, further revealed in its National Digital Infrastructure Lab (NDIL) report that this multi-billion dollar project will be executed in two phases. The first phase, which runs until end-2021, will see a gradual shutdown of the existing 3G networks, improvement of the existing 4G network for the deployment of 5G, and targets to widen coverage to reach all populated areas with higher average mobile broadband speeds. Phase 2 will run from 2022-onwards and will focus on ensuring nationwide 4G coverage as well as 5G planning and roll out. It will aim to reach wireless speeds of 100 Mbps using 5G.

In this insight, we look at the current state of mobile network experience across 13 states and three federal territories of Malaysia. We focused on different aspects of mobile network experience as outlined for Phase 1 plans in the NDIL report. These should also act as a yardstick by which to measure future mobile progress.

A digital divide exists between East and West Malaysia when it comes to 4G access

Since the NDIL report emphasizes 4G reach and accessibility, let us first look at 4G Availability — the proportion of time our 4G users spent connected to 4G services. Opensignal found that national 4G Availability stands at 86%, which means that on average our 4G users in Malaysia are able to access a 4G network 86% of the time. When we look at 4G Availability in the states and federal territories, we observed that a digital divide exists between West and East Malaysia. Our users in regions of West Malaysia saw relatively higher 4G Availability than their counterparts in East Malaysia.

At the top of the list for 4G Availability is the federal territory and the capital city of Kuala Lumpur (91.8%), followed by Pulau Pinang (89.7%) and Putrajaya (89.5%). Our Malaysian 4G users in seven regions, including Selangor, Melaka, Perlis and Labuan, reported 4G Availability above the national average of 86%. At the same time, those in Johor and Kedah were less than one point away. In contrast, users in Sabah, Sarawak and Kelantan experienced significantly lower 4G Availability, reporting 80% or less.

By re-using 3G spectrum, Malaysian operators could more quickly deploy additional capacity to support the increasing user demand for 4G. However, the mobile industry and Malaysian government will need to take steps to ensure that digital exclusion does not increase and that users have the ability to connect to 4G. 

4G Download Speed needs to improve by almost three-fold to match the planned 35 Mbps benchmark

With Phase 1 of the JENDELA project, the government also wants to provide the rakyat with high-speed mobile internet with speeds averaging 35 Mbps. But when we analyzed current real-world national 4G speeds we found that Malaysian users observed an average 4G Download Speed of 12.7 Mbps, or approximately one third of the planned goal. At Opensignal, our tests are run to the typical servers and content delivery networks that host popular mobile apps and websites, so our results reflect the real user experience. 

At a regional level we see an extremely varied mobile experience. Regionally, the average 4G Download Speed ranged between 16.5 Mbps (Putrajaya) to 10.5 Mbps (Kedah). Our users in seven regions saw 4G Download Speeds averaging above the national score, while those in the remaining nine regions, including Kelantan, Sabah and Perlis, saw comparatively similar or lower 4G Download Speeds.

With the planned sunset of 3G networks and added capacity by the end of Phase 1, mobile download speeds should grow, provided operators can keep pace with growing data usage. To help reach the 35 Mbps speed target, MCMC also plans to boost network reach and improve signal power levels by deploying more new cell towers and by upgrading 1,972 existing transmitters in Sabah. Demand plays a huge role in the typical everyday speeds that consumers experience. The more users connect to a network, the more its capacity is spread thin, which can cause average speeds to fall. Though operators tend to concentrate their network and capacity upgrades in the big cities, often those improvements can't keep ahead of the mobile data demands generated by dense population centres.

Moving towards Quality of Experience, rather than just QoS, is the right approach

Unlike the National Fiberisation and Connectivity Plan (NFCP) or other initiatives launched earlier, the JENDELA project has incorporated Quality of Experience (QoE) into its plans because speeds alone do not enable a high-quality mobile service. We see this when we compare the results of mobile download speed with Opensignal’s Video Experience and other experiential measures. Often, operators manage types of mobile data traffic differently, which makes it impossible to extrapolate from speed to how mobile users’ smartphone apps behave in the real-world.

Malaysian smartphone users wait 3 to 4.3 seconds for a video to load and begin streaming while connected to mobile networks

One of the Quality of Experience (QoE) benchmarks recommended in the NDIL report is Video Streaming — HD 720p video without buffering, 90% of the time. But Opensignal’s mobile analytics indicate that a user’s perceived video experience depends on a combination of technical characteristics that we measure, including: the time for a video to start playing; the video resolution; the stalling time; and the video codec used, among other factors. Together, we combine these measurements using an ITU-based mean opinion score (MOS) methodology to create a score of users’ experience. We measure video experience on a scale from 0 to 100 — the higher the score, the better the video experience. 

Opensignal looked at the 4G Video Experience in Malaysia across a range of video resolutions and found that users enjoy a “Good” 4G Video Experience at a national level. This result was consistent with the experience in most regions, except Putrajaya and Kuala Lumpur, where users enjoyed a “Very Good” 4G Video Experience. This means that video streams exhibited generally fast loading times and only occasional stalling, but the experience might be somewhat inconsistent across users and/or video providers/resolutions, while a Good rating indicates a less consistent experience, even from the same video streaming provider and particularly for higher resolutions, with noticeably slower loading times and stalling not being uncommon.

We also looked at one of the supporting metrics — 4G Video Load Time — which measures the average amount of time in seconds, a consumer waits for a video to load and begin streaming while on a 4G connection. Our Malaysian users experienced a noticeable delay before mobile video began playing. In Putrajaya users enjoyed the fastest video load time of 3 seconds, followed by Kuala Lumpur (3.2 seconds), Selangor and Pahang (3.4 seconds), while their peers in the rest of the regions reported video-load time between 3.5-3.9 seconds. Sabah was the only region that lagged with a video load time of 4.3 seconds. 

Even these small amounts of delay have a significant impact on users. A study by University of Massachusetts Amherst and Akamai found that viewers start to abandon a video if it takes more than 2 seconds to start up, with each incremental delay of 1 second resulting in a 5.8% increase in the abandonment rate.

4G Voice App Experience: Malaysia already ahead with low UDP latency, but packet loss needs attention

Looking at the quality of experience for over-the-top (OTT) voice services — mobile voice apps such as WhatsApp, Skype, Facebook Messenger, etc. The national 4G Voice App Experience of our Malaysian users was in the “Acceptable” category. Like 4G Video Experience, the regional 4G Voice App Experience did not vary significantly from the national experience: Putrajaya was the only region where our users perceived a Good experience.

An Acceptable rating means users were satisfied with the quality of the call and were generally able to comprehend without repetition, while some may have suffered from perceptible call quality impairments such as clicking sounds or distortion. Likewise, a “Good” rating means many users were satisfied, while some experienced minor call quality impairments.

When we looked into metrics related to Voice App Experience — 4G Packet Loss and 4G UDP Latency — again we saw large regional differences indicating the importance of a national initiative to help Malaysians. Our users saw latency between 127.8 ms (Putrajaya) to 205.5 ms (Sabah) depending on the region they were located in. Meanwhile, users’ packet loss ranged between 1.6% and 3.5%. The NDIL has set out the goal for Malaysia to achieve under 150 ms of round-trip latency delay and under 0.5% of packet loss on voice over IP (VoIP) calls.

Games Experience ranges from Poor to Fair

We found that playing multiplayer mobile games or Esports over cellular networks in Malaysia can be challenging. Our users in most regions perceived a “Poor” mobile multiplayer Games Experience, except for Putrajaya, Kuala Lumpur, Pahang and Selangor, where users reported a “Fair” Experience.

But what factors contribute to a good Games Experience? There are a range of technical network characteristics that together affect the Games Experience, including: UDP Latency, Packet Loss and Jitter. In Opensignal’s testing we have found that often there is a complicated, and nonlinear, relationship between these three inputs and their effect on a player’s experience. Again, Opensignal’s Games Experience measure uses a model that converts real-world technical measurements into a score of users’ mobile experience.

The variation in mobile network experience highlights the connectivity gap that exists between the densely and sparsely populated regions, and the challenges faced by mobile operators to provide uniform mobile network experience across Malaysia's vast archipelagic geography. The JENDELA initiative should help to narrow the digital divide. 

In our recent insight, we saw that the introduction of the Movement Control Order (MCO) coupled with free data services and relaxed data limits in Malaysia had a significant impact on our users’ experience and revealed the vulnerabilities in the nation's existing digital infrastructure. While Malaysia is still adjusting to the new normal of 2020, the pandemic has given the digital inclusion agenda a new-found sense of urgency due to the need to enable the Malaysian population to continue to work, learn, stay connected and get entertained. That’s likely one of the reasons why the NDIL formulated the JENDELA project to help accelerate the expansion of 4G and improve Malaysia’s digital connectivity. And in the run-up to Malaysia becoming a 5G country, existing networks will be important and will be an essential foundation for the smooth transition to next-generation 5G networks.