Telco 2.0™ Research

The Future Of Telecoms And How To Get There

MobiNEX: The Mobile Network Experience Index, Q4 2015

Summary: STL Partners quantifies the customer ‘app experience’ on twenty-seven mobile networks in seven countries. MobiNEX - The Mobile Network Experience Index - benchmarks mobile operators' network speed and reliability by measuring download speed; average latency; error rate and latency consistency. It’s based on billions of real customer data points provided by our partners Apteligent. Congratulations to the top three performers – Bouygues, Free and Orange (all in France). (Executive Briefing Service, April 2016) 

MobiNEX: The Mobile Network Consumer Experience Index, Q4 2015

 

Read in Full (Members only)   To Subscribe click here 

Update (October 2016): We have released a follow-up MobiNEX report using data from H1 2016.
You can view an extract here.

 

Below is a 3 page extract from this 30 page Telco 2.0 Report that can be downloaded in full in PDF format by subscribers to the Executive Briefing Service here. The full report includes an Executive Summary, along with further detailed content and figures outlined below the extract. To find out more about how to join or access this report please see here or call +44 (0) 207 247 5003.

Data for the report was provided by mobile app intelligence specialist Apteligent.

Executive Summary

In response to customers’ growing usage of mobile data and applications, STL Partners has developed MobiNEX: The Mobile Network Customer Experience Index, which benchmarks mobile operators’ network speed and reliability by measuring the consumer app experience, and allows individual players to see how they are performing in relation to competition in an objective and quantitative manner.

We assign operators an individual MobiNEX score based on their performance across four measures that are core drivers of customer app experience: download speed; average latency; error rate; latency consistency (the percentage of app requests that take longer than 500ms to fulfil). Apteligent has provided us with the raw data for three out of four of the measures based on billions of requests made from tens of thousands of applications used by hundreds of millions of users in Q4 2015. We plan to expand the index to cover other operators and to track performance over time with twice-yearly updates.

Encouragingly, MobiNEX scores are positively correlated with customer satisfaction in the UK and the US suggesting that a better mobile app experience contributes to customer satisfaction.

The top five performers across twenty-seven operators in seven countries in Europe and North America (Canada, France, Germany, Italy, Spain, UK, US) were all from France and the UK suggesting a high degree of competition in these markets as operators strive to improve relative to peers:

  • Bouygues Telecom in France scores highest on the MobiNEX for Q4 2015 with consistently high scores across all four measures and a total score of 76 out of 100.
  • It is closely followed by two other French operators. Free, the late entrant to the market, which started operations in 2012, scores 73. Orange, the former national incumbent, is slightly let down by the number of app errors experienced by users but achieves a healthy overall score of 70.
  • The top five is completed by two UK operators: EE (65) and O2 (61) with similar scores to the three French operators for everything except download speed which was substantially worse.

The bottom five operators have scores suggesting a materially worse customer app experience and we suggest that management focuses on improvements across all four measures to strengthen their customer relationships and competitive position. This applies particularly to:

  • E-Plus in Germany (now part of Telefónica’s O2 network but identified separately by Apteligent).
  • Wind in Italy, which is particularly let down by latency consistency and download speed.
  • Telefónica’s Movistar, the Spanish market share leader.
  • Sprint in the US with middle-ranking average latency and latency consistency but, like other US operators, poor scores on error rate and download speed.
  • 3 Italy, principally a result of its low latency consistency score.

Surprisingly, given the extensive deployment of 4G networks there, the US operators perform poorly and are providing an underwhelming customer app experience:

  • The best-performing US operator, T-Mobile, scores only 45 – a full 31 points below Bouygues Telecom and 4 points below the median operator.
  • All the US operators perform very poorly on error rate and, although 74% of app requests in the US were made on LTE in Q4 2015, no US player scores highly on download speed.

 

MobiNEX scores – Q4 2015

MobiNEX

 Source: Apteligent, OpenSignal, STL Partners analysis

 

MobiNEX vs Customer Satisfaction

MobiNEX

Source: ACSI, NCSI-UK, STL Partners

 

 

 

...to access the other 27 pages of this 30 page Telco 2.0 Report, including...   

  • Introduction
  • Mobile app performance is dependent on more than network speed
  • App performance as a measure of customer experience
  • MobiNEX: The Mobile Network Experience Index
  • Methodology and key terms
  • MobiNEX Q4 2015 Results: Top 5, bottom 5, surprises
  • MobiNEX is correlated with customer satisfaction
  • Segmenting operators by network customer experience
  • Error rate
  • Quantitative analysis
  • Key findings
  • Latency consistency: Requests with latency over 500ms
  • Quantitative analysis
  • Key findings
  • Download speed
  • Quantitative analysis
  • Key findings
  • Average latency
  • Quantitative analysis
  • Key findings
  • Appendix: Source data and methodology
  • STL Partners and Telco 2.0: Change the Game
  • About Apteligent

...and the following figures...

  • MobiNEX scores – Q4 2015
  • MobiNEX vs Customer Satisfaction
  • Figure 1: MobiNEX – scoring methodology
  • Figure 2: MobiNEX scores – Q4 2015
  • Figure 3: Customer Satisfaction vs MobiNEX, 2015
  • Figure 4: MobiNEX operator segmentation – network speed vs network reliability
  • Figure 5: MobiNEX operator segmentation – with total scores
  • Figure 6: Major Western markets – error rate per 10,000 requests
  • Figure 7: Major Western markets – average error rate per 10,000 requests
  • Figure 8: Major Western operators – percentage of requests with total roundtrip latency greater than 500ms
  • Figure 9: Major Western markets – average percentage of requests with total roundtrip latency greater than 500ms
  • Figure 10: Major Western operators – average weighted download speed across 3G and 4G networks (Mbps)
  • Figure 11: Major European markets – average weighted download speed (Mbps)
  • Figure 12: Major Western markets – percentage of requests made on 3G and LTE
  • Figure 13: Download speed vs Percentage of LTE requests
  • Figure 14: Major Western operators – average total roundtrip latency (ms)
  • Figure 15: Major Western markets – average total roundtrip latency (ms)
  • Figure 16: MobiNEX benchmarks

 

...Members of the the Executive Briefing Service can download the full 30 page report in PDF format here. For non-members, to find out more about how to join or access this report please see here or call +44 (0) 207 247 5003. 

Technologies and industry terms referenced include: 3, 500 ms, App, Apteligent, Bouygues, Customer Experience, Customer Satisfaction, Downlink, Download Speed, E-Plus, EE, Error Rate, Free, Index, Latency, LTE, Mobile, MobiNEX, Movistar, Network, O2, Operator, Orange, Speed, Sprint, STL, T-Mobile, Telco 2.0, Wind