Mobile Experience Benchmark - MWC 2015 Edition

Introduction
This report provides an overview of three key aspects to
consider when developing mobile apps:
1. State of the mobile ecosystem, across operating systems
and devices
2. The digital customer experience for mobile apps, across
app store categories
3. Differences in key metrics for wireless networks, across
different markets & geos
The data in this report is aggregated from Crittercisms leading
mobile application performance management (mAPM)
solution that processes over 50,000 requests per second from
1 billion mobile users in over 120 countries, every day.

Whats the state of the mobile ecosystem?

Why the state of the mobile ecosystem matters

The smarts of Android on screens big & small
iOS8 is the biggest iOS release ever for developers and everyone else
Knowledge about the installed base of devices & OS inform design, feature and architecture decisions.

iOS 8 passing 50% of iOS app load traffic in 50 days

Apple ships a new OS bundled with a new flagship device, and pushes the OS update
immediately to the entire installed base of devices, resulting in quick uptake.
0%
10%
20%

50% share
of app load
traffic

30%

iOS 8

40%

50 days

50%
Sept 17th

Nov 5th

60%
70%
80%

iOS 7

90%

iOS 6

100%
Apr/14

May/14

Jun/14

Jul/14

Aug/14

Sep/14

Oct/14

Nov/14

Dec/14

Jan/15

Feb/15

Apple iOS, Share of App Load Traffic by Version, April 2014 January 2015

Other
5

iPhone 6 / 6 Plus reaching ~20% app load traffic within ~20 weeks
iPhone 5S as the previous flagship device achieved largest share of a single iOS device, indicating how
good Apple is at providing latest computing technology to consumers.
100%

iPhone 6 Plus

19% share of
app load
traffic

90%

iPhone 6

80%

~20 weeks /
4.5 months

Sept 17th
70%

Feb1st

60%

iPhone 5S

iPhone 5C

50%

iPhone 5

40%

iPhone 4S

30%

iPhone 4
20%

Other

10%
0%
Apr/14

May/14

Jun/14

Jul/14

Aug/14

Sep/14

Oct/14

Nov/14

Dec/14

Jan/15

Feb/15

Apple iPhone, Share of App Load Traffic by Device Generation, April 2014 January 2015

Android Lollipop <2% share of app load traffic, KitKat still growing
Google, device OEMs, carriers provide phased upgrades to the existing installed base of Android devices,
resulting in a uptake of a new Android version thats slower by an order of magnitude compared to iOS.
100%

Lollipop 5.0

<2% share of app load traffic

Nov 12th

90%

81days

Feb1st

80%
70%

KitKat 4.4
60%
50%
40%

Jelly Bean 4.3

30%

Jelly Bean 4.2

20%

Jelly Bean 4.1

10%

Ice Cream
Sandwich 4.0

0%
Apr/14

Other
May/14

Jun/14

Jul/14

Aug/14

Sep/14

Oct/14

Nov/14

Dec/14

Jan/15

Feb/15

Google Android, Share of App Load Traffic by Version, April 2014 January 2015

Samsung dominates Android, with S5 supplanting S3

Data indicates that owners of Samsung devices follow a 2-year upgrade cycle (e.g. from S3 to S5)
along with carrier subsidies.
100%
90%

20% share of
app load traffic

80%

Samsung Galaxy S5

~30 weeks / 7 months

Apr 1st

Nov 1st

70%

Motorola Moto G

60%
Samsung Galaxy S4

50%
40%
30%

Samsung Galaxy S3

20%

Samsung Galaxy S3 Mini

10%

Droid Razr HD
Samsung Galaxy Note 3

0%
Apr/14

Samsung Galaxy Tab 2

May/14

Jun/14

Jul/14

Aug/14

Sep/14

Oct/14

Nov/14

Dec/14

Jan/15

Feb/15

Samsung Galaxy Note 2

Android, Share of App Load Traffic by Device Type, April 2014 January 2015

OS updates can cause unforeseen problems

Despite extensive testing & preparation by OEMs and carriers, things can still go wrong.

Source: https://twitter.com/azkop2010/status/569312645527908352; https://twitter.com/dianaturner/status/569971859573358592

Whats the digital customer experience

of my mobile apps ?

10

Why the digital customer experience for mobile apps matters

Badly functioning apps are the source for customer frustration:
Your app has one job: online check in, so how come its never available?

Source: https://twitter.com/British_Airways/status/560238312540364800; https://twitter.com/British_Airways/status/560055280604835840

11

iOS 8 crash rates stabilized within a single quarter

Initial iOS 8 crash rate of 3.3%, dropping to 2.2% and converging with crash rates of iOS 6 (2.2%) and iOS 7
(2.0%). Improvements due to Apple quickly providing OS updates and developers fixing errors & bugs.
3.5%

Sept 17th

12 Weeks

Dec 14th

3.3%
3.1%

Crash rate
declines from
3.3% to 2.2%

2.9%
2.7%
2.5%

iOS 8

2.3%

iOS 6

2.1%

iOS 7

1.9%
1.7%
1.5%
Apr/14

May/14

Jun/14

Jul/14

Aug/14

Sep/14

Oct/14

Nov/14

Dec/14

Jan/15

Feb/15

iOS Crash Rates by Version, April 2014 February 2015

12

Anatomy of an iOS release cycle: Maturity of old OS

Crash rates are continuously declining as developers get more acquainted with the installed OS, continuously
improving the quality of their apps.
2.4%
iPhone 4

2.2%
iPhone 4S
iPhone 6 plus

2.0%

iPhone 5
iPhone 6

1.8%

iPhone 5C
iPhone 5S

1.6%

1.4%

1.2%
Apr/14

iOS 7 maturing, with crash rates continuously

going down and reversely the quality of the
in-app experience goes up

May/14

Jun/14

Jul/14

Aug/14

Sep/14

Oct/14

Nov/14

Dec/14

Jan/15

Feb/15

iOS Crash Rates by Device, April 2014 February 2015

13

Crash rates spike following the introduction of a new OS version

New OS features while tested - are breaking apps, as end points are hit in a new way. Developers need at
least two new builds to fix errors, implying two App Store review cycles with ~20 days of wait time.
2.4%

iOS 8 launch on
Sept 2014

iPhone 4

2.2%
iPhone 4S
iPhone 6 plus

2.0%

iPhone 5
iPhone 6

1.8%

iPhone 5C
iPhone 5S

1.6%

1.4%

1.2%
Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

iOS Crash Rates by Device, April 2014 February 2015

14

New cycle of developer productivity starts, driving down crash rates

Quality of apps goes back up as Apple stabilizes the OS and developers continuously fix errors and optimize
the delivery of their apps. Dev efforts are also shifting away from older devices such as iPhone 4.
2.4%
iPhone 4

Devs moving
on from iPhone 4

2.2%

iPhone 4S
iPhone 6 plus

2.0%

iPhone 5
iPhone 6

1.8%

iPhone 5C
iPhone 5S

1.6%
Developers getting more acquainted
with the new OS, creating higher
quality apps, but also moving on
from older hardware platforms in
the case of iPhone 4

1.4%

1.2%
Apr-14

May-14

Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14

Dec-14

Jan-15

Feb-15

iOS Crash Rates by Device, April 2014 February 2015

15

App Store category crash rates

Apps that consume a lot of resources across the stack (e.g. CPU, GPU, bandwidth) like Photo & Video and
Games tend to crash more frequently than other categories.
2.5%

2.4%

Best-in-class apps operate within a crash rate

band of 0.02-0.05% on both iOS and Android

2.1%
1.8%

1.8%

1.8%
1.6%

1.5%

0.05%
0.02%
Photo & Video

Games

Social
Networking

Health &
Fitness

Business

Entertainment

Lifestyle

News

AppStore Crash Rates by Category December 2014

16

Google Play category crash rates

On average higher crash rates per category than for the App Store, same phenomenon for resource-intensive
apps like Entertainment and Photography.
5.3%

Best-in-class apps operate within a crash rate

band of 0.02-0.05% on both iOS and Android

3.4%

3.3%

3.3%
2.8%

2.5%
1.8%

1.7%

0.05%
0.02%
Entertainment

Business

Photography

Shopping

Games

News &
Magazines

Social

Health &
Fitness

Google Play Crash Rates by Category December 2014

17

What are key differences for wireless networks

across different markets & geos?

18

Why networks & geographies matter for mobile apps

Carrier-enforced firmware for Android devices can cause an app to drain battery life; network settings can
lead to unexpected consequences for the availability of mobile apps:
Yes, it does work on wifi but does not work over AT&Ts network.

Source: https://twitter.com/wuhelp/status/539823712766144512

19

France

Bouygues scores best on latency among all French carriers, with the highest ARPU. France has the fastest
networks and with it the highest ARPUs in the Eurozone countries.
Wireless Carriers Ranked by Speed
Carrier

Relative Split between iOS & Android

Avg. Latency

iOS

Android

ARPU

1.

Bouygues

250ms

37%

63%

26

2.

Free

262ms

35%

65%

17

3.

Orange

274ms

41%

59%

23

4.

SFR

293ms

40%

60%

23

= highest ARPU in a market

Split between iOS and Android calculated based on app usage / traffic.

20

Germany

Vodafone with the fastest network and the highest ARPU; E-Plus by far the slowest, with lowest ARPU
Wireless Carriers Ranked by Speed
Carrier

Relative Split between iOS & Android

Avg. Latency

iOS

Android

ARPU

1.

Vodafone

300ms

45%

55%

15

2.

T-Mobile

317ms

44%

56%

14

3.

O2

320ms

4.

E-Plus

448ms

= highest ARPU in a market

69%

77%

31%

23%

Split between iOS and Android calculated based on app usage / traffic.

11

21

United Kingdom

O2 with the fastest network and the highest ARPU. All carriers with highly similar OS split and latency metrics,
leading to similar ARPUs.
Wireless Carriers Ranked by Speed
Carrier

Avg. Latency

Relative Split between iOS & Android

iOS

Android

ARPU

1.

O2

327ms

2.

Vodafone

344ms

87%

13%

18 / ~24

3.

EE

347ms

87%

13%

19 / 26

= highest ARPU in a market

82%

18%

Split between iOS and Android calculated based on app usage / traffic.

19 / ~26

22

Italy

No clear correlation between low relative latency / high relative ARPU.

Wireless Carriers Ranked by Speed
Carrier

Relative Split between iOS & Android

Avg. Latency

iOS

Android

ARPU

1.

WIND

282ms

2.

TIM

339ms

55%

45%

12

3.

Vodafone

343ms

55%

45%

13

= highest ARPU in a market

7%

11

93%

Split between iOS and Android calculated based on app usage / traffic.

23

Spain

No clear correlation between low relative latency / high relative ARPU; Vodafone with highest ARPU and
highest share of iOS usage.
Wireless Carriers Ranked by Speed
Carrier

Relative Split between iOS & Android

Avg. Latency

1.

Orange

314ms

2.

Yoigo

334ms

3.

Vodafone

358ms

4.

Movistar

431ms

= highest ARPU in a market

iOS
8%

Android

15

92%

23%

32%

26%

ARPU

77%

68%

74%

Split between iOS and Android calculated based on app usage / traffic.

13

18

15

24

United States

AT&T with the slowest network and the lowest ARPU in the market.
Wireless Carriers Ranked by Speed
Carrier

Relative Split between iOS & Android

Avg. Latency

iOS

Android

ARPU

1.

T-Mobile

327ms

45%

55%

$44

2.

Sprint

340ms

44%

56%

$49

3.

Verizon

356ms

4.

AT&T

362ms

= highest ARPU in a market

69%

77%

31%

23%

Split between iOS and Android calculated based on app usage / traffic.

$55

$42

25

Mobile Experience Benchmark - MWC 2015 Edition