+

WHEN THE CLOUD DECIDES: DESIGNING FOR

PREDICTIVE MACHINE LEARNING FOR THE IOT
Mike Kuniavsky
OReilly Design Conference
January 21, 2016
2015 PARC

Good afternoon and thanks for having me here. In this talk I want to look at the
design challenges of systems that anticipate users needs and then act on them.
That means that it sits at the intersection of the internet of things, user experience
design and machine learning, which to me is new territory for designers who may
have dealt with one of those disciplines before, but rarely all three at once.
The talk is divided into several parts: it starts with an overview of how I think
Internet of Things devices are primarily components of services, rather than being
self-contained experiences, how predictive behavior enables key components of
those services, and then I finish by trying to to identify user experience issues
around predictive behavior and suggestions for patterns to ameliorate those issues.
A couple of caveats:
- I focus almost exclusively on the consumer internet of things. Although predictive
behavior is an important part of the Industrial Internet of Things for things like
preventitive maintenance or energy savings in building environments, I feel its
REALLY key to the consumer IoT because of its potential ability to cut through the
information and data fog we live in.
- I want to point out that few if any of the issues I raise are new. Though the term
internet of things is hot right now, the ideas have been discussed in research
circles for decades. Search for ubiquitous computing, ambient intelligence, and
pervasive computing and itll help you keep from reinventing the wheel.
- Finally, most of my slides dont have words on them, so Ill make the complete
deck with a transcript available as soon Im done.

OReilly Design 2016

Let me begin by telling you a bit about my background. Im a user

experience designer. I was one of the first professional Web
designers. This is the navigation for a hot sauce shopping site I
designed in the spring of 1994.

OReilly Design 2016

Ive also worked on the user experience design of a lot of

consumer electronics products from companies youve probably
heard of.

OReilly Design 2016

I wrote a couple of books based on my experience as a designer.

One is a cookbook of user research methods, and the second
describes what I think are some of the core concerns when
designing networked computational devices.

OReilly Design 2016

I also started a couple of companies. The first, Adaptive Path, was

primarily focused on the web, and with the second one, ThingM, I
got deep into developing hardware.

OReilly Design 2016

Today I work for PARC, the famous research lab that invented the
personal computer and laser printer, as a principal in its Innovation
Services group. We help companies reduce the risk of adopting
novel technologies using a mix of ethnographic research, user
experience design and innovation strategy. We do everything from
developing novel products for our clients to coaching teams in how
they can be much more strategic and effective with their innovation
efforts.

R. G. Shoup, 1971
PARC

OReilly Design 2016

PARC also started thinking about what we call the IoT long before
most other companies.
It was at PARC in 1971 that Dick Shoup, and early PARC
researcher, wrote that eventually processors would be as
common, and as invisible as electric motors. This clearly
outlines the destiny of connected computer: that eventually it will
become as boring and as common as electric motors are today.

Mark Weiser, 1996

PARC

OReilly Design 2016

20 years later, also at PARC, Mark Weiser coined the term

ubiquitous computing to describe a future when the number of
computers surpassed the number of people, in everyday usage.
In this chart from 20 years ago, he predicted that would happen
around 2005. He didnt live to see that crossover, but he was
basically rightthe iPhone launched in 2007and we now live
in the world he envisioned.

OReilly Design 2016

But it didnt appear all at once. Weve only started the transition to
the ubiquitous computing world, and as such, were seeing a lot
of bad ideas about what the Internet of Things is and it isnt.
There are so many bad ideas in fact that there are entire
Tumblrs dedicated to mocking stupid IoT ideas. One is about
dumb smart things and the other is just about smart
refrigerators.

+ CONNECTING STUFF TO

THE INTERNET IS EASY

AND POINTLESS

OReilly Design 2016

Simply connecting existing stuff to the internet does not produce

customer value

10

11

OReilly Design 2016

Simple connectivity helps when youre trying to maximize the

efficiency of a fixed process, but thats not a problem that most
people have. Weve been able to simply connect various
devices to a computer since a Tandy Color Computers could
lights off and on over X10 in 1983. The problem is that that
wasnt very useful then, and its not very useful now. If you
replace the Tandy with an iPhone and the lamp with a washing
machine

11

12

OReilly Design 2016

or an egg carton, you still have the same problem, and its a user
experience problem.
The UX problem is that end users have to connect all the dots to
coordinate between a wide variety of devices, and to interpret
the meaning of all of these sensors to create personal value. For
many simply connected products there is so little efficiency to be
had relative to the cognitive load that its just not worth it. Whats
worse, the extra cognitive load is exactly opposite to what the
product promises, and customers feel intensely disappointed,
perhaps even betrayed, when they realize how little they get out
of such a product That makes most such products effectively
WORSE than useless. That promise gap is what distinguishes
an optional and marginal gadget from a tool.
This strategy worked very poorly for Quirky.

12

+ IOT SERVICE AVATARS +

MACHINE LEARNING =
PREDICTIVE BEHAVIOR

OReilly Design 2016

How do you create a tool that reduces

cognitive load instead of creating it, that
exchanges peoples precious time for
significant value? One approach is to couple
cloud-based services with predictive machine
learning models to anticipate what behaviors
will maximize the chances of a desirable
outcome in a given situation. Designing the UX
for this kind of solution is what Ill talk about
today, but first let me unpack these two
concepts a bit.

13

14

OReilly Design 2016

When I talk about services, Im talking about thinking of hardware

devices as avatars of cloud-services, which makes them very
different than traditional consumer electronics. Historically, a
company made an electronic product, say a turntable, they found
people to sell it for them, they advertised it and people bought it.
That was traditionally the end of the companys relationship with
the consumer until that person bought another thing, and all of the
value of the relationship was in the device. With the IoT, the sale of
the device is just the beginning of the relationship and holds almost
no value for either the customer or the manufacturer.

14

15

SERVICE

AVATARS

OReilly Design 2016

Value now shifts to services and the devices, software applications

and websites used to access itits avatarsbecome
secondary. A camera becomes a really good appliance for
taking photos for Instagram, while a TV becomes a nice
Instagram display that you dont have to log into every time, and
a phone becomes a convenient way to check your friends
pictures on the road.
Hardware becomes simultaneously more specialized and devalued
as users see through each device to the service it represents.
The avatars exist to get better value out of the service.

15

16

OReilly Design 2016

Amazon really gets this. Heres a telling older ad from Amazon for
the Kindle. Its saying Look, use whatever device you want. We
dont care, as long you stay loyal to our service. You can buy our
specialized devices, but you dont have to.

16

17

OReilly Design 2016

When Fire was released 3 years ago, Jeff Bezos even called it a
service.

17

18

OReilly Design 2016

Amazon Dash is a service thats enabled by dedicated devices, or

by a Dash button whose only purpose is to be an avatar for a
macaroni and cheese service.

18

19

OReilly Design 2016

Most large-scale IoT products are service avatars. They use

specialized sensors and actuators to support a service, but have
little valueor dont work at allwithout the supporting service.
Smart Things, which was acquired by Samsung last year, clearly
states its service offering right up front on their site. The first
thing they say about their product line is not what the
functionality is, but what effect their service will achieve for their
customers. Their hardware products functionality, how they will
technically satisfy the service promise, is almost an afterthought.

19

20

OReilly Design 2016

Compare that to X10, their spiritual predecessor thats been in the

business for more than 20 years. All that X10 tells is you is what
the devices are, not what the service will accomplish for you. I
dont even know if there IS a service. Why should I care that
they have modules? I shouldnt, and I dont.

20

+ PREDICTIVE BEHAVIOR,
ENABLED BY MACHINE
LEARNING

OReilly Design 2016

I think the real consumer value connected services offer is their ability to make
sense of the world on peoples behalf, to reduce cognitive load by enabling
people to interact with devices at a higher level than simple telemetry, at
the level of intentions and goals, rather than data and control. Humans are
not built to collect and make sense of huge amounts of data across many
devices, or to articulate our needs as systems of mutually interdependent
components. Computers are great at it.

21

22

OReilly Design 2016

They can make statistical models from many data sources across
space and time and then try to maximize the probability of a
desired outcome. A model learned from thousands of samples
across many people and long periods of time can compensate for
much wider variety of situations in a more nuanced way than an
individual will ever be able to. Because people and their machines
act pretty consistently, these systems can essentially predict the
future, which is how Waze knows that youre probably driving
home when you get in the car after work without you ever telling it
your home address and schedule.

22

23

OReilly Design 2016

Such prediction and response is at the heart of the value

proposition many of the most compelling IoT products offering,
starting with the Nest. The Nest says that it knows you. How
does it know you? It predicts what youre going to want based
on your past behavior.

23

24

OReilly Design 2016

Amazons Echo speaker says its continually learning. How is that?

Predictive machine learning.

24

25

OReilly Design 2016

The Birdi smart smoke alarm says it will learn over time, which is
again the same thing.

25

26

OReilly Design 2016

Jaguar, learningAND intelligent.

26

27

OReilly Design 2016

The Edyn plant watering system adapts to every change. What is

that adaptation? Predictive analytics.

27

28

OReilly Design 2016

Canary, a home security service.

28

29

OReilly Design 2016

Heres foobot, an air quality service.

[I also like how one of its implicit service promises is to identify
when your kids are smoking pot.]

29

+ ISSUES WITH THE UX OF

PREDICTIVE BEHAVIOR

OReilly Design 2016

Predictive behavior creates a seductive world of espresso

machines that start brewing as youre thinking its a good time
for coffee; office lights that dim when its sunny and mac and
cheese that never runs out. The problem is that although the
value proposition is of a better user experience, its unspecific in
the details. All the previous machine learning systems that
existed were used in areas such as predictive maintenance and
finance were made by and for specialists. Now that these
systems are for general consumers, we have some significant
UX questions. Exactly how will our experience of the world, our
ability to use all the collected data, become more efficient and
more pleasurable?
Were still early in our understanding of predictive UX, so right now
the problems are worse than solutions and I want to start by
articulating the issues Ive observed in our work.

30

+ ISSUE 1: EXPECTATIONS OF

31

AUTONOMOUS BEHAVIOR

OReilly Design 2016

Weve never had mechanical things that make significant decisions

on their own, the first major issue is around expectations. As
devices adapt their behavior, how will they communicate that
theyre doing so? Do we treat them like animals? Do we stick a
sign on them that says adapting, like the light on a video
camera says recording? Should my chair vibrate when
adjusting to my posture? How will users, or just passers-by,
know which things adapt? I could end up sitting uncomfortable
for a long time, waiting for my chair to change, before realizing it
doesnt adapt on its own. How should smart devices set the
expectation that they may behave differently in what appears to
be identical circumstances?
How do we know HOW intelligent these devices are? People
already often project more smarts on devices than those devices
actually have, so a couple of accurate predictions may imply a
much better model than actually exists. How do we know were
not just homesteading the uncanny valley here?
Chair by Raffaello D'Andrea, Matt Donovan and Max Dean.
31

+ ISSUE 2: UNCERTAINTY

32

OReilly Design 2016

The irony in predictive systems is that theyre pretty unpredictable, at least at first.
When machine learning systems are new, theyre often inaccurate, which is not
what we expect from our digital devices. 60%-70% accuracy is typical for a first
pass, but even 90% accuracy isnt enough for a predictive system to feel right,
since if its making decisions all the time, its going to be making mistakes all the
time, too. Its fine if your house is a couple of degrees cooler than youd like, but
what if your wheelchair refuses to go to a drinking fountain next to a door because
its been trained on doors and it cant tell thats not what you mean in this one
instance? For all the times a system gets it right, its on the mistakes that we judge
it and a couple such instances can shatter peoples confidence. Anxiety is a kind of
cognitive load, and a little doubt about whether a supposedly smart system is going
to do the right thing is enough to turn a UX thats right most of the time into one
thats more trouble than its worth. When that happens, youve more than likely lost
your customer.
Unfortunately, sooner than we think, such inaccurate predictive behavior isnt going to
be an isolated incident. Soon were going to have 100 connected devices
simultaneously acting on predictions about us. If each is 99% accurate, then one is
always wrong. So the problem is: How can you design a user experience to make a
device still functional, still valuable, still fun, even when its spewing junk behavior?
How can you design for uncertainty?
Photo CC BY 2.0 photo 2011 Pop Culture Geek taken by Doug Kline:
https://www.flickr.com/photos/popculturegeek/6300931073/

32

+ ISSUE 3: CONTROL

33

OReilly Design 2016

The last issue comes as a result of the previous two: control. How
can we maintain some level of control over these devices, when
their behavior is by definition statistical and unpredictable?
On the one hand you can mangle your devices predictive behavior
by giving it too much data. When I visited Nest once they told me
that none of the Nests in their office worked well because theyre
constantly fiddling with them. In machine learning this is called
overtraining. The other hand, if I have no direct way to control it
other than through my own behavior, how do I adjust it? Amazon
and Netflixs recommendation systems, which are machine
learning systems for predicting what you may like, give you some
context about why they recommended something, but what do I do
when my only interface is a garden hose?

33

34

From: Sheridan and Verplank, 1978

OReilly Design 2016

At the dawn of computing cyberneticists like Norbert Weiner

philosophized about the increasingly complex relationship
between people and computers, and how it was fundamentally
different than with other kinds of machines. Developers working
in supervisory control of manufacturing machines and robotics
have had to deal with these questions pragmatically for about 30
years, but I think this is now a problem that is going to be a
general UX problem for everyone going forward.

34

+ SHARING CONTROL AND

35

RESPONSIBILITY WITH ALGORITHMS

Automation
Level

The computer offers no assistance: humans must make all decision and
actions

The computer offers a complete set of decision/action alternatives

Narrows the selection down to a few

Suggests one alternative

Executes that suggestion if the human approves

Allows the human a restricted time to veto before automatic execution

Executes automatically, then necessarily informs humans

Informs the human only if asked

Informs the human only if it, the computer, decides to

10

The computer decides everything and acts autonomously, ignoring the

From: Parasuraman, Sheridan, Wickens, 2000
human

OReilly Design 2016

Heres a more modern framework from the industrial automation

world that defines a spectrum of responsibility between people
and computers. It ranges from humans doing all the work to
computers doing all the work completely autonomously. Of
course the goal is to get a system to level 9 or 10. Thats the
maximum reduction in cognitive load. However, for a system to
qualify for that, it has to be very stable, its effects need to be
highly predictable and, equally importantly, it needs to be
adequately embedded in society that its OK for a computer to
take on that level of responsibility. At the airport we trust the
monorail computers to work without human intervention, but we
dont trust the plane computers to do that, even though-as I
understand itplanes can basically fly themselves these days.
I think predictive Internet of Things services fall between 5 and 7
on this scale right now. The problem is that this is the exact
range where youre increasing cognitive load, so the result of the
automation had better be worth this new extra effort.

35

+ FIVE PATTERNS FOR

ADDRESSING PREDICTIVE
UX ISSUES

OReilly Design 2016

Here are 5 UX design patterns Ive observed in developing

predictive IoT systems.

36

+ PATTERN 0: HAVE A USER STORY

37

FOR EVERY STAGE OF PREDICTION

Acquire
Behave

Extract

Train

Classify
Model

OReilly Design 2016

My first pattern isnt really a pattern, but a general approach. To design these
systems you need to have a user model for every stage of the machine
learning and prediction process. There needs to be a story to tell about each
step, even if its a step that seems like it would invisible to customers.
Starting with acquisition: how will you incentivize people o add data to the
system at all? Why should I upload my cars dashcam video to your traffic
prediction system EVERY DAY? Next, how will you communicate youre
extracting features? I like the way that Google speech to text shows you
partial phrases as youre speaking into it, and how it corrects itself. That
small bit of feedback tells people its pulling information out and it trains
users how to meet the algorithm halfway. How do machine-generated
classifications compare to peoples organization of the same phenomena?
How is a context model presented to end users and developers? How will
you get people to train it and tell you when the model is wrong? Does the
final behavior actually match their expectation?
Machine learning algorithms used to be strictly behind-the-scenes, but in the
IoT they are actors in our lives, so as designers its our responsibility to
understand the situations where the algorithms and the devices they control
interact with peoples lives, especially since theres a deep symbiotic
relationship between the data that comprises the models, the behavior those
models induce and the people who are the intended beneficiaries.

37

+ PATTERN 1: BUILD AND MAINTAIN

38

TRUST

VS

OReilly Design 2016

Because these systems are working on peoples behalf, at the core

of the user experience is that the user has to trust them to act
autonomously. If you dont trust a machine learning system,
youll be worried about it. If you worry, thats cognitive load.
Every unit of cognitive load has to be balanced by end-user
value.

38

+ COMPONENTS OF TRUST IN

39

PREDICTIVE UX

Benevolence

Humor

Integrity

Balance

Control
From: Yoo and Gretzel, 2011
Kantor, et al, 2011

OReilly Design 2016

What makes for a trustworthy experience? Computers are social actors and predictive
systems are especially so, the qualities of a trustworthy digital assistant are
essentially the same as those of a trustworthy human apprentice.
Here are some factors recommender system researchers identified as building blocks
of trust when interacting with a system making decisions on your behalf. Why do
you trust a Slack chatbot more than one from, say, AT&T? Is it because its funny,
and if its funny then it cant be evil, right? For me every one of these bricks
represents an interesting cluster of UX, branding and messaging challenges that
are unique to each device.

Benevolence, the recommender systems caring about the user and acting in the
users interest
Integrity, adherence to a set of principles (e.g. honesty) that the user finds acceptable
Balance. several studies have demonstrated that communicators can enhance their
trustworthiness when they provide both sides of the argument - the pros and the
cons - rather than arguing only in their own favor
Humor. A number of studies found positive effects of humor on communicator
trustworthiness judgments but rarely on judgments of expertise
Familiarity. products that were familiar to users were helpful in establishing users trust
in recommender systems
Control. users showed more positive affective reactions to recommender systems
when they had increased control

39

+ PATTERN 2: SET BEHAVIOR

40

EXPECTATIONS

OReilly Design 2016

Our current expectation for digital systems is that theyll behave consistently
and the reasons for their behavior will be clear. Neither of these is true for the
user experience of predictive systems, which dont necessarily behave
identically in what appear to be similar circumstances, whose behavior
changes over time, and where the reasons for the behavior may not be
obvious. If we undermine peoples confidence in a system by violating their
expectations, theyre likely to be disappointed and stop using it.
The first thing a predictive UX needs to do is to set peoples expectations
appropriately. It needs to explain the nature of the device, to describe it is
trying to predict, that its trying to adapt, that its going to sometimes be wrong,
to explain how its learning, and how long itll take before it crosses over from
creating more trouble than benefit.
Recommender systems, such as Google Now, describe why a certain kind of
content was selected, and that sets the expectation that in the future the
system will recommend other things based on other kinds of content youve
requested. Nests FAQ explains that you shouldnt expect your thermostat to
make a model of when youre home or not until its been operating for a week
or so.

40

41

From: Arnall, 2006

OReilly Design 2016

About ten years ago Timo Arnall and his students tried to address a
similar set of questions around interactions with RFID-enabled
devices by creating an iconography system that communicated
to potential users that these devices had functionality that was
invisible from the outside. Perhaps we need something like this
for behavior created by predictive analytics?

41

+ PATTERN 3: EXPLAIN SEQUENCES

42

WHEN A STATE HAS CHANGED OR IS

GOING TO CHANGE

OReilly Design 2016

Predictive behavior is all about time, about sequences of activities.

Many predictive UX issues around expectations and uncertainty
have time as their basis: what were you expecting to happen
and why. If it didnt happen, why? If something else happened,
or it happened at an unexpected time, why did that happen?
Knowing that a device has acted on your behalf, and that its going
to actand HOW its going to actin the future is important to
giving people a model of how its working, to set their
expectations, to reduce the uncertainty. Nest, for example, has a
calendar of its expected behavior, and it shows when its acting
on your behalf to change the temperature, and when you can
expect that temperature will be reached.

42

+ EXPLANATION COMPONENTS

43

What just happened?

Why did it happen?
What is going to happen next?
When will it happen?
What can I do?

OReilly Design 2016

Explanations should not be technical, and they should not

overwhelm people with information, but a user should be able to
find out these things if they want. This should be prioritized
relatively low in the interfacethe Nests schedule is pretty
hiddenbut it should be clear and available for someone to
unpack what just happened.

43

+ PATTERN 4: CREATE CLEAR WAYS

44

FOR PEOPLE TO ADJUST THE

PREDICTIVE BEHAVIOR

OReilly Design 2016

That last element in explanations is especially important, because it points to

the difficulty of adjusting the behavior of a machine learning system once
that system has built a model, and how important it is to give people
affordances for adjusting that model.
Remember the Wall Street Journal story from about 15 years ago about a guy
(Basil Iwanyk) who thought that his TiVo thought he was gay because of
the shows it recommended for him? Thats funny, but the second part is
frustrating: he didnt want it to think he was gay, so he tried to correct it the
only way he could think of, by clicking thumbs-up on what he thought were
the least gay things he could imagine, war movies. The TiVo then decided
he was a Nazi (which he also wasnt). You have to give people a clear way
to teach the system and tell it when its model is wrong. Statistical systems,
by definition, dont have simple rules that can be changed. There arent
obvious handles to turn or dials to adjust, because everything is
probabilistic. If the model is made from data collected by several devices,
which device should I interact with to get it to change its behavior? Google
Now asks whether I want more information from a site I visited, Amazon
shows a explanation of why it gave me a suggestion. Mapping this to the
consumer IoT means way more explanation than were currently getting,
which is either that a thing has happened, or it hasnt.

44

+ PATTERN 5: SUPPORT PEOPLE,

45

DONT REPLACE THEM

OReilly Design 2016

Finally, dont automate. These systems shouldnt try to replace

people, but to support them, to augment and extent their
capabilities, not to replace them.
One of my favorite current examples of this kind of system is from
Meshfire, which is a social media management tool that has a
machine learning assistant. Its machine learning assistant,
called Ember, doesnt try to replace the social media manager.
Instead it manages the media managers todo list. It adds things
that it thinks are going to be interesting, deletes old things, and
reprioritizes the managers list based on what it thinks is
important. I think this is a good model for how such systems can
add value to a persons experience without creating a situation
where random, unexplained behaviors confuse people, frustrate
them and make them feel powerless. Ember is an augmentation
to the social media manager, it helps that person focus on
whats important so that they can be smarter about their
decisions. It doesnt try to be smarter than they are. How can
our devices HELP us, rather than trying to replace us?
45

+ ANTIPATTERN: DONT MAKE PEOPLE

46

DO ALL THE TRAINING, THEY WONT

1:11

1:26

1:36

1:47

1:59

2:25

2:38

2:57

OReilly Design 2016

Finally, an antipattern: making people do all of the training, asking

them to identify whether a behavior is appropriate or not, should
be done selectively and infrequently. Yes, it will really help your
supervised models accuracy to have people identify the correct
positives from the false positives, but unless youre paying these
people, its incredibly annoying to have customers do it all the
time. Last Friday one consumer IoT product with a machine
learning system Im playing with asked me to classify its output
at 1:11PM, then again at 1:26, and again at 1:47 and again and
again. I think it was on roughly ten-minute sensing cycle, and at
every cycle it tried to make a decision, and asked me to verify it.
Im sure its still doing it, but I turned off all notifications from it,
and now Im considering turning it off entirely. People will
sometimes willingly act as sensors and actuators for your
system, but because they are not machines, they will not do it all
the time and youre just going to have to find a better way to
train your model.

46

+ ITS NOT ABOUT THE

THINGS

OReilly Design 2016

Finally, for me the IoT is not about the things, but the experience
created by the services for which the things are avatars.

47

48

The wheel is an extension of the foot.

The book is an extension of the eye.
Clothing is an extension of the skin.
Electric circuitry is an extension of the
central nervous system.

Marshall McLuhan,
The Medium is the Massage, 1967

OReilly Design 2016

Ultimately we are using these tools to extend our capabilities, to

use the digital world as an extension of our minds. To do that well
we have to respect that as interesting and powerful as these
technologies are, they are still in their infancy, and our job as
entrepreneurs, developers and designers will be to create systems,
services, that help people, rather than adding extra work in the
name of simplistic automation. What we want to create is a
symbiotic relationship where we, and our predictive systems, work
together to create a world that provides the most value, for the
least cost, for the most people, for the longest time.

48

+ Thank you!
Mike Kuniavsky
mikek@parc.com
@mikekuniavsky

OReilly Design 2016

Thank you.

49