Good

day. Thanks for having me here. Today I want to give you a bit of my
perspec;ve on user experience design for the internet of things. The talk will be in
two main parts: a general overview of what I believe the Internet of Things is beyond
the marke;ng hype and deni;onal slippage in the news, and to talk about some
specic approaches Ive taken to developing products in this space. I normally dont
go deep into the niBy griBy of what I do, but I thought this group may appreciate
knowing at least a bit about my design prac;ce.
Overall, however, remember that this eld is new. Not just the Internet of Things, but
user experience design in general, so one of things I urge you to do is to think about
about all of things Im missing, all of the dierent ways to think about this space and
how to design in it. Were just at the beginning here, so the eld is wide open for
innova;on at all levels.

Let me begin by telling you a bit about my background. I m a user experience

designer. I was one of the first professional Web designers. This year I am celebrating
professionally designing Web sites for 20 years. This is the navigation for a hot sauce
shopping site I designed in the spring of 1994.

Ive also worked on the user experience design of a lot of consumer electronics
products from companies youve probably heard of.

Ive wriBen a couple of books based on my experience as a designer. One is a

cookbook of user research methods, and the second describes what I think are some
of the core concerns when designing networked computa;onal devices.

I also started a couple of companies. The rst, Adap;ve Path, was primarily focused
on the web, and with the second one, ThingM, I got deep into developing hardware.

Today I work for PARC, the famous research lab, as a principal in its Innova;on
Services group.

PARC has been thinking about what we now call the Internet of Things for a long
;me, nearly since its incep;on. Heres part of an internal strategy memo Dick Shoup
wrote in 1971.

PARC is also in many ways where the Internet of Things began. In the late 1980s Mark
Weiser, who was then the CTO of PARC predicted that just as mainframesone
computer to many peoplegave rise to personal compu;ngone computer per
personthere would be a next period, which he called ubiquitous compu;ng, that
would happen when there was more than one computer per person. Thats his chart
from 20 years ago predic;ng that the crossover would happen around 2005, which
was basically right. I never knew him and he died in 1999, but I wish he was around
today to give his thoughts about whats going on today.

The Internet of Things is a really challenging name to work with, because there are so
many deni;ons. This is Time Magazines illustra;on of the Internet of Things for
their Best Inven;ons of 2008 edi;on. I love this illustra;on is because it makes no
sense no maBer how you think about it, which is actually quite an accurate
representa;on of how confusing the many deni;ons of the Internet of Things are
right now.

The rst thing to note is that there are two deni;ons of the Internet of Things,
which causes a bit of confusion. In the original one Kevin Ashton looked at how RFIDs
could aect the logis;cs industry and saw an analogy to the Internet. RFIDs could be
like the packet header in Internet data packets, trucks were like wires, depots like
routers, warehouses like caches.

Somewhere around 2010 the deni;on slipped and rather than being an analogy
between shipping and the Internet, it came to mean what Mark Weiser had earlier
described as ubiquitous compu;ng, which means that everyday objects as fulledged ci;zens of the Internet. Instead of being LIKE packets, the objects now send
packets themselves.

Whats especially confusing is that since so many of the new objects are mobile, the
rst deni;on ALSO kind of applies. When youre using Waze to guide you around
trac, its trea;ng your car like a data packet because your phone has made your car
into a networked compu;ng device.

10

11

12

I want to start by talking about Moores Law, since thats where all conversa;ons
about the implica;on of digital technology start. When people talk about Moores
Law, its ogen in the context of maximum processing power. But its actually
something dierent. Its actually a descrip;on of the cost of processing power. Its a
model of how much more processing power we can t into a single chip thats priced
at a predictable pricing point this year than we could last year. This means that its
not just that processors are gehng more powerful, its that PROCESSING is gehng
cheaper.

For example, at the beginning of the Internet era we had the 486 as the state of the
art and it cost $1500 in today s dollars. It s the processor that the Web was built for
and with. Today, you can buy that same amount of processing power for 30 cents,
and it uses only a frac;on of the energy. That decrease in price is the same orders of
magnitude drop as the increase in speed. This is not a coincidence, because both are
the product of the same underlying technological changes.

What this means in prac;ce is that embedding powerful informa;on processing
technology into anything is quickly approaching becoming free.

13

This means that because of compe;;ve pressure, anything that can have networked
informa;on processing embedded in it, will. Everyday objects will make sophis;cated
autonomous decisions and act using arbitrary informa;on., which is as deep an
infrastructural change in our world as electrica;on, steam power, and mechanical
prin;ng. Maybe it s as big of a deal as bricks.

14

The other dominant trend right now is of course pervasive data communica;on. This
is an image from Timo Arnall thats envisioning how saturated our environment is
with networks, and its not even coun;ng the mobile phone network. This means
that virtually any device, anywhere can share data with the cloud at any ;me. This
means it can ooad func;onality that used to be local to the cloud, which means
that almost all processing, storage and most baBery life issues disappear. If your ;ny,
wimpy device can talk to the cloud, it can be eec;vely as powerful as the most
powerful computer on earth.

15

The implica;ons of this technological change are broad, but today I want to focus on
two aspects: how consumers rela;onships to devices change, and how companies
value chains change.

16

The core of this change is a shift from generic devices and software to specialized
devices and software. When computing was expensive, you had general purpose
devices and general purpose software that had deal with almost every situation. This
necessitated design compromises that resulted in devices and software that could do
almost everything, but did none of it well.
Now that processing is so cheap, you can have a combination of 10, 20, or 30
computing devices and apps for the price of that one device, and you can acquire new
functionality as needed. This means that every device and software package can have a
narrower purpose.
Adobe brush
Haiku deck

17

The second trend is that much of the value of compu;ng is now remote. Today, most
people understand that the experience you see on one device is ogen a part of
something thats distributed throughout the world. Theres no longer a need to pack
everything into a single piece of sogware, and theres no expecta;on that everything
will be there.

Foursquare as a whole doesnt live on your phone in any meaningful way, and people
know that.

18

If we chart these two tends, two broad classes of digital products emerge.

If we follow the local to remote axis, we nd general-purpose devices that do roughly
the same set of things, and dier primarily in size. They exist to provide access to
online services, in a form factor thats appropriate to the context in which theyre used.
I call these devices terminals.

If we follow the general to specic axis, we see a shig is to more narrow-func;on
devices that are designed to do a small set of things with specialized hardware. A
parking meter can take quarters, which your phone cant do, and a digital SLR has a
giant lens on it, which you probably dont want on your phone. These devices differ in
their specialized hardware. I call these devices appliances.

19

The most interesting thing, however, to me is that these shifts are part of an even larger
transition, one where devices are simultaneously specific AND deeply tied to online
services. In this model, the service provides the majority of the value, and can be
represented either as a dedicated appliance, an app running on a terminal, or anything
in between.
I call these devices service avatars. And this is where much of the Internet of Things
currently lies, at least for consumer products.

20

As value shifts to services, the devices, software applications and websites used to
access itits avatarsbecome secondary. A camera becomes a really good appliance
for taking photos for Flickr, while a TV becomes a nice Flickr display that you dont
have to log into every time, and a phone becomes a convenient way to take your Flickr
pictures on the road.
Hardware becomes simultaneously more specialized and devalued as users see
through each device to the service it represents. The hardware exists to get better
value out of the service.

21

Another example is the Kindle. Here s a telling older ad from Amazon for the Kindle.
Its saying Look, use whatever device you want. We don t care, as long you stay
loyal to our service. You can buy our specialized devices, but you don t have to.

22

At rst Withings was kind of a gimmick. You can tweet your weight to your friends!
was one of the ways it was originally pitched. Thats of course not par;cularly
interes;ng, but that was not the purpose of the device. The scale is the avatar to
Withings health service that helps you track your weight. The scale originally
dieren;ated Withings from other weight tracking apps, but the value is not in the
scale, but in the service, which you can only fully experienced using other avatars,
such as the web site and the phone and tablet apps.

Withings has since expanded the service to include a bunch of other hardware and
sogware avatars. Again, the value is not in the devices, but in the knowledge that
they create by collec;ng simple pieces of informa;on and then providing users with
the full power of cloud-based services to make use of that piece of informa;on.
Withings can keep adding avatars, new sensors and new ways to display the
informa;on the sensors collect, without fundamentally changing the promise of the
service.

23

Smart Things clearly states its service oering right up front on their site. The rst
thing they say about their product line is not what the func;onality is, but what eect
their service will achieve for their customers. Their hardware products func;onality,
how they will technically sa;sfy the service promise, is almost an agerthought.

24

Compare that to X10, their spiritual predecessor thats been in the business for more
than 20 years. All that X10 tells is you is what the devices are, not what the service
will accomplish for you. I dont even know if there IS a service. Why should I care that
they have modules? I shouldnt, and I dont.

25

In other words

26

Simple connec;vity helps when youre trying to maximize the eciency of a xed
process, but thats not a problem that most people have. Weve been able to simply
connect various devices to a computer since a Tandy Color Computers could lights o
and on over X10 in 1983. That wasnt very useful then, and its not very useful now. If
you replace the Tandy with an iPhone and the lamp with a washing machine

27

Here are two tumblrs dedicated to mocking this simplis;c approach. One is about
dumb smart things and the other is just about smart refrigerators.

28

Unfortunately exactly that product is what Quirky bet a lot of their resources on, and
we see where that got them.

Simple telemetry means that youre making connect all the dots to coordinate
between a wide variety of devices, and to interpret the meaning of all of these
sensors to create personal valueThere is so liBle eciency to be gained rela;ve to
the cogni;ve load that its just not worth it for the vast majority of people and tasks.
Whats worse, the extra cogni;ve load is exactly opposite to what the product
promises, so the disappointment people feel when they realize how liBle they get out
makes such products eec;vely WORSE than useless. That promise gap is what
dis;nguishes an op;onal and marginal gadget from a tool.

29

30

One approach is to change from an ownership model to a subscrip;on model. Now

the device gives access to a desired end result, without the burdens of ownership or
maintenance. The IoT technology is what gives an ecient way to track and charge
for assets. Car sharing, bike sharing, Uber and AirBNB follow this model. You dont
use it every day, so why own it? High-end clothing is going this way. Do you really
need to own that Prada handbag so you can use it twice a year?

31

HewleB Packards printer division is really an ink company that also makes ink
consump;on devices. Similarly Amazon is trying to corner the market on all
consumables, whether theyre digital

32

..or physical. Thats what their dash buBon is for. Its an avatar to their cloud-based
replenishment service.

33

Predic;on is at the heart of the value proposi;on many of the most compelling IoT
products oering, star;ng with the Nest. The Nest says that it knows you. How does
it know you? It predicts what youre going to want based on your past behavior.

34

Amazons Echo speaker says its con;nually learning. How is that? Predic;ve
analy;cs, predic;ve machine learning.

35

The Birdi smart smoke alarm says it will learn over ;me, which is again the same
thing.

36

Jaguar comes right out with it. They even obliquely reference the 40 years of ar;cial
intelligence research that powers predic;ve analy;cs by calling their car not just
learning, but learning AND intelligent.

37

The Edyn plant watering system adapts to every change. What is that adapta;on?
Predic;ve analy;cs.

38

Canary, a home security service.

39

Heres foobot, an air quality service. I think these services are just the ;p of the
iceberg and that theyre the most interes;ng part of the whole IoT and they
encompass both previous models. Imagine a world of espresso machines that start
brewing as youre thinking its a good ;me for coee, which reorder your favorite
blend; oce lights that dim when its sunny, power is cheap and youre not doing
anything that needs them; and food truck caravans that show up just as the crowd in
the park is gehng hungry.

Making the experience of that is really hard. Its what we spend most of our ;me
working on, but I think services that leverage devices ability to learn and predict is by
far the most interes;ng part of the whole IoT.

40

But now back to my core point

41

Heres the slide of advice I give to IoT startups.

Basically, because hardware is hard, it should be used very carefully. It should be
used tac;cally to create a clear advantage against compe;tors for whom it will be
hard to copy in the short run, but who you can assume will duplicate the device
func;onality very quickly if youre successful. It should create lock-in, but not based
on some kind Keurig DRM-protected ridiculousness, but through experience quality,

I feel that the long-term greatest leverage will be through a cloud-based service,
rather than any avatar, no maBer how clever or specialized that avatar is. I feel the
same way about the Skully motorcycle helmet as I do about Fitbit. Its temp;ng to
carried away by the physicality of making hardwareI personally love it but if you
let the thrill of making something you can hold in your hand go to your head, you risk
sliding into gadget and gizmo territory, where compe;;ve advantages are likely to be
very short.

42

In prac;ce this means that the range of things that need to be designed is enormous.
One minute youre deba;ng the business model, the next youre sihng next to an
engineer picking out sensors. Theres a lot of uncertainty.

43

And heres the process Im currently using to help us push the bounds of exis;ng products
and services with some idea of what people value and whats technologically feasible. Its
explicitly based on the Lean Startup approach, but geared toward hardware.

The top layer describes a highly itera;ve exploratory phase that exists solely to help us
understand what experiences provide value to people and what technology our team can
reasonably deliver. Probes, which I will go to in more detail later, are things that resemble
nished productsperhaps theyre nonfunc;onal apps or boxes with lightsbut exist to
explore hypotheses about what kinds of products and services people will respond well to.
We give them to people for a couple of days or a week and then we interview them when
theyve had a chance to live with them. Proofs of concept exist to test the core technological
ideas. I dont care if we have to throw a $3000 server at the problem when the end product
has to cost $10, is it possible for us to do that thing AT ALL? This kind of technical proof-ofconcept is what Alan Kay used to call a ;me machine. If we can do it for $3000 today, well be
able to do it for $10 in ve years, so its as if were paying a bunch of money to get a device
from the future.

Ager this phase is done, we throw everything we did away and start designing the real
product from scratch, and thats where the prototyping process starts. We itera;vely
develop prototypes with addi;onal user feedback un;l were preBy sure it represents a
valuable and technologically feasible service. Then we deliver that to our client who throws
away all of our work, but uses it as a reference design that they can then tear apart and
rebuild using their own in-house technologies or processes or teams.

Today we will focus on probes.

44

Heres our version of the probe process. Itll look familiar from various new product
development and user centered design prac;ces. Ive tried to emphasize explicit
hypothesis genera;on and evalua;on because when youre making an unknown
product in a novel category for an untested market, which is where we work, one of
the only points of stability is an clear statement about what you think the value of the
idea is, and thats the hypothesis, so that when the probe process calls it into
ques;on, that provides guidance about how to iterate it.

45

The design ar;facts you make are all over the board. Im going to go through a
handful that Ive found to be useful, but like I said at the beginning, its a new
discipline and one of the most interes;ng design challenges is actually designing the
ar;facts that use to understand the problems youre trying to solve and how to
communicate your ideas to the appropriate audiences in the appropriate way.

46

Many of my conversa;ons with clients start our with a lot of the typical whiteboard
and post-it note discussion where we describe what they believe the value of their
service will be to poten;al customers. Typically no one has very good answers at this
point, so we take guesses.

Based on those guesses I put together a high-level descrip;on of what pieces the
service is going to contain. The diagram separates data sources, analy;cs, displays,
controllers, sensors and actuators so I can start a discussion about how many avatars
were actually going to design and the value each one adds to customers and to the
business. The diagram is super simple, but one of its goals is to reinforce that all of
the pieces are connected and that neither the app or the watch is more important.
That helps me talk to stakeholders about the scope of what were going to build.

47

Next, someone on the product team needs to create one of these. Its a Business
Model Canvas, from Alex Osterwalder and Yves Pigneurs Business Model Genera;on
book, which is a single-page checklist that describes what the service is going to do
for people, whos its for, and why theyll pay for it.

The user experience of most mul;-touchpoint Internet of Things services is much
more closely ;ed to their business model than in a tradi;onal tradi;onal consumer
electronics company. For example, say youre making a smart lock. How much
func;onality should you put in the lock mechanism? Well, that depends on what
customer problem the service is trying to address. If you primary audience is AirBNB
property owners who want to create a unique code for each tenant, thats a dierent
set of needs than for a security-concerned homeowner. Same lock, perhaps even
same locking hardware, but a dierent emphasis based on the key business model.

48

Ager the Business Model Canvas, were in straighuorward UX design for a while. I like
personas, so I create 4-5 dierent users of the product based on facets of the
business modelheres one we did for a Stanford project. These can be built with
some up-front ethnography, which is what I prefer, or one of the other techniques
that gets you a reasonable persona.

49

Then you get to use scenarios and storyboards. These help to tell the story of how
people interact with various avatars, what role each does and does not play in
peoples lives. Its useful for both communica;ng a vision among team members and
stakeholders, and you ogen learn important UX lessons just by talking through a
couple of stories. This is one we did with Smart Design. My approach to wri;ng
scenarios is to start by assuming that people dont want to use your product, but
your product helps them accomplish a goal faster, easier or more elegantly than they
would have otherwise, so my scenarios and storyboards ogen feature situa;ons
where personas dont use the product much, or only use one aspect of it, but it plays
a key role in their lives.

Storyboard by Smart Design

50

Next, you start bringing things together by puhng various touchpoints against
important goals your personas and your stakeholders goals. These touchpoints can
be full-edged service avatars or just interac;ons with the service. One way to do
that is by making a swim lane diagram that puts the touchpoints across the top and
then maps interac;ons across.

This is a chunk of a journey diagram I put together for Sigeo, the game company, a
couple of years ago. We developed a set of goals for the UX of the devices, and those
goals spanned the documenta;on, the interface on the devices, and the Web site.
Across the top is a set of touchpoints, which range from the text on the box to the
instruc;ons that people wont read, to the rmware and the site. Horizontally I
created swim lanes for peoples mo;va;on, what informa;on we gave them, what
they can do at that point, etc. And then across this grid I mapped out a strategy for
interac;ons that we hoped would lead people to do things such as downloading the
sogware and buying new games.

51

Next comes the design for all of the avatars. I divide this process into 2D, which is
essen;ally screen-based interac;on design like how you would design an app or a
Web page

52

and 3D. Here we take our process cues from industrial design prac;ce and start
with sketches then move to models and renderings, with looks-like prototypes

This is a rendering from a project we did with Smart Design.

53

and works-like prototypes. This is by Yoichi Nagashima for a project he did, but I
think its a good representa;on of what our stu looks like at the crazier end.

This process is essen;ally the tradi;onal consumer electronic product development
process, dont think its easy. Its not and its where much of your ;me is going to go,
but its fairly well established, so Im glossing over it here.

54

In fact, we ogen make the renderings less realis;c so we can focus on the concept,
rather than the actual execu;on. Todays sogware make it really easy to produce
images or 3D models that are completely nonviable as products and at this stage
were much more interested in understanding whats valuable than making things
that are preBy. Too much delity leads to stakeholders and poten;al customers
focusing on the wrong aspects of the device.

Also by Smart Design

55

And of course in between many of these steps is a lot of user research. Heres a my
colleagues using video ethnography to document peoples behaviors. Im a rm
believer that no idea is worthwhile unless its been validated by people who are not
your team. Customer test and valida;on is a cri;cal part, and we try to include it as
ogen as we can, which is never as much as wed like.

56

So, stepping back, why do this? Why go through this process where I as a designer
dont end up doing that much design? Well, its because I think that theres
enormous possibility in the Internet of Things, but its not in the things.

57

To me why we make Internet of Things is not about crea;ng new gadgets, but
developing services that extend ourselves from our bodies out into the world. Our
biggest challenge as innovators, designers, developers and entrepreneurs will not be
adding cheap electronics to everyday objects. Thats a solved problem. Our challenge
will be to understand how to use those things to create value, and to create
sustained and sustainable change, in a world where there is an enormous amount of
poten;al for embedded, networked technology, and an enormous number of paper
thin, one note gadgets. I once heard an industrial designer say that everything you
design is garbage from the minute it comes o the assembly line, and that its your
responsibility to keep it out of the trash bin for as long as possible. I think that in the
internet of things that is especially true and today I think that our biggest challenge
as designers will be to think about why we want to make things at all, and then how
we can make them provide the most value, for the least cost, for the most people, for
the longest ;me.

58

Thank you.

59

60