Sixth Sense Technology

Turning the Physical World into a Computer Interface

Presented by

K.Shireesha

III B.tech ECE S.Kishore

JYOTHISMATHI INSTITUTE
OF TECHNOLOGY AND SCIENCE
NUSTULAPUR, KARIMNAGAR
Email
id:shireeshakandu@yahoo.com
III B.tech ECE
JYOTHISMATHI INSTITUTE
OF TECHNOLOGY AND SCIENCE
NUSTULAPUR ,KARIMNAGAR
Email Id:sangisetty.kishore@gmail.com

ABSTRACT
The fact that makes human beings unique
from the other species is that we possess a
unique sixth sense apart from the basic five
senses of sight, hearing, taste, smell and
touch. This sixth sense or an extrasensory
perception (ESP) is the reception of
information not inferred through the
recognized senses and is not gained from
experience. ESP is usually referred to as a
sixth sense or a gut instinct. Scientists had
constantly been working on connecting the
digital world and the physical world and
several breakthroughs have been got. Now
here is a technology that connects our
physical world with our digital world. This
sixth sense technology is a gesture based
interface that connects the physical world
with the digital world. Any activity we do
involves gestures and such a device would
be an ideal one to connect our gesture to
perform digital operations. This technology
ensures us to carry the digital world with us
always and it constantly converts our
gestures to give digital outputs.
1. INTRODUCTION
All of us are aware of the five basic senses –
seeing, feeling, smelling, tasting and
hearing. But there is also another sense
called the sixth sense. It is basically a
connection to something greater than what
their physical senses are able to perceive. To
a layman, it would be something
supernatural. Some might just consider it to
be a superstition or something
psychological. But the invention of sixth
sense technology has completely shocked
the world. Although it is not widely known
as of now but the time is not far when this
technology will change our perception of the
world.
Pranav Mistry, 28 year old, of Indian origin
is the mastermind behind the sixth sense
technology. He invented ‘ Sixth Sense /
WUW ( Wear UR World) ‘ which is a
wearable gestural , user friendly interface
which links the physical world around us
with digital information and uses hand
gestures to interact with them. He is a PhD
student at MIT and he won the ‘Invention of
the Year 2009 ‘- by Popular Science. The
device sees what we see but it lets out
information that we want to know while
viewing the object. It can project
information on any surface, be it a wall,
table or any other object and uses hand / arm
movements to help us interact with the
projected information. The device brings us
closer to reality and assists us in making movements and arrangements of these
right decisions by providing the relevant fiducials are interpreted into gestures that act
information, thereby, making the entire as interaction instructions for the projected
world a computer application interfaces.

The maximum number of tracked fingers is

only constrained by the number of unique
fiducials, thus SixthSense also supports
2. COMPONENTS multi-touch and multi-user interaction.

The SixthSense prototype is composed of a 3. WORKING

pocket projector, a mirror, and a camera.
The hardware components are coupled in a
pendant-like mobile wearable device. Both
the projector and the camera are connected
to the mobile computing device in the user’s
pocket, working such as

THE PROJECTOR: projects visual

information, enabling surfaces, walls and
physical objects around the wearer to be The Sixth Sense prototype consists of a
used as interfaces; pocket projector, mirror and a camera. The

THE CAMERA AND HANDS: device is pendant shaped like mobile

recognizes and tracks the user's hand wearing devices. Both the projector and the

gestures and physical objects using camera are connected to the mobile

computer-vision based techniques. computing device in the user’s pocket. The

projector projects visual information
THE SOFTWARE PROGRAM: It
enabling surfaces, walls and physical objects
processes the video stream data took by the
around us to be used as interfaces; while the
camera and tracks the locations of the
camera recognizes and tracks user’s hand
colored markers (visual tracking of colored
gestures and physical objects using
fingers) at the tip of the user’s fingers using
computer-vision based techniques. The
simple computer-vision techniques. The
software program processes the video SIXTH SENSE TECHNOLOGY MAY
stream data captured by the camera and CHANGE HOW WE LOOK AT THE
tracks the locations of the colored markers WORLD FOREVER
(visual tracking fiducials) at the tip of the
It is a system that projects information
user’s fingers using simple computer-vision
about what surrounds you over objects'
techniques. It also supports multi touch and
surfaces, but it's also much more. You just
multi user interaction.

have to watch it:

4. DIGITAL WORLD MEETS THE
PHYSICAL WORLD Basically, Sixth Sense is a mini-projector
coupled with a camera and a cellphone—
Most noticeably, the device is the first
which acts as the computer and your
attempt to link connected digital devices
connection to the Cloud, all the information
directly to the physical world. Instead of
stored on the web. Sixth Sense can also obey
information being confined to paper or to a
handgestures.
screen connected to a computer, every
physical object has the potential to become a
computer interface.

The projector and the camera are both

connected to whatever mobile computing
device might be in the user’s pocket. The
projector projects the visual information
onto whatever surface is available: a wall, a
table, even your hand.

But the real genius lies in the ability to use

natural hand gestures and arm movements to
interact with the information being made
available. The camera is able to recognize a
user’s hand gestures through computer-
vision based techniques.
However, instead of requiring you to be in
front of a big screen like Tom Cruise, Sixth
Sense can do its magic—and a lot more—
everywhere, even while you are jumping
hysteric over Oprah's sofa.
The camera recognizes objects around you boarding pass, Sixth Sense will grab real
instantly, with the micro-projector time information about your flight and
overlaying the information on any surface, display it over the ticket. You won't need to
including the object itself or your hand. do any action. Just hold it in front of your
Then, you can access or manipulate the and it will work.
information using your fingers. Need to
The key here is that Sixth Sense recognizes
make a call? Extend your hand on front of
the objects around you, displaying
the projector and numbers will appear for
information automatically and letting you
you to click. Need to know the time? Draw a
access it in any way you want, in the
circle on your wrist and a watch will appear.
simplest way possible.
Want to take a photo? Just make a square
with your fingers, highlighting what you
Clearly, this has the potential of becoming
want to frame, and the system will make the
the ultimate "transparent" user interface for
photo—which you can later organize with
accessing information about everything
the others using your own hands over the
around us. If they can get rid of the colored
air.
finger caps and it ever goes beyond the
initial development phase, that is. But as it
But those are just novelty applications. The
with the real world and truly give everyone
true power of Sixth Sense lies on its
complete awareness of the environment
potential to connect the real world with the
around us.
Internet, and overlaying the information on
the world itself. Imagine you are at the 5. ADVANTAGES
supermarket, thinking about what brand of
1. The system is very simple, portable
soap is better. Or maybe what wine you
and multisensory.
should get for tonight's dinner. Just look at
objects, hold them on your hands, and Sixth 2. There is no burden to carry the
Sense will show you if it's good or bad, or if system
it fits your preferences or not.
3. The system is easy to operate.
Now take this to every aspect of your
everyday life. You can be in a taxi going to
the airport, and just by taking out your
 4. This technology enables us to use additional information on the text,
any surface as Computer Interface to comments and lot more add on features.
view information While picking up any good at the grocery
store, the user can get to know whether the
5. Any shape can be depicted by simply
product is eco friendly or not.
drawing them with fingers.

Imagine using your finger to sketch the @

6. It facilitates instant mobile web
symbol in the air and immediately having
access.
your email projected upon the wall in front
7. APPLICATIONS of you? How about placing your fingers and
hands into a framing gesture and having the
The device has a huge number of
camera snap a picture? Or drawing a circle
applications. Firstly, it is portable and easily
on your wrist to have the device project an
to carry as you can wear it in your neck. The
analog watch onto your arm?
drawing application lets user draw on any
surface by observing the movement of index Perhaps, with the right gesture, even helping
finger. Mapping can also be done anywhere you decide which toilet paper to purchase
with the features of zooming in or zooming
Display of time
out. The camera also helps user to take
pictures of the scene he is viewing and later The device is cool enough that it can
he can arrange them on any surface. That’s display current time on your wrist. The
not it. Some of the more practical uses are gesture of drawing a circle on the user’s
reading a newspaper. Imagine reading a wrist projects an analog watch to effectively
newspaper and viewing videos instead of the tell time
photos in the paper. Or live sports updates
while reading the newspaper. The device
can also tell you arrival, departure or delay
time of your air plane on your tickets. For
book lovers it is nothing less than a blessing.
Open any book and you will find the
Amazon ratings of the book. To add to it,
pick any page and the device gives
Easy location of places on maps
The sixth sense also implements map which
lets the user display the map on any physical
surface and find his destination by just using
isgestures.
Interaction instructions
letting the user zoom in, zoom out or pan
using intuitive hand movements. The
drawing application lets the user draw on
any surface by tracking the fingertip
movements of the user’s index finger.
SixthSense also recognizes user’s freehand
gestures (postures).
For example, it implements a gestural
camera that takes photos of the scene the
user is looking at by detecting the ‘framing’
gesture. The user can stop by any surface or
wall and flick through the photos he/she has
taken.
SixthSense also lets the user draw icons or
symbols in the air using the movement of
the index finger and recognizes those
symbols as interaction instructions.
A newspaper can show live video news or
dynamic information can be provided on a
regular piece of paper.
Connecting real world objects
One can pick up images/content from any
surface (say newspaper, book etc) and place
it into your system, as this device interlinks
the two! Though there are some ethical
issues involved with the technology, the
entry of this device which offers lot of
features can take life to an extraordinary
level. With the entry of this technology into
the market, and with several individuals
owning this device, life would go to a sci-fi
level.

Camera recognizes objects which are around

you instantly,with the micro-projector
overlying the information on any 8.Disadvantages
surface,including the objects itself or your
There seems to be no disadvantages of this
hand.then you can acess or manipulate the
technology. But there is a risk of losing the
information using your finger. This
privacy in accessing the person information.
technology’s power lies in the fact that it can
connect real world and internet. Personal Information like the interests of a
person, contacts of a person and many more
Pick and Place
details when projected ought to get
everybody’s attention in a public place
posing a threat to the privacy of the person
using sixth sense technology.

At present cost of the product is also more

for it is a new technology But this could
come down soon with mass production.

9. Future Scope
• A UK television spot for Dixons uses
a Minority Report style interface for
motion gesture-based interaction.
Aiming to bring the world to a level
of ubiquitous integrated mobile
computing, the Microsoft Research
team has demonstrated a shrunken
Mobile Surface application where a
projection of a motion-touch
interface can be placed on almost
any flat object turning it into a
responsive active display screen.
• Creating a mobile interface that will
integrate into many parts of our life,
giving us access to information for
making optimal decisions throughout
our day.
• A video by TechFlash shows
researchers playing a simulated drum
set which is projected on a table,
allowing the user to tap his fingers or
drumsticks to initiate the drum
sounds.
The goal of the sixth sense technology is to
combine the intuitive process of creation
that is inherent in paper and pencil with the
power of computing that the digital design
tools provide.
Intution also extends the natural work-
practice of using physical paper by giving
the pen the ability to control the design in
physical three dimensional space, freeing it
from its tie to the paper. The intuition of pen
and paper are still present, but lines are
captured and translated into shapes in the
digital world. It provides a novel interaction
mechanism for drawing and designing using
above surface pen movements. Thus this
new technology with its highly potential
applications will soon enter our daily lives.
11.References
• http://hubpages.com/hub/pranav-
mistry-sixth-sense-technology-
physical-world-to-digital-world

• To integrate this technology in future

• www.tedpoint.com
games for the Project Natal game
control system for Xbox 360 have • www.wikipedia.com
led to advancements in natural user
• http://www.zdnetasia.com
interfaces (NUI) and is essentially
the Wii of the future. • www.youtube.com

10.Conclusions
• http://techpp.com/2009/11/22/must-
watch-pranav-mistrys-sixth-sense-
technology/