Name: Mohammad Mashhood Roll No: 2021F-BBM-070

Assignment#03
A Study on Sixth Sense Technology
Ajesh M.S & Subin P Sabu
Student of BCA 2nd Year, Department of Computer Under the guidance of: Ms. Mintu Movi
Science, Christ College of Science and Management, Head of Department of Computer Science, Christ College of
Bangalore-University, Karnataka, Email: Science and Management, Bangalore University, India, Email:
[email protected], [email protected] [email protected]

Abstract- ‘Sixth Sense’ is a wearable gesture interface that augments the physical world around us with digital information and lets us use natural hand gestures
to interact with that information. Sixth Sense Technology may change how we look at the world forever. Sure, it is a system that projects information about what
surrounds you over objects' surfaces, but it's also much more. By using a camera and a tiny projector, Sixth Sense sees what you see and visually augments any
surfaces or objects we are interacting with.
This wearable gestural interface attempts to free information from its confines by seamlessly integrating it with reality, and thus making the entire world our
computer. To a layman, it would be something supernatural. Some might just consider it as a superstition or something psychological. But the invention of Sixth
Sense Technology has completely shocked the world. Although it is not widely known as of now but the time is not far when this technology will change our
perception of the world.

Keywords— wearable device, gesture recognition, extra sensory perception, real time access.
world. Right now, we use our devices (computers, mobile
I. INTRODUCTION phones, tablets, etc.) to go into the internet and get
We have evolved over millions of years to sense the world information that we want. With Sixth Sense we will use a
around us. When we encounter something, someone or device no bigger than current cell phones and probably
some place, we use our five natural senses which include eventually as small as a button on our shirts to bring the
eye, ear, nose, tongue mind and body to perceive internet to us in order to interact with our world! Sixth
information about it; that information helps us make Sense will allow us to interact with our world like never
decisions and chose the right actions to take. But arguably before. We can get information on anything we want from
the most useful information that can help us make the right anywhere within a few moments! We will not only be able
decision is not naturally perceivable with our five senses, to interact with things on a whole new level but also with
namely the data, information and knowledge that mankind people. One great part of the device is its ability to scan
has accumulated about everything and which is objects or even people and project out information
increasingly all available online. regarding what you are looking at.
Although the miniaturization of computing devices
WHAT IS SIXTH SENSE?
allows us to carry computers in our pockets, keeping us
continually connected to the digital world, there is no link Sixth Sense in scientific (or non-scientific) terms is defined
between our digital devices and our interactions with the as Extra Sensory Perception or in short ESP. It involves the
physical world. Information is confined traditionally on reception of information not gained through any of the five
paper or digitally on a screen. Sixth Sense bridges this gap, senses. Nor is it taken from any experiences from the past
bringing intangible, digital information out into the or known. Sixth Sense aims to more seamlessly integrate
tangible world, and allowing us to interact with this online information and tech into everyday life. By making
information via natural hand gestures. Sixth Sense frees available information needed for decision-making beyond
information from its confines by seamlessly integrating it what we have access to with our five senses, it effectively
with reality, and thus making the entire world your gives users a sixth sense.
computer. Sixth Sense is a wearable gesture based‖ device II. LITERATURE REVIEW
that augments the physical world with digital information
and let people use natural hand gestures to interact with This paper deals with the Sixth Sense, a tool that connects
that information. It was developed by Pranav Mistry, a PhD the physical world with the world of data. It’s a high tech
student in the Fluid Interfaces Group at the MIT Media Lab. device which can supplement the physical world around us
He says that the movies Robocop and Minority Report gave with digital information. This wearable device has the
him the inspiration to create his view of a world not capability of turning everything to touch screen just by
dominated by computers, digital information and human capturing the gestures of a person. But alongside it has led
robots, but one where computers and other digital devices to invention to another concept having the same point but
enhance people’s enjoyment of the physical by using commands instead of gestures. For orders face
recognition technique is used algorithms with the sixth
Name: Mohammad Mashhood Roll No: 2021F-BBM-070

2
sense method. Here the database is used which will be phone. They initially produced a wristband that would
initially trained for storage. With this model, our goal is to read a Radio Frequency Identification tag to know, for
demonstrate, within a neurobiological framework, the ability example, which book a user is holding in a store. They also
of network that process details of any person just by had a ring that used infrared to communicate by beacon to
capturing the natural photographs. It implements the similar supermarket smart shelves to give you information about
command which will gain access to the operation from the products. As we grab a package of macaroni, the ring
mobile device associated to it and action finally projector is would glow red or green to tell us if the product was
used for projection over any surface. Because of its cost organic or free of peanut traces whatever criteria we
constraint it’s more likely to be implemented in future. [1] program into the system.
The Sixth Sense ‘An Extrasensory Perception’ has turned They wanted to make information more useful to people
into the new called ‘The Sixth Sense Technology’ which has in real time with minimal effort in a way that doesn’t
emerged in few years. We all have evolved over millions of require any behavior changes. The wristband was getting
years to sense the world around us. We always make use of close, but we still had to take out our cell phone to look at
our five natural senses to perceive the information around the information. That’s when they struck on the idea of
us whenever we come across a thing, a person or a place. accessing information from the internet and projecting it. So
That information helps us make judgments and embrace someone wearing the wristband could pick up a paperback
the appropriate action to be taken. But arguably, the most in the bookstore and immediately call up reviews about the
useful information that can help us take the right decision book, projecting them onto a surface in the store or doing a
and judgments is not naturally cognoscible information keyword search through the book by accessing digitized
with the help of our five senses, namely the data, pages on Amazon or Google books.
information and knowledge that mankind has They started with a larger projector that was mounted
amalgamated but, rather it’s the ‘Sixth Sense Technology.’ on a helmet. But that proved cumbersome if someone was
This paper focuses on and makes us aware with the sixth projecting data onto a wall then turned to speak to friend
sense technology which provides an integration of the the data would project on the friends face. [4]
digital world with the real world; it helps us understand V. RECENT PROTOTYPE
how the sixth sense device had overpowered the five Now they have switched to a smaller projector and created
natural senses; it also pours light over its various the pendant prototype to be worn around the neck. The
applications, its security related issues and further Sixth Sense prototype is composed of a pocket projector, a
implications. [2] mirror and a camera. The hardware components are
The recent trends in technology have revolutionized the coupled in a pendant-like mobile wearable device. Both the
means of interaction between the digital world and real projector and the camera are connected to the mobile
time applications. The primary focus of human computer computing device in the user’s pocket. We can very well
interaction is to improve the intercommunication between consider the Sixth Sense Technology as a blend of the
user and computer by making computer more receptive to computer and the cell phone. It works as the device
the user's needs. Mouse too has undergone a significant associated to it is hanged around the neck of a person and
revolution right from its invention starting with mechanical thus the projection starts by means of the micro projector
mouse with two buttons and scroll to an optical mouse and attached to the device. Therefore, in course, you turn out to
finally to a cordless mouse and is still used as a pre- be a moving computer in yourself and the fingers act like a
dominant means to interact with a computer. In this paper mouse and a keyboard. [6]
our aim is to implement an invisible computer mouse that The prototype was built from an ordinary webcam and a
enables interaction with computer without attaching a battery-powered 3M projector, with an attached mirror all
hardware mouse. The methodology used is based on the connected to an internet-enabled mobile phone. The setup,
Sixth Sense Technology where the user will be able to move which costs less than $350, allows the user to project
the cursor by the movement of fingers. Our focus is to information from the phone onto any surface walls, the
move the cursor on the screen as the user moves his/her body of another person or even your hand. Mistry wore the
fingers. [3] device on a lanyard around his neck, and colored Magic
III. OBJECTIVES Marker caps on four fingers (red, blue, green and yellow)
helped the camera distinguish the four fingers and
1. To understand the possibilities of sixth sense
recognize his hand gestures with software that Mistry
technology.
created. [5]
2. To learn about the advantages of virtual computing
VI. COMPONENTS
and real world interaction held together.
3. To find the usefulness of this technology in the The devices which are used in Sixth Sense Technology
future. are:
IV. EARLIER SIXTH SENCE PROTOTYPE  Camera: It captures the image of the object in view
and tracks the users hand gesture. The camera
Maes (media lab) _MIT group, which includes seven
recognizes individuals, images, pictures, gestures
graduate students, were thinking about how a person could
that user makes with his hand. The camera then
be more integrated into the world around them and access
sends this data to a smart phone for processing.
information without having to do something like take out a
Name: Mohammad Mashhood Roll No: 2021F-BBM-070

3
Basically the camera forms a digital eye which the neck. The mirror reflects the image on to a
connects to the world of digital information. desire surface. Thus finally the digital image is
 Colored Marker: There are color markers placed at freed from its confines and placed in the
the tip of user’s fingers. Marking the user’s fingers physical world Flight Updates: The system will
recognize your
with red, yellow, green and blue colored tape helps
boarding pass and let you know whether your flight is
the webcam to recognize the hand gestures. The
on time and if the gate has changed.
movements and arrangement of these markers are
.
interpreted into gestures that act as an interaction
VII. WORKING
instruction for the projected application interfaces.
 Mobile Component: The Sixth Sense device consists The Sixth Sense Technology works as follows:
of a web enabled smartphone which process the 1. It captures the image of the object in view and tracks
data send by the camera. The smartphone searches the users hand gestures.
the web and interprets the hand gestures with the 2. There are color markers placed at the tip of users
help of the colored markers placed at the finger tips. fingers. Marking the user’s fingers with red, yellow,
Basic processing works on computer vision green and blue colored tape helps the webcam to
algorithms where approx. 50,000 lines of code are recognize the hand gestures. The movements and
used written in Symbian C++. arrangement of these markers are interpreted into
 Projector: The information that is interpreted gestures that act as an interaction instruction for the
through the smartphone can be projected into any projected application interfaces.
surface. The projector projects the visual information 3. The smartphone searches the web and interprets the
enabling surfaces and physical objects to be used as hand gestures with the help of the colored markers
interfaces. The projector itself consists of batteries placed at the finger tips.
which have 3 hours of battery life. A Tiny LED 4. The information that is interpreted through the
projector displays the data sent from the smartphone can be projected into any surface.
smartphone on any surface in view-object, wall or 5. The mirror reflects the image on to a desired surface.
person. The downward facing projector projects the
image on to a mirror.

  Pictures: Anohher application of VIII. APPLICATIONS

Sixth Sense devices is the implementation of a
The Sixth Sense device has a huge number of applications.
gestural camera. This camera takes the photo of The following are few of the applications of Sixth Sense
the location user is looking at by detecting the Technology:-
framing gesture. After taking the desired number
 Viewing Map: With the help of a map application
of photos we can project them onto any surfaces the user can call upon any map of his/her choice and
and then use gestures to sort through those photos navigate through them by projecting the map on to
and organize and resize them. any surface. By using the thumb and index fingers
 Interacting with Physical Objects: The Sixth movements the user can zoom in, zoom out or pan
Sense system also helps to interact with physical the selected map.
objects we use in a better way. It augments  Drawing Application: The drawing application
physical objects by projecting more information allows the user you to draw on any surface
about these objects projected on them. For by tracking the fingertip movements of the user’s
example, a gesture of drawing a circle on the index finger. The pictures that are drawn by the user
user’s wrist projects a watch on the user’s hand. can be stored and replaced on any other surface.
Similarly a newspaper can show live video news The user can also shuffle through various pictures
or dynamic information can be provided on a and drawing by using the hand gesture movements.
regular piece of paper.  Making Calls: We can make calls with the help of
 Mirror: The usage of a mirror is important as the Sixth Sense device. The Sixth Sense device is used to
 projector dangles pointing downward from
Name: Mohammad Mashhood Roll No: 2021F-BBM-070

4
protect the keyboard into your palm and using that IX. FUTURE ENHANCEMENTS
virtual keypad we can make calls to anyone.
1. To get rid of color markers.
 Interacting with Physical Objects: The Sixth 2. To incorporate camera and projector inside mobile
Sense system also helps to interact with Computing device.
physical objects we use in a better way. It 3. Whenever we place pendant- style wearable device
augments physical objects by projecting on table, it should allow us to use the table as multi
more information about these objects touch user interface.
projected on them. For example, a gesture of 4. Applying this technology in various interest like
gaming, education systems etc.
drawing a circle on the user’s wrist projects 5. To have 3D gesture tracking.
a watch on the user’s hand. Similarly a 6. To make sixth sense work as fifth sense for disabled
newspaper can show live video news or person.
dynamic information can be provided on a
X. FINDINGS
regular piece of paper. • Flight Updates: The
system will recognize your boarding pass Through the paper we found out that the sixth sense
and let you know whether your flight is on technology is an emerging innovation in the field of
time and if the gate has changed. computing. It will be used widely all over the world
because of its various advantages of virtual computing
 Taking Pictures: Another application of Sixth
and real world interaction held together.
Sense devices is the implementation of a
gestural camera. This camera takes the XI. CONCLUSION
photo of the location user is looking at by
The key here is that Sixth Sense recognizes the objects
detecting the framing gesture. After taking around you, displaying information automatically and
the desired number of photos we can project letting you access it in any way you want, in the simplest
them onto any surfaces and then use way possible.
gestures to sort through those photos and
Clearly, this has the potential of becoming the ultimate
organize and resize them "transparent" user interface for accessing information about
 Other Applications: Sixth Sense also lets the user everything around us. If they can get rid of the colored
draw icons or symbols in the air using the finger caps and it ever goes beyond the initial development
movement of the index finger and recognizes those phase, that is. But as it is now, it may change the way we
symbols as interaction instructions. For example, interact with the real world and truly give everyone
drawing a magnifying glass symbol takes the user complete awareness of the environment around us.
to the map application or drawing a @‖ symbol lets REFERENCES
the user check his mail
[1] Charan prathap sing, prathap srivasthava. “face recognization
 ADVANTAGES approches using sixth sense technology : a review”,2014.
[2] Aakanksha chopra, natasha narang. “a study on-the sixth sense
1. Sixth Sense is a user friendly interface which technology and its various security thruts”,2013.
integrates digital information into the physical [3] Swarali naveker, manali godse. “vision baced analysis using sixth
world and its objects, making the entire world your sense technology”,2013.
computer. [4] Rakesh D. Desale, Vandana S. Ahire “A Study on Wearable
2. Sixth Sense does not change human habits but Gestural Interface – A SixthSense Technology”, IOSR Journal of
causes computer and other machines to adapt to Computer Engineering (IOSR-JCE) e-ISSN: 2278-0661, p- ISSN:
human needs. 2278-8727Volume 10, Issue 5 (Mar. - Apr. 2013), pp. 10-16,
3. It uses hand gestures to interact with digital Available: www.iosrjournals.org
information, supports multi-touch and multi-user [5] Monika Arora, “Basic Principles of Sixth Sense Technology”,
interaction. VRSD International Journal Of Computer Science and
4. Data access directly from machine in real time. It is Information Technology, VSRD-IJCSIT, Vol. 2 (8), 2012, pp.687-
an open source and cost effective and we can mind 693, Available: www.vsrdjournals.com.
map the idea anywhere. Nigel Davies, Manfred Lau, Chris Speed, Tom Cherrett, Janet Dickinson,
5. It is gesture-controlled wearable computing device and Sarah Norgate, “Sixth Sense Transport: Challenges in Supporting
that feeds our relevant information and turns any Flexible Time Travel”,Published in: HotMobile '12- Proceedings of the
surface into an interactive display. Twelfth Workshop on Mobile Computing System & Application, Article
6. It is portable and easy to carry as we can wear it in No-8, ACM New York, NY, US©2012, ISBN: 978-1-4503
our neck.
7. The device could be used by anyone without even a
basic knowledge of a keyboard or mouse. There
is no need to carry a camera anymore.
8. If we are going for a holiday, then from now on
wards it will be easy to capture photos by using
mere fingers.
Name: Mohammad Mashhood Roll No: 2021F-BBM-070