Visvesvaraya Technological University: Mitre Att&Ck Framework
Visvesvaraya Technological University: Mitre Att&Ck Framework
BELAGAVI-590018.
Science &Engineering
Submitted by
Rakesh R Amin (1KT20CS063)
CERTIFICATE
The framework categorizes these tactics and techniques into a matrix that aligns with the
stages of a typical cyberattack, from initial access to data exfiltration. Each cell in the
matrix represents a specific technique used by adversaries during an attack, along with
details on how the technique works and examples of known threats that have employed
it.By mapping out the tactics and techniques used by adversaries, the MITRE ATT&CK
Framework helps organizations improve their cybersecurity defenses. It enables them to
better understand potential threats, assess their security posture, and develop more effective
strategies for detecting and mitigating cyberattacks. Additionally, it serves as a common
language for cybersecurity professionals to communicate and collaborate on threat .
i
ACKNOWLEDGEMENT
I consider it is a special privilege to express a few words of gratitude and respect to all
those who have guided and inspired us in completing the technical seminar work. The success
of the project depends largely on the encouragement and guideline of many other. I take this
opportunity to express my gratitude to the people who have been instrumental in successful
completion of the Technical Seminar report.
I am also indebted to all who directly or indirectly rendered their valuable help for
completion of the Technical Seminar(18CSS84) report.
I am also thankful to our parents who have always been our mentor.
Rakesh R Amin(1KT20CS063)
ii
Table of Content
ABSTRACT.........................................................................................................i
ACKNOWLEDGEMENT..................................................................................ii
Table of Content.................................................................................................iii
List of Figure........................................................................................................v
CHAPTER 1: INTRODUCTION……………………………………………..1
1.1 Tactics
1.2 Mitre attack techniques
CHAPTER 2 MITRE ATTACK HEATMAP...............................................10
2.1 Heatmap
2.2 constructuction of heatmap
2.3 Interpretation of Heatmaps
2.4 Application of Heatmap
iii
Table of Content
5.6 Network
5.7 Containers
CONCLUSION………………………………………………………………..28
REFRENCES....................................................................................................29
iv
List of Figure
v
Sixth Sense Technology
CHAPTER 1 TRODUCTION
Sixth Sense Technology is the process by which humans modify nature to meet their needs and
wants. Increase in the needs leads to the increase in the technology and vice versa. When it comes to the topic
of technology it is not wise to use the word impossible in anything. Because every impossible of past has
become possible in the present and day by day the proof has been stronger. The development of technology has
made people amazed each day and thus it has been a difficult task for people to recognize or to see what is
coming next.
As technology is a vast topic, entering every sector and explaining everything is a difficult task with
limitation of words, sources and time. This paper will discuss Sixth Sense Technology, one of the strong topics
of now, which has challenged the people with the word impossible on their lips and created another revolution
in its own field. As the topic itself is interesting, revealing the real meaning of sixth sense technology and the
implementation of this topic in the field of technology is the main aim of this thesis paper. But since the topic
is a new concept, finding suitable content materials for research was a very challenging task.
The sixth sense technology has combined the digital and physical worlds. A variety of excellent
technologies are combined to create the amazing Sixth Sense technology. Its incredible integration of all those
technologies into a single, portable, and affordable gadget is what makes it so amazing. It links many
technologies together, including image processing, picture recording, and hand motion recognition. It overlays
the virtual world over the physical one. It also seeks to receive information, much as how our senses help us
take in information about our surroundings in various ways. In actuality, the sixth sense is about understanding
information beyond what our senses can now see. And today, in addition to the physical world, which is where
we obtain our information, there is also the digital world, which has ingratiated itself into our daily lives. We
now value the digital world just as much as the actual one. Additionally, the digital world may be multiplied
many times over through the internet.
This WuW, or Sixth Sense, technology is a model for the next evolution of
connecting the digital and physical worlds. With the use of this technology, individuals may
communicate virtually as if they were speaking face-to-face. Augmented reality (AR), which
is one of the main tenets of Sixth Sense technology, includes gesture recognition. It aims to
decipher human gestures using computer codes and mathematical formulas.
Pranav Mistry is the creator of the concept underlying sixth sense technology., an
inventor, and a researcher at the MIT Media Lab. It enables individuals to interact with the
virtual environment using their natural hand motions and movements without the use of a
keyboard, mouse, or touch screen. The technology comprises a tiny wearable gadget with a
camera, a projector, and several sensors that can be carried in a user's pocket or attached to
clothing. The computer analyses the motions and converts them into commands that may be
used to interact with digital information such as images, videos, and web pages. The camera
records the user's movements and delivers the data to the computer. The digital content is
then projected onto a wall or other flat surface, turning it into an interactive interface that the
user may engage with using their hands and fingers. The idea behind this technology is to
create an intuitive and natural interface between the physical and digital worlds, enabling
users to access and work with digital data. Many possible uses for the technology exist,
ranging from entertainment and gaming to business, healthcare, and education.
Chapter 2
People are aware of the senses that they have. Those senses when encounter a new experience
tries to analyze that. The analyzed experience is then used to modify the interaction with the environment.
There are many small-sized computers or computing devices which are helping people to connect to the digital
world in different situations from different parts of the world. But now rather than being connected to the
digital world, the demand of people is the use the technology as their sixth sense which in other word is to
bridge the gap between two worlds namely digital and real. (Udayakumar & Khanaa 2013.)
2.1 Technology
The term technology itself is the fusion of two Greek words techne and logia which gives the
meaning of making of tools, machines, crafts, systems or methods to solve a problem. In other word, the use of
science in any field to invent feasible device for doing different tasks or to solve any problem is called
technology. According to Encyclopedia Britannica, the application of scientific knowledge to the practical aims
of human life or to the change and manipulation of the human environment is called technology (Encyclopedia
Britannica 2014.) The broad meaning of technology implies the meaning of practical art and mining to means
of communication, medicine, and military technology. (Merrill 1968.)
The word when appeared in English in 17th century had a definite meaning but in the today’s
world, it is not easy to confine the meaning of technology into few words as it covers wide range of area and
the range extends from computers and software to the totality of all tools devised by or for humankind
(Encyclopedia Britannica 2014.) Now, almost each and every human activity is covered with the technologies.
The progress and the changes in the technological field not only has helped the people but also has created a
new hope every day with better future. And since progress is being every day the hope of better future is also
becoming stronger. Compared to the past the progress rate of technological development has been increasing
each day and every second. This progress has been making the world digital every day. The developments have
made peoples life easier and convenient but now the time has come to pause and think. There are many digital
devices which have surrounded people and have helped them in their daily work. But somehow those devices
have been holding people with themselves inside the virtual world and have taken the people away from the
real nature or environment. From this perspective it is essential for people to modify the technology in such a
The scientific definition of sixth sense is power of perception seemingly independent of the five senses
which are hear, touch, smell, sight and taste. That is to say the reception of information apart from the five
senses. This also means Extra Sensory Perception (ESP). Challenge is what people like and change is what
they want. To make a device which has power of perception will be the new change in technology and being
successful in such task will be a great challenge. Using this definition engineers tried to develop an interface to
receive the information from the surroundings and interact with people. Since the objective of the device was
to receive the human interaction and use the device’s own sense (according to the embedded program) to
analyze the interaction and give the results, it was named as sixth sense device. The reason for development of
this kind of device is to compel the technology to adapt peoples’ environment. People would then be able to
stand up from the chair in front of the computer monitor and would enjoy the real world at the same time
interacting with the digital device using it as their extra 5 sense, sixth sense. (Merriam-Webster.com 2014; the
Free Dictionary 2014; Mistry 2009.)
The first one to make a device with sixth sense was Steve Mann. By developing the concept and
making the wearable computer in 90’s when he was Media Lab student, Steve Mann became the father of Sixth
Sense Technology. He implemented the concept as the neck worn projector with the camera system. Currently
he is the professor at University of Toronto. He is also a director of Eye Tap Personal Imaging (ePi) Lab and
FL_UI_D laboratory. Apart from this, Mann has contributed in many different projects and published many
articles and books. After Mann, the concept was developed further by Pranav Mistry, the Head of Think Tank
Team and Director of Research of Samsung Research America, when he was a Research Assistant and PhD
candidate at the MIT ( Massachusetts Institute of Technology) Media Lab. (Mann 1990; Mistry 2013; Arora
2012.)
The working mechanism of the device is very simple. It gathers data from the users’ surroundings,
makes query using the internet as a data store, processes the query and presents information back to user via a
display. Simple computer vision algorithm and gesture recognition technology are used to understand the input
fed by the user as the input is received by the device as the image or the gesture produced by the hand or face.
Chapter 3
3.1 Telepointer
The first prototype of Sixth Sense technology was developed by Steve Mann. The
name of this device is Telepointer. It is hands-free, headwear-free-device that al- lows the
wearer to experience a visual collaborative telepresence, with text, graphics, and a shared
cursor, displayed directly on real world objects (Mann 2000, 177-178). Mann has referred
this device as “Synthetic Synesthesia of Sixth Sense”. It is a real-time groupware interface
feature for indicating where other users are pointing which can also act as a communication
channel by conveying gestural messages. (Xia, Sun, and Chen 2005.) It has been tested in
telemedicine, and has potential to become a great influence in improving quality of health
care. The other reason of its great possibility is its capability of effective communication,
precise diagnosis which helps to take a better decision by means of discussion and
consultation between the expert and the junior doctors. Since it has large potential for
wider acceptance in real life applications, more improvement in real time positioning
accuracy is needed.
According to Steve Mann (Mann 2000, 177.), Telepointer operates through a Reality
User Interface (RUI) which allows direct interaction with the real world, establishing a kind
of computing that is completely free of metaphors. Graph 1 shows how the device works.
The device was proposed as a Direct User Interface (DUI) to fulfill the desire of the
augmented reality. It works was the pointing device. The question may arise, if it is a
pointing device then what makes it different than other devices with same functionality. In
order to get the answer, it is first essential to know the working mechanism of the device in
detail.
This device establishes the connection between the long distance objects. If somebody is in
Helsinki and wants to buy some cloths from a shop in Kokkola, the person in Kokkola wears
Telepointer device, and the video is projected on the screen of person who is in Helsinki. The
person does not have to produce the sound or to choose the object; all he needs to do is to
point a object on his projected screen with laser pointer. The laser point is pointed exactly on
the same object in a shop in Kokkola as of the screen in Helsinki. At the same time both, per-
son in Helsinki and in Kokkola see the point on the same object. If he scribbles any image
in the projected screen the one wearing the device sees the same scribbled image in the
real object. Graph 2 shows how Telepointer looks. It has also become a powerful tool in the
field of medicine as it enhances the effective- ness of long distance communication. (Karim,
Zakaria, Zulkifley, Mustafa, Sagap & Latar 2013.)
After Steve Mann, seven students of MIT started to think about integrating the world
around them to access information without taking out any devices from their pocket. They
were successful to develop a wristband which read radio frequency identification tag for
recognizing different things, for example which book a user is holding in a store. They had
also made a ring using infrared to communicate by beacon. The ring was used in supermarket
smart shelves to find information about the products. According to criteria of the program
inside the system, the ring would blink in different LED lights to determine if the product
was organic or not. (Kohil 2013; Sharma, Agarwal, Sharma, & Gupta 2013, 277-282.)
Since the device is wearable and we can interact with the real world it was given a name
Wear Ur World (WUW). Gesture recognition is used to understand the gestures by webcam;
computer vision algorithms are used to identify the objects through camera. Graph 3
describes the prototype or connection of the WUW. (Mis- try 2013.)
Now a days WUW is widely called as “Sixth Sense”. The Mistry has published the
beta software version of this device written in C++ and C# in the Google Code project and he
also has given the name of the project WUW. This prototype also won an Invention Award in
2009 from Popular Science. (Mistry 2009; Arora 2012; Bhartia et al. 2012; Gupta &
Mohammad 2011; McManus 2009; Venukumar 2010Kohil 2013; Sharma et al. 2013.)
The camera streams the video and the gestures made by the user’s hand. The user’s
fingers’ tips are covered with different distinct colors which helps the cam- era to recognize
the gesture. The camera has a sensor to understand the gesture. These streaming images of
objects and gestures are taken as instructions by the computing device. After which the
device works as the instruction and also per- forms query to the internet if needed and passes
the information back. That information is projected out through the projector and mirror
reflects it to the desired surface or object. The user can then interact with the projected image
to give an- other instruction. The working is illustrated in Graph 4. (Mistry 2009; Arora
2012; Bhartia et al. 2012; Gupta & Mohammad 2011; McManus 2009; Venukumar
2010Kohil 2013; Sharma et al. 2013.)
etal.2013.)
Chapter 4
REQUIREMENTS OF WUW
Requirements can be defined as a careful assessment of the needs for the system. Any
system is designed to fulfill a certain function or task. The components or functions needed
to fulfill such task are known as requirements. (Mylopoulos 2004.) The technical device is a
composition of hardware and software. As one does not work in the absence of other, both
are essential components. To make WUW work, certain hardware and software are required.
Mistry made the requirement list available in the Google Code. The required things are
explained below.
4.1 Software
The set of instructions that tells computer or computing device about the task to be
performed is called software. According to the definition of West’s Encyclopedia of
American Law, software is intangible personal property consisting of mathematical codes,
programs, routines, and other functions that controls the functioning and operation of a
computer’s hardware. (West’s Encyclopedia of American Law 2005.) The instructions are
written using the language like C, C++, C# and JAVA. It is not a physical component.
4.2 Hardware
Hardware is a physical component of a computer system or a computing device. All the
equipment like circuit boards, peripheral devices, cables and connectors falls inside this
category. It is not that difficult to make a sixth sense device be- cause of cost efficiency. Also
the components that are required are easily available. The essential components to make the
sixth sense device are color markers, camera, computing device, projector, mirror and
microphone. The requirement of those components for making the sixth sense hardware
device is explained in de- tail below. (Mistry 2012.)
projected application interfaces. This is essential to pass input to the device. Any distinct
color can be used to mark the finger. Even nail polish with different colors can work
sometimes, or multicolor tape, paper, or paint can be used as a marker. (Mistry 2009; Arora
2012; Bhartia et al. 2012; Gupta & Mohammad 2011; McManus 2009; Venukumar
2010Kohil 2013; Sharma et al. 2013.)
4.2.2 Camera
The work of camera is to capture images or videos. In sixth sense also, it performs the
same task acting as an input device. Acting as user’s third eye, it captures the image, streams
video of objects and tracks the gestures. After recognizing all the input, it sends the data to a
computing device for processing. The camera used in this device should have the sensor to
track the gesture created by the users. For the program written by Mistry, he has suggested to
use Logitech QuickCam Pro for notebooks. (Mistry 2009; Arora 2012; Bhartia et al. 2012;
Gupta & Mohammad 2011; McManus 2009; Venukumar 2010Kohil 2013; Sharma et al.
2013.)
4.2.4 Projector
Projector projects the visual information enabling surface and physical objects to be
used as interfaces. In the device made by Mistry, the projector has inbuilt three hour battery.
But since it the device can be made by user himself, it depends upon the choice and wish of
user to modify it. It acts as the output device. The content, digital information and a graphical
user interface are provided by computing de- vice.
4.2.5 Mirror
The mirror performs special task in this prototype. Since the projector is pointed
downward, the mirror helps to reflect the image on to a desired surface such that the digital
image is freed from the confined screen and placed in the physical world. It is attached in a
way that it can be tilted anywhere as per user’s desire of location to project the image. It
provides the ability for user to choose the size and place of projecting the result. Varying the
angle of reflection the size of the projection can also be varied. (Mistry 2009; Arora 2012;
Bhartia et al. 2012; Gupta & Mohammad 2011; McManus 2009; Venukumar 2010Kohil
2013; Sharma et al. 2013.)
4.2.6 Microphone
Chapter 5
Though the concept of sixth sense or WUW is quite similar as Telepointer, there are some factors that
make sixth sense different and somehow complex also. (Mann 2000; Mistry 2009.) One of those factors is the
technology that is behind it. Different technological concepts behind WUW are ubiquitous computing,
augmented reality, gesture recognition, computer vision and radio frequency identification. (Kumar &
Pandithurai 2013, 947-953.) The detailed information of what these terms and terminologies are described in
the following topics.
The term ubiquitous computing (Ubicomp) was coined by late Mark Weiser (1952- 1999). As the
verbal meaning of ubiquitous is to be present, appear or found every where it can be said that ubiquitous
computing is the concept of making computer available everywhere. According to the Oxford Dictionary of
the Internet, this term is used to describe a future where computers are so numerous that they affect every
action people take. This includes computers that are part of people’s clothing and which monitor their vital
functions, computers which control traffic, computers which are integrated with television sets and computers
that control people’s homes. (Oxford Reference 2009.)
The term became popular and caught attention of other researchers when Weiser introduced the vision
of Ubiquitous Computing in an article titled “The Computer for the 21st Century”, written in 1988 but
published during 1991 in Scientific Ameri- can, while he was at Xerox PARC. According to him, it is a
method of enhancing computing use not only by making many devices available throughout the physical
environment but making them effectively invisible to the user. (Weiser 1991.) Weiser also mentioned in
his article that these computers reside in the human world and pose no barrier to personal interaction. Today,
Ubicomp has become the inspiration through the development of the new paradigm of computation (off the
desktop), where the interaction between humankind and the computer was natural and it has slowly left the
paradigm keyboard/mouse/display through the PC generation. (Agushinta, Kusuma, Junatas & Trihasta 2009.)
Augmented reality (AR) is the term that describes a combination of technologies that enable real time
mixing of computer-generated content with live video dis- plays. (Mullen, 2011.) This in other word means to
The name, AR, was first coined in 1990 by a researcher named Tom Caudell at The Boeing Company
and the major development in this technology occured at the end of 90’s when Hirokazu created ARToolKit, a
powerful library of tools for creating AR applications. (Mullen, 31, 2011.) Its endless possibilities in different
fields are one of the reasons of its growing popularity. (Geroimenko, 2012.) And now there are already a
handful of applications available on smartphones or port- able devices.
The process of recognizing the gesture (hand movements, facial expressions, body movement etc.) is
called gesture recognition and the computer interface us- ing this method is called gesture recognition
technology. It is a topic in technology language which interprets human gestures via mathematical algorithms.
This technology is a better alternative to the text and graphical user interface since it does not require any
mouse, track pad, joysticks or keyboard. There are more possibilities that all such devices will be replaced by
sixth sense technology just like touch screen replacing the keypad of phones. (Patch 2003, 1.)
The Magic Cube, evoMouse, Kinect, The Leap are some of the recent technolo- gies which
implement gesture recognition. There are numbers of companies work- ing on it and creating or developing
technologies based on gesture recognition every day. Sixth sense implements this technology to feed input.
As for now, the webcam recognizes the gestures made by the colored marked fingers of users. (Celluon
2012; Microsoft 2013; LeapMotion 2013.)
It is commonly accepted that the father of computer vision is Larry Roberts, who in his Ph.D. thesis in
MIT discussed the possibilities of extracting 3D geometrical in- formation from 2D perspective views of
blocks. (Aloimonos, 1992) Computer vision is a field that includes methods for acquiring, processing,
analyzing, and under- standing images and, in general, high-dimensional data from the real world in or- der to
produce numerical or symbolic information. This technology includes the aspect of Artificial Intelligence (AI)
techniques like pattern recognition. Image sensors are the essential factors of this device since it has to detect
electromagnetic radiation which is in the form of light rays. Computer vision algorithm is used to obtain
important information about the spatial position of objects in a scene. But the efficient exploration of these
Radio Frequency Identification (RFID) is a means of identifying objects by interro- gation a unique
characteristic of the object using radio wave. (National Academies Press, 2004.) It is a rapidly growing
segment of automatic identification data col- lection (AIDC). According to AIDC researchers, the first
implementation of RFID was completed during the World War II to identify and authenticate the allied planes.
Even though it is currently being used in numerous applications throughout the world, the ratio of its uses has
not been in the way as it should. It transmits the identity of an object wirelessly, using radio magnetic waves.
The main purpose of this technology is to enable the transfer of a data via a portable device. RFID has an
ability to identify objects without the requirement of a line of sight meaning that the objects can be identified
even when they are tightly packed together or their surface markings are removed, marred, or obscured.
(National Academies Press, 2004.)
Chapter 6
APPLICATIONS AND USES OF WUW
It is normal that people search for the application and its uses before they buy it. WUW has wide range
of applications and uses. Many different applications of this device demonstrate the usefulness, viability and
flexibility of the system. (Mistry 2009.) It has applications which perform the task of multiple devices that
people are using nowadays. Due to those applications, it offers us freedom from many dif- ferent devices.
Among many of its applications and uses, some of them which can be used in daily basis are going to be
described below.
6.1 Applications
WUW implements wide range of applications which are not only useful in daily life but are also very
different from applications that people have been experiencing before. For example a newspaper can show live
video related to the subject mat- ter. The computing device accesses internet for searching out the related
Dept of CSE, SKIT 16 2023-24
Sixth Sense Technology
subject and passes the information in the form of video through projector over the news- paper. Thus the user
can read the news in the paper as well as see the video. Apart from this, some of the applications are described
below. (Mistry 2009; Arora 2012; Bhartia et al. 2012; Gupta & Mohammad 2011; McManus 2009;
Venukumar 2010Kohil 2013; Sharma et al. 2013.)
This application has wide range of uses. It can be used to navigate, to find places, to
find the shortest path and many more things. This application starts when the user forms the
sign of magnifying glass. The gesture recognition technology helps to understand this. When
the application starts, a user can zoom in and zoom out using two fingers’ gesture just like in
Graph 7. The device is developed on multi- touch based system.
FIGURE 5. Search the map application. (Adapted from Sharma et al. 2013.)
Now, color pens and papers or any wide device is not necessary for drawing picture.
Using this small portable device one can let out his artistic talent anywhere on any surface.
Understanding the gesture of the colored fingertip of the user, this application draws the
image and projects it on the surface. The drawing is saved in the computing device and it can
be retrieved anytime wherever the user wants. Graph 8 shows the user drawing on a wall
using the gesture of a hand through drawing application.
FIGURE 7. Take a picture gesture (Framing gesture). (Adapted from Sharma et al.
2013.)
6.1.5 Clock
It is not necessary to buy watches anymore to see the time. Users do not have to take
their phone out from the pocket either. Using clock application of sixth sense device, in order
to check the time, all that the users need to do is show their wrist in front of the camera and
just draw a circle on it. This will display the clock in the hand of a user. To exit the
application, the user can slide his fingertip from left to right. Graph 10 shows how easily the
clock can be projected on the wrist of a user. (Mistry 2009; Arora 2012; Bhartia et al.
2012; Gupta & Mohammad 2011; McManus 2009; Venukumar 2010Kohil 2013; Sharma
et al. 2013.)
6.2 Uses
Sixth sense is a technology that can be used to carry the real world objects in to
digital world with a minimum effort and a higher accuracy. (Mistry 2009; Arora 2012;
Bhartia et al. 2012; Gupta & Mohammad 2011; McManus 2009; Venukumar
2010Kohil 2013; Sharma et al. 2013.) Everyone will have the capability and right to develop
the applications and use them as per their wish. Sixth sense can be used as phone, as a
camera, as a projector or even as a computer. Among the variety of uses the key use of this
device is as fifth sense and in industries.
Simple modification in this device can help disabled people to use this device as their
fifth sense. For example just adding the speaker can help people who cannot speak as they
can use a gesture to explain their opinions or instruction which the computing device will
understand and will transfer the information in the form of sound waves. And the other
people can hear that information through speakers. Tt will not be necessary for everyone to
learn sign language to understand them. (Mistry 2009; Arora 2012; Bhartia et al. 2012;
Gupta & Mohammad 2011; McManus 2009; Venukumar 2010Kohil 2013; Sharma et al.
2013.)
There are different industries like metal, chemical or plastic. Works in those indus-
tries are sensitive and difficult as everyone might not know how to handle the dif- ferent
machines or objects. This device can be used to provide information about those objects. The
device also provides easy control over the machineries in dif- ferent industries. (Mistry 2009;
Arora 2012; Bhartia et al. 2012; Gupta & Mohammad 2011; McManus 2009; Venukumar
2010Kohil 2013; Sharma et al. 2013.)
Chapter 7
As the development of the technology has reached its peak stage the question that
appears is what to develop next or what to do next. In such context, the con- cept of
implementing sixth sense in the technological devices became the answer of all the
questions. The concept has been taken into consideration by different companies. And now,
there are different products that can be seen in market place which are related to the sixth
sense. Several technological companies have al- ready started to implement the sixth sense in
their devices or products.
Whirlpool has launched its washing machine model AWOE 8758 which uses the sixth
sense technology to monitor and to control the washing cycle. This helps to obtain the perfect
result in the most efficient way. The inbuilt sensor of the device adjusts water, temperature
and time levels to save energy and natural resources. Today, there are many washing
machines found of this type. These devices really help to preserve the environment and
implements green technology. (Whirlpool 2014.)
There are different distractions in the environment. These distractions limit the ability
of people to use all of their senses to find out the things happening around them. While
driving, there might be other vehicles in the blind spot or stopped in the area that is difficult
or impossible to see, or is about to enter the same intersec- tions same as us. For this, there is
General Motors V2V (vehicle-to-vehicle) com- munication system which communicates with
other similar devices and warns the driver about the possible danger. (RITA 2014.)
After Mistry revealed the concept of SixthSense device in TED (Technology, En-
tertainment, Design) talks in 2009, it took time for everyone to view if the concept in real is
possible or not. Everyone was waiting for the open source promised by Mistry. Finally in
year 2011, he made beta version, WUW_v0.1, of device code available for everyone in
Google Code with the title sixthsense and project name WUW. This work created hope in
people and proved that the concept is possible and it works. The code is written in C# with
OpenCV (for .NET) and it is tested of Windows. (Mistry 2009; McManus 2009; Mistry
2012.)
Since 2012 there has not been any update in WUW project in Google Code. Nei- ther
Mistry nor his project members has updated it. Other users also have not up- loaded the new
modified version of the code. Looking through the comments post- ed on different pages of
the site, it seems like everyone is busy developing the be- ta version of WUW in their home
and school project only. Yet in GitHub, the work to port the code in JAVA is on progress.
The members have mentioned that they are working on an Android port. There is no change
in the codes of the C# since a year but the work in JAVA has been updated 8 month ago i.e.
during July 2013. And PointR, SizeR, REctangleR, Category, NBestList, NBestResult
and Ge- ometricRecognizer have already been successfully ported.
Even though there are many users being associated with the project and following the
work, only four of the members seem to be working actively on this. But look- ing at the
activity log or the trend of users being associated with the project, there is a hope that
someday somebody will bring a change. It can also be predicted that the users are working
on the codes behind the scenes. It is just the matter of time to find the good solution and post
it on the site. (GitHub 2013.)
Even though the new development of the project WUW is not clearly visible at the
moment outside the Media lab or in any news or articles, and one that is visible is slow, the
development of the sixth sense technology on progress implemented by different companies
in their own way. There are different products like Google Goggles and Eye Tap which are
based on the concept of sixth sense.
Chapter 8
8.1 Advantages
As the device is small in size it is portable. (Sharma et al. 2013; Gupta & Moham-
mad 2011) It supports multi-touch and multi-user interactions. (Kohil 2013.) It connects a
digital world and physical world as user do not have to take their device out from their
pocket and press words to search the information. The task can be performed through a
user’s movements and gestures. (Mistry 2009; McManus 2009; Arora 2012; Bhartia et al.
2012; Gupta & Mohammad 2011; Kohil 2013; Sharma et al. 2013; Venukumar 2010.)This in
an extent also saves user’s time, the time taken by user take the device out and compute the
search query. Data can be accessed directly from the machine in real time wherever users are
and the ideas can be mapped anywhere users want. (Mistry 2009; Kohil 2013..) It can have
different applications for different developers depending on how user imagines and wants.
The project is an open source which enables individuals to devel- op their own application
depending on their needs and imagination. (Mistry 2009; Kohil 2013.) It can perform task of
different devices by itself so it saves the money that users use to buy the devices like camera,
phone, laptop or watch. (Kohil 2013.)Even after all these the cost of this setup is not
expensive so it helps to make the quality of living better. (Bhartia et al. 2012.)
Finding the good vision based algorithm has been always the main challenging part.
There are many challenges associated with the accuracy and usefulness of gesture
recognition software. There are limitations on the equipment used and im- age noise for
image-based gesture recognition. Images or video may not be under consistent lighting, or in
the same location. Items in the background or distinct features of the users may make
recognition more difficult. The variety of implementations for image-based gesture
recognition may also cause issue for viability of the technology to general usage. For
example, an algorithm calibrated for one camera may not work for a different camera. The
amount of background noise also causes tracking and recognition difficulties, especially
when occlusions (partial and full) occur. Furthermore, the distance from the camera, and the
camera's resolution and quality also cause variations in recognition accuracy. (Sousaet al.
2013.)
8.3 Enhancements
In past, there used to be the gloves for the recognition of hand gesture. They have
been replaced by the use of color markers in this device. Searching the alternative for the
color markers would be another asset of this device. Kinect and The Leap have shown that it
is possible. Enhancing the device to be practiced in various fields like gaming, education
systems, entertainment fields will generate wide range of costumer. Extra speaker can be
added or mobile computing device with inbuilt speaker can be used to prepare this device.
This allows developers to develop and application that can translate the gesture into sound.
Device with the gesture to sound converter application helps the people with disabilities to
communicate with others and to be stronger. Integrating camera and projector in the mobile
computing device would be the perfect idea. As the devices like The Leap, Kinect have
inbuilt sensors to recognize the gesture, replacing camera with one of these devices would be
nice idea to solve the limitation of camera algorithm. (Patch 2003;Celluon 2012; Leap
Motion 2013.)
The suggestion for the enhancement of the device would be first to finish porting the
code in JAVA and to adjust it for Android also. Eventually, phones with projector can be
used with little modifications like adding The Leap just below the cam- era section and
adding the mirror just below the projector. After the modification, shaping the device as
wearable is not a problem. This modification provides multiple options to the users. They can
use the phone by touching in the front screen or just turn the device to back and use it by
gestures and images. By adding speech integrated circuit, the device can be built useful for
the users with disabilities. The devices like Samsung i7410 for the Android version, LG
EXPO (GW820) for the Windows version or components like monolith for iPhone version
codes are benefit for the enhancements. (Samsung 2009; LG 2009.) The prototype of this
proposal is shown in Graph 11.
Mobile computing
Device
In the given concept, the mobile computing device has inbuilt microphone and
speakers. Also the device has port to insert microphone and speakers if the user wants the
external devices. All the other components of the device are similar except the added
component, The Leap. The Leap is a device that tracks movement of both hands and all 10
fingers through the open space between a user and users computing device. It operates using
LED lights and camera sensors. The soft- ware detects user’s hands and fingers and translates
the data into information for the device (LeapMotion 2013.)
Chapter 9
CONCLUSION
Sixth sense is the science of tomorrow with the aim of connecting the digital world
with the physical world seamlessly, eliminating hardware devices. Even though the sixth
sense devices are in development stage and have not been used in widely, it is predictable
that this technology will revolutionize the way people interact with the digital world.
Sixthsense has potential to become the extreme transparent user interface for accessing
information about everything around it. And as different de- vices have already started to
come out with the implementation of this concept, it can be predicted that in near future
everyone will have this device in the way they are having cellphones now. Diminishing the
mouse and keyboard from everywhere is a challenge. Regardless of that challenge it will for-
sure reduce the number of users who uses devices like keyboard and mouse.
There is no doubt that the future of Sixth Sense technology will continue. Once this
device is developed, there is no need of printing all the details of components available in the
supermarket. Because this device can scan the object and show all the information, details,
compositions, and even the costumers’ review. All the details can be stored in the cloud and
this device can retrieve that information just like the QR code and project it in the real
environment in real time. Finding the suitable algorithm for the feasible working of the
device, implementing the security system for the personal informations, are however, the
biggest general problems of this concept. If the challenges and limitations are overcome with
the enhance- ments of the device, no doubt the device will be a benefit to everyone in the
entire world.
REFERENCES
Aloimonos, Y. 1992. Special Issue on Purposive and Qualitive Active Vision, CVGIP B:
Image Understanding (Vol. 56)
Arora, M. 2012. Basic Principles of Sixth Sense Technology. VSRD International Journal of
Computer Science and Information Technology, VSRD-IJCSIT, Vol.
2(8), 687-693
Azuma R. Baillot Y. Behringer, R., Feiner, S., Julier, S. and MacIntyre, B. 2001. Recent
advances in augmented reality
Bhartia, A., Bhatia, R., Bhatia, S. And Rani, G. 2012. Sensing the sixth sense technology,
International Journal of information technology and knowledge man- agement, Volume 5,
201-204.
Bimber, O. and Raskar, R. 2005. Spatial Augmented Reality: Merging Real And Virtual
Worlds, A K Peters Ltd.
Dhingra, S. 2011. Sixth Sense Technology will Revolutionize the World, The Views Paper,
Available: http://theviewspaper.net/sixth-sense-technology-will- revolutionize-the-world/.
Accessed 18th March 2014
Geroimenko, V. 2012. Augmented reality technology and art: the analysis and vis- ualization
of evolving conceptual models
Grznar, M. 2013. Sixth Sense Device: What it is and how to get it. Available
http://indiegadgets.com/2013/05/sixth-sense-device-what-it-is-and-how-to-get-it/
http://indiegadgets.com/2013/05/sixth-sense-device-what-it-is-and-how-to-get-it/. Accessed
17th February 2014.
Karim, R. A., Zakaria, N. F., Zulkifley, M. A., Mustafa, M. M., Sagap, I. And Latar,
N. H. M. 2013. Telepointer technology in telemedicine: a review, Available:
http://www.biomedical-engineering-online.com/content/12/1/21. Accessed at 19
February 2014
McManus, E. 2009. An interview with Pranav Mistry, the genius behind Sixth Sense, 11
March 2009, Available: http://blog.ted.com/2009/03/11/sixth_sense_pranav/. Accessed at
17th February 2014
Sousa, E. R., Tanase, A., Hannig, F. and Teich, J. 2013, A Prototype of an adap- tive
computer vision algorithm on MPSoC architecture, Design and Architectures for Signal and
Image Processing (DASIP), 2013 Conference
Xia, S., Sun, D., Sun, C. and Chen, D. 2005. Object-Associated Telepointer for Real-Time
Collaborative Document Editing Systems.