CSI4005

Augmented Reality
and Virtual Reality
Dr.Priyanka N
 Virtual Reality and Augmented
Reality?

• Virtual reality (VR) and augmented reality

(AR) are both technologies that create a
sense of immersion in a digital
environment, but they differ in how they
do it
 What is Augmented Reality?
• Augmented Reality (AR) is a
technology that adds digital
information to the real world, such
as images, videos, or 3D models, to
create immersive experiences.
 What is Augmented Reality?
Some examples of AR include:
❑Pokemon Go

❑Ikea furniture arrangements

❑Home Depot color options

❑Smithsonian Museum dinosaur experience

❑Snapchat
 What is Augmented Reality?
Key Components of an AR System:
1. Camera: Captures the real-world environment.
2. Processor: Processes the camera input and generates
the augmented reality scene.
3. Display: Projects the augmented scene onto a screen
or directly into the user's field of view.
4. Sensors: Provide information about the user's
position and orientation.
 What is Virtual Reality?
• Creates a fully-immersive, computer-
generated environment that users can
explore and interact with.
• Users can wear devices like headsets or
goggles to interact with the environment.
• VR is used in 3D films and video games.
 What is Virtual Reality?
Key Components of a VR System:

1. Head-Mounted Display (HMD): Covers the

user's eyes and ears, providing visual and
auditory input.

2. Sensors: Track the user's head movements and

provide input to the VR system.

3. Controllers: Allow users to interact with the

virtual environment.

4. Powerful Computer: Processes the virtual

world and generates the immersive
experience.
Introducing importance
and applications of
augmented and virtual
reality systems
Importance of Augmented and Virtual
Reality Systems
• Augmented Reality (AR) and Virtual Reality
(VR) are transformative technologies that
blend the physical and digital worlds, offering
immersive and interactive experiences.

• These technologies are reshaping industries

by enhancing how people visualize, interact
with, and experience information.
 Importance of Augmented and Virtual
Reality Systems
Their importance lies in their ability to:
1. Enhance Engagement: AR and VR create compelling experiences that
boost user engagement, making them powerful tools for education,
training, and marketing.

2. Enable Realistic Simulations: VR provides fully immersive environments

for safe and controlled simulations, vital in areas like healthcare,
military, and engineering.

3. Bridge Physical and Digital Realms: AR overlays digital content on the

physical world, enriching real-world environments with contextually
relevant information.

4. Facilitate Accessibility: These technologies enable users to interact with

complex systems or remote locations, democratizing access to
education, entertainment, and professional tools.
 Applications of Augmented and Virtual Reality Systems
Healthcare:
• Training and Simulation: VR provides realistic scenarios for
surgical training and medical procedures, allowing
practitioners to hone their skills without risking patient
safety.

• Therapy and Rehabilitation: VR is used for treating

phobias, Post-traumatic stress disorder (PTSD), and
physical rehabilitation by creating controlled and
customizable therapeutic environments.

• AR in Diagnostics: AR enhances diagnostic procedures by

superimposing vital information directly onto a patient’s
body during medical examinations.
Applications of Augmented and Virtual Reality Systems
Education and Training:
• Interactive Learning: AR brings textbooks and
learning materials to life with 3D models and
animations, increasing comprehension and
retention.

• Skill Development: VR is used for technical

training in industries like aviation,
construction, and manufacturing, providing
hands-on experience in virtual environments.
Applications of Augmented and Virtual Reality Systems

Entertainment and Gaming:

• Immersive Experiences: AR and VR have
revolutionized the gaming industry by
providing highly immersive gameplay and
interactive storytelling.

• Virtual Concerts and Events: VR enables users

to attend live events and concerts from
anywhere in the world.
Applications of Augmented and Virtual Reality Systems

Retail and E-commerce:

• Virtual Try-Ons: AR allows customers to
visualize products like clothing, furniture, or
makeup in real-time, enhancing the shopping
experience.

• Immersive Showrooms: VR creates virtual

showrooms where users can explore products
in detail without visiting physical stores.
Applications of Augmented and Virtual Reality Systems

Real Estate and Architecture:

• Virtual Property Tours: VR lets buyers and
investors explore properties remotely, saving
time and effort.

• Design Visualization: AR enables architects to

overlay designs onto real-world settings,
offering clients a preview of construction
projects.
Applications of Augmented and Virtual Reality Systems

Manufacturing and Maintenance:

• Prototyping: VR allows designers and
engineers to test and iterate on prototypes in
virtual spaces, reducing costs and time.

• AR for Maintenance: Technicians can use AR

to access step-by-step repair guides and
overlays for equipment troubleshooting.
Applications of Augmented and Virtual Reality Systems

Travel and Tourism:

• Virtual Tours: VR allows users to experience
travel destinations virtually, providing a
preview of real-world visits.

• Cultural Preservation: AR enhances museums

and historical sites with interactive exhibits
and contextual information.
Applications of Augmented and Virtual Reality Systems

Workplace Collaboration:
• Remote Collaboration: AR and VR enable
virtual meetings and collaborative
workspaces, enhancing productivity for
distributed teams.

• Training in Hazardous Environments:

Simulations in VR prepare employees for
high-risk tasks, ensuring safety and efficiency.
 History and differences
between augmented and
virtual reality systems
 Virtual Reality (VR)
1. Early Concepts (1800s-1960s):
• Stereoscopes (1838): Early devices like the
stereoscope used two images to create a sense of
depth, a precursor to VR.
 Virtual Reality (VR)
1. Early Concepts (1800s-1960s):
• Sensorama (1956): Morton Heilig created the
Sensorama, a multi-sensory device offering
simulated experiences like riding a motorcycle.

American cinematographer and

inventor Morton Heilig described his vision
of a multi-sensory theater in a 1955 paper
entitled "The Cinema of the Future."
 Virtual Reality (VR)
1. Early Concepts (1800s-1960s):
• The Ultimate Display (1965): Ivan Sutherland
envisioned a "virtual world" in his essay.
• This concept inspired the creation of the first head-
mounted display (HMD).
 Virtual Reality (VR)
2. Development Era (1970s-1990s):
• First HMD (1968): Ivan Sutherland developed the
first HMD, "The Sword of Damocles," tethered to a
computer.

Sutherland created a system called

the “Sword of Damocles”, which used a
head-mounted display to project images
onto a screen in front of the user. This
system was the first to allow users to
experience a virtual world in real time
and interact with it.
 Virtual Reality (VR)
2. Development Era (1970s-1990s):
• VPL Research (1980s):
▪ VPL Research was one of the first
companies that developed and
sold virtual reality products.
▪ It was founded by computer
scientist Jaron Lanier in 1984
▪ VPL stood for "Virtual Programming
Languages"
▪ Jaron Lanier popularized the term
"virtual reality" and developed early
VR gear like the DataGlove and
EyePhone.
 Virtual Reality (VR)
3. Modern VR (2000s-Present):
• Oculus Rift (2012): Kickstarted the modern VR era
with high-resolution visuals and motion tracking.

Product type Virtual reality

headset (PC,
tethered)
Owner Oculus VR
Country United States
Introduced April 2012
Discontinued April 2021
Markets Worldwide
Tagline Step into the Game
 Virtual Reality (VR)
3. Modern VR (2000s-Present):
• Mass Adoption (2016-): Companies like HTC, Sony,
and Valve launched consumer-grade VR systems for
gaming, education, and professional use.
 Augmented Reality
1. Early Innovations (1950s-1980s):
• Heads-Up Display (HUD): Initially developed for
military aviation, HUDs displayed crucial information
overlaid on a pilot's view.
 Augmented Reality
1. Early Innovations (1950s-1980s):
• Videoplace (1974): Myron Krueger created an early
AR-like system where users interacted with virtual
objects.

❑ What we think of today, often

known as the "goggles and gloves"
archetype of virtual reality is not the
kind of virtual reality Krueger was
interested in exploring.

❑ He tended to favor the "art of

interactivity" over art that just so
happens to be interactive.

https://www.youtube.com/watch?v=dmmxVA5xhuo&t=4s
 Augmented Reality
2. Commercial Beginnings (1990s):
• AR Term Coined (1990):
 Augmented Reality
2. Commercial Beginnings (1990s):
• First AR Systems: Applications in medical imaging
and industrial design emerged.
 Augmented Reality
3. Expansion Era (2000s-Present):
• AR on Smartphones (2008): The rise of
smartphones enabled AR apps, starting with tools
like Wikitude and later Pokémon GO (2016).

Wikitude initially focused on

providing location-based augmented
reality experiences through the
Wikitude World Browser App.
 Augmented Reality
3. Expansion Era (2000s-Present):
• AR Glasses: Devices like Google Glass (2013)
and Microsoft HoloLens (2016) popularized
wearable AR technology.
Differences Between AR and VR
 Convergence and Future
• Mixed Reality (MR):
▪ Combines elements of AR and VR to seamlessly blend
real and virtual environments.

• Extended Reality (XR):

▪ Umbrella term encompassing AR, VR, and MR
technologies.
▪ As hardware becomes more compact and software
more advanced, the lines between AR and VR are
increasingly blurred, paving the way for versatile
applications across industries.
Basics of computer vision
and multimodal
interactions in AR VR
 Basic of Computer Vision

https://www.almabetter.com/bytes/tutorials/artificial-intelligence/computer-
vision-in-ai
• Computer vision and multimodal
interactions are foundational in
Augmented Reality (AR) and Virtual
Reality (VR) applications, enhancing how
users perceive and interact with virtual
environments.
 Basics of Computer Vision in AR/VR
• Computer vision involves enabling
computers to interpret and understand
visual data from the world.

• In AR/VR, it plays a crucial role in bridging

the gap between the real and virtual
worlds.
 Basics of Computer Vision in AR/VR
Core Concepts:
1.Image Processing:
• Techniques like edge detection, filtering, and
segmentation help extract useful features from
images or videos.
• Example: Identifying boundaries of objects in a
user's environment.

2. Object Detection and Recognition:

• Recognizing and tracking objects in real-time for
interaction or overlaying virtual content.
• Example: Identifying furniture for placement of
virtual objects in AR.
 Basics of Computer Vision in AR/VR
Core Concepts:
3. Tracking and Pose Estimation:
• Determining the position and orientation of the
camera or objects relative to the environment.
• Used for head tracking (VR) or aligning AR
elements with real-world objects.

4. Depth Estimation:
• Using stereoscopic vision, LiDAR (Light Detection
and Ranging), or depth cameras to measure
distances in a scene.
• Critical for accurate placement of virtual objects in
AR.
 Basics of Computer Vision in AR/VR
Core Concepts:
5. SLAM (Simultaneous Localization and Mapping):
• Builds a map of an environment while tracking the
user's location.
• Essential for AR applications to anchor content in
physical spaces.

6. Gesture Recognition:
• Analyzing hand or body movements to enable natural
interaction.
• Example: Using hand gestures to manipulate objects in
VR.
 Multimodal Interactions in AR/VR

Multimodal interaction integrates

multiple sensory inputs—visual,
auditory, haptic, and more—for
intuitive user experiences.
 Multimodal Interaction

http://humancarinteraction.com/multimodal-interaction.html
 Multimodal Interactions in AR/VR
Key Modalities:

1.Visual Interaction:
• Augmented visual feedback through overlays,
holograms, or enhanced scenes.
• Example: Highlighting paths or objects in an AR
game.

2. Auditory Interaction:
• Spatial audio to provide directional sound cues.
• Example: Hearing footsteps approaching from
behind in VR.
 Multimodal Interactions in AR/VR
Key Modalities:

3. Haptic Feedback:
• Simulating touch sensations using vibrations,
force, or tactile feedback.
• Example: Feeling the texture of a virtual object in
VR.
4. Speech and Voice Interaction:
• Natural Language Processing (NLP) for voice
commands and conversational agents.
• Example: Asking a virtual assistant to modify the
environment in AR.
 Multimodal Interactions in AR/VR
Key Modalities:

5. Eye Tracking:
• Monitoring gaze to focus interactions and reduce
computational load (foveated rendering).
• Example: Selecting items in VR by looking at them.

6. Hand and Body Motion:

• Using sensors or cameras to capture gestures and
body movements for interaction.
• Example: Waving to control navigation or using hand
gestures to sculpt a virtual object.
 Applications in AR/VR:
• Gaming: Immersive worlds with gesture-
based controls.
• Education: Interactive learning using 3D
visualizations.
• Healthcare: AR-assisted surgeries or VR for
therapy.
• Retail: Virtual try-ons or showroom
experiences.
• Industrial Training: Simulating scenarios for
hands-on practice.
 Summary

By integrating computer vision with

multimodal interaction, AR/VR systems
deliver more immersive and intuitive
experiences, paving the way for
innovations in entertainment, education,
healthcare, and beyond.