Scribd
Upload
276 views30 pages

Computer Vision Used in Self-Driving Cars

Computer vision is used in self-driving cars for tasks like object detection, scene segmentation, and object tracking to help the vehicle perceive its environment. Convolutional neural networks are commonly used for tasks like object detection. Region-based convolutional neural networks (RCNN) were developed to improve object detection in images. Google's self-driving car uses a combination of sensors like cameras, lasers, and ultrasonic sensors along with computer vision algorithms to navigate. Some challenges for self-driving cars include cybersecurity, high costs, handling extreme weather conditions, and addressing ethical concerns.

Uploaded by

jaseem CK
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
276 views30 pages

Computer Vision Used in Self-Driving Cars

Computer vision is used in self-driving cars for tasks like object detection, scene segmentation, and object tracking to help the vehicle perceive its environment. Convolutional neural networks are commonly used for tasks like object detection. Region-based convolutional neural networks (RCNN) were developed to improve object detection in images. Google's self-driving car uses a combination of sensors like cameras, lasers, and ultrasonic sensors along with computer vision algorithms to navigate. Some challenges for self-driving cars include cybersecurity, high costs, handling extreme weather conditions, and addressing ethical concerns.

Uploaded by

jaseem CK
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 30

Computer Vision used in Self-

driving cars

Guided by: Jaseem C K


Prof. Sruthy Manmadhan S7 CSE
NSSCE Palakkad Roll no : 59
OUTLINE
❖ Introduction to Autonomous vehicles
➢ Perception
➢ Localization
➢ Path planning
➢ Control
❖ Computer Vision
➢ Object Detection
➢ Semantic scene segmentation
➢ Object Tracking
❖ Object Detection using CNN
❖ Technologies that power Google’s self driving car
❖ Challenges and future of Autonomous vehicle
1. INTRODUCTION
❖ What is self-driving car?
➢ A truly driverless car must be capable of navigating to a destination,
avoiding obstacles and parking without any human intervention.
❖ Artificial intelligence is everywhere in self-driving cars and especially in the
perception of the environment.
➢ Computer vision and sensor fusion are called perceptron. It’s all about
understanding the environment.
❖ Main steps in the operation of an autonomous vehicle
➢ Perception
➢ Localization
➢ Path planning
➢ Control
2. COMPUTER VISION
❖ Computer vision is a discipline that allows a computer equipped with a
camera to understand its environment.
❖ Computer vision is used in autonomous vehicles to detect pedestrians
or other objects.
❖ Cameras are key to a variety of essential tasks: lane finding, road
curvature estimation, obstacle detection and classification, traffic sign
detection and classification, traffic light detection and classification,
and more.
Computer vision solutions in self-driving cars

Object detection Semantic scene segmentation Object tracking


Object Detection

❖ Object detection is a computer


technology related to computer vision
and image processing that deals with
detecting instances of semantic
objects of a certain class (such as
humans, buildings, or cars) in digital
images and videos.
Semantic scene segmentation

❖ Segmentation is essential for image analysis tasks. Semantic


segmentation describes the process of associating each pixel of an
image with a class label, (such as flower, person, road, sky, ocean, or
car).
❖ It is the task of assigning a class to every pixel in a given image.
❖ Self-driving vehicles rely heavily on such segmented images to
navigate through routes
Semantic segmentation
Object Tracking

❖ Object Tracking is an important domain in computer vision. It


involves the process of tracking an object which could be a person,
ball or a car across a series of frames. For people tracking we
would start with all possible detections in a frame and give them an
ID.
❖ Object tracking is the task of taking an initial set of object detections,
creating a unique ID for each of the initial detections, and then
tracking each of the objects as they move around frames in a video,
maintaining the ID assignment.
Object detection using CNN

Neural Networks

Convolutional Neural
Networks

RCNN
Artificial neural networks or connectionist systems are computing systems that
are inspired by, but not identical to, biological neural networks that constitute
animal brains. Such systems "learn" to perform tasks by considering examples,
generally without programmed with task-specific rules
How does a machine look at an image?

Every image is an arrangement of


dots (a pixel) arranged in a special
order. If you change the order or
color of a pixel, the image would
change as well
Feature Extraction
We need three basic components to define a basic convolutional network.

1. The convolutional layer


2. The Pooling layer[optional]
3. The output layer

The weight matrix behaves like a filter in an image extracting particular


information from the original image matrix. A weight combination might be
extracting edges, while another one might a particular color, while another one
might just blur the unwanted noise.
Stride and Padding

● If the weight matrix moves 1 pixel at a time, we call it as a stride of 1.


● size of image keeps on reducing as we increase the stride value.
● Padding the input image with zeros across it solves this problem for us.
● The output from the each filter is stacked together forming the depth
dimension of the convolved image.
● The activation map is the output of the convolution layer.
Pooling Layer

● Pooling is done for the sole purpose of reducing the spatial size of the image.
● It is then desired to periodically introduce pooling layers between subsequent
convolution layers.
● After multiple layers of convolution and padding, we would need the output in
the form of a class.
● The convolution and pooling layers would only be able to extract features and
reduce the number of parameters from the original images.
● To generate the final output we need to apply a fully connected layer to
generate an output equal to the number of classes we need.
Region-Based Convolutional Neural Network

● Instead of working on a massive number of regions, the RCNN algorithm


proposes a bunch of boxes in the image and checks if any of these boxes
contain any object. RCNN uses selective search to extract these boxes from
an image (these boxes are called regions).
● There are basically four regions that form an object: varying scales, colors,
textures, and enclosure.
● Selective search identifies these patterns in the image and based on that,
proposes various regions.
Fast RCNN

● Fast RCNN uses a single model which extracts features from the regions,
divides them into different classes, and returns the boundary boxes for the
identified classes simultaneously.
5. Google’s Self-driving Car
❖ Laser range finder
❖ Front camera for near vision
❖ Aerial that reads precise geo-location
❖ Ultrasonic sensors in the rear wheels
❖ Devices within the car
❖ Synergistic combining of sensors
❖ Programmed to interpret common road signs
❖ Programming real life road behaviour
6. Challenges of self-driving cars

❖ Cyber security and privacy protection


❖ Non-affordable cost
❖ Ethical considerations
❖ Extreme weather conditions
❖ Adaptation to different terrains
❖ Awareness
REFERENCES
Nakul Agarwal, “A study on computer vision techniques for self-driving cars”, International
conference on frontier computing, 19 May 2019

Ajeet Ram Pathak, Manjusha Pandey, Siddharth Rautaray, “Application of Deep Learning for
Object Detection”, International Conference on Computational Intelligence and Data
Science,2018

Dishashree Gupta, “Architecture of Convolutional Neural Networks (CNNs) demystified”,


Analytics Vidhya, June 2017

Pulkit Sharma, “A Step-by-Step Introduction to the Basic Object Detection Algorithms”,


Analytics Vidhya, October 2018

Mike Daily, “Self driving cars” - IEEE Published Computer, volume:50 issue:12, December
2017, Pages 18-23
24

THANK YOU

You might also like