Scribd
Upload
1 views11 pages

SDS_fyp

Uploaded by

Awaiz Adnan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
1 views11 pages

SDS_fyp

Uploaded by

Awaiz Adnan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 11

Cutting edge real-time

reinforcement learning model


implementation on
autonomous rover

Date: 11/26/2022

Supervisor: Group Members:


Dr. Farhan Khan Awaiz Adnan 2019095
Jahanzaib Khan Ludin 2019187
Faiz ul Hasan gardezi 2019123
Zain ul furqan shahid 2019554

Co-Supervisor:
Engr. Badre Munir
Project Title

Revision History:

Revision Date Comments


History
1.00 11/23/2022 Initial draft
2.00 11/24/2022 Adding tables and diagrams
3.00 11/26/2022 Finalized version

Document Approval:
The following document has been accepted and approved by the following:

Signature Date Name


Project Title

List of Contents
1. INTRODUCTION......................................................................6
1.1. PURPOSE...........................................................................................6
1.2. PRODUCT SCOPE..............................................................................6
1.3. OVERVIEW........................................................................................6
2. THE OVERALL DESCRIPTION....................................................7
2.1. PRODUCT PERSPECTIVE......................................................................7
3. WORK BREAKDOWN STRUCTURE.............................................7
4. DESIGN..................................................................................8
4.1 ARCHITECTURAL DESIGN..................................................................8
4.2. WHY WE CHOOSE ????? ARCHITECTURE DESIGN?.......................................9
4.3. MODULE IDENTIFICATION.................................................................9
Project Title

List of Figures
Figure 1 Work Breakdown Structure.............................................................7
Figure 4.1.1 Electronic Design.......................................................................8
Figure 4.1.2 Chasis Design
Figure 4.1.3 Cloud Design
Figure 4.3.1 Arduino Nano
Figure 4.3.2 Raspbery Pi Pico
Figure 4.3.3 ESP8266

List of Tables
Table 1 Context
Table..................................................................................................5
Project Title

1. INTRODUCTION
Scientists have been sending unmanned probes and rovers into the solar
system for decades. With a broad number of purposes and targets, these
unmanned vehicles have sent back a stunning array of scientific data.
However, there is a critical issue for unmanned, semi-autonomous
spaceships: navigating unknown and potentially hazardous terrain. Whilst a
human operator can respond to unexpected terrain with their senses, an
unmanned, robotic vehicle cannot rely on human operators, especially at
such vast distances. They must make use of their programming and having
the software to adapt to unknown terrain is essential. The ability to explore
and map previously unknown areas is key for missions using autonomous
vehicles. It can impact time-critical mission parameters (such as fuel
depletion) and, ultimately, can be the difference between the success and
failure of the mission itself. Deep Reinforcement Learning is a field of
computer science that can provide software-based solutions to this
mission-critical challenge.

1.1. PURPOSE
The purpose of this project is to explore reinforcement learning and further
our research into areas where it is still in infancy, we can use this discovery
rover or autonomous rover to discover unknown areas without any prior
training. The model is still being trained in real life and able to
handlesituations a normal deep learning model cannot.

1.2. PRODUCT SCOPE

The scope of this project is to build an autonomous rover that can explore
areas initially in our university premises and encounter any obstacle and be
able to compute an optimal policy in the virtual simulation and take that
decision.

Table 1: Terms used in this document and their description


Name Description
CNN Convolutional Neural Network
RL Reinforcement Learning
Deep Learning Type of Machine Learning
Agent The acting machine (rover)
Q Learning Type of Reinforcement Learning
Rocker Bogie Type of Susupension
Project Title

1.3. OVERVIEW

Area discovery is the feature that our project or rover will be focused
around the basic function will be to find the best path from point A to point
B with the help of a virtually simulated 2D environment that will be mapped
using LIDAR and proximity sensors then using reinforcement learning
algorithms we will find the optimal policy and feed that value back into the
rover, all of these computations will be done on a cloud because the micro-
controllers are constraint devices and can’t handle the load.

2. THE OVERALL DESCRIPTION


The description of the proposed system is given below. This description
defines the product in different directions.

We are building a system whereby the sensor data is collected and a


simulation of the environment is created in real time. Our Reinforcement
learning model will then take this data as input tensors and perform
iterations with our rover as a simulated agent which in turn gives us the
optimal pathway. This pathway is then mapped onto the rover as output.
This model will not need pre-trained data or training, hence can work in
unknown and inhospitable situations. So, to demonstrate our model
capability, we will be building a rover with custom designed electronics and
hardware. Primary sensors are LIDAR and SONAR to work in sync to
generate the simulation of nearby environment. The rover will have
autonomous navigation, obstacle avoidance and geo-locating capabilities

2.1. PRODUCT PERSPECTIVE


The rover when fabricated as a finished product can be used as
autonomous exploration ground vehicle for mapping and navigating
through unknown or previously unexplored territory. The intended end user
for such a product is the scientific community for collecting data in
inaccessible environments such as a planetary rover. Another type of end
user can be search and rescue teams or military. Hence the rover, as a
product, will provide path-finding, navigating, mapping and autonomy
capabilities to the end user.
Project Title

3. WORK BREAKDOWN STRUCTURE


The different stages of the architecture are illustrated in the work
breakdown structure below.

Figure 1 Work Breakdown Structure


Project Title

4. Design
4.1 ARCHITECTURAL DESIGN
The electronic design is illustrated in the schematic given below.

Fig. 4.1.1 Electronic Design

Electronic components and modules:-


- Arduino Nano
- Raspberry Pi Pico
- ESP8266 WiFi
- L293d Motor Driver
- A1M8 LiDAR Sensor

All data and communication lines of each component/module is connected


to the I2C BUS via SCL and SDA pins.
Project Title

The chassis is modeled after Rocker Bogie suspension.

Fig. 4.1.2 Rocker Bogie

The overall cloud (remote server) design can be deduced from the following
flow chart.

Fig. 4.1.3 Cloud Architenture


Project Title

4.2. Why we choose ????? Architecture Design?


This particular architecture allows us to interface multiple sensors and keep
the sensor data processing and collecting in separate modules. We also
have a requirement of real-time training that is fulfilled by cloud connection
via ESP8266.

4.3. MODULE IDENTIFICATION


For Data collection we interface LiDAR and secondary sensors with Arduino
Nano. The data processing part is carried out by Raspberry Pi Pico. For
connectivity to cloud we use ESP8266 WiFi module. A high performance
Linux PC will act as cloud (remote server).

Fig. 4.3.1 Arduino Nano


Project Title

Fig. 4.3.2 Raspberry Pi Pico

Fig. 4.3.3 ESP8266 WiFi Module

You might also like