Gesture Control Robot

1. INTRODUCTION

The word “Robot” was first used in a 1921 play titled R.U.R. Rossum’s Universal Robots, by
Czechoslovakian writer Karel Capek. Robot is a Czech word meaning “worker”. The robot is usually
an electro-mechanical machine that can perform tasks automatically. Some robots require some degree
of guidance, which may be done using a remote control or with a computer interface. Robots can be
autonomous, semi-autonomous or remotely controlled. Robots have evolved so much and are capable
of mimicking humans that they seem to have a mind of their own. Thus finally came out with a
hand gesture recognition automaton, which might follow the commands created by hand gestures.

These days a number of wireless robots are being developed and put to various applications
and uses. In order to enhance the contribution of robot in our daily lives we need to find an effective
way of communicating with robots. For this purpose, there have been certain developments in area of
human -machine interaction. One common form of communication is Gestures that are not only
limited to face, body and fingers but also hand gestures. In order to increase the use of robot in
places where conditions are not certain like rescue operations, robots can be made to follow the
instructions of human operator and perform the task accordingly. A hand Gesture Control Robot is a
kind of robot which is controlled by the hand gestures and not by using buttons.

The robot is equipped with two sections - Transmitting section and Receiving section. In the
Transmitting section, the Accelerometer is mounted on hand of the user capturing its gesture and
moving the robot accordingly. These captured data is in analog output from accelerometer which is
converted into digital by using microcontroller that is ESP32 node MUC that performs encoding of the
data. Later the encoded data is transmitted by the transceiver section of the node MCU. In the receiving
section, ESP32 Cam module receives the transmitted data and decodes it. These decode data then
passed onto a motor driver to rotate the motors in a special configuration to move the robot in
the same direction as that of the hand movement which may be forward, backward, right or left.

The applications of robotics mainly involve in automobiles, medical, construction, defense and also
used as a fire fighting robot to help the people from the fire accident. But controlling the robot
with a remote or a switch is quite complicated.

Dept. of E&CE, BIET, January 2022 Page 1

Davanagere
 Gesture Control Robot

2. LITERATURE SURVEY

1. B. Chaitanya Varma, P. Manikanta P, Venkateswaralu Reddy, K.Venu Gopal Electronics &

Communication Engg. St. Ann’s college of engineering & technology Chirala, India,
presented a paper Gesture Controlled Robot with Wireless Camera Monitoring in the
march 2017, PP 395 - 399.

Gesture controlled robot which can be controlled by human gestures. The

transmitter circuit in which includes sensor accelerometer, the user needs to wear this
device. The sensor will accept the movement of hand in a specific direction which will result
in the motion of the robot in the respective direction. The system design is divided into
two parts namely, Accelerometer part and robot part. The robot part consists microcontroller
unit (MCU), which receives signal from transmitter circuit and sends the motor controlling
signal to the motor driver IC. Then the robot can move forward, backward, leftward &
rightward directions.

2. Nathan David, Chinedu Udengwu, Department of Electronic Engineering, University of

Nigeria, presented a paper A Gesture Controlled System in the November 2015, PP 397 -
405.
There are two main components that are used in the system, an Arduino
microcontroller and an accelerometer. Through accelerometer, a passage of data signal is
received and it is processed with the help of Arduino microcontroller. The microcontroller
gives command to the robot to move in the desired direction. Accelerometer-based gesture
control is studied as a supplementary or an alternative interaction modality. Gesture
commands freely trainable by the user can be used for controlling external devices with
handheld wireless sensor unit. Gesture commands were found to be natural, especially for
commands with spatial association in design environment control. The work consists of the
integration of three technologies as wireless, gesture & embedded.

Dept. of E&CE, BIET, January 2022 Page 2

Davanagere
 Gesture Control Robot

3. Balaji Sivakumar, Pravin kumar, Bhuvaneswari Balachander, presented a paper Design of

Hand Gesture Controlled Robot using Arduino Lilypad in the August 2019, PP 250 -
252.

This paper proposes a style of hand gesture controlled automaton victimization

Arduino Lilypad. The model projected is controlled through a motion device that is
mounted on the hand gloves. This style helps physically challenged folks and additionally
for sure tasks educated by human. the most aim of this style is to manage the automaton
victimization hand gesture. measuring device utilized in the planning senses the direction
of hand movement and sends an indication to Arduino Lilypad. Four main Hand gesture
movements like forward, backward, left and right area unit detected and enforced.

4. Ashutosh zagade, Vishakha Jamkhedkar, Shraddha Dhakane, Vrushali Patankar,

Presented a paper A Study on Gesture Control Arduino Robot, PP 385 - 392.

The user just needs to wear a gesture device in which a sensor is included. The
sensor will record the movement of hand in a specific direction which will result in the
motion of the robot in the respective directions. The robot and the Gesture instrument are
connected wirelessly through radio waves. User can interact with the robot in a more
friendly way due to the wireless communication. We can control the car using
accelerometer sensors connected to a hand glove. The sensors are intended to replace the
remote control that is generally used to run the car. It will allow user to control the forward,
backward, leftward and rightward movements, while using the same accelerometer sensor
to control the throttle of the car. Movement of car is controlled by the differential
mechanism. The mechanism involves the rotation of both forth & rear wheels of left or
right side to move in the anticlockwise direction and the other pair to rotate in the
clockwise direction which makes the car to rotate about its own axis without any kind of
forward or backward motion. The main advantage of this mechanism is the car with this
mechanism can take sharp turn without any difficulty.

Dept. of E&CE, BIET, January 2022 Page 3

Davanagere
 Gesture Control Robot

5. D Arulselvam Duraisamy, Snekha Williams, Department of Electrical and Electronics

Engineering, Sri Sairam Engineering College, Chennai. Presented a paper Robotic Hand
Gesture System with GSM and Camera Control in the April 2020, PP 72 - 78.

This movement can be commanded by simple hand gestures. It consists of an

Accelerometer sensor that controls the wheelchair direction through hand gestures given by
the users. Using Arduino, values from the sensor are monitored and accordingly the direction
changes. Based on the direction of the Acceleration, Arduino controls the wheelchair ways
like Right, Left, Backward, & Forward. Additionally, this system is provided with a camera
module, to access the vehicle from the user’s location. It also provides an emergency alert
system by means of Global system for mobile communication module.

6. Tajim Md. Niamat Ullah Akhund, Jahangirnagar University, Daffodil International

University, presented a paper Designing and Implementation of a Low-cost Wireless
Gesture Controlled Robot for Disable People in the April 2018, PP 1 - 67.

In this paper they designed a low-cost Microcontroller Based Gesture Recognition

system without image processing. This can move a robot in 360 degree. This system is
cheaper than image processing. They have used MPU 6050 Accelerometer Gyroscope sensor
for measuring the gesture of a human hand. Then the collected data will be sent to the robot
via Radio Frequency communication and the robot will move forward, backward, left and
right direction by following the hand gesture .

Dept. of E&CE, BIET, January 2022 Page 4

Davanagere
 Gesture Control Robot

2.1 SUMMARY

Some of the parameters of interest in the papers referred could be summarized as follows:

Table 1: Summary of literature survey

Paper Authors Microcontrolle Transmitter/ Additional Number of

r Receiver Feature Microcontroller
B.Chaitanya Varma, AT89C51 RF Transmitter Wireless
Manikanta P, (Recei and RF Receiver camera 1
Gesture Controlled Robot ver
with Wireless Camera section Monitoring
Monitoring
NathanDavi, Chinedu ATMega328 Transmitter
Udengwu, (Transmitter &Receiver - 1
A Gesture Controlled section)
System
Ashutosh zagade, Vishakha AT mega 328p
Jamkhedkar,
Design of Hand Gesture RF Transmitter - 1
Controlled Robot using and RF Receiver
Arduino Lilypad
Balaji Sivakumar, Pravin
kumar, Arduin RF
A Study on Gesture o Lilypad Transmitter - 2
Control Arduino Robot and RF
Receiver
D Arulselvam Duraisamy, Arduino UNO
Snekha Williams, RF GSM and Camera 1
Robotic Hand Gesture System Transmitter
with GSM and Camera and RF
Control Receiver
Tajim Md. Niamat Ullah
Akhund, Ardiuno Transmitter
Designing and NANO &Receiver - 1
Implementation of a Low-
cost Wireless
Gesture Controlled Robot
for Disable People

Dept. of E&CE, BIET, January 2022 Page 5

Davanagere
 Gesture Control Robot

3. PROBLEM IDENTIFICATION

There is a need to create a much more convenient means of communicating with machines that
would feel more natural to human users. The traditional methods of doing so involved the use of
keyboards, mouse and other controller I/O devices to control machines. These methods had some
bottlenecks of which include the complexity of the controller device and the generic unnaturalness
of the interaction as a whole, and the series of wired connections which limits the range of operation
of the machine by the user. This project aims to demonstrate an easier and efficient way of
interacting with machines.
The traditional wired buttons controlled robot becomes very bulgy and it also limits the distance the
robot goes. The Wireless Hand controlled Robot will function by a wearable hand glove from which
the movements of the hand can be used as the input for the movement of the robot. The basic idea
of our project is to develop a system (Robot) which can recognize the Human Interaction with it to
accomplish the certain tasks assigned to it. In our project we will design a wearable Hand Glove
which will contain the sensors mounted on it to capture the movement of the hand and convert the
raw mechanical data into electrical form. This data will be further processed and converted into an
understandable format for the lilypad mounted on the Glove. This lilypad will act as a transmitter of
the data for wireless communication purpose. Once the transmitted data is received by the receiver
module which will be connected to the Microcontroller, it will be processed and further sent to the
Microcontroller. Microcontroller will deduce the commands and accordingly it will actuate the
motor drivers to control the Motors for various tasks on the robot.

Dept. of E&CE, BIET, January 2022 Page 6

Davanagere
 Gesture Control Robot

4. OBJECTIVES & METHODOLOGY

4.1 OBJECTIVES

The main objectives of using robot are:

• The aim of the project is to develop a human machine interface used for control robot arm.
Our objective is to make this device simple as well as cheap so it can be produced and used
for number of purposes.

• The objective of this project is to build a car that can be controlled by gesture wirelessly.

• In this project user is also able to control motions of the car by wearing controller glove and
performing predefined gestures. This can be also used in many potential applications such
as wireless controller car racing etc.

4.2 METHODOLOGY

➢ Transmitter section: The transmitter units contains MPU6050 accelerometer power supply
and ESP32 module. The accelerometer Detect the angle of tittle and give analog output,
these analog output is converted into digital output ESP32 and its has transceiver functions
so it sends these digital data serially to receiver section wirelessly. The serial data is by the
function of encoder.
➢ Receiver section: The receiver unit contains ESP32 module, motor drivers, DC motor with
four wheels. The transmitted digital data is received by ESP32 module it converts serial
data into motor operated parallel data by the function of decoder then give to motor driver.
The motor driver take low control current signals and then turn it into a high current signal
that can drive a motor and give to DC motor. According to gesture data the wheels attaches
to DC motors are move either forward or backward or left or right.

Dept. of E&CE, BIET, January 2022 Page 7

Davanagere
 Gesture Control Robot

4.3 BLOCK DIAGRAM

Figure 1: Block diagram

Dept. of E&CE, BIET, January 2022 Page 8

Davanagere
 Gesture Control Robot

4.4 FLOW CHART

Figure 2: Transmission Flow Chart

Figure 3: Receiving Flow Chart

Dept. of E&CE, BIET, January 2022 Page 9

Davanagere
 Gesture Control Robot

5. HARDWARE DESCRIPTION

1. ESP32 NODEMCU

ESP32 is a series of low-cost, low-power system on a chip microcontrollers with

integrated Wi-Fi and dual-mode Bluetooth. The ESP32 series employs either
a Tensilica Xtensa LX6 microprocessor in both dual-core and single-core variations,
Xtensa LX7 dual-core microprocessor or a single-core RISC-V microprocessor and
includes built-in antenna switches, RF balun, power amplifier, low-noise receive amplifier,
filters, and power-management modules. ESP32 is created and developed by Espressif
Systems, a Shanghai-based Chinese company, and is manufactured by TSMC using their
40 nm process.[2] It is a successor to the ESP8266 microcontroller.

Features

❖ Processors:
➢ CPU: Xtensa dual-core (or single-core) 32-bit LX6 microprocessor, operating at 160 or
240 MHz and performing at up to 600 DMIPS.
❖ Memory: 320 KB RAM, 448 KiB ROM.
❖ Wireless connectivity:
➢ Wi-Fi: 802.11 b/g/n
➢ Bluetooth: v4.2 BR/EDR and BLE (shares the radio with Wi-Fi)
❖ Peripheral interfaces:
➢ 34 × programmable GPIOs
➢ 12-bit SAR ADC up to 18 channels
➢ 2 × 8-bit DACs
➢ 10 × touch sensors (capacitive sensing GPIOs)
➢ 3 × UART
➢ Motor PWM
➢ LED PWM (up to 16 channels).

Figure 4: ESP32 Node MCU

Dept. of E&CE, BIET, January 2022 Page 10

Davanagere
 Gesture Control Robot

2. ESP32 CAM MODULE

The ESP32-CAM is a small size, low power consumption camera module based on ESP32. It
comes with an OV2640 camera and provides onboard TF card slot. The ESP32-CAM can be widely
used in intelligent IoT applications such as wireless video monitoring, WiFi image upload, QR
identification, and so on.

SPI Flash default 32Mbit

RAM internal520KB+external 4M PSRAM
Bluetooth bluetooth4.2BR/EDR and BLE standards
Wi-Fi 802.11 b/g/n/e/i
Support TF card Maximum support 4G
IO port 9
Image output format JPEG (only supported by OV2640), BMP, GRAYSCALE
Spectrum range 2412 ~ 2484MHz
Antenna form onboard PCB antenna , gain 2dBi
802.11b: 17±2 dBm (@11Mbps)
Transmit power 802.11g: 14±2 dBm (@54Mbps)
802.11n: 13±2 dBm (@MCS7)
CCK, 1 Mbps: -90dBm
6 Mbps (1/2 BPSK): -88dBm
Receiving sensitivity
54 Mbps (3/4 64-QAM): -70dBm
MCS7 (65 Mbps, 72.2 Mbps): -67dBm
Power supply range 5V

Table 2: ESP32 CAM Module Specifications

Figure 5: ESP32 CAM Module

Dept. of E&CE, BIET, January 2022 Page 11

Davanagere
 Gesture Control Robot

3. DC MOTOR

A DC motor is any of a class of rotary electrical motors that converts direct current electrical
energy into mechanical energy. The most common types rely on the forces produced by magnetic
fields. Nearly all types of DC motors have some internal mechanism, either electromechanical or
electronic, to periodically change the direction of current in part of the motor.

DC motors were the first form of motor widely used, as they could be powered from existing direct-
current lighting power distribution systems. A DC motor's speed can be controlled over a wide
range, using either a variable supply voltage or by changing the strength of current in its field
windings. Small DC motors are used in tools, toys, and appliances. The universal motor can operate
on direct current but is a lightweight brushed motor used for portable power tools and appliances.
Larger DC motors are currently used in propulsion of electric vehicles, elevator and hoists, and in
drives for steel rolling mills.

Figure 6: DC Motor

4. MOTOR DRIVER

Motor drivers acts as an interface between the motors and the control circuits.So the function
of motor drivers is to take a low-current control signal and then turn it into a higher-current signal
that can drive a motor.

Motor driver: L298N

Motor channels: 2
Maximum operating voltage: 46 V
Peak output current per channel: 2 A
Minimum logic voltage: 4.5 V

Dept. of E&CE, BIET, January 2022 Page 12

Davanagere
 Gesture Control Robot

Figure 7: Motor Driver

5. ACCELEROMETER MPU6050

MPU6050 has a 3-axis gyroscope, 3- axis Accelerometer and a Digital motion processor integrated
on a single chip. It works on the power supply of 3V-5V. MPU6050 uses the I2C protocol for
communication and transfer of data. This module has a built-in 16-bit ADC which provides great
accuracy.
Technical specifications of the MPU-6050 3-Axis Gyro & Accelerometer module

• Supply voltage: 2.3–3.4 V.

• Consumption: 3.9 mA max.
• Accelerometer: Measuring ranges: ±2 g ±4 g ±8 g ±16 g.
• Gyroscope.
• I2C interface.
• Embedded temperature sensor.
• Selectable jumpers on CLK, FSYNC and AD0.
• Operating temperature: -40 °C to +85 °C.

Figure 8: Accelerometer

Dept. of E&CE, BIET, January 2022 Page 13

Davanagere
 Gesture Control Robot

6. SOFTWARE DESCRIPTION

PROGRAMMING ARDUINO SOFTWARE

ARDUINO IDE Software is used for programming the boards.

INTRODUCTION TO ARDUINO IDE

IDE stands for Integrated Development Environment - official software introduced by

Arduino.cc, that is mainly used for writing, compiling and uploading the code in the Arduino
Device. Almost all Arduino modules are compatible with this software that is an open source and
is readily available to install and start compiling the code on the go.

Arduino IDE is open-source software that is mainly used for writing and compiling the code into
the Arduino Module. It is official Arduino software, making code compilation too easy that even a
common person with no prior technical knowledge can get their feet wet with the learning process.

It is easily available for operating systems like MAC, Windows, Linux and runs on the Java Platform
that comes with inbuilt functions and commands that play a vital role for debugging, editing
and compiling the code in the environment. A range of Arduino modules available including
Arduino Uno, Arduino Mega, Arduino Leonardo, Arduino Micro and many more. The IDE
environment mainly contains two basic parts: Editor and Compiler where former is used for writing
the required code and later is used for compiling and uploading the code into the given Arduino
Module.

Dept. of E&CE, BIET, January 2022 Page 14

Davanagere
 Gesture Control Robot

7. ADVANTAGES
• This mode of interaction with machines is more natural as opposed to the traditional use of
keyboards and mouse since it greatly reduces the complexity of operation.

• Gesture controlling is very helpful for handicapped and physically disabled people to achieve
certain tasks, such as driving a vehicle or moving about in a wheelchair without needing to
manually push the wheels themselves.

• Gestures can be used to control interactions for entertainment purposes such as gaming to
make the game player's experience more interactive or immersive.

• It does not require GUI to be functional which further emphasizes the simplicity of gesture
controls.

• The machine can be remotely controlled from a distance when high RF transceiver modules
are implemented.

8. APPLICATIONS
• Gesture can be used to control interactions for entertainment purposes such as gaming to
make the game player’s experience more interactive or immersive.

• Through the use of gesture recognition, remote control with the wave of a hand of various
devices is possible.

• Industrial application for trolley to control, lift control, etc.

• Military application to control robotics.

• Medical application for surgery purpose.

• Construction application.

Dept. of E&CE, BIET, January 2022 Page 15

Davanagere
 Gesture Control Robot

REFERENCES

[1] Swetha N, Design and Implementation of Accelerometer based Robot motion and
Speed control with Obstacle detection. IJSERT, volume 3, issue 2, pp 749-755, March
2013.
[2] Saikat patra et al., Controlling a robotic car. Electronics for you., volume 5, issue 2 , pp
95-96, October-2016.
[3] Rohan Sheoran et al., A Study of Hand Gesture Technique to Communicate with a
Robot. IJERT, Vol. 4 Issue 01, pp. 12- 13, January-2015.
[4] Bonny Varghese; B. Thilagavathi, Design and Wireless Control of Anthropomorphic
Robotic Arm. 978-1-4799- 6818-3/15/$31.00©2015 IEEE.
[5] Roy Chowdhury,D (2003). Linear Integrated Circuits, 2nd.Ed.New Age
International(p) Ltd, 207-212.
[6] https://electronissome.com/ht12e-encoder-ic-remote-control- systems/
[7] Diksha Goyal; Dr. S.P.S. Saini, Accelerometer Based Hand Gesture Controlled
Wheelchair IJET, volume 4, issue 2, pp 15-20, june-2013.
[8] https://electronissome.com/ht12d-decoder-ic-remote-control- systems/
[9] Ajay V Deshmukh, (2012). Microcontrollers, TATA McGraw Hill publications,
[10] Muhammad Ali Mazidi, Janice Gillsple Mazidi, and Rollin
D. McKiniay; The 8051 microcontroller and embedded systems, 2nd Ed. Pearson
Eductation india,507-512.
[11] http://en.wikipedia.org/wiki/IP_camera
[12] https://www.elprocus.com/simulation-process/
[13] https://www.engineersgarage.com/foruns/8051/how-to- dump-code-at89c51
microcontroller
[14] Rahul Ranjan Singh et al., Wireless Controlled Robot Movement System Designed
Using Microcontroller. IJETT, volume 32, issue 5, pp 204-207, February 2016.
[15] http://www.electronicshub.org/interfacing-dc-motor-8951-microcontroller

Dept. of E&CE, BIET, January 2022 Page 16

Davanagere
 Gesture Control Robot

Dept. of E&CE, BIET, January 2022