VISVESVARAYA TECHNOLOGICAL UNIVERSITY

JNANASANGAMA, BELAGAVI - 590018

“ IMAGE RECOGNITION
USING NEURAL NETWORK &
DEEP LEARNING”
Thesis submitted in partial fulfillment of the curriculum prescribed for
the award of the degree of Bachelor of Engineering in
Computer Science & Engineering by
1CR14CS138 Siddharth
1CR14CS163 Vivekanand Vivek
1CR14CS007 Akarsh Ramesh Khatagalli
1CR14CS134 Shreyans Maitrey
Under the Guidance of

Mr. Kartheek GCR

Assistant Professor
Department of CSE, CMRIT, Bengaluru

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

#132, AECS LAYOUT, IT PARK ROAD, BENGALURU - 560037

2017-18
 VISVESVARAYA TECHNOLOGICAL UNIVERSITY
JNANASANGAMA, BELAGAVI - 590018

Certificate
This is to certify that the project entitled “ IMAGE RECOGNITION US-
ING NEURAL NETWORK & DEEP LEARNING” is a bonafide work car-
ried out by Siddharth , Vivekanand Vivek , Akarsh Ramesh Khatagalli
and Shreyans Maitrey in partial fulfillment of the award of the degree of Bache-
lor of Engineering in Computer Science & Engineering of Visvesvaraya Technological
University, Belgaum, during the year 2017-18. It is certified that all corrections / sug-
gestions indicated during reviews have been incorporated in the report. The project
report has been approved as it satisfies the academic requirements in respect of the
project work prescribed for the Bachelor of Engineering Degree.

Signature of Guide Signature of HoD Signature of Principal

Mr. Kartheek GCR Dr. Jhansi Rani P Dr. Sanjay Jain
Assistant Professor Professor & Head Principal
Department of CSE Department of CSE CMRIT,
CMRIT, Bengaluru - 37 CMRIT, Bengaluru - 37 Bengaluru - 37

External Viva

Name of the Examiners Institution Signature with Date

1.

2.
 Acknowledgement
We take this opportunity to thank all of those who have generously
helped us to give a proper shape to our work and complete our BE project
successfully. A successful project is fruitful culmination efforts by many
people, some directly involved and some others indirectly, by providing
support and encouragement.

We would like to thank Dr. SANJAY JAIN , Principal , CMRIT ,

for providing excellent academic environment in the college.

We would like to express our gratitude towards Dr. JHANSI RANI

, Professor & HOD , Dept of CSE , CMRIT , who provided guidance and
gave valuable suggestions regarding the project.

We consider it a privilege and honour to express our sincere gratitude

to our Internal Guide Mr. KARTHEEK GCR , Asst. Professor , De-
partment of Computer Science & Engineering , CMRIT , for his valuable
guidance throughout the tenure of this project work.

Siddharth
Vivekanand Vivek
Akarsh Ramesh Khatagalli
Shreyans Matriey

i
Table of Contents

Table of Contents ii

List of Figures iv

List of Tables v

Abstract vi

1 PREAMBLE 1
1.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 MOTIVATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 FACE DETECTION APPROACHES . . . . . . . . . . . . . . . . . . . 3

2 LITERATURE SURVEY 5
2.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 LITERATURE SURVEY . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 PAPER 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 PAPER 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 PAPER 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 THEORETICAL BACKGROUND 8
3.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 PERCEPTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 SIGMOID NEURONS . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4 THE ARCHITECTURE OF NEURAL NETWORKS . . . . . . . . . . 11
3.5 A SIMPLE NETWORK TO CLASSIFY FACES . . . . . . . . . . . . . 12
3.6 DEEP NEURAL NETWORK . . . . . . . . . . . . . . . . . . . . . . . 13
3.7 CONVOLUTIONAL NEURAL NETWORK . . . . . . . . . . . . . . . 14

4 SYSTEM REQUIREMENT SPECIFICATION 16

4.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.2 FUNCTIONAL REQUIREMENTS . . . . . . . . . . . . . . . . . . . . 18

ii
 4.3 NON-FUNCTIONAL REQUIREMENTS . . . . . . . . . . . . . . . . . 18
4.4 HARDWARE REQUIREMENTS . . . . . . . . . . . . . . . . . . . . . 21
4.5 SOFTWARE REQUIREMENTS . . . . . . . . . . . . . . . . . . . . . 22
4.6 SOFTWARE QUALITY ATTRIBUTES . . . . . . . . . . . . . . . . . 22

5 SYSTEM ANALYSIS 24
5.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.2 FEASIBILITY STUDY . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6 SYSTEM DESIGN 26
6.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
6.2 SYSTEM DEVELOPMENT METHODOLOGY . . . . . . . . . . . . . 27
6.3 DESIGN USING UML . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.4 DATA FLOW DIAGRAM . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.5 USE CASE DIAGRAM . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.6 ACTIVITY DIAGRAM . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.7 SEQUENCE DIAGRAM . . . . . . . . . . . . . . . . . . . . . . . . . . 33

7 IMPLEMENTATION 35
7.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
7.2 TRAINING MODULE CODE . . . . . . . . . . . . . . . . . . . . . . . 36
7.3 IMAGE RECOGNITION CODE . . . . . . . . . . . . . . . . . . . . . 41

8 TESTING AND RESULTS 45

8.1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
8.2 TESTING METHODOLOGIES . . . . . . . . . . . . . . . . . . . . . . 46
8.3 TEST CASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

9 CONCLUSION & FUTURE SCOPE 50

9.1 CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
9.2 FUTURE SCOPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

References 52

iii
List of Figures

3.1 Neurons Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.2 Sigmoid neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.3 Sigmoid Fuction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4 Step Fuction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.5 Nuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.6 Nuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.7 Used Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.8 Typical Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

6.1 Waterfall Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

6.2 Dataflow Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.3 Usecase Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.4 Activity Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.5 Sequence Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

7.1 Training Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

7.2 Training Console Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7.3 Training Data Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

8.1 Input Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

8.2 Recognition Console Input . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
8.3 Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

iv
List of Tables

6.1 Symbols Used In UML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

v
 Abstract
The main idea of the project is to be able to classify product category
by looking only at the image, we think that it make sense to use Deep
Convolutional Neural Networks. To begin with, we are planning to use
transfer learning and fine-tuning on ResNet50, VGG-19, and some other
pre-trained on ImageNet data networks that are accessible in Keras. For
this step we are planning to use only small subset of the whole training
data it should also represent only small subset of the whole classes that
we need to classify data into (50-100 classes out of total 5000 classes).

We are planning to rely on transfer learning, because it seems to be

very time consuming to train the Deep Convolutional Neural Network on
such a large dataset from scratch, we will need tens of training epochs, each
of them should run through at least some big subset of the initial train
set and then update the weights of the network. Each of these epochs will
take few days even on GPU, then to train from scratch we need tens of
days on GPU which we dont have. Thus we decided to stop on transfer
learning which have been reported as a very powerful way to deal with
image classification problems. Because on the top level most object on
images are built from the same lines, and corners we then just need to
specify combinations of these lines and corners and their connection with
the classes in the last layers of the pretrained networks. Next steps would
involve training on a cluster because of the reasons stated in the back-
ground section. For this task some Keras functions would be very useful
to us and would save some time from reimplementing functions for deal-
ing with big training sets of data , because in this project we should be
concerned about RAM usage.

As a final step we are planning to use the hierarchical structure of the

target classification labels. Because we have 3 level tree of categories with
roughly 50 nodes at the first layer, 50-100 children per node at 2nd and
3rd levels we can use it in a way that we first classify the image to one (or
top 5) of the upper level categories, then based on this we can do 2nd level
classification and 3rd (level of interest) classification and the idea is that
because we are fixing some class on the upper level the lower level should
have smaller number of choices and be able to better learn from smaller
amounts of training data.

vi
Chapter 1

PREAMBLE

1
Image Recognition Using Neural Network & Deep Learning Chapter 1

1.1 INTRODUCTION
As the necessity for higher levels of security rises, technology is bound to swell to fulfill
these needs. Any new creation, enterprise, or development should be uncomplicated
and acceptable for end users in order to spread worldwide. This strong demand for
user-friendly systems which can secure our assets and protect our privacy without
losing our identity in a sea of numbers, grabbed the attention and studies of scientists
toward whats called biometrics. There is a more scientific Mathematical Introduction
For Face Recognition: Pixel Arithmetic for readers who are interested in the mathe-
matical perspective and representation of pixels in face recognition applications. The
link also contains a VB.NET implementation of the Pixel class.
Biometrics is the emerging area of bioengineering; it is the automated method of
recognizing person based on a physiological or behavioral characteristic. There exist
several biometric systems such as signature, finger prints, voice, iris, retina, hand
geometry, ear geometry, and face. Among these systems, facial recognition appears
to be one of the most universal, collectable, and accessible systems.
Biometric face recognition is a particularly attractive biometric approach, since it
focuses on the same identifier that humans use primarily to distinguish one person
from another: their faces. One of its main goals is the understanding of the complex
human visual system and the knowledge of how humans represent faces in order to
discriminate different identities with high accuracy.
The face recognition problem can be divided into two main stages: face verification
(or authentication), and face identification (or recognition).

Dept Of CSE, CMRIT, Bengaluru - 560037 2

Image Recognition Using Neural Network & Deep Learning Chapter 1

1.2 MOTIVATION
Face recognition has been a sought after problem of biometrics and it has a variety
of applications in modern life. The problems of face recognition attracts researchers
working in biometrics, patternrecognition eld and computer vision . Several face
recognition algorithms are also used in many dierent applications apart from biomet-
rics , such as video compressions , indexings etc. They can also be used to classify
multimedia content, to allow fast and ecient searching for material that is of interest
to the user. An ecent face recognition system can be of great help in forensic sciences,
identication for law enforcement, surveillance , authentication for banking and secu-
rity system, and giving preferential access to authorised users i.e. access control for
secured areas etc. The problem of face recognition has gained even more importance
after the recent increase in the terrorism related incidents. Use of face recognition
for authentication also reduces the need of remembering passwords and can provide
a much greater security if face recognition is used in combination with other security
measures for access control. The cost of the license for an ecient commercial Face
recognition system ranges from 30,000 $ to 150,000 $ which shows the signicant value
of t he problem. Though face recognition is considered to be a very crucial authenti-
cation system but even after two decades continuous research and evolution of many
face recognition algorithms , a truely robust and ecient system that can produce good
results in realtime and normal conditions is still not available. The Face Recognition
Vendor Test (FRVT) that has been conducted by the National Institute of Standards
and Technology (NIST), USA, has shown that the commercial face recognition sys-
tems do not perform well under the normal daily conditions. Some of the latest face
recognition algorithm involving machine learning tools perform well but sadly the
training period and processing time is large enough to limit its use in practical ap-
plications. Hence there is a continuous strife to propose an eective face recognition
system with high accuracy and acceptable processing time.

1.3 FACE DETECTION APPROACHES

Some of the main face detection methods are discussed here. 1) Knowledge based
methods are developed on the rules derived from the researchers knowledge of human
faces. Problem in this approach is the diculty in translating human knowledge into
welldened rules. 2) Featured-based methods: Invariant features of faces are used
for detecting texture, skin color. But features from such algorithm can be severely
corrupted due to illumination, noise and occlusion. 3) Template matching: Input
image is compared with predened face template. But the performance here suers
due to variations in scale, pose and shape. 4) Appearance-based method: In template
Dept Of CSE, CMRIT, Bengaluru - 560037 3
Image Recognition Using Neural Network & Deep Learning Chapter 1

matching methods, the templates are predened by experts. Whereas, the templates in
appearance based methods are learned from examples in images. Statistical analysis
and machine learning techniques can be used to nd the relevant characteristics of face
and non-face images.

Dept Of CSE, CMRIT, Bengaluru - 560037 4

Chapter 2

LITERATURE SURVEY

5
Image Recognition Using Neural Network & Deep Learning Chapter 2

2.1 INTRODUCTION
Literature survey is mainly carried out in order to analyze the background of the
current project which helps to find out flaws in the existing system and guides on which
unsolved problems we can work out. So, the following topics not only illustrate the
background of the project but also uncover the problems and flaws which motivated
to propose solutions and work on this project.

2.2 LITERATURE SURVEY

Literature survey is the documentation of a comprehensive review of the published
and unpublished work from secondary sources data in the areas of specific interest to
the researcher. The library is a rich storage base for secondary data and researchers
used to spend several weeks and sometimes months going through books, journals,
newspapers, magazines, conference proceedings, doctoral dissertations, master’s the-
ses, government publications and financial reports to find information on their research
topic. Reviewing the literature on the topic area at this time helps the research er to
focus further interviews more meaningfully on certain aspects found to be important
is the published studies even if these had not surfaced during the earlier question-
ing .So the literature survey is important for gathering the secondary data for the
research which might be proved very helpful in the research. The literature survey
can be conducted for several reasons. The literature review can be in any area of the
business.

Dept Of CSE, CMRIT, Bengaluru - 560037 6

Image Recognition Using Neural Network & Deep Learning Chapter 2

2.3 PAPER 1
In FFNN, the neurons are connected in a directed way having clear start and stop
place i.e., the input layer and the output layer. The layer between these two layers,
are called as the hidden layers. Learning occurs through adjustment of weights and
the aim is to try and minimize error between the output obtained from the output
layer and the input that goes into the input layer. The weights are adjusted by pro-
cess of back propagation (in which the partial derivative of the error with respect to
last layer of weights is calculated). The process of weight adjustment is repeated in a
recursive manner until weight layer connected to input layer is updated.

2.4 PAPER 2
An Artificial Neural Network (ANN) is an information processing paradigm that is
inspired by the way biological nervous systems, such asthe brain, process information.
The key element of this paradigm is the novel structure of the information processing
system. It is composed of a large number of highly interconnected processing elements
(neurons) working in unison to solve specific problems. ANNs, like people, learn by
example. An ANN is configured for a specific application, such as pattern recognition
or data classification, through a learning process. Learning in biological systems in-
volves adjustments to the synaptic connections that exist between the neurons. This
is true of ANNs as well.

2.5 PAPER 3
We propose a deep convolutional neural network architecture codenamed Inception
that achieves the new state of the art for classification and detection in the ImageNet
Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of
this architecture is the improved utilization of the computing resources inside the
network. By a carefully crafted design, we increased the depth and width of the
network while keeping the computational budget constant. To optimize quality, the
architectural decisions were based on the Hebbian principle and the intuition of multi-
scale processing. One particular incarnation used in our submission for ILSVRC14 is
called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the
context of classification and detection.

Dept Of CSE, CMRIT, Bengaluru - 560037 7

Chapter 3

THEORETICAL BACKGROUND

8
Image Recognition Using Neural Network & Deep Learning Chapter 3

3.1 INTRODUCTION
The process of image classification involves two steps, training of the system followed
by testing. The training process means, to take the characteristic properties of the
images and form a unique description for a particular class. The process is done for
all classes depending on the type of classification problem; binary classification or
multi-class classification. The testing step means to categorize the test images under
various classes for which system was trained. This assigning of class is done based
on the partitioning between classes based on the training features. Since 2006, deep
structured learning, or more commonly called deep learning or hierarchical learning,
has emerged as a new area of machine learning research . Several definitions are
available for Deep Learning; coating one of the many definitions from Deep Learning
is defined as: A class of machine learning techniques that exploit many layers of
nonlinear information processing for supervised or unsupervised feature extraction
and transformation and for pattern analysis and classification. This work aims at
the application of Convolutional Neural Network or CNN for image classification.
The image data used for testing the algorithm includes remote sensing data of aerial
images and scene data from SUN database. The rest of the paper is organized as
follows. Section 2 deals with the working of the network followed by section 2.1 with
theoretical background. The working of CNN gives the experimental procedure in
detail.

3.2 PERCEPTION
Perceptrons were developed in the 1950s and 1960s by the scientist Frank Rosenblatt,
inspired by earlier work by Warren McCulloch and Walter Pitts. Today, it’s more
common to use other models of artificial neurons - in this book, and in much modern
work on neural networks, the main neuron model used is one called the sigmoid neuron.
We’ll get to sigmoid neurons shortly. But to understand why sigmoid neurons are
defined the way they are, it’s worth taking the time to first understand perceptrons.
So how do perceptrons work? A perceptron takes several binary inputs, x1,x2,, and
produces a single binary output:

Figure 3.1: Neurons Output

Dept Of CSE, CMRIT, Bengaluru - 560037 9

Image Recognition Using Neural Network & Deep Learning Chapter 3

3.3 SIGMOID NEURONS

Suppose we have a network of perceptrons that we’d like to use to learn to solve some
problem. For example, the inputs to the network might be the raw pixel data from
a dataset, image of a face. And we’d like the network to learn weights and biases so
that the output from the network correctly classifies the Face. To see how learning
might work, suppose we make a small change in some weight (or bias) in the network.
What we’d like is for this small change in weight to cause only a small corresponding
change in the output from the network. As we’ll see in a moment, this property will
make learning possible. Schematically, here’s what we want :

Figure 3.2: Sigmoid neurons

In fact, the exact form of isn’t so important - what really matters is the shape of the
function when plotted. Here’s the shape:

Figure 3.3: Sigmoid Fuction

This shape is a smoothed out version of a step function:

Dept Of CSE, CMRIT, Bengaluru - 560037 10

Image Recognition Using Neural Network & Deep Learning Chapter 3

Figure 3.4: Step Fuction

3.4 THE ARCHITECTURE OF NEURAL

NETWORKS
In preparation for that, it helps to explain some terminology that lets us name different
parts of a network. Suppose we have the network:

Figure 3.5: Nuron

As mentioned earlier, the leftmost layer in this network is called the input layer, and
the neurons within the layer are called input neurons. The rightmost or output layer
contains the output neurons, or, as in this case, a single output neuron. The middle
layer is called a hidden layer, since the neurons in this layer are neither inputs nor
outputs. The term ”hidden” perhaps sounds a little mysterious - the first time I heard
the term I thought it must have some deep philosophical or mathematical significance
- but it really means nothing more than ”not an input or an output”. The network
above has just a single hidden layer, but some networks have multiple hidden layers.
For example, the following four-layer network has two hidden layers:

Dept Of CSE, CMRIT, Bengaluru - 560037 11

Image Recognition Using Neural Network & Deep Learning Chapter 3

Figure 3.6: Nuron

3.5 A SIMPLE NETWORK TO CLASSIFY

FACES
We’ll focus on writing a program to solve the problem, that is, classifying individual
Faces. We do this because it turns out that the segmentation problem is not so
difficult to solve, once you have a good way of classifying Faces. There are many
approaches to solving the segmentation problem. One approach is to trial many
different ways of segmenting the image, using the individual Face classifier to score
each trial segmentation. A trial segmentation gets a high score if the individual Face
classifier is confident of its classification in all segments, and a low score if the classifier
is having a lot of trouble in one or more segments. The idea is that if the classifier is
having trouble somewhere, then it’s probably having trouble because the segmentation
has been chosen incorrectly. This idea and other variations can be used to solve the
segmentation problem quite well. So instead of worrying about segmentation we’ll
concentrate on developing a neural network which can solve the more interesting and
difficult problem, namely, recognizing individual Faces. To recognize individual Faces
we will use a three-layer neural network:

Dept Of CSE, CMRIT, Bengaluru - 560037 12

Image Recognition Using Neural Network & Deep Learning Chapter 3

Figure 3.7: Used Network

3.6 DEEP NEURAL NETWORK

Deep Neural Networks are distinguished from the more commonplace single-hidden-
layer neural networks by their depth; that is, the number of node layers through which
data passes in a multistep process of pattern recognition.
Earlier versions of neural networks such as the first perceptrons were shallow,
composed of one input and one output layer, and at most one hidden layer in between.
More than three layers (including input and output) qualifies as deep learning. So
deep is a strictly defined, technical term that means more than one hidden layer.
In deep neural networks, each layer of nodes trains on a distinct set of features
based on the previous layers output. The further you advance into the neural net,
the more complex the features your nodes can recognize, since they aggregate and
recombine features from the previous layer.
This is known as feature hierarchy, and it is a hierarchy of increasing complexity
and abstraction. It makes deep-learning networks capable of handling very large,
high-dimensional data sets with billions of parameters that pass through nonlinear
functions.
Above all, these nets are capable of discovering latent structures within unlabeled,
unstructured data, which is the vast majority of data in the world. Another word
for unstructured data is raw media; i.e. pictures, texts, video and audio recordings.
Therefore, one of the problems deep learning solves best is in processing and clustering
the worlds raw, unlabeled media, discerning similarities and anomalies in data that
no human has organized in a relational database or ever put a name to.

Dept Of CSE, CMRIT, Bengaluru - 560037 13

Image Recognition Using Neural Network & Deep Learning Chapter 3

3.7 CONVOLUTIONAL NEURAL NETWORK

Computational models of neural networks have been around for a long time, first model
proposed was by McCulloch and Pitts. Neural networks are made up of a number
of layers with each layer connected to the other layers forming the network. A feed-
forward neural network or FFNN can be thought of in terms of neural activation and
the strength of the connections between each pair of neurons.
In FFNN, the neurons are connected in a directed way having clear start and stop
place i.e., the input layer and the output layer. The layer between these two layers,
are called as the hidden layers. Learning occurs through adjustment of weights and
the aim is to try and minimize error between the output obtained from the output
layer and the input that goes into the input layer. The weights are adjusted by process
of back propagation.
The process of weight adjustment is repeated in a recursive manner until weight
layer connected to input layer is updated.

Figure 3.8: Typical Neural Network

Dept Of CSE, CMRIT, Bengaluru - 560037 14

Image Recognition Using Neural Network & Deep Learning Chapter 3

3.7.1 WHY ARE WE USING CONVOLUTIONAL

NEURAL NETWORK ?
Convolutional Neural Networks (CNN) is variants of MultiLayer Perceptron which
are inspired from biology. These filters are local in input space and are thus better
suited to exploit the strong spatially local correlation present in natural images. Con-
volutional neural networks are designed to process two dimensional image . A CNN
architecture used in this project is that defined in . The network consists of three
types of layers namely convolution layer, sub sampling layer and the output layer.

Dept Of CSE, CMRIT, Bengaluru - 560037 15

Chapter 4

SYSTEM REQUIREMENT
SPECIFICATION

16
Image Recognition Using Neural Network & Deep Learning Chapter 4

4.1 INTRODUCTION
This chapter describes about the requirements. It specifies the hardware and software
requirements that are required in order to run the application properly. The Software
Requirement Specification (SRS) is explained in detail, which includes overview of
dissertation as well as the functional and non-functional requirement of this disserta-
tion.
A SRS document describes all data, functional and behavioral requirements of the
software under production or development. SRS is a fundamental document, which
forms the foundation of the software development process. Its the complete descrip-
tion of the behavior of a system to be developed. It not only lists the requirements
of a system but also has a description of its major feature. Requirement Analysis
in system engineering and software engineering encompasses those tasks that go into
determining the need or conditions to meet for a new or altered product, taking ac-
count of the possibly conflicting requirements of the various stakeholders, such as
beneficiaries or users. Requirement Analysis is critical to the success to a develop-
ment project. Requirement must be documented, measurable, testable, related to in
identified business needs or opportunities, and defined to a level of detail sufficient for
system design.
The SRS functions as a blueprint for completing a project. The SRS is often
referred to as the ”parent” document because all subsequent project management
documents, such as design specifications, statements of work, software architecture
specification, testing and validation plans, and documentation plans, are related to it.
It is important to note that an SRS contains functional and non-functional require-
ments only.
Thus the goal of preparing the SRS document is to

• To facilitate communication between the customer, analyst, system developers,

maintainers.

• To serve as a contrast between purchaser and supplier.

• To firm foundation for the design phase.

• Support system testing facilities.

• Support project management and control.

• Controlling the evolution of the system.

Dept Of CSE, CMRIT, Bengaluru - 560037 17

Image Recognition Using Neural Network & Deep Learning Chapter 4

4.2 FUNCTIONAL REQUIREMENTS

Functional Requirement defines a function of a software system and how the system
must behave when presented with specific inputs or conditions. These may include
calculations, data manipulation and processing and other specific functionality. In
this system following are the functional requirements:-

• Input test case must not have compilation and runtime errors.

• The application must not stop working when kept running for even a long time.

• The application must function as expected for every set of test cases provided.

• The application should generate the output for given input test case and input
parameters.

• The application should generate on-demand services.

4.3 NON-FUNCTIONAL REQUIREMENTS

Non-functional requirements are the requirements which are not directly concerned
with the specific function delivered by the system. They specify the criteria that
can be used to judge the operation of a system rather than specific behaviors. They
may relate to emergent system properties such as reliability, response time and store
occupancy. Non-functional requirements arise through the user needs, because of
budget constraints, organizational policies, the need for interoperability with other
software and hardware systems or because of external factors such as:-

• Product Requirements

• Organizational Requirements

• User Requirements

• Basic Operational Requirements

In systems engineering and requirements engineering, a non-functional requirement

is a requirement that specifies criteria that can be used to judge the operation of a
system, rather than specific behaviours. This should be contrasted with functional
requirements that define specific behaviour or functions. The plan for implementing
non-functional requirements is detailed in the system architecture. Broadly, functional
requirements define what a system is supposed to do and non- functional requirements
define how a system is supposed to be. Functional requirements are usually in the

Dept Of CSE, CMRIT, Bengaluru - 560037 18

Image Recognition Using Neural Network & Deep Learning Chapter 4

form of system shall do requirement, an individual action of part of the system,

perhaps explicitly in the sense of a mathematical function, a black box description
input, output, process and control functional model or IPO Model. In contrast, non-
functional requirements are in the form of system shall be requirement, an overall
property of the system as a whole or of a particular aspect and not a specific function.
The systems’ overall properties commonly mark the difference between whether the
development project has succeeded or failed. Non-functional requirements of our
project include:

• Response time The time the system takes to load and the time for responses
on any action the user does.

• Processing time - How long is acceptable to perform key functions or export /

import data?

• Throughput The number of transactions the system needs to handle must be

kept in mind.

• Storage - The amount of data to be stored for the system to function.

• Growth Requirements As the system grows it will need more storage space to
keep up with the efficiency.

• Locations of operation - Geographic location, connection requirements and the

restrictions of a local network prevail.

• Architectural Standards The standards needed for the system to work and
sustain.

4.3.1 PRODUCT REQUIREMENTS

• Portability: Since the SLR system is designed to run using Python (whose
library is written in C), the system is portable.

• Correctness: It follows a well-defined set of procedures and rules to compute

and also rigorous testing is performed to confirm the correctness of the data.

• Ease of Use: The front end is designed in such a way that it provides an interface
which allows the user to interact in an easy manner.

• Modularity: The complete product is broken up into many modules and well-
defined interfaces are developed to explore the benefit of flexibility of the prod-
uct.

Dept Of CSE, CMRIT, Bengaluru - 560037 19

Image Recognition Using Neural Network & Deep Learning Chapter 4

• Robustness: This software is being developed in such a way that the overall
performance is optimized and the user can expect the results within a limited
time with utmost relevancy and correctness.

whereas evolution quality involves testability, maintainability, extensibility or scal-

ability.

4.3.1.1 ORGANIZATIONAL REQUIREMENTS

Process Standards: IEEE standards are used to develop the application which is the
standard used by the most of the standard software developers all over the world.
Design Methods: Design is one of the important stages in the software engineering
process. This stage is the first step in moving from problem to the solution domain.
In other words, starting with what is needed design takes us to work how to satisfy
the needs.

4.3.1.2 USER REQUIREMENTS

The user requirements document (URD) or user requirements specification is a doc-

ument usually used to software engineering that specifies the requirements the user
expects from software to be constructed in a software project. Once the required infor-
mation is completely gathered it is documented in a URD, which is meant to spell out
exactly what the software must do and becomes part of the contractual agreement. A
customer cannot demand feature not in the URD, whilst the developer cannot claim
the product is ready if it does not meet an item of the URD. The URD can be used as
a guide to planning cost, timetables, milestones, testing etc. The explicit nature of the
URD allows customers to show it to various stakeholders to make sure all necessary
features are described. Formulating a URD requires negotiation to determine what is
technically and economically feasible. Preparing a URD is one of those skills that lies
between a science and economically feasible. Preparing a URD is one of those skills
that lies between a science and an art, requiring both software technical skills and
interpersonal skills.

4.3.1.3 BASIC OPERATIONAL REQUIREMENTS

Operational requirement is the process of linking strategic goals and objectives to

tactic goals and objectives. It describes milestones, conditions for success and explains
how, or what portion of, a strategic plan will be put into operation during a given
operational period, in the case of, a strategic plan will be put into operation during
a given operational period, in the case of commercial application, a fiscal year or
another given budgetary term. An operational plan is the basis for, and justification
Dept Of CSE, CMRIT, Bengaluru - 560037 20
Image Recognition Using Neural Network & Deep Learning Chapter 4

of an annual operating budget request. Therefore, a five-year strategic plan would

typically require five operational plans funded by five operating budgets. Operational
plans should establish the activities and budgets for each part of the organization for
the next 1-3 years. They link the strategic plan with the activities the organization
will deliver and the resources required to deliver them. An operational plan draws
directly from agency and program strategic plans to describe agency and program
missions and goals, program objectives, and program activities. Like a strategic plan,
an operational plan addresses four questions:

• Where are we now?

• Where do we want to be?

• How do we get there?

The customers are those that perform the eight primary functions of systems engi-
neering, with special emphasis on the operator as the key customer. Operational
requirements will define the basic need and, at a minimum, will be related to these
following points:

• Mission profile or scenario: It describes about the procedures used to accomplish

mission objective. It also finds out the effectiveness or efficiency of the system.

• Performance and related parameters: It points out the critical system parame-
ters to accomplish the mission

• Utilization environments: It gives a brief outline of system usage. Finds out

appropriate environments for effective system operation.

• Operational life cycle: It defines the system lifetime

4.4 HARDWARE REQUIREMENTS

System Requirement for training the Model:

• Operating System : Linux or MacOS

• RAM : 16GB Minimum.

• Internal storage : 500 MB.

• GPU : Nvidia

• VRAM : 6GB

Dept Of CSE, CMRIT, Bengaluru - 560037 21

Image Recognition Using Neural Network & Deep Learning Chapter 4

System Requirement for Recognition using Trained Model:

• Operating System : Linux or MacOS

• RAM : 8GB Minimum.

• Internal storage : 500 MB.

• GPU : IntelHD, Nvidia, AMD

• VRAM : 1.5GB

4.5 SOFTWARE REQUIREMENTS

System Requirement for training the Model:

• Operating System : Linux or MacOS

• Coding Language : Python 2.7

• Tools : PyCharm

• Library : DLib, Keras, Open CV, numpy, Tinker.

System Requirement for Recognition using Trained Model:

• Operating System : Linux or MacOS

• Coding Language : Python 2.7

• Tools : PyCharm

• Library : Keras,Open CV,numpy,Tinker.

4.6 SOFTWARE QUALITY ATTRIBUTES

• Functionality: the capability of the software to provide functions which meet
stated and implied needs when the software is used under specified conditions.

• Reliability: the capability of the software to maintain its level of performance

when used under specified conditions.

• Usability: the capability of the software to be understood, learned, used and

liked by the user, when used under specified conditions.

• Efficiency: the capability of the software to provide the required performance,

relative to the amount of resources used, under stated conditions.
Dept Of CSE, CMRIT, Bengaluru - 560037 22
Image Recognition Using Neural Network & Deep Learning Chapter 4

• Maintainability: the capability of the software to be modified. Modifications

may include corrections, improvements or adaptation of the software to changes
in environment, and in requirements and functional specifications.

• Portability: the capability of software to be transferred from one environment

to another.

Dept Of CSE, CMRIT, Bengaluru - 560037 23

Chapter 5

SYSTEM ANALYSIS

24
Image Recognition Using Neural Network & Deep Learning Chapter 5

5.1 INTRODUCTION
Design is a meaningful engineering representation of something that is to be built.
It is the most crucial phase in the developments of a system. Software design is a
process through which the requirements are translated into a representation of soft-
ware. Design is a place where design is fostered in software Engineering. Based on the
user requirements and the detailed analysis of the existing system, the new system
must be designed. This is the phase of system designing. Design is the perfect way
to accurately translate a customers requirement in the finished software product. De-
sign creates a representation or model, provides details about software data structure,
architecture, interfaces and components that are necessary to implement a system.
The logical system design arrived at as a result of systems analysis is converted into
physical system design.

5.2 FEASIBILITY STUDY

The feasibility of the project is analyzed in this phase and business proposal is put
forth with a very general plan for the project and some cost estimates. During system
analysis the feasibility study of the proposed system is to be carried out. This is
to ensure that the proposed system is not a burden to the company. For feasibility
analysis, some understanding of the major requirements for the system is essential.
Three key considerations involved in the feasibility analysis are

• Operational Feasibility

• Economical Feasibility

• Technical Feasibility

• Social Feasibility

Dept Of CSE, CMRIT, Bengaluru - 560037 25

Chapter 6

SYSTEM DESIGN

26
Image Recognition Using Neural Network & Deep Learning Chapter 6

6.1 INTRODUCTION
Design is a meaningful engineering representation of something that is to be built.
It is the most crucial phase in the developments of a system. Software design is a
process through which the requirements are translated into a representation of soft-
ware. Design is a place where design is fostered in software Engineering. Based on the
user requirements and the detailed analysis of the existing system, the new system
must be designed. This is the phase of system designing. Design is the perfect way
to accurately translate a customers requirement in the finished software product. De-
sign creates a representation or model, provides details about software data structure,
architecture, interfaces and components that are necessary to implement a system.
The logical system design arrived at as a result of systems analysis is converted into
physical system design.

6.2 SYSTEM DEVELOPMENT

METHODOLOGY
System development method is a process through which a product will get completed
or a product gets rid from any problem. Software development process is described as
a number of phases, procedures and steps that gives the complete software. It follows
series of steps which is used for product progress. The development method followed
in this project is waterfall model.

6.2.1 MODEL PHASES

The waterfall model is a sequential software development process, in which progress
is seen as flowing steadily downwards (like a waterfall) through the phases of Require-
ment initiation, Analysis, Design, Implementation, Testing and maintenance.

• Requirement Analysis: This phase is concerned about collection of require-

ment of the system. This process involves generating document and requirement
review.

• System Design: Keeping the requirements in mind the system specifications

are translated in to a software representation. In this phase the designer em-
phasizes on:-algorithm, data structure, software architecture etc.

• Coding: In this phase programmer starts his coding in order to give a full
sketch of product. In other words system specifications are only converted in to
machine readable compute code.

Dept Of CSE, CMRIT, Bengaluru - 560037 27

Image Recognition Using Neural Network & Deep Learning Chapter 6

• Implementation: The implementation phase involves the actual coding or

programming of the software. The output of this phase is typically the library,
executables, user manuals and additional software documentation

• Testing: In this phase all programs (models) are integrated and tested to en-
sure that the complete system meets the software requirements. The testing is
concerned with verification and validation.

• Maintenance: The maintenance phase is the longest phase in which the soft-
ware is updated to fulfill the changing customer need, adapt to accommodate
change in the external environment, correct errors and oversights previously
undetected in the testing phase, enhance the efficiency of the software.

6.2.2 REASON FOR CHOOSING WATERFALL MODEL

AS DEVELOPMENT METHOD
• Clear project objectives.

• Stable project requirements.

• Progress of system is measurable.

• Strict sign-off requirements.

• Helps you to be perfect.

• Logic of software development is clearly understood.

• Production of a formal specification

• Better resource allocation.

• Improves quality. The emphasis on requirements and design before writing a

single line of code ensures minimal wastage of time and effort and reduces the
risk of schedule slippage.

• Less human resources required as once one phase is finished those people can
start working on to the next phase.

Dept Of CSE, CMRIT, Bengaluru - 560037 28

Image Recognition Using Neural Network & Deep Learning Chapter 6

Figure 6.1: Waterfall Model

6.3 DESIGN USING UML

Designing UML diagram specifies, how the process within the system communicates
along with how the objects with in the process collaborate using both static as well as
dynamic UML diagrams since in this ever-changing world of Object Oriented applica-
tion development, it has been getting harder and harder to develop and manage high
quality applications in reasonable amount of time. As a result of this challenge and
the need for a universal object modeling language every one could use, the Unified
Modeling Language (UML) is the Information industries version of blue print. It is a
method for describing the systems architecture in detail. Easier to build or maintains
system, and to ensure that the system will hold up to the requirement changes.

Dept Of CSE, CMRIT, Bengaluru - 560037 29

Image Recognition Using Neural Network & Deep Learning Chapter 6

Table 6.1: Symbols Used In UML

6.4 DATA FLOW DIAGRAM

A data flow diagram (DFD) is graphic representation of the ”flow” of data through an
information system. A data flow diagram can also be used for the visualization of data
processing (structured design). It is common practice for a designer to draw a context-
level DFD first which shows the interaction between the system and outside entities.
DFDs show the flow of data from external entities into the system, how the data
moves from one process to another, as well as its logical storage. There are only four
symbols: 1. Squares representing external entities, which are sources and destinations
of information entering and leaving the system. 2. Rounded rectangles representing
processes, in other methodologies, may be called ’Activities’, ’Actions’, ’Procedures’,
’Subsystems’ etc. which take data as input, do processing to it, and output it. 3.
Arrows representing the data flows, which can either, be electronic data or physical
items. It is impossible for data to flow from data store to data store except via a
process, and external entities are not allowed to access data stores directly. 4. The
flat three-sided rectangle is representing data stores should both receive information
for storing and provide it for further processing.

Dept Of CSE, CMRIT, Bengaluru - 560037 30

Image Recognition Using Neural Network & Deep Learning Chapter 6

Figure 6.2: Dataflow Diagram

6.5 USE CASE DIAGRAM

A use case defines a goal-oriented set of interactions between external entities and the
system under consideration. The external entities which interact with the system are
its actors. A set of use cases describe the complete functionality of the system at a
particular level of detail and it can be graphically denoted by the use case diagram.

Dept Of CSE, CMRIT, Bengaluru - 560037 31

Image Recognition Using Neural Network & Deep Learning Chapter 6

Figure 6.3: Usecase Diagram

6.6 ACTIVITY DIAGRAM

An activity diagram shows the sequence of steps that make up a complex process. An
activity is shown as a round box containing the name of the operation. An outgoing
solid arrow attached to the end of the activity symbol indicates a transition triggered
by the completion.
Activity diagram is another important diagram in UML to describe the dynamic
aspects of the system.
Activity is a particular operation of the system. Activity diagrams are not only used
for visualizing the dynamic nature of a system, but they are also used to construct
the executable system by using forward and reverse engineering techniques. The only
missing thing in the activity diagram is the message part.
It does not show any message flow from one activity to another. Activity diagram is
sometimes considered as the flowchart. Although the diagrams look like a flowchart,
they are not. It shows different flows such as parallel, branched, concurrent, and
single.
Dept Of CSE, CMRIT, Bengaluru - 560037 32
Image Recognition Using Neural Network & Deep Learning Chapter 6

Figure 6.4: Activity Diagram

6.7 SEQUENCE DIAGRAM

Sequence diagram are an easy and intuitive way of describing the behavior of a system
by viewing the interaction between the system and the environment. A sequence
diagram shows an interaction arranged in a time sequence. A sequence diagram
has two dimensions: vertical dimension represents time, the horizontal dimension
represents the objects existence during the interaction.

Dept Of CSE, CMRIT, Bengaluru - 560037 33

Image Recognition Using Neural Network & Deep Learning Chapter 6

Figure 6.5: Sequence Diagram

Basic elements:

• Vertical rectangle: Represent the object is active (method is being per-

formed).

• Vertical dashed line: Represent the life of the object.

• X: represent the life end of an object. (Being destroyed from memory)

• Horizontal line with arrows: Messages from one object to another.

Dept Of CSE, CMRIT, Bengaluru - 560037 34

Chapter 7

IMPLEMENTATION

35
Image Recognition Using Neural Network & Deep Learning Chapter 7

7.1 INTRODUCTION
The implementation phase of the project is where the detailed design is actually
transformed into working code. Aim of the phase is to translate the design into a
best possible solution in a suitable programming language. This chapter covers the
implementation aspects of the project, giving details of the programming language
and development environment used. It also gives an overview of the core modules
of the project with their step by step flow. The implementation stage requires the
following tasks:

• Careful planning.

• Investigation of system and constraints.

• Design of methods to achieve the changeover.

• Evaluation of the changeover method.

• Correct decisions regarding selection of the platform.

• Appropriate selection of the language for application development.

7.2 TRAINING MODULE CODE

import math
from s k l e a r n import n e i g h b o r s
import os
import os . path
import p i c k l e
from PIL import Image , ImageDraw
import f a c e r e c o g n i t i o n
from f a c e r e c o g n i t i o n . f a c e r e c o g n i t i o n c l i
import i m a g e f i l e s i n f o l d e r

ALLOWED EXTENSIONS = { ’ png ’ , ’ j p g ’ , ’ j p e g ’ }

def t r a i n ( t r a i n d i r , m o d e l s a v e p a t h=None , n n e i g h b o r s=None ,

k n n a l g o= ’ b a l l t r e e ’ , v e r b o s e=F a l s e ) :
X = []
y = []
for c l a s s d i r in os . l i s t d i r ( t r a i n d i r ) :

Dept Of CSE, CMRIT, Bengaluru - 560037 36

Image Recognition Using Neural Network & Deep Learning Chapter 7

i f not os . path . i s d i r ( os . path . j o i n

( train dir , class dir )):
continue

for img path in i m a g e f i l e s i n f o l d e r

( os . path . j o i n ( t r a i n d i r , c l a s s d i r ) ) :
image = f a c e r e c o g n i t i o n . l o a d i m a g e f i l e ( img path )
face bounding boxes = face recognition . f a c e l o c a t i o n s
( image )
i f len ( f a c e b o u n d i n g b o x e s ) != 1 :
i f verbose :
print ( ” Image {} not s u i t a b l e f o r t r a i n i n g : {} ” .
format ( img path , ”Didn ’ t f i n d a f a c e ” i f len
( f a c e b o u n d i n g b o x e s ) < 1 e l s e ”Found more
than one f a c e ” ) )
else :
X. append ( f a c e r e c o g n i t i o n . f a c e e n c o d i n g s
( image , k n o w n f a c e l o c a t i o n s=f a c e b o u n d i n g b o x e s ) [ 0 ] )
y . append ( c l a s s d i r )

i f n n e i g h b o r s i s None :
n n e i g h b o r s = int ( round ( math . s q r t ( len (X) ) ) )
i f verbose :
print ( ” Chose n n e i g h b o r s a u t o m a t i c a l l y : ” , n n e i g h b o r s )

k n n c l f = n e i g h b o r s . K N e i g h b o r s C l a s s i f i e r ( n n e i g h b o r s=n n e i g h b o r s ,
a l g o r i t h m=knn alg o , w e i g h t s= ’ d i s t a n c e ’ )
k n n c l f . f i t (X, y )

i f m o d e l s a v e p a t h i s not None :
with open ( m o d e l s a v e p a t h , ’wb ’ ) as f :
p i c k l e . dump( k n n c l f , f )

return k n n c l f

def p r e d i c t ( X img path , k n n c l f=None ,

model path=None , d i s t a n c e t h r e s h o l d = 0 . 6 ) :
Dept Of CSE, CMRIT, Bengaluru - 560037 37
Image Recognition Using Neural Network & Deep Learning Chapter 7

i f not os . path . i s f i l e ( X img path ) or

os . path . s p l i t e x t ( X img path )
[ 1 ] [ 1 : ] not in ALLOWED EXTENSIONS:

r a i s e Ex cep ti on ( ” I n v a l i d image path : {} ”

. format ( X img path ) )

i f k n n c l f i s None and model path i s None :

r a i s e Ex cep ti on ( ”Must s u p p l y c l a s s i f i e r e i t h e r
thourgh k n n c l f o r model path ” )

i f k n n c l f i s None :
with open ( model path , ’ rb ’ ) as f :
knn clf = p i c k l e . load ( f )

X img = f a c e r e c o g n i t i o n . l o a d i m a g e f i l e ( X img path )

X f a c e l o c a t i o n s = f a c e r e c o g n i t i o n . f a c e l o c a t i o n s ( X img )

i f len ( X f a c e l o c a t i o n s ) == 0 :
return [ ]

faces encodings = face recognition . face encodings

( X img , k n o w n f a c e l o c a t i o n s=X f a c e l o c a t i o n s )

c l o s e s t d i s t a n c e s = knn clf . kneighbors ( faces encodings ,

n n e i g h b o r s =1)
a r e m a t c h e s = [ c l o s e s t d i s t a n c e s [ 0 ] [ i ] [ 0 ] <=
d i s t a n c e t h r e s h o l d for i in range ( len
( X face locations ))]

return [ ( pred , l o c ) i f r e c e l s e ( ”unknown” , l o c ) for pred ,

l o c , r e c in zip ( k n n c l f . p r e d i c t ( f a c e s e n c o d i n g s )
, X fa ce lo cati ons , are matches ) ]

def s h o w p r e d i c t i o n l a b e l s o n i m a g e ( img path , p r e d i c t i o n s ) :

p i l i m a g e = Image . open ( img path ) . c o n v e r t ( ”RGB” )
draw = ImageDraw . Draw ( p i l i m a g e )
Dept Of CSE, CMRIT, Bengaluru - 560037 38
Image Recognition Using Neural Network & Deep Learning Chapter 7

for name , ( top , r i g h t , bottom , l e f t ) in p r e d i c t i o n s :

draw . r e c t a n g l e ( ( ( l e f t , top ) , ( r i g h t , bottom ) )
, o u t l i n e =(0 , 0 , 2 5 5 ) )

name = name . encode ( ”UTF−8” )

t e x t w i d t h , t e x t h e i g h t = draw . t e x t s i z e ( name )
draw . r e c t a n g l e ( ( ( l e f t , bottom − t e x t h e i g h t − 1 0)
, ( r i g h t , bottom ) )
, f i l l =(0 , 0 , 2 5 5 ) , o u t l i n e =(0 , 0 , 2 5 5 ) )
draw . t e x t ( ( l e f t + 6 , bottom − t e x t h e i g h t − 5 ) ,
name , f i l l =(255 , 2 5 5 , 2 5 5 , 2 5 5 ) )
del draw

p i l i m a g e . show ( )

if name == ” m a i n ” :
print ( ” T r a i n i n g c l a s s i f i e r . . . ” )
c l a s s i f i e r = t r a i n ( ” t r a i n ” , m o d e l s a v e p a t h=
” t r a i n e d m o d e l . dat ” , n n e i g h b o r s =2)
print ( ” T r a i n i n g complete ! ” )

for i m a g e f i l e in os . l i s t d i r ( ” t e s t ” ) :
f u l l f i l e p a t h = os . path . j o i n ( ” t e s t ” , i m a g e f i l e )

print ( ” Looking f o r f a c e s i n {} ” . format ( i m a g e f i l e ) )

p r e d i c t i o n s = p r e d i c t ( f u l l f i l e p a t h , model path=
” t r a i n e d m o d e l . dat ” )

for name , ( top , r i g h t , bottom , l e f t ) in p r e d i c t i o n s :

print ( ”− Found {} a t ( { } , {}) ” . format
( name , l e f t , top ) )

s h o w p r e d i c t i o n l a b e l s o n i m a g e ( os . path . j o i n
(” test ” , image file ) , predictions )

Dept Of CSE, CMRIT, Bengaluru - 560037 39

Image Recognition Using Neural Network & Deep Learning Chapter 7

7.2.1 TRAINING MODULE SET,INPUT AND OUTPUT

SCREENSSHOTS

Figure 7.1: Training Data Set

Figure 7.2: Training Console Input

Dept Of CSE, CMRIT, Bengaluru - 560037 40

Image Recognition Using Neural Network & Deep Learning Chapter 7

Figure 7.3: Training Data Output

7.3 IMAGE RECOGNITION CODE

import f a c e r e c o g n i t i o n as f a c e r e c o g n i t i o n
from PIL import Image , ImageTk
import T k i n t e r
import cv2
import c o n s t a n t s as CONSTANTS
import s y s

videoCApture = cv2 . VideoCapture ( 0 )

r e t , image = videoCApture . r ead ( )

print ” Enter F i l e NAme o f Known People ( S i n g l e Line S e p e r a t e d

By Spaces ) ”
knownImages = raw input ( ) . s p l i t ( ’ ’ )
print ” Enter O r i g i n a l NAme OF Unknown P e o p l e s ( S i n g l e Line
S e p e r a t e d By Spaces ) ”
knownNames =raw input ( ) . s p l i t ( ’ ’ )

knownImagesPreEncoded = [ ]

Dept Of CSE, CMRIT, Bengaluru - 560037 41

 Image Recognition Using Neural Network & Deep Learning Chapter 7

print knownNames , ” And ” , knownImages

#Create Encoding For A l l The Known Images

for knownImage in knownImages :
tempImageBinary = f a c e r e c o g n i t i o n . l o a d i m a g e f i l e ( knownImage )
image2Learn =( f a c e r e c o g n i t i o n . f a c e e n c o d i n g s ( tempImageBinary ) )
i f ( len ( image2Learn ) ! = 1 ) :
i f image2Learn == 0 :
print ( ” Unable To R e c o g o n i z e Image ” , knownImage ,
” COntains ” , len ( image2Learn ) , ”FAces” )
exit ()
else :
print ( ”Make Sure There i s J u s t One Face ” , knownImage ,
” COntains ” , len ( image2Learn ) , ”FAces” )
exit ()
knownImagesPreEncoded . append ( ( f a c e r e c o g n i t i o n . f a c e e n c o d i n g s
( tempImageBinary ) ) [ 0 ] )

del knownImages

def updateImage ( r o o t , canvas ) :

try :
#Get t h e Image From VIdeo Camera
r e t , frame = videoCApture . read ( )
#S c a l e I t DOwn For Speed
s m a l l f r a m e = cv2 . r e s i z e ( frame , ( 0 , 0 ) ,
f x=CONSTANTS. SCALING X ,
f y=CONSTANTS. SCALING Y)
#s e t t h i s Image as MAin IMage
image = cv2 . c v t C o l o r ( s m a l l f r a m e ,
cv2 .COLOR BGR2RGB)

#i d e n t i f y Image
f a c e l o c a t i o n s = f a c e r e c o g n i t i o n . f a c e l o c a t i o n s ( image )

face encodings = face recognition . face encodings

( image , f a c e l o c a t i o n s )

Dept Of CSE, CMRIT, Bengaluru - 560037 42

Image Recognition Using Neural Network & Deep Learning Chapter 7

tempReco = [ ]

for i in range ( len ( f a c e e n c o d i n g s ) ) :

peopleBools = f a c e r e c o g n i t i o n . compare faces
( knownImagesPreEncoded , f a c e e n c o d i n g s [ i ] ,
CONSTANTS.COMPARISON TOLERENCE)
tempReco . append ( ”Unknown” )
print p e o p l e B o o l s
for j in range ( len ( p e o p l e B o o l s ) ) :
i f p e o p l e B o o l s [ j ] and tempReco [ i ]==”Unknown” :
tempReco [ i ]=knownNames [ j ]
e l i f peopleBools [ j ] :
tempReco [ i ]=tempReco [ i ] + ” OR ” + knownNames [ j ]

for f s e t in range ( len ( f a c e l o c a t i o n s ) ) :

image [ f a c e l o c a t i o n s [ f s e t ] [ 0 ] ] [ min
( face locations [ fset ][1] ,
f a c e l o c a t i o n s [ f s e t ] [ 3 ] ) : max( f a c e l o c a t i o n s [ f s e t ] [ 1 ] ,
face locations [ fset ][3])]=
CONSTANTS.BOUNDING BOX COLOR
image [ f a c e l o c a t i o n s [ f s e t ] [ 2 ] ] [ min( f a c e l o c a t i o n s [ f s e t ] [ 1 ] ,
face locations [ fset ] [ 3 ] ) :
max( f a c e l o c a t i o n s [ f s e t ] [ 1 ] ,
face locations [ fset ][3])]=
CONSTANTS.BOUNDING BOX COLOR
for i in range (min( f a c e l o c a t i o n s [ f s e t ] [ 0 ]
, face locations [ fset ][2])
,max( f a c e l o c a t i o n s [ f s e t ] [ 0 ] , f a c e l o c a t i o n s [ f s e t ] [ 2 ] ) ) :
image [ i ] [ f a c e l o c a t i o n s [ f s e t ] [ 1 ] ]
=CONSTANTS.BOUNDING BOX COLOR
image [ i ] [ f a c e l o c a t i o n s [ f s e t ] [ 3 ] ]
=CONSTANTS.BOUNDING BOX COLOR
im = Image . f r o m a r r a y ( image , ’RGB ’ )
canvas . image = ImageTk . PhotoImage ( im )
canvas . c r e a t e i m a g e ( 0 , 0 , image=canvas . image , anchor= ’nw ’ )
for f s e t in range ( len ( f a c e l o c a t i o n s ) ) :
canvas . c r e a t e t e x t ( ( f a c e l o c a t i o n s [ f s e t ] [ 1 ] +
face locations [ fset ][3])/2
Dept Of CSE, CMRIT, Bengaluru - 560037 43
Image Recognition Using Neural Network & Deep Learning Chapter 7

,min( f a c e l o c a t i o n s [ f s e t ] [ 0 ] , f a c e l o c a t i o n s [ f s e t ] [ 2 ] )
, f i l l =CONSTANTS.LABEL TEXT COLOR
, f o n t=CONSTANTS. LABEL FONT, t e x t=tempReco [ f s e t ] )

canvas . pack ( )

print ( ” Updating . . . ” )
except I n d e x E r r o r :
print ” E r r o r : ” , s y s . e x c i n f o ( )
canvas . a f t e r ( 1 , updateImage , r o o t , canvas )

#Create A WIndow
top = T k i n t e r . Tk ( )
#Add CAnvas To Window To D I s p l a y RGB Image
canvas = T k i n t e r . Canvas ( top , h e i g h t=image . shape [ 0 ] ∗
CONSTANTS. SCALING X
, width=image . shape [ 1 ] ∗CONSTANTS. SCALING Y)
#Update CAnvas As Fast As P o s s i b l e
updateImage ( top , canvas )
#L I s t e n For Evwnts
top . mainloop ( )

#R e l e a s e Video Permission
videoCApture . r e l e a s e ( )

Dept Of CSE, CMRIT, Bengaluru - 560037 44

Chapter 8

TESTING AND RESULTS

45
Image Recognition Using Neural Network & Deep Learning Chapter 8

8.1 INTRODUCTION
Testing is an important phase in the development life cycle of the product this was
the phase where the error remaining from all the phases was detected. Hence testing
performs a very critical role for quality assurance and ensuring the reliability of the
software. Once the implementation is done, a test plan should be developed and run
on a given set of test data. Each test has a different purpose, all work to verify that all
the system elements have been properly integrated and perform allocated functions.
The testing process is actually carried out to make sure that the product exactly does
the same thing what is suppose to do. Testing is the final verification and validation
activity within the organization itself. In the testing stage following goals are tried to
achieve:-

• To affirm the quality of the project.

• To find and eliminate any residual errors from previous stages.

• To validate the software as the solution to the original problem.

• To provide operational reliability of the system.

During testing the major activities are concentrated on the examination and modifi-
cation of the source code. The test cases executed for this project are listed below.
Description of the test case, steps to be followed; expected result, status and screen-
shots are explained with each of the test cases.

8.2 TESTING METHODOLOGIES

There are many different types of testing methods or techniques used as part of the
software testing methodology. Some of the important types of testing are:

8.2.1 WHITE BOX TESTING

White Box Testing is a testing in which in which the software tester has knowledge
of the inner workings, structure and language of the software, or at least its purpose.
It is purpose. It is used to test areas that cannot be reached from a black box level.
Using white box testing we can derive test cases that:

• Guarantee that all independent paths within a module have been exercised at
least once.

• Exercise all logical decisions on their true and false sides.

Dept Of CSE, CMRIT, Bengaluru - 560037 46

Image Recognition Using Neural Network & Deep Learning Chapter 8

• Execute all loops at their boundaries and within their operational bounds.

• Execute internal data structure to assure their validity.

8.2.2 BLACK BOX TESTING

Black Box Testing is testing the software without any knowledge of the inner workings,
structure or language of the module being tested. Black box tests, as most other kinds
of tests, must be written from a definitive source document, such as specification or
requirements document, such as specification or requirements document. It is a testing
in which the software under test is treated, as a black box .you cannot see into it. The
test provides inputs and responds to outputs without considering how the software
works. It uncovers a different class of errors in the following categories:

• Incorrect or missing function.

• Performance errors.

• Initialization and termination errors.

• Errors in objects.

8.2.3 UNIT TESTING

Unit testing is usually conducted as part of a combined code and unit test phase of
the software lifecycle, although it is not uncommon for coding and unit testing to be
conducted as two distinct phases. Test strategy and approach
Field testing will be performed manually and functional tests will be written in detail.

8.3 TEST CASE

Dept Of CSE, CMRIT, Bengaluru - 560037 47

Image Recognition Using Neural Network & Deep Learning Chapter 8

Figure 8.1: Input Images

Figure 8.2: Recognition Console Input

Dept Of CSE, CMRIT, Bengaluru - 560037 48

Image Recognition Using Neural Network & Deep Learning Chapter 8

Figure 8.3: Output

Dept Of CSE, CMRIT, Bengaluru - 560037 49

Chapter 9

CONCLUSION & FUTURE

SCOPE

50
Image Recognition Using Neural Network & Deep Learning Chapter 9

9.1 CONCLUSION
This project was the first attempt to develop a system of this nature. We idntified from
the beginning that producing a complete result would be impossible within the given
time frame. We viewed the project as a journey where we learnt many lessons and
gained insights to the subject which we tried to share in this report and summarised
in this chapter. We tried to look at the problem from many points of view which
generated some new ideas that could be explored in future. We suggested formal
approaches for modelling and analysing the system which are by no means complete
but could become the initiation for further research. We also created a working system
and algorithms which we claim to be useful and extensible. However, as we have seen
in these chapter, all these achievements are only partialy successful.
Personally, we would consider this project a success if the ideas described in the
report can beme a useful reference for future work on the subject.

9.2 FUTURE SCOPE

This Project might, in the future, allow:

• robots that can see, feel, and predict the world around them

• improved stock prediction

• common usage of self-driving cars

• composition of music

• handwritten documents to be automatically transformed into formatted word

processing documents

• trends found in the human genome to aid in the understanding of the data
compiled by the Human Genome Project

• self-diagnosis of medical problems using neural networks

• and much more!

Dept Of CSE, CMRIT, Bengaluru - 560037 51

References

[1] Li Deng and Dong Yu ,Deep Learning: methods and applications, by Microsoft
research

[2] Lillesand, T.M. and Kiefer, R.W. and Chipman, J.W., in Remote Sensing and
Image Interpretation

[3] About deep learning http://research.microsoft.com/pubs/209355

[4] About Neural Network http://www.uow.edu.au/œphung

[5] A back-propagation neural network based method for post life expectancy esti-
mation of thoracic surgery patients Abhishek Kumar Pandey; Mohd Anas Khan;
Aleena Swetapadma 2017 International Conference On Smart Technologies For
Smart Nation .

52