An Overview of Machine Learning Methods
Uploaded by
MedichamAn Overview of Machine Learning Methods
Uploaded by
MedichamDeep Learning and Neural Networks -
Mid-report for SoS
Rohit Kaulgud
April 2020
Mentor: Siddhesh Pawar
Contents
Title i
Contents i
1 Introduction 1
2 Supervised Learning 2
2.1 Historical Background . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Supervised Learning Algorithms . . . . . . . . . . . . . . . . . 3
2.3 Example: Donuts and Bagels . . . . . . . . . . . . . . . . . . 3
2.4 Precision and Recall . . . . . . . . . . . . . . . . . . . . . . . 4
3 Unsupervised Learning 5
3.1 K-means Clustering Algorithm . . . . . . . . . . . . . . . . . . 5
4 Updated Plan of Action 6
5 References 6
i
1 Introduction
According to Wikipedia, Machine learning (ML) is the study of computer
algorithms that improve automatically through experience. It is seen as a
subset of artificial intelligence. Machine learning algorithms build a mathe-
matical model based on sample data, known as ”training data”, in order to
make predictions or decisions without being explicitly programmed to do so.
Deep learning is a class of machine learning algorithms that uses multiple
layers to progressively extract higher level features from the raw input. For
example, in image processing, lower layers may identify edges, while higher
layers may identify the concepts relevant to a human such as digits or letters
or faces.
Figure 1: Deep Learning as a subset of AI and ML
The ”deep” deep learning refers to the number of layers through which data
is transformed. Deep learning has exciting applications in the fields of com-
puter vision, speech recognition, natural language processing, audio recog-
nition, social network filtering, machine translation, bio-informatics, drug
design, where it have produced results comparable to and in some cases sur-
passing human expert performance.
Machine learning can be further classified into supervised learning, unsuper-
vised learning and reinforcement learning.
1
2 Supervised Learning
2.1 Historical Background
Supervised learning, as the term suggests is the process in which an entity
shows the machine what to do and provides the target, i.e. each example in
the dataset is associated with a label or target.
In 1958, psychologist Frank Rosenblatt was inspired by the Dartmouth Con-
ference to design a machine that could simulate a human neuron. A human
neuron is a single cell that is made up of a dendrites and an axon, with a
synapse separating the dendrites of one neuron from the axon of the other.
Electrical impulses are passed through these synapses from one neuron which
accumulate in the other neuron, which in turn fires its own impulses upon
reaching a threshold value.
Figure 2: A Human Neuron
Rosenblatt designed a simple computer about the size of a large piano called
the Perceptron, which was equipped with a 400-pixel camera, in order to clas-
sify shapes as triangles or not triangles. Each time an image was held before
the camera, each pixel would send a different electrical signal to the computer,
depending on whether it detected ink or paper. These signals would then
be added up and if they were greater than threshold values, the computer
would send an output signal classifying the shape as a triangle. Initially, the
signal strengths were random and the Perceptron would essentially guess if
the shape was a triangle or not. So to train it with supervision, Rosenblatt
used ‘yes’ and ‘no’ buttons which he would press if the Perceptron’s guesses
were correct or incorrect. If the Perceptron guess was incorrect, the signal
strengths and threshold levels would be adjusted so that its future guesses
would be more accurate.
2
Figure 3: Rosenblatt and his Perceptron
2.2 Supervised Learning Algorithms
In modern supervised learning systems, each neuron receives an input multi-
plied by a certain weight, which corresponds to the strength of each signal.
The threshold is associated with a special weight called bias which can be
adjusted based on the neuron’s eagerness to fire. The inputs multiplied by
their respective weights are then combined, and a mathematical function gets
the results. In the simlest systems, this function can just be a step-function.
Figure 4: A Simple Perceptron
2.3 Example: Donuts and Bagels
Consider a perceptron designed to distinguish between donuts and bagels
based on two properties - weight and diameter. If the two properties are
plotted on a graph, a decision boundary will be determined by the per-
ceptron based on the weights and bias.Each object is then plotted as a point
3
on the graph[(x, y) = (weight, diameter)], and the perceptron determines
whether it is a donut or a bagel depending on which side of the boundary it
falls. Of course, the machine initially makes incorrect guesses, and adjusts
its weights and bias based on the supervisor feedback, which is given by the
update rule.
Perceptron Guess- Donut Bagel
Result when actually Donut Correct Incorrect
Update Rule 0 +n or -n
Table 1: Update Rule
The process is carried out over the entire dataset and the final weights and
bias are fixed.
2.4 Precision and Recall
Consider the above example in which the perceptron has (after training) clas-
sified 100 objects as donuts or bagels.
Classified Donut Classified Bagel Total
Actually Donut 8 17 25
Actually Bagel 2 73 75
Total 10 90 100
Table 2: Summary of classified items
Precision: The percentage of items correctly identified by the perceptron,
i.e. the measure of our trust in the classification. In this case, the precision
is (8+73)/100 = 81%
Recall: The percentage of the desired object correctly identified by the
perceptron. Say donut is our desired item, then the recall in this case will
be 8/25 = 32%
Precision and recall can be especially important in some cases, such as email
spam recognition, in which case a high recall for important messages is nec-
essary, even if it comes at the expense of some spam messages sneaking into
the inbox.
4
3 Unsupervised Learning
Unsupervised learning is a type of machine learning algorithm used to draw
inferences from datasets consisting of input data without labeled responses.
In other words, the world is the teacher!
3.1 K-means Clustering Algorithm
K-means clustering requires a way to compare observations, a way to guess
how many clusters exist in the dataset and a way to calculate the averages
for each cluster.
The algorithm works in two steps:
1. Predict: The perceptron assumes certain averages and classifies the
nearest points into that cluster.
2. Learn: The average is recalculated for all the above cluster points and
the points are reclustered based on the updated average.
Figure 5: K-clusters
5
4 Updated Plan of Action
With roughly six weeks remaining(27th April - 7th June), the following is my
updated plan of action:
Week 1 Machine Learning basics continued
Week 2 Feed-Forward Neural Networks
Week 3 Convolutional Neural Networks
Week 4 Recurrent Neural Networks
Week 5 Autoencoders, Neural Language Models, GANs
Week 6 Practical applications and end report completion
5 References
• Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville.
MIT Press, 2016.
