Scribd
Upload
19 views

Evaluation Metrics

The document discusses evaluation methods in machine learning, focusing on the confusion matrix, accuracy, precision, recall, and the F₁ score. It explains how the confusion matrix summarizes classification performance and highlights the limitations of accuracy as a metric in skewed datasets. Additionally, it describes the relationship between precision and recall, and how the F₁ score serves as a balanced measure of both metrics.

Uploaded by

qubefexe
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
19 views

Evaluation Metrics

The document discusses evaluation methods in machine learning, focusing on the confusion matrix, accuracy, precision, recall, and the F₁ score. It explains how the confusion matrix summarizes classification performance and highlights the limitations of accuracy as a metric in skewed datasets. Additionally, it describes the relationship between precision and recall, and how the F₁ score serves as a balanced measure of both metrics.

Uploaded by

qubefexe
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 20

EVALUATION

METHODS
IN MACHIENE LEARNING
AGENDA

01 02 03 04
What is the What is What is the What is an F₁
Confusion Accuracy, need for a score in
matrix in Precision and Confusion machine
machine Recall? matrix? learning?
learning?
WHY WE USE
CONFUSION MATRIX ?
ACCURACY
ACCURACY

 Accuracy is the number of times you predicted something correctly divided by


how many times you predicted it.

 Classification accuracy alone can be misleading if you have an unequal number


of observations in each class or if you have more than two classes in your
dataset.

 Hence, Accuracy is not a suitable metric to use in situations where the data is
skewed, so in these situations, we use a confusion matrix.
WHAT IS THE
CONFUSION
MATRIX ?
CONFUSION MATRIX

 This is a confusion matrix, it shows you all the possible scenarios of the predictions of a
model vs the ground truth.
 It is a technique for summarizing the performance of a classification algorithm.
Let's co-relate this to a spam classifier dataset.
Case 1: Our model predicts a message as spam and actually it is
spam (TP - True Positive)
Case 2: Our model predicts a message as spam but in reality, it
is not (FP - False Positive)
Case 3: Our model predicts a message as not spam, but in
reality, it is spam (FN - False Negative)
Case 4: Our model predicts a message as not spam, and
actually it is not spam(TN - True Negative)
WHAT IS THE NEED FOR A
CONFUSION MATRIX?
WHAT IS PRECISION AND RECALL ?
PRECISION = Out of all the positive classes we have predicted
correctly, how many are actually positive or how good is our
model at not making false accusations?
RECALL: Out of all the predictions, how many are actually
correct?. This metric basically tells us how good our model is at
identifying relevant samples.
Recall and precision are related such that high
precision leads to low recall, and low precision leads
to high recall.
By making sure that the False Negative and False
positive are almost the same we can say that the
train and test data are balanced and having high
precision and recall.
We can calculate this by using a confusion matrix.
WHAT IS AN F₁ SCORE IN MACHINE LEARNING?
The F₁ score is the harmonic mean of precision and recall. The harmonic mean is a
special type of mean(average) which is explained by this formula.

Now if we optimize our model to have an F₁ score, we can have high precision and
recall.

You might also like