Scribd
Upload
11 views

Ch 7 - notes evaluation

Chapter 7 discusses the importance of evaluating AI models to ensure their reliability and effectiveness in predicting outcomes. It outlines various evaluation metrics such as accuracy, precision, recall, and the F1 score, emphasizing the need to balance precision and recall for optimal model performance. The chapter also highlights common reasons for AI model inefficiencies, including lack of training data and inefficient coding.

Uploaded by

dhanashriam06
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
11 views

Ch 7 - notes evaluation

Chapter 7 discusses the importance of evaluating AI models to ensure their reliability and effectiveness in predicting outcomes. It outlines various evaluation metrics such as accuracy, precision, recall, and the F1 score, emphasizing the need to balance precision and recall for optimal model performance. The chapter also highlights common reasons for AI model inefficiencies, including lack of training data and inefficient coding.

Uploaded by

dhanashriam06
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

Ch 7- Evaluation

Introduction
In the Evaluation stage, we will explore different methods of evaluating an AI model. Model
Evaluation is an integral part of the model development process. It helps to find the best model
that represents our data and how well the chosen model will work in the future. Evaluation is
the process of understanding the reliability of any AI model, based on outputs by feeding test
dataset into the model and comparing with actual answers.

Importance of Evaluation –
 Following are the some of the advantages of evaluating a model:
 Evaluation ensures that the model is operating correctly and optimally.
 Evaluation is an initiative to understand how well it achieves its goals.
 Evaluation helps to determine what works well and what could be improved in a
program

Reasons for inefficiency of AI Model –


Lack of training data – This could be due to less data available for developing an AI model
Unauthenticated / wrong data – If the data is not unauthenticated and correct due to
negligence or data collected from unauthorized resources then the model will not give good
results.
Inefficient coding – If the written algorithms are not correct and relevant, AI model will not
give desired output.
Not Tested- If the model is not tested properly, then it will not be efficient.

Terminologies of Model Evaluation – Evaluation of AI model can be done using


various terminologies. Let us understand this with an example
Imagine that you have come up with an AI based prediction model which has been deployed
in a forest which is prone to forest fires. Now, the objective of the model is to predict whether
a forest fire has broken out in the forest or not. Now, to understand the efficiency of this model,
we need to check if the predictions which it makes are correct or not. Thus, there exist two
conditions which we need to ponder upon: Prediction and Reality. The prediction is the output
which is given by the machine and the reality is the real scenario in the forest when the
prediction has been made. Now let us look at various combinations that we can have with these
two conditions.
True Positive - a forest fire has broken out in the forest. The model predicts a Yes which means
there is a forest fire. The Prediction matches with the Reality. Hence, this condition is termed
as True Positive.
True Negative- Here there is no fire in the forest hence the reality is No. In this case, the
machine too has predicted it correctly as a No. Therefore, this condition is termed as True
Negative.
False Positive - Here the reality is that there is no forest fire. But the machine has incorrectly
predicted that there is a forest fire. This case is termed as False Positive.
False Negative. - Here, a forest fire has broken out in the forest because of which the Reality
is Yes but the machine has incorrectly predicted it as a No which means the machine predicts
that there is no Forest Fire. Therefore, this case becomes False Negative.
Confusion matrix
The result of comparison between the prediction and reality can be recorded in what we call
the confusion matrix. The confusion matrix allows us to understand the prediction results. Note
that it is not an evaluation metric but a record which can help in evaluation.

Prediction and Reality can be easily mapped together with the help of this confusion matrix.
Evaluation Methods
Accuracy-
Accuracy is defined as the percentage of correct predictions out of all the observations.
A prediction can be said to be correct if it matches the reality. Here, we have two conditions in
which the Prediction matches with the Reality: True Positive and True Negative.

Precision
Precision is defined as the percentage of true positive cases versus all the cases where the
prediction is true. That is, it takes into account the True Positives and False Positives.
Recall
It can be defined as the fraction of positive cases that are correctly identified. it considers True
Positives (There was a forest fire in reality and the model predicted a forest fire) and False
Negatives (There was a forest fire and the model didn’t predict it)

Which Metric is Important


we must say that if we want to know if our model’s performance is good, we need these two
measures: Recall and Precision. For some cases, you might have a High Precision but Low
Recall or Low Precision but High Recall. But since both the measures are important, there is a
need of a parameter which takes both Precision and Recall into account.
F1 Score- F1 score can be defined as the measure of balance between precision and recall.

An ideal situation would be when we have a value of 1 (that is 100%) for both Precision and
Recall. In that case, the F1 score would also be an ideal 1 (100%). It is known as the perfect
value for F1 Score.

You might also like