Scribd
Upload
4 views

Al Evaluation

Uploaded by

poornimagidagar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
4 views

Al Evaluation

Uploaded by

poornimagidagar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

AI Evaluation

Evaluation
Evaluation refers to systematically checking and analysing the merit,
correct- ness and reliability of an AI model based on the outputs
produced by it.
Evaluation Metrics
Evaluation Metrics refers to the measures used to test the quality of
the Al model.
The Causes behind the Performance of AI Model are:
1.Overfitting
Overfitting refers to a situation when an Al model performs so well as
the test data it got, fitted exactly against its training data and thus AI
model always produced correct result.
2.Underfitting
Underfitting refers to a situation when an Al model is not complex
enough to capture the structure and relationships of its training data
and predict effective outcomes.
3. Generalization
Generalization refers to how well the concepts learned by a machine
learning model apply to specific examples not seen by the model
when it was learning. The goal of a good machine learning model is to
generalize well from the training data to any data from the problem
domain. This allows us to make predictions in the future on data the
model has never seen.
Ideally, an Al model should be balanced between underfitting and
overfitting to be a good fit.
Confusion Matrix
A Confusion Matrix is a technique using a chart or table for
summarizing the performance of a classification-based Al model by
listing the predicted values of an Al model and the actual/correct
outcome values.
A confusion table includes both predictive and actual values in
context of AI model, which are:
◆ the Actual Value represents the actual result (observed or
measured).
True
Actual Values
False

◆ the Predicted Value is the value of the outcome/result of the Al


model, produced on the basis of its algorithm and learning.
Positive
Predicted Values
Negative

Before we proceed to how to create and use confusion matrices, some


terms associated with confusion matrix are:
(i) True Positive (TP). True positive refers to an instance for which
both predicted value of the AI model and actual value are positive.
For example, while testing a patient for Covid, if the test also
produced the result (predicted value) as positive and the actual result
(actual value) is also positive, it is True positive.
(ii) True Negative (TN). True negative refers to an instance for which
both predicted value of the AI model and actual value are negative.
For example, while testing a patient for Covid, if the test also
produced the result (predicted value) as negative and the actual result
(actual value) is also negative, it is True negative.
(iii) False Positive (FP) (also called Type I Error). False positive
refers to an instance for which predicted value of an Al model is
positive but actual value is negative. For example, while testing a
patient for Covid, if the test produced the result (predicted value) as
positive and the actual result (actual value) is negative, it is False
positive.
(iv) False Negative (FN) (also called Type II Error). False negative
refers to an instance for which predicted value of an Al model is
negative but actual value is positive. For example, while testing a
patient for Covid, if the test produced the result (predicted value) as
negative and the actual result (actual value) is positive, it is False
negative.
Note
A Confusion Matrix is a technique using a chart or table (N x N
matrix) for summarizing the performance of a classification-based Al
model by listing the predicted values of an Al model and the
actual/correct outcome values.
Using the confusion matrices, you need to compute the following
values to evaluate an AI model:
◆ Accuracy rate. This is the percentage of times the predictions out
of all the observations are correct.
◆ Precision rate. This is the rate at which the desirable predictions
turn out to be correct (True Positives out of all positives).
◆ Recall. It is a rate of correct positive predictions to the overall
number of positive instances in the dataset.
◆ F1 score. It is a measure of balance between precision and recall.

F1 Score refers to a metric that balances Precision and Recall and


hence balances the impact of False Positives and False negatives.
The metrics Precision, Recall and F1 score range from 0 to 1.
For all the Al models developed, the AI model with the higher F1
score is chosen.

You might also like