Scribd
Upload
8 views20 pages

Bias_Variance_Trade_Off

The document discusses the bias-variance trade-off, highlighting the need to balance bias and variance to avoid underfitting and overfitting in models. It explains L1 and L2 regularization techniques, with L1 promoting sparsity and robustness to outliers, while L2 penalizes weights without creating a sparse solution. The document emphasizes the importance of regularization in improving generalization error on test datasets.

Uploaded by

poojaexplore1
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
8 views20 pages

Bias_Variance_Trade_Off

The document discusses the bias-variance trade-off, highlighting the need to balance bias and variance to avoid underfitting and overfitting in models. It explains L1 and L2 regularization techniques, with L1 promoting sparsity and robustness to outliers, while L2 penalizes weights without creating a sparse solution. The document emphasizes the importance of regularization in improving generalization error on test datasets.

Uploaded by

poojaexplore1
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 20

BIAS – VARIANCE TRADE OFF &

REGULARIZATION (L1 and L2)

Dr. V. Sowmya,
Associate
Professor,
Amrita School of AI,
Coimbatore,
23-01-2025.
Bias-variance trade-
off is tension between the

error introduced by the bias and the error

produced by the variance.

------------Required to avoid underfit or


overfit. https://www.kdnuggets.com/2020/09/understanding-bias-variance-trade-off-3-
Errors due to
Bias
The distance between the predictions of a model and
the true values.

The model pays little attention to training data.

and oversimplifies the model and doesn't learn the


patterns.
- learns the wrong relations by not taking in
account

all the features.


https://www.kdnuggets.com/2020/09/understanding-bias-variance-trade-off-3-
Errors due to
Variance
A value that tells us the spread of
our data.

The model pays lot of attention in


training data.

Not flexible to generalize on the data


which it

hasn’t seen before.


https://www.kdnuggets.com/2020/09/understanding-bias-variance-trade-off-3-
Bias — Variance Trade-
Off

https://www.kdnuggets.com/2020/09/understanding-bias-variance-trade-off-3-
Underfitti
ng
Overfittin
g
Regulariza
tion
Any modification or change in the
learning algorithm that helps
reduce its error over a test dataset,
commonly known as generalization
error but not on the supplied or
training dataset.
https://medium.com/analytics-vidhya/regularization-understanding-l1-and-l2-regularization-for-deep-learning-
https://medium.com/analytics-vidhya/regularization-understanding-l1-and-l2-regularization-for-deep-learning-
L1 Parameter
Regularization

https://medium.com/analytics-vidhya/regularization-understanding-l1-and-l2-regularization-for-deep-learning-
L1 Parameter
Regularization
L1 regularization is robust in dealing with
outliers.

It creates sparsity in the solution (most of the


coefficients
of the solution are zero), which means the less
important
features (feature selection) or noise terms will
be zero.

It makes L1 regularization robust to outliers.


https://medium.com/analytics-vidhya/regularization-understanding-l1-and-l2-regularization-for-deep-learning-
L2 Parameter
Regularization

https://medium.com/analytics-vidhya/regularization-understanding-l1-and-l2-regularization-for-deep-learning-
L2 Parameter
Regularization
1. Ridge regularization forces the weights to be
small but does not
make them zero and does not give the sparse
solution.

2. Ridge is not robust to outliers as square terms


blow up the error
differences of the outliers, and the regularization
term tries to fix
it by penalizing the weights.

3. Ridge regression performs better when all the input


features
influence the output, and all with weights are of
https://medium.com/analytics-vidhya/regularization-understanding-l1-and-l2-regularization-for-deep-learning-
https://medium.com/analytics-vidhya/regularization-understanding-l1-and-l2-regularization-for-deep-learning-
https://medium.com/@alejandro.itoaramendia/l1-and-l2-regularization-part-1-a-complete-guide-
https://medium.com/@alejandro.itoaramendia/l1-and-l2-regularization-part-1-a-complete-guide-
https://medium.com/@alejandro.itoaramendia/l1-and-l2-regularization-part-1-a-complete-guide-
https://medium.com/@alejandro.itoaramendia/l1-and-l2-regularization-part-1-a-complete-guide-
https://medium.com/@alejandro.itoaramendia/l1-and-l2-regularization-part-1-a-complete-guide-
https://medium.com/@alejandro.itoaramendia/l1-and-l2-regularization-part-1-a-complete-guide-
https://medium.com/@alejandro.itoaramendia/l1-and-l2-regularization-part-1-a-complete-guide-

You might also like