Scribd
Upload
63 views

Ensemble Learning: HARIS JAMIL (FA18-RCS-003) HOOIRA ANAM (FA18-RCS-004)

The document discusses ensemble learning, which is a machine learning technique that combines multiple models to obtain better predictive performance than from any of the constituent models. It can be applied to both regression and classification problems. There are two main steps: generating base models using the same or different algorithms, and then combining their predictions. Common ensemble methods mentioned are voting/averaging, stacking, bagging/bootstrap aggregating, and boosting. Bagging generates models from random samples and combines them, while boosting iteratively adjusts the weights of misclassified examples.

Uploaded by

suppi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
63 views

Ensemble Learning: HARIS JAMIL (FA18-RCS-003) HOOIRA ANAM (FA18-RCS-004)

The document discusses ensemble learning, which is a machine learning technique that combines multiple models to obtain better predictive performance than from any of the constituent models. It can be applied to both regression and classification problems. There are two main steps: generating base models using the same or different algorithms, and then combining their predictions. Common ensemble methods mentioned are voting/averaging, stacking, bagging/bootstrap aggregating, and boosting. Bagging generates models from random samples and combines them, while boosting iteratively adjusts the weights of misclassified examples.

Uploaded by

suppi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

Ensemble Learning

HARIS JAMIL (FA18-RCS-003)


HOOIRA ANAM (FA18-RCS-004)
Ensemble Learning

 Ensemble Learning is a technique that create multiple models and then combine
them them to produce improved results.
 Ensemble learning usually produces more accurate solutions than a single model
would.
Ensemble Learning

 Ensemble learning methods is applied to regression as well as classification.


 Ensemble learning for regression creates multiple repressors i.e. multiple
regression models such as linear, polynomial, etc.
 Ensemble learning for classification creates multiple classifiers i.e. multiple
classification models such as logistic, decision tress, KNN, SVM, etc.
Ensemble Learning

 There are two steps in ensemble learning:


 Multiples machine learning models were generated using same or different machine
learning algorithm. These are called “base models”.
 The prediction perform on the basis of base models.
Ensemble Learning

 Techniques/Methods
 Voting and Averaging
 Stacking
 Bootstrap Aggregating/Bagging
 Boosting
Ensemble Learning Techniques/Methods

 Voting and Averaging


 Voting used in classification
 Averaging used in regression
Ensemble Learning Techniques/Methods

 Voting and Averaging


 Voting used in classification
Ensemble Learning Techniques/Methods

 Voting and Averaging


 Averaging used in Regression
Ensemble Learning Techniques/Methods

 Stacking
 Also known as stacked generalization
 Multiple machine learning models are combined using another (combiner) machine
learning algorithm.
 The basic idea is to train machine learning algorithms with training dataset and then
generate a new dataset with these models. Then this new dataset is used as input for
the combiner machine learning algorithm.
Ensemble Learning Techniques/Methods

 Stacking
Ensemble Learning Techniques/Methods

 Bootstrap Aggregating / Bagging


 Again at first step multiple machine learning models are generated. These models are
generated using the same machine learning algorithm with n random observations of m
sub-samples of the original dataset using bootstrap sampling method.
 The second step is aggregating the result generated from these models. Well known
methods, such as voting and averaging, are used for this purpose.
 For example, Random Forest algorithm uses the bagging technique.
Ensemble Learning Techniques/Methods

 Bootstrap Aggregating / Bagging


 Voting and averaging is used in second step for final prediction
Ensemble Learning Techniques/Methods

 Boosting
 The term “boosting” is used to describe a family of algorithms which are able to convert
weak models to strong models.
 Boosting incrementally builds an ensemble by training each model with the same
dataset but where the weights of instances are adjusted according to the error of the
last prediction
 Adaboost is a widely known algorithm which is a boosting method.
Thank You for listening Us 

Any Question?????

You might also like