3.3.

1 Decision Tree
A decision tree is a simple representation for classifying examples. Decision tree learning is
one of the most successful techniques for supervised classification learning.
A decision tree or a classification tree is a tree in which each internal (non-leaf) node is
labeled with an input feature. The arcs coming from a node labeled with a feature are
labeled with each of the possible values of the feature. Each leaf of the tree is labeled with a
class or a probability distribution over the classes.
" A decision tree is a simple representation for classifying examples. A decision tree or a
classification tree is a tree in which each internal node is labeled with an input feature. The
arcs coming from a node labeled with a feature are labeled with each of the possible values
of the feature. Each leaf of the tree labeled with a class or a probability distribution over
the classes.
" In this method a set of training examples is broken down into smaller and smaller subsets
while at the same time an associated decision tree get incrementally developed. At the end
of the learning process, a decision tree covering the training set is returned.
The key idea is to use a decision tree to partition the data space into cluster (or dense)
regions and empty (or sparse) regions.
Decision tree consists of
1. Nodes : test for the value of acertain attribute
2. Edges :correspond to the outcome ofa test and connect to the next node or leaf
3. Leaves : terminal nodes that predict the outcome

Training

New example Classification

Fig. 3.3.1
" In decision tree learning, a new example is classified by submitting it to a series of tests that
determine the class label of the example. These tests are organized in a hierarchical
structure called a decision tree.
" Decision tree has three other names :

1. Classification tree analysis is a term used when the predicted outcome is the class to
which the data belongs.
2. Regression tree analysis is a term used when the predicted outcome can be considered a
real number (e.g. the price of a house,or a patient's length of stay in a hospital).
Internal node denotes a test on an attribute. Branch represents an outcome of the test. Leaf
nodes represent class labels or class distribution.
" A decision tree is a flow-chart-like tree structure, where each node denotes a test on an
attribute value, each branch represents an outcome of the test, and tree leaves represent
classes or class distributions. Decision trees can easily be converted to classification rules.
> Decision Tree Algorithm
To generate decision tree from the training tuples of data partition D.
> Input :
1. Data partition ( D)
2. Attribute list

3. Attribute selection method

> Algorithm :
1. Create a node (N)
2. If tuples in D are all of the same class then
3. Return node (N) as a leaf node labeled with the class C.
4. If attribute list is empty then return Nas a leaf node labeled with the majority class in D
S. Apply Attribute selection method(D, attribute list) to find the "best" splitting criterion;
6. Label node N with splitting criterion;
7. If splitting attribute is discrete-valued and multiway splits allowed
8. Then attribute list -> attribute list -> splitting attribute
9. For (each outcome j of splitting criterion )
10. Let D,be the set of data tuples in Dsatisfying outcome j;
 11. IfD, is empty then attach a leaf labeledwith the majority class in Dto node N;
12. Else attach the node returned by Generate decision tree(Dj, attribute list) to node N;
13. End of for loop
14. Return N;
" CART analysis is a term used to refer to both of the above procedures. The name CART is
an acronym from the words Classification And Regression Trees, and was first introduced
by Breiman et al.
 3.3.2 Advantages and Disadvantages of Decision Trees
> Advantages :
1. Decision trees can handle both nominal and numeric input attributes.
2. Decision tree representation is rich enough to represent any discrete value classifier.
3. Decision trees are capable of handling datasets that may have errors.
4. Decision trees are capable of handling datasets that may have missing values.
5. It is self-explanatory and when compacted they are also easy to follow.
Disadvantages
1. Most of thealgorithms require that the target at-tribute will have only discrete values.
2. Most decision-tree algorithms only examine a single field at a time.
3. Decision trees are prone to errors in classification problems with many class.
4. As decision trees use the "divide and conquer" method, they tend to perform well if a
few highly relevant attributes exist, but less so if many complex interactions are present.