Scribd
Upload
6 views21 pages

Svm

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classifying data into different classes by finding a hyperplane that separates them. SVM can handle both classification and regression problems and utilizes the kernel trick to classify non-linear data by transforming it into a higher-dimensional space. The document also includes Python code examples for implementing SVM using the Iris dataset.

Uploaded by

abhijaychauhan88
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
6 views21 pages

Svm

Support Vector Machine (SVM) is a supervised machine learning algorithm used for classifying data into different classes by finding a hyperplane that separates them. SVM can handle both classification and regression problems and utilizes the kernel trick to classify non-linear data by transforming it into a higher-dimensional space. The document also includes Python code examples for implementing SVM using the Iris dataset.

Uploaded by

abhijaychauhan88
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 21

SUPPORT VECTOR MACHINE

WITH PYTHON CODE


What Is SVM?

 SVM (Support Vector Machine) is a supervised machine learning


algorithm which is mainly used to classify data into different classes.
Unlike most algorithms, SVM makes use of a hyperplane which acts like
a decision boundary between the various classes.
 The objective of the support vector machine algorithm is to find a
hyperplane in an N-dimensional space(N — the number of features)
that distinctly classifies the data points.
Features Of SVM:

 SVM is a supervised learning algorithm. This means that SVM trains on a


set of labelled data. SVM studies the labelled training data and then
classifies any new input data depending on what it learned in the
training phase.
 A main advantage of SVM is that it can be used for both classification
and regression problems. Though SVM is mainly known for classification,
the SVR (Support Vector Regressor) is used for regression problems.
 SVM can be used for classifying non-linear data by using the kernel trick.
The kernel trick means transforming data into another dimension that
has a clear dividing margin between classes of data. After which you can
easily draw a hyperplane between the various classes of data.
How Does SVM Work?

 Maximal Margin Classifier


 If the classes are separable by a linear boundary, we can use a
Maximal Margin Classifier to find the classification boundary.
What is Hyperplane?

 Before we start let’s first understand “What is Hyperplane?” In a P-


Dimensional space, a hyperplane is a flat subspace having dimensions
“P-1” i.e., in a 2-D space a hyperplane will be a line-
B0+B1X1+B2X2=0
 While in an m-dimensional space-
B0+B1X1+B2X2+…+BmXm=0
 A hyperplane divides the space into two parts –
B0+B1X1+B2X2+…+BmXm< 0, and
B0+B1X1+B2X2+…+BmXm> 0
Equation of line : y = mx+b -- line
General form:
 ax+by+c = 0 ( line )
 -by = ax+c  y = -a\b * x – c/b
m = -a/b, b = -c/b
Use: ax+by+c = 0  ax1+bx2+c  w1x1+w2x2+w0 = 0
 w1x1+w2x2+w3x3+….wnxn+w0 = 0
W = [w1,w2,w3,..,wn] x = [x1,x2,x3,..,xn]
w1x1+w2x2+w3x3+….wnxn+w0 = 0  w.x+w0=0  wTx+w0 = 0  wTx = 0
 y^ = +1 if w.x+b>= 0
 -1 if w.x+b<0
Hyperplane
Support Vectors

 In the example shown above we can see that there are 4 points which
are nearest to the boundary or are defining boundary, these points are
called “Support Vectors”.
The Non –Separable Case or Nonlinear
data

 In the scenario below, we can’t have linear hyper-plane between the


two classes, so how does SVM classify these two classes? Till now, we
have only looked at the linear hyper-plane.
 SVM can solve this problem. Easily! It solves this problem by
introducing additional feature. Here, we will add a new feature
z=x^2+y^2. Now, let’s plot the data points on axis x and z:
Kernel Trick

 Kernel trick allows us to operate in the original feature space without


computing the coordinates of the data in a higher dimensional space.
 K(x, y) = <f(x), f(y)>. Here K is the kernel function, x, y are n dimensional
inputs. f is a map from n-dimension to m-dimension space. < x,y> denotes
the dot product. usually m is much larger than n.

 Simple Example: x = (x1, x2, x3); y = (y1, y2, y3). Then for the function f(x)
= (x1x1, x1x2, x1x3, x2x1, x2x2, x2x3, x3x1, x3x2, x3x3), the kernel is K(x, y
) = (<x, y>)².
 Let’s plug in some numbers to make this more intuitive: suppose x = (1, 2, 3);
y = (4, 5, 6). Then:
f(x) = (1, 2, 3, 2, 4, 6, 3, 6, 9)
f(y) = (16, 20, 24, 20, 25, 30, 24, 30, 36)
<f(x), f(y)> = 16 + 40 + 72 + 40 + 100+ 180 + 72 + 180 + 324 = 1024
 Now let us use the kernel instead:
K(x, y) = (4 + 10 + 18 ) ^2 = 32² = 1024
Same result, but this calculation is so much easier.
Kernel functions

 linear
 nonlinear
 Polynomial
 radial basis function (RBF)
 sigmoid
SVM Classifier:

import pandas as pd
from sklearn.datasets import load_iris
iris = load_iris()

iris.feature_names
iris.target_names

df = pd.DataFrame(iris.data,columns=iris.feature_names)
df.head()
df['target'] = iris.target
df.head()

df[df.target==1].head()
df[df.target==2].head()

df['flower_name'] =df.target.apply(lambda x: iris.target_names[x])


df.head()
df[45:55]
df0 = df[:50]
df1 = df[50:100]
df2 = df[100:]

import matplotlib.pyplot as plt


%matplotlib inline

plt.xlabel('Sepal Length')
plt.ylabel('Sepal Width')
plt.scatter(df0['sepal length (cm)'], df0['sepal width (cm)'],color="green",marker='+')
plt.scatter(df1['sepal length (cm)'], df1['sepal width (cm)'],color="blue",marker='.')
plt.xlabel('Petal Length')
plt.ylabel('Petal Width')
plt.scatter(df0['petal length (cm)'], df0['petal
width(cm)'],color="green",marker='+')
plt.scatter(df1['petal length (cm)'], df1['petal width (cm)'],color="blue",marker='.')

from sklearn.model_selection import train_test_split


X = df.drop(['target','flower_name'], axis='columns')
y = df.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
len(X_train)
len(X_test)

from sklearn.svm import SVC


model = SVC()
model.fit(X_train, y_train)

model.score(X_test, y_test)
model.predict([[4.8,3.0,1.5,0.3]])
SVM Regression

You might also like