PROGRAMMING USING R

NITHISHKUMAR. S
RA2332241040023
DEPARTMENT OF MCA
1ST YEAR
 TITLE : SUPPORT VECTOR MACHINE (SVM)

CONTENTS :

 INTRODUCTION TO SVM
 CLASSIFICATION OF SVM
 USAGE OF SVM
 IMPLIMENTATION
 ADVANTAGES
 DISADVANTAGES
 CONCLUSION
SUPPORT VECTOR MACHINE (SVM) :

Support vector machine (SVM)

is a supervised machine learning algorithm that analyzes and classifies
data into one of two categories — also known as a binary classifier.

• It is a supervised machine learning algorithm used mainly for

classification tasks, but it can also be applied to regression and outlier
detection problems.
• The primary goal of SVM is to find the hyperplane that best separates
the data points into different classes. This hyperplane is chosen in such
a way that it maximizes the margin between the classes, i.e., the
distance between the hyperplane and the nearest data points from
each class.In the case of non-linearly separable data, SVM can use a
technique called the kernel trick to transform the input features into a
higher-dimensional space, where the classes become linearly
separable. Common kernel functions include linear, polynomial, radial
basis function (RBF), and sigmoid.

• Support Vector Machine (SVM) is a supervised machine learning

algorithm used for both classification and regression. Though we say
regression problems as well it’s best suited for classification. The main
objective of the SVM algorithm is to find the optimal hyperplane in an
N-dimensional space that can separate the data points in different
classes in the feature space. The hyperplane tries that the margin
between the closest points of different classes should be as maximum
as possible.
• The dimension of the hyperplane depends upon the number of
features. If the number of input features is two, then the hyperplane is
just a line. If the number of input features is three, then the
hyperplane becomes a 2-D plane. It becomes difficult to imagine when
the number of features exceeds three.

• SVM algorithms are very effective as we try to find the maximum

separating hyperplane between the different classes available in
the target feature.
CLASSIFICATION OF SVM :

• In SVM Classification, the data can be either linear or non-linear. There

are different kernels that can be set in an SVM Classifier. For a linear
dataset, we can set the kernel as 'linear'. On the other hand, for a non-
linear dataset, there are two kernels, namely 'rbf' and 'polynomial'.

Based on the nature of the decision boundary, Support Vector

Machines (SVM) can be divided into two main parts :

 Linear SVM

 Non-linear SVM
  Linear SVM :

• Linear SVMs use a linear decision boundary to separate the data

points of different classes. When the data can be precisely linearly
separated, linear SVMs are very suitable. This means that a single
straight line (in 2D) or a hyperplane (in higher dimensions) can entirely
divide the data points into their respective classes. A hyperplane that
maximizes the margin between the classes is the decision boundary.

• LinearSVC is a powerful tool for classification tasks, especially when

dealing with large datasets and linearly separable data. Its flexibility in
terms of penalties and loss functions, as well as its scalability, make it a
popular choice among machine learning practitioners.

• When the data is perfectly linearly separable only then we can use
Linear SVM. Perfectly linearly separable means that the data points
can be classified into 2 classes by using a single straight line(if 2D).
 Non-Linear SVM :

• Non-Linear SVM: When the data is not linearly separable then we can
use Non-Linear SVM, which means when the data points cannot be
separated into 2 classes by using a straight line (if 2D) then we use
some advanced techniques like kernel tricks to classify them. In most
real-world applications we do not find linearly separable datapoints
hence we use kernel trick to solve them.

• Nonlinear SVM (Support Vector Machine) is necessary when the data

cannot be effectively separated by a linear decision boundary in the
original feature space. Nonlinear SVM addresses this limitation by
utilizing kernel functions to map the data into a higher-dimensional
space where linear separation becomes possible.

• The kernel function computes the similarity between data points,

allowing SVM to capture complex patterns and nonlinear
relationships between features. This enables nonlinear SVM to handle
intricate data distributions, such as curved or circular decision
boundaries. By leveraging the kernel trick, nonlinear SVM provides a
powerful tool for solving classification problems.
 USAGE OF SVM :

• SVMs can be used for a variety of tasks, such as text classification,

image classification, spam detection, handwriting identification, gene
expression analysis, face detection, and anomaly detection. SVMs are
adaptable and efficient in a variety of applications because they can
manage high-dimensional data and nonlinear relationships.

• SVM algorithms are very effective as we try to find the maximum

separating hyperplane between the different classes available in
the target feature.

• SVMs are used in applications like handwriting recognition, intrusion

detection, face detection, email classification, gene classification, and
in web pages. This is one of the reasons we use SVMs in machine
learning. It can handle both classification and regression on linear and
non-linear data.

• Handwriting detection
• Handwriting recognition has beenone of the most fascinating and
challengingresearch areas of image processingand pattern recognition
in the recentyears. It contributes enormously to theimprovement of
automation process andupgrades the interface between man
andmachine in numerous applications whichinclude, reading aid for
blind, librarycataloguing, ledgering, processing of forms,cheques and
faxes and conversion of anyhandwritten document into editable
textetc. As a result, the off-line handwritingrecognition continues to be
an active area ofresearch towards exploring the innovativetechniques
to produce adequate accuracy.Even though, sufficient studies
haveperformed in foreign scripts like Chinese,Japanese and Arabic
characters, onlyfew works can be traced for handwrittencharacter
recognition of Indian scripts.

• Face detection

• In the component system we first locate facial components, extract

them and combine them into a single feature vector which is classi-
fied by a Support Vector Machine (SVM). The two global systems
recognize faces by classifying a single feature vec- tor consisting of the
gray values of the whole face image.
• E-mail classification

• n the SVM algorithm, we plot each data item as a point in n-

dimensional space (where n is a number of features you have) with
the value of each feature being the value of a particular coordinate.
Then, we perform classification by finding the hyper-plane that
differentiates the two classes very well.

• A spam detector is software that will detect any message as spam or

not spam. This type of software is very useful to protect people from
fraud. As in today’s time, there are a lot of people out there in the
world who send people fraud SMS to make fool of them. These SMS
contains various kinds of schemes like making money daily or some
sort of these things. But all these messages are there to get money out
of your pocket.

• These SMS are created in such a way that it is hard to detect whether
these are real or spam messages for a normal person. That’s why most
people get caught in these fraud schemes and lose their hard-earned
money. It’s not just about money, there are other types of fraud too
that takes place just through SMS.
IMPLEMENTATION OF SVM :

Steps involved in Implementation of svm

 Import the dataset

 Explore the data to figure out what they look likePre-process the
data

 Split the data into attributes and labels

 Divide the data into training and testing sets

 Train the SVM algorithm

 Make some predictions

 Evaluate the results of the algorithm

Some of these steps can be combined depending on how you handle

your data. We'll do an example with a linear SVM and a non-linear SVM.
Face detection using SVM in R programming

Implementation code

# Install and load required packages

install.packages("e1071") library(e1071)
# Example data
# Let's create a simple dataset with two features and two classes#
Replace this with your actual dataset
set.seed(123)
# For reproducibility
X <- matrix(rnorm(100), ncol = 2)
y <- factor(rep(c("Class A", "Class B"), each = 50))

# Train-test split
train_indices <- sample(1:nrow(X), 0.7 * nrow(X))
# 70% for training
X_train <- X[train_indices, ]
y_train <- y[train_indices]
X_test <- X[-train_indices, ]
y_test <- y[-train_indices]

# Train SVM model

svm_model <- svm(x = X_train, y = y_train, kernel = "radial")

# Predict using the trained model

predictions <- predict(svm_model, newdata = X_test)

# Calculate accuracy
accuracy <- mean(predictions == y_test)
cat("Accuracy:", accuracy, "\n")
Output :

The output of the code will display the accuracy of the SVM model on
the test data. Here's what the output might look like

Accuracy: 0.85

This indicates that the SVM model achieved an accuracy of 85% on the
test data, meaning it correctly classified 85% of the test samples.
 ADVANTAGES OF SVM :
Support Vector Machines (SVMs) offer several advantages, making them
a popular choice for classification and regression tasks in machine
learning:
• Effective in High-Dimensional Spaces:
SVMs perform well even in high-dimensional spaces, making them
suitable for problems with many features, such as text classification,
image recognition, and genomics.
• Memory Efficient:
SVMs use a subset of training points (support vectors) in the decision
function, which makes them memory efficient, particularly when dealing
with large datasets.
• Versatile Kernels:
SVMs can use different kernel functions, such as linear, polynomial,
radial basis function (RBF), and sigmoid, allowing them to handle
complex decision boundaries and non-linear relationships between
features.
• Robust to Overfitting:
SVMs tend to generalize well even in cases where the number of
features exceeds the number of samples. This is because SVMs maximize
the margin between classes, which helps prevent overfitting.
• Effective in Data with Few Samples:
SVMs are effective even when the number of samples is less than the
number of features. This is particularly useful in scenarios where
collecting large amounts of labeled data is difficult or expensive.
• Global Optimum:
The objective function of SVMs leads to a convex optimization
problem, ensuring that the solution is the global optimum rather than a
local one.
• Tolerant to Noise:
SVMs are less sensitive to noisy data due to the use of margin
maximization, which helps in better generalization.
• Regularization Parameter:
SVMs have a regularization parameter (C) that helps control the trade-
off between maximizing the margin and minimizing the classification
error, providing flexibility in model tuning.
 DISADVANTAGES OF SVM :
While Support Vector Machines (SVMs) have many advantages, they also
come with certain limitations and disadvantages:
• Difficulty in Choosing Appropriate Kernel:
The performance of SVMs heavily depends on the choice of kernel
function and its parameters. Selecting the right kernel and tuning its
parameters can be challenging and often requires domain expertise or
extensive experimentation.
• Computational Complexity:
Training an SVM can be computationally intensive, especially for large
datasets. The time complexity of SVM training is generally between
O(n^2) and O(n^3), where n is the number of samples. This can make
SVMs less practical for very large datasets or real-time applications.
• Memory Intensive:
SVMs require storing all support vectors in memory, which can
become memory-intensive, especially when dealing with large datasets
with a high number of support vectors.
• Sensitivity to Noise:
SVMs are sensitive to noise in the training data, especially when using
complex kernel functions. Noisy data can lead to overfitting, reducing the
generalization performance of the model.
• Difficulty with Large-Scale and Streaming Data:
While SVMs perform well with small to medium-sized datasets, they
may not scale efficiently to very large datasets or streaming data due to
their computational complexity and memory requirements.
• Binary Classification:
SVMs inherently perform binary classification and need to be
extended for multi-class classification tasks using techniques like one-vs-
one or one-vs-all, which can increase complexity and computational
overhead.
• Interpretability:
SVMs typically provide a black-box model, meaning they offer limited
interpretability compared to simpler models like decision trees or logistic
regression. Understanding the decision boundaries learned by SVMs can
be challenging, especially with complex kernel functions.
 CONCLUSION :

• In conclusion, Support Vector Machines (SVMs) are a versatile and

powerful class of machine learning algorithms with several
advantages, including their effectiveness in high-dimensional spaces,
memory efficiency, robustness to overfitting, and ability to handle
non-linear relationships through kernel functions. SVMs perform well
with small to medium-sized datasets, making them suitable for a wide
range of applications such as text classification, image recognition, and
bioinformatics.

• However, SVMs also have their limitations and challenges, including

the difficulty in choosing appropriate kernel functions, computational
complexity, sensitivity to noise, and lack of direct probabilistic outputs.
SVMs may not be suitable for very large datasets or real-time
applications due to their computational requirements and memory
intensity. Additionally, interpreting the decision boundaries learned by
SVMs can be challenging, and they may require careful tuning of
hyperparameters for optimal performance.

• Despite these drawbacks, SVMs remain a popular choice for many

machine learning tasks due to their effectiveness, versatility, and solid
theoretical foundation. When used appropriately and with proper
consideration of their limitations, SVMs can yield high-quality results
and serve as valuable tools in the data scientist's toolkit.