Kannada Handwritten Digit Recognition Using

PCA and SVM on Kannada-MNIST Dataset

Ramesh. G, Prasanna G. B, Santosh Bhat, Chandrashekar Naik, Champa H. N

Abstract- Handwritten digit recognition is domains. An important domain in this field is

one of the classical problem in the field of efficient and generic handwritten digit
image classification, a subfield of computer recognition. The handwritten digit recognition
has many potential real applications such as
vision. In this paper, we have used Kannada-
marks digitization, banking utilities, postal code
MNIST dataset for digit recognition to reading and tax form. The isolated handwriting
evaluate the performance of support vector recognition process can be broken down into
machines (SVM) and principal component three stages: preprocessing, feature extraction
analysis (PCA). Efforts were made previously and classification. Feature extraction plays an
to recognize handwritten digits of different important role in getting high accuracy rates.
languages with this approach. However, due But along with this proper preprocessing of data
also contributes to high accuracy. Many research
to lack of standard MNSIT dataset for
activities are made in this regard for English
Kannada numerals, Kannada Handwritten numerals and impressive outputs are obtained.
digit recognition was left behind. With the However, there is room for more improvement
introduction of MNIST dataset for Kannada when it comes to Kannada numerals.
digits, we budge towards solving the problem Support Vector Machines (SVM) is widely
statement and show how applying PCA for used for classification of numbers in handwritten
dimensionality reduction before using the digit recognition due to high accuracy. But when
Principal Component Analysis (PCA) is used as
SVM classifier increases the accuracy on
a preprocessing step along with SVM much
RBF kernel. We have used 60,000 images for higher accuracy rate is seen. PCA reduces the
training and 10,000 images for testing the number of features, and then they use some
model. We were able to achieve accuracy of Principal Components (the eigen vectors of the
97.63%. We have also evaluated our covariance matrix) as the new features. This in
technique based on precision, recall and F1- turn removes the non predictive features and
score. gives much better results.
This project presents recognizing the
Keywords- Computer Vision, handwritten Kannada digits (0 to 9) from the
famous MNIST dataset, comparing SVM
Dimensionality Reduction, Handwritten Digit
classifier and the cascade of SVM-PCA on RBF
Recognition, Kannada-MNIST Dataset,
kernel. Various performance factors like the
Principal Component Analysis, Support
accuracy, precision, recall, F1 score have been
Vector Machine. compared for the kernel and the two classifiers.
I. INTRODUCTION A. Motivation
Machine learning and deep learning plays an
Kannada numerals has a very long and
important role in computer technology and
artificial intelligence. With the use of machine rich history. The earliest inscription having
learning, human effort can be reduced in all 9 Kannada numerals have been engraved
recognizing, learning, predicting and many more in the Gudnapur Inscription which dates
areas. It is a fast-growing field of computer back to the time of Kadamba
science that is making its way into all other Ravivarma(485 A.D. to 519 A.D.). The
 symbols used to represent 0–9 in the recognition of Kannada numerals. The
language are distinct from the modern recognition of isolated Kannada characters was
Hindu-Arabic numerals. Even today, people first explored by [Kunte], where wavelet
of Karnataka use Kannada digits for day-to- features were extracted from the character
day affairs. contour and used as features. A character
Kannada numerals also got itself a full- recognition accuracy of 56% was achieved using
fledged Kannada-MNSIT dataset in 2019. a Multi-layer feedforward neural network with a
There have been numerous works around single hidden layer.
Kannada digits in ML before this. However, In [{Pal} and N. {Sharma}] recognition of
Kannada-MNIST data provides sufficient off-line handwritten numerals of six popular
amount of data for training and testing. The Indian scripts has been done using a quadratic
state of the art classifiers like SVM along classifier-based scheme. A follow up to this
with PCA have been used for recognition of work can be found in [rajesh] where zone and
handwritten digits in different languages. distance metric based feature extraction is used
However, this method is yet to be using SVM for classification and recognition of
implemented on standard native language Kannada script.
datasets. This motivated us to use the SVM In [Rajput] Support Vector Machine is
classifier along with PCA on the Kannada- employed for recognition of printed and
MNIST dataset. handwritten mixed Kannada numerals
recognition with an accuracy of 97.76% for a
B. Contribution
data set size of 5000 mixed numerals image.
Structural and topological features of numerals
are used for recognition of handwritten Kannada
C. Organization
vowels and consonants in [angadi2015structural]
The work is unfolded over the pages in and the test characters are categorized into
8 sections. Section I provides the vowel/consonant classes using the multiclass
introduction and Section II provides the SVM classifier, obtaining 89.84 % efficiency in
insight about the previous works on the recognizing Kannada vowels (swaragalu) and
problem statement. Section III is about the 85.14% efficient in recognizing consonants.
motivation behind this work and Section [ramappa2013comparative] throws light on
IV describes clearly the problem statement the ongoing research in Optical Character
we have approached to solve. Section V Recognition Systems with focus on different
narrates the system architecture and methods for segmentation, feature extraction and
Section VI provides a detailed description for classification.
of the proposed system. Section VII is the [aravinda2014template] adopts a Correlation
results section and Section VIII concludes Technique for recognition of Kannada
the work, while Section IX addresses the Handwritten Characters. The digitized input
acknowledgement. image is subjected to various pre-processing
techniques and the processed image is then
II. RELATED WORK segmented into individual characters using
The work on Kannada handwritten digit simple segmentation algorithm. The segmented
recognition is very limited and less research has individual character is correlated with the stored
been taken place in this field. There is scope for templates and template with maximum
improvement in the techniques used till date for correlation value is displayed in editable format.
 In [pasha2015handwritten] Wrapping based 2) Preprocessing: In this step, the data is
Curvelet transform is proposed to perform converted into binary format. The images
feature extraction by performing dimensionality are converted to grayscale for better
reduction using principal component analysis. efficiency and all the images are
Nearest neighbor classifier is used to recognize transformed into 28x28.We approach
the handwritten Kannada characters and an different methods to reduce the noise in the
overall accuracy of 90% was achieved. data. By applying different measures to fill
out the missing data or ignore the less
III. PROBLEM STATEMENT
important data.
This works aims to solve the “Recognition of
3) Feature Extraction using PCA: PCA is
handwritten Kannada digits”, by using the state
used to map the high dimensionality space
of the art classifier, which gives very high
of input features to lower dimensions by
accuracy along with reliability. This effort is
throwing away some columns. It selects
focused on using SVM with PCA on the
some columns of features, which has low
Kannada-MNSIT dataset to recognize the digits
standard deviation and deletes it from the
efficiently and overcome the limitations of the
feature matrix. The data points with the
existing works in this domain.
highest correlation will be retained after
IV. SYSTEM ARCHITECTURE applying this method and has good impact
on further training process.
The flowchart of the system architecture is
shown in the Fig. 1 4) Classification using SVM: It is a
supervised machine learning method, which
classifies the digits based on the features of
the array of grayscale images. Mean and
standard deviation of each digit is
considered as the features here.  Using these
features, multi-class SVM classifiers are
trained to separate different classes of digits
by finding the hyperplanes with maximum
margins.

5) Data Visualization: By using pictorial

elements like charts, graphs, and maps, data
visualization tools provide an accessible
Fig. 1. System Architecture way to see and understand trends, outliers,
The steps involved in the method are as follows: deviation and patterns in data. By this, we
can compare the expected and predicted
1) Dataset Collection and Division: The output of the models. We can also
dataset consists of images of handwritten distinguish the performance of different
numerals in Kannada with 60,000 images in algorithms by representing data in a pictorial
training set and 10,000 images in the test form. 
set. Data Set Dimensions:
Training set: 60000x28x28 6) Output Analysis: Output analysis is the
Test set: 10000x28x28 essential step to evaluate the performance of
 the model. The good model should give significant features. It finds a sequence
satisfactory results for the input data. We of linear combination of the variables that
can use confusion matrix, F1 score, have maximum variance and are mutually
uncorrelated. Apart from finding, the
classification accuracy, mean squared errors
features with major significance it also used
etc. to evaluate the performance of the for data visualization.
model.
Z1=ϕ11X1+ϕ21X2+…+ϕp1Xp
V. PROPOSED SYSTEM
The first principal component of a set of
A. Kannada Numerals: features X1, X2,…, Xp is the normalized
linear combination of the features with the
largest variance.
C. Support Vector Machine:

Main purpose of the support vector

machine algorithm is to find a hyperplane in
an N-dimensional space (N- Number of
features) which contains all the distinct
features of the data.

Fig.2. Handwritten digits in the

Kannada MNIST dataset

Kannada is the official language of the

southern Indian state of Karnataka. It ranks
at top in the list of Dravidian languages of
India. It is as old as Tamil, the truest
language of the Dravidian family. The
earliest inscription having all 9 Kannada
numerals have been engraved in the
Gudnapur Inscription which dates back to
the time of Kadamba Ravivarma(485 A.D. Fig. 4.1 Possible hyperplanes
to 519 A.D.). The symbols used to represent
0–9 as shown in Fig.2 are distinct from the
modern Hindu-Arabic numerals.

B. Principal Component Analysis:

Principal component analysis (PCA) is a

statistical method, which uses to convert a
set of observations of correlated data into a
set of values of linearly uncorrelated data
called principal components using an
orthogonal transformation. It produces a
low-dimensional representation of a data set
by eliminating the columns containing less
Fig. 4.2 Best fit hyperplane Equation (2) is the improvised equation
of (1).It has the slack variable ‘ε' to
The objective is to find the best-fit separate the noise and outliers in the
hyperplane of all the possible data.
hyperplanes. The plane is called best fit We can represent the model using
when it has the maximum distance different kernels like sigmoid,
between the data points of different polynomial and rbf kernels to make it
classes. linearly separable.
Support vectors are the points, which In this paper we used RBF as kernel
are closer to the hyperplane and impact function which is defined as :
on the position and orientation of the
hyperplane. We can maximize the K(xi , xj) = exp(−γ||xi −xj ||2 ) -(3)
distance of the data points, which lie in
different classes using these support where γ is the parameter of kernel
vectors. function. This parameter has influence
on the quality of classifier, so tuning the
value of it is an important task.

D. Algorithm:

Our system uses the following algorithm

to solve the problem statement:

Fig. 5. Illustration of support vectors

Hyperplane that separates the features

of different classes is defined by the
equation:
yi (w· xi +b) ≥ 1 for 1 ≤ i ≤ n -(1)
where xi ∈ S are data points represented
as vectors in d-dimensional space, n is
the number of data points, yi ∈ {−1,1}
are classes of corresponding instances
and w and b are parameters of the
hyperplane. Hyperplane should lie as far
as possible from data points of both
classes. The distance can be maximized
using 2/ ||w||.

Real word information contains noise

and outliers that can be eliminated using
the equation:

yi(w· xi +b) ≥ 1−εi , εi ≥ 0, 1 ≤ i ≤ n (2)

 The model is first trained only using SVM
classifier without applying PCA and the
accuracy obtained is 95.43%. After the
application of Principal Component Analysis on
the data, the accuracy of the model increased to
97.63%. The measure of performance such as
precision, recall and f1-score have also been
calculated for the model as shown in Fig _, for
the given data set. The model’s performance for
the data set can be studied using these measures
and proper tuning of parameters can be done.
The confusion matrix shown in Fig _ clearly
shows that a maximum misclassification error is
observed between the digits '0' and '1'. The
second most misclassification is seen between
the digits '6' and '7'. The reasons for
misclassification may be due to similarity of
shape found between some of the digits, but it
also depends on different writing styles followed
by individuals, which make the samples of a
particular class closer to other class. The
accuracy obtained for individual digits is shown
in Table 1.
Table 1. Individual Digit Accuracy

Fig. 3 Flow chart of SVM-PCA merge Recognized as

algorithm Numeral '0' '1' '2' '3' '4' '5' '6' '7' '8' '9'
'0' 1745 44 1 5 0 0 0 0 5 0
VII. RESULT '1' 15 1777 1 1 1 1 0 1 1 2
'2' 8 1 1774 4 0 8 0 4 1 0
The proposed methodology has been applied '3' 5 2 7 1748 3 13 5 15 1 1
to a dataset [vinayprabhu] consisting of 70000 '4' 0 1 1 3 1769 13 1 0 2 10
samples of isolated handwritten Kannada '5' 1 1 19 8 14 1751 4 1 1 0
numerals. For the experimentation, 60000 '6' 0 2 0 2 3 0 1750 41 0 2
samples are used for training the model and the '7' 1 0 1 29 1 0 33 1728 0 7
rest 10000 samples are used for model testing. '8' 20 11 2 0 2 3 0 0 1761 1
The size of image considered is 28 * 28 pixel. '9' 4 0 0 4 26 1 20 3 2 1740
The results of the evaluation are carried out in
two-fold, first without applying the PCA and
then with PCA. Of the various kernel functions
available in SVM such as linear, RBF and
polynomial, due to the non-linearity of the data,
RBF kernel appeared to be the best fit and with
higher accuracies than the other functions.
 Digits
0 number={1Supplement},
pages={417-420},

1 year={2000},
doi={10.3154/jvs.20.1Supplement_417}

2 }
@INPROCEEDINGS{4377015, author={U.
{Pal} and N. {Sharma} and T.
{Wakabayashi} and F. {Kimura}},
booktitle={Ninth International Conference
on Document Analysis and Recognition
(ICDAR 2007)},
title={Handwritten Numeral Recognition of Six
Popular Indian Scripts},
VIII. CONCLUSION
year={2007},
In this paper we have used SVM classifier volume={2},
along with PCA for recognition of Kannada
Handwritten digits using the Kannada-MNSIT number={},
dataset. Using PCA along with SVM helps in
pages={749-753},
increasing the accuracy as we can see from the
Table 2. }

Author Method Accuracy

SVM 95.43 @inproceedings{rajashekararadhya2009support,
Proposed system SVM+ PCA 97.63
title={Support vector machine based
Table 2. Comparison handwritten numeral recognition of Kannada
script},
author={Rajashekararadhya, SV and Ranjan, P
Vanaja},
booktitle={2009 IEEE International Advance
Computing Conference},
REFERENCES
pages={381--386},
@article{2000417, year={2009},
title={170Wavelet Features based On-line organization={IEEE}
Recognition of Handwritten},
}
author={R.Srinivasa Rao Kunte and
R.D.Sudhaker Samuel},
journal={Journal of the Visualization Society
of Japan},
@article{article,
volume={20},
author = {Rajput, Ganapatsingh and Horakeri, year={2013}
and Rajeswari, and Sidramappa, Chandrakant},
}
year = {2010},
month = {08},
pages = {},
@inproceedings{aravinda2014template,
title = {Printed and Handwritten Mixed Kannada
Numerals Recognition Using SVM}, title={Template matching method for Kannada
Handwritten recognition based on correlation
volume = {2}, analysis},
journal = {International Journal on Computer author={Aravinda, CV and Prakash, HN},
Science and Engineering}
booktitle={2014 International Conference on
} Contemporary Computing and Informatics
(IC3I)},
@article{angadi2015structural,
pages={857--861},
title={Structural features for recognition of
hand written Kannada character based on year={2014},
SVM},
organization={IEEE}
author={Angadi, SA and Angadi,
Sharanabasavaraj H}, }

journal={International Journal of Computer

Science, Engineering and Information @inproceedings{pasha2015handwritten,
Technology},
title={Handwritten Kannada character
volume={5}, recognition using wavelet transform and
number={2}, structural features},

pages={25--32}, author={Pasha, Saleem and Padma, MC},

year={2015} booktitle={2015 International Conference on

Emerging Research in Electronics, Computer
} Science and Technology (ICERECT)},
@article{ramappa2013comparative, pages={346--351},
title={A comparative study of different feature year={2015},
extraction and classification methods for
recognition of handwritten Kannada numerals}, organization={IEEE}

author={Ramappa, Mamatha Hosalli and }

Krishnamurthy, Srikantamurthy}, @article{prabhu2019kannada,
journal={International Journal of Database title={Kannada-mnist: A new handwritten
Theory \& Application}, digits dataset for the kannada language},
volume={6}, author={Prabhu, Vinay Uday},
number={4}, journal={arXiv preprint arXiv:1908.01242},
pages={71--90}, year={2019}
}