2021 6th International Conference for Convergence in Technology (I2CT)

Pune, India. Apr 02-04, 2021

Multilingual Text & Handwritten Digit Recognition

and Conversion of Regional languages into
Universal Language Using Neural Networks
Dr. Bhushan Vidhale Prof. Ganesh Khekare, Prof.Chetan Dhule,
Department. of Electronics & Department of Information and Department of Information and
Telecommunication Engg. Technology, Technology,
G H Raisoni College of Engineering, G H Raisoni College of Engineering, G H Raisoni College of Engineering,
Nagpur, India Nagpur, India Nagpur, India
[email protected],
[email protected]

Dr. Pankaj Chandankhede, Prof. Abhijit Titarmare, Prof. Meenal Tayade,

Department. of Electronics & Department. of Electronics & Department. of Electronics &
Telecommunication Engg. Telecommunication Engg. Telecommunication Engg.
2021 6th International Conference for Convergence in Technology (I2CT) | 978-1-7281-8876-8/21/$31.00 ©2021 IEEE | DOI: 10.1109/I2CT51068.2021.9418106

G H Raisoni College of Engineering, G H Raisoni College of Engineering G H Raisoni College of Engineering,

Nagpur, India Nagpur, India Nagpur, India

Abstract—The Character recognition techniques equate I. INTRODUCTION

an illustrative identity with the image of character.
Handwritten human character recognition is a machine's Handwriting recognition has acquired a special place in
ability to obtain and recognize handwritten information from the field of pattern recognition and machine learning due to its
various sources such as papers, photos, tactile touch devices enormous applications. This technique is the solution to the
etc. Recognition of handwriting and computer characters is problem when it is difficult for the viewer to understand
an evolving field of study and has broad uses in banks, offices someone else's handwriting. For this we are using a certain
and industries. The key objective of this research work is to type of deep neural network which is Convolutional Neural
develop a knowledgeable framework for "Handwritten Networks. A GUI is build in which we can draw the digit and
Character Recognition (HCR) victimization Neural Network" recognize it straight away. This Python project requires you to
which might effectively acknowledge selected type-format have basic knowledge of Python programming, deep learning
character victimization as the substitute Neural Network with a library for building GUI.
approach. Neural method is the best method for controlling
images, thus style parts square measure less all around plot as For character recognition and translation, we will use a
compared to various designs. Neural computers do parallel matlab code. Optical character recognition ( OCR) is actually
results. Neural computers square measure run during a a computer code established to translate pictures that are
manner that's utterly different from traditional operation. captured employing a scanner and regenerate into machine
Neural computers square measure conditioned (not editable text, or to remodel pictures of characters or symbols
programmed) in such a way, that how it's given in an explicit into standard ASCII or Unicode formats. [1] In this particular
beginning state (data input); they either assign the project, a character recognition framework by employing
information (input file or computer file) into one amongst the JAVA is build. There is a great need of recognition program
quantity of categories or permit the initial data to evolve to of this type these days. In today's world, where our main focus
maximize an explicit fascinating property. In this research or concentration is on using efficient and cost-effective
work, a purely handwritten digit recognition using machine processes, many people are now using technology to perform
learning model as well as character recognition matlab model
activities that used to be both laborious and time-consuming.
is used. A translator using MATLAB to beat the barrier of
various languages is designed. The projected style is also used The nature of our modern gadgets is their productivity and
for English, Marathi and Guajarati text to speech conversion their role in reducing the strength of labor. Computers are now
into English language. Input is taken in English, Marathi and being used as a more well-ordered and well-regulated
Gujrati text manually to the interface or image of written text substitute for taking and handling notes and for eradicating
or handwritten text and output can be translated in English potential issues around handwriting consistency or paper
Language by facilitating use of Optical Character sheets being lost. Additional advantages of choosing
Recognition (OCR) technique. The projected methodology is
technology include the ease of later rendering such notes
also used to produce help to folks that lack the ability of
easier, as well as reducing the waste of physical space.
speech or non-native speakers. On the other hand, purely
handwritten digit recognition using machine learning Those electronic documents would be placed on drives as
algorithms is used to interpret the human handwriting to the such. Another example of the advantages of the current
second person easily and effectively. technology is the use of tablets for both professionals and
students, eliminating the need for multiple books and various
Keywords—: Off-line Handwritten Recognition, important documents, once again avoiding the loss of
Handwritten Character, Pattern Recognition, Feature
additional physical space and enabling all the necessary data
Extraction, Neural Network.
to be stored in a small piece of equipment. Data management
has achieved considerable popularity in executions in

978-1-7281-8876-8/21/$31.00 ©2021 IEEE 1

Authorized licensed use limited to: Carleton University. Downloaded on May 25,2021 at 17:03:15 UTC from IEEE Xplore. Restrictions apply.
engineering. Since the industry sector has been steadily string matching rule to urge match aspirant, and conjointly the
increasing over the past several decades, a significantly 2nd stage uses loads of acceptable close distance live, just like
higher market can be met for automated machinery. This the Levenshtein distance. Like this process shows little
automatic machinery is highly dependent on data enhancements in recall, with small decreases in preciseness.
management in the form of utility character identification.
A simple example for this can be the analogy of data III. INDIAN CHARACTER RECOGNITION
management in a particular distribution complex close to Not many makes it, a several trials are done on the word
mail organization, where all data is processed by OCR recognition of Indian character sets. Even so, some vital works
platforms. Then decisions are made by means of automated square measure according on Nagari script. A few makes an
machine arms to route the correct mail to the right lane so attempt on Tamil, Gujarathi, Malayalam, Marathi and Bengali.
that all mails are routed according to the places they need to Word recognition and written communication is of nice
be sent. importance for electronic conversion of historical knowledge
alongside letters and varied documents. The matter is hard as a
II. LITERATURE SURVEY result of human handwriting unevenness, variable skew and
ManikVarma, “Character Recognition In Natural direction still as noise and deformation like fleck, smears,
Images” they implement the matter of realizing characters light-weight print, and so on.
in pictures of natural scenes. Specially, they concentrate on
recognizing characters in things that might historically not Recognition of written Indian manuscripts considerably of
be handled well by OCR techniques. They gave associate gujrati, still as English, Marathi etc. Majority of the Indian
degree anno-tated data of images carrying South and scripts have five hundred or heaps of characters, words or
English Dravidian characters.[2] The data contains images symbols utilized in flowing text, by way of quantity of basic
of street scenes taken in city, Asian natiSon using a consonants and vowels isn't quite fifty. The quantity is inflated
standard device. The matter addressed in the associate by 3 forms of vowel modifiers that is in a position to be
object categorization framework supported a counts-of- appendant below the consonants, thus generating threefold
words illustration. They evaluate the performance of consonant-vowel mixtures. A great deal of increase in vary is
assorted choices supported nearest neighbor and SVM feasible wherever consonant creates a rhetorical writing kind
classification. It’s incontestable that the performance of the discovered as compound characters. For a couple of
projected methodology, victimization as few as fifteen manuscripts like Malayalam, kannada and syllabary languages
coaching pictures, may be way superior to it of economic compromise monumental vary of compound letters. These
OCR systems. what is more, the strategy will take pleasure compound letters may take vowel transformers to come back
in synthetically generated coaching information, preventing up with threefold tons of shapes. Hence, writing symbols may
the requirement for dearly-won information assortment and run of the structure of a thousand.
annotation. Purpose of study
In more recent add a similar space; Lopresti and Zhou Recently, it tends to be within the condition that people
dynasty calculated the performance of the many classical forever carry giant resolution digital devices. For this
and magnified IR models exploitation simulated OCR rationale, image recovery with queries captured by a digital
information. to strengthen ancient IR models to agitate the device is paid further attention. Throughout this a tendency to
flawed information, they used estimated string matching particularly concentrate on document image recovery, that
and symbolic logic. At large, they were able to reveal that would be the duty to search out a document image from an
the new ways that area unit extra robust to strident information comparable to a question achieved by clicking a
information than the initial ways that, suggesting that document. This method will give people with the knowledge
simple enhancements is used to improve performance [3]. that is related to the recovered document within the
Ohta et al, delineate a system for word (content) search information. In alternative words, along the assistance of
throughout that they augment three possibilities any document image recovery, paper documents will be observed
retrieval ways in which with info regarding awaited OCR as media for accessing varied information; pictures, texts and
errors.The approach used uncertainty data for certain additional. In addition to that it focuses on text recognition of
characters, in conjunction along with bi-gram potentialities traditional languages such as gujrati and Marathi. The
of character existence to make numerous possible search projected methodology is also use to produce help to folks that
terms for each primary search term when activity the search lack the ability of speech or non-native speakers.
with each latest term, the rationality of came documents IV. OBJECTIVE
depends on the hesitancy and bi-gram prevalence
probabilities.. [4]The results claim will increase from 2-3 1. Using Neuronal signals in the realm of literature.
percent in recall with decreases or 4-5 percent in exactitude. 2. Man-power reduced for manually converting old
Fujisawa Associate in NursingdMarukawa used constant literature into digitized form.
approach throughout that they use confusion statistics to
urge Associate in nursing enhanced finite state machine for 3. The suggested program acted as a reference and
question terms in Japanese text. functioned in identification fields of character.
A different approach to making an attempt to alter the 4. Making the digitized language library rich.
question to influence low quality is to alter the parallel
(matching) rule, as depicted by Takasu. To urge rate, the
process uses a 2 stage rule where the first stage uses a fast

Authorized licensed use limited to: Carleton University. Downloaded on May 25,2021 at 17:03:15 UTC from IEEE Xplore. Restrictions apply.
 V. SYSTEM ARCHITECTURE 1) Import the libraries and load the dataset
Firstly, import all the modules that will need to training
our model. The Keras library already contains some datasets
and MNIST is one of them. So it can easily import the dataset
and start working with it. The mnist.load_data() method
returns us the training data, its labels and also the testing data
and its labels. Files of handwritten images is made and
converts it into .csv file and make it read.
2) Preprocess the data
The image data cannot be fed directly into the model so we
need to perform some operations and process the data to make
it ready for our neural network. The dimension of the training
Fig. 1. System Architecture data is (60000,28,28). The CNN model will require one more
dimension so we reshape the matrix to shape (60000,28,28,1).
VI. TECHNICAL DETAILS 3) Create the model
A. For character recognition and translation Now, create our CNN model in Python data science
1. Input an Image project. A CNN model generally consists of convolutional and
pooling layers. It works better for data that are represented as
2. Image Preprocessing grid structures, and so CNN works well for image
classification problems. The dropout layer is used to
3. Selection (Segmentation) deactivate some of the neurons and while training, it reduces
4. Extracting features (Feature Extraction) over fitting of the model.
5. Categorization (Classification)[6] 4) Train the model
The model.fit() function of Keras will start the training of
1) Input an image the model. It takes the training data, validation data, epochs,
Input image is also any written scanned image that and batch size. It takes some time to train the model. After
contains Bangla numeric digits. The image could contain training, we save the weights and model definition in a certain
single or connected digit(s) and size of letter could dissent file.
from one another.
5) Evaluate the model
2) Image Preprocessing We have 10,000 images in our dataset which will be used
The image is preprocessed victimisation varied to evaluate how good our model works. The testing data was
algorithms for image process, like Inverting image, grey not involved in the training of the data therefore, it is new data
Scale Conversion, and image cutting. for our model. The MNIST dataset is well balanced so we can
3) Segmentation get around 99% accuracy.
After preprocessing of the image selection is finished. 6) Create GUI to predict digits
This can be through along with the assistance of following For GUI, we need to create a new file in which we build
steps: an interactive window to draw digits on canvas and with a
1 .Take out the borderlines. button, we can recognize the digit. The Tkinter library comes
in the Python standard library. For that create a function that
2. Split the text (Characters) into series, takes the image as input and then uses the trained model to
3. Split the rows (horizontal lines) into text. predict the digit. We create a canvas where we can draw by
capturing the mouse event and with a button, we trigger the
4 .Bisect the word (text) into characters function and display the results.
4) Feature Extraction VII. ALGORITHM
Once the word (letter) is divided binary glyphs is
produced and find the summation as options of every row x To make this model, first study and search for the
including column values. handwritten digits and the actual computerized digits.

5) Classification x We need to look for the dataset available already or if

In this part, focus is on area unit aiming to instruct and we want, we can create our own dataset which can be
check the Neural Network. in the “.csv” file format.

B. For Handwritten digit recognition x MNIST dataset can be accessed through OpenML
library, while “.csv” dataset can be made to read
1. Import the libraries and load the dataset. through panda’s library.
2. Preprocess the data
3. Create the model x Next we need to check for the dimensions of the
images in the dataset, for this model we are using
4. Train the model (28*28) dimension that is 28 rows and 28 columns.
5. Evaluate the model
6. Create GUI to predict digits x Then work on the python code and import necessary
modules, libraries and create functions for the same.
We can check for the desired digit then and it will

Authorized licensed use limited to: Carleton University. Downloaded on May 25,2021 at 17:03:15 UTC from IEEE Xplore. Restrictions apply.
 show us accuracy of the digit it has been matched
maximum with.
VIII. EXPERIMENTAL RESULTS
We have used MATLAB and ANACONDA software
system for our implementation on character recognition and
translation & handwritten digit recognition respectively.
The experiments were performed on several datasets, take a
look at pictures having differing types of numerals in
Gujrathi, Hindi and English literature. First we tend to
browse the scanned input image then eliminate the noise
from the scanned image. The preprocessed input image is
divided line by line by scanning and indicating the left-top
edge and right-bottom fringe of the road from the input
image. The segmentation is extracting a line of numerals.
All sides of those lines of numerals can specifically bit the
road boundary of the bounding box. Then we segment on
individual Characters from each segmented line. The Fig. 5. Human handwriting dataset
scanned image is resized to 32x32 pixels and convert them
into English We collect hand written numeral for input
from set of different people and a selected dataset.

Fig. 2. Text Recognition of English Language

Fig. 6. Recognized Digit Testing

IX. CONCLUSION
Regional languages throughout the globe have completely
distinct sorts of writing that may be recognized. The correct
algorithms and methods with HCR systems are implemented.
To acknowledge characters is learnt. It’s been found that
written character recognition becomes troublesome for
multiple characters because of the presence of weird
characters or similarity in shapes. To induce a clean image, the
scanned image is pre-processed, and also the characters area
unit outlying into independent characters.

Fig. 3. Text Recognition and Conversion of Marathi to English Pretreated work is applied within standardization;
purification is applied, exploitation pro-cessation measures
that deliver hushed and unstained performance. Managing our
assessment formula along with correct coaching, evaluating
alternative step-wise method with higher potency can lead to a
triple-crown output of system. Exploitation some applied
mathematics options and geometric options through neural
network can offer higher character recognition result. This
analysis ought to be of profit to the researchers in their
analysis on another story.
REFERENCES
[1] Li, H. and Doermann, D., Automatic Identification of Text in Digital
Video Key Frames, Proceedings of IEEE International Conference of
Pattern Recognition, pp. 129- 132, 1998
[2] de Campos, Teofilo&Babu, Bodla&Varma, Manik. (2009). Character
Fig. 4. Text Recognition and Conversion of Gujrathi to English Recognition in Natural Images.. VISAPP 2009 - Proceedings of the 4th

Authorized licensed use limited to: Carleton University. Downloaded on May 25,2021 at 17:03:15 UTC from IEEE Xplore. Restrictions apply.
 International Conference on Computer Vision Theory and [8] Ohya, J., Shio, A., and Akamatsu, A., Recognition of characters in
Applications. 2. 273-280. scene images. IEEE Transactions on Pattern Analysis and Machine
[3] Doermann, D.. "The Indexing and Retrieval of Document Images: A Intelligence, vol. 16, no. 2, pp. 214- 220, 1994.
Survey" , Computer Vision and Image Understanding, academic [9] Sato, T., Kanade, T., Hughes, E.K., and Smith, M.A., Video OCR for
press,US,vol,no.70,no.3,1 june1998 (199806).PP287-298 digital news archives. IEEE Int. Workshop on Content-Based Access of
[4] ChandralekhaPadvekar, Pranoti Shukla, Neha Shah, Image and Video Database, 1998.
KetkeeSonawane, 2014, Text Document Analysis and Recognition, [10] Bhushan Vidhale, SS Dorle, Performance analysis of routing protocols
INTERNATIONAL JOURNAL OF ENGINEERING RESEARCH in realistic environment for vehicular Ad Hoc networks, 2011 21st
& TECHNOLOGY (IJERT) Volume 03, Issue 02 (February 2014), International Conference on Systems Engineering, 267-272
[5] Dalal, j., &daiya, s. (2018). Image processing based optical [11] SSDorle, B Vidhale, M Chakole, Evaluation of multipath, unipath and
character recognition using matlab. International journal of hybrid routing protocols for vehicular Ad hoc networks, 2011 Fourth
engineering sciences & research technology,7(5), 406-411. International Conference on Emerging Trends in Engineering &
[6] RenukaMarutiraoPujari, Vikas D. Hajare.” Analysis of ultrasound Technology
images for identification of Chronic Kidney Disease stages", 2014 [12] Bhushan R. Vidhale, Ganesh Khekare, Chetan Dhule, Meenal Tayade,
First International Conference on Networks & Soft Computing Design of Automized Inspection Vehicle for RTO, International
(ICNSC2014), 2014 Journal of Engineering and Advanced Technology (IJEAT) ISSN: 2249
[7] Lienhart, R., Automatic Text Recognition for Video Indexing, – 8958, Volume-9 Issue-5, June 2020
Proceedings of ACM Multimedia 96, pp. 11-20, 1996.

Authorized licensed use limited to: Carleton University. Downloaded on May 25,2021 at 17:03:15 UTC from IEEE Xplore. Restrictions apply.