International Journal of Engineering and Advanced Technology (IJEAT)

ISSN: 2249 – 8958, Volume-3, Issue-3, February 2014

Automatic Speech Recognition: A Review

Anchal Katyal, Amanpreet Kaur, Jasmeen Gill

Abstract - The research, development and the accuracy of

automatic speech recognition (ASR) remains one of the most II. AUTOMATIC SPEECH RECOGNITION (ASR)
important research challenges over the years e.g. speaker and
language variability, vocabulary size and domain, noise. This It means an automated process that inputs human speech and
paper describes the recent progress and the author’s perspective tries to find out what was said. ASR is useful, for example,
of ASR and gives an overview of major technological perspective in speech-to-text applications (dictation, meeting
and appreciation of the fundamental progress of Automatic transcription, etc.), speech-controlled interfaces, search
speech recognition. engines for large speech or video archives, and speech-to-
speech translation [8] [4].In recent years, automatic speech
Index Terms - ASR, HMM, Classifications of ASR, Speech recognition technology has advanced to the point where it is
Recognition Process, HNN.
used by millions of individuals to automatically create
documents from dictation. One of the major problems in
I. INTRODUCTION
automatic speech recognition technologies is the sensitivity
Speech recognition is a computer science term and is also
of recognizers to any interfering sounds. Since natural
known as automatic speech recognition. It is a feature that
environments often include other sound sources, the
turns speech into text. One of the main advantages for
performance of the existing technologies is severely limited.
speech recognition services is the cut back on misspelled For a phonetic language, there always exits a one to one
words that some typists may suffer from when typing. The
mapping between their pronunciation and orthography [2].
service cuts down on the amount of time editing and fixing
In addition to this, unlike English and other European
spelling corrections [1] [3]. It is also a big advantage to
languages, these languages possess a large number of
people who may suffer from disabilities that affect their
phonemes such as retroflex and aspirated stops. Some of the
writing ability but can use their speech to create text on
major application areas of Automatic speech recognition
computers or other devices. The overall advantage is the
systems are dictation, controlling the programs, automatic
time management. Most people can speak faster than they
telephone call processing and query based information
can type with fewer mistakes.
system such as travel information system , weather report
1.1 Types of speech recognition information system etc. In simple words, Speech
Recognition is the process to take the audio format as input
A. Text-To-Speech and then convert the text format from it as output [8].
Text-To-Speech (or TTS) will manipulate a string of text
into an audio clip. It is useful for blind people to be able to 2.1 Benefits of ASR
use computers but can also be used to simply improve  Accessibility for the deaf and hard of hearing
computer experience. There are several programs available  Cost reduction through automation
that perform TTS, some of which are command-line based  Searchable text capability
(ideal for scripting) and others which provide a handy GUI
[2]. 2.2 Mathematical representation of ASR
In statistical based ASR systems an utterance is represented
B. Simple Voice Control/Commands by some sequence of acoustic feature observations O,
This is the most basic form of Speech-To-Text application. derived from the sequence of words W. The recognition
These are designed to recognize a small number of specific, system needs to find the most likely word sequence, and
typically one-word commands and then perform an action. given the observed acoustic signal is formulated by [1]:
This is often used as an alternative to an application
launcher, allowing the user for instance to say the word W = argmaxW P(W|O) (i)
“Firefox” and have his OS open a new browser window [1].
C. Full dictation/recognition In "equation (i)" the argument, P(W|O) i.e. the word
sequence W is found which shows maximum probability,
Full dictation/recognition software allows the user to read given the observation vector O[1][2]. Using Baye’s rule it
full sentences or paragraphs and translates that data into text can be written as:
on the fly. This could be used, for instance, to dictate an
entire letter into the window of an email client [3]. In some W = argmaxW P (W|O). P (W)/P (O) (ii)
cases, these types of applications need to be trained to your
voice and can improve in accuracy the more they are used. In "equation (ii)”,P (O) is the probability of observation
sequence and is not considered as it is a constant w.r.t. W.
Hence,

W = argmaxW P (W|O) P(W) (iii)

Manuscript received February, 2014.
Anchal Katyal, M.Tech Scholar (CSE) RIMT, Mandi Gobindgarh In "equation (iii)”, P (W) is determined by a language model
Ms.Amanpreet Kaur, AP (CSE) BBSBEC, Fatehgarh Sahib like grammar based model and P(O|W) is the observation
Ms. Jasmeen Gill, AP (CSE) RIMT-IET, Mandi Gobindgarh likelihood and is evaluated based on an acoustic model.

Published By:
71 Blue Eyes Intelligence Engineering
& Sciences Publication Pvt. Ltd.
 Automatic Speech Recognition: A Review

Among the various models, Hidden Markov Model (HMM) C. Language modeling
is so far the most widely used technique due to its efficient It defines what kind of phoneme and word sequences are
algorithm for training and recognition. possible in the target language or application at hand and
2.3 Phases of ASR what are their probabilities.
Automatic speech recognition system involves two phases: D. Decoding
A. Training phase The acoustic models and language models are used in for
searching the recognition hypothesis that fits best to the
A rigorous training procedure is followed to map the basic
models. Recognition output can then be used in various
speech unit such as phone, syllable to the acoustic
applications.
observation. In training phase, known speech is recorded,
pre-processed and then enters the first stage i.e. Feature
IV. SPEECH RECOGNITION PROCESS
extraction. The next three stages are HMM creation, HMM
training and HMM storage [2]. In essence, the basic task involved in speech recognition is
that of going from speech recordings to word labels. As the
B. Recognition phase pattern recognition approach to speech recognition is the
The recognition phase starts with the acoustic analysis of most widely used approach. There are two main variants of
unknown speech signal. The signal captured is converted to the basic speech recognition task, namely isolated word
a series of acoustic feature vectors. Using suitable recognition and connected word recognition [2] [3].
algorithm, the input observations are processed. The speech
is compared against the HMM’s networks and the word 4.1 Variants of the Speech Recognition Task
which is pronounced is displayed. An ASR system can only A. Isolated word recognition
recognize what it has learned during the training process. Isolated word recognition refers to the task of recognizing a
But, the system is able to recognize even those words, which single spoken word where the choice of words is not
are not present in the training corpus and for which sub- constrained to task syntax or semantics. HMMs can be used
word units of the new word are known to the system and the to build an isolated word recognizer. HMM approach is a
new word exists in the system dictionary [4]. well-known and widely used statistical method of
characterizing the spectral properties of the frames of a
III. CLASSIFICATIONS OF ASR pattern. HMMs are particularly suitable for speech
The goal of an ASR system is to accurately and efficiently recognition as the speech signal can be well characterized as
convert a speech signal into a text message transcription of a parametric random process and the parameters of the
the spoken words independent of the speaker, environment stochastic process can be determined in a precise, well-
or the device used to record the speech [7]. defined manner.
Speech B. Fluent speech Recognition
Fluent speech recognition is a more complicated task than
isolated word recognition. In this case the task is to
Feature Extraction recognize a continuous string of words from the vocabulary.
C. Feature Extraction and Pattern Recognition
Acoustic Language The input into an automatic speech recognition system is the
Feature Modeling speech signal. The two major tasks involved in speech
ModelingDecoding recognition are feature extraction and pattern recognition.
Feature Extraction
In all speech recognition systems the first step in the process
Recognized Text is signal processing. Initially a spectral / temporal analysis
of the speech signal is performed to give observation vectors
Text which can be used to train the HMMs. One way to obtain
Speech Retrieval Machine Translator observation vectors from speech samples is to perform
spectral analysis. A type of spectral analysis that is often
used is linear predictive coding.
Fig1: Main Components of anTranslator
Automatic Speech
Recognition System Pattern Recognition
A. Feature Extraction Pattern recognition refers to the matching of features. The
pattern recognition process consists of training and testing.
Signal processing techniques are applied to the speech During training, a model of each vocabulary word must be
signal in order to dig out the features that distinguish created. Each model consists of a set of features extracted
different phonemes from each other. from the speech signal. The exact form of the model
B. Acoustic modeling depends on the type of pattern-recognition algorithm used.
It provides probabilities for different phonemes at different During testing, a similar model is created for the unknown
word [3]. The pattern-recognition algorithm compares the
time instants. It is the statistical mapping from the units of
speech to all the features of speech. These are used for model of the unknown word with the models of known
speech sounds to phoneme and from phoneme to word. words and selects the word whose model score is highest.
There are many different pattern matching techniques.

Published By:
72 Blue Eyes Intelligence Engineering
& Sciences Publication Pvt. Ltd.
 International Journal of Engineering and Advanced Technology (IJEAT)
ISSN: 2249 – 8958, Volume-3, Issue-3, February 2014

These include templates, Dynamic Time Warping and C. Types of HMMs

HMMs. There are many different types of hidden Markov models. In
the ergodic or fully connected HMM, every state of the
V. HIDDEN NEURAL NETWORKS model can be reached (in a single step) from every other
A general framework for hybrids of hidden Markov models state of the model. Inspeech processing, the left-right model
(HMMs) and neural networks (NNs) called hidden neural or Bakis model has been used. The benefit of this model is
networks (HNNs) [16]. A neural network model consists of that it can model signals whose properties change over time
units with associated basis or activation functions, which are [7] [16].
connected through weights. These units are arranged in
layers. Although ANN structures like the radial basis
function networks exist where the parameters of the
activation functions are estimated during training, the ANNs
used in this thesis use fixed activation functions. The
connecting weights are the free parameters to be estimated.
Every ANN has an input layer with one unit for each
dimension of the observation vector and an output layer with
one unit for each target class. Within these layers one or
more layers are included, which have no particular desired
behavior and have no direct connection to the outer world.
For this reason they are called hidden layers. Hidden units Fig.2: Illustration of 2 types of HMMs. (a) A 4-state ergodic
are able to discover regularities in the data and enrich the model. (b) A 4-state left-right model
family of functions the network is able to approximate [7].
VI. CONCLUSION
A. Hidden Markov Model Automatic Speech Recognition is the challenging problem
Hidden Markov models are special cases of neural networks to deal with. In this paper, we discussed the various
[16]. Despite huge amounts of research trying to create an techniques of Automatic Speech Recognition and Hidden
intelligent speech recognition machine, we are far from Markov Model (HMM) of how the technology has
achieving the desired goal of a machine that can understand progressed from the last years. ASR is more than automatic
spoken discourse on any subject by all speakers in all text to speech; ASR requires fast computers with lots of data
environments. To date, the best results in speech recognition capacity and memory a necessary condition for complex
systems have been achieved by those based on hidden recognition tasks, and the involvement of speech scientists,
Markov models. Hence, most current automatic speech linguists, computer scientists, mathematicians, and
recognition systems are based on HMMs. engineers.
B. Three Fundamental Problems of HMM design
REFERENCES
HMM design is characterized by three fundamental [1]. Giuseppe Riccardi, Dilek Hakkani-tur, “ Active learning: theory and
problems namely: applications to Automatic speech recognition,” IEEE transactions on
 The evaluation of the probability of a sequence of speech and audio processing, vol. 13, no. 4, july 2005.
observations given a specific HMM. [2]. Santosh K.Gaikwad, Bharti W.Gawali, “A review on speech
recognition technique,” International Journal of computer
 The determination of a best sequence of model states. applications (0975 – 8887) Volume 10– no.3, November 2010.
 The adjustment of model parameters to best account for [3]. Thomas Hain, Asmaa El Hannani, “Automatic Speech Recognition
the observed signal. for scientific purposes – webasr,”2008 ISCA, september 22-26,
brisbane australia.
There are various methods for solving the above problems. [4]. Parwinderpal Singh, Er. Bhupinder Singh, “ Speech recognition as
The most popular technique used to solve problem 1. emerging revolutionary technology,”International Journal of
The Forward-Backward procedure, is an algorithm for advanced research in computer science and software engineering
computing the probability of a particular observation Volume 2, issue 10, october 2012 issn: 2277 128x.
[5]. Wiqas Ghai, Navdeep Singh, “Analysis of automatic speech
sequence. recognition systems for indo-aryan languages: Punjabi a case study,”
The Viterbi algorithm is a dynamic programming algorithm International Journal of Soft computing and engineering (IJSCE)
for finding the most likely sequence of hidden states that issn: 2231-2307, volume-2, issue-1, march 2012.
results in a sequence of observed events. This algorithm is a [6]. Preeti Saini, Parneet Kaur, “Automatic Speech Recognition: a
review,” International Journal Of Engineering Trends And
popular technique for solving problem 2, that of finding the Technology- Volume4issue2- 2013.
best state sequence for the given observation sequence [15]. [7]. Mohit Dua, R.K.Aggarwal, “ Punjabi Automatic Speech
The third and most difficult problem in the design of HMMs Recognition using htk,” IJCSI international journal of computer
is the problem of determining a method to maximise the science issues, vol. 9, issue 4, no 1, july 2012 issn (online): 1694-
0814.
probability of the observation sequence given the model. As [8]. Wiqas Ghai, Navdeep Singh, “Literature review on automatic
Rabiner mentions, there is no known way to analytically speech recognition,” international journal of computer applications
solve this problem, neither is there an optimal way of (0975 – 8887) volume 41– no.8, march 2012.
estimating the model parameters. There are various iterative [9]. I. A. Muslea, “active learning with multipleviews,” ph.d.
Dissertation, Univ. Southern california, los angeles, 2000.
procedures such as the Baum-Welch method, and [10]. M. Tang, x. Luo, and s. Roukos, “active learning for statistical
expectation modification method or gradient techniques that natural Language parsing,” in proc. 40th anniversary meeting of
can be used to choose model parameters. The standard association For computational linguistics (acl-02), philadelphia, pa,
criterion for estimation of HMM parameters is maximum 2002, Pp. 120–127.
[11]. G. Riccardi and d. Hakkani-tür, “active and unsupervised learning
likelihood. for Automatic speech recognition,” in proc. Eurospeech, 2003.

Published By:
73 Blue Eyes Intelligence Engineering
& Sciences Publication Pvt. Ltd.
 Automatic Speech Recognition: A Review

[12]. M.a.anusuya , s.k.katti “speech recognition by machine :A review”

international journal of computer science and Information security
2009.
[13]. Samudravijay k “speech and speaker recognition report”
Source:http://cs.jounsuu.fi/pages/tkinnu/reaserch/index.html Viewed
on 23 feb. 2010.
[14]. Sadokifuruki ,Tomohisa ichiba, “Cluster-based modeling For
ubiquitous speech recognition, Tokyo institute of technology
Interspeech 2005.
[15] Spector, Simon kinga and Joe Frankel, “Recognition ,speech
Production knowledge in automatic speech recognition ,Journal of
acoustic society of america,2006.
[16]. Ranu Dixit,Navdeep Kaur, “Speech Recognition Using Stochastic
Approach: A Review,” International Journal of Innovative Research
in Science, Engineering and Technology Vol. 2, Issue 2, February
2013.

Author 1: Jasmeen Gill, a dynamic lecturer. She has completed M. Tech

and B Tech in Computer Science Engineering having an experience of 8
years. She has published 3 national and international papers. Her areas of
interest are Artificial Intelligence and Image Processing. She has a
membership of IEEE.

Author 2: AmanPreet Kaur, an effectual lecturer has an experience of 10.5

years. She has completed M. Tech and B Tech in Computer Science
Engineering and pursuing P.hd. She has published 10 national and
international papers and pursuing research work on Punjabi Speech
recognition. She has a membership of IEEE.

Author 3: Anchal Katyal has completed B.Tech and Pursuing M.Tech.She

has published four national and international papers.She has a membership
of IEEE.

Published By:
74 Blue Eyes Intelligence Engineering
& Sciences Publication Pvt. Ltd.