A REPORT

ON

Hardware Implementation of Audio Processing

Algorithms
BY

Soumya Ganguly ID: 2010A3PS590H

BIRLA INSTITUTE OF TECHNOLOGY & SCIENCE, PILANI

HYDERABAD CAMPUS

Under the supervision of

Dr. Amlan Chakrabarti

A.K. Choudhury School of Information

Technology, University of Calcutta

1|Page
CERTIFICATE

This is to certify that the report entitled Hardware Implementation of Audio

Processing Algorithms and submitted by SOUMYA GANGULY at the
A.K.Choudhury School of Information Technology, University of Calcutta
embodies the work done by him under my supervision.

(Signature of the supervisor)

Dr. Amlan Chakrabarti

2|Page
ACKNOWLEDGEMENTS

I would express my gratitude to my instructor Prof. Amlan Chakrabarti for

giving me an opportunity to work on this project and his whole hearted support,
guidance and motivation throughout the project. I am thankful to Mr. Chandrajit
Pal and Mr. Tanmay Biswas for their continuous feedback and instructions
which helped me to explore the vast area of Audio Processing on Hardware and
make improvements in my project.

I would also like to thank all the people at the A.K.Choudhury School of
Information Technology for accommodating me and their support during the
tenure of my project.

3|Page
ABSTRACT

This project aims to come up with novel algorithms and efficient hardware
implementations of signal processing front ends for automatic speech
recognition. The initial phase of the project has been completed. The
architectures of various Field Programmable Devices were studied and the
architecture of the Xilinx Field Programmable Gate Arrays (FPGA) was
analyzed in detail. After having learnt to use the Xilinx System Generator Tool
for DSP, I am currently implementing existing signal processing algorithms on
FPGA. The Hardware Description Language being used is VHDL. In the future,
we plan to implement several novel algorithms related to feature extraction on
hardware and determine their advantages.

4|Page
TABLE OF CONTENTS

 Introduction................................................................................................6
 Programmable Logic..................................................................................7
 Architecture of FPGAs.............................................................................10
 Special FPGA functions...........................................................................11
 Xilinx System Generator Tool for DSP...................................................11
 Speech Recognition Frontend..................................................................12
 Proposed Feature Extraction Algorithm..................................................12
 Hardware Implementation of Audio Processing Algorithms.................14
 Conclusion and Future Work...................................................................15
 References................................................................................................16

5|Page
INTRODUCTION
Audio Processing has been an active area of research for many years. Audio
Processing, especially speech recognition, finds widespread applications in
voice dialing on cell phones, robotics and desktop software packages. Although
commercial speech recognizers are still extremely limited in their abilities, the
very best research systems are now approaching their ultimate goal—large-
vocabulary, continuous, speaker independent, real-time recognition. These
systems are large vocabulary in that they can handle on the order of 60,000
words; continuous in that they recognize natural human speech, spoken without
deliberate pauses; and speaker-independent in that no user-specific training is
required [1]. Unfortunately, these systems are also extremely computationally
intensive, requiring the full processing resources of a modern desktop machine
in order to run in real-time. This completely rules out high-quality speech
recognition for many of the applications where one might want it most—in
particular, for mobile applications.

With advances in VLSI technology, it has become possible to incorporate these

algorithms in hardware, and thus, liberate speech recognition from the
unreasonable limitations of software. Accordingly, this project aims to explore
the implementation of various signal processing algorithms on Hardware and
come up with novel designs for efficient hardware implementations of signal
processing front ends for automatic speech recognition. The architectures of
Xilinx FPGAs and the hardware description language VHDL is studied in
detail. The Xilinx System Generator Tool for DSP is subsequently learnt and
the existing signal processing architectures are then downloaded on a Xilinx
FPGA. In the future, we plan to implement several novel algorithms related to
feature extraction on hardware and compare them with these existing designs
and thus, determine the advantages of these novel designs.

6|Page
Programmable Logic
Programmable logic can be defined as an integrated circuit that can be
programmed /reprogrammed with a digital logic of a certain level. The concept
of programmable logic first originated in the late 70s and the field is constantly
growing thereafter. Some of the advantages that programmable logic offers are:

Short Development time

Reconfigurable

Saves board space

Flexible to changes

No need for ASIC expensive design and production

Fast time to market

Bugs can be fixed easily

Of the shelf solutions are available

Flexibility

Processors
Instruction Flexibility
90% Area Overhead
(Cache , Predictions)

FPGA
Device-wide flexibility
99% Area Overhead
(Configuration)

ASIC
No Flexibility
20% Area Overhead
(Testing)

Speed , Power Efficiency

7|Page
Some of the earlier programmable logic circuits were:

PLA — a Programmable Logic Array (PLA) contains two levels of logic, an

AND-plane and an OR-plane, where both levels are programmable. It was well
suited for implementation of logic functions in Sum of Product form. However,
the two levels of configurable logic produced significant propagation delays and
programmable logic planes were difficult to manufacture. Hence, it suffered
from poor speed performance and was expensive.

PAL — a Programmable Array Logic (PAL), a relatively small field

programmable device, has a programmable AND-plane followed by a fixed
OR-plane. It was manufactured by Advanced Micro Devices.

SPLD — Simple Programmable Logic Device (SPLD) contains a Single AND

Level and also consists of Flip-Flops and feedbacks.

8|Page
CPLD — a more Complex PLD that consists of an arrangement of multiple
SPLD-like blocks on a single chip. Alternative names sometimes adopted for
this style of chip are Enhanced PLD (EPLD), Super PAL, Mega PAL, and
others.

PLDs had the option of being programmed in batches in a factory or in the field
(field programmable). However, one of their disadvantages was that
programmable logic was hard-wired between logic gates.

In 1985, Xilinx co-founders Ross Freeman and Bernard Vonderschmitt invented

the first commercially viable field programmable gate array– the XC2064. The
XC2064 had programmable gates and programmable interconnects between
gates, and this was the beginning of a new technology and market. Field-
Programmable Gate Arrays feature a general structure that allows very high
logic capacity. Whereas CPLDs feature logic resources with a wide number of
inputs (AND planes), FPGAs offer more narrow logic resources. FPGAs also
offer a higher ratio of flip-flops to logic resources than do CPLDs.

9|Page
 Architecture of FPGAs
The field programmable gate arrays are mainly composed of:

1) Programmable logic blocks: These blocks implement combinational and

sequential logic. They are based on Look Up Tables and D-flip flops. In
Xilinx FPGAs, they are known as configurable logic blocks.

y
a

b N Input
LUT
SET
c MUX D Q q

d
CLR Q
clk

rst

2 Programmable Input/output blocks: These are Configurable I/Os for

external connections and support various voltages and tri-states.

3 Programmable interconnect: These consist of wires to connect inputs,

outputs and logic blocks. They also include clocks, short distance local
connections and long distance connections across chip.

10 | P a g e
Special FPGA Functions
Modern FPGAs often include higher level functionality embedded into silicon.
Some of the specialized functions include:

Internal SRAM

Embedded Multipliers and DSP blocks

Embedded logic analyzer

Embedded CPUs

High speed I/O (~10GHz)

DDR/DDRII/DDRIII SDRAM interfaces

PLLs

These functions are directly implemented (not through LUTs, as is the case with
normal combinational circuits).

The DSP block is a highly useful function that is extensively used in my project.
Various signal processing algorithms like Fast Fourier Transform, Filtering,
MAC blocks etc. can be easily and efficiently implemented through this block.
One of the most useful tools for the implementation of DSP algorithms on
Xilinx FPGAs is the Xilinx System Generator Tool.

Xilinx System Generator Tool for DSP

System Generator for DSP™ is the industry’s leading high-level tool for
designing high-performance DSP systems using FPGAs. [5] It can be employed
to:

 Develop highly parallel systems with the industry’s most advanced FPGAs
 Provide system modeling and automatic code generation from Simulink® and
MATLAB
 Integrate RTL, embedded, IP, MATLAB and hardware components of a DSP
system

11 | P a g e
Some of its key features are:

 DSP modelling
 Bit and cycle accurate floating and fixed-point implementation
 Automatic code generation of VHDL or Verilog from Simulink
 Hardware co-simulation
 Xilinx Power Analyzer (XPA) Integration
 Hardware / software co-design of embedded systems

Speech Recognition Frontend

In a typical automatic speech recognition system, a signal processing front-end
transforms the speech waveform from an input device such as a microphone to a
parametric representation. This parametric representation, often referred to
as”features”, is then used to drive the speech recognition decoder process. Two
of the most widely used features are Mel Frequency Cepstral Coefficients and
Linear Prediction Cepstral Coefficients.

Proposed Feature Extraction Algorithm

This proposed algorithm calculates Mel-Frequency Cepstral Coefficients and
Linear Predictive Coding Coefficients simultaneously and their weighted sum is
used for feature extraction (new coefficients). The weights assigned to each of
these co-efficients have to be determined experimentally for different
implementations. Matlab(software) implementations have been found to give
best results when the coefficients are approximately 0.6 for MFCC and 0.4 for
LPC. The complete diagram detailing the feature extraction steps are given
below.

12 | P a g e
13 | P a g e
Hardware implementation of Audio
Processing Algorithms
The acoustic front end is all DSP. Therefore, DSP algorithms such as FFT,
DCT, filtering, windowing etc. are often encountered. The System Generator
tool comes in handy in these situations and hence is extensively used. The
behavioural, structural and dataflow styles of modelling of the VHSIC
Hardware Description Language (VHDL) are often combined in the various
designs. Several improvements have been suggested and work on them is
underway. These improvements include:

1) Use of the same FFT block for calculating the FFT (Needed for MFCC) as
well as Auto-Correlation Coefficients (needed for LPC) employing the Wiener–
Khinchin theorem, thus saving considerable amount of Hardware.

2) Subsequent use of Split-Levinson algorithm and a folded architecture for the

extraction of LPC formants. (I’m working on HW implementation of superfast
algorithms which have “taking an FFT” as their first step, for solving toeplitz
systems. If such an algorithm could be realized on Hardware, not only will the
toeplitz system be solved faster, but also the extra Inverse FFT calculation in the
first step can be avoided ) Thus, significant reduction in delay will occur.

3) Use of rectangular filter bank for MFCC, which will completely avoid
multiplication and will require less memory when compared to the existing
designs.

4) Use of weighted sum of MFCC and LPCC for template matching using
HMM / DTW / ANN. Results from the software implementation of the design
in Matlab have shown that at approx. 0.6*MFCC + 0.4*LPCC, the maximum
accuracy is achieved. Therefore, this particular weighted sum of the coefficients
is implemented using a carry-save adder.

We are currently working on these designs and hope to implement them on

hardware soon. A small part of the design implemented using Xilinx DSP
Blockset elements is shown below.

14 | P a g e
Conclusion and Future Work
This project on the Hardware implementation of Audio Processing Algorithms,
though still in its initial stage, has helped me a lot in understanding the internal
architecture of FPGAs and the implementation of various Signal Processing
Algorithms on them. I have also learnt the basics of VHDL and have become
familiar with the Xilinx System Generator Tool for DSP. In future, we intend to
implement on hardware, several novel algorithms related to Speech Feature
Extraction that I have developed and determine their advantages over the
existing architectures.

15 | P a g e
References
[1] Lin, Edward C., Kai Yu, Rob A. Rutenbar, and Tsuhan Chen. "Moving
speech recognition from software to silicon: the in silico vox project." In
INTERSPEECH. 2006.

[2] Serge Karabchevsky “Programmable logic and FPGA”

[3] Bhasker, Jayaram, and Jayaram Bhasker. A VHDL Primer. Prentice Hall
PTR, 1999.

[4] Brown, Stephen, and Jonathan Rose. "FPGA and CPLD architectures: A
tutorial." Design & Test of Computers, IEEE 13, no. 2 (1996): 42-57.

[5] Xilinx System Generator for DSP User Guide, r10.1.1, April 2008

16 | P a g e