REPORT ON

TUNETYPE: CLASSIFYING SOUNDS WITH NEURAL

NETWORKS

Submitted By

Nidhi Sakhare – 06
Kanak Arora - 20

Under the Guidance of

Prof. Shubhangi Shambharkar

Department Of Artificial Intelligence And Data Science

YESHWANTRAO CHAVAN COLLEGE OF ENGINEERING,

NAGPUR

SESSION 2024-2025

1
 Description of the Used Tool/Framework

a. Overview of the Deep Learning Tool or Framework

In this project, we utilized TensorFlow, an open-source deep learning framework developed

by Google Brain. TensorFlow is designed to handle large-scale machine learning and deep
learning tasks. It provides a flexible platform for implementing various neural networks, and
it supports both research and production environments.
TensorFlow’s ease of use with Keras API allows fast prototyping and provides several pre-
built modules that help in handling complex tasks such as image recognition, natural
language processing, and, as in this project, sound classification. TensorFlow’s
comprehensive ecosystem also includes tools for deployment in mobile and web applications,
making it suitable for extending the project's future goals.

o TensorFlow: Developed by Google Brain in 2015, TensorFlow is widely used

for both research and production purposes. It supports deep learning models
for tasks like image classification, natural language processing, and sound
classification.
o PyTorch: Released by Facebook's AI Research lab (FAIR) in 2016, PyTorch is
known for its dynamic computation graphs, which allow flexibility in model
building and debugging. It is popular among researchers and widely used in
academia for experimentation.
o Keras: Keras, originally an independent library, is now a part of TensorFlow
as its high-level API. It simplifies the process of building neural networks and
is known for its ease of use and fast prototyping capabilities.

b. Why This Tool Was Chosen

TensorFlow was chosen for its high efficiency, ease of scalability, and powerful
debugging tools like TensorBoard. The framework supports multiple GPUs and
distributed computing, which is crucial when training on large datasets like audio
classification tasks. The compatibility with Keras also simplifies the model-building
process, enabling quick experimentation with different architectures.

2
 Additionally, TensorFlow has extensive community support, making it easier to find
documentation and relevant resources for audio classification.

o TensorFlow: It's a comprehensive end-to-end platform that provides a robust

set of tools for deep learning projects. TensorFlow's wide array of pre-built
modules, libraries for handling large datasets, and deployment capabilities
(such as TensorFlow Lite) make it suitable for production-level audio
classification tasks.
o PyTorch: PyTorch is often preferred for projects involving heavy
experimentation due to its ability to dynamically alter the computational graph
during training. In audio classification, it allows for easier debugging, testing,
and fine-tuning.
o Keras: Keras, emphasize its simplicity, which allows you to quickly build
models and experiment with different architectures. It's also tightly integrated
with TensorFlow, offering the same scalability benefits.
 Comparison:
o TensorFlow vs PyTorch: While TensorFlow offers a wider production
ecosystem, PyTorch's dynamic graph creation makes it easier for beginners
and those focusing on research. TensorFlow’s static graph approach ensures
efficient computation at scale.
o Why Not MXNet or CNTK?:
MXNet is a highly flexible and efficient deep learning framework, particularly suited for
distributed computing and dynamic computation graphs. While MXNet offers several
advantages, there are key reasons why it may not be the preferred choice:

 Community and Ecosystem Support: While MXNet is efficient, the support

community is significantly smaller compared to TensorFlow or PyTorch. This can
make it harder to find tutorials, troubleshooting resources, and advanced tools,
particularly for specific domains like audio classification.
 Ecosystem for Audio Data: TensorFlow and PyTorch have more extensive libraries
and APIs designed for audio processing and neural network experimentation.
MXNet's ecosystem, though robust, lacks the extensive integration for audio-related

3
 tasks. For example, TensorFlow offers TensorFlow Audio, a dedicated library for
handling and preprocessing audio data, which simplifies the workflow.
 Popularity and Longevity: TensorFlow is widely adopted in both academic and
industrial settings. As a result, it has more frequent updates, better cross-platform
support, and a larger collection of pretrained models, which can be useful for transfer
learning. MXNet, while powerful, has fewer pretrained models in audio domains and
less integration into research papers, making TensorFlow a more suitable choice for
research and production.

CNTK is another strong contender in the deep learning framework space, known for its
efficiency, particularly in large-scale applications like speech recognition and natural
language processing. However, several factors may make TensorFlow a better fit for this
project:

 Focus on Speech over General Audio: While CNTK has proven successful in
speech recognition tasks (e.g., Microsoft's Cortana), it is more specialized toward
speech data, which is slightly different from the general urban sound classification
your project focuses on. TensorFlow, on the other hand, has more generalized support
across diverse types of audio classification problems.
 Lower Adoption Rate: Although CNTK is highly optimized for specific tasks, its
overall adoption rate is lower compared to TensorFlow. This results in fewer public
resources, tutorials, and pretrained models that could accelerate development.
 Tooling and Visualization: TensorFlow provides powerful tools such as
TensorBoard, which is crucial for visualizing complex models and debugging the
training process. While CNTK offers some debugging tools, the visualization
experience with TensorFlow is more mature, making it easier to track the model's
performance across epochs.
 Cross-platform Capabilities: TensorFlow supports a wide range of deployment
environments, from mobile apps (via TensorFlow Lite) to web apps (via
TensorFlow.js), which is beneficial if you want to deploy your sound classifier to
different platforms in the future. CNTK is relatively limited in this area, making
TensorFlow more future-proof for projects that require real-time sound classification
on diverse devices.

4
 Description of the Data Used

a. Detail the Dataset Selected

For this project, we used the UrbanSound8K dataset, which is widely used in urban sound
classification tasks. The dataset contains a total of 8,732 labeled sound excerpts collected
from a variety of urban settings.
Each sound excerpt is stored as a WAV file and has a duration of up to 4 seconds. The dataset
provides audio clips with different environmental noise levels, which makes it particularly
challenging for machine learning models to distinguish between classes with overlapping
frequency ranges (e.g., car horns and sirens). This dataset is designed for use in audio
classification tasks and helps develop models that can identify sound events in real-world
noisy environments.
The data is well-suited for this project due to its balance across sound categories, ensuring
that the model has an equal opportunity to learn distinguishing patterns across all types of
urban sounds. Additionally, since the sound events occur in varying environments, this
dataset introduces the model to real-world noise, improving its robustness.

b. Preprocessing Steps
Preprocessing audio data is an essential step before feeding it into the neural network,
especially when using techniques like Convolutional Neural Networks (CNNs). Below are
the key preprocessing techniques used:
1. Resampling:
o Audio files in the UrbanSound8K dataset have varying sample rates, which
can introduce inconsistencies when training a neural network. To ensure
uniformity, all audio files were resampled to 16 kHz. This is a common
practice in audio processing, as it standardizes the data and reduces the size of
the input without losing significant audio detail for classification.
2. Trimming/Padding:
o Not all audio clips in the UrbanSound8K dataset are exactly the same length,
with some being shorter or longer than the target duration of 4 seconds. To
handle this variability, we applied trimming and padding techniques:

5
  Trimming: If an audio clip exceeded 4 seconds, it was trimmed to the
desired length to maintain consistency.
 Padding: If an audio clip was shorter than 4 seconds, it was padded
with zeros (silence) to ensure that all clips had equal length. This is
important as varying lengths would otherwise complicate model input
handling.

3. Spectrogram Generation:
o Since raw waveforms are difficult for CNNs to interpret, the audio signals
were transformed into Mel Spectrograms. A Mel Spectrogram is a visual
representation of the audio where the x-axis represents time and the y-axis
represents frequency, with color intensity depicting the amplitude of
frequencies over time.
o Mel Spectrograms were chosen because they closely resemble image data,
allowing CNNs to extract meaningful features using their strong pattern
recognition capabilities. The Mel scale is perceptually relevant for humans,
meaning that it aligns with how we perceive pitch differences, which improves
model performance.

4. Normalization:
o To ensure numerical stability during training, the raw waveform data was
normalized. All input data values were scaled to a range between -1 and 1,
which helps prevent exploding gradients and ensures that all audio data is
treated uniformly by the network. This is especially crucial when using deep
learning models with gradient-based optimization techniques.
These preprocessing steps ensure that the dataset is ready for input into the neural network,
reducing noise, standardizing the data, and converting the raw audio into a format that the
model can easily interpret and learn from.

Analysis of Why the Selected Deep Learning Technique is Suitable

a. Justify the Choice of Deep Learning Technique

6
For the project TuneType: Classifying Sounds with Neural Networks, we opted to use
Convolutional Neural Networks (CNNs) as the primary deep learning architecture. The
choice of CNNs is grounded in several compelling reasons:
 Success with Spatial Data: CNNs have shown remarkable success in various tasks
involving spatial data, such as image recognition and classification. Given that Mel
Spectrograms of audio data are essentially 2D representations similar to images,
CNNs are particularly well-suited for processing this type of data. They leverage
convolutional layers to efficiently capture local patterns, hierarchies, and spatial
features that are critical for distinguishing between different sound categories.
 Feature Extraction: One of the defining characteristics of CNNs is their ability to
automatically learn hierarchical features from the input data. In the case of Mel
Spectrograms, CNNs can identify important features such as pitch, timbre, and
frequency patterns across both time and frequency domains. This ability to extract
relevant features significantly reduces the need for manual feature engineering,
allowing the model to adaptively learn complex representations necessary for accurate
classification.
 Efficiency and Speed: CNNs are generally faster than other architectures, such as
Recurrent Neural Networks (RNNs), especially when working with 2D data. They
can process multiple parts of the input simultaneously due to their parallel
architecture, making them ideal for handling larger datasets and improving training
times. This speed is advantageous in audio classification tasks, where processing
efficiency can greatly impact the overall workflow.
 Combining CNNs with LSTMs: Although CNNs alone are effective, we also
explored the potential of integrating Long Short-Term Memory networks (LSTMs)
with CNNs to leverage their strengths. LSTMs are specifically designed to handle
sequential data and are capable of capturing long-range dependencies. By combining
CNNs and LSTMs, the model can not only identify spatial features from the
spectrograms but also learn temporal patterns in the audio sequences. This hybrid
approach allows the model to account for the time-dependent nature of sound events,
enhancing its ability to classify sounds accurately.
 Robustness to Noise: Urban environments are often characterized by background
noise and overlapping sounds. CNNs are inherently more robust to such variations
compared to RNNs, which may struggle to maintain stability in noisy environments.

7
 The localized feature extraction in CNNs helps focus on critical patterns while
minimizing the influence of irrelevant noise.

b. Discuss the Expected Outcomes and the Problem Being Addressed

The primary goal of this project is to accurately classify urban sound data into distinct
categories. The expected outcomes can be outlined as follows:
 High Classification Accuracy: We anticipate that the CNN-LSTM model will
achieve high accuracy in classifying the audio clips from the UrbanSound8K dataset.
By effectively capturing both temporal and spatial patterns from the Mel
Spectrograms, the model should demonstrate an ability to distinguish between various
sound categories accurately.
 Feature Learning: The model is expected to learn features that are representative of
each sound class, enabling it to generalize well to unseen audio samples. As a result, it
should be able to correctly identify sounds in real-world urban settings, providing
insights into the auditory landscape.
 Challenges with Similar Sound Classes: While we expect the model to perform well
overall, we also recognize the inherent challenges in distinguishing between similar
sound classes. For instance, sounds like sirens and car horns may share overlapping
frequency ranges, complicating the classification task. These similarities can lead to
misclassification, which is a common challenge in audio recognition systems.
Addressing these challenges is crucial to improving the model's performance.
 Potential Insights into Urban Sound Dynamics: Beyond classification accuracy, the
project aims to provide insights into urban sound dynamics by understanding the
relationships between different sound categories. For instance, we may explore how
certain sounds tend to co-occur in specific urban contexts, which can inform city
planning, noise management, and public safety measures.

Screenshots of the Implementation

1. Importing Libraries:-

8
2. Reading CSV File:-

9
3. Plotting Raw Wave Files:-

4. Individual Audio Files and their visuals:-

5. Spectograms:-

10
6. Spectral Rolloff:-

11
7. Chroma Feature:-

8. Zero Crossing Rate:-

12
 9. Building the model:-

10. Model evaluation:-

Results

a. Present the Results Obtained from the Implementation

The performance of the sound classification model was assessed using several key
metrics, including accuracy, precision, recall, and the F1 score.

13
  Accuracy: The model achieved an accuracy of XX% on the test set. This
accuracy level demonstrates a competitive performance relative to state-of-the-
art models in the domain of sound classification. The model's ability to
generalize across various urban sound categories indicates effective feature
extraction from Mel Spectrograms.
 Precision and Recall: Precision measures the accuracy of the positive
predictions, while recall indicates the ability of the model to identify all relevant
instances within a category. The calculated precision and recall for each class
provide insight into the model's performance and its tendency to misclassify.
 F1 Score: The F1 score, which balances precision and recall, offers a
comprehensive view of the model’s accuracy across different classes. A higher
F1 score indicates better overall performance.
The results from these metrics can be summarized in a table format:
Metric Value

Accuracy XX%

Precision (Class 1) XX%

Recall (Class 1) XX%

F1 Score (Class 1) XX%

Precision (Class 2) XX%

Recall (Class 2) XX%

F1 Score (Class 2) XX%

... ...
This table should continue for each class to give a detailed view of the model's
performance.
c. Tables and Graphs
The training and validation accuracy and loss curves provide visual insight into
the model's learning process over the epochs. A well-performing model should
show increasing accuracy and decreasing loss over time.
d. Analysis of Misclassifications

Analyzing misclassifications provides insight into the model’s weaknesses.

For example, in our implementation, we noticed that the model often struggled to
differentiate between:

14
  Car Horns and Sirens: Both sound types produce similar frequency components,
leading to overlapping features in the Mel Spectrogram. This misclassification can
occur due to their proximity in sound characteristics, which can confuse the CNN.
 Children Playing and Street Music: While these sounds belong to different
categories, they can share similar environmental noise components, leading to
potential confusion in classification.

Future Improvements

To enhance model accuracy, especially in cases of misclassification, several strategies could

be employed:

1. Advanced Data Augmentation: Techniques such as pitch shifting, time stretching,

and adding background noise can create a more diverse training set, helping the model
generalize better across different sounds.
2. Ensemble Learning: Combining multiple models can leverage the strengths of each,
potentially reducing misclassifications and improving overall performance.
3. Hyperparameter Tuning: Conducting extensive hyperparameter tuning can help
identify optimal model configurations, enhancing classification accuracy.
4. Transfer Learning: Utilizing pre-trained models on similar tasks may provide a
stronger foundation for feature extraction and improve performance in sound
classification.
5. Larger Datasets: Expanding the dataset with more diverse examples could help the
model learn finer distinctions between similar classes.

By addressing these areas, the model can be further refined to achieve higher accuracy and
robustness in classifying urban sounds.

Conclusion with Interpretation of Results

a. Summarize the Key Findings

In this project, we successfully built a CNN-based sound classification model using the
UrbanSound8K dataset. The use of Mel Spectrograms as input data proved to be effective,
and the model demonstrated competitive performance, achieving high accuracy in classifying
urban sounds.

b. Interpret the Results and Discuss Their Significance

The results show that deep learning models, particularly CNNs, can effectively learn from
spectrogram representations of audio data. Our model performed well on most sound
categories, although there were some misclassifications in overlapping categories. This
suggests that with further tuning or more complex architectures, the model can achieve even
better performance.

c. Future Work

15
For future improvements:

 Data Augmentation: Introduce more advanced augmentation techniques, such as

time-stretching or adding environmental noise, to make the model more robust to real-
world variations.
 Model Enhancements: Explore hybrid architectures, such as combining CNNs with
LSTMs or Transformers, to better capture both spatial and temporal features in the
sound data.
 Deploying the Model: Consider deploying the model on edge devices for real-time
sound classification in smart city applications.

16