Presented to the

DEPARTMENT OF ELECTRONICS ENGINEERIN G

College of Engineering and Architecture

FILIPINO SIGN LANGUAGE ALPHABET

In Partial Fulfilment

OF THE REQUIREMENT FOR THE SUBJECT

By
A6
BONETE, CYNTHIA O.
CARLOS, ARIOLLE JEIY
DIMATULAC, ALAN LEEUWEN

DECEMBER 2024

1
 TABLE OF CONTENTS

ABSTRACT…………………………………………………………………………………………… 3

INTRODUCTION………………………………………………………………................................... 4

THEORIES……………………………………………………………………………………………. 4

PROJECT OBJECTIVE…………………………………………….....................................................5

METHODOLOG………………………………………………………………………………………6

DATA GATHERED, PROOF, AND ANALYSIS…………………………………………………8

LEARNINGS…………………………………………………………………………………………9

CONCLUSION……………………………………………………………………………………….10

RECOMMENDATION………………………………………………………………………………10

SOURCE CODE………………………………………………………………………………………11

STEPS ON HOW TO DEPLOY THE PROJECT………………………………………………….13

REFERENCES………………………………………………………………………………………..16

2
 ABSTRACT

Communication is an essential thing for our everyday life, and if someone has a communication
barrier from the people around them, they truly suffer from miscommunication and sometimes can’t keep
up with others. That is why Sign Language was invented and gladly Filipinos have their own version of
it. According to Balanquit (2019) as a language system, Filipino Sign Language (FSL) is a mixture of
unique traditional and regional signs, signs borrowed from American Sign Language (ASL) (including
phonological and semantic variants), and lexicalized signs based on Manually-Coded English (MCE).
Sign Language is a helpful tool for the deaf and hard-of-hearing community, but there is still a
communication barrier between the community and the people who can hear. That is why this project is
proposed. This project Filipino Sign Language Recognition is a tool to help both communities to
communicate. Can be used in facilities like schools, government stations, or even at home. Using the
Convolutional Neural Networks (CNN) model of Deep Learning, the system is made in MATLAB
Software. The system can recognize Filipino Sign Language (FSL) vowels (a, e, i, o, u) and still can
improve and add other alphabets, numbers, or phrases. Through this project, it can help people in both
communities to communicate with each other and break the communication barrier.

3
INTRODUCTION

Sign language is a visual language that uses hand gestures, facial expressions, and body
movements to convey meaning which is used by deaf and hard-of-hearing individuals to communicate
with each other and with hearing individuals who know sign language. This language is also beneficial
for both hearing and deaf people to improve communication, and increase cultural awareness, and career
opportunities.

In the sign language community, each country has different sign languages, and the American
Sign Language (ASL) is the most common sign language used. Moreover, in the Philippines around
1590, the first Filipino Sign Language was first documented in Dulag, Leyte by a priest who was writing
about his encounters with local deaf people. Last January 17, 2022, The Implementing Rules and
Regulations of Republic Act No. 11106, known as the “Filipino Sign Language Act” was released.
(Gatchalian, n.d.)

Unfortunately, not everyone is educated enough in using sign language. People who are deaf and
have hearing difficulties face communication barriers that can lead to misunderstandings, frustration, and
exclusion from the hearing world.

As the world evolves, there are many sources available to help deaf and hard-of-hearing
individuals overcome these challenges or barriers, and because we are now in the modern era with
Artificial Intelligence (AI) and Machine Learning taking over, making a big help in the society, Sign
language recognition is invented. Sign Language Recognition (SLR) uses computer vision and machine
learning techniques to recognize and interpret sign language gestures. Sign language recognition systems
use cameras or other sensors to capture images or videos of sign language gestures, and then use machine
learning algorithms to analyze the data and recognize the gestures. (Sign Language Recognition, n.d.)

The researchers proposed a Sign Language Recognition System, specifically, Filipino Sign
Language (FSL). The purpose of this project is to develop a Filipino Sign Language (FSL) Recognition
model capable of recognizing FSL alphabet vowels (a, e, i, o, u) using a Convolutional Neural Networks
(CNNs) model under Deep Learning in Machine Learning in MATLAB. The data is limited due to lack
of time and preparation.

THEORIES

Under this section are the theories used behind this project. Listed here are the theories and concepts
related to sign language recognition.

Carr's (2021) article regarding the Semiotic Framework concluded that sign language interpretation can
be defined within a semiotic framework, considering the broader field of translation and the unique
aspects of sign language.

4
Dean & Pollard's (2011) Demand-Control Theory addresses the complexities beyond language that affect
sign language interpretation, such as environmental dynamics and the interpreter’s cognitive process.

Limaye et al., (2022) research provides a system that is based on Computer vision and Convolutional
Neural Networks, an end-to-end application is created to provide a customizable solution for each user
with a graphic user interface.

PROJECT OBJECTIVES

The objective of this project is to help disabled people easily identify vowels without misinterpreting the
hand gestures of the presenter. These problems are certain every time that the hand gesture of the
presenter is unable to distinguish people with disability, that is why this solution was proposed to help
with the problem being dealt with. This can also be a way for them to be more interactive and able to
communicate with other people not just personally but also through other digital platforms.

5
METHODOLOGY

a. Data Collection

Step 1: For this step, we have created our dataset to ensure compatibility with MATLAB’s input
layer and to increase accuracy. Using high-resolution images of the sign language gestures, we
have taken 300 pictures of every letter. The background should be white blank and the lighting
conditions are also considered.

Step 2: Data augmentation technique is used like setting the resolution of the image to improve
the robustness of the model.

Step 3: The preprocessed data is used to train the CNN model, which keeps an eye on the training
process to make sure the data is not being over- or underfitted.

Step 4: Deploy the model to recognize the sign language gestures in real-time.

b. Data Preparation/ Data Pre-processing/ Data Cleaning

The data is categorized into folders per letter and processed through MATLAB’s installed
webcam add-on. First, select the folder that you want to put a dataset on, and then by webcam
capturing, steady the hand gesture to the processing area box it will automatically capture 300
copies of the data set. Then, by using the training network, the data will be processed.

c. Choosing and Training the Model

The first model that we proposed is the Bayes Theorem model but we changed it to the Deep
learning algorithm, Convolutional Neural Network (CNN), because of some complications
during the process. CNN is a popular model for sign language recognition which is why we chose
this model. In training the model, since we used MATLAB software, it is the right choice to
choose the CNN model because MATLAB provides a comprehensive environment for building
and training CNN models for sign language recognition. We just install and use MATLAB’s Deep
Learning Toolbox to implement and train the model easily.

d. Model Optimization

The process of adjusting a machine learning model's parameters following a criterion is known
as model optimization. This can be accomplished by fine-tuning the model on the data and
modifying the model's hyperparameters, such as the learning rate, batch size, and number of
layers. Unfortunately, we weren’t able to optimize the program.

6
 e. Model Evaluation

In the model evaluation of the Convolutional Neural Network (CNN) Model, various techniques
such as accuracy, precision, recall, F-1 Score, and confusion matrix. Accuracy measures the
percentage of correctly classified samples, Precision measures the proportion of true positives
among all positive predictions, Recall measures the proportion of true positives among all actual
positives, and F1-Score is the harmonic mean of precision and recall. By using a confusion matrix,
it can evaluate the performance of the model. It is a table that summarizes the number of true
positives, false positives, true negatives, and false negatives that can help identify which classes
are being misclassified and classified correctly.

f. Model Deployment

In this recognition system, the researchers developed the myNet1 dataset which allows the users to
recognize hand gestures into visual aid letters. This model can also add more hand gestures to this
system.

DOCUMENTATION

7
DATA GATHERED, PROOF, AND ANALYSIS

Data gathered

The data that has been gathered is already processed by the input code. This will help the system to determine the
hand gestures that will be recognized.

Testing the outcome of the input being processed.

8
Proofs

Proof of trial and error of the system. This proves that it is accurately tested to make sure that there will be no error
once the system is being used. The expected outcome is the system should determine and recognize each hand
gesture precisely.

Analysis
Analyzing the data that has been presented, the input data should be clear and precise for less error outcomes. Be
accurate with the given hand gestures being stored to avoid misinterpretation with the system that will conduct the
proposed project. In creating this system, one must consider the background, lighting, shadows, and other outside
images that may affect the recognition of the system. It varies with different places, such as walls, and other
backgrounds.

LEARNINGS

● Utilizing Machine Learning, people can have systems that can help them in their everyday lives.
Being an engineer is a big help to the community.
● To create a precise and functional recognition system, data preparation, analysis, and gathering
were essential stages for this project.

9
 ● To train and evaluate the CNN model using machine learning algorithms, the researchers and
developers had to produce a dataset that was compatible with MATLAB's input layer.

CONCLUSION

In conclusion, using machine learning to predict Filipino Sign Language can help the deaf and hard-of-
hearing community to be able to communicate with others and can lessen the communication barrier. This system
can be installed in schools, hospitals, government facilities, and even at home. The data that is been supplied in
the system is Filipino Sign vowels (a, e, i, o, u). Though the original proposal is alphabets due to lack of time
preparation, the developer wasn’t able to supply it. Nonetheless, this project proves its objective and can also be
improved.

RECOMMENDATIONS

● Even though the data supplied for FSL recognition has limits, this effort creates possibilities for
additional advancements in the recognition system. Phrases and body movements can also be
applied.
● Investing in these advanced projects shall be implemented by the government for our country to
be more inclusive to all Filipinos.

10
SOURCE CODE

For the source code, there are 3 parts;

Data Set;

clc
close all
warning off
cam=webcam;
x=100;
y=100;
height=200;
width=200;
bboxes=[x y height width];
temp=0;
while temp<=300
e=cam.snapshot;
IFaces = insertObjectAnnotation(e,'rectangle',bboxes,'Processing Area');
imshow(IFaces);
filename=strcat(num2str(temp),'.bmp');
es=imcrop(e,bboxes);
es=imresize(es,[227 227]);
imwrite(es,filename);
temp=temp+1;
drawnow;
end
clear c;

DeepLearning;

clc
close all
warning off
g = alexnet;
layers = g.Layers;
layers(23) = fullyConnectedLayer(6);
layers(25) = classificationLayer;

allImages = imageDatastore('Hand Gestures', 'IncludeSubfolders', true, 'LabelSource',

Camera-Hand Detection;

clc;

close all;

warning off

c=webcam;

load myNet1.mat

x=100;

y=100;

height=200;

width=200;

bboxes=[x y height width];

while true

e=c.snapshot;

IFaces = insertObjectAnnotation(e,'rectangle',bboxes,'Processing Area');

es=imcrop(e,bboxes);

es=imresize(es,[227,227]);

label=classify(myNet1,es);

imshow(IFaces);

title(char(label));

drawnow;

end

12
STEPS ON HOW TO DEPLOY THE PROJECT

First, download the following add-ons that were shown on the picture below.

Second. On the first script, copy the code on the data set section. This section
makes a data set that will recognize a certain hand gesture that will be captured on
300 copies. Since the data set is already set in the given folder, there is no need to
do this section.

13
Third, Make another script for deep learning. This section is where the function of the main project is.
Copy the given code for deep learning and save it as “myNet1” in a MATLAB file format. This will
serve as the main function of the project

Run the script and let the training finish before going to the last script.

14
Lastly, the camera hand gesture detection. Make another script and copy the given code for this section.
After copying the code, double-check if your camera is plugged in properly (if you are in a desktop setup)
and run the code.

Place the hand gesture on the box where the processing area is and it will show the hand gesture at the
top of the camera.

15
REFERENCES

Buarqoub, I. (2019). Language barriers to effective communication.

Dean, R. K., Rochester Institute of Technology, Pollard, R. Q., & Rochester Institute of Technology.
(2011). Context-based Ethical Reasoning in Interpreting A Demand Control Schema Perspective. The
Interpreter and Translator Trainer, 1–17. https://www.researchgate.net/publication/290940042

Filipino Sign Language with Bayani Generoso. (n.d.). https://www.gridmagazine.ph/story/30-mins-

Limaye, H., Shinde, S., Bapat, A., & Samant, N. (2022). Sign Language Recognition using Convolutional
Neural Network with Customization. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4169172

Zlatev, J., Devylder, S., Defina, R., Moskaluk, K., & Andersen, L. B. (2023). Analyzing polysemiosis:
language, gesture, and depiction in two cultural practices with sand drawing. Semiotica, 2023(253), 81–
116. https://doi.org/10.1515/sem-2022-0102

16