Bangla Food Review Sentimental Analysis Using Machine Learning
Bangla Food Review Sentimental Analysis Using Machine Learning
Machine Learning
Mohd. Istiaq Hossain Junaid, Faisal Hossain, Udyan Saha Upal, Anjana Tameem, Abul Kashim & Ahmed Fahmin
Department of Electrical and Computer Engineering
North South University
Dhaka, Bangladesh
[email protected], [email protected], [email protected], [email protected],
[email protected] & [email protected]
2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC) | 978-1-6654-8303-2/22/$31.00 ©2022 IEEE | DOI: 10.1109/CCWC54503.2022.9720761
Abstract— In this modern age, people are dependent on the food delivery platforms can get insightful information from
internet. They prefer to order food online or Food App rather the user's sentiment analysis. Nowadays, proper analysis of
than the restaurant. They are giving various reviews online user/customer feedback is critical to an organization's success.
about the foods. In this project, we aim to build a machine If a restaurant owner cannot recognize consumers' issues
learning model to analyze the sentiment of that reviews. In through their feedback, the authority's ability to understand
Bangladesh, internet users are increasing day by day. So we company difficulties becomes more difficult. Sentiment
have decided to build the model for the Bangla language. We analysis systems can review user feedback more proactively
have found no Bangla dataset for food reviews that we can use to the rapid change in the market and improve the restaurant's
for our project. Then we have collected more than one thousand
business view. To measure a user's opinion, one can identify
Bangla food reviews from various online platforms like
Foodpanda, Hungrynaki, Shohoz food, Pathao food, etc., and
the market position of a particular restaurant. Making the
labeled them. After some necessary preprocessing, we have model about the total reviews and their target levels, positive
extracted various features from cleaned data and used them to or negative, will categorize new user reviews [1].
train and test for machine learning and deep learning models. According to BTRC, the approximate number of internet
We have come to the result that Long Term Short Term users in Bangladesh will be 120.95 million by June 2021.
(LSTM), a deep learning model giving the best accuracy, that is They usually post reviews in Bangla, English, or Phonetic
90.89%, where we have used word2sequence as feature
Bangla, which is trendy among smartphone users. It is
extraction. Our research contribution will help the food
impossible to examine the reviews carefully because there is
industry by using this model. This model can help them to
understand the Bangla food review sentiment.
no precise guideline for posting them, and so many comments
are made regularly. That's why an automated system that can
Keywords— Bangla language processing, Sentiment analysis, identify the polarity of reviews can be beneficial. The features
Customer reviews, Natural Language Processing (NLP), of the user's emotional expression in general, whether positive
Recurrent Neural Network (RNN), Deep Learning (DL), Long or negative, may be found in Bangla or, more particularly,
short-term memory (LSTM), Gated recurrent unit (GRU). Phonetic Bangla. Our study collected over 1000 reviews from
various online food delivery platforms like Foodpanda,
I. INTRODUCTION Shohoz food, Pathao food, HungryNaki.
Sentiment Analysis, also known as opinion mining, is a Our objective is to create a machine learning system that
method of Natural Language Processing (NLP) that identifies can detect the possibility to the users and automatically
and extracts the emotional tone from text-based data. This display the ratio of negative to positive reviews using
system helps businesses gather insightful and socio sentiment analysis via natural language processing (NLP). We
sentimental information from unstructured text data. On the preprocessed the data with punctuation removal, stopwords
verge of technological advancement and the internet era, it can removal, tokenization, stemming, and other useless symbols.
be easily noticeable that text-based sharing reviews on various Transforming text to feature vector, also known as
internet platforms, especially restaurants through online food vectorization, we use Count Vectors, TfidfVectorizer, N-gram
delivery platforms, are common. Online text-based reviews to test machine learning and deep learning techniques. Finally,
reflect customers' opinions on a particular product. By to classify positive and negative reviews, we preferred
expressing their own opinions or sentiments, users rate the Random Forest Classifier, LinearSVM, Naïve Bayes,
restaurants and their food services. These reviews might Decision Tree Classifier, Logistic Regression, Multinomial
indicate whether the restaurant is of sufficient quality or poor Naïve Bayes, LSTM, GRU, RNN. We have also dealt the
quality. For this reason, those text reviews can be the source phonetic reviews like "Khabar ta besh valo chilo," which
for sentiment analysis about a restaurant. In recent years, translates "খাবারটা বেশ ভাল ছিল" in actual Bengali. We
internet users have been growing radically in Bangladesh.
did these types of translations manually.
Online food delivery systems offer users to share their
feedback to care about their customer's fundamental interest In summary, in our study, we collected over 1000 Bengali
in their food services. For the rapid growth of these platforms, food reviews and annotated them manually, which will be a
enormous text reviews could be analyzed to find out the user's
sentiment on particular food items. Restaurant owners and
Authorized licensed use limited to: Chungnam National University. Downloaded on February 23,2023 at 02:18:30 UTC from IEEE Xplore. Restrictions apply.
Fig. 1 Infrastructure of our experiment
public dataset for future research. Those reviews were Most of these research works are not adequately utilized
collected manually from online food delivery platforms. We for Bangla reviews sentiment analysis. One such research was
implemented deep learning techniques to get higher accuracy implemented by Sharif et al. [6]. They collected English
rather than only machine learning techniques. reviews from Facebook pages and groups and manually
annotated and classified positive or negative reviews. Then
II. LITERATURE REVIEW translated the reviews from English to Bangla. Their best
Natural language processing (NLP) is a major model MNB achieved 80.48% accuracy on validation sets.
preprocessing task in any script-based research work in the Another similar research, Rahman et al. [7] of a translated
machine learning discipline. Many scholars and researchers dataset of online shopping review, achieved 78% accuracy for
have done sentiment-based analysis on opinion mining where SVM and 83% accuracy for the CNN model. But sentiment
different emotions are categorized, and various machine analysis is highly dependent on language context and structure
learning models were implemented after applying other NLP where machine translation lacks human interactions. For this
techniques. Yet, the essential part of doing any textual-based purpose, researchers are now more focused on collecting
preprocessing is solely dependent on the language, which is native Bangla language data for machine learning purposes.
not universal. While there are lots of works and NLP tools for For instance, Shafin et al. [8] worked on online shopping
the English language, scarcity of Bangla annotated datasets product review opinion mining, collected 1020 reviews, and
and no proper NLP tools for Bangla text preprocessing are annotated them positively and negatively accordingly. Using
some main restrictions on more research in this discipline. TF-IDF vectorizer as model features, SVM acquired the best
testing accuracy of 88.81%. Though this was a product review
Sentiment analysis for different contextual datasets has based on opinion mining, more focused work on restaurant
experimented with English datasets. Hossain et al. [2] worked and food reviews has been done recently. Hossain et al. [9]
on a restaurant review dataset where they collected about 1k created a dataset from FoodPanda and Shohoj food websites
reviews from the PRIYO review website. Manually by manually annotating 500 positive and 500 negative reviews
annotated the reviews under positive and negative and experimented on a combined CNN-LSTM model. Though
impressions, and POS tagged the dataset for feature they achieved high training accuracy, validation accuracy was
extraction along with the TF-IDF vectorizer. stuck at 75.01%. Haque et al. [10] experimented with different
Experimenting with three machine learning models for MNB, feature extraction and model implementation techniques to
SVM, KNN, and LR - they achieved the best accuracy of 77% compare restaurant opinion mining. They collected 1500
on their validation set. Similar experimentation was done by reviews from Facebook, YouTube videos, blogs and
Asiful et al. [3], where they collected a dataset from a annotated them bad, good, or excellent. Their experiment
Facebook page named FOODBANK. Still, their dataset differentiated different features of N-gram and vectorizing
yielded an unbalanced one consisting of 500 positive reviews techniques with SVM, DT, and LR models. They found that
with 200 negative reviews. As feature extraction is an TF-IDF Vectorizer with Bigram acquired the best accuracy on
essential part of any machine learning research, different SVM with SVM 75.58%.
approaches were taken by researchers in NLP. Hasan et al. [4]
Our background research has concluded that there is a
has taken 5000 restaurant reviews and extracted four various
scarcity of rich Bangla annotated datasets for food and
features - BOW, TF-IDF, Skip-Gram, CBOW and
restaurant reviews. So, this paper aims to construct a dataset
experimented with three machine learning models. Among
on opinion mining for food and restaurant reviews. Implement
them, they showed that skip-gram had the best accuracy
and experiment with different features and create a machine
among all the models. Bhuiyan et al. [5] implemented deep
learning and deep learning model for our custom dataset.
learning models and compared CNN, CNN with attention
mechanism and LSTM on an English dataset and achieved
98.4% accuracy.
0348
Authorized licensed use limited to: Chungnam National University. Downloaded on February 23,2023 at 02:18:30 UTC from IEEE Xplore. Restrictions apply.
III. METHODOLOGY C. Feature Extraction
Different features were extracted from the review text to
feed the classifier models. As previously mentioned, we used
Food Reviews traditional machine learning techniques and deep learning
models to classify our food review dataset. Both of these
models are unable to classify textual representations. To
analyze and predict different sentiments, they need numeric
representations of various properties and features of the text
and sentences. We used other parts for different classifiers to
experiment and get better results. We'll go over those features
briefly in this section:
Count Vectorizer: The frequency of the word is calculated
for each document using a count vectorizer to represent text
data in a matrix form. All unique words in a dataset are
represented in a sparse matrix by a CountVectorizer. Each
word in a document is assigned an index number, representing
Negative Positive the frequency of that word in the paper. After that, the vector
representation is used as a feature. We extracted count vectors
from our dataset and fed them into various models.
Fig. 2 Pie chart of collected Data
0349
Authorized licensed use limited to: Chungnam National University. Downloaded on February 23,2023 at 02:18:30 UTC from IEEE Xplore. Restrictions apply.
E. Model Implementation
Table 1 Comparative analysis of other's model
0350
Authorized licensed use limited to: Chungnam National University. Downloaded on February 23,2023 at 02:18:30 UTC from IEEE Xplore. Restrictions apply.
Table 2. Average evaluation metrics for ML models Table 3. Average evaluation metrics for DL models
Multinom 0(0.66) 0(0.80) 0(0.71) 69.23% From table 3, we can see that by using the word to sequence
ial Naive 1(0.74) 1(0.59) 1(0.66) as feature extraction, we get a good result from LSTM, which
Bayes was fine-tuned. The accuracy is 90.86%. We can also observe
Decision 0(0.69) 0(0.72) 0(0.70) 69.71% that LSTM performed well when using Glove as feature
Tree 1(0.71) 1(0.67) 1(0.69)
Classifier extraction. The accuracy is 87.5%. Mohd Istiaq Hossain
Junaid et al. [14], for binary text classification, achieved more
Logistic 0(0.72) 0(0.69) 0(0.71) 71.15%
Regressio 1(0.70) 1(0.73) 1(0.72)
than 98% from GRU by using word2sequence as feature
n extraction. The difference between that dataset has more
vocab per sentence than this dataset. We can conclude that
N-Gram Random 0(0.73) 0(0.73) 0(0.73) 73.08%
Vector Forrest 1(0.73) 1(0.73) 1(0.73) Deep learning models are better than machine learning
Classifier models. Among the Deep learning Models, LSTM performs
Linear 0(0.73) 0(0.73) 0(0.73) 73.08% well by using word2Sequence as feature extraction. We can
SVM 1(0.73) 1(0.73) 1(0.73) say that feature extraction plays a vital role in our experiment.
Multinom 0(0.70) 0(0.83) 0(0.76) 73.55%
It helps the model to increase its accuracy.
ial Naive 1(0.79) 1(0.64) 1(0.71)
Bayes Fig. 4 shows the ROC curves of the deep learning models
Decision 0(0.72) 0(0.65) 0(0.68) 69.71% using the word2sequence as feature extraction. We can see
Tree 1(0.68) 1(0.74) 1(0.71
Classifier
that the highest AUC (Area under Curve) Score is 0.94, which
Logistic 0(0.72) 0(0.75) 0(0.73) 72.59% means it can distinguish between bad and good—the highest
Regressio 1(0.74) 1(0.70) 1(0.72 AUC score from GRU.
n
Tf-Idf Random 0(0.75) 0(0.64) 0(69) 72.0%
Forrest 1(0.69) 1(0.79) 1(0.75)
Classifier
Linear 0(0.74) 0(0.64) 0(0.69) 71.0%
SVM 1(0.69) 1(0.78) 1(0.73)
0351
Authorized licensed use limited to: Chungnam National University. Downloaded on February 23,2023 at 02:18:30 UTC from IEEE Xplore. Restrictions apply.
V. CONCLUSION & FUTURE WORK
Bangla Food Review sentiment analysis is relatively new
research. Since internet users are increasing daily in
Bangladesh, people are moving their activities more from
offline to online. So this is important to work on such a topic
for the future. We have created our dataset, preprocessed it,
and trained on various machine learning and deep learning
models. We have the best result from LSTM by using
word2sequence as feature extraction.
For future work, we can extend the datasets and classify
more than two categories: classify datasets into neutral, worst,
best, etc. Since the pre-trained model is quite bigger, we plan
to use distilling knowledge to have a small model with better
accuracy. By this, any person can use this model with any
device. Our contribution will have a significant impact on the
food industry. This work will benefit the food industries, as
well as customers.
0352
Authorized licensed use limited to: Chungnam National University. Downloaded on February 23,2023 at 02:18:30 UTC from IEEE Xplore. Restrictions apply.
NLP and Machine Learning in Bangla Language," in International
Conference on Computer and Information Technology (ICCIT),
DHAKA, Bangladesh,, 2020.
[9] N. Hossain, Md. R. Bhuiyan, Z. N. Tumpa, and S. A. Hossain,
"Sentiment Analysis of Restaurant Reviews using Combined CNN-
LSTM," in 11th International Conference on Computing,
Communication and Networking Technologies (ICCCNT),
Kharagpur, India, 2020 .
[10] F. Haque, Md. M. H. Manik, and M. M. A. Hashem, "“Opinion
Mining from Bangla and Phonetic Bangla Reviews Using
Vectorization Methods,”," in 4th International Conference on
Electrical Information and Communication Technology (EICT), Dec.
2019, Khulna, Bangladesh.
[11] R. A. Laksono, K. R. Sungkono, R. Sarno and C. S. Wahyuni,
"Sentiment Analysis of Restaurant Customer Reviews on
TripAdvisor using Naïve Bayes," in 12th International Conference
on Information & Communication Technology and System (ICTS),
2019 .
[12] M. H. Rahman, M. S. Islam, M. M. U. Jowel, M. M. Hasan and M.
S. Latif, "Classification of Book Review Sentiment in Bangla
Language Using NLP, Machine Learning and LSTM," in 2021 12th
International Conference on Computing Communication and
Networking Technologies (ICCCNT), 2021 .
[13] R. R. Chowdhury, M. Shahadat Hossain, S. Hossain and K.
Andersson, "Analyzing Sentiment of Movie Reviews in Bangla by
Applying Machine Learning Techniques," in 2019 International
Conference on Bangla Speech and Language Processing (ICBSLP),
2019, .
[14] M. I. Hossain Junaid, F. Hossain and R. M. Rahman, "Bangla Hate
Speech Detection in Videos Using Machine Learning," 2021 IEEE
12th Annual Ubiquitous Computing, Electronics & Mobile
Communication Conference (UEMCON), 2021, pp. 0347-0351, doi:
10.1109/UEMCON53757.2021.9666550.
0353
Authorized licensed use limited to: Chungnam National University. Downloaded on February 23,2023 at 02:18:30 UTC from IEEE Xplore. Restrictions apply.