Proceedings of the SMART–2021, IEEE Conference ID: 52563

10th International Conference on System Modeling & Advancement in Research Trends, 10th–11th December, 2021
Faculty of Engineering & Computing Sciences, Teerthanker Mahaveer University, Moradabad, India

Multi Class Fake News Detection using

2021 10th International Conference on System Modeling & Advancement in Research Trends (SMART) | 978-1-6654-3970-1/21/$31.00 ©2021 IEEE | DOI: 10.1109/SMART52563.2021.9676333

LSTM Approach

Bhaskar Majumdar1, Md. RafiuzzamanBhuiyan2, Md. Arid Hasan3,

Md. Sanzidul Islam4, Sheak Rashed Haider Noori5
Dept. of CSE, Daffodil International University, Dhaka, Bangladesh
1,2,3,4,5

E-mail: [email protected], [email protected], [email protected]

1

4
[email protected], [email protected]

Abstract—Nowadays the spread of fake news or very difficult to distinguish between false news and true
information is having a detrimental effect on society. Due to news raised from fabricated information here. As a result,
the widespread spread of fake news, we sometimes believe a researchers have focused on research on various types of
lot of fake news is true. As a result, we face issues and deprive social media, such as Twitter, Facebook, WhatsApp, online
ourselves of a lot of good and realistic news. To protect
news portals, and other recent news media. And through the
people’s lives from these various problems, we need to work
success of these studies, every effort is being made to deal
to automatically detect fake news. Fake news detection is
very complex task. In this paper we present our approach to with the spread of false fiction [2].
address multi class fake news detection using Deep Learning. Promoting fake news is nothing new in our human
We used a Long Short Term Memory (LSTM) model for society. We have this problem for a long time but now due
multi class fake news detection using data provided by the to the popularity of social media, people are facing many
task organizers. Our best performing model on the training problems through spreading fake news. We know that
data achieved an accuracy of 0.98. Our trained model gave an manually checking any news fake or real is very difficult,
accurate response to the detection of fake news. time consuming and expensive. Again as a result of various
Keywords: LSTM, Deep learning, Spread, Fake News, news checks through people it can often be biased and its
Detection. credibility can be largely questioned. So in recent times
I. Introduction researchers have been searching for solutions to these
problems automatically. All of these automated detection
The term “fake news” means publishing a news story methods will further improve the false news detection
in a fictitious manner without any scrutiny, propagating it process, taking less time and hassle. Recommended
without verifying the truth and falsehood, and deliberately solutions include solving the problem of detecting fake
spreading rumors to create chaos in society [12]. news using NLP methods with deep learning [3, 13 15].
The use of the Internet and technological advances in A system can be created using Natural Language
our lives have had a profound effect on society. We are Processing tools and Artificial Intelligence that will
now relying on social media to get any kind of information automatically boost detect false or wrong news. The
because using this social media we can collect any kind of method of detecting false reports or messages is extremely
information and news very easily and in a short time. Any challenging, a certain good dataset is needed to compare any
kind of news can spread very fast using social media so news with reality. Using AI and Deep Learning Algorithms
that any information is reaching people very fast. However, with NLP, we can create false news identification methods.
using this facility of social media, many kinds of misleading Fake news detection methods will help news writers gain
and baseless false rumors are spreading and many of us are credibility, adhere to guidelines for news coverage, and act
easily believing them, which sometimes creates a chaotic as predictors of real and wrong verification of messages
environment in society. Not only social media but also before online news broadcasts.
various types of newspapers, magazines and televisions are The dataset we applied for false news detection gave a
now serving various types of news that is presented to the rating by comparing text and titles for fake news detection
people without any sort of true or false verification which and these ratings could be False, Partial False, True, and
has created a lot of confusion in society [4]. Other. Fake news can be in a textual or a visual format [21].
In today’s world, publishing and spreading fake news Textual means to present a piece of news or information to
has become a big problem for every state. Now media is the people in written form or text form. Again, visual means
not just a source of information or news. We are spreading various news or information that is expressed through
our personal information through this media i.e. social pictures and videos. We’ve worked with text-related data
media. Now it has reached such a stage that it has become for this fake news detection.
Copyright © IEEE–2021 ISBN: 978-1-6654-3970-1 75

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on August 06,2024 at 03:37:05 UTC from IEEE Xplore. Restrictions apply.
 10th International Conference on System Modeling & Advancement in Research Trends, 10th–11th December, 2021
Faculty of Engineering & Computing Sciences, Teerthanker Mahaveer University, Moradabad, India

II. Literature Review they used labeled information bearing fake news. For this
So far many research works related to fake news work they used Classification algorithms and used text
have been done. The spread of fake news has recently data classification deep learning path. For NLP related
made everyone very worried. We have already seen many work BERT model play very important role to solve
research papers about fake news for different language. problem [9].
Many researchers have used machine learning algorithms Yunfei Long et al. proposed Long Short Term Memory
and many have applied deep learning algorithms to detect method for fake news detection with a view of speaker
fake information. profiles. In this system the speaker profiles have worked in
NataliRuchansky et al. applied CSI method for false two measures - first focused this model for learning news
news that is defined by Capture, Score, and Integrate text and then includes as extra input data [13]. William
modules. This modules are based on response, text, Yang Wang worked on the fake news detection problem by
behavior of users, categorize an article as fake or real [6]. using LIAR dataset which has 12.8K data and it can be used
SomyaRanjanSahoo et al. worked fake news detection for information checking purpose. For news delegation he
through machine learning and deep learning method based used LSTM and for vector delegation of speaker scheme
classification using some features related to social media. used Bi-LSTM [20].
They used more than 15000 data mixed by fake and real AyatAbedalla et al. focused on fake news related work
news and their accuracy was very good [16]. using DL method and they used different type of model-
Gautam Kishore Shahi et al. described the fake news CNN, LSTM, Bi-LSTM approach and they use article body
spread during the covid-19 epidemic via social media and headline in dataset. For this work the best accuracy
Twitter. They collected 1,500 fake tweets with the help was 71.2 percent [15]. Rahul Mishra proposed on a unique
of fact-checking [18] agencies and from here they worked consumer to consumer mutual attention development
to identify multiclass fake news. Through this research process to version affect concerns within users for fake
they have come up with the idea of one thing that partial news detection. In this work he used two types of twitter
false news spreads much faster than false news. In this datasets (Twitter15 and Twitter16) [7].
work, they explained in detail where the misinformation Sachin Kumar et al. incorporated a DL approach
came from and how it spread, and made many important for fake news detection. For this work they used 1356
recommendations for resolving the crisis [5]. Gautam news data from Twitter and other media and they used
Kishore Shahi et al. collected articles related to fake news CNN, LSTM and Bi-LSTM model. In this work they get
for covid-19 using cross domains. Their dataset features
maximum accuracy 88.78 percent [2]. HoonKo et al. used
a system that automatically detects fake news in about 40
the reverse-tracking system of the news that’s are send on
languages in 105 countries. In their work they collected data
the Cognitive method. In this case study for fake news the
from various online media platforms. They adopted machine
maximum accuracy was 85 percent [17].
learning algorithms to categorize the fake information about
Chaitra K Hiramath et al. discussed about fake
covid-19 and thereby initially prevented the spread of fake
news detection based on different types of classification
information during this pandemic time [22].
algorithms- Random forest, Naive bayes, Logistic
Hunt Allcott et al. discussed about the impact of fake
news on the 2016 US presidential election. Democracy and regression, Support vector machine and deep neural
elections in America have repeatedly been influenced by network [1]. Sherry Girgis et al. addressed the fake news
the news spread in the media. One thing is for sure, the detection problem by training several DL models using the
impact of fake news, features and fake news spread on LIAR dataset. Here they used LSTM and RNN technology
social media is causing a lot of concern in the neighborhood - Vanilla, GRU model. For better accuracy they used CNN
[12]. Sebastian Kula et al. worked fake news detection by with GRU [19].
using BERT neural network type architectures and BERT L. Rubin et al. worked on three variants of the fake
model connecting with RNN. This work are the scientific news detection problem [14]. AshwiniThota et al. adopted a
achievement to the Natural Language Processing research neural network architecture using the headline of a dataset
area [10]. and the body of the article. In this work they used Tf-IDF
Huang et al. applied deep learning approach for his vector with DNN model and the accuracy rate was 94.21
work. In this work they have used 4 different types of DL percent [8].
model- LIWC CNN, Embedding with LSTM, N-gram In this research we detected multi-level output for fake
CNN and Depth LSTM and they also used cross-domain news using LSTM model. The LSTM model worked very
intractability expression and the maximum accuracy was well for the data set we used. We saw a number of research
99.4 percent [11]. Ignacio Palacio Marín et al. racked papers that used deep learning and machine learning
of a specific event in the taxonomy of the complicated algorithm. Very few studies have previously worked on
continuity of fake and wrong information phenomena and multi-level output for fake news detection.
76 Copyright © IEEE–2021 ISBN: 978-1-6654-3970-1

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on August 06,2024 at 03:37:05 UTC from IEEE Xplore. Restrictions apply.
 Multi Class Fake News Detection using LSTM Approach

III. Methodology so that engine can understand. For data processing there
In this section we will discuss our methodology in are few steps we follow and completed our work. The
details. In this work we used Long Short Term Memory preprocessing steps are presented in Figure-2.
architecture with tensor flow 2.2.0 GPU for model training.
But before model training, we had to collect datasets and Add Contraction
process them. Our working flow is presented in Figure-1.

Remove Stop Word

Spelling Checker

Remove Punctuation

Clean Text
Fig. 2: Data Preprocessing Steps
Add Contraction: The concise form of words or
characters is called as contractions. For writing in English
an apostrophe is used to reveal the position of the absence
letters. Contractions are generally used in English written
text unceremonious terms of writing like- can not to replace
can’t, does not to replace doesn’t etc.
Remove Stop Word: We used Stop Word for data
Fig. 1: Workflow of Our Model processing. For NLP related work NLTK can remove the
stop word and trace the catalog of stop words in the corpus
A. Dataset Narration
module. It can be used safely without changing the meaning
To address the fake news detection, we used the of the sentence.
CheckThat!2021 task3a dataset provided by the lab Spelling Checker: This step we check spelling for our
organizers [23]. For this work, the training dataset has 900 dataset. There were many spelling mistakes in the dataset
data and the test dataset has 364 data. For this dataset the that we fixed with this method.
principal demand built in the item is true, partially true, false Remove Punctuation: In this dataset there are many
and other news detection. We must have a good dataset to needless punctuations. These we were able to remove with
use the deep learning algorithm. Good results are obtained this technology. It is the part of purified test of our model.
using deep learning algorithms for large and clean datasets.
In this dataset for model learning outcome depends on
C. Word Embedding
rating and the considerable attributes is public id, text, title. This research based on an English Language with multi
For outcome, our rating contains four labels- True, False, class fake news detection. Word embedding is a significant
Partially false and Other. and strong way to perform texts and documents. Word
Outcome false means the article or news is not true. embedding is a method where a word has a resembling
Outcome partially false means the news or information is meaning for the resembling precedent. The embedding
false and true mixed. Although the information registered system method for deep learning is used to radically
here is not 100% true and it is not 100% false. For outcome change the way words and documents are presented. We
true means the full information or element is right. Now in used “word2vec” [11, 23] word embedding system for this
this dataset other class means it is used in situations where work and it help us for time saving. We finalized our words
the information given here is neither true nor false due to before training and counted the words.
lack of evidence. D. Proposed Model
B. Data Preprocessing We used deep learning algorithms for our work.
After data collection, we must preprocess the data to There are several deep learning algorithms that are used
train the model. If we don’t do data preprocessing properly, for different purposes. We have only used LSTM for this
we won’t get the right output. It is the process of converting fake news related work under Natural Language Processing
raw data into a particular ordination and creating it decent, (NLP) area. Now we will describe the LSTM model below-
Copyright © IEEE–2021 ISBN: 978-1-6654-3970-1 77

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on August 06,2024 at 03:37:05 UTC from IEEE Xplore. Restrictions apply.
 10th International Conference on System Modeling & Advancement in Research Trends, 10th–11th December, 2021
Faculty of Engineering & Computing Sciences, Teerthanker Mahaveer University, Moradabad, India

Long Short Term Memory: LSTM acts as a special form IV. RESULT AND Discussion
of neural network for text data. LSTM is an architecture In this work we used LSTM architecture with tensor
that acts like memory and can retain previous information flow 2.2.0 version. The machine will be able to automatically
in memory for a long time. LSTM is connected to memory detect fake news when the dataset training of the model
and can make decisions based on previous data. This is stopped. In this model we applied Adam optimizer for
architecture is very efficient for the input of written data count the learning rate of every parameter. It is designed
or information related to NLP. LSTM has the capability to to form deep neural networks. For loss function purpose in
capture the feeling of the sentence. Its architecture services this model we applied categorical cross entropy.
very good in the case of large sequential data. In this research our proposed model work very good for
E. Experimental Setup multi-level fake news detection. Here our training accuracy
In order to use LSTM model (Figure-3), at first we was 98% and validation accuracy was 55%. In work our F1-
used collected news as input. We fit those inputs data to the macro accuracy was not good for this noisy, real and small
embedding layer with “word2vec” system [11, 23]. Then dataset. Using our LSTM model F1-macro score is 0.23.
a dropout was added to avoid over fitting. We used the For good result using deep learning method we need clean
hidden layer of LSTM. After using an activator, again we and large dataset. So, the work of our model cannot be said
added a dropout. Finally, we used the softmax function to to be bad for this multi-level English dataset.
detection false or partial false or true or other from our input After learning the model through the parameters we
data. Some parameters are destined for the training scheme added some findings, when we reduced the size of our
which are called hyper-parameters. This model we applied epochs from 30 to 15 then we found that using the shorter
few hyper-parameters in (Table 1) for better accuracy. duration epochs gave our prediction results more accurately.
Table 1: Hyper-parameters for Our Model Table 2: Our Result Predicting in Tabular Form
Hyper-parameter Value Actual Predicted
News Title
LSTM size 128 Result Result

1st Dropout rate 0.25 The rookie politician’s first month Partially
Partially false
looks a lot li ... false
2nd Dropout rate 0.35
Nearly 700 more deaths than average
True True
Batch size 32 were recorded...
Epochs 30 Get email updates with the day’s True
False
biggest stories I ...
Learning rate 0.00001
Though health officials have warned False False
Americans to ...
If you needed any more proof that the
Others Others
migrant cara ...

Lisa Page, former FBI lawyer under

False False
James Comey ...

Coronavirus horror: Volunteer in

Others False
Oxford University ...

V. Conclusion
The work related to the detection of fake news in the
natural language processing area is a very timely research
for the present time. To solve this task we applied deep
learning based model for multi-class fake news detection
by using news articles. We have tried to detect four types of
situation which is- false, partial false, true and other from
the collected news articles and accordingly we applied the
architectures. The LSTM model gives good results in any
work related to NLP and text data. This is why we have
used LSTM architecture in our work. Our model has given
satisfactory accuracy after using real time fake news dataset.
We will be able to enrich the dataset in the future
by adding more data to the dataset we have used to
Fig. 3: Proposed Model accomplish this task. Besides that here we can use some
78 Copyright © IEEE–2021 ISBN: 978-1-6654-3970-1

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on August 06,2024 at 03:37:05 UTC from IEEE Xplore. Restrictions apply.
 Multi Class Fake News Detection using LSTM Approach

more architectures related to deep learning in the future for [12] Hunt Allcott, Matthew Gentzkow: Social Media and Fake News in
the 2016 Election. Journal of Economic Perspectives—Volume 31,
fake news detection and can compare them using different 2017, DOI: 10.1257/jep.31.2.211.
architectures and find the best one out of them. [13] Yunfei Long, Qin Lu, Rong Xiang, Minglei Li and Chu-Ren Huang:
Fake News Detection ThroughMulti-Perspective Speaker Profiles.
References 8th International Joint Conference on Natural Language Processing,
[1] Chaitra K Hiramath, Prof. G.C Deshpande: Fake News Detection pages 252–256, volume 2, Taipei, Taiwan, 2017.
Using Deep Learning Techniques. 1st International Conference on [14] Victoria L. Rubin, Yimin Chen, Nadia K. Conroy: Deception
Advances in Information Technology, 2019 (ICAIT), DOI: 10.1109/ detection for news: three types of fakes, Proceedings of the
ICAIT47043.2019.8987258. Association for Information Science and Technology, 2015, https://
[2] Sachin Kumar, Rohan Asthana, ShashwatUpadhyay, NidhiUpreti, doi.org/10.1002/pra2.2015.145052010083.
Mohammad Akbar: Fake news detection using deep learning models: [15] AyatAbedalla, Aisha Al-Sadi and Malak Abdullah: A Closer
A novel approach. Transactions on Emerging Telecommunications Look at Fake News Detection: A Deep Learning Perspective.
Technologies (2019), https://doi.org/10.1002/ett.3767. ICAAI 2019, Istanbul, Turkey, ISBN 978-1-4503-7253-4 DOI:
[3] Choras, M., Gielczyk, A., Demestichas, K.P., Puchalski, D., Kozik, 10.1145/3369114.3369149.
R.: Pattern recognition solutions for fake news detection. 17th [16] SomyaRanjanSahoo, B.B. Gupta: Multiple features based approach
International Conference, CISIM 2018, Olomouc, Czech Republic, for automatic fake news detection on social networks using deep
vol. 11127. learning. Applied Soft Computing Journal 100 (2021) 106983,
[4] FeyzaAltunbeyOzbay, Bilal Alatas: Fake News Detection within https://doi.org/10.1016/j.asoc.2020.106983.
Online Social Media Using Supervised Artificial Intelligence [17] HoonKo, Jong Youl Hong, Sangheon Kim, Libor Mesicek, In Seop
Algorithms. Physica A (2019). Na: “Human-machine interaction: a case study on fake news detection
[5] Gautam Kishore Shahi, Anne Dirkson, Tim A. Majchrzak: An using a backtracking based on a cognitive system”. Cognitive
exploratory study of COVID-19 misinformation on Twitter. Online Systems Research, Volume 55, 2019, https://doi.org/10.1016/j.
Social Networks and Media, Volume 22, March 2021, page 100104. cogsys.2018.12.018.
[6] NataliRuchansky, SungyongSeo, Yan Liu: CSI: A Hybrid Deep [18] Nakov P. et al. (2021) The CLEF-2021 CheckThat! Lab on
Model for Fake News Detection. CIKM’17, November 2017, Detecting Check-Worthy Claims, Previously Fact-Checked
Singapore, DOI: https://doi.org/10.1145/3132847.3132877. Claims, and Fake News. In: Hiemstra D., Moens MF., Mothe
[7] Rahul Mishra: “Fake News Detection using Higher-order User J., Perego R., Potthast M., Sebastiani F. (eds) Advances in
to User Mutual-attention Progression in Propagation Paths”, 2020 Information Retrieval. ECIR 2021. Lecture Notes in Computer
IEEE/CVF Conference on Computer Vision and Pattern Recognition Science, vol 12657. Springer, Cham. https://doi.org/10.1007/978-
Workshops (CVPRW),doi: 10.1109/CVPRW50498.2020.00334. 3-030-72240-1_75.
[8] AswiniThota, PriyankaTilak, Simeratjeet Ahluwalia and [19] Sherry Girgis, EslamAmer and Mahmoud Gadallah: Deep Learning
NibhratLohia: Fake News Detection: A Deep Learning Approach. Algorithms for Detecting Fake News in Online Text. 2018 13th
SMU Data Science Review: Vol. 1: No. 3, Article 10. International Conference on Computer Engineering and Systems
[9] Ignacio Palacio Marín, David Arroyo (2021): Fake News (ICCES), DOI: 10.1109/ICCES.2018.8639198.
Detection. 13th International Conference on Computational [20] William Yang Wang: liar, liar pants on fire: A new benchmark
Intelligence in Security for Information Systems (CISIS 2020), dataset for fake news detection. Published in ACL 2017, https://doi.
CISIS 2019. Advances in Intelligent Systems and Computing, org/10.18653/v1/P17-2067.
Springer, doi: https://doi.org/10.1007/978-3-030-57805-3_22. [21] XinyiZhou(B) , Jindi Wu and Reza Zafarani: SAFE: Similarity-
[10] Sebastian Kula, Michał Choraś and Rafał Kozik: Application Aware Multi-modal Fake News Detection. (2020), doi: https://doi.
of the BERT-Based Architecture in Fake News Detection. 13th org/10.1007/978-3-030-47436-2_27.
International Conference on Computational Intelligence in [22] Gautam Kishore Shahi, DurgeshNandini: FakeCovid- A
Security for Information Systems (CISIS 2020), Advances in Multilingual Cross-domain Fact Check News Dataset for
Intelligent Systems and Computing, Springer, doi: https://doi. COVID-19. Workshop Proceedings of the 14th International
org/10.1007/978-3-030-57805-3_23. Conference on Web and Social Media, 2020, http://workshop-
[11] Yin-Fu Huang, Po-Hong Chen: Fake News Detection Using an proceedings.icwsm.org/pdf/2020_14.pdf.
Ensemble Learning Model Based on Self-adaptive Harmony Search [23] Gautam Kishore Shahi, Struß,Julia Maria and Thomas Mandl:
Algorithms. Expert Systems with Applications (2020), doi: https:// Overview of the {CLEF}-2021 {CheckThat}! Lab Task 3 on Fake
doi.org/10.1016/j.eswa.2020.113584. News Detection, 2021, Working Notes of CLEF 2021---Conference
and Labs of the Evaluation Forum, Bucharest, Romania (online).

Copyright © IEEE–2021 ISBN: 978-1-6654-3970-1 79

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on August 06,2024 at 03:37:05 UTC from IEEE Xplore. Restrictions apply.