Expert Systems With Applications 231 (2023) 120716

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Detecting fake news by RNN-based gatekeeping behavior model on social

networks
Bailin Xie ∗, Qi Li
School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou, 510006, China

ARTICLE INFO ABSTRACT

Keywords: Social network users are not only news disseminators and consumers, but also gatekeepers. News gatekeepers
Social networks are regular users who actively participate in news propagation. This study introduces the concept of
Fake news gatekeepers into social network fake news detection and then presents a recurrent neural network (RNN)-
Gatekeeper
based gatekeeping behavior model (RGBM). Based on this, we propose a social network fake news detection
Recurrent neural network
method. The proposed method includes model training and fake-news detection. In the fake news detection
GRU
phase, every observation sequence is updated in real time and the output of every observation sequence is
calculated in real time. Therefore, the proposed method can detect social network fake news in real time.
Several RNNs are compared on real datasets from Twitter and Weibo. The experimental results show that the
gate recurrent unit (GRU) achieves the best comprehensive performance. On the Twitter and Weibo datasets,
the proposed method had an overall accuracy of 0.985, recall of 0.978, F1 of 0.976 and loss of 0.058. In
a comparison test, the proposed method outperformed several state-of-the-art approaches. The experimental
results of the timeliness evaluation also demonstrated that the proposed method can effectively detect fake
news in the early and middle stages of news propagation.

1. Introduction networks (Bodaghi & Oliveira, 2022; Khan et al., 2021; Shan et al.,
2022). For example, social networks are flooded with lots of fake news
Social networks are online social broadcast media platforms where during the COVID-19 pandemic (Ceron et al., 2021; Iwendi et al.,
users can share knowledge and information. With their revolutionized 2022). In the face of social panic caused by the COVID-19 pandemic,
methods of instantaneous information dissemination and consumption, a large number of people are more willing to believe fake news, and
social networks have gained significant popularity over the last decade. even become active forwarders of such fake news.
As of the fourth quarter of 2021, the number of daily active users of Social network fake news has immeasurable negative effects on indi-
Facebook and Twitter, the two most notable social networks in the viduals, societies and nations (Olan et al., 2022). For example, the fake
world, has reached 1930 and 217 million respectively. The large and news posted by the compromised Twitter account of Associated Press
growing daily active user base makes social networks a great place claimed that the explosion in the White House and the US president’s
for news dissemination. For instance, approximately 68 percent of injury caused national panic and wiped out $130 billion in stock value.
American adults read their news from social media in 2018 (Zhou &
Social network fake news is also considered one of the greatest threats
Zafarani, 2020).
to democracy, justice and public trust. During the 2016 US presidential
However, on the flip side, social networks are also becoming a
election, a large amount of fake news found its way on social networks,
breeding ground for fake news, for several reasons (Lazer et al., 2018;
which had a serious impact on election results (Allcott & Gentzkow,
Shu et al., 2017). First, the brevity of social network news can lead
2017). Therefore, the rapid and accurate detection of social network
to inaccurate or incomprehensible interpretations of real-world events.
fake news and ensuring the authenticity of social network news are
Second, the immediacy and user-generated nature of social network
urgent problems to be solved.
news makes it difficult to prevent fake news from being published.
Third, the social nature of social network news could mislead users In light of this, social network fake news detection has been widely
into inadvertently participate in the spread of fake news. In addition, researched in recent years (Li et al., 2022; Raponi et al., 2022; Rohera
due to the lack of rigorous gatekeepers (Garimella et al., 2018), it is et al., 2022; Song et al., 2022; Wang et al., 2022). The existing methods
extremely easy for fake news to spread rapidly and widely on social can be divided into two categories. The first category uses external

∗ Corresponding author.
E-mail addresses: [email protected] (B. Xie), [email protected] (Q. Li).

https://doi.org/10.1016/j.eswa.2023.120716
Received 30 June 2022; Received in revised form 15 August 2022; Accepted 3 June 2023
Available online 8 June 2023
0957-4174/© 2023 Elsevier Ltd. All rights reserved.
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

sources of information to detect fake news. The second category of 2.1. Fake news detection based on external information sources
methods, in contrast, relies on the signals available on social networks
to detect fake news. For example, based on the attributes extracted from Suzuki and Nadamoto (2011) presented an approach to measure the
metadata, post content, the trace of propagation and user interaction, credibility of messages on social media based on content similarity with
machine learning classifiers or deep learning models can be built to credible articles from Wikipedia. Al-Khalifa and Al-Eidan (2011) pro-
distinguish truthful and fake news. posed a similar prototype system that assesses the credibility levels of
This study proposes a new method for social network fake news news content on Twitter according to its similarity with verified content
detection. The method adopts the output of gatekeeping behavior on from Google News. Liang et al. (2012) proposed a fake news detection
social networks and detects fake news by mining the confidence and framework involving active human participation. In particular, they
opinions of users who forward and/or comment on corresponding built an index of user expertise based on the user generated content.
news. In particular, the concept of gatekeepers for news was introduced When a piece of news must be assessed, the task is assigned to an expert
in this method. The rationale is that social network users are not who matches the given news. In general, this method is based on the
only news consumers and disseminators but also gatekeepers. News availability of similar content from external sources (Shim et al., 2021),
gatekeepers are regular users who read news and actively participate which makes it unsuitable for detecting fake news in real time.
in news propagation.
News gatekeepers on social networks do not have the authority 2.2. Fake news detection based on information available on social networks
or resources to identify news as fake. However, they tend to make
judgments implicitly by applying their background and knowledge, and Earlier studies focused on finding a good set of features, such
they can express negative, neutral, or positive attitudes toward news. as the features extracted from metadata, post content, and traces of
When news spreads, the propagation process can be described by propagation and user interaction. Subsequently, supervised classifi-
a sequence of observations comprising gatekeeper opinions and confi- cation algorithms were used to detect fake news (Bibi et al., 2022;
dence levels. Specifically, we infer gatekeeper opinions by mining ad- Ebrahimi Fard et al., 2019; Reis et al., 2019). For instance, Castillo
ditional comments, and estimate gatekeeper confidence levels based at- et al. (2011) adopted supervised classifiers to predict whether a tweet
tributes that reflect gatekeeper trustworthiness and relevance to news. is credible or newsworthy. Specifically, they chose four types of fea-
In this study, we found differences in gatekeeping behavior models for tures: user-based, message-based, propagation-based, and topic-based.
truthful and fake news. Therefore, a new method for detecting social Propagation-based characteristics include the number of initial tweets
network fake news was proposed based on the gatekeeping behavior of the topic and several measurements of the retweet tree, such as
model. the maximum level size, total size, average and maximum depth, and
The main contributions of this study can be summarized as follows: average and maximum degree. Other types of features are extracted
from the profile of the authors or the content of the tweets. Gupta et al.
• This is the first study to introduce the concept of gatekeepers into (2012) analyzed the credibility of Twitter events. They proposed an ap-
social network fake news detection. proach to assess event credibility by analyzing not only the attributes of
• We proposed a recurrent neural network (RNN)-based gatekeep- different entities (i.e., tweets, users, and events) but also the inter-entity
ing behavior model (RGBM). The model takes news gatekeepers’ credibility relationships. In particular, the features proposed in Castillo
opinions and confidence levels as observations and uses the gate et al. (2011) are chosen, along with some novel ones, to build an SVM
recurrent unit (GRU) to describe the gatekeeping behavior of classifier and then use the predictions given by the SVM classifier to
news gatekeepers. The random forest classification algorithm was initialize the credibility of different nodes in a graph of users, tweets,
adopted to estimate the gatekeeper confidence levels and infer the and events. Finally, they adopted page rank-like credibility propagation
gatekeeper opinions. and event-graph optimization methods to detect fake news.
• Based on the gatekeeping behavior model, we propose a fake Xia et al. (2012) investigated the credibility of emergency events
news detection method that detects fake news in real time by circulating on Twitter. They first adopted a sequential k-means algo-
monitoring gatekeepers who appear in the process of news prop- rithm to identify sudden bursts of related tweets. When an emergency
agation. situation is detected, the Bayesian network-based classifier is used to
• We compared several recurrent neural networks (RNNs) based on predict whether the tweets related to the situation are fake. Four types
real datasets from Twitter and Weibo, and the experimental re- of features, content-based, author-based, diffusion-based, and topic-
sults showed that GRU had the best comprehensive performance. based, were chosen for classification. Yang et al. (2012) proposed one
We also compared our proposed fake news detection method of the earliest studies of news credibility on Weibo. The authors found
with several existing methods, and the test results confirmed the that differences in the types of trending topics and language could make
effectiveness and superiority of our method. We evaluated the some useful features on Twitter not applicable to Weibo. In this regard,
timeliness of our method, and the experimental results show that they chose a set of account-based, content-based, and propagation-
our method can effectively detect fake news in the early and based features, and they also selected a few unique features available
middle stages of news propagation. on Weibo. Their experimental results demonstrated that unique features
can increase the accuracy of the classifier.
The remainder of this paper is organized as follows. Section 2 Other researchers have introduced artificial neural networks to
reviews recent studies on fake news detection. In Section 3, we de- detect fake news (Davoudi et al., 2022), including recurrent neural
scribe the gatekeeping behavior model. Section 4 presents the proposed networks (Jin et al., 2017; Ma et al., 2016, 2018; Mittal et al., 2021;
fake-news detection method. The experimental results are presented Ruchansky et al., 2017), convolutional neural networks (Liu & Wu,
in Section 5, and the results are discussed in Section 6. Finally, we 2018; Monti et al., 2019; Yu et al., 2017, 2019), graph convolutional
conclude the paper in Section 7. networks (Bian et al., 2020; Chandra et al., 2020; Liao et al., 2020; Xu
et al., 2022), Bayesian neural networks (Zhang et al., 2019), recursive
2. Related works neural networks (Ma et al., 2020), generative adversarial networks (Ma
et al., 2019), auto-encoders (Cheng et al., 2020; Khattar et al., 2019),
The existing approaches for social network fake news detection can and attention mechanisms (Chen et al., 2018; Guo et al., 2018; Huang
be divided into two categories. The first category involves the detection et al., 2020; Liu et al., 2018). For example, Ma et al. (2016) introduced
of fake news using external information sources. In contrast, the second recurrent neural networks to capture the temporal linguistic features of
category relies on information available on social networks. a continuous stream of user comments in order to detect fake news. Yu

2
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

et al. (2017) used convolutional neural networks to learn high-level in- 3.3. Gatekeeping behavior modeling
teractions among them to detect fake news. Bian et al. (2020) proposed
a graph convolutional network-based model to capture the structures of Based on the above definitions of gatekeeper confidence and opin-
rumor dispersions. Guo et al. (2018) proposed a detection model based ion, the propagation process of one piece of news can be considered as
on attention-based neural networks, with handcrafted social features a sequence of observations that are composed of the confidence levels
serving as attention at different semantic levels. of gatekeepers and opinions of gatekeepers, that is, 𝑋1 , 𝑋2 , . . . , 𝑋𝑡 ,
. . . , where 𝑋𝑡 denotes the observation at the 𝑡th time that the news
3. RNN-based gatekeeping behavior model is forwarded or commented. 𝑋𝑡 is obtained based on the opinion and
confidence level of the 𝑡th gatekeeper.
On social networks, users forward or comment on news to express
their opinions, feelings, or simply interest in news they happen to read. ⎧ 𝑣 𝑖𝑓 𝑜𝑝𝑖𝑛𝑖𝑜𝑛 𝑖𝑠 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒
⎪
In this study, we proposed a gatekeeping behavior model for fake news ⎪ 𝑣 + 𝑉 𝑖𝑓 𝑜𝑝𝑖𝑛𝑖𝑜𝑛 𝑖𝑠 𝑛𝑒𝑢𝑡𝑟𝑎𝑙
detection. 𝑋𝑡 = ⎨ (1)
⎪ 𝑣 + 2𝑉 𝑖𝑓 𝑜𝑝𝑖𝑛𝑖𝑜𝑛 𝑖𝑠 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒
⎪
3.1. Gatekeeping on social networks ⎩𝑣 + 3𝑉 𝑖𝑓 𝑜𝑝𝑖𝑛𝑖𝑜𝑛 𝑖𝑠 𝑛𝑜 𝑐𝑜𝑚𝑚𝑒𝑛𝑡
In Eq. (1), 𝑣 denotes the confidence level of the 𝑡th gatekeeper, that
In this study, news gatekeepers are defined as users who actively is, 𝑣 ∈{1,2, . . . , 𝑉 }, and 𝑉 denotes the maximum level of gatekeeper
participate in news propagation. Users can perform gatekeeping on confidence.
social networks as follows. As the news spreads, gatekeepers often receive more relevant infor-
Forward: Users only forward news without appending personal mation and opinions from others. Therefore, the confidence and opin-
comments. ions of the gatekeepers vary over time. Gatekeepers forward or com-
Forward & Comment: Users append personal comments as they ment on news for different purposes (e.g., benign or malicious). Based
forward news. on these characteristics, we consider that the gatekeeping behavior out-
Comment: Users comment directly on news without forwarding the put is driven by a recurrent neural network, as shown in Fig. 2, where
news. regular pentagons, squares, and six-pointed stars represent different
Therefore, news gatekeepers are merely regular users who read gatekeepers who perform gatekeeping in different ways.
interesting news such that they decide to forward or append personal Recurrent neural networks (RNNs) are feedforward neural networks
comments. This does not necessarily mean that gatekeepers must con- that introduce the concept of time into the model. They are charac-
firm that news is truthful. However, gatekeepers inevitably make their terized by ‘‘memory’’ as they take information from prior inputs to
own judgments based on their knowledge and background. influence current input and output. RNNs utilize sequential or time-
series data for learning. They can capture long-term temporal de-
3.2. Gatekeeping behavior analysis
pendencies, overcoming the major limitations of the Markov models.
RNNs can model inputs and/or outputs that contain sequences of de-
Based on the confidence and opinions expressed by news gatekeep-
pendent elements, and simultaneously model sequential and temporal
ers, we analyze the gatekeeping behavior of gatekeepers using RNNs
dependencies at multiple scales.
from two perspectives: who they are and what they think. Gatekeeper
In Fig. 2, ℎ𝑡 represents the hidden cell activation at time step 𝑡 and
confidence and opinions are defined as follows.
𝑌𝑡 represents the output at time step 𝑡, expressed as follows:
( )
3.2.1. Gatekeeper confidence estimation ℎ𝑡 = 𝑠𝑜𝑓 𝑡𝑚𝑎𝑥 𝑊ℎ(0) ℎ𝑡−1 + 𝑊𝑖(0) 𝑋𝑡 + 𝑏(0) (2)
ℎ
News gatekeeper confidence refers to the gatekeeper’s ability to
confirm the truthfulness of the news. The confidence level of the
( )
gatekeeper was estimated based on these two types of features. The
𝑌𝑡 = 𝑠𝑜𝑓 𝑡𝑚𝑎𝑥 𝑊𝑦(0) ℎ𝑡 + 𝑏(0)
𝑦 (3)
first type of feature reflects the credibility of the gatekeeper, such
as registration age, number of followers, and verification status. The
second type of feature reflects the extent to which the gatekeeper knows If the RNN cell is an LSTM cell, then ℎ𝑡 and 𝑌𝑡 are calculated using
about the news, such as the distance between the gatekeeper and where the following equations:
the news occurs, and the number of keywords (i.e., the frequent words ( )
in the gatekeeper’s previous posts and the hash tags in the gatekeeper’s 𝑖(1) (1) (1)
𝑡 = 𝜎 𝑊𝑖𝑥 𝑋𝑡 + 𝑊𝑖ℎ ℎ𝑡−1 + 𝑏𝑖
(1)
(4)
profile) matched in the news text. Then, we use machine learning
algorithms to train a classifier to predict the confidence level of any ( )
given gatekeeper. 𝑓𝑡(1) = 𝜎 𝑊𝑓(1)
𝑥 𝑡
𝑋 + 𝑊𝑓(1) ℎ + 𝑏(1)
ℎ 𝑡−1 𝑓
(5)

3.2.2. Gatekeeper opinion inference ( )

A gatekeeper opinion is defined as representing what the gate- 𝑔𝑡(1) = 𝑡𝑎𝑛ℎ 𝑊𝑔𝑥
(1) (1)
𝑋𝑡 + 𝑊𝑔ℎ ℎ𝑡−1 + 𝑏(1)
𝑔 (6)
keeper thinks about the news, which is derived from the gatekeeper’s
comment. In this study, we consider four types of opinions: positive,
( )
negative, neutral, and no comment. In the case where the gatekeeper
𝑜(1) (1) (1) (1)
𝑡 = 𝜎 𝑊𝑜𝑥 𝑋𝑡 + 𝑊𝑜ℎ ℎ𝑡−1 + 𝑏𝑜 (7)
forwards the news without appending any words, the gatekeeper’s
opinion is no comment. In other cases, gatekeepers’ opinions can be
derived from their comments by using sentiment analysis schemes.
𝑐𝑡(1) = 𝑖𝑡(1) ∗ 𝑔𝑡(1) + 𝑓𝑡(1) ∗ 𝑐𝑡−1
(1)
(8)
Given a training set with positive, neutral, and negative comment
samples, we can train a classification model and use it to infer the
opinions of gatekeepers from their comments in real time. We adopted ( )
different sentiment analysis schemes for the English language (Pak & ℎ𝑡 = 𝑡𝑎𝑛ℎ 𝑐𝑡(1) ∗ 𝑜(1)
𝑡 (9)
Paroubek, 2010) and Chinese language (Wu & Xiao, 2013), as shown
in Fig. 1. In addition, we adopt the random forest to train classification
(1)
models instead of SVM and naïve Bayes in the original schemes. 𝑌𝑡 = 𝑊𝑦ℎ ℎ𝑡 + 𝑏(1)
𝑦 (10)

3
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Fig. 1. Opinion inference.

Fig. 2. RNN-based gatekeeping behavior model.

If the RNN cell is a PLSTM cell, then ℎ𝑡 and 𝑌𝑡 are calculated using
the following equations:
( ) 𝑟(2) (2)
𝑡 = 𝑊𝑟ℎ ℎ𝑡 (17)
𝑖(2) (2) (2) (2) (2) (2)
𝑡 = 𝜎 𝑊𝑖𝑥 𝑋𝑡 + 𝑊𝑖𝑟 𝑟𝑡−1 + 𝑊𝑖𝑐 𝑐𝑡−1 + 𝑏𝑖
(2)
(11)

( ) 𝑝(2) (2)
𝑡 = 𝑊𝑝ℎ ℎ𝑡 (18)
𝑓𝑡(2) =𝜎 𝑊𝑓(2)
𝑥 𝑡
𝑋 + 𝑊𝑓(2) 𝑟(2)
𝑟 𝑡−1
+ 𝑊𝑓(2) 𝑐 (2)
𝑐 𝑡−1
+ 𝑏(2)
𝑓
(12)

( ) 𝑌𝑡 = 𝑊𝑦𝑟(2) 𝑟(2) (2) (2) (2)

𝑡 + 𝑊𝑦𝑝 𝑝𝑡 + 𝑏𝑦 (19)
𝑔𝑡(2) (2)
= 𝑡𝑎𝑛ℎ 𝑊𝑔𝑥 𝑋𝑡 + 𝑊𝑔𝑟(2) 𝑟(2)
𝑡−1
+ 𝑏(2)
𝑔 (13)
If the RNN cell is a GRU cell, then ℎ𝑡 and 𝑌𝑡 are calculated using the
following equations:
( )
𝑐𝑡(2) = 𝑓𝑡(2) ∗ 𝑐𝑡−1
(2)
+ 𝑖(2) (2)
𝑡 ∗ 𝑔𝑡 (14) 𝑟(3) (3) (3)
𝑡 = 𝜎 𝑊𝑟𝑥 𝑋𝑡 + 𝑊𝑟ℎ ℎ𝑡−1 + 𝑏𝑟
(3)
(20)

( ) ( )
𝑜(2) (2) (2) (2) (2) (2) (2)
𝑡 = 𝜎 𝑊𝑜𝑥 𝑋𝑡 + 𝑊𝑜𝑟 𝑟𝑡−1 + 𝑊𝑜𝑐 𝑐𝑡 + 𝑏𝑜 (15) 𝑧(3) (3) (3) (3)
𝑡 = 𝜎 𝑊𝑧𝑥 𝑋𝑡 + 𝑊𝑧ℎ ℎ𝑡−1 + 𝑏𝑧 (21)

( ) ( ( ) )
ℎ𝑡 = 𝑜(2) (2)
𝑡 ∗ 𝑡𝑎𝑛ℎ 𝑐𝑡 (16) (3)
ℎ̃ 𝑡 = 𝑡𝑎𝑛ℎ 𝑊ℎ𝑥 𝑋𝑡 + 𝑟(3) (3) (3)
𝑡 ∗ 𝑊ℎℎ ℎ𝑡−1 + 𝑏ℎ (22)

4
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Table 1
Twitter dataset.
Popular tweets Total Training set Testing set
Truthful news 3450 2300 1150
Fake news 3030 2000 1030

Table 2
Weibo dataset.
Popular Weibo posts Total Training set Testing set
Truthful news 4070 2700 1370
Fake news 3850 2500 1350

5.1.1. Twitter data set

To collect truthful news samples on Twitter, we used the search
function to search for tweets on the hottest topics. The truthful tweets
were then manually extracted. Besides, to collect fake news samples,
we first use search keywords such as ‘‘Really?’’, ‘‘What?’’, ‘‘Is this true?’’
to obtain the suspicious tweets. We then manually examined the search
results to select fake news samples. In this way, we obtained 3450 pop-
ularly truthful tweets and 3030 widespread fake tweets. Through the
Twitter API, we can collect relevant information on all the gatekeepers
of these tweets. The dataset is summarized in Table 1. The period of
data collection on Twitter was from December 2014 to June 2017.

5.1.2. Weibo data set

Weibo is an important social network in China. Similar to the
Twitter data collection, we obtained 4070 popular truthful posts and
3850 widespread fake posts on Weibo. Then, the relevant information
Fig. 3. Fake news detection flowchart.
of all gatekeepers is collected using the official API provided by Weibo.
The dataset is summarized in Table 2. The time period of the Weibo
data collection was from January 2015 to June 2017.
( )
ℎ𝑡 = 1 − 𝑧(3)
𝑡 ∗ ℎ𝑡−1 + 𝑧(3) ̃
𝑡 ∗ ℎ𝑡 (23) 5.2. Estimation criteria

( ) In the gatekeeper confidence estimation and sentiment analysis

(3)
𝑌𝑡 = 𝜎 𝑊𝑦ℎ ℎ𝑡 (24) experiments, we adopted precision and recall as the evaluation metrics.
In the equations above, W with different superscripts and subscripts In the fake news detection experiments, accuracy, recall, F1, and loss
represents different weight matrices, 𝑏 with different superscripts and were used as evaluation metrics. The true positive rate (TPR) and false
subscripts represents different bias vectors, 𝜎 denotes the sigmoid positive rate (FPR) were used to plot receiver operating characteristic
function, 𝑡𝑎𝑛ℎ denotes the hyperbolic tangent function, and ∗ denotes (ROC) curves. The evaluation metrics are expressed as follows:
the element-wise product. True positive (TP): The number of samples in the positive class that
For evaluation, we compare several recurrent neural networks, the model correctly predicts.
including GRU (Cho et al., 2014), DSTP-RNN (Liu et al., 2020), In- True negative (TN): The number of samples in the negative class
dRNN (Li et al., 2018), PLSTM (Sak et al., 2014), LSTM (Hochreiter that the model correctly predicts.
& Schmidhuber, 1997), and simple RNN (Lipton, 2015). False positive (FP): The number of samples in the positive class that
the model incorrectly predicts.
4. Fake news detection False negative (FN): The number of samples in the negative class
that the model incorrectly predicts.
The fake news detection method proposed in this study comprises 𝑇𝑃
𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = (25)
two phases. In the first phase, we trained the RNN-based gatekeeping 𝑇𝑃 + 𝐹𝑃
behavior model (RGBM) using a training dataset with truthful and fake
news. In the second phase, every trending news item was monitored in 𝑇𝑃
real time. Once trending news is commented or forwarded, the corre- 𝑟𝑒𝑐𝑎𝑙𝑙 = (26)
𝑇𝑃 + 𝐹𝑁
sponding observation sequence is updated, and the output is calculated
using Eq. (3). If the output is larger than a predefined threshold, the
𝑇𝑃 + 𝑇𝑁
news will be considered truthful. Otherwise, we classify the news as 𝑎𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (27)
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁
fake news and remove it from the monitoring targets. A fake news
detection flowchart for the proposed method is shown in Fig. 3. The ( )
proposed method can detect fake news in real time. 𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 × 𝑟𝑒𝑐𝑎𝑙𝑙
𝐹1 = 2 × (28)
𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 + 𝑟𝑒𝑐𝑎𝑙𝑙
5. Experiments
𝑇𝑃
5.1. Data sets 𝑇𝑃𝑅 = (29)
𝐹𝑁 + 𝑇𝑃

In this study, we collected truthful and fake news from Twitter and
𝐹𝑃
Weibo to evaluate the proposed method. 𝐹𝑃𝑅 = (30)
𝑇𝑁 + 𝐹𝑃

5
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Fig. 4. Precision and recall results for gatekeeper confidence estimation on Twitter Fig. 5. Training and testing time results for gatekeeper confidence estimation on
dataset. Twitter dataset.

training phase. Regarding the testing time, C4.5 and random forest are
∑[ ( ) ( ) ( )] the fastest algorithms, both finishing within 0.1 s. Naïve Bayes and SVM
𝑙𝑜𝑠𝑠 = − 𝑢𝑡 × ln 𝑢∗𝑡 + 𝑢𝑡 − 1 ln 1 − 𝑢∗𝑡 (31) use approximately 2 s, whereas kNN requires over 5 s.
𝑡
Hot news usually attracts bursts of forwards and comments as it
In Eq. (31), 𝑢𝑡 is the actual label value, 𝑢∗𝑡 is the RNN-predicted label spread. When we keep track of gatekeeping behavior in real time, we
value. The lower the loss value, the better is the model. The higher the need to predict the confidence levels of gatekeepers at high speed.
accuracy, precision, recall, and F1 values, the more successful is the Random forest offers good classification accuracy and a much faster
model. In addition, the training and testing times are important metrics online classification speed, according to the above experimental results.
for evaluating the performance of the model. Therefore, random forest is considered the best option, and we use it
in the following steps.
5.3. Gatekeeper confidence estimation results
5.4. Sentiment analysis results
Based on the predefined features, we first performed the k-means
algorithm to divide the gatekeepers of the collected dataset into k The analysis schemes proposed in Pak and Paroubek (2010) and Wu
groups. We then manually checked and filtered the gatekeepers in and Xiao (2013) are adopted to infer the opinions of gatekeepers,
each cluster such that each cluster corresponded to a confidence level. from their English and Chinese comments, respectively. Both schemes
Because the checking and filtering of gatekeepers all require manual were used to train machine learning models that divided input texts
intervention, if the number of confidence levels is too large, it becomes into three classes: positive, negative, and neutral. The base learning
difficult to distinguish between the different levels. Otherwise, if the algorithms used in the original schemes were naïve Bayes and SVM.
number is too small, the difference in the confidence of the different In addition, we explored the feasibility of using the random forest as
gatekeepers is not reflected. Therefore, we referred to the five-star the base classifier.
rating system and divided the confidence of the gatekeepers into five For the purpose of evaluation, we collected 3000 comments for
levels. each class of positive, neutral, and negative on the Twitter and Weibo
Then, 3000 gatekeepers were randomly extracted in each cluster datasets. Next, we investigated the effectiveness and efficiency of gate-
to construct the training set. Then, a supervised learning algorithm keeper opinion mining. We used 10-fold cross-validation based on
was used to train the classifier to predict the confidence level of the the above-labeled comment datasets. In this experiment, precision and
given gatekeeper. For evaluation, we compared several popular learn- recall were used as evaluation metrics. We first examine Twitter data
ing algorithms. Precision and recall were used as evaluation metrics. in English. Fig. 6 shows the precision and recall results. Naïve Bayes
The experimental results derived from the two datasets exhibited the (NB), which is the base classifier used in the original scheme in Pak and
same pattern. Here, we only present Twitter results owing to space Paroubek (2010), achieved a precision of 0.751 and recall of 0.726. In
limitations. Based on the Twitter dataset, Fig. 4 presents the precision comparison, random forest (RF) yielded a higher precision rate of 0.783
and recall results for the different learning algorithms. In this exper- and a higher recall rate of 0.775. Fig. 7 shows the efficiency results
iment, 10-fold cross-validation was adopted. As shown in Fig. 4, the in terms of the training and testing times in seconds. We can see that
precision of SVM was the highest. Random forest (RF) has the next naïve Bayes is extremely fast in training and uses only 0.043 s, whereas
highest precision, followed by C4.5. The precision of kNN (k = 9) and random forest is much faster in testing and uses only 0.047 s.
naïve Bayes (NB) was relatively low at approximately 0.92. SVM also Figs. 8 and 9 illustrate the results obtained from the Weibo dataset
yielded the highest recall, followed by random forest, C4.5, and naïve in the Chinese language. The base classifier used in the original scheme
Bayes. The recall rate of kNN was lower than 0.9. in Wu and Xiao (2013), SVM, yielded 0.594 precision and 0.558 recall.
Fig. 5 compares the different learning algorithms in terms of the Again, random forest increased the precision and recall rates to 0.665
time used for training and testing on the Twitter dataset. Note that and 0.647, respectively. Random forest is faster than SVM in terms of
the results are obtained on a computer with 4 × 3.46 GHz Intel training. More importantly, it requires much less time than the SVM
Xeonprocessors and 8 GB of RAM. The training times of C4.5 and for classification. That is, 0.058 s versus 1.532 s. We can see that the
naïve Bayes are both less than 0.2 s. SVM and random forest required accuracy of sentiment analysis in English is better than that in Chinese.
approximately 3 s to finish the training. The kNN does not have a

6
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Fig. 9. Training and testing time results for sentiment analysis on Weibo dataset.
Fig. 6. Precision and recall results for sentiment analysis on Twitter dataset.

Fig. 7. Training and testing time results for sentiment analysis on Twitter dataset.

Fig. 10. Histogram of observation sequence lengths on Twitter dataset.

the naïve Bayes-based and SVM-based methods. Therefore, we used the

random forest for opinion mining in our gatekeeper behavior analysis
framework.

5.5. Fake news detection results

In this section, we evaluate the effectiveness of the proposed method

for fake news detection on the Twitter and Weibo datasets. Based on
the estimated gatekeeper confidence level and the predicted gatekeeper
opinion, we can derive the observation sequences of the gatekeeping
behavior. A histogram of the observation sequence lengths on the
Twitter dataset is shown in Fig. 10. On the Twitter dataset, the ob-
servation sequence length of truthful news was mainly concentrated
in 4000–7000, and the observation sequence length of fake news was
mainly concentrated in 1000–4000. Fig. 11 presents a histogram of
the observation sequence lengths on the Weibo dataset. On the Weibo
Fig. 8. Precision and recall results for sentiment analysis on Weibo dataset. dataset, the observation sequence length of truthful news was mainly
concentrated in the 4000–6000, and the observation sequence length
of fake news was mainly concentrated in the 1000–3000.
In summary, we find that random forest is a better option for the Then, we randomly selected a number of observation sequences as
base classifier in sentiment analysis schemes for both English and Chi- the training set and the corresponding residual observation sequences
nese. In terms of prediction accuracy and testing time, it outperformed as the testing set. The training and testing sets of the Twitter and

7
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Fig. 12. Training accuracy on Twitter dataset.

Fig. 11. Histogram of observation sequence lengths on Weibo dataset.

Weibo datasets are listed in Tables 1 and 2, respectively. For evalu-

ation purposes, we compared a number of RNNs, including GRU (Cho
et al., 2014), DSTP-RNN (Liu et al., 2020), IndRNN (Li et al., 2018),
PLSTM (Sak et al., 2014), LSTM (Hochreiter & Schmidhuber, 1997),
and simple RNN (Lipton, 2015), to find the best RNN to detect social
network fake news. In the training phase of the RNN, the accuracy
and loss were used as evaluation metrics. In the testing phase of
the RNN, the accuracy, recall, F1, and loss were used as evaluation
metrics. The experimental environment was built on a computer with
4 × 3.46 GHz Intel Xeonprocessors and 8 GB of RAM, Ubuntu OS
(version:20.04.1), Keras (version:2.2.5), Python (version:3.6.2), and
Tensorflow (version:1.14.0).

5.5.1. Detection results on Twitter data set

The training accuracy and training loss are shown in Figs. 12 and
13, respectively, with the number of epochs changing from 1 to 100.
When a complete dataset passes through the RNN once and back once,
the process is called an epoch. The epoch represents the learning phase
Fig. 13. Training loss on Twitter dataset.
of the RNN. In the training phase, the DSTP-RNN achieved a training
accuracy of 0.9985, which is higher than that of the other RNNs. The
training accuracy of GRU was 0.9978, followed by IndRNN, PLSTM,
and LSTM. The simple RNN had the lowest training accuracy of 0.9620.
In contrast, DSTP-RNN exhibited the lowest training loss at 0.0119. The
lowest training loss values for GRU, IndRNN, PLSTM, and LSMT were
0.0176, 0.0192, 0.0207, and 0.0258, respectively. When the epoch was
82, the training loss of the simple RNN reached the lowest value of
0.1323.
After training, we used the testing set to evaluate the different
RNNs. The comparative test results are shown in Fig. 14. The DSTP-
RNN had the highest accuracy, recall, and F1 values of 0.990, 0.982,
and 0.981, respectively. GRU had the second highest accuracy, re-
call, and F1 values of 0.988, 0.981, and 0.979, respectively. Mean-
while, DSTP-RNN achieved the lowest loss, followed by GRU, IndRNN,
PLSTM, and LSTM. The simple RNN performs the worst, with the lowest
accuracy, recall, F1, and highest loss.
The training and testing times of the different RNNs are listed in
Table 3. The simple RNN requires the least training and testing times.
However, the accuracy of the simple RNN is significantly lower than
that of other RNNs. Moreover, the accuracy of the simple RNN fluctu-
ated significantly with the epoch. The training and testing times of the
GRU were much shorter than those of the DSTP-RNN, IndRNN, PLSTM,
and LSTM. Therefore, GRU has the best comprehensive performance;
Fig. 14. Comparative testing results on Twitter dataset.
that is, the classification accuracy is good, and the online classification

8
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Table 3
Training time and testing time of different RNNs on twitter dataset.
RNNs Training time (s) Testing time (s)
GRU 15,682 76
DSTP-RNN 63,462 558
IndRNN 17,329 104
PLSTM 47,125 403
LSTM 19,856 119
Simple RNN 6341 35

Table 4
Training time and testing time of different RNNs on Weibo dataset.
RNNs Training time (s) Testing time (s)
GRU 16,425 133
DSTP-RNN 69,588 617
IndRNN 23,556 149
PLSTM 52,674 456
LSTM 25,132 175
Simple RNN 8281 72

Fig. 16. Training loss on Weibo dataset.

Fig. 15. Training accuracy on Weibo dataset.

speed is much faster. Therefore, GRU is more suitable for practical

applications.
Fig. 17. Comparative testing results on Weibo dataset.

5.5.2. Detection results on Weibo data set

As the number of epochs changes from 1 to 100, the training accu-
racy and training loss on the Weibo dataset are shown in Figs. 15 and 5.5.3. Comparison with the hidden Markov model
16. The DSTP-RNN achieved the highest training accuracy of 0.9977. The hidden Markov model (HMM) is a doubly stochastic pro-
cess, with a discrete-time finite-state Markov chain as the underlying
When the number of epochs was 53, the training accuracy of the GRU
stochastic process. The state sequence is not observable but influences
achieved the highest value of 0.9972. The highest values of training
another stochastic process that produces a sequence of observations. A
accuracy for IndRNN, PLSTM, and LSMT were 0.9969, 0.9966, and
variable number of observations can be obtained for each state. HMM
0.9957, respectively. When the number of epochs was 90, the training
is widely used in learning from sequence data.
accuracy of the simple RNN achieved the highest value of 0.9538. The
For comparison, we also analyzed the performance of the hidden
training losses of the DSTP-RNN and GRU were the lowest at 0.013 and
Markov model using the same training set and testing set as the RNNs.
0.0184, respectively.
Based on the variation in the false positive rate on the 𝑥-axis and the
After training, we tested the different RNNs. A comparison of the true positive rate on the 𝑦-axis, we plotted receiver operating character-
test results is presented in Fig. 17. The training and testing times of istic (ROC) curves on different axes. The area under the ROC curve was
the RNNs are listed in Table 4. The experimental results again show used as a benchmark for the models. The larger the area under the ROC
that GRU has the best comprehensive performance. Therefore, we chose curve, the better the performance of the model. Fig. 18 presents the
GRU in the gatekeeping behavior model. ROC curves of the HMM on the Twitter and Weibo datasets. It can be
It can be observed that the experimental results of the RNN-based seen that HMM outperforms the Weibo dataset on the Twitter dataset.
model on the Twitter dataset are better than those on the Weibo The test results are shown in Fig. 19. For the Twitter dataset, the
dataset. This is because the accuracy of English opinion inference accuracy, recall, F1, and loss of HMM were 0.968, 0.961, 0.957, and
is significantly higher than that of Chinese opinion inference (see 0.085, respectively. HMM, meanwhile, had an accuracy of 0.960, recall
Section 5.4). of 0.953, F1 of 0.948, and loss of 0.091 on the Weibo dataset. It is clear

9
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Fig. 20. Timeliness evaluation results on the first 1/3 part of the testing set of Twitter
Fig. 18. ROC curves of HMM. dataset.

Table 5
Comparison results on twitter dataset.
Methods Accuracy Recall F1
RGBM 0.988 0.981 0.979
Davoudi et al. (2022) 0.973 0.965 0.962
Xu et al. (2022) 0.892 0.886 0.885
Reis et al. (2019) 0.865 0.857 0.853
Ruchansky et al. (2017) 0.952 0.946 0.948

Table 6
Comparison results on Weibo dataset.
Methods Accuracy Recall F1
RGBM 0.982 0.975 0.973
Davoudi et al. (2022) 0.970 0.961 0.958
Xu et al. (2022) 0.887 0.883 0.880
Reis et al. (2019) 0.859 0.852 0.847
Ruchansky et al. (2017) 0.947 0.942 0.939

Table 7
Statistical analysis of detection results on twitter dataset.
Fig. 19. Testing results of HMM.
Statistical features Accuracy Recall F1 Loss
Maximum 0.991 0.984 0.983 0.057
Minimum 0.984 0.976 0.972 0.053
that GRU, DSTP-RNN, IndRNN, PLSTM, and LSTM all outperformed Mean 0.988 0.981 0.979 0.055
HMM. Variance 0.000011 0.000012 0.000016 0.000003

5.5.4. Comparison with existing approaches

Table 8
In this section, we compared our proposed method with several Statistical analysis of detection results on Weibo dataset.
state-of-the-art approaches (Davoudi et al., 2022; Reis et al., 2019; Statistical features Accuracy Recall F1 Loss
Ruchansky et al., 2017; Xu et al., 2022). Accuracy, recall, and F1
Maximum 0.987 0.979 0.978 0.063
were used as the evaluation metrics. The comparison results based Minimum 0.978 0.970 0.967 0.058
on the Twitter and Weibo datasets are presented in Tables 5 and 6, Mean 0.982 0.975 0.973 0.061
respectively. The results shown in the tables makes it clear that our Variance 0.000013 0.000015 0.000019 0.000008
method outperforms the existing approaches.

5.5.5. Statistical analysis of fake news detection results we analyzed the performance of our method on new testing sets. The
Based on the Twitter and Weibo datasets, we conducted multiple
evaluation results for the new testing sets of the Twitter dataset are
rounds of training and testing using the method proposed in this paper.
presented in Figs. 20 and 21. The evaluation results for the new Weibo
The statistical analysis of the detection results is presented in Tables 7
testing sets are shown in Figs. 22 and 23.
and 8. It can be observed that the performance of the proposed method
was relatively stable. It can be seen that our method can effectively detect fake news in
the early and middle stages of news propagation. In the early stage of
5.5.6. Timeliness evaluation news propagation, our method had an overall accuracy of 0.940, recall
In this section, the first 1/3 and first 2/3 parts of all sequences of of 0.935, F1 of 0.934, and loss of 0.077. In the middle stage of news
the testing set are extracted as new testing sets (roughly corresponding propagation, the overall accuracy, recall, F1, and loss of our method
to the early and middle stages of news propagation). Subsequently, were 0.971, 0.967, 0.964, and 0.071, respectively.

10
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Fig. 21. Timeliness evaluation results on the first 2/3 part of the testing set of Twitter Fig. 23. Timeliness evaluation results on the first 2/3 part of the testing set of Weibo
dataset. dataset.

7. Conclusion

In the study, we focused on investigating the issues faced during

social network fake news detection. For this purpose, we considered
users who are actively involved in the news dissemination process
as gatekeepers. We then analyzed gatekeeping behavior using RNNs,
from the perspectives of ‘who they are’ and ‘what they think’. The
random forest classification algorithm was then used to estimate the
confidence level of the gatekeeper, achieving a precision of 0.968 and
recall of 0.957. The gatekeeper opinion was inferred using the random
forest classification algorithm, with a precision of 0.783 and a recall
of 0.775 for English and a precision of 0.665 and a recall of 0.647
for Chinese. We compared several RNNs to find the best RNN for
detecting social network fake news. The experimental results based on
Twitter and Weibo datasets show that GRU has the best comprehensive
performance. The fake news detection method proposed in this paper
achieved an overall accuracy of 0.985, recall of 0.978, F1 score of
Fig. 22. Timeliness evaluation results on the first 1/3 part of the testing set of Weibo 0.976, and loss of 0.058, outperforming several existing methods.
dataset. The timeliness evaluation results demonstrated that our method can
effectively detect fake news in the early and middle stages of news
propagation.
6. Discussion The fake news detection method proposed in this study does not
consider the credibility of the news publishers. In future research, we
In the RNN-based gatekeeping behavior model, the random forest intend to integrate the credibility assessment of the news publisher
is used to infer the gatekeepers’ opinions. On the Twitter dataset, into our method to improve its accuracy. We also intend to use more
random forest had a precision of 0.783 in English opinion inference. diverse datasets to test the performance of our method, and test the
In contrast, on the Weibo dataset, the precision of the random forest performance of our method online on the actual social networks.
in Chinese opinion inference was 0.665. In this study, gatekeepers’
opinions were divided into four types: negative, positive, neutral, and CRediT authorship contribution statement
no comment. If a gatekeeper forwards news without commenting, we
can obtain the gatekeeper’s opinion directly, without using sentiment
Bailin Xie: Investigation, Conceptualization, Methodology, Data
analysis. On the Twitter and Weibo datasets, for 90% of the news, about
curation, Writing – original draft. Qi Li: Software, Validation.
65% of the gatekeepers only forward the news without commenting.
We can infer the opinions of these gatekeepers with 100% accuracy;
Declaration of competing interest
therefore, our gatekeeping behavior analysis framework can achieve an
overall precision of 0.909 and 0.861 for English opinion inference and
Chinese opinion inference, respectively. Therefore, the performance of The authors declare that they have no known competing finan-
sentiment analysis had little impact on our fake news detection method. cial interests or personal relationships that could have appeared to
In addition, a large amount of new news spreads on social networks influence the work reported in this paper.
every day, and most new news is not widespread. Therefore, even if
there is fake news in unpopular news, it will not confuse the public’s Data availability
attention, which means that we need to detect fake news only in the
trending news on social networks. Hence, our method is cost efficient. Data will be made available on request.

11
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Acknowledgments Khan, T., Michalas, A., & Akhunzada, A. (2021). Fake news outbreak 2021: Can we
stop the viral spread? Journal of Network and Computer Applications, 190, Article
This work was supported by the Guangdong Basic and Applied 103112. http://dx.doi.org/10.1016/j.jnca.2021.103112.
Khattar, D., Goud, J. S., Gupta, M., & Varma, V. (2019). MVAE: Multimodal variational
Basic Research Foundation, China (Grant No. 2018A0303130045), and
autoencoder for fake news detection. In The world wide web conference (pp. 2915–
the Science and Technology Program of Guangzhou, China (Grant No. 2921). New York, NY, USA: Association for Computing Machinery, http://dx.doi.
201904010334) org/10.1145/3308558.3313552.
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F.,
References Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., Schudson, M., Sloman, S.
A., Sunstein, C. R., Thorson, E. A., Watts, D. J., & Zittrain, J. L. (2018). The science
Al-Khalifa, H. S., & Al-Eidan, R. M. (2011). An experimental system for measuring of fake news. Science, 359(6380), 1094–1096. http://dx.doi.org/10.1126/science.
the credibility of news content in Twitter. International Journal of Web Information aao2998.
Systems, 7, 130–151. http://dx.doi.org/10.1108/17440081111141772. Li, K., Guo, B., Liu, J., Wang, J., Ren, H., Yi, F., & Yu, Z. (2022). Dynamic probabilistic
Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. graphical model for progressive fake news detection on social media platform. ACM
Journal of Economic Perspectives, 31(2), 211–236. http://dx.doi.org/10.1257/jep.31. Transactions on Intelligent Systems and Technology, 13(5), http://dx.doi.org/10.1145/
2.211. 3523060.
Bian, T., Xiao, X., Xu, T., Zhao, P., Huang, W., Rong, Y., & Huang, J. (2020). Li, S., Li, W., Cook, C., Zhu, C., & Gao, Y. (2018). Independently recurrent neural
Rumor detection on social media with bi-directional graph convolutional networks. network (indrnn): Building a longer and deeper RNN. In Proceedings of the IEEE
Proceedings of the AAAI Conference on Artificial Intelligence, 34(01), 549–556. http: conference on computer vision and pattern recognition.
//dx.doi.org/10.1609/aaai.v34i01.5393. Liang, C., Liu, Z., & Sun, M. (2012). Expert finding for microblog misinformation
Bibi, M., Abbasi, W. A., Aziz, W., Khalil, S., Uddin, M., Iwendi, C., & Gadekallu, T. R. identification. In COLING (Posters) (pp. 703–712). Citeseer.
(2022). A novel unsupervised ensemble framework using concept-based linguistic Liao, H., Liu, Q., Shu, K., & xie, X. (2020). Incorporating user-comment graph for fake
methods and machine learning for twitter sentiment analysis. Pattern Recognition new detection. (pp. arXiv–2011). arXiv e-prints.
Letters, 158, 80–86. http://dx.doi.org/10.1016/j.patrec.2022.04.004. Lipton, Z. C. (2015). A critical review of recurrent neural networks for sequence
Bodaghi, A., & Oliveira, J. (2022). The theater of fake news spreading, who plays learning. CoRR abs/1506.00019. arXiv:1506.00019.
which role? A study on real graphs of spreading on Twitter. Expert Systems with Liu, Y., Gong, C., Yang, L., & Chen, Y. (2020). DSTP-RNN: A dual-stage two-phase
Applications, 189, Article 116110. http://dx.doi.org/10.1016/j.eswa.2021.116110. attention-based recurrent neural network for long-term and multivariate time series
Castillo, C., Mendoza, M., & Poblete, B. (2011). Information credibility on Twitter. In
prediction. Expert Systems with Applications, 143, Article 113082. http://dx.doi.org/
Proceedings of the 20th international conference on world wide web (pp. 675–684).
10.1016/j.eswa.2019.113082.
New York, NY, USA: Association for Computing Machinery, http://dx.doi.org/10.
Liu, Y., & Wu, Y.-F. (2018). Early detection of fake news on social media through prop-
1145/1963405.1963500.
agation path classification with recurrent and convolutional networks. Proceedings
Ceron, W., de-Lima-Santos, M.-F., & Quiles, M. G. (2021). Fake news agenda in the
of the AAAI Conference on Artificial Intelligence, 32(1), 354–361.
era of COVID-19: Identifying trends through fact-checking content. Online Social
Liu, Q., Yu, F., Wu, S., & Wang, L. (2018). Mining significant microblogs for
Networks and Media, 21, Article 100116. http://dx.doi.org/10.1016/j.osnem.2020.
misinformation identification: An attention-based approach. ACM Transactions on
100116.
Intelligent Systems and Technology, 9(5), http://dx.doi.org/10.1145/3173458.
Chandra, S., Mishra, P., Yannakoudakis, H., Nimishakavi, M., Saeidi, M., & Shutova, E.
(2020). Graph-based modeling of online communities for fake news detection. CoRR Ma, J., Gao, W., Joty, S., & Wong, K.-F. (2020). An attention-based rumor detec-
abs/2008.06274. arXiv:2008.06274. tion model with tree-structured recursive neural networks. ACM Transactions on
Chen, T., Li, X., Yin, H., & Zhang, J. (2018). Call attention to rumors: Deep attention Intelligent Systems and Technology, 11(4), http://dx.doi.org/10.1145/3391250.
based recurrent neural networks for early rumor detection. In M. Ganji, L. Rashidi, Ma, J., Gao, W., Mitra, P., Kwon, S., Jansen, B. J., Wong, K.-F., & Cha, M. (2016).
B. C. M. Fung, & C. Wang (Eds.), Trends and applications in knowledge discovery and Detecting rumors from microblogs with recurrent neural networks. In Proceedings
data mining (pp. 40–52). Cham: Springer International Publishing. of the 25th international joint conference on artificial intelligence (IJCAI 2016) (pp.
Cheng, M., Nazarian, S., & Bogdan, P. (2020). Vroc: Variational autoencoder-aided 3818–3824). New York, USA: AAAI Press.
multi-task rumor classifier based on text. In Proceedings of the web conference 2020 Ma, J., Gao, W., & Wong, K.-F. (2018). Rumor detection on twitter with tree-
(pp. 2892–2898). New York, NY, USA: Association for Computing Machinery. structured recursive neural networks. In Proceedings of the 56th annual meeting of
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., the association for computational linguistics (ACL 2018) (pp. 1980–1989). Association
& Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for Computational Linguistics.
for statistical machine translation. arXiv:1406.1078. Ma, J., Gao, W., & Wong, K.-F. (2019). Detect rumors on Twitter by promoting
Davoudi, M., Moosavi, M. R., & Sadreddini, M. H. (2022). DSS: A hybrid deep model for information campaigns with generative adversarial learning. In The world wide
fake news detection using propagation tree and stance network. Expert Systems with web conference (pp. 3049–3055). New York, NY, USA: Association for Computing
Applications, 198, Article 116635. http://dx.doi.org/10.1016/j.eswa.2022.116635. Machinery, http://dx.doi.org/10.1145/3308558.3313741.
Ebrahimi Fard, A., Mohammadi, M., Chen, Y., & Van de Walle, B. (2019). Computa- Mittal, M., Iwendi, C., Khan, S., & Rehman Javed, A. (2021). Analysis of security and
tional rumor detection without non-rumor: A one-class classification approach. IEEE energy efficiency for shortest route discovery in low-energy adaptive clustering
Transactions on Computational Social Systems, 6(5), 830–846. http://dx.doi.org/10. hierarchy protocol using Levenberg-Marquardt neural network and gated recurrent
1109/TCSS.2019.2931186. unit for intrusion detection system. Transactions on Emerging Telecommunications
Garimella, K., De Francisci Morales, G., Gionis, A., & Mathioudakis, M. (2018). Technologies, 32(6), Article e3997. http://dx.doi.org/10.1002/ett.3997.
Political discourse on social media: Echo chambers, gatekeepers, and the price of Monti, F., Frasca, F., Eynard, D., Mannion, D., & Bronstein, M. M. (2019). Fake news
bipartisanship. In Proceedings of the 2018 world wide web conference (pp. 913–922). detection on social media using geometric deep learning. CoRR abs/1902.06673.
Republic and Canton of Geneva, CHE: International World Wide Web Conferences arXiv:1902.06673.
Steering Committee, http://dx.doi.org/10.1145/3178876.3186139. Olan, F., Jayawickrama, U., Arakpogun, E. O., Suklan, J., & Liu, S. (2022). Fake
Guo, H., Cao, J., Zhang, Y., Guo, J., & Li, J. (2018). Rumor detection with hi-
news on social media: the impact on society. Information Systems Frontiers, 1–16.
erarchical social attention network. In Proceedings of the 27th ACM international
http://dx.doi.org/10.1007/s10796-022-10242-z.
conference on information and knowledge management (pp. 943–951). New York, NY,
Pak, A., & Paroubek, P. (2010). Twitter as a corpus for sentiment analysis and opinion
USA: Association for Computing Machinery, http://dx.doi.org/10.1145/3269206.
mining. In Proceedings of the seventh international conference on language resources
3271709.
and evaluation (LREC’10). Valletta, Malta: European Language Resources Association
Gupta, M., Zhao, P., & Han, J. (2012). Evaluating event credibility on Twitter. In
(ELRA).
Proceedings of the 2012 SIAM international conference on data mining (pp. 153–164).
Raponi, S., Khalifa, Z., Oligeri, G., & Pietro, R. D. (2022). Fake news propagation: A
http://dx.doi.org/10.1137/1.9781611972825.14.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, review of epidemic models, datasets, and insights. ACM Transactions on the Web,
9(8), 1735–1780. http://dx.doi.org/10.1162/neco.1997.9.8.1735. http://dx.doi.org/10.1145/3522756.
Huang, Q., Yu, J., Wu, J., & Wang, B. (2020). Heterogeneous graph attention networks Reis, J. C. S., Correia, A., Murai, F., Veloso, A., & Benevenuto, F. (2019). Supervised
for early detection of rumors on Twitter. In 2020 international joint conference on learning for fake news detection. IEEE Intelligent Systems, 34(2), 76–81. http:
neural networks (pp. 1–8). http://dx.doi.org/10.1109/IJCNN48605.2020.9207582. //dx.doi.org/10.1109/MIS.2019.2899143.
Iwendi, C., Mohan, S., khan, S., Ibeke, E., Ahmadian, A., & Ciano, T. (2022). Covid- Rohera, D., Shethna, H., Patel, K., Thakker, U., Tanwar, S., Gupta, R., Hong, W.-C.,
19 fake news sentiment analysis. Computers and Electrical Engineering, 101, Article & Sharma, R. (2022). A taxonomy of fake news classification techniques: Survey
107967. http://dx.doi.org/10.1016/j.compeleceng.2022.107967. and implementation aspects. IEEE Access, 10, 30367–30394. http://dx.doi.org/10.
Jin, Z., Cao, J., Guo, H., Zhang, Y., & Luo, J. (2017). Multimodal fusion with 1109/ACCESS.2022.3159651.
recurrent neural networks for rumor detection on microblogs. In Proceedings of Ruchansky, N., Seo, S., & Liu, Y. (2017). CSI: A hybrid deep model for fake news
the 25th ACM international conference on multimedia (pp. 795–816). New York, NY, detection. In Proceedings of the 2017 ACM on conference on information and knowledge
USA: Association for Computing Machinery, http://dx.doi.org/10.1145/3123266. management (pp. 797–806). New York, NY, USA: Association for Computing
3123454. Machinery, http://dx.doi.org/10.1145/3132847.3132877.

12
B. Xie and Q. Li Expert Systems With Applications 231 (2023) 120716

Sak, H., Senior, A., & Beaufays, F. (2014). Long short-term memory based recurrent Yu, F., Liu, Q., Wu, S., Wang, L., & Tan, T. (2017). A convolutional approach for
neural network architectures for large vocabulary speech recognition. arXiv:1402. misinformation identification.. In Proceedings of international joint conference on
1128. artificial intelligence (pp. 3901–3907).
Shan, G., Zhao, B., Clavin, J. R., Zhang, H., & Duan, S. (2022). Poligraph: Yu, F., Liu, Q., Wu, S., Wang, L., & Tan, T. (2019). Attention-based convolutional
Intrusion-tolerant and distributed fake news detection system. IEEE Transactions on approach for misinformation identification from massive and noisy microblog posts.
Information Forensics and Security, 17, 28–41. http://dx.doi.org/10.1109/tifs.2021. Computers & Security, 83, 106–121. http://dx.doi.org/10.1016/j.cose.2019.02.003.
3131026. Zhang, Q., Lipani, A., Liang, S., & Yilmaz, E. (2019). Reply-aided detection of
Shim, J.-S., Lee, Y., & Ahn, H. (2021). A link2vec-based fake news detection model misinformation via Bayesian deep learning. In The world wide web conference
using web search results. Expert Systems with Applications, 184, Article 115491. (pp. 2333–2343). New York, NY, USA: Association for Computing Machinery,
http://dx.doi.org/10.1016/j.eswa.2021.115491. http://dx.doi.org/10.1145/3308558.3313718.
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017). Fake news detection on social Zhou, X., & Zafarani, R. (2020). A survey of fake news: Fundamental theories, detection
media: A data mining perspective. ACM SIGKDD Explorations Newsletter, 19(1), methods, and opportunities. ACM Computing Surveys, 53(5), http://dx.doi.org/10.
22–36. http://dx.doi.org/10.1145/3137597.3137600. 1145/3395046.
Song, C., Teng, Y., Zhu, Y., Wei, S., & Wu, B. (2022). Dynamic graph neural network
for fake news detection. Neurocomputing, 505, 362–374. http://dx.doi.org/10.1016/
j.neucom.2022.07.057.
Suzuki, Y., & Nadamoto, A. (2011). Credibility assessment using wikipedia for messages Bailin Xie received his B.S. degree from Qingdao University,
on social network services. In 2011 IEEE ninth international conference on dependable, China. He received his M.S. degree and PhD. degree from
autonomic and secure computing (pp. 887–894). http://dx.doi.org/10.1109/DASC. Sun Yat-Sen University, China. He is currently an assistant
2011.149. professor at the school of Information Science and Technol-
Wang, H., Wang, S., & Han, Y. (2022). Detecting fake news on Chinese social media ogy in Guangdong University of Foreign Studies, China. His
based on hybrid feature fusion method. Expert Systems with Applications, 208, Article research interests focus on online social network, network
118111. http://dx.doi.org/10.1016/j.eswa.2022.118111. security and network behavior model algorithm.
Wu, W., & Xiao, S. (2013). Sentiment analysis of Chinese micro-blog based on
multi-feature and combined classification. Journal of Beijing Information Science &
Technology University, 28, 39–45.
Xia, X., Yang, X., Wu, C., Li, S., & Bao, L. (2012). Information credibility on Twitter Qi Li received her B.S. degree from Guangdong University
in emergency situation. In M. Chau, G. A. Wang, W. T. Yue, & H. Chen (Eds.), of Foreign Studies, China. She is currently a master at the
Intelligence and security informatics (pp. 45–59). Berlin, Heidelberg: Springer Berlin school of Information Science and Technology in Guangdong
Heidelberg. University of Foreign Studies, China. Her research interests
Xu, W., Wu, J., Liu, Q., Wu, S., & Wang, L. (2022). Evidence-aware fake news focus on online social network and network security.
detection with graph neural networks. In Proceedings of the ACM web conference
2022 (pp. 2501–2510). New York, NY, USA: Association for Computing Machinery,
http://dx.doi.org/10.1145/3485447.3512122.
Yang, F., Liu, Y., Yu, X., & Yang, M. (2012). Automatic detection of rumor on sina
weibo. In Proceedings of the ACM SIGKDD workshop on mining data semantics. New
York, NY, USA: Association for Computing Machinery, http://dx.doi.org/10.1145/
2350190.2350203.

13