Received 17 December 2023, accepted 7 January 2024, date of publication 15 January 2024, date of current version 1 February 2024.

Digital Object Identifier 10.1109/ACCESS.2024.3354165

Detection of Turkish Fake News From Tweets

With BERT Models
GÜLSÜM KAYABAŞI KORU AND ÇELEBİ ULUYOL
Computer Forensics Department, Gazi University, 06640 Ankara, Turkey
Corresponding author: Gülsüm Kayabaşi Koru ([email protected])

ABSTRACT As the number of people using social networks increases, more people are using social media
platforms to meet their news needs. Users think that it is easier to follow the agenda by accessing news,
especially on Twitter, rather than newspaper news pages. However, fake news is increasingly appearing
on social media, and it is not always possible for people to obtain correct news from partial news pages
or short Twitter posts. Understanding whether the news shared on Twitter is true or not is an important
problem. Detecting fake tweets is of great importance in Turkish as well as in any language. In this
study, fake news obtained from verification platforms on Twitter and real news obtained from the Twitter
accounts of mainstream newspapers were labeled and, preprocessed using the Zemberek natural language
processing tool developed for the Turkish language, and a dataset named TR_FaRe_News was created.
Then, the TR_FaRe_News dataset was explored using ensemble methods and BoW, TF-IDF, and Word2Vec
vectorization methods for fake news detection. Then a pre-trained BERT deep learning model was fine-
tuned, and variations of the model extended with Bi-LSTM and Convolutional Neural Network (CNN)
layers with the frozen and unfrozen parameters methods were explored. The performance evaluation was
conducted using seven comparable datasets, namely BuzzFeedNews, GossipCop, ISOT, LIAR, Twitter15,
and Twitter16, including an LLM-generated fake news dataset. Analyzing Turkish tweets and using fake
news datasets generated by LLM is considered an important contribution. Accuracy values between 90 and
94% were obtained with the BERT and BERTurk + CNN models with 94% accuracy.

INDEX TERMS Fake news, generated news, ensemble learning, deep learning, BERT.

I. INTRODUCTION 5.6 billion active Internet users worldwide as of January 2023

Internet journalism is a concept frequently used in our age. [3] and according to the research conducted by eBizMba,
Examples, which started with the messages sent by news news sites are among the 15 most popular web pages in
groups to their subscribers, have led to the emergence of 2022 and the first quarter of 2023 [4].
virtual journalism, which we call Internet journalism, with the When the ‘‘We Are Social Digital 2023 Global and Türkiye
development of various software and hardware applications Report’’ is examined, Twitter ranks 4th in the most preferred
over the Internet [1]. With the emergence of virtual journal- social media list after Instagram, WhatsApp and Facebook
ism, the tendency to receive news through social media has with a rate of 66.5% [5]. Considering the statistics above,
begun. it is clear that people access news via Twitter as much as they
Unlike traditional media, social media should not be con- prefer to access news from web pages [6].
sidered just as media; it is an integrated concept that includes On the other hand, fake news causes serious harm to soci-
many dimensions. Journalists need to reach audiences in ety and increasingly appear on news pages and on Twitter.
the cyberworld beyond known means of communication [2]. This has triggered several studies on fake news, includ-
Social media statistics show that there are approximately ing the development of fake news detection and real news
verification techniques. News verification experts have cre-
The associate editor coordinating the review of this manuscript and ated platforms that detect fake news in Turkish to verify
approving it for publication was Wentao Fan . the news that users share on social networks. The news
2024 The Authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.
14918 For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ VOLUME 12, 2024
G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

shared on the webpages of these platforms is also shared • The performance evaluation was conducted with
on Twitter accounts [7]. The main feature of these accounts seven comparable datasets: BuzzFeedNews, GossipCop,
is that they prove whether the news they verify is fake. ISOT, LIAR, Twitter15, and Twitter16, including even a
The mentioned accounts include teyitorg, dogrulukpayicom, GPT-2-generated fake news dataset,
dogrulaorg, gununyalanlari, and malumatfurusorg. This study • The TR_FaRe_News dataset was built in Turkish lan-
utilizes these accounts to identify and uncover instances of guage and used for classification of fake news generated
fake news. Teyitorg and dogrulukpayicom accounts are affil- by LLM,
iated with the International Fact-Checking Network (IFCN), • Accuracy values between 90 and 94% were obtained
an organization that unites verification platforms worldwide with the BERT and BERTurk + CNN models with 94%
to enhance supervision and responsibility [8]. The other accuracy.
accounts (dogrulaorg, gununyalanlari and malumatfurusorg) The subsequent sections of this work are structured as fol-
are non-members. They meet the network’s criteria. These lows: Section II presents a comprehensive analysis of existing
accounts engage in the dissemination of fake news and also literature on the subject matter. The specific architectural
create posts to increase public awareness and facilitate the components of the proposed model are outlined in Section III.
identification of fake news.This study utilizes these stories to The experiments are showcased in Section IV. The findings
identify and uncover instances of misinformation. Real news, and analysis are outlined in Section V. Section VI outlines
which is another focal point in this study, was taken from the issues and constraints. Section VII provides an overview
Twitter accounts of the most clicked mainstream newspapers of potential enhancements for study, future endeavors, and
[8]. Detailed information regarding this section is included in final conclusions.
the dataset section of the study.
This study considered three aspects. First, within the scope
of the study, a dataset named TR_FaRe_News was cre- II. RELATED WORKS
ated, consisting of tweets taken between January 2020 and Scientists in the field of natural language processing (NLP)
December 2022 and shared on the Twitter [7] platform. Sec- are employing machine learning and deep learning techniques
ond, the manually labeled dataset was converted into a vector to identify and counteract fake news, a challenging endeavor
using word representation methods, and the fake news classi- that necessitates thorough comprehension and effective coun-
fication process was carried out with our model created using termeasures.
machine learning algorithms and BERT. Finally, we included Fake news detection is the basis of many tasks, such
fake news datasets generated by large language models like as news accuracy detection and classification. In the lit-
GPT-2 in our scope and compared the model we built with erature, there are many studies on fake news detection.
BERT with human-generated datasets like our own dataset. Because collecting these studies under a single heading would
To summarize the purpose of the study, it was to create cause semantic confusion, they are analyzed under three
a fake news data set consisting of fake and real news in headings in this article. The initial two research employ super-
Turkish, introduce it to the literature and make it available vised machine learning algorithms and ensemble learning
to future researchers, and conduct experiments on the fake techniques, whereas the latter studies utilize deep learning
news dataset created by a large language model such as GPT- methods. The third heading is studies that detect fake news
2 with human-generated datasets and state-of-the-art models by creating a Turkish dataset, since the language used in our
we have created. study was Turkish. The primary objective of this study is to
The following are the contributions to the literature made analyze research conducted in languages other than Turkish,
by this study: categorizing them into two distinct groups. The aim is to
highlight the achievements of studies conducted in Turkish,
considering the intricacies of the language, and to compare
• Fake news obtained from verification platforms on Twit- them with our own research. Additionally, this study aims
ter and real news obtained from the Twitter accounts of to create a dataset in Turkish that can be utilized by other
mainstream newspapers are labeled, researchers.
• The labeled news items were preprocessed with the
Zemberek NLP tool for Turkish, A. FAKE NEWS DETECTION USING MACHINE LEARNING
• The dataset generated after preprocessing is named AND ENSEMBLE LEARNING APPROACHES
TR_FaRe_News, A multitude of research projects utilize machine-learning
• Then the TR_FaRe_News dataset was explored using algorithms to identify and categorize fake news, and a sub-
ensemble methods and BoW, TF-IDF, and Word2Vec stantial amount of inquiries have been undertaken, so enhanc-
vectorization methods for fake news detection, ing the existing body of knowledge on ensemble learning
• After that, a pre-trained BERT deep learning model was techniques [9].
fine-tuned, and variations of the model extended with A study combining linguistic features and knowledge-
Bi-LSTM and CNN layers with the frozen and unfrozen based approaches achieved 94.4% accuracy, outperforming
parameter methods were tested, 89.4% using linguistic features separately. Support Vector

VOLUME 12, 2024 14919

 G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

Machine (SVM) and Random Forest (RF) achieved 97% The study proposes an intelligent detection system using an
accuracy using LIAR dataset [10]. Ensemble Voting Classifier for real and fake news classifica-
The study discusses spam detection using n-gram analysis, tion, utilizing 11 machine learning algorithms and detection
highlighting its advantages for fake content, with a 90% techniques like Gradient Boosting and Ada Boosting, achiev-
success rate achieved using the SVM algorithm [11]. ing 94.5% accuracy [22].
The study [12] presents two news datasets for fake news The UNBiased dataset, a new corpus of text, uses advanced
detection, detailing data collection, annotation, validation, linguistic features, word embeddings, ensemble algorithms,
and linguistic differences. Comparative analyses show a 73% and SVMs to accurately classify fake news [23].
f1-score for automatic and manual identification.
A study conducted to identify fake news on COVID-19 in B. FAKE NEWS DETECTION USING DEEP LEARNING
both Hindi and English languages produced an impressive APPROACHES
accuracy rate of 93.45% in English and 97% in Hindi [9]. The study investigates the efficacy of 19 machine learn-
The study successfully detected fake news on multiple ing methods in identifying fake news across three English
languages using conventional machine learning algorithms, datasets. Out of the total of 19 models, 8 were conventional
with results ranging from 81% for TwitterBR to 95% for deep learning models, while the remaining 5 were pre-trained
btvlifestyle [13]. sophisticated language models such as BERT. The findings
The study used the TF-IDF method to obtain vector repre- indicate that models based on BERT exhibit superior perfor-
sentations of news texts, followed by classification successes mance compared to other models, but Naïve Bayes algorithms
using 23 supervised AI algorithms, and evaluation metrics can reach comparable outcomes [24].
were compared [14]. A study using a Convolutional Neural Network (CNN)
FakeNewsNet, an extensive compilation of fake news, was achieved 85% accuracy in analyzing fake news, highlighting
launched in [15] with the purpose of facilitating studies on the need for a comprehensive understanding of its character-
fake news by offering a wide-ranging collection of news arti- istics [25].
cles, social context, and spatio-temporal data. The analysis Another study proposes a system using a deep learn-
examines the datasets from BuzzFeed and GossipCop from ing model to convert any word in an information message
several angles and emphasizes the benefits of FakeNewsNet into an ideal measurement vector. Word vectorization effec-
in identifying fake news on social media. tively manages high-dimensional data variation, with LSTM
This study conducts a comparison of supervised model’s accuracy reaching 91.73%, surpassing CNN and
machine-learning algorithms in order to automatically detect RNN models [26].
fake news. The systems are evaluated based on the features A study using active learning techniques achieved a 97.1%
extracted from the news [16]. f1-score performance using Multilingual-BERT for solving
A study [17] assessed the efficacy of five machine learning multilingual fake news detection problems [27].
and three deep learning models on two distinct datasets by Researchers developed a deep neural network model to
employing deep learning in conjunction with conventional automatically detect truth in Arabic news or claims, achieving
techniques. The claim was that the key to achieving high test 91% accuracy when applied to an Arabic dataset [28].
accuracy was chunking. The AugFake-BERT approach employs a cutting-edge
The study employed capsule neural networks to detect fake BERT language model to classify data and mitigate under-
news, adopting several word embedding models for news of classing problems by generating synthetic fake data. This
different durations. Static word embeddings are employed for approach achieves an impressive accuracy score of 92.45%
brief news articles, whereas non-static embeddings enable [29].
progressive training and updating during the training phase A method for automatically detecting fake news integrates
for moderate and extensive news articles [18], [19]. both textual and visual characteristics, while maintaining
The work presents a machine learning ensemble method the semantic connections among words. The model attained
to automatically classify news articles, utilizing linguistic classification accuracies of 93% and 92% for the PolitiFact
characteristics to differentiate between fake and real content. and GossipCop datasets, respectively [30].
The approach outperforms individual learners in four real- The study aims to predict fake news items using a NLP-
world datasets [20]. based classifier, comparing results from multiple models and
The paper suggests utilizing an ensemble learning tech- presenting a new design with an attention-like mechanism in
nique to tackle the issue of imbalanced data in Indonesian a CNN [31].
fake news datasets. The study showcased that the random for- The researchers used RNN to read news headlines and
est classifier surpassed the multinomial classifier in ensemble articles, comparing it with advanced systems, but found a
classification, achieving an impressive f-1 score of 0.98. significant issue with overfitting [32].
The Naïve Bayes and support vector machine classifiers, The Bi-LSTM model demonstrated the highest accu-
which were not ensemble models, were used to evaluate racy in feature extraction and stance classification using
660 documents. The f-1 scores obtained were 0.43 and 0.74, deep neural networks, outperforming RNN models and their
respectively [21]. extensions [33].

14920 VOLUME 12, 2024

G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

A study used deep learning techniques to develop a classi- ing fake news. Fig. 1 displays the architectural design of the
fier for predicting fake news stories using RNN models and model.
LSTMs, utilizing the LIAR dataset [34].
The study effectively attained a 98.9% accuracy rate by
employing a BERT-based deep learning method to categorize
parallel segments of a single-layer deep CNN with different
kernel sizes and filters [35].
An investigation on the impact of margin loss CNNs exhib-
ited inferior performance on the LIAR dataset in comparison
to the ISOT dataset for the purpose of fake news detection
[36].

C. FAKE NEWS DETECTION USING TURKISH LANGUAGES

The study focuses on fake news detection in Turkish using
SVM and NB classifiers. The datasets used include term
frequency, TF-IDF, n-gram, style markers, slang usage, url,
accessible link features, Headline and News content compat-
ibility, and Exaggerated Headline usage. The study obtained
a 79% f1-measure using an SVM classifier [37].
The study suggests automatic mechanisms to verify digital
content reliability in libraries without manual verification,
preventing fake news spread. A dataset was generated by uti-
lizing both real news and fake news. The ExtraTrees classifier
was employed, resulting in an impressive accuracy rate of
96.81%. [38].
A study on Twitter identified two popular fake news topics
in August 2019. Data was collected using Twitter Scraper
and labeled using a labeling platform. The dataset comprises
1287 tweets and is the inaugural attempt at identifying Turk-
ish fake news on social media [39].
The study presents Turkish fake news detection approaches
using the SOSYalan Turkish dataset, demonstrating that deep
learning models outperform existing literature in both Turk-
ish and English [40].
This study highlights the importance of developing a fake
news identification model for COVID-19 in the Turkish lan-
guage, proposing an advanced deep-language transformer.
The study developed a model to identify genuine COVID- FIGURE 1. Machine learning models.
19 news in Turkey sourced from social media. There are five
conventional machine learning algorithms and deep learn-
ing algorithms like LSTM, Bi-LSTM, CNN, and GRU were As seen in Fig. 1., the tweet texts in the TR_FaRe_News
tested, with BERT and variations improving efficiency and dataset were first passed through data preprocessing steps.
achieving 98.5% accuracy [41]. Feature extraction was performed with BoW, TF-IDF and
The study evaluated supervised and unsupervised word2Vec and then classified.
machine-learning algorithms on Turkish pseudonym datasets, The performance of the classification models was assessed
achieving an 86% f1-score for supervised algorithms and using commonly employed measures such as accuracy and f1
72% for unsupervised algorithms [42]. values. Calculating these measures necessitates the use of two
parameters: True Positive (TP), True Negative (TN), False
Positive (FP), and False Negative (FN).
III. MACHINE LEARNING AND BERT BASED MODELS The study presents the formulas for calculating the per-
A. EXPLORED MACHINE LEARNING MODELS formance metrics that are used to evaluate the model
This study employed four machine learning algorithms, performance. These formulas are shown in Equations (1)
including MultinomialNB (MNB), RF, LR, and SVM, to clas- and (2).
sify fake news. The study also evaluated the effectiveness of TP + TN
the Voting Classifier (VC) and ensemble systems in classify- Accuracy := (1)
TP + TN + FP + FN
VOLUME 12, 2024 14921
 G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

2xTP
F1 − Measure := (2) pre-trained tasks utilizing unlabeled data [43]. Features were
2xTP + FP + FN extracted from a pre-trained model [44]. The fine-tuning
Classification results are shown in Table 1 below. method incurs lower costs compared to the pre-training
process. The fine-tuning process begins by utilizing the
TABLE 1. Machine learning results. pre-trained parameters. The parameters were updated with
labelled data prepared according to the type of study. Except
for the output layers, the architectures used for fine-tuning
and pre-training were the same. DistilBERTurk and BERTurk
were used in this study.

1) DISTILLBERT/DISTILLBERTURK (MODEL 1)
DistilBERTurk [45] has a transformer architecture that is
similar to that of BERTurk. Distillation, a process carried out
during the pre-training phase, is executed in the fine-tuning
step based on the specific task at hand. The number of lay-
ers was reduced by half, and the algebraic processes were
optimized. By implementing multiple adjustments, Distill-
BERTurk achieved comparable outcomes despite its 40%
smaller size compared to BERTurk [46].
The sentences in the dataset were first tokenized using the
DistilBERTurk tokenizer trained to create word embeddings
(768 dimensions), converted into tensors, and provided to
the model. Subsequently, a basic neural network architecture
consisting of Dense and Dropout layers was used for the for-
ward classification task and training with DistilBERTClass.
TWhen Table 1 is examined, when we compare the This is the first model we created for BERT models within
TR_FaRe_News dataset we created with the fake news the scope of this study.
dataset created by GPT-2, which is a large language model, DistilBERT is a variant of BERT that demonstrates com-
the success rate of the community systems’ voting classifier mendable performance. Consequently, we incorporated it into
is 89% with the word2vec development process, the most our analysis to evaluate its performance in comparison to
performant algorithm for the TR_FaRe_News dataset. For the other BERT models. The settings utilized in the DistilBER-
GPT-2 dataset, the best performing algorithm was found to be Turk configuration are enumerated in Table 2.
the voting classifier and achieved 77.6% success.

B. BERT MODELS TABLE 2. DistillBERT parameters.

The BERT language representation model is considered a
very powerful model for language based tasks [43]. It was
first introduced in 2018. It is designed to perform pre-training
in a bi-directional way in all layers that can handle unlabelled
sequential data. The BERT model employs an attention mech-
anism to acquire the contextual associations among words
in the input text of the transformer. The system employs an
encoder to analyze the input text and build word embed-
dings, and a decoder to forecast the outcome [43]. In this DistilBERT is a highly effective variant of BERTurk.
way, because the vectors are produced in accordance with Consequently, we utilized it to evaluate its efficacy in compar-
the semantic context of the words, the language is better ison to other BERT models. The DistilBERT configuration
understood and homophones used for different purposes can includes the settings indicated in Table 2.
be distinguished [12]. The results obtained in line with the parameters used were
A pre-trained state-of-the-art BERT model can be adapted calculated as the accuracy, precision, recall, and f1-score. The
or fine-tuned for various tasks. These tasks can include ques- truth table of DistillBERTurk is presented in Table 3.
tion answering, language recognition, etc. The BERT model Additionally, the DistillBERTurk model used was tested on
is considered state-of-the-art because it consistently achieves BuzzFeed, GosspCop, ISOT, LIAR, Twitter15, Twitter16 and
the best accuracies in several natural language processing GPT-2 fake news datasets. The fields used in the datasets were
tasks [43]. The process of adapting BERT involved both updated to suit our model and were used in our experiments.
pre-training and fine-tuning. During the pre-training phase of The parameters used in our dataset were also used in the
the BERT model, the model underwent training for various datasets in question.

14922 VOLUME 12, 2024

G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

TABLE 3. DistillBERTurk truth table.

A comparison of the results obtained with our DistillBER-

Turk model with the TR_FaRe_News dataset created within
the scope of this study is presented in Table 4.

TABLE 4. A comparison table of the results obtained with our

DistillBERTurk model.

FIGURE 2. Optimal model search architecture.

2) BERT/BERTURK(MODEL2-6)
The Turkish BERT model (BERTurk) [47]was pre-trained on
the Oscar Corpus, Opus Corpora and Wikipedia. The model
consists of 12 transducer layers. BERTurk models vary in
word sizes of 32K and 128K, and both are available in cased FIGURE 3. BERTurk+CNN architecture.

and caseless versions.

The classification task in this study utilized the orig-
inal BertForSequenceClassification model. Subsequently, Five models were created using BERTurk. The aim was to
the BERT model underwent fine-tuning, followed by the obtain an optimal model by using these models. The process
replication of the same experiment. Finally, following the followed to reach the optimal model is illustrated in Fig. 2.
fine-tuned model, additional layers were added for both In this study, six different models were created for
BERTurk+CNN and BERTurk+Bi-LSTM, such as freezing BERTurk and DistillBERTurk. The initial model was eluci-
and unfreezing parameters in the fine-tuned model. Training dated in the preceding section. The second model was defined
was then performed for model adaptation, hyperparameteri- as the fine-tuned BERTurk model. The third model is defined
sation, and testing. In addition to the success of BERTurk in as a BERTurk fine-tuned model with CNN layers and frozen
word-embedding tasks, the aim of using CNN is to provide parameters. The fourth model was defined as the BERTurk
a deep learning model that uses neural network layers to fine-tuned model with CNN layers without frozen parame-
define the model and is powerful enough to process large ters. The fifth model is defined as the BERTurk fine-tuned
amounts of data using a network of hidden layers [50]. When model with BiLSTM layers with frozen parameters. Finally,
using Bi-LSTM and BERT together, the aim is to extract the the sixth model was defined as a BERTurk fine-tuned model
improved features of BERTurk and achieve better learning with BiLSTM layers without frozen parameters.
performance. It is also believed that Bi-LSTM will better cap- There are two techniques available when using BERTurk.
ture the global context. The results of CNN and Bi-LSTM on these techniques are MLM and NSP techniques. Optimization
short sentences datasets such as Twitter are very good when during training ensures that the loss when using these two
analyzed in the literature. The fact that each component of techniques is minimized. Since the features extracted in the
the Bi-LSTM input sequence contains information from both first stage are generic, parameter freezing was performed to
past and present has helped us to produce more meaningful access meaningful features more easily. The use of frozen
output [51]. The procedure of our main model, the improved parameters in large language models and in our own Turkish
BERTurk model, is presented in Figure 3. dataset shows the originality of the study.

VOLUME 12, 2024 14923

 G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

BERTurk fine-tuning (Model 2) study, the calculation shown in (4) was used for test accuracy.
The learning rate utilized for Model 2, also known as correctclassifiednewsnumber
BERT fine-tuning, was set to 2e-5. The model underwent four (4)
totalnewsnumber
rounds of fine-tuning.
BERTurk fine-tuning + CNN (Model 3 and Model 4) The second metric used is the ROC AUC score. The ROC
For Models 3 and 4, two CNN layers of kernel size (1,768) AUC represents the extent of the area enclosed by the ROC
and (2,768) were added after the BERTurk fine-tuned Model. curve. The ROC AUC score ranges from 0 to 1, and a num-
The activation procedure is succeeded by a maximum pooling ber close to 1 indicates excellent performance in predicting
layer, where the kernel size is set to the previous output size classifications. The final metric was the F1 score. The F1
and the step size is determined by the prior height of this score, similar to the AUC value, varies between 0 and 1,
output. Ultimately, a linear layer is employed, followed by and is calculated by remembering the precision results. The
the application of a softmax function. Equation (3) defines accuracy value is calculated by dividing the number of real
the softmax activation function. positive outcomes by the total number of positive results.
Recall, also known as sensitivity, is calculated by dividing
ezi the number of real positive results by the total number of
σ (z)i = PK (3)
zj samples that should be classified as positive. A higher score
j=1 e
corresponds to superior achievement. The calculation of the
Because the news in our dataset has two classes (fake f1-score is demonstrated by Equation (5).
and real), K was set to 2.Variable zrepresents the input. precision·recall
The class that yields the greatest value when sent through f 1 − score= 2x (5)
precision + recall
the softmax activation function can be regarded as the out-
come of the classification process.In addition, the learning Five models were developed using BERTurk to assess and
rate of Models 3 and 4 is 2e-5and the number of rounds compare the effectiveness of fake news classification.
is four,as in the fine-tuning of BERTurk. The architecture of After training the created models on our TR_FaRe_News
the BERTurk+CNN model is illustrated in Fig. 3. dataset consisting of Turkish news tweets, we obtained var-
BERTurk fine-tuning + Bi-LSTM (Model 5 and Model 6) ious results when we tested them with a validation dataset
After the BERTurk model for Model 5 and Model 6, 2 created with tweet sentences that were not in our dataset,
Bi-LSTM layers were applied to Model 5 and 1 Bi-LSTM which we call live data. The validation dataset consists of
layer was applied to Model 6. Afterwards, a linear layer was parody accounts on Twitter and current tweets of mainstream
utilized using a softmax activation function. In addition, the Twitter accounts. All model results with GPT-2 Fake News
learning rates for Models 5 and 6 were 5e-5,the number of Dataset are presented in Table 5 and all model results with
rounds used for Model 5 was 10,and the number of rounds TR_FaRe_News Dataset are presented in Table 6.
used for Model 6 was 6.The structure of the BERTurk+Bi-
TABLE 5. Model results with GPT-2 fake news dataset.
LSTM model is depicted in Fig. 4.

TABLE 6. Model results with TR_FaRe_news dataset.

FIGURE 4. BERT+BiLSTM architecture.

Various measurements were used to test the prediction

results of classifiers on the fake news dataset. First, test A series of operations were conducted to make it easier
accuracy was calculated as the primary measurement. In this for us to access the results in Table 6. The ReLU activation

14924 VOLUME 12, 2024

G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

function was utilized due to its low computational cost, while formula (6).
softmax activation created 2 classes and 2 nodes. Max pool- AxB
Pn
i=1 Ai Bi
ing and dropout were employed to reduce large dimensions cos (θ) = =r (6)
∥A∥ x ∥B∥ qP
and improve learning rate. Max pooling was employed to Pn 2 n 2
i=1 Ai i=1 Bi
minimize the time-consuming process of selecting the largest
value in a matrix to reduce its size. Dropout was employed to After sorting similar tweets, tweets containing words with
prevent data memorizing, while AdamW optimization func- high word counts that would negatively affect the classifi-
tion was utilized for optimization, enhancing learning rate cation in the fake and real news datasets were also edited.
and creating a timer. The word frequencies for the fake and real news datasets are
The ROC curve of the classification process with the depicted in Fig. 6 and Fig. 7, correspondingly.
TR_FaRe_News dataset for Turkish is shown in Figure 5.

FIGURE 6. Real news dataset word count.

FIGURE 5. ROC curve.

Furthermore, ROC charts were generated for each indi-

vidual model. Binary classification is a commonly employed
technique, particularly when constructing ROC curves in aca-
demic research. The primary metric of interest in the ROC
curve, which is a properly scaled graphical representation,
is the Area Under the Curve (AUC). AUC, or the area under
FIGURE 7. Fake news dataset word count.
the receiver operating characteristic (ROC) curve, quantifies
the accuracy and effectiveness of the model. The value must The manually created TR_FaRe_News dataset was used
fall within the range of 0 and 1. A model is considered more for the Turkish fake news classification. In addition, Buz-
successful as its AUC value approaches 1. zFeedNews, GossipCop, ISOT, LIAR, Twitter15, and Twit-
ter16, which are used to detect fake news in English, were
IV. EXPERIMENTS used for verification. Datasets created using large-language
A. DATASETS models were also used. A fake news dataset created by GPT-2
For this study, a dataset called TR_FaRe_News (Turkish Fake was also included in this study. The dataset for the experi-
and Real News) consisting of Turkish fake and real news was ments is divided into three parts: train, validation (val), and
created. Tweets taken using the Tweepy module developed test. The dataset statistics are listed in Table 7.
for Twitter were manually labelled so that tweets taken from
mainstream newspapers were real, and false news shared by B. DATA PREPROCESSING
the fact-checking platform was labelled fake news. After the Data preprocessing involves editing the dataset before con-
labelling process, similar tweets were found in the data and verting it into numbers that can be understood by the com-
sorted. Before extraction, cosine similarity was calculated for puter. Since Turkish is an agglutinative language, it presents
the news in the dataset. Tweets that were very similar were difficulties in terms of natural language processing [48].
eliminated. Here, if the news tweet in the real news dataset To overcome these difficulties, Zemberek was used to analyze
is considered A and the news tweet in the fake news dataset texts and access word roots [49]. The StemmingAndLemma-
is considered B, the mathematical process is calculated using tization package of Zemberek, an open source NLP toolkit

VOLUME 12, 2024 14925

 G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

TABLE 7. Data istatistics. TABLE 8. News tweet statistics for the TR_FaRe_news dataset.

Following the preparation procedures, our data corpus had

a total of 46,818 tweet sentences, but after applying the
similarity theorems explained in the previous section and
deleting duplicate records again, 18,695 tweet sentences were
implemented in Java, was utilized for processing the Turkish included in our dataset. The steps we performed for prepro-
language. What to do here is: cessing are as follows.
• The stemming process attempts to find the word root by • Repeated tweets have been deleted,
cutting the suffixes and prefixes in a word, • Special characters, emojis and punctuation marks
• Lemmatization is based on the morphological analysis in tweets are written with words in the tweet sen-
of words. Therefore, the algorithm requires a detailed tences. This prevents NLP. Therefore, these have
dictionary to obtain the word root. been deleted.
To effectively perform text classification and create a clas- • Erroneous words or missing letters entered by
sification model, a dataset must be preprocessed. Studies the users (even by mistake) were normalized
have shown that the results obtained during the classifica- or corrected by appliying them to the data
tion of data from preprocessed datasets are better than those corpus.
obtained from non-preprocessed data [48]. The objective of • Twitter mentions (starting with @) and tweet hash-
the preprocessing procedures is to enhance performance by tags (starting with #) were deleted.
reducing the size of the vector space and dimensions. The • Stop words that had no meaning in the sentences or
TR_FaRe_News dataset was utilized for our investigation, have no effect on the meaning of the sentence were
and the open-source Zemberek Library [49] was chosen for deleted.
natural language processing (NLP). The code developments • Because the lowercase mode of the BERTurk model
in the library were implemented using Java. The total number was used, all words were converted to lower-
of news tweets was calculated according to the operations case letters, which have been used in all other
performed on the news tweets taken from Twitter. The news models.
tweet statistics for the TR_FaRe_News dataset are listed in The outputs obtained from the applied process are listed in
Table 8. Table 9.

14926 VOLUME 12, 2024

G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

TABLE 9. An example of preprocessing an original tweet. TABLE 11. Experimental results.

TABLE 12. BERTurk comparison table with all datasets.

C. PARAMETER SETTINGS
The parameters of our first model, DistillBERT, are presented
in Section III. An further five models were developed via
BERTurk in order to assess and contrast the efficacy of
false news classification. The hyperparameter tables for these
models are listed in Table 10. this thesis, it is seen that it achieves a better accuracy rate
in the ISOT dataset than the TR_FaRe_News dataset with
TABLE 10. Hyperparameters used for the BERTurk model. a slight difference. We can say that the main reason here
is that our dataset consists of short tweets on Twitter and
the language analysis difficulties of the Turkish languages.
However, if we look at the Section II, we see that good results
are obtained with the TR_FaRe_News dataset.

V. RESULTS AND DISCUSSIONS

The word clouds of fake and real news in the TR_FaRe_News
dataset created within the scope of the study were extracted.
Word clouds were created for the first 50 most common words
D. EXPERIMENTAL RESULTS in the dataset. The word clouds are shown in Figure 8.
The common table of the results we obtained when we tested In order to assess the effectiveness of the BERT models
the 6 models we created with BERTurk and DistillBERTurk developed for Turkish fake news detection and classification
with BuzzFeedNews, GossipCop, ISOT, LIAR, Twitter15, using deep learning, a comparison was conducted among
Twitter16, GPT-2 and TR_FaReNews, the dataset we created, Turkish studies focused on fake news detection. The compar-
is presented in Table 11 A comparison of the results obtained ison is displayed in the Table 13.
with our BERTurk model with the TR_FaRe_News dataset When we analyze the table, we observe that among super-
created within the scope of the thesis is presented in Table 12. vised machine learning algorithms, the SVM models perform
As seen in Table 11, if we compare our BERTurk model better for Turkish fake news detection. When [37] and [38]
with the dataset TR_FaRe_News created within the scope of were examined, it was observed that the performance of

VOLUME 12, 2024 14927

 G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

TABLE 13. The comparison of Turkish studies on fake news detection.

FIGURE 8. Real and fake news dataset word clouds.

the model we created was higher when word2vec word

embedding was performed. Additionally, when the ensemble
learning perspective we offer in our machine learning model
was added, this performance increased even more with the
voting classifier, and a performance of 89% was achieved
when trained with the TR_FaRe_News dataset and 77.6%
when trained with the GPT-2 fake news dataset.
This study, which has not yet been shared with the literature
but creates a specific dataset from their studies, includes
tweets based on three topics [39]. Because the authors gave
f1 score results when evaluating performance, it would not
be appropriate to make a comparison in terms of accuracy
rate. However, when we compare the f1 score results, it is
seen that the score results of the models presented in the study
in question are between 57% and 89%, and these results are
below those of the BERT models in our study. The f1 score
of all five BERT models in our study was over 90%.
Furthermore, the SOSYalan dataset, a notable study in
the literature for identifying fake news in Turkish, yielded
successful outcomes, but when evaluated in terms of dataset
size, it is estimated that the classification success is likely to
be high because it is a small dataset [40]. The BERT models
were not included in the study.
In terms of the data in the dataset, the study created by
selecting tweets on the subject of Covid-19 appears to be
very similar to our study. Since our study does not include
a single subject, it has a good rate among the literature
studies, although there is a slight decrease in performance.
In addition, their study did not include a comparison table
with important datasets in the literature [41] and [42] has been
explained in related works.
Experiments were also conducted for the fake news dataset
produced by GPT-2, a large language model we used in our

14928 VOLUME 12, 2024

G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

TABLE 13. (Continued.) The comparison of Turkish studies on fake news research, examining their role in the experiments conducted
detection.
within the study’s scope. In this context;
- First, The study created Turkish datasets using
18695 tweets and two tags, with 13736 for training,
3433 for testing, and 1526 for verification in the Twitter
environment. The Turkish language dataset has been
used for detecting fake news, providing a comprehensive
source for future studies and contributing to the literature
on fake news detection.
- Then, the data was collected and labeled, then prepro-
cessed using the Zemberek library for Natural Language
Processing (NLP) processes, bringing the roots to a
study. As a result of the experiments, an accuracy between
semantic level. This is suitable for models utilizing
62.7% and 95% was obtained. This usage does not appear
machine learning and deep learning.
in any study in the Turkish literature. In addition, the perfor-
mance achieved for our dataset, TR_FaReNews, was between Machine learning and deep learning studies on
71% and 94%. This performance is considered to be success- TR_FaRe_News dataset detected fake news tweets with six
ful when considering the dataset we prepared. models, including DistillBERTurk, achieving accuracy val-
ues of 90-94%. The models tested on different datasets exhibit
VI. CHALLENGES AND LIMITATIONS various differences. The model’s creation for the Turkish
The challenges and limitations we faced in the study are listed language and its current results indicate a difference in usage
below; patterns among BERT models developed for the Turkish
language.
• In the Twitter API’s free version, only tweet sentences This study analyzed Twitter tweet news, focusing on short
from the last seven days were available, making it texts with low word frequency and frequent topics. Despite
difficult to obtain data. these disadvantages, high accuracy rates were achieved due
• We retrieved the data using Twitter usernames, so we to considerations like special expressions, abbreviations,
had all tweets from these usernames in the last seven spelling errors, and NLP losses.
days. Considering that we extracted tweets over a The study created a Turkish dataset called TR_FaRe_News
two-year period (from January 2020 to December using Twitter data from fact-checking platforms and main-
2022), a maximum of 3250 tweets each time the app stream news agencies. The dataset was divided into 10 parts
was run. for reliability and classification performance was evaluated
• Considering the way Twitter is used in terms of the using supervised machine-learning algorithms and a voting
number of characters, it has been observed that there classifier. The BERTurk + CNN model achieved 94% accu-
are also posts consisting of very few words. One of racy.
the factors that reduce the success of text classification The study utilized GPT-2’s fake news dataset, a large
processes is the low number of words. As a result of language model, in experiments, resulting in impressive per-
the data preprocessing steps, tweets that did not contain formance, unlike any previous Turkish study using such news.
any words were also encountered. The Turkish fake news detection field is limited by a lack
• Turkish exhibits a markedly distinct structure com- of references and an accessible dataset in literature. Our study
pared to English. Therefore, the Zemberek Natural yielded higher results than other Turkish fake news detection
Language Processing (NLP) tool was employed to classification studies, with similar results observed in English
examine word affixes, since there is a scarcity of avail- fake news detection studies. Our study utilized the GPT-2
able sources. fake news dataset, which is not commonly used in literature
• The large size of the datasets generated by a large for detecting fake news through the creation of a model. The
language model such as GPT-2, has led to temporal language models developed in this study are deemed to be
problems in utilizing the entire dataset. both original and successful.
We plan to utilize the TR_FaRe_News dataset developed
VII. CONCLUSION within our study’s scope in various models in the future. The
This study utilized news accuracy platforms to detect and study will also involve experiments using GRU models and
classify Twitter posts’ success, creating classification models various other generative models. The models we developed
and testing their effectiveness using real-time tweet data. will be tested with larger language models beyond GPT-
The studies conducted a literature review on detecting fake 2. Our study is anticipated to serve as a valuable resource
news on Twitter and discussing its connection to disinforma- for future research in Turkey. The TR_FaRe_News dataset,
tion, deception, and misinformation. The study detailed the utilizing classification models and findings, will significantly
datasets used in fake news detection applications, tests, and contribute to the literature in Turkish Fake News Detection.

VOLUME 12, 2024 14929

 G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

This dataset serves as a crucial foundation for academic [18] M. H. Goldani, S. Momtazi, and R. Safabakhsh, ‘‘Detecting fake news
research. with capsule neural networks,’’ Appl. Soft Comput., vol. 101, Mar. 2021,
Art. no. 106991.
[19] S. Hakak, M. Alazab, S. Khan, T. R. Gadekallu, P. K. R. Maddikunta, and
DATA AVAILABILITY STATEMENT W. Z. Khan, ‘‘An ensemble machine learning approach through effective
The datasets utilized in the studies, with the exception of the feature extraction to classify fake news,’’ Future Gener. Comput. Syst.,
vol. 117, pp. 47–58, Jun. 2021.
TR_FaRe_News dataset, are readily accessible to the public [20] I. Ahmad, M. Yousaf, S. Yousaf, and M. O. Ahmad, ‘‘Fake news detec-
and can be obtained by following the links provided below: tion using machine learning ensemble methods,’’ Complexity, vol. 2020,
pp. 1–11, Oct. 2020.
• BuzzFeedNews, GossipCop: https://github.com/ KaiD-
[21] H. S. Al-Ash, M. F. Putri, P. Mursanto, and A. Bustamam, ‘‘Ensemble
MML/FakeNewsNet learning approach on Indonesian fake news classification,’’ in Proc. 3rd
• ISOT: https://onlineacademiccommunity.uvic.ca/ Int. Conf. Informat. Comput. Sci. (ICICoS), Oct. 2019, pp. 1–6.
isot/2022/11/27/fake-news-detection-datasets/ [22] A. Mahabub, ‘‘A robust technique of fake news detection using ensemble
voting classifier and comparison with other classifiers,’’ Social Netw. Appl.
• LIAR: https://paperswithcode.com/dataset/liar Sci., vol. 2, no. 4, p. 525, Apr. 2020.
• Twitter15, Twitter16: https://www.kaggle.com/ [23] G. Gravanis, A. Vakali, K. Diamantaras, and P. Karadais, ‘‘Behind the
datasets/lhyimp/twitter1516 cues: A benchmarking study for fake news detection,’’ Expert Syst. Appl.,
vol. 128, pp. 201–213, Aug. 2019.
• GPT-2 Fake News Dataset: https://openaipublic.
[24] J. Y. Khan, M. T. I. Khondaker, S. Afroz, G. Uddin, and A. Iqbal, ‘‘A bench-
blob.core.windows.net/gpt-2/output-dataset/v1/ mark study of machine learning models for online fake news detection,’’
• TR_FaRe_News: To acquire it, simply write an email to Mach. Learn. With Appl., vol. 4, Jun. 2021, Art. no. 100032.
the corresponding author of the article. [25] A. Saha, K. Tawhid, M. B. Miah, A. A. Somudro, and D. Nandi, ‘‘A com-
parative analysis on fake news detection methods,’’ in Proc. 2nd Int. Conf.
Comput. Advancements, Mar. 2022, pp. 392–399.
REFERENCES [26] K. K. Kumar, S. H. Rao, G. Srikar, and M. B. Chandra, ‘‘A novel approach
[1] I. A. Taşkıran, ‘‘Dijital Yerli Gazetelerin Sosyal Medya Stratejileri ve for detection of fake news using long short term memory (LSTM),’’ Int. J.,
Sosyal Medyanın Haber Okunurluğuna Etkisi,’’ Akdeniz Üniversitesi vol. 10, no. 5, 2021.
Iletişim Fakültesi Dergisi, vol. 30, pp. 218–240, Apr. 2018. [27] T. Pu, ‘‘An active learning framework for trustworthy classification over
[2] C. Beckett, SuperMedia: Saving Journalism so it Can Save World. Hobo- COVID-19 multilingual fake news dataset,’’ Doctoral dissertation, Ford-
ken, NJ, USA: Wiley, 2011. ham Univ., Bronx, NY, USA, 2023.
[3] 25 İCİN 2023+ Sosyal Medya İstatİstiklerİ, GerçEkler Ve Trendler. [28] F. Harrag and M. K. Djahli, ‘‘Arabic fake news detection: A fact checking
Accessed: 2023. [Online]. Available: https://www.websiterating. based deep learning approach,’’ ACM Trans. Asian Low-Resource Lang.
com/tr/research/internet-statistics-facts/#chapter-1 Inf. Process., vol. 21, no. 4, pp. 1–34, Jul. 2022.
[4] (2023). Top 15 Most Popular News Websites. Accessed: Jun. 26, 2023. [29] A. J. Keya, M. A. H. Wadud, M. F. Mridha, M. Alatiyyah, and M. A. Hamid,
[Online]. Available: http://www.ebizmba.com/articles/ ‘‘AugFake-BERT: Handling imbalance through augmentation of fake news
[5] We Are Social Dijital 2023 Global ve Türkiye Raporu Yayınlandı. using BERT to enhance the performance of fake news classification,’’ Appl.
Accessed: 2023. [Online]. Available: https://omgiletisim.com/we-are- Sci., vol. 12, no. 17, p. 8398, Aug. 2022.
social-dijital-2023-global-ve-turkiye-raporu-yayinlandi/ [30] B. Palani, S. Elango, and V. Viswanathan K, ‘‘CB-fake: A multimodal
[6] En Popüler Web Siteleri Sıralaması. Accessed: 2023. [Online]. Available: deep learning framework for automatic fake news detection using cap-
https://www.similarweb.com/tr/top-websites/turkey/news-and-media/ sule neural network and BERT,’’ Multimedia Tools Appl., vol. 81, no. 4,
[7] Twitter. İşte Olup Bitenler. Accessed: 2023. [Online]. Available: pp. 5587–5620, Feb. 2022.
https://twitter.com/?lang=tr [31] B. Samir, ‘‘The pope has a new baby!’’ in Fake News Detection Using Deep
[8] H. A. Ünver, Edam, TürkıYe’de doǧruluk kontrolü ve doğrulama kuru- Learning, 2017.
luşlari,’’ Centre Econ. Foreign Policy Stud., Tech. Rep., 2020. [32] A. Ågren, and C. Ågren, ‘‘Combating fake news with stance detection
[9] N. Seddari, A. Derhab, M. Belaoued, W. Halboob, J. Al-Muhtadi, using recurrent neural networks,’’ M.S. thesis, Univ. Gothenburg, Gothen-
and A. Bouras, ‘‘A hybrid linguistic and knowledge-based analysis burg, Sweden, 2018.
approach for fake news detection on social media,’’ IEEE Access, vol. 10, [33] G. Rajendran, B. Chitturi, and P. Poornachandran, ‘‘Stance-in-depth deep
pp. 62097–62109, 2022. neural approach to stance classification,’’ Proc. Comput. Sci., vol. 132,
[10] D. Choudhury and T. Acharjee, ‘‘A novel approach to fake news detection pp. 1646–1653, Jan. 2018.
in social networks using genetic algorithm applying machine learning clas- [34] S. Girgis, E. Amer, and M. Gadallah, ‘‘Deep learning algorithms for
sifiers,’’ Multimedia Tools Appl., vol. 82, no. 6, pp. 9029–9045, Mar. 2023. detecting fake news in online text,’’ in Proc. 13th Int. Conf. Comput. Eng.
[11] A. Hadeer, ‘‘Detecting opinion spam and fake news using n-gram analysis Syst. (ICCES), Dec. 2018, pp. 93–97.
and semantic similarity,’’ Ph.D. dissertation, 2017. [35] R. K. Kaliyar, A. Goswami, and P. Narang, ‘‘FakeBERT: Fake
[12] V. Pérez-Rosas, B. Kleinberg, A. Lefevre, and R. Mihalcea, ‘‘Automatic news detection in social media with a BERT-based deep learning
detection of fake news,’’ 2017, arXiv:1708.07104. approach,’’ Multimedia Tools Appl., vol. 80, no. 8, pp. 11765–11788,
[13] P. H. A. Faustini and T. F. Covões, ‘‘Fake news detection in multi- Mar. 2021.
ple platforms and languages,’’ Expert Syst. Appl., vol. 158, Nov. 2020, [36] M. H. Goldani, R. Safabakhsh, and S. Momtazi, ‘‘Convolutional neural
Art. no. 113503. network with margin loss for fake news detection,’’ Inf. Process. Manage.,
[14] F. A. Ozbay and B. Alatas, ‘‘Fake news detection within online social vol. 58, no. 1, Jan. 2021, Art. no. 102418.
media using supervised artificial intelligence algorithms,’’ Phys. A, Stat. [37] U. Mertoglu, H. Sever, and B. Genc, ‘‘Savunmada Yenilikçi bir Dijital
Mech. Appl., vol. 540, Feb. 2020, Art. no. 123174. Dönüşüm Alanı: Sahte Haber Tespit Modeli,’’ Savtek, vol. 9, pp. 771–778,
[15] K. Shu, D. Mahudeswaran, S. Wang, D. Lee, and H. Liu, ‘‘FakeNewsNet: Jan. 2018.
A data repository with news content, social context, and spatiotemporal [38] U. Mertoğlu and B. Genç, ‘‘Automated fake news detection in the age
information for studying fake news on social media,’’ Big Data, vol. 8, of digital libraries,’’ Inf. Technol. Libraries, vol. 39, no. 4, pp. 1–19,
no. 3, pp. 171–188, Jun. 2020. Dec. 2020.
[16] J. C. S. Reis, A. Correia, F. Murai, A. Veloso, and F. Benevenuto, ‘‘Super- [39] S. G. Taskin, E. U. Kucuksille, and K. Topal, ‘‘Detection of Turkish fake
vised learning for fake news detection,’’ IEEE Intell. Syst., vol. 34, no. 2, news in Twitter with machine learning algorithms,’’ Arabian J. Sci. Eng.,
pp. 76–81, Mar. 2019. vol. 47, no. 2, pp. 2359–2379, Feb. 2022.
[17] T. Jiang, J. P. Li, A. U. Haq, A. Saboor, and A. Ali, ‘‘A novel stacking [40] G. Güler and S. Gündüz, ‘‘Deep learning based fake news detection
approach for accurate detection of fake news,’’ IEEE Access, vol. 9, on social media,’’ Int. J. Inf. Secur. Sci., vol. 12, no. 2, pp. 1–21,
pp. 22626–22639, 2021. Jun. 2023.

14930 VOLUME 12, 2024

G. K. Koru, Ç. Uluyol: Detection of Turkish Fake News From Tweets With BERT Models

[41] M. Bozuyla and A. Özçift, ‘‘Developing a fake news identification model GÜLSÜM KAYABAŞI KORU received the B.Sc.
with advanced deep languagetransformers for Turkish COVID-19 mis- and M.Sc. degrees in computer engineering. She
information data,’’ Turkish J. Electr. Eng. Comput. Sci., vol. 30, no. 3, is currently pursuing the Ph.D. degree in computer
pp. 908–926, Mar. 2022. forensics with Gazi University, Ankara, Turkey.
[42] S. G. Taskin, E. U. Kucuksille, and K. Topal, ‘‘Twitter üzerinde Türkçe She is also a Computer Engineer with the Ministry
sahte haber tespiti,’’ Balıkesir Üniversitesi Fen Bilimleri Enstitüsü Dergisi, of National Defense. Her current research interests
vol. 23, no. 1, pp. 151–172, 2021. include software project management, information
[43] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, ‘‘BERT: Pre-training systems security, social network analysis, and pre-
of deep bidirectional transformers for language understanding,’’ 2018,
dictive machine learning.
arXiv:1810.04805.
[44] A. S. Alammary, ‘‘BERT models for Arabic text classification: A system-
atic review,’’ Appl. Sci., vol. 12, no. 11, p. 5720, Jun. 2022.
[45] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, ‘‘DistilBERT, a dis-
tilled version of BERT: Smaller, faster, cheaper and lighter,’’ 2019,
arXiv:1910.01108.
[46] A, Wani, I, Joshi, S. Khandve, V. Wagh, and Joshi, ‘‘Evaluating deep
learning approaches for COVID19 fake news detection,’’ in Proc. Int.
Workshop Combating Line Hostile Posts in Regional Languages During
Emergency Situation, Cham, Switzerland: Springer, 2021, pp. 153–163.
[47] (2023). BERTurk. Accessed: Jun. 16, 2023. [Online]. Available:
https://huggingface.co/dbmdz/bert-base-turkish-128k-uncased
[48] K. Oflazer, and M. Saraçlar, ‘‘Turkish and its challenges for language
ÇELEBİ ULUYOL received the B.Sc. and M.Sc.
and speech processing,’’ in Turkish Natural Language Processing, 2018,
degrees in electronics and computer education
pp. 1–19.
[49] A. A. Akın, and M. D. Akın, ‘‘Zemberek, an open source NLP framework
and the Ph.D. degree in educational technologies
for Turkic languages,’’ Structure, vol. 10, pp. 1–5, Jun. 2007. from Gazi University, Ankara, Turkey. He is cur-
[50] P. Sharma and R. Sahu, ‘‘Fake news detection using deep learning based rently a Professor with the Department of Forensic
approach,’’ in Proc. Int. Conf. Circuit Power Comput. Technol. (ICCPCT), Informatics, Gazi University. His current research
Aug. 2023, pp. 651–656. interests include education, computer education
[51] A. R. Merryton and M. G. Augasta, ‘‘A novel framework for fake news and instructional technology, computer sciences,
detection using double layer BI-LSTM,’’ in Proc. 5th Int. Conf. Smart Syst. and software.
Inventive Technol. (ICSSIT), Jan. 2023, pp. 1689–1696.

VOLUME 12, 2024 14931