Classi cation of Diverse AI Generated Content: An

In-Depth Exploration using Machine Learning and

Knowledge Graphs
Vismay Vora
DJSCE
Jenil Savla (  [email protected] )
DJSCE
Deevya Mehta
DJSCE
Aruna Gawade
DJSCE
Ramchandra Mangrulkar
DJSCE

Research Article

Keywords: Machine Learning, AI-Generated Data, Deepfake, LLMs, Generative AI, Knowledge Graphs,
BERT, CNN, Multi-Modal Detection

Posted Date: October 31st, 2023

DOI: https://doi.org/10.21203/rs.3.rs-3500331/v1

License:   This work is licensed under a Creative Commons Attribution 4.0 International License.
Read Full License

Additional Declarations: No competing interests reported.

 Classification of Diverse AI Generated Content:
An In-Depth Exploration using Machine Learning
and Knowledge Graphs
Vismay Vora1*† , Jenil Savla1† , Deevya Mehta1† , Dr
Aruna Gawade1 , Dr Ramchandra Mangrulkar1
1 Dept. of Computer Eng., DJSCE, Mumbai, 400056, Maharashtra, India.

*Corresponding author(s). E-mail(s): [email protected];

Contributing authors: [email protected];
[email protected]; [email protected];
[email protected];
† These authors contributed equally to this work.

Abstract
As the use of AI-generated content grows, it becomes imperative to recognize
which data is human-developed and which is not. AI-generated content comes
in several modalities such as deepfakes, text and images. This article proposes
techniques that can be used to tackle this problem. To identify deepfakes we
can take advantage of characteristics that make us human: blood circulation,
human emotions and winks among other things. Text recognition takes a different
approach, using a combination of large-scale language models, style analysis, and
visual analysis. The generated images can be recognized by analyzing the pixels
and common colors and patterns. It can be concluded that AI-generated content
is the future, and it is difficult to develop a model that can identify it with 100%
accuracy. Validating content before submitting it and using it responsibly are the
steps we should take to adapt to a world where both types of content coexist.

Keywords: Machine Learning, AI-Generated Data, Deepfake, LLMs, Generative AI,

Knowledge Graphs, BERT, CNN, Multi-Modal Detection

1
Statements and Declarations
Competing Interests
The authors have no competing interests to declare that are relevant to the content
of this article.

Author’s Contributions
All authors contributed to the study conception and design. Material preparation,
data collection and analysis were performed by Vismay Vora, Jenil Savla and Deevya
Mehta. The first draft of the manuscript was written by them and all authors com-
mented on previous versions of the manuscript. All authors read and approved the
final manuscript.

Data Availability Statement

The data sets analysed during the current study are available in the ML
Olympiad Kaggle Contest repository; https://www.kaggle.com/competitions/ml-
olympiad-detect-chatgpt-answers/data, and the CIFAR-10 and CIFAR-100 dataset
source; https://www.cs.toronto.edu/kriz/cifar.html with its associated tech report;
https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf

Funding
No funding was received for conducting this study.

1 Introduction
The contemporary landscape is marked by a remarkable transition into the realm of
technology. The forefront of this transformation is adorned by Smart Machines and
Artificial Intelligence (AI), with ChatGPT and BingAI garnering considerable atten-
tion. These Language Model (LLM) based chatbots have witnessed a surge in daily
content generation. Rapid integration of these models into various applications under-
scores their utility. The ability to elicit curated responses with a single prompt has
bestowed unprecedented empowerment upon individuals, facilitating effortless content
generation. Gone are the days when content creation entailed laborious processes of
scouring diverse resources, gathering information, and amalgamating data to construct
coherent narratives. Traditional hurdles in content acquisition have receded, render-
ing generated content readily accessible to all.
However, this newfound capability also brings forth its own set of challenges. The
imperative for responsible content generation and utilization becomes evident. As users
harness AI to produce information, concerns over the authenticity and credibility of
such content emerge. Instances of content generated by AI infiltrating platforms like
StackOverflow have prompted temporary bans, as the influx of responses may initially
appear accurate but subsequently prove erroneous upon scrutiny. These AI models
can be tailored by users, thereby facilitating the dissemination of misleading or false

2
information—essentially enabling the propagation of self-inserted misinformation.
This privilege is sometimes exploited by students, who employ AI-generated content
to craft essays and undertake cognitive tasks, thereby negating the essence of the
assignments. Consequently, there exists a pressing need to establish mechanisms for
discerning AI-generated content. This paper delves into the methods of identifying
AI-generated text, images, and deepfake content, advocating for curated and filtered
content generation and distribution.

2 Related Work
The initial modality encompasses videos, wherein Generative AI techniques are har-
nessed to produce synthetic content, often derived from textual inputs. Within this
context, a distinct subset emerges—deepfake videos. A deepfake constitutes a syn-
thetic methodology facilitated by deep learning algorithms. It entails substituting the
visage of an individual in an existing image or video with the features or resemblance of
another individual. The applications of deepfakes span a spectrum of intentions, rang-
ing from legitimate to malevolent. These encompass purposes such as entertainment,
educational purposes, propaganda, spreading misinformation, harassment, blackmail,
and more. Broadly, deepfakes can be categorized into two principal classes: Respon-
sible Deepfakes, serving benign intentions, and Malicious Deepfakes, often entailing
harmful objectives.
• Responsible Deepfakes:
Responsible deepfakes are used for masking identities in video conversations, the
metaverse, and event participation in general. There is an increasing demand for
solutions that allow people to engage in video calls, virtual meetings, and virtual
settings while protecting their privacy and anonymity. In this context, research
into clever masking techniques that allow facial expressions to be exhibited without
exposing identity are critical. The field of responsible deepfakes entails creating
powerful algorithms that can change facial features in real-time while protecting
privacy and preserving the authenticity of facial expressions. While this field is
concerned with safeguarding privacy and anonymity, the major goal is to enable
individuals to communicate in virtual places without jeopardizing their identity.
• Malicious Deepfakes:
Malicious deepfakes are used for spreading misinformation and disinformation, as
well as propaganda and fake news. Deepfake technology is increasingly being utilised
for malevolent reasons, such as creating and disseminating deceptive content, fake
news, and propaganda. These deepfakes can have serious societal ramifications, such
as misinformation, loss of confidence, and even harmful outcomes. Detecting and
countering such deepfake films is critical for preserving information integrity and
safeguarding individuals from manipulation.
While responsible deepfakes prioritise privacy and do not involve deepfake detec-
tion, irresponsible or malicious deepfakes necessitate the development of powerful
deepfake detection techniques to counteract disinformation dissemination and protect
individuals from being tricked. In this research paper, the emphasis is on developing

3
effective deepfake detection systems that can detect and flag hostile deepfakes, hence
reducing the dissemination of misleading information and protecting the validity of
media content.
Research paper [1] synthesized 112 pertinent publications from 2018 to 2020 that
presented a variety of methodologies. They are grouped into four buckets: deep learning
approaches, classic machine learning methods, statistical techniques, and blockchain-
based strategies. The work [2] investigates novel approaches that implement GANs
as well as VAEs that are focused on faces. A convolutional EfficientNet B0 used to
extract features is integrated with ViTs, achieving performance equivalent to some
very recent ViT approaches. A basic inference process based on a simple voting scheme
for dealing with many faces in a single video shot has been described. A 0.951 AUC
and 88.0% F1 score was attained by the best model, very close to the state-of-the-art
on the DFDC Challenge[3].
Lastly, [4] suggests utilizing an ensemble of multiple CNNs to detect face swaps
and manipulation in video footage. Using attention layers and siamese training,
the suggested approach mixes several models obtained from a base network (Effi-
cientNetB4). The method is evaluated on openly accessible datasets containing over
119,000 clips, and it showed promising results in recognizing deepfakes and other facial
modification techniques.
One prevalent manifestation of generated content is text, and the advancement of
Large Language Models is perpetuating this trend. Research within this domain is
actively underway. A literature review on detecting AI-generated text unveils a spec-
trum of methodologies. In Article [9], published in the MIT Technology Review, the
discourse revolves around diverse approaches. This includes the employment of Large
Language Models, repurposed through retraining on human-written text to distin-
guish it. The introduction of watermarks during generation is suggested as an effective
demarcation between AI-generated and human-generated text. A pivotal deduction
from the article is that human judgment remains a potent means of detecting generated
text, as the presentation might not align with a general reader’s preferences.
Delving into the realm of research, Article [10] investigates the direct utilization
of generative AI models for detecting AI-generated content. However, this approach’s
accuracy hinges on shared training data between both systems.
In a nuanced evaluation, Article [11] conducts a differential analysis of AI-generated
content, focusing on scientific context. A distinctive framework is formulated, incor-
porating features encompassing syntax, semantics, pragmatics, and writing style to
unveil subtler errors. This contributes to refining AI models for improved content
quality, addressing ethical and security concerns.
The traceability of AI-generated text is a focal point in Article [13]. The article
delves into techniques employed to obscure AI-generated content, including paraphras-
ing tools. The exploration also entails a comparison between the detection model and a
random classifier. The study underscores the vulnerability of watermarking and intro-
duces the concept of spoofing attacks, whereby text is manipulated to contain hidden
signatures, evading detection.
Article [14] elucidates the utilization of watermarks as a segregation strategy
for distinct text types. Employing invisible text spans, detectable only via specific

4
algorithms, the study assesses its efficacy using models from the Open Pretrained
Transformer family. Watermarks are endorsed as robust and secure markers for
differentiation.
Images constitute the third modality. [15] conducts a systematic investigation on
the detection of CNN-generated images by utilizing the systematic deficit present in
these images in duplicating high-frequency Fourier spectrum decay features. However,
the study shows that the disparities in those decay attributes are not intrinsically
present in CNN-based generative models.
Another subsequent investigation creates an artificial data set that resembles the
CIFAR-10 dataset [17], containing sophisticated optical characteristics such as life-
like reflections, and gives the CIFAKE dataset [16]. This dataset creates a binary
classification task in order to distinguish between actual and AI-generated photos.
It proposes a simple neural network design for performing binary classification and
distinguishing between fake and real photos.
Notwithstanding these attempts, there is still a lot of room for advancement in
the field of AI-generated content detection research. The current study proposes a
method to detect deepfake content and is primarily concerned with the detection and
classification of AI-generated material in the text and image modalities with the use of
knowledge graphs. This research provides a feature-based model training methodology
for text detection as well as a neural network-based image detection method. The
following section goes over these approaches in depth.

3 Methodology
While the proposed approach for deepfake detection has been elucidated in subsec-
tion A, the technique for detecting AI-generated content for both text and images is
explained in two separate sections. Subsection B presents a method to identify text
produced by Large Language Models (LLMs) using a language model that relies on
knowledge-based features. Subsection C describes a technique to recognize images cre-
ated by artificial intelligence using a convolutional neural network with multiple layers.
Subsection D briefly proposes how knowledge graphs can be used to combine detection
for the text and image modalities.

3.1 Deepfake Detection

A novel technique for identifying deepfake videos utilising biological signals such
as photoplethysmography (PPG) which monitors blood volume changes in the face
produced by the heart beating is proposed in this section.
The proposed approach comprises two main components: a deepfake detector and a
deepfake generator. The deepfake detector employs a CNN architecture that processes
video frames and produces a probability score signifying the authenticity of the frame.
On the other hand, the deepfake generator employs a generative adversarial network
(GAN), taking both a source and target image as inputs to generate a synthesized
image that imparts the appearance of the source image onto the target image.
The deepfake detector capitalizes on the observation that deepfake videos often
lack genuine biological cues such as PPG, eye gaze, or facial expressions. To obtain

5
PPG signals from the facial region of the input frame, the detector utilizes a PPG
extraction module. Subsequently, these PPG signals are fed into a PPG classification
module, which discerns their authenticity. Moreover, the detector incorporates an eye
gaze estimation module to predict the eye gaze direction of the face captured in the
input frame. The direction of the eye gazing is then sent into an eye gaze classification
module, which predicts whether the direction of the eye gaze is congruent with the
face orientation or not. The detector combines the outputs of the PPG classification
module with the eye gaze classification module to determine if the input frame is real
or fake.
The deepfake generator is designed to generate realistic and diversified synthetic
images while preserving the biological signals of the source image. To generate a syn-
thesised image, the generator employs a mix synthesis module that learns to combine
and blend numerous source images. The mix synthesis module is divided into two sub-
modules: composition and style. To establish a coherent framework for the synthesised
image, the composition module learns to choose and align diverse parts from several
source images.The style module learns to transfer and harmonise the source images’
colour, texture, and lighting to match the target image. The generator additionally
employs a residual interpretation module, which learns to comprehend and adjust
residuals between the synthesised image and the target image by utilising biological
signals such as PPG, facial expressions, and head attitude. To make the synthesised
image more realistic or diversified, the residual interpretation module can boost or
decrease particular biological signals.
The proposed technique also has potential applications for privacy enhancement
and social media anonymization. For example, it can be used to create privacy-
enhancing deepfakes that can replace or mask the identity of a person in a video while
preserving their biological signals and expressions. This can help users to protect their
personal information and avoid unwanted exposure or harassment online.

3.2 AI Generated Text Detection

Text detection that is generated by AI models requires performing three main steps:
data preprocessing, feature extraction based on knowledge, and model selection and
training. These steps involve several tasks that are explained in more detail later.

3.2.1 Data Preprocessing

In this research study, the dataset employed originates from the ML Olympiad Detect
ChatGPT Answers Kaggle competition [12]. The dataset encompasses both training
and test sets, each composed of prompts in diverse categories. These prompts con-
stitute questions spanning a wide range of topics, including opinion-based inquiries,
general knowledge queries, scientific facts, and intricately designed ones to challenge
complexity. Notably, each prompt incorporates an undisclosed ratio of human-
generated responses to ChatGPT-generated responses, with an example ratio of 1
ChatGPT response to 5 human-generated responses.
To transform this dataset into a structured format, a process of data cleaning is
applied. Specifically, unlabelled data points and rows with missing values are excluded.

6
 Fig. 1: Flowchart for Proposed Deepfake Detection Approach

The dataset is then partitioned into an 80:20 ratio for training and validation purposes,
respectively. The training set is employed to train the model, while the validation set
is used to assess the model’s performance and derive key evaluation metrics.

3.2.2 Feature Extraction

The feature extraction phase utilizes a carefully curated labeled dataset, which
exclusively comprises complete and representative data instances. In the context of
feature extraction, the selection of features is tailored to the distinctive attributes
of the preprocessed dataset. Leveraging existing domain-specific knowledge graphs
holds promise as an approach to derive informative features. By aligning the entities,
concepts, and relationships from the knowledge graph with the textual content, a
selection of features can be discerned for integration, including -
• Vocabulary-based Features:
Two pertinent vocabulary-based features are of significance - vocabulary size and
lexical richness. Vocabulary size serves as a metric to gauge the diversity and
expansiveness of word usage. The AI model is anticipated to exhibit a distinctive

7
Algorithm 1 Feature Extraction
1: Input: Text
2: Output: Extracted Features
3: {Vocabulary-based features}
4: Tokenize and preprocess text
5: Calculate vocabulary size and lexical richness
6: {Syntactic features}
7: POS tag tokens and count nouns and verbs
8: {Semantic features}
9: Sentiment analysis to compute average sentiment score
10: {Stylistic features}
11: Calculate average sentence length and punctuation count
12: return features

vocabulary pattern encompassing rare lexicon and technical terminology. Lexi-

cal richness, indicated by the Token-Type Ratio (TTR), quantifies the proportion
of unique words in relation to the total word count. In light of limited training
data, AI-generated text typically demonstrates reduced lexical richness compared
to human-generated text, thus rendering vocabulary-based features conducive to
differentiation.
• Syntactic Features:
Syntactic attributes inherent to each text can differ among writers. Examination of
the distribution of Parts of Speech (POS) tags, including nouns, verbs, adjectives,
and adverbs, yields valuable insights. AI-generated text is liable to manifest distinct
syntactic patterns, possibly displaying more straightforward sentence structures.
Hence, metrics such as average sentence length, clause count, and syntactic depth
tree can measure syntactic complexity, enabling the discrimination between AI and
human-generated text.
• Semantic Features:
Semantic coherence, pivotal for smooth transitions between sentences and para-
graphs, may elude AI-generated text, potentially giving rise to abrupt topic
shifts. Techniques encompassing word embeddings and knowledge graph integra-
tion enable sentence similarity comparison. Here, AI-generated text could exhibit
diminished semantic coherence due to its potential lack of comprehensive contextual
understanding.
• Stylistic Features:
Stylistic attributes encompassing sentence structure, tone, formality, and contextual
adherence merit analysis. Stylistic variations can be assessed through parame-
ters like sentence length variability and punctuation utilization. AI-generated text
may deviate in stylistic patterns compared to human-generated text, potentially
unveiling inconsistencies in writing style.
For feature extraction, the identified vocabulary-based, syntactic, semantic, and
stylistic features are encompassed. Vocabulary feature extraction commences with
tokenization of the text, followed by vocabulary size determination by considering

8
exclusively alphabetic words that are not stopwords. This aids in computing lexical
richness by calculating the ratio of unique words to the total word count. Syntactic
features involve quantifying the count of specific Parts of Speech (POS) tags such as
nouns and verbs. The SemanticIntensityAnalyzer module from the nltk library serves
to extract semantic scores for each sentence in the text. Furthermore, integration
with the knowledge graph yields knowledge-driven features that encapsulate structural
attributes, semantic associations, and domain-specific insights. Additionally, stylish
features encompass average sentence length and punctuation count, contributing to
the assessment of text style.

Algorithm 2 AI Detection using BERT

Input: List of texts, truncation, padding
Output: Encoded inputs, labels
1: train encodings ← Tokenizer(train texts, truncation)
2: test encodings ← Tokenizer(test texts, truncation)
3: train dataset ← BuildData(train encodings, train labels)
4: test dataset ← BuildData(test encodings, test labels)
5: model ← BuildModel()
6: CompileModel(model, optimizer, loss, metrics)
7: TrainModel(model, train dataset, epochs, batch size)
8: test accuracy ← EvaluateModel(model, test dataset)
9: return test accuracy

3.2.3 Model Selection and Training

The subsequent step involves the critical process of model selection and training, piv-
otal for the accurate detection of AI-generated text. This phase mandates the judicious
choice of a suitable machine learning or deep learning model tailored to address the
distinctive challenges posed by AI-generated content detection. An array of prospec-
tive models presents itself, ranging from classical methods such as Logistic Regression
and Support Vector Machines (SVM) to ensemble techniques like Random Forests and
Gradient boosting models (e.g., XGBoost, LightGBM). Deep learning models such as
Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and their
variants (LSTM, GRU) as well as transformer-based models (e.g., BERT, GPT) con-
stitute compelling choices. Selection hinges upon factors including dataset size, feature
representation, interpretability, and available computational resources.
For model training, the dataset undergoes preprocessing to conform to a suit-
able representation. Techniques such as bag-of-words, TF-IDF, word embeddings (e.g.,
Word2Vec, GloVe), or contextual embeddings (e.g., BERT, GPT) are enlisted to
achieve this. The training process is accompanied by a rigorous evaluation employ-
ing performance metrics encompassing accuracy, loss, and other pertinent measures.
Hyperparameters are optimized based on the model’s performance on the validation
set.

9
 In this research, the potency of a pre-trained Large Language Model (LLM)
– BERT (Bidirectional Encoder Representations for Transformers) – is harnessed
for detecting AI-generated text. Renowned for its efficacy, BERT is pre-trained on
extensive corpora, endowing it with nuanced understandings of word and sentence
representations, thus encompassing intricate syntactic and semantic relationships.
Contextualized word embeddings, intrinsic to BERT, further enrich its capacity
to comprehend content intricacies. The amalgamation of knowledge-driven features
derived from the knowledge graph and the learned representations of LLMs heightens
accuracy by capitalizing on their synergistic strengths. Accordingly, BERT emerged
as the model of choice for this experimental classification task.
Training the BERT model commences with dataset retrieval from a CSV file,
followed by partitioning into training and testing subsets. Tokenization via the
BERT tokenizer ensues, initializing the BERT model for subsequent steps. Tokenizer
encodings materialize into TensorFlow Dataset objects, fueling training and testing
endeavors. Fine-tuning transpires on the training dataset, with accuracy serving as
the evaluation metric on the test dataset.

3.2.4 Experimental Setup

The experimental setup is a cornerstone that shapes the course of model training and
evaluation. It encompasses pivotal choices, ranging from optimization strategies and
loss functions to evaluation metrics and the training process itself.
• Optimization Strategy:
The Adam optimizer was strategically chosen to propel the optimization strat-
egy. Leveraging the combined strengths of AdaGrad and RMSProp, Adam emerges
as an apt choice for training intricate deep neural networks. Its adaptive learn-
ing rate mechanism dynamically tailors weight updates based on gradients, thereby
engendering effective weight adjustments.
• Loss Function and Evaluation Metrics:
The selection of the Sparse Categorical cross-entropy loss function attests to a
thoughtful decision. Recognized as a staple for multi-class classification scenarios,
this loss function’s suitability is underscored. Notably, the logits argument was des-
ignated as True to eschew the use of normalized probabilities. By juxtaposing logits
with integer target labels, the loss function orchestrates loss computation through
the categorical cross-entropy paradigm. Evaluation metrics encompass accuracy,
precision, and recall. While accuracy proffers a comprehensive gauge of overall
predictive performance, precision quantifies the ratio of true positive predictions rel-
ative to all positive predictions. Concurrently, recall gauges the proportion of true
positive predictions among the actual positive instances.
• Training Process:
The model embarks on a training trajectory spanning a fixed number of epochs,
with each epoch epitomizing a comprehensive traversal through the training dataset.
For this study, the model underwent training across three epochs, a decision aligned
with the dataset’s nature and the task’s intricacies. The choice of a batch size set at
16 merits attention, signifying that a batch comprising 16 input strings undergoes

10
 processing before modulating the model’s weights. This judicious approach expedites
training iterations and facilitates expedient model evaluation.

3.3 AI Generated Image Detection

In the preceding section, we delved into the detailed methodology for detecting AI-
generated text using the BERT model. Now, we transition to the current subsection,
where we elucidate the approach for detecting AI-generated images. This is achieved
through the implementation of a multi-layer Convolutional Neural Network (CNN)
architecture.

3.3.1 Data Preprocessing and Knowledge Graph Construction

The CIFAKE dataset [16] comprises 60,000 artificially generated images and an addi-
tional 60,000 authentic photographs sourced from Krizhevsky and Hinton’s CIFAR-10
dataset [17]. This dataset is categorized into two distinct classes: REAL and FAKE.
The REAL images were extracted from the CIFAR-10 dataset. To simulate the CIFAR-
10 structure for the FAKE images, the implementation involves using Stable Diffusion
version 1.4. In total, the dataset encompasses 100,000 images for the training phase,
distributed as 50,000 images per class, along with an additional 20,000 images for
testing, equally split into 10,000 images per class.
In this subsection, we describe the preprocessing steps and propose a use case for
knowledge graphs. The use of knowledge graphs allows us to leverage structured knowl-
edge representation and graph-based algorithms for better multi-modal AI-generated
content detection and explainability of decisions.
• Image Resizing: The initial stage of data preprocessing involves Image Resizing.
This step is executed while loading the training data and employs the image size
option set at (32, 32), effectively resizing the images to dimensions of 32x32 pixels.
• Normalization: After the images are resized, a normalization process is implemented.
This procedure involves dividing the pixel values by 255.0 and converting them
into floating-point values within the range of [0, 1]. This meticulous normalization
strategy ensures uniform scaling of the input data, ultimately facilitating quicker
convergence during the model training process.
In addition to the above data preprocessing steps, we can incorporate the construc-
tion of a knowledge graph for the CIFAKE dataset described above. A knowledge graph
is a structured representation of knowledge that captures entities, their attributes, and
the relationships between them. We can construct a knowledge graph by extracting
visual features from the images using techniques like convolutional neural networks
(CNN) as described in the model architecture section below or large models like pre-
trained CNNs (e.g., VGG16 or ResNet) to extract high-level visual features from
the images and encoding them as nodes in the graph. The attributes of the nodes
can include image features, metadata, and class labels. We also establish relation-
ships between the nodes based on semantic similarities, such as visual similarity
between images or shared attributes. This knowledge graph serves as a comprehensive
representation of the image dataset, enabling better analysis and reasoning.

11
Algorithm 3 Knowledge Graph Construction for AI-Generated Images
Input: CIFAKE Dataset (60,000 AI-generated images, 60,000 real photos)
Output: Knowledge Graph representing relationships and attributes of images
1: Construct Nodes:
2: Extract high-level visual features using CNN or pre-trained models (e.g., VGG16).
3: Encode images as nodes with attributes like image features, metadata, and class
labels.
4: Establish Relationships:
5: Create relationships between nodes based on semantic similarities.
6: Determine relationships through visual similarity or shared attributes.

3.3.2 Model Architecture

The model architecture to detect AI-generated images is a convolutional neural net-
work. The constructed knowledge graph can act as an additional input or as an
auxiliary feature in the model. This integration allows the model to leverage the struc-
tured knowledge representation and relationships captured in the graph for improved
detection performance. The architecture of the model encompasses several essential
layers, each serving a distinct purpose:
Input Layer: The input layer is configured based on the dimensions of the
images present in the CIFAKE dataset. A sequential layer is constructed using the
tf.keras.Sequential command.
Rescaling Layer: To ensure consistent data scaling, a rescaling layer is incorpo-
rated. This layer operates by normalizing the pixel values of the input images, dividing
them by 255.0.
Convolutional Layers: The model starts with a Conv2D layer employing a 3x3
kernel size, 32 filters and the ReLU activation function. This initial convolutional
layer extracts intricate features from the input images via convolutional techniques.
Subsequently, a MaxPooling2D layer is applied, following the first convolutional layer,
in order to downsample the feature maps, thereby retaining the most significant
attributes. Another Conv2D layer featuring 64 filters and a 3x3 kernel size is intro-
duced, employing the ReLU activation function. Convolution involves the application
of a small window (kernel/filter) across the input image and calculating dot products
between the the corresponding pixels in the windowand the kernel. The utilization of
the MaxPooling2D layer contributes to augmenting the model’s ability for translation
invariance and its overall generalization prowess.
Flattening Layer: Post the convolutional layers, a flattening layer is implemented.
This layer serves to transform the 2D feature maps into a singular 1D feature vector.
This reconfiguration of data prepares it for processing within the artificial neural
network’s (ANN) fully connected layers.
Fully Connected Layers (ANN): A Dense layer containing 128 units accompa-
nied by the ReLU activation function processes the flattened feature vector. This layer
captures intricate, high-level representations of the feature attributes. To counteract
the potential for overfitting, a Dropout layer is introduced, featuring a dropout rate of
0.5. This dropout layer randomly deactivates 50% of the neurons during the training

12
process. The architecture culminates with an output layer that includes a Dense layer
with 1 unit and a sigmoid activation function. This specific configuration generates
prediction probabilities tailored to the binary classification task, offering indications
as to whether an image is AI-generated or not.
Model Compilation: The final model is compiled employing the Adam optimizer,
a binary cross-entropy loss function, and evaluation metrics encompassing accuracy,
precision, and recall.

3.3.3 Experimental Setup

The experimental setup for the training process has been described below. During
training, the knowledge graph can be used for additional tasks such as graph regular-
ization or graph-based loss functions to enforce structural constraints and encourage
the model to learn from the graph’s information.
• Optimization Strategy:
To optimize the model’s performance, the Adam optimizer was selected as the
preferred strategy. Adam seamlessly integrates the merits of both AdaGrad and
RMSProp, rendering it particularly well-suited for training complex deep neural
networks. By dynamically adapting the learning rate according to gradient varia-
tions, this optimizer effectuates judicious weight updates, a cornerstone of effective
training.
• Loss Function and Evaluation Metrics:
For steering the training process, the binary cross-entropy loss function was chosen,
a ubiquitous choice for binary classification endeavors. This loss function endeavors
to minimize the discrepancy between predicted probabilities and actual outcomes.
Assessment of model performance is executed through a triad of evaluation metrics:
accuracy, precision, and recall. Accuracy provides a comprehensive measure of the
model’s overall predictive prowess, precision gauges the proportion of correct posi-
tive predictions among all positive predictions, and recall quantifies the fraction of
accurate positive predictions in relation to the entire set of actual positive instances.
• Training Process:
The model’s training process entails a predetermined number of epochs, wherein
each epoch corresponds to a full iteration through the training dataset. In this
specific investigation, the model’s training is conducted across 10 epochs. To expe-
dite both the training and evaluation phases, a batch size of 32 was designated.
Consequently, every iteration processes a group of 32 image samples prior to updat-
ing the model’s weights. This strategic deployment of batch processing facilitates
accelerated model training and real-time assessment.

3.4 Multimodal Knowledge Graphs

This section proposes the combined usage of the proposed text and image knowledge
graphs to employ a truly multimodal knowledge graph for effective detection of AI-
generated content across both modalities of the text and image formats individually
as well as a mix of them.

13
Fig. 2: Architecture of Model for AI-Generated Image Detection
14
By integrating the knowledge graphs constructed for AI-generated images and
AI-generated text, we create a multi-modal knowledge graph that captures the interde-
pendencies and correlations between different modalities. This integration allows us to
leverage the strengths of both text and image analysis techniques, some of which have
been described above, leading to improved multi-modal AI-generated content detec-
tion. The combined knowledge graph facilitates cross-modal reasoning and provides a
holistic understanding of the data.

3.4.1 Graph-Based Algorithms for Similarity Measurement

To measure the similarity or differences between AI-generated content and real con-
tent using the constructed knowledge graphs, we can define similarity metrics and
utilize graph-based algorithms.
These algorithms leverage the rich representation and structural properties of the
knowledge graphs to compute similarity scores or distances between nodes or sub-
graphs. For example, graph similarity algorithms such as graph edit distance or graph
kernel methods can be employed to compare the structures of AI-generated and real
content subgraphs in the knowledge graph.
Additionally, similarity metrics can be defined based on node attributes, such as
visual features or textual embeddings, to quantify the similarity between AI-generated
and real content. These similarity measurements provide valuable insights into the
characteristics and patterns of AI-generated content, enabling better detection and
analysis. Therefore, these metrics and algorithms quantify the relationships and pat-
terns captured within the knowledge graphs, providing insights into the discriminative
characteristics of AI-generated content.

4 Results and Discussion

The results observed after the implementation of the two above-mentioned techniques
are discussed in this section. The discussion is structured into two subsections: Sub-
section A outlines the results pertaining to the text detection model, while Subsection
B provides visualizations for the image detection model.

4.1 Text Detection

The process of detecting text generated by AI was executed using a BERT Classifier
model. This model underwent training for three epochs, employing a batch size of 16.
Following each epoch, the loss function and accuracy were computed. Notably, the
initial epoch demonstrated a loss value of 0.5012, resulting in an accuracy of 0.7701.
This performance was enhanced in the subsequent epoch, wherein the loss diminished
to 0.3194, accompanied by an accuracy of 90.44%. Concluding the training, the final
epoch revealed a loss measure of 0.2576, with an associated accuracy of 90.10%.
Subsequently, the above mentioned trained model was tested against the valida-
tion dataset. Concurrently, SVM and Random Forest Classification models were also
trained and assessed using the identical dataset. The attained outcomes are shown
below:

15
 Fig. 3: Accuracy Comparison Plot

The accuracy comparison graph (Figure 3) presents a box plot depicting the
achieved accuracy for each model. The assessment of these models was conducted
using identical datasets to ensure uniformity. Notably, the BERT model exhibited the
highest accuracy of 0.8999. Subsequently, the Random Forest Classifier achieved an
accuracy of 0.7867, whereas the SVM model recorded an accuracy of 0.7334.

Fig. 4: Confusion Matrix Fig. 5: Confusion Matrix Fig. 6: Confusion Matrix

(BERT Model) (SVM Model) (Random Forest Model)

The Confusion Matrices illustrated in figure 4, figure 5, and figure 6 correspond to

BERT, SVM, and the Random Forest Classifier, respectively. These matrices provide
a comprehensive view of the distribution of true positive, true negative, false positive,
and false negative predictions. It is noteworthy that the BERT model attained a
notable count of true positives and true negatives, surpassing the performance of the
SVM and Random Forest models. However, it should be acknowledged that BERT also
exhibited a slightly elevated count of false positives and false negatives. This specific
area presents an opportunity for further enhancement.
The ROC curves, as depicted in Figure 7, display the models’ capacity to differenti-
ate between positive and negative classes. Notably, these curves exhibit an inclination
towards the upper-left quadrant of the graph. Remarkably, BERT showcased the most
prominent area under the curve (AUC) value of 0.90, signifying its robust discrimina-
tory prowess. Conversely, SVM achieved an AUC of 0.79, whereas the Random Forest
registered an AUC of 0.82. The elevated AUC values show the superior performance
of both BERT and SVM in accurately classifying the text.

16
 Fig. 7: ROC Curve Fig. 8: F1 Score Plot

The graph displayed above illustrates the F1 score curve, as represented in Figure
8, delineating the equilibrium between precision and recall for each model. Noteworthy
is the fact that the F1 score for the BERT model surfaced as the highest among the
models, signifying its adeptness at achieving a harmonious balance between precision
and recall. Conversely, both SVM and the Random Forest models exhibited marginally
lower F1 scores, implying prospective compromises between precision and recall within
their predictive outcomes.

Fig. 9: Precision Recall Plot

The area beneath the precision-recall curve, as depicted in Figure 9, demonstrates

a significant elevation for the BERT model. This pronounced value establishes a cor-
relation with high precision and recall, followed consecutively by the Random Forest
Classifier and the SVM model. The substantial precision and recall values signal that
the model’s predictions will be accurate across a majority of instances.
Upon comprehensive evaluation of the aforementioned metrics, it becomes evident
that the BERT Classification model excels beyond its SVM and Random Forest Clas-
sifier counterparts in the identification of AI-generated text through the application
of the designated features for the classification task.

17
4.2 Image Detection
The outcomes for the CNN model employed in AI-generated image detection are pre-
sented as follows:
The CNN-ANN model underwent training and evaluation using the CIFAKE dataset
to identify AI-generated images. The results obtained showcase its promising perfor-
mance, clearly distinguishing between AI-generated and real-world images. The model
achieved a minimal loss value of 0.1929, showcasing its capability to minimize the
disparity between predicted and actual labels post 10 epochs of training. On the val-
idation dataset, the model exhibited an accuracy of 93.55%, signifying a remarkably
high rate of accurate predictions overall.
To perform a comprehensive comparative analysis of the CNN model in conjunction
with its contemporaries, namely the Support Vector Machine (SVM) and Random
Forest (RF) models, all three were subjected to training using the CIFAKE dataset.
Visualizing the results, the bar plot presented in Figure 10 offers an illuminating
comparison of achieved accuracy levels.
Significantly, the CNN model stands out by attaining an impressive accuracy of
93.55%. This performance surpasses both the SVM and RF models, which achieved
accuracy rates of 84.84% and 83.41%, respectively. This disparity underscores the
exceptional discriminative prowess of the CNN model in distinguishing between AI-
generated and authentic images.

Fig. 10: Accuracy Comparison Plot

The confusion matrices for each of the models are displayed in the annotated
figures 11, 12, and 13, while a tabulated overview comparing the performance of these
confusion matrices is presented in the subsequent table. In this context, ”Positive”
denotes an AI-generated image, whereas ”Negative” signifies a real image.

Table 1: Performance Comparison

Model True Positive True Negative False Positive False Negative
CNN 9498 9220 780 502
SVM 8545 8422 1578 1455
RF 8723 7959 2041 1277

18
 Fig. 13: Confusion
Fig. 11: Confusion Fig. 12: Confusion Matrix (Random Forest
Matrix (CNN Model) Matrix (SVM Model) Model)

Drawing from these findings, it can be inferred that the CNN model demonstrates
superior performance compared to both the SVM and Random Forest models in effec-
tively distinguishing between AI-generated and real images. The CNN model attained
the highest instances of accurate predictions (18718) along with the lowest occurrences
of incorrect predictions (1292), underscoring its exceptional performance.

Fig. 14: ROC Curve

The ROC curve, displayed in Figure 14, provides a visual representation of the
models’ performance. This curve adeptly highlights the intricate equilibrium between
the true positive rate and the false positive rate. The metric of the area under the
curve (AUC) serves as a robust indicator of the models’ overall efficacy, with elevated
values correlating to enhanced class distinction. Specifically, within this context, the
CNN model demonstrated an AUC of 0.9359. In comparison, the SVM and Random
Forest models secured AUC values of 0.9267 and 0.9132, respectively.
The Precision-Recall curve, juxtaposing the three models, is visualized in figure 15.
Precision, gauging the ratio of accurately predicted AI-generated images to all pro-
jected AI-generated images, stood at 0.9469 for the CNN model. This commendable
precision score underlines the model’s efficacy in curtailing false positives, thereby
mitigating the likelihood of misidentifying genuine images as AI-generated. In addi-
tion, the model achieved a recall rate of 0.9122, signifying the proportion of correctly

19
 Fig. 15: Precision Recall Curve

identified AI-generated images out of the entire set of actual AI-generated images.
This notable recall value suggests the model’s proficiency in minimizing false nega-
tives, ensuring the detection of a significant majority of AI-generated images within
the dataset.

Fig. 16: F1 Score Plot

Finally, the comparative F1 score plot is depicted in Figure 16 above. The F1

score amalgamates precision and recall into a unified metric, offering a comprehensive
evaluation of the models’ overall performance.

In conclusion, the presented CNN model showcased exceptional capabilities in

detecting AI-generated images within the CIFAKE dataset. It exhibited superior per-
formance across various metrics, including accuracy, precision, recall, AUC, and F1
score, effectively surpassing the performance of both the SVM and RF models.

20
5 Conclusion
The proliferation of AI-generated content has ushered in a surge in the velocity and
quantum of information available online. In this context, the imperative to differen-
tiate human-developed data from its AI-generated counterparts becomes paramount.
The detection of AI-generated text employs an alternative strategy, harnessing a fusion
of extensive language models, style assessment, and visual scrutiny to discern text
fashioned by AI models themselves. Conversely, AI-generated images can be discerned
through meticulous pixel analysis, with deviations in color, pattern, and structure
serving as telltale signs of synthetic content. Detecting deepfake content necessitates
consideration of naturally occurring biological cues often overlooked during synthetic
content creation. Cues encompassing eye movements, blood pulsations, and quintessen-
tial human expressions emerge as discernible parameters. Consolidating the insights
presented in this research, it is evident that AI-generated content is poised to domi-
nate the future landscape. Yet, the creation of a model that achieves absolute precision
in identifying such content remains a formidable challenge. Although countermea-
sures such as watermarking and cryptographic signatures will evolve, the potency of
systems generating synthetic content will concurrently heighten. Hence, content val-
idation prior to dissemination and responsible usage emerge as pivotal strategies as
society adjusts to the coexistence of both content paradigms.

6 Future Scope
The realm of generative artificial intelligence is rapidly advancing, offering numerous
potential directions for expanding this research. One avenue involves working with
larger and more diverse datasets, delving into the scalability and architecture of these
models. The success of each model is tightly intertwined to the datasets they are
trained upon. Another avenue to explore is integrating the audio modality, followed by
the incorporation of videos or deepfakes. A proposed approach for the latter has been
briefly mentioned. Additionally, fusing all these diverse models to create a cross-modal
detector ould yield more potent and resilient detection outcomes, achieved through the
exploration of novel techniques. Furthermore, a prospective avenue is the development
of a real-time detection system. Such a system could offer significant value by ensuring
prompt content assessment. Finally, while the current models demonstrate predictions
with high accuracy, their intricate architecture might pose challenges in terms of inter-
pretation. Strategies such as attention mechanisms or visualizing learned traits could
greatly enhance the models’ interpretability, shedding light on the underlying reasons
behind their decisions.

References
[1] M. S. Rana, M. N. Nobi, B. Murali and A. H. Sung, ”Deepfake Detection: A
Systematic Literature Review,” in IEEE Access, vol. 10, pp. 25494-25513, 2022,
doi:10.1109/ACCESS.2022.3154404.

21
 [2] Coccomini, Davide Alessandro, et al. ”Combining efficientnet and vision trans-
formers for video deepfake detection.” Image Analysis and Processing–ICIAP
2022: 21st International Conference, Lecce, Italy, May 23–27, 2022, Proceedings,
Part III. Cham: Springer International Publishing, 2022.

[3] Deepfake Detection Challenge — Kaggle Contest

https://www.kaggle.com/competitions/deepfake-detection-
challenge/discussion/145721

[4] Bonettini, Nicolo, et al. ”Video face manipulation detection through ensemble of
cnns.” 2020 25th international conference on pattern recognition (ICPR). IEEE,
2021.

[5] Mathews, S., Trivedi, S., House, A. et al. ”An explainable deepfake detection
framework on a novel unconstrained dataset”. Complex Intell. Syst. (2023).

[6] Dagar, D., Vishwakarma, D.K. ”A literature review and perspectives in deep-
fakes: generation, detection, and applications”. Int J Multimed Info Retr 11,
219–289 (2022).

[7] Ma, Yongqiang, Jiawei Liu, and Fan Yi. ”Is this abstract generated by ai? a
research for the gap between ai-generated scientific text and human-written
scientific text.” arXiv preprint arXiv:2301.10416 (2023).

[8] Bonettini, Nicolo, et al. ”Video face manipulation detection through ensemble of
cnns.” 2020 25th international conference on pattern recognition (ICPR). IEEE,
2021.

[9] How to spot AI-generated text — MIT Technology Review

https://www.technologyreview.com/2022/12/19/1065596/how-to-spot-ai-
generated-text/

[10] New AI classifier for indicating AI-written text — OpenAI

https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text

[11] Yongqiang Ma, Jiawei Liu, Fan Yi, Qikai Cheng, Yong Huang, Wei Lu, Xiaozhong
Liu (2023), ”AI vs. Human – Differentiation Analysis of Scientific Content Gen-
eration”, arXiv:2301.10416

22
[12] ML Olympiad - Detect ChatGPT Answers — Kaggle Contest
https://www.kaggle.com/competitions/ml-olympiad-detect-chatgpt-
answers/data

[13] Vinu Sankar Sadasivan and Aounon Kumar and Sriram Balasubramanian and
Wenxiao Wang and Soheil Feizi (2023), ”Can AI-Generated Text be Reliably
Detected?”, arXiv:2303.11156

[14] John Kirchenbauer and Jonas Geiping and Yuxin Wen and Jonathan Katz and
Ian Miers and Tom Goldstein (2023), ”A Watermark for Large Language Mod-
els”, arXiv:2301.10226

[15] Chandrasegaran, Keshigeyan, Ngoc-Trung Tran, and Ngai-Man Cheung. ”A

closer look at fourier spectrum discrepancies for cnn-generated images detec-
tion.” Proceedings of the IEEE/CVF conference on computer vision and pattern
recognition. 2021.

[16] Bird, Jordan J., and Ahmad Lotfi. ”CIFAKE: Image Classification and
Explainable Identification of AI-Generated Synthetic Images.” arXiv preprint
arXiv:2303.14126 (2023).

[17] The CIFAR-10 AND CIFAR-100 Dataset Source -

https://www.cs.toronto.edu/ kriz/cifar.html

[18] C. Anitha Mary and A. Boyed Wesley. ”Optimization Assisted Autoregressive

Method with Deep Convolutional Neural Network-Based Entropy Filter for
Image Demosaicing.” ICTACT Journal on Soft Computing, vol. 13, issue 3, pp.
2977-2985 (2023)

[19] Alexander Turchin, Stanislav Masharsky and Marinka Zitnik. ”Comparison of

BERT implementations for natural language processing of narrative medical
documents”. Informatics in Medicine Unlocked, vol. 36, 101139 (2023).

[20] Ravichandra Madanu, Maysam F. Abbod, Fu-Jung Hsiao, Wei-Ta Chen

and Jiann-Shing Shieh. ”Explainable AI (XAI) Applied in Machine
Learning for Pain Modeling: A Review”.Technologies Vol. 10, Issue 3.
doi:10.3390/technologies10030074 (2022)

23