AD2602-Generative AI UNIT IV

UNIT IV
APPLICATIONS OF GENERATIVE AI
Syllabus : Image generation and manipulation – Text generation and natural language
processing – Anomaly detection and data augmentation – Style transfer and artistic applications
– Real-world use cases (Art & Design, Medical Imaging, Content creation, Chatbots, Virtual
Assistants, Cybersecurity, etc.) and industry examples. Guest Lectures by Industry Experts, and
Researchers
Image Generation and Manipulation
Image generation and manipulation involve using generative models like Generative
Adversarial Networks (GANs), Variational Autoencoders (VAEs), and diffusion models to
create or edit visual content. GANs, for instance, are widely used in high-resolution image
synthesis, style transfer, and editing tasks. Applications include creating photorealistic images,
modifying facial features, or generating artistic variations of existing artwork. Techniques like
DeepDream and Neural Style Transfer enable blending artistic styles into photographs. In
practical scenarios, tools such as DALL·E, MidJourney, and RunwayML demonstrate the
capability of AI in generating unique and customized visual content. Manipulation tasks also
extend to object removal, inpainting (filling missing parts of an image), and image super-
resolution, which have applications in fields such as design, forensics, and entertainment.
Image generative AI models are a subset of generative models that aim to create realistic and
coherent images from scratch. These models use complex algorithms and deep learning
techniques to learn patterns and features from a vast amount of training data. They can be
broadly classified into two categories: Variational Autoencoders (VAEs) and Generative
Adversarial Networks (GANs).
Variational Autoencoders (VAEs): VAEs are probabilistic models that encode images into a
latent space, where they are represented as vectors. The decoder then reconstructs the images
from the encoded vectors, enabling the model to generate new images by sampling from the
latent space.
Generative Adversarial Networks (GANs): GANs consist of two neural networks, a generator,
and a discriminator, engaged in a competitive process. The generator creates synthetic images
to fool the discriminator, which, in turn, aims to distinguish between real and fake images. This
back-and-forth battle results in the generation of highly realistic images.
2. Text-to-Image Synthesis
Text-to-image synthesis, a fascinating application of image generative AI models, enables the
conversion of textual descriptions into corresponding visual representations. This process
involves combining natural language processing (NLP) techniques with image generative
models to achieve impressive results.
Conditional GANs for Text-to-Image: Researchers have successfully integrated text
embeddings with GANs, allowing for the generation of images conditioned on specific textual
input. This means that by providing a detailed textual description, the model can generate
highly specific and accurate images.

1
AD2602-Generative AI UNIT IV

Applications of Text-to-Image Synthesis: Text-to-image technology finds applications in

diverse fields, such as e-commerce, content creation, and virtual reality. For instance, in e-
commerce, this technology can generate product images from textual descriptions, aiding in
faster product design and development.
3. State-of-the-Art Text-to-Image Models
Over the years, several state-of-the-art models have emerged, showcasing the impressive
capabilities of text-to-image synthesis:
DALL-E: Developed by OpenAI, DALL-E is a prominent text-to-image model capable of
creating unique and creative images based on textual prompts. It has the ability to generate art,
objects, and even fictional creatures with remarkable precision.
CLIP: Another innovation by OpenAI, CLIP enables the model to understand and generate
images based on natural language descriptions. It combines vision and language pre-training
to achieve impressive cross-modal capabilities.
CM3leon: One of the latest and most promising additions to the world of image generative AI
models is Meta CM3leon. This groundbreaking multimodal model, introduced by Meta AI, has
garnered significant attention for its unparalleled text-to-image generation capabilities, coupled
with unmatched compute efficiency.
TEXT GENERATION AND NATURAL LANGUAGE PROCESSING
Text generation and natural language processing (NLP) leverage advanced language models
like GPT (Generative Pre-trained Transformer) and BERT to create coherent and contextually
relevant text. These models can generate essays, write poetry, summarize documents, or
simulate human-like conversations. Text generation finds applications in creative writing,
chatbots, automated journalism, and code generation. Furthermore, NLP extends to tasks such
as sentiment analysis, translation, and question-answering systems. Real-world tools like
ChatGPT and Google Translate are prime examples of these capabilities. Recent advancements
in large language models (LLMs) have further enabled complex tasks like summarizing legal
documents, writing marketing copy, and generating code snippets in programming.
Text generation is the process of automatically producing coherent and meaningful text, which
can be in the form of sentences, paragraphs or even entire documents. It involves various
techniques, which can be found under the field such as natural language processing (NLP),
machine learning and deep learning algorithms, to analyze input data and generate human-like
text. The goal is to create text that is not only grammatically correct but also contextually
appropriate and engaging for the intended audience.
Natural language generation (NLG) and natural language understanding (NLU) are 2 essential
components of a robust natural language processing (NLP) system, but they serve different
purposes.
Natural language understanding (NLU) is the ability of a machine to comprehend, interpret and
extract meaningful information from human language in a valuable way. It involves tasks like
sentiment analysis, named entity recognition, part-of-speech tagging and parsing. NLU helps
machines understand the context, intent and semantic meaning of human language inputs.

2
AD2602-Generative AI UNIT IV

Natural language generation (NLG) is the ability of a machine to produce human-like text or
speech that is clear, concise and engaging. It involves tasks like text summarization,
storytelling, dialogue systems and speech synthesis. NLG helps machines generate meaningful
and coherent responses in a way that is easily understood by humans.
NLU focuses on understanding human language, while NLG focuses on generating human-like
language. Both are crucial for building advanced NLP applications that can effectively
communicate with humans in a natural and meaningful way.
Text generation techniques
Statistical models: These models typically use a large dataset of text to learn the patterns and
structures of human language, and then use this knowledge to generate new text. Statistical
models can be effective at generating text that is similar to the training data, but they can
struggle to generate text that is both creative and diverse. N-gram models and conditional
random fields (CRF) are popular statistical models.
N-gram models: These are a type of statistical model that uses the n-gram language model,
which predicts the probability of a sequence of "n-items" in a given context.
Conditional random fields (CRFs): These are a type of statistical model that uses a
probabilistic graphical model to model the dependencies between words in a sentence. CRFs
can be effective at generating text that is both coherent and contextually appropriate, but this
type of text generation model can be computationally expensive to train and might not perform
well on tasks that require a high degree of creative language generation.
Neural networks: These are machine learning algorithms that use artificial neural networks to
identify data patterns. Through APIs, developers can tap into pretrained models for creative
and diverse text generation, closely mirroring the training data's complexity. The quality of the
generated text heavily relies on the training data. However, these networks demand significant
computational resources and extensive data for optimal performance.
Recurrent neural networks (RNNs): These are a foundational type of neural network
optimized for processing sequential data, such as word sequences in sentences or paragraphs.
They excel in tasks that require understanding sequences, making them useful in the early
stages of developing large language models(LLMs). However, RNNs face challenges with
long-term dependencies across extended texts, a limitation stemming from their sequential
processing nature. As information progresses through the network, early input influence
diminishes, leading to the "vanishing gradient" problem during backpropagation, where
updates shrink and hinder the model's ability to maintain long-sequence connections.
Incorporating techniques from reinforcement learning can offer strategies to mitigate these
issues, providing alternative learning paradigms to strengthen sequence memory and decision-
making processes in these networks.
Long short-term memory networks (LSTMs): This is a type of neural network that uses a
memory cell to store and access information over long periods of time. LSTMs can be effective
at handling long-term dependencies, such as the relationships between sentences in a
document, and can generate text that is both coherent and contextually appropriate.

3
AD2602-Generative AI UNIT IV

Transformer-based models: These models are a type of neural network that uses self-attention
mechanisms to process sequential data. Transformer-based models can be effective at
generating text that is both creative and diverse, as they can learn complex patterns and
structures in the training data and generate new text that is similar to the training data. Unlike
historical approaches such as RNNs and LSTMs, transformer-based models have the distinct
advantage of processing data in parallel, rather than sequentially. This allows for more efficient
handling of long-term dependencies across large datasets, making these models especially
powerful for natural language processing applications such as machine translation and text
summarization.
Generative pretrained transformer (GPT): GPT is a transformer-based model that is trained
on a large dataset of text to generate human-like text. GPT can be effective at generating text
that is both creative and diverse, as it can learn complex patterns and structures in the training
data and generate new text that is similar to the training data.
Bidirectional encoder representations from transformers (BERT): BERT is a transformer-
based model that is trained on a large dataset of text to generate bidirectional representations
of words. That means it evaluates the context of words from both before and after a sentence.
This comprehensive context awareness allows BERT to achieve a nuanced understanding of
language nuances, resulting in highly accurate and coherent text generation. This bidirectional
approach is a key distinction that enhances BERT's performance in applications requiring deep
language comprehension, such as question answering and named entity recognition (NER), by
providing a fuller context compared to unidirectional models.
Thus, text generation techniques, especially those implemented in Python, have revolutionized
the way we approach generative AI in the English language and beyond. Using trained models
from platforms like Hugging Face, developers and data scientists can access a plethora of open
source tools and resources that facilitate the creation of sophisticated text generation
applications. Python, being at the forefront of AI and data science, offers libraries that simplify
interacting with these models, allowing for customization through prefix or template
adjustments, and the manipulation of text data for various applications. Furthermore, the use
of metrics and benchmarks to evaluate model performance, along with advanced decoding
strategies, ensures that the generated text meets high standards of coherence and relevance.
Examples of text generation
Text generation is a versatile tool that has a wide range of applications in various domains.
Here are some examples of text generation applications:
• Blog posts and articles:
It can be used to automatically generate blog posts and articles for websites and blogs. These
systems can automatically generate unique and engaging content that is tailored to the reader's
interests and preferences.
• News articles and reports:
It can be used to automatically generate news articles and reports for newspapers, magazines
and other media outlets. These systems can automatically generate timely and accurate content
that is tailored to the reader's interests and preferences.

4
AD2602-Generative AI UNIT IV

• Social media posts:

It can be used to automatically generate social media posts for Facebook, Twitter and other
platforms. These systems can automatically generate engaging and informative content that is
tailored to the reader's interests and preferences.
• Product descriptions and reviews:
It can be used to automatically generate product descriptions and reviews for e-commerce
websites and online marketplaces. These systems can automatically generate detailed and
accurate content that is tailored to the reader's interests and preferences.
• Creative writing:
It can be used to automatically generate creative writing prompts for writers with powerful AI
models. These systems can automatically generate unique and inspiring ideas that are tailored
to the writer's interests and preferences.
• Language translation:
It can be used to automatically translate text between different languages. These systems can
automatically generate accurate and fluent translations that are tailored to the reader's interests
and preferences.
• Chatbot conversations:
It can be used to automatically generate chatbot conversations for customer service and
support. These systems can automatically generate personalized and engaging conversations
that are tailored to the reader's interests and preferences.
• Text summaries:
It condenses lengthy documents into concise versions, preserving key information through
advanced natural language processing and machine learning algorithms. This technology
enables quick comprehension of extensive content, ranging from news articles to academic
research, enhancing information accessibility and efficiency.
• Virtual assistant interactions:
Text generation can be used to automatically generate virtual assistant interactions for home
automation and personal assistance. These systems can automatically generate personalized
and convenient interactions that are tailored to the reader's interests and preferences.
• Storytelling and narrative generation:
Text generation can be used to automatically generate stories and narratives for entertainment
and educational purposes. These systems can automatically generate unique and engaging
stories that are tailored to the reader's interests and preferences.

5
AD2602-Generative AI UNIT IV

ANOMALY DETECTION AND DATA AUGMENTATION

Anomaly detection uses generative models to identify unusual patterns or outliers in data,
making it invaluable for industries like finance, cybersecurity, and healthcare. For instance,
autoencoders and VAEs can model normal data patterns and flag deviations as anomalies. This
is crucial for fraud detection, network intrusion monitoring, and detecting manufacturing
defects. Data augmentation, on the other hand, enhances the quality and quantity of datasets by
artificially generating variations. Techniques such as flipping, rotating, or cropping images, and
using GANs to create synthetic data, improve model robustness and generalization. In NLP,
data augmentation involves paraphrasing, synonym substitution, and back-translation to
diversify training datasets. These techniques are particularly critical in scenarios with limited
labeled data.
Data augmentation is a powerful technique used to enhance the performance of machine
learning models, particularly in scenarios where the available training data is limited. In the
context of anomaly detection, it plays a crucial role in improving the model's ability to identify
rare or unexpected events.
Common Data Augmentation Methods
Traditional data augmentation methods often involve simple transformations such as:
• Rotation: Rotating the image by a certain angle.
• Translation: Shifting the image horizontally or vertically.
• Scaling: Resizing the image.
• Flipping: Flipping the image horizontally or vertically.
• Cropping: Extracting a portion of the image.
While these methods can increase the diversity of the training data, they may sometimes
generate samples that are significantly different from the original image, potentially misleading
the model.
Advanced Data Augmentation Techniques
To address the limitations of traditional methods, more sophisticated techniques have been
developed:
• AugMix: This method combines the original image with multiple augmented versions,
creating a diverse set of synthetic images that preserve the original image's features.
• Mask Pool: This technique involves masking out portions of the image, forcing the
model to focus on different regions and improving its ability to generalize to unseen
anomalies.
• Learnable Data Augmentation: This approach automatically learns the optimal
augmentation parameters for a specific dataset and task, leading to more effective and
efficient training.
Anomaly Detection Methods
Several approaches can be used for anomaly detection:

6
AD2602-Generative AI UNIT IV

• Embedding-based Methods: These methods utilize pre-trained models to extract

high-level features from images. Anomalies are identified as data points that are
significantly different from the normal data distribution in the feature space.
• Reconstruction-based Methods: These methods aim to reconstruct the input data from
a compressed representation. Anomalies are detected as data points that cannot be
accurately reconstructed.
• Local Outlier Factor (LOF): LOF is a density-based method that calculates the local
density of each data point based on its nearest neighbors. Data points with significantly
lower density than their neighbors are considered outliers.
Anomaly Detection in Industries
Anomaly detection has numerous applications across various industries, including:
• Manufacturing: Identifying defective products, predicting equipment failures, and
optimizing production processes.
• Healthcare: Detecting medical abnormalities in images, monitoring patient health, and
identifying potential outbreaks.
• Finance: Detecting fraudulent transactions, identifying market anomalies, and assessing
credit risk.
• Security: Detecting intrusions in computer networks, identifying suspicious activities,
and preventing cyberattacks.
By effectively leveraging data augmentation techniques and employing suitable anomaly
detection methods, organizations can significantly improve their ability to identify and respond
to unexpected events, leading to increased efficiency, safety, and productivity.
STYLE TRANSFER AND ARTISTIC APPLICATIONS
Style transfer refers to applying the artistic style of one image to another, often achieved using
convolutional neural networks (CNNs) and GANs. This technology powers creative tools that
allow artists and designers to reimagine their works in the styles of famous painters or other
visual aesthetics. Neural Style Transfer, for example, can blend Van Gogh’s or Monet’s style
with a photograph, creating visually captivating results. Artistic applications extend to music
composition, video editing, and even fashion design, enabling AI to act as a creative
collaborator. Industries such as gaming and entertainment utilize style transfer to enhance the
visual appeal of their products and create immersive experiences.
Neural Style Transfer
NST is a deep learning technique that allows us to merge the content of one image with the
artistic style of another, resulting in visually stunning and unique artworksAt its core, Neural
Style Transfer is a process of taking two images: a content image and a style image, and
combining them to create a new image. The resultant image retains the content of the content
image while adopting the artistic style of the style image. This technique is inspired by the way
our brains perceive and interpret art. When we look at a painting, for instance, we can recognize
the objects in the scene (content) and the unique artistic strokes and colors used by the artist
(style).

7
AD2602-Generative AI UNIT IV

Here’s a simple breakdown of the steps involved in Neural Style Transfer:

Input Images: As mentioned, you need two images — a content image and a style image.
Neural Network: You use a pre-trained deep neural network, typically a Convolutional Neural
Network (CNN), to extract both the content and style features from these images.
Loss Functions: You define two loss functions, one for content and one for style. The content
loss measures the difference in content between the generated image and the content image.
The style loss measures the difference in style between the generated image and the style image.
Optimization: The objective is to minimize the content and style loss simultaneously. You
update the generated image iteratively until you achieve a satisfactory result.
Neural style transfer is an optimization technique used to take two images, a content image and
a style reference image (such as an artwork by a famous painter), and blend them so the output
image looks like the content image, but “painted” in the style of the style reference image. This
technique is used by many popular Android iOS apps such as Prisma, DreamScope,
and PicsArt.
VGG-19 Architecture overview
VGG-19 is a convolutional neural network (CNN) architecture from the VGG family of
models. The Visual Graphics Group (VGG) at the University of Oxford introduced the VGG
models, which are known for their simplicity and uniform architecture. VGG-19 has 19 layers,
including 16 convolutional layers and 3 fully connected layers. The following are the key
features of the VGG-19 architecture:
Input Layer:
• Accepts 224×224 pixel images with three colour channels (RGB) as input.
Convolutional Blocks (Blocks 1–5):
• VGG-19 is made up of five convolutional block sets. Each block is made up of several
convolutional layers followed by max-pooling layers.
• Convolutional layers commonly employ small 3×3 filters with a stride of one and
rectified linear unit (ReLU) activation functions.
• To reduce spatial dimensions, max-pooling layers with 2×2 filters and a stride of 2 are
used.
Layers that are fully connected (FC6, FC7, and FC8):
• There are three fully connected layers (FC6, FC7, and FC8) following the convolutional
blocks.
• The FC6 and FC7 layers each contain 4096 neurons and employ ReLU activation
functions.
• The FC8 layer (output layer) contains 1000 neurons with softmax activation, which
correspond to the 1000 classes in the ImageNet dataset on which VGG-19 was trained.
Parameters:

8
AD2602-Generative AI UNIT IV

• Although VGG-19 is known for its simplicity, it has a large number of parameters,
owing to the fully connected layers.
• There are approximately 143.7 million trainable parameters in total.
Pre-trained Model:
• VGG-19 is a popular pre-trained model for a variety of computer vision tasks.
Researchers pre-trained it on large datasets such as ImageNet, allowing it to capture a
diverse set of features from various categories.
The neural style transfer paper uses feature maps generated by intermediate layers of VGG-19
network to generate the output image. This architecture takes style and content images as input
and stores the features extracted by convolution layers of VGG network.

9
AD2602-Generative AI UNIT IV

Losses in Neural Style Transfer

REAL-WORLD USE CASES

• Art & Design: AI-generated art and design tools, such as Adobe’s AI-powered applications
and platforms like Artbreeder, enable creators to generate and customize digital artwork.
AI can also assist in product design by generating prototypes or iterating on existing ideas.
• Medical Imaging: Generative models are transforming medical imaging by enhancing
image resolution, filling gaps in incomplete scans, and simulating realistic data for training
purposes. GANs are used to generate synthetic MRI or CT images for disease detection,
while style transfer techniques improve scan visualization.
• Content Creation: AI-powered tools like Jasper and Writesonic automate content writing
for blogs, advertisements, and social media. Video generation platforms like Synthesia
create lifelike avatars for educational or marketing content.
• Chatbots and Virtual Assistants: AI-driven systems like ChatGPT, Alexa, and Google
Assistant use NLP to deliver seamless conversational experiences. These tools help
businesses provide 24/7 customer support and improve user engagement.

10
AD2602-Generative AI UNIT IV

• Cybersecurity: Generative AI aids in simulating attack scenarios for training, detecting

malware, and generating realistic phishing email templates for security awareness training.
Models like VAEs help identify anomalous behavior in network traffic, enhancing intrusion
detection systems.

11