NAGARJUNA COLLEGE OF ENGINEERING AND TECHNOLOGY

An Autonomous College under VTU, Belagavi

PRESENTATION ON

“IMAGE GENERATION USING DIFFUSION MODELS”

submitted in partial fulfillment for the completion of the course
Project work(21CSMP76)

Presented by

AMRUTHAVARSHINI P - 1NC21CS008
KUNJAL KEDIA - 1NC21CS049
SAMVID ROY - 1NC21CS096
SHREYASI BISWAS - 1NC21CS102

Under the guidance of

Prof. Praveenraj
Dept of CSE, NCET
 CONTENTS

1. Title
2. Abstract
3. Introduction
4. Literature Survey
5. Problem Formulation/Objectives of the Project
6. Methodology
7. Conclusion
 TITLE

“IMAGE GENERATION

USING
DIFFUSION MODELS”
 ABSTRACT

Diffusion model is the gerneration models that are used to generate the images.
They are a class of latent variable models to generate high quality images. Basis for
GLIDE and DALLE-E 2 (by OpenAl) and ImageGen (by Google Brain). The
models also provide a progressive lossy decompression scheme that can be
interpreted as a generalization of autoregressive decoding. On unconditional
CIFAR10 dataset, DDPM obtains an Inception score of 9.46 and a state-of-the-art
FID score of 3.17. On 256x256 LSUN, sample quality is similar
to ProgressiveGAN.

.
 INTRODUCTION

In recent years, deep generative models have revolutionized the field of artificial intelligence by
enabling the creation of high-quality synthetic images, audio, and other media. Among these models,
diffusion probabilistic models represent an innovative approach to image generation by reversing a
process of progressively adding random noise to data, similar to gradually blurring an image and
then learning to un-blur it. Unlike traditional models such as Generative Adversarial Networks
(GANs) that face challenges like stability and sample diversity, diffusion models excel at generating
highly realistic images by iteratively refining a noisy input until it becomes a clear, detailed image.
This process involves training the model to learn complex noise patterns and reconstruct original
images, resulting in exceptional sample quality.
 LITERATURE SURVEY
SL. Title Author Year Description
We aim to generate images or perform image in-
Customized Textual Shubham Paliwal, Arushi painting with controlled text attributes that
1 Image Generation using Jain, Monika Sharma, Vikram 2024
seamlessly integrate into a reference image layout,
Diffusion Models Jamwal, Lovekesh Vig ensuring high harmonization and photo-realism.

Denoising Diffusion Jonathan Ho, Ajay Jain, Pieter Achieving state-of-the-art image synthesis using
2020 diffusion models with advanced training techniques
2 Probabilistic Models Abbeel
Customized Textual Image Generation using Diffusion Models
his article examines diffusion models as leading deep
Research on Denoising generative models, analyzing their mathematical
3 Diffusion Probabilistic foundations, comparing them to other generative
Yiyang Jiang 2024
Models approaches, and discussing their advantages,
limitations, and future research directions.

Improved Denoising Alex Nichol, Prafulla Diffusion probabilistic models (DDPMs) can achieve
4 Diffusion Probabilistic 2021 competitive log-likelihoods and better sample quality
Dhariwal
Models on diverse datasets like ImageNet.
 LITERATURE SURVEY
SL. Title Author Year Description

Image generation using Dr Mahmoud Elbattah Diffusion and Stable Diffusion models generate images
5 diffusion and Stable 2024 by progressively refining random noise, leveraging
diffusion models learned patterns and structures from training data.

enhancing the residual network architecture by

Introduces wide increasing width instead of depth. his approach
Sergey Zagoruyko, Nikos 2016 improves model performance and training efficiency,
6 residual networks Komodaki
demonstrating competitive results on various
Customized Textual Image Generation using Diffusion
benchmark Models
datasets.
Explores the It presents methods to bridge these techniques,
relationship between Salimans, Kingma and improving posterior approximations in probabilistic
7 Markov Chain Monte Welling 2015 modeling and enhancing efficiency in inference
Carlo (MCMC) processes.
It enhancing prediction accuracy by integrating
Predictive sampling autoregressive structures, enabling improved temporal
Auke J Wiggers, Emiel
8 with forecasting 2020 data forecasting and offering valuable insights for
Hoogeboom applications in time series analysis and predictive
autoregressive models.
modeling.
 LITERATURE SURVEY
SL. Title Author Year Description
We aim to generate images or perform image in-
Customized Textual Shubham Paliwal, Arushi painting with controlled text attributes that
9 Image Generation using Jain, Monika Sharma, Vikram 2024
seamlessly integrate into a reference image layout,
Diffusion Models Jamwal, Lovekesh Vig ensuring high harmonization and photo-realism.

Denoising Diffusion Jonathan Ho, Ajay Jain, Pieter Achieving state-of-the-art image synthesis using
2020 diffusion models with advanced training techniques
10 Probabilistic Models Abbeel
Customized Textual Image Generation using Diffusion Models
his article examines diffusion models as leading deep
Research on Denoising generative models, analyzing their mathematical
Diffusion Probabilistic foundations, comparing them to other generative
11 Yiyang Jiang 2024
Models approaches, and discussing their advantages,
limitations, and future research directions.

Improved Denoising Alex Nichol, Prafulla Diffusion probabilistic models (DDPMs) can achieve
12 Diffusion Probabilistic 2021 competitive log-likelihoods and better sample quality
Dhariwal
Models on diverse datasets like ImageNet.
 PROBLEM FORMULATION
In the realm of image synthesis, we aim to enhance the quality and efficiency of generating high-fidelity
images using diffusion probabilistic models. Our primary objective is to develop a robust training
methodology that maximizes the performance of these models through a carefully designed weighted
variational bound. A key aspect of our approach is the implementation of a progressive lossy decompression
scheme, which can be seen as a sophisticated generalization of autoregressive decoding.
 OBJECTIVES OF THE PROJECT

• To develop a diffusion model that can generate high-quality images from

random noise or given conditions (e.g., textual descriptions, sketches).
• Develop and implement a weighted variational bound for effective model
training.
 METHODOLOGY

1. Forward Diffusion Process 2. Reverse Diffusion Process 3. U-Net Architecture 4. Variational Inference
Noise Addition: The forward Denoising Steps: The reverse Model Design: U-Net is commonly Probabilistic Modeling:
process gradually adds Gaussian process aims to recover the used as the backbone architecture for DDPMs utilize variational
noise to the data over a series of original data from the noisy the denoising network due to its inference to model, the
time steps, transforming the original samples by iteratively ability to capture multi-scale features model can learn to generate
data into a pure noise distribution. denoising them, allowing for effectively. The architecture consists samples that closely
the reconstruction of the of an encoder-decoder structure with resemble the training data.
original image. skip connections.

7. Data Augmentation
5. Noise Schedule 6. Training
Enhancing Training Data: To improve
Controlled Noise Addition: Techniques Loss Functions: The
model robustness and generalization,
Techniques such as linear or cosine training of DDPMs typically
data augmentation techniques can be
schedules can be employed to involves minimizing a loss
applied to the training dataset. This
optimize the noise levels at each function that measures the
includes transformations such as rotation,
step, impacting the quality of the difference between the predicted
scaling, and color adjustments, which
generated images. noise and the actual noise added
help the model learn from a more diverse
during the forward process.
set of examples.
 SYSTEM DESIGN

Fig.1 Training model of image generation

 CONCLUSION

The ultimate goal is to create a robust diffusion model capable of producing realistic images, while
also allowing for flexibility in terms of conditions for generation. This structured approach provides
a clear pathway for research and development in the domain of image generation using
diffusion models.