CS492D: Diffusion Models and Their

Applications
Assignment 2 Session

SEUNGWOO YOO

Fall 2024
KAIST

CS492(D): Diffusion Models and Their Applications (Fall 2024)

 How was Assignment 1?

CS492(D): Diffusion Models and Their Applications (Fall 2024) 2

 Introduction
In Assignment 2, we implement Denoising Diffusion Implicit Models
(DDIM) to accelerate the generation process of pretrained DDPMs.

Denoising Diffusion Implicit Models, Song et al., ICLR 2021

CS492(D): Diffusion Models and Their Applications (Fall 2024) 3

 Introduction
We also explore Classifier-Free Guidance (CFG), a simple technique
to enhance image quality in conditional generation.
“Pembroke Welsh corgi”

Weak Guidance Scale Strong Guidance Scale

Diffusion Models Beat GANs on Image Synthesis, Dhariwal and Nichol, PMLR 2021

CS492(D): Diffusion Models and Their Applications (Fall 2024) 4

 Introduction
The skeleton code and instructions are available at:
https://github.com/KAIST-Visual-AI-Group/Diffusion-Assignment2-DDIM-CFG

CS492(D): Diffusion Models and Their Applications (Fall 2024) 5

 Introduction
Copy and paste your DDPM implementation from Assignment 1.

We assume that you completed it, so finish it before starting Assignment 2.

CS492(D): Diffusion Models and Their Applications (Fall 2024) 6

 Important Notes
• All programming assignments are due two weeks after the assignment
session.

• Late submission will incur 20% penalty for each late day!

• Please carefully check the README of each assignment.

• Missing items in your submission will also incur penalties.

CS492(D): Diffusion Models and Their Applications (Fall 2024) 7

 What to Do: Overview
You need to implement:

• [2D Swiss Roll] Reverse Process of DDIMs

• [AFHQ] Class Conditioning Mechanism in U-Net

• [AFHQ] CFG Training and Sampling

CS492(D): Diffusion Models and Their Applications (Fall 2024) 8

 What to Do: Task 1
In the 2D example, replace a single line

1 1 − 𝛼! !
𝒙!"# = 𝒙! − 𝜖$ 𝒙! + 𝜎! 𝜖!
𝛼! 1 − 𝛼&!

in your DDPM implementation with

𝒙! − 1 − 𝛼! 𝜖$! 𝒙! % !
𝒙!"# = 𝛼!"# + 1 − 𝛼!"# − 𝜎! 𝜖$ 𝒙! + 𝜎! 𝜖!
𝛼!

CS492(D): Diffusion Models and Their Applications (Fall 2024) 9

 What to Do: Task 1
Implement functions

• ddim_p_sample

• ddim_p_sample_loop

2d_plot_diffusion_todo/ddpm.py

CS492(D): Diffusion Models and Their Applications (Fall 2024) 10

 What to Do: Task 1
Run all cells in 2d_plot_diffusion_todo/ddpm_tutorial.ipynb to train DDPM
and generate 2D points via DDIM sampling

CS492(D): Diffusion Models and Their Applications (Fall 2024) 11

 What to Do: Task 2
The AFHQ dataset we used previously contains images of 3 classes.

Wildlife

Cat

Dog

CS492(D): Diffusion Models and Their Applications (Fall 2024) 12

 What to Do: Task 2
We will use one-hot encoding to distinguish different classes.

Wildlife
(100)

Cat
(001)

Dog
(010)

CS492(D): Diffusion Models and Their Applications (Fall 2024) 13

 What to Do: Task 2
Add a class conditioning mechanism in our U-Net implementation.

Hint 1: Use self.class_embedding

Hint 2: Add class embeddings to temb

image_diffusion_todo/network.py

CS492(D): Diffusion Models and Their Applications (Fall 2024) 14

 What to Do: Task 2
Randomly replace some class labels to null (000) vector for CFG training.

image_diffusion_todo/network.py

CS492(D): Diffusion Models and Their Applications (Fall 2024) 15

 What to Do: Task 2
Implement noise computation with CFG.

image_diffusion_todo/model.py

CS492(D): Diffusion Models and Their Applications (Fall 2024) 16

 What to Do: Task 2
After implementing the functions, start training the model by running

python train.py –-use-cfg

⚠ Do NOT forget to add the flag –-use-cfg.

⚠ Otherwise, the trained model CANNOT be used for CFG sampling!

CS492(D): Diffusion Models and Their Applications (Fall 2024) 17

 What to Do: Task 2
After training your model, generate samples by

python sampling.py –ckpt-path {CKPT PATH} \

!--save-dir {SAVE DIR}

–-use-cfg –-cfg-scale {CFG Scale}

Try CFG scale of 0.0 and 7.5 and observe how sample quality changes.

⚠ Just like before, do NOT forget to add flag –-use-cfg.

CS492(D): Diffusion Models and Their Applications (Fall 2024) 18

 What to Do: Task 2
Compute the FID by running

Python fid/measure_fid.py {GT_DIR} {GEN_DIR}

Specifically, use

• GT_DIR: data/afhq/eval

• GEN_DIR: The path you passed as save_dir to sampling.py

CS492(D): Diffusion Models and Their Applications (Fall 2024) 19

 What to Submit
Compile the following items into a PDF file: {NAME}_{ID}.pdf.
Task 1
• A screenshot of the Chamfer Distance measured using DDIM;
• A visualization of samples generated using your DDIM.
Task 2
• A screenshot of the computed FIDs with CFG scale= {0.0, 7.5}
• 8 images generated with CFG scale = {0.0, 7.5} (16 images in total)

CS492(D): Diffusion Models and Their Applications (Fall 2024) 20

 What to Submit
Create a single ZIP file {NAME}_{ID}.zip including:

• The PDF file, formatted according to the guideline;

• Your code without any model checkpoints, training data, and outputs.

Your score will be deducted by 10% for each missing item.

Please check carefully!

CS492(D): Diffusion Models and Their Applications (Fall 2024) 21

 Grading
You will receive up to 20 points from this assignment.

Task 1

• 10 points: Achieve CD lower than 60 in DDIM sampling.

• 5 points: Achieve CD greater, or equal to 60 and less than 80.

• 0 point: Otherwise.

CS492(D): Diffusion Models and Their Applications (Fall 2024) 22

 Grading
You will receive up to 20 points from this assignment.

Task 2

• 10 points: Achieve FID lower than 30 in both CFG scales = {0.0, 7.5}.

• 5 points: Achieve FID between 30 and 50 in one of the two CFG scales.

• 0 point: Otherwise.

CS492(D): Diffusion Models and Their Applications (Fall 2024) 23

 Thank You

CS492(D): Diffusion Models and Their Applications (Fall 2024) 24