Scribd
Upload
7 views

Prompt Engr Module 9

The document provides an introduction to DALL-E 2 and prompt engineering. It explains that DALL-E 2 is an AI tool that can generate images from text descriptions using diffusion models. It also outlines the process that DALL-E 2 uses to generate images and discusses factors to consider when crafting prompts.

Uploaded by

enonimoussse
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
7 views

Prompt Engr Module 9

The document provides an introduction to DALL-E 2 and prompt engineering. It explains that DALL-E 2 is an AI tool that can generate images from text descriptions using diffusion models. It also outlines the process that DALL-E 2 uses to generate images and discusses factors to consider when crafting prompts.

Uploaded by

enonimoussse
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

DALL-E 2

1
Introduction to Prompt Engineering

Module 009 – DALL-E


At the end of this module you are expected to:
1. Understand the basics of DALL-E 2 and how it can be used to generate
images from text descriptions.
2. Learn how to create and evaluate prompts for DALL-E 2.

DALL-E 2

DALL-E 2 is a large language model (LLM) developed by OpenAI that can generate images from text
descriptions. It is a powerful tool that can be used to create realistic and creative images, but it is
important to understand its limitations.

DALL-E 2 is a tool that can create images from text descriptions. It can be used to create realistic images
of objects, scenes, and concepts. For example, you could ask DALL-E 2 to create an image of a cat wearing
a cowboy hat and boots, or a painting of a cityscape in the style of Van Gogh.

DALL-E 2 works by using a technique called diffusion modeling. Diffusion modeling starts with a blank
image and gradually adds detail to it until it matches the text description. This process is similar to how a
sculptor might create a statue by starting with a block of marble and gradually chipping away at it until it
takes the desired shape.

The diffusion modeling process used by DALL-E 2 is controlled by a neural network. This neural network
is trained to generate images that are both realistic and creative. The neural network is trained using a
technique called supervised learning. In supervised learning, the neural network is given a set of input
data and output data. The neural network then learns to map the input data to the output data.

DALL-E 2 uses a diffusion model to generate images. A diffusion model is a type of generative model that
starts with a random image and gradually transforms it into an image that matches the text description.
The transformation is done by gradually adding noise to the image, and then reducing the noise until the
image matches the text description.

The diffusion model in DALL-E 2 is trained on a dataset of text descriptions and corresponding images.
The dataset includes a wide variety of images, from simple objects to complex scenes. This allows the
diffusion model to learn the relationship between text and images for a wide variety of concepts.

Course Module
DALL-E 2
2
Introduction to Prompt Engineering

When a user gives DALL-E 2 a text description, the LLM first converts the description into a set of
features. These features represent the objects, colors, and relationships in the image. The diffusion model
then starts with a random image and gradually transforms it to match the features.

The diffusion model is able to generate images with a high degree of realism and detail. However, it is still
under development, and it can sometimes generate images that are not accurate or realistic.

The diffusion model is a generative adversarial network (GAN). GANs are a type of machine learning
model that can be used to generate realistic images.

The diffusion model in DALL-E 2 is trained on a dataset of text descriptions and corresponding images.
The dataset includes a wide variety of images, from simple objects to complex scenes. This allows the
diffusion model to learn the relationship between text and images for a wide variety of concepts.

The diffusion model in DALL-E 2 is able to generate images with a high degree of realism and detail.
However, it is still under development, and it can sometimes generate images that are not accurate or
realistic.

DALL-E 2 Process:
 The user gives DALL-E 2 a text description of the image they want to generate.
 The LLM converts the text description into a set of features. These features represent the objects,
colors, and relationships in the image.
 The diffusion model starts with a random image and gradually transforms it to match the features.
 The diffusion model does this by gradually adding noise to the image, and then reducing the noise
until the image matches the features.
 The diffusion model also uses a technique called CLIP to ensure that the generated image is
realistic and matches the text description. CLIP is a neural network that can compare images and
text descriptions, and it can be used to identify images that are not realistic or that do not match
the text description.
 The diffusion model continues to transform the image until it reaches a certain level of realism or
until it fails to improve the image any further.
 The final image is then output to the user.

DALL-E 2 is still under development, but it has already been used to create some amazing images. It has
the potential to be a powerful tool for artists, designers, and creative professionals. It could also be used
to create educational materials, marketing materials, and even new forms of art.

Course Module
DALL-E 2
3
Introduction to Prompt Engineering

Example Code:

import torch
import numpy as np

# Load the diffusion model


model = torch.load("dall-e-2.pt")

# Create a text description of the image we want to generate


text_description = "A painting of a cat in the style of Picasso"

# Convert the text description into a set of features


features = CLIP.encode_text(text_description)

# Generate the image


image = model.generate(features)

# Save the image


np.save("image.npy", image)

This code first loads the diffusion model, which is a neural network that can be used to generate images.
It then creates a text description of the image we want to generate, and converts the text description into
a set of features. The features represent the objects, colors, and relationships in the image.

The code then generates the image by running the diffusion model on the features. The diffusion model
starts with a random image and gradually transforms it to match the features. The image is saved to a file
called "image.npy".

Course Module
DALL-E 2
4
Introduction to Prompt Engineering

References and Supplementary Materials


Books and Journals
1. https://www.researchgate.net/publication/360310862_Prompt_Engineering_for_Tex
t-Based_Generative_Art
2. https://arxiv.org/pdf/2107.13586.pdf
3. Oppenlaender, Jonas. (2022). Prompt Engineering for Text-Based Generative Art.
Online Supplementary Reading Materials
1. https://www.classcentral.com/course/chatgpt-for-developers-180241
2. https://www.flowrite.com/blog/introduction-to-prompt-engineering
3. https://docs.cohere.com/docs/prompt-engineering
4. https://solutions.yieldbook.com/content/dam/yieldbook/en_us/documents/publicat
ions/using-chatgpt-with-prompt-engineering.pdf
Online Instructional Videos
1. https://youtu.be/dOxUroR57xs?feature=shared
2. https://youtu.be/JTxsNm9IdYU?feature=shared
3. https://youtu.be/BP9fi_0XTlw?feature=shared

Course Module

You might also like