Module 2:

Planning a Generative AI
Project

1
 Module On completion, you will be able to do
objectives the following:
• Discuss the technical foundations and key
terminology for generative artificial
intelligence (generative AI).
• Explain the steps for planning a
generative AI project.
• Identify some of the risks and mitigations
when using generative AI.

2
Technical foundations and
terminology for generative AI
Generative AI fundamentals
Generative AI in practice
Generative AI context

3
Foundation models

2 5

1 3 4

4
Evolution of ML and the emergence of generative AI

Content Question and

Content generation Code generation
summarization answer

5
How are foundation models created?

6
Pretraining

Large amount Training algorithm Foundation model

of unlabeled
data

7
Components of FMs
Unlabeled data Large model
• Easier to obtain compared to • Billions of parameters
labeled data
• Pretraining models of this
• Pretraining models take size requires access to:
context into account from all • Sufficient quantity and quality of
of this training data training data
• Tracks the relationships in • Large-scale training
sequential data infrastructure

8
How does this unlabeled data get
processed?

9
Transformer The transformer architecture is a type
of neural network that is efficient, easy
to scale and parallelize, and can model
interdependence between input and
output data.

10
Transformer Benefits of transformers include the
benefits following:
• Memory efficient and can be trained
in parallel on GPUs
• Retain and use information about
meaning and position
• Pay attention to multiple things at
once, in different ways

11
Technical foundations and
terminology for generative AI
Generative AI fundamentals
Generative AI in practice
Generative AI context

12
A puppy is to dog as kitten is to…

13
How a transformer model completes a sentence

Tokenization
Word
and Decoding Outputs
embedding
encoding

14
Tokenization and encoding

“A puppy is to dog as kitten is to…”

pupp kitte
[CLS] A is to dog as is to [SEP]
y n

1702 1840
101 1037 2003 2000 3899 2004 2003 2000 102
2 1

15
Word embedding

16
Words with similar meaning are
located closer to each other in this
vector space.

17
Decoder A puppy is to dog as kitten is to_______.
cat 9.2
bird 5.5
bear 4.7
human 3.7

18
Output

“A puppy is to dog as kitten is to cat.”

19
Technical foundations and
terminology for generative AI
Generative AI fundamentals
Generative AI in practice
Generative AI context

20
Context Context is a private exchange between
the user and the model.
• It does not persist.
• There is an upper limit on the number
of tokens.
• Initial information the model is using
can be lost.

21
Example The transformer model has to figure out what
this refers to in the context of the current
conversation.
Prompt (Turn 1)

Hi, what's the best place in Seattle to visit?

Response:

The Columbia Center offers breathtaking views of the city

skyline […].

Prompt (Turn 2, continue interaction):

Will this be fun for children?

Response:

No. There are much better places for children in Seattle.

22
Why generative
AI is like an
interview
• Maximum amount
of time
• Information isn’t
retained for new
sessions
• Each session starts
over

23
Question 1
What is the term for a Choice Response

type of neural network A Foundation model

that is efficient, easy to
scale and parallelize, and
B Model pretraining
can model
interdependence
between input and output C Transformer architecture
data?
D Word embedding

24
Question 1 answers
What is the term for a Choice Response

type of neural network A Foundation model

that is efficient, easy to
scale and parallelize, and
B Model pretraining
can model
interdependence C
between input and output correc Transformer architecture
data? t
D Word embedding

25
Planning a generative AI project
Steps in planning a generative AI
project

26
Planning a generative AI project

Step 1: Step 2: Step 3:

Step 4: Use
Define the Select a Adapt the
the model
scope model model

27
Step 1: Define the scope

• Do your customers want this?

• Can your organization do this?
• Should your organization do
this?

28
Long-term and Assess the long-term and short-term
short-term impacts that each solution provides.
impacts • What are the trade-offs of having
faster development cycles for your
developers?
• What about implementation time of
the solutions?

29
Step 2: Select a model

Pretrained model Fine-tuning a pre-existing model

• Provides a quick solution for integrating • Offers the flexibility to tailor the model
generative AI capabilities to your specific needs
• Suitable for general tasks • Specialized tasks
• Good choice when the need for • Wider range of open-source models
customization is minimal and for when • Downsides include greater
speed is important computational resources, expertise, and
time
30
Discussion Chatbot example
Work backwards to determine the
solutions you intend to provide and
where that data will come from.
• Will they be general questions that
could be facilitated by a pretrained
chatbot model?
• Will they be more focused on
datasets that appear uniquely in your
organization?

31
Step 3: Adapt the model

Fine-tuning a continuation of pre-

Prompt engineering
training
• Designing and refining your • Create a new specialized model
prompts or inputs to generate • When fine-tuning, you change the
specific types of outputs that suit parameters in the model and
your needs. create a new model specific to
your solution.
32
Step 4: Use the model
 Have you managed all of the responsible AI concerns?
 Do you have a plan for feedback from users?
 How are you going to track performance of your FM over
time?
 How are you tracking changes to the pretrained model so
you can retrain your fine-tuned model?

33
Activity Chatbot example
Apply the four-step framework:
1. What is the scope?
2. Would you use a pretrained model, or fine-
tune a pre-existing model?
3. Would you use prompt engineering or fine-
tuning a continuation of pretraining?
4. When using the model, what would you
consider?

34
Evaluating the use of
generative AI for your project
Risks and mitigation

35
Evaluating the • Not all problems can or should be
use of solved by generative AI.
generative AI • Take into account the risks of using
generative AI, including whether or
not those risks can be mitigated.

36
Generative AI considerations

Fairness Privacy

37
Concerns of generative AI

Toxicity Hallucinations Intellectual property

Plagiarism and Disruption of the nature

cheating of work
38
Discussion Think about the use case you
came up with earlier in the course.
• What were some of the risks?
• How could those risks be mitigated?

39
 Module You should now be able to do the
summary following:
• Discuss the technical foundations and
key terminology for generative AI.
• Explain the steps for planning a
generative AI project.
• Identify some of the risks and
mitigations when using generative AI.

40