Copyright Notice

These slides are distributed under the Creative Commons License.

DeepLearning.AI makes these slides available for educational purposes. You may not use or distribute
these slides for commercial purposes. You may make copies of these slides and use or distribute them for
educational purposes as long as you cite DeepLearning.AI as the source of the slides.

For the rest of the details of the license, see https://creativecommons.org/licenses/by-sa/2.0/legalcode

 Software
Applications
Using generative AI in
software applications
Examples of software applications
Writing Reading Chatting
Is there parking for staff? Classify the sentiment of the Welcome to BettaBurgers!
following review as either
Yes, employees can park on positive or negative: I’d like a cheeseburger for
levels 1 and 2 of the office lot. delivery!
You can get a sticker at [...] The mochi is excellent!

Positive Awesome. Anything else?

That’s it thanks!

Ok, it’ll be there in 20

minutes. Enjoy!

Andrew Ng
Andrew Ng
Supervised learning for restaurant reputation
monitoring
Input Output Input (A) Output (B)
Best soup dumplings I’ve ever eaten. Positive
A B The colorful tablecloths made me smile! Positive
Not worth the 3 month wait for a reservation Negative

Get Train Deploy

labeled AI model (run)
data on data model

Input: “Best bubble tea I’ve ever had” Output: Positive

Andrew Ng
Prompt-based development
prompt = """
Classify the following review
as having either a positive or Instruction text
negative sentiment:

The banana pudding was really

Review text
tasty!
"""

response = llm_response(prompt) Code to call LLM

print(response) Code to print output

Andrew Ng
Workflow using Generative AI

Get Train Deploy

Supervised learning labeled AI model (run)
data on data model

1 month 3 months 3 months

Deploy
Specify
Prompt-based AI prompt
(run)
model

minutes/hours hours/days

Andrew Ng
 Software
Applications
Trying generative AI code
yourself (optional)
 Coding platform (Jupyter notebook)

Play button

The one
command
to know:

shift + enter

Andrew Ng
 Software
Applications
Lifecycle of a
generative AI project
Lifecycle of a generative AI project

Build/improve Internal
Scope project
system evaluation

Classify the sentiment

Initially a prototype, of the following review
that we will improve as either positive or
over time negative:

The custard tart was

amazing!
Positive

Andrew Ng
Lifecycle of a generative AI project

Build/improve Internal
Scope project
system evaluation

Classify the sentiment

Initially a prototype, of the following review
that we will improve as either positive or
over time negative:

My pasta was cold

Positive Incorrect
response

Andrew Ng
Lifecycle of a generative AI project

Build/improve Internal Deploy and

Scope project
system evaluation monitor

Classify the sentiment

Initially a prototype, of the following review
that we will improve as either positive or
over time negative:

My miso ramen tasted

like tonkotsu ramen.
Positive

Andrew Ng
Tools to improve performance

Building Generative AI is a highly empirical (experimental)

process – we repeatedly find and fix mistakes.
• Prompting

Andrew Ng
Tools to improve performance

Building Generative AI is a highly empirical (experimental)

process – we repeatedly find and fix mistakes.
• Prompting
• Retrieval augmented generation (RAG)
• Give LLM access to external data sources

Andrew Ng
Tools to improve performance

Building Generative AI is a highly empirical (experimental)

process – we repeatedly find and fix mistakes.
• Prompting
• Retrieval augmented generation (RAG)
• Give LLM access to external data sources
• Fine-tune models
• Adapt LLM to your task

Andrew Ng
Tools to improve performance

Building Generative AI is a highly empirical (experimental)

process – we repeatedly find and fix mistakes.
• Prompting
• Retrieval augmented generation (RAG)
• Give LLM access to external data sources
• Fine-tune models
• Adapt LLM to your task
• Pretrain models
• Train LLM from scratch

Andrew Ng
Lifecycle of a generative AI project

Build/improve Internal
Scope project Build system
system evaluation

Take food Let team

order order…
I’d like a mushroom
and swiss burger

I’m sorry, we don’t Incorrect

have mushrooms, how
about onions?
response

Andrew Ng
Lifecycle of a generative AI project

Internal
Scope project Build system
evaluation

Take food Let team

order order…
Do you have pickles?

Yes – would you like to

add some?

Andrew Ng
Lifecycle of a generative AI project

Build/improve Internal Deploy and

Scope project
system evaluation monitor

Take food Let customers

order order, monitor
LLM responses
How many calories in
the cheeseburger?
I’m sorry, I don’t know

Andrew Ng
Lifecycle of a generative AI project

Build/improve Internal Deploy and

Scope project
system evaluation monitor

Take food Let customers

order order, monitor
LLM responses
How many calories in
the cheeseburger?
About 375 calories

Andrew Ng
 Software
Applications
Cost intuition
How much does it cost?
Example prices
OpenAI/GPT3.5 OpenAI/GPT4 Google/PaLM 2 Amazon/Titan Lite

Input tokens $0.0015/1K tokens $0.03/1K tokens $0.0005/1K tokens $0.0003/1K tokens

Output tokens $0.002/1K tokens $0.06/1K tokens $0.0005/1K tokens $0.0004/1K tokens

What is a token?
the example Andrew 1 token

translate programming 2 tokens

tonkotsu 4 tokens Roughly, 1 token = 3/4 words

Andrew Ng
Estimating Cost

Typical adult reading speed: 250 words/minute.

How much would it cost to keep someone occupied for 1 hour?

Andrew Ng
 Advanced
Technologies:
Beyond Prompting
Retrieval Augmented
Generation (RAG)
Retrieval Augmented Generation (RAG) example

General Chatbot Chatbot with RAG

Is there parking for employees? Is there parking for employees?

I need more specific information Yes, employees can park on

about your workplace to answer levels 1 and 2 of the office lot. You
that question. can get a sticker at […]

Andrew Ng
Retrieval Augmented Generation (RAG) example

1. Given question, search relevant 2. Incorporate retrieved text into

documents for answer an updated prompt
Is there parking for employees? Use the following pieces of context
to answer the question at the end.

Company documents Parking Policy: All employees may

park on levels 1 and 2 of the office
lot. Use the entrance on Front St […]

Is there parking for employees?

Benefits Leave Facilities Payroll
policy

Andrew Ng
Retrieval Augmented Generation (RAG) example

3. Generate answer from the new

prompt with additional context
Use the following pieces of context
to answer the question at the end.

Parking Policy: All employees may

park on levels 1 and 2 of the office
lot. Use the entrance on Front St […]

Is there parking for employees?

Yes, employees can park on […] Link to full onboarding policy

Andrew Ng
Examples of RAG applications

Chat with PDF files

Chatting with an Accenture report using ChatPDF

Andrew Ng
Examples of RAG applications

Answer questions based on a website’s articles

Coursera Coach Snapchat Hubspot

Andrew Ng
 Examples of RAG applications

New form of web search

Microsoft/Bing Chat Google You.com

Andrew Ng
Big Idea: LLM as reasoning engine

• LLMs have a lot of general knowledge, but they don’t know

everything

• By providing relevant context in the prompt, we ask an LLM to

read a piece of text, then process it to get an answer

• We’re using it as a reasoning engine to process information,

rather than using it as a source of information

Andrew Ng
 Advanced
Technologies:
Beyond Prompting
Fine-tuning
Pretraining and Fine-tuning

Pretraining Fine-tuning
My favorite food is a bagel with cream cheese What a wonderful chocolate cake
The novel was thrilling

Input (A) Output (B) Input (A) Output (B)

My favorite food is a bagel What a

My favorite food is a bagel with What a wonderful

My favorite food is a bagel with cream What a wonderful chocolate

My favorite food is a bagel with cream cheese What a wonderful chocolate cake

Learns from 100Bs of words Learns from 1000s to 10,000s of words

Andrew Ng
Why fine-tune?

To carry out a task that isn’t easy to define in a prompt.

Example 1: Summarize in certain style or structure
Customer: Hi, my monitor won’t and reinserting? Customer tells agent
turn on. Customer: Ok, doing that
Agent: I’m sorry to hear that. now. Hmm, the image about a problem with
What model is it? appeared for a few seconds monitor.
Customer: It’s the 27-inch 4K. then disappeared.
Agent: Can you tell me the Agent: Ok, try jiggling the
model number? cable while it is in the socket.
Customer: Hang on one sec…I’m Customer: Oh, I see the
looking on the back…it’s the image coming and going.
MK401-27X. Agent: Ok, I think you have a
Agent: Ok, thank you. Can you defective cable. We will send
try unplugging the HDMI cable you a replacement.

Andrew Ng
Why fine-tune?

To carry out a task that isn’t easy to define in a prompt.

Example 1: Summarize in certain style or structure
Customer: Hi, my monitor won’t and reinserting? MK401-27KX was
turn on. Customer: Ok, doing that
Agent: I’m sorry to hear that. now. Hmm, the image reported as broken by
What model is it? appeared for a few seconds customer 5402. Cable
Customer: It’s the 27-inch 4K. then disappeared.
Agent: Can you tell me the Agent: Ok, try jiggling the
identified as problem.
model number? cable while it is in the socket. Replacement issued.
Customer: Hang on one sec…I’m Customer: Oh, I see the
looking on the back…it’s the image coming and going.
MK401-27X. Agent: Ok, I think you have a
Agent: Ok, thank you. Can you defective cable. We will send
try unplugging the HDMI cable you a replacement.

Andrew Ng
Why fine-tune?

To carry out a task that isn’t easy to define in a prompt.

Example 2: Mimicking a writing or speaking style
Write a speech about generative AI Write a speech about generative AI
in the voice of Andrew Ng in the voice of Andrew Ng

Hello everyone, I'm Andrew Ng, and Hey everyone, good to see all of you.
I'm here today to talk to you about So, what I want to do today, is share
generative AI. Generative AI is a type some thoughts with you about how
of artificial intelligence that can generative AI is going to change the
create new content […] way that your company […]

Pretrained model Fine-tuned model

Andrew Ng
Why fine-tune?

To help LLM gain specific knowledge.

Medical notes Pt c/o SOB, DOE. PE: RRR, JVD absent, CTAB.
EKG: NSR. Tx: F/u w/ PCP, STAT CXR, cont. PRN O2.

Extract from admission note for patient with shortness of breath

Andrew Ng
Why fine-tune?

To help LLM gain specific knowledge.

Legal documents Licensor grants to Licensee, per Section

2(a)(iii), a non-exclusive right to use the
intellectual property, contingent upon
compliance with fiduciary duties outlined in
Section 8, paragraphs 1-4, and payment as
specified in Schedule B, within 15 days hereof.

Andrew Ng
Why fine-tune?

To help LLM gain specific knowledge.

Financial documents Counterparties engaged in OTC derivatives

transactions must comply with margin
requirements pursuant to Regulation (EU)
No 648/2012, including initial and variation
margin calculations."

Andrew Ng
Why fine-tune?
Large model: Small model:
To get a smaller model to perform a task. 100B+ parameters 1B parameters

• Lower cost/latency to deploy

• Can run on mobile/laptop (edge devices)

Classify the following review as Classify the following review as Classify the following review as
having a positive or negative having a positive or negative having a positive or negative
sentiment: sentiment: sentiment:

Really enjoyed my vegetable The noodle soup was too salty The staff sang happy birthday to
biryani me!
Positive Negative Positive

Can work well with 500-1000 examples.

Andrew Ng
 Advanced
Technologies:
Beyond Prompting
Pretraining an LLM
When should you pretrain an LLM?
Many teams are pretraining general-purpose LLMs by learning from
internet text.
• May take $10s of millions, many months, huge amount of data

For building a specific application:

• Option of last resort
• Could help if have a highly specialized domain

Andrew Ng
 Advanced
Technologies:
Beyond Prompting
Choosing a model
Model size

1B parameters: Pattern matching and basic knowledge Restaurant

of the world. review sentiment

10B parameters: Greater world knowledge. Can follow basic Food order
instructions. chatbot

100B+ parameters: Rich world knowledge. Complex reasoning. Brainstorming

partner

Andrew Ng
Closed or open source?

Closed-source models Open-source models

(Cloud programming interface)

• Easy to use in applications • Full control over model

• More large/powerful models • Can run on your own
• Relatively inexpensive device (on-prem, PC, etc.)
• Some risk of vendor lock-in • Full control over data
privacy/access

Andrew Ng
 Advanced
Technologies:
Beyond Prompting
How LLMs follow
instructions: Instruction
tuning and RLHF (optional)
How do chat systems learn to follow instructions?

Pretraining
My favorite food is a bagel with cream cheese What is the capital of France?

Input (A) Output (B) What is the capital of Germany?

Where is Mumbai?
My favorite food is a bagel
Is Mt Fuji or Mt Kilimanjaro taller?
My favorite food is a bagel with […]
My favorite food is a bagel with cream
My favorite food is a bagel with cream cheese

Andrew Ng
How do chat systems learn to follow instructions?

What is the capital of South Korea? Write a Haiku poem about Japan’s
cherry blossoms.
The capital of South Korea is Seoul.
Pink blooms fill the air
Japan's cherry blossoms dance,
Help me brainstorm some fun Spring's beauty unfolds.
museums to visit in Bogotá.

Sure, here are some suggestions: Tell me how to break into Fort Knox.
[…]
I can’t assist with that.

Andrew Ng
How do chat systems learn to follow instructions?

Fine-tuning Help me brainstorm some fun

museums to visit in Bogatá.

Sure, here are some suggestions:

[…]
Input (A) Output (B)
Help me brainstorm some fun museums to visit in Bogatá. Sure,
Help me brainstorm some fun museums to visit in Bogatá. Sure, here
Help me brainstorm some fun museums to visit in Bogatá. Sure, here are
Help me brainstorm some fun museums to visit in Bogatá. Sure, here are some

… …
Help me brainstorm some fun museums to visit in Bogatá. Sure, here are some suggestions:

Andrew Ng
Reinforcement learning from human feedback
(RLHF)
Input (A) Output (B)
Helpful, Honest, Harmless
Score
Response (reward)
Step 1: Train an answer quality
(reward) model I’m happy to help! Here are some 5
steps to follow […]
Prompt: Just try your best! 3
Advise me on how to apply for
a job. It’s hopeless – why bother? 1

Step 2: Have LLM generate a lot of answers. Further train it to

generate more responses that get high scores.

Andrew Ng
 Advanced
Technologies:
Beyond Prompting
Tool use and agents
(optional)
Tool use for food order taking

Send me a burger!

Ok, it’s on the way!

Andrew Ng
Tool use for food order taking
Passed to restaurant ordering system:
Send me a burger!
You have ordered:Item: Burger
ORDER(Burger, 9876, 1 burger UserID: 9876
1234 My Street) for delivery to: Address: 1234 My Street
User message: 1234 My Street
Ok, it’s on the way Displayed to user as response
Is this correct? (If yes, your credit
card will be charged)
Ok, it’s on the way!
Yes No

Andrew Ng
Tools for reasoning
How much would I have after 8 LLMs are not great at precise math
years if I deposit $100 into a
bank account that pays 5%
interest?

After compounding $100 at 5%

interest for 8 years, you would
have $147.04 dollars.
Incorrect value

Andrew Ng
Tools for reasoning
How much would I have after 8
years if I deposit $100 into a
bank account that pays 5%
interest?

After compounding $100 at 5% External calculator program

interest for 8 years, you would
have CALCULATOR(100 * 1.05^8)
100 * 1.05^8 = 147.74
dollars.

After compounding $100 at 5%

interest for 8 years, you would
have 147.74 dollars.

Andrew Ng
Agents
• Use LLM to choose and carry out complex sequences of actions
• Cutting edge area of AI research

Help me research BetterBurgers SEARCH(”BetterBurgers Competitors”)

top competitors

Here are steps I need to carry VISIT(http://www.fastburger.com)

out to research competitors:
1. Search top competitors VISIT(http://www.burgerworld.com)
2. Visit web site of each
competitor
…
3. For each competitor, write Summarize the following text:
summary based on homepage At Fast Burger, we pride ourselves on the
content speed of our delivery …

Andrew Ng