today we are going to build end to

endend llm project called cold email

generator using Lama 3.1 open source llm

we will also use chrom ADB Vector store

Lang chain and streamlet let us first

understand the problem that this tool is

solving there are software services

companies like TCS infosis I also have

this company at lck Technologies and

these companies help their clients build

software clients are like Nike JP Morgan

Kroger there are a lot of businesses

throughout the globe who want to build

software and these companies have a huge

staff of software engineers and they

provide their services to these clients

now there's a lot of competition in this

market and many times the sales teams

from these companies what they do is

they use variety of marketing techniques

to buid and to acquire projects one of

the techniques is C email so a person

from from software Services Company such

as attick will send a cold email to a

potential client now how do they know if

these clients have a specific

requirement one of the techniques that

they use is they will go to their

potential clients job portal so let's

say I'm on Nike's job portal where they

have this requirement for a software

engineer in AI ml area and you will look

into all the skills Etc that they need

and then what attick will do or CCS will

do is send a cold email saying that hey

it looks like you need a software

engineer in AIML domain and we have uh

these people so instead of hiring

full-time maybe you can hire people from

us because uh many times these uh

companies operate in countries like

India Indonesia where the cost is low

and also clients like Nike don't have to

hire full-time employees you know if

they can hire them on contract based so

there are several benefits of using this

model so at will send this kind of C

email to Nike and many times this cold

emailing techniques work now a sales

representative or bde Business

Development executive at at Lake or TCS

will have to form a nice email which is

relevant to the job post and one of the

things that we can use um is llm using

llm we can generate a cold email so

let's say you build this kind of tool

where you give a job posting okay and

when you submit it will go to that job

post it will figure out what kind of

skills are needed and based on that it

will write an email which is relevant so

let's say in that job post uh they

needed a person who has machine learning

with python and devops knowledge see

these two and the email will exactly uh

talk about that profile that skills you

will also give this kind kind of

portfolio so this will be like

at.com python at lick.com devops you

will give that portfolio portfolio means

the past projects that you have done in

that area in terms of technical

architecture we will go to that career

page extract text from that page using

Lang chain framework then we will use

llm Lama 3.1 to extract uh job role job

skills description in a Json format then

we give that to chroma DB which is a

vector database in that Vector database

we have previously stored the skills and

the relevant portfolios for at company

okay so let's say I have stored the

skills for example for python this is my

portfolio

www.com okay SL

python so these kind of links are

already stored we will look into it more

uh as we uh proceed in this particular

video uh you'll know more so it will

basically find relevant information for

this uh skill set so if you're talking

about react it will fetch you uh

portfolio links related to react and

then it will give this portfolio link as

well as the job description okay this

Json it will give these two things to

llm and that will generate this kind of

nice email you all probably know that

Lama 3.1 is a super amazing open-source

tool that you can use it for free many

people download this model locally on

their computer and run their inference

many people also use tools such as olama

and this olama tool will make certain

things easier for you you can say o Lama

pull Lama 3.1 o Lama run Lama 3.1 that

way it makes things easier for you now

we are not going to use locally

downloaded model or maybe use AMA

because when I previously used AMA the

inference was very slow to generate a

response it was taking around 5 minutes

so what we will do is use something

called Gro so Gro is a platform which

allows you to run Lama

3.1 in cloud and the inference is very

very fast because they use something

called lpu language processing unit that

makes the inference which means when you

ask a question the response will come

back up very fast so go to console.

gro.com either create a login using

email or just log in with Google I have

already logged in I am on a playground

you can type uh you know messages like

first person to walk on the moon was and

you

can submit see see you see it was so

fast and it is using uh llama 38b 8192

model they support other models too such

as mixl whisper large you can just look

into it they keep on adding new models

we are going to use Lama 370 billion for

our project and for that you need API

key so you will click here API key

create an API key okay so let's say my

project whatever just give your project

name submit and then you have to copy

this API key and save it somewhere this

is like a pass password it is your own

individual API key that you should not

be sharing with others I'm going to

delete this key by the way so that

should be okay once this is

generated you will uh create a folder so

I have created this C code code email

generator tool so I have created M

folder and then I uh went to that folder

and just ran python um let me just show

you so I ran python M notebook or

jupyter

notebook and here I'm going to create a

new notebook I'll probably call it

tutorial

Gro I want to test it okay

so chat grock is a class which Lang

chain framework offers you might be

aware that Lang chain is a framework

which makes building llm application

easier it's a open source module so you

just do p install Lang chain install it

and then uh you also install pip install

Lang chain Gro so first you install pip

install Lang chain the P install Lang

chain Gro to install that particular

module and then um I'm just going to

copy paste this code

here you import chat grock class from

this uh module and then uh you can use

mixl whatever but I'm going to use llama

3.1 okay so let me use that and I have

put my API once again I'm going to

delete this um but I'm just showing you

for demonstration purpose and simply you

call llm do invoke and you ask the

question this is my question you store

that in response variable and then

response.

content see it was super fast it went to

Gro Cloud so you need internet

connection OB obviously it asks this

question to Lama 3.1 70b model and

receive the response let me take a

moment to talk about today's sponsor or

firu is your pandas program running slow

you can make it faster by installing

fire du pip install firu and then use

the import hook utility where you supply

dhm option while running your Python

program no code changes required and

your program will run super fast fireex

is a compiler as a cated data frame

Library fully compatible with Panda's

API it can utilize multiple cores on

your computer and also it has some

optimization strategies which will make

your program faster it not only helps

you do your data analysis in more

efficient way it will also help

environment by reducing CO2 emissions

because your compute performance is

optimized check video description below

for fire duux documentation okay let's

go back to our video now as a Next Step

step we need to set up a vector database

and we are going to use chroma DB there

are other Solutions such as Pine con

milver Etc but chroma DB is open source

very lightweight so we will use that in

case you don't know what exactly is a

vector database then go to YouTube

search for code Basics Vector database

watch this uh simple 6 minute video it

will explain you what exactly it is all

right so I'm on chrom ADB website I will

go go to documentation and here to

install chrom ADB simply run pip install

chroma DB in your command prompt so let

me go to my command prompt clear this

and run this I have already installed so

it's not going to do anything after

installation you will go to getting

started guide and you will write simple

code to uh create a sample collection so

here I will create a new Python 3

notebook call it tutorial chroma DB and

here import chroma DB do client so this

is my

client and then client do create

collection so my collection name is

whatever my collection so this is a

database in database I'm creating a

collection it's like a table where I can

insert my records so I will

call it a collection control enter

execute it then to add documents

documents is nothing but the records or

data records that you're adding so

collection. add and

here

documents okay now these documents are

usually big text blocks in our case

since we're testing this thing we will

just provide a simple sentence so we

have provided some simple sentences and

then

IDs is equal to

id1 id2 I will explain you in few

seconds why we are not using traditional

database why we are using this uh

special Vector database so control enter

so documents are added now uh you can do

something called

collection.

getet and these

are all the documents you can see this

way you can get all the documents you

can also get an individual document by

ID you can say collection doget ID is

equal to this and it will give you only

ID one now here is the most interesting

part why we don't use traditional like

MySQL type of database so here you can

make a

query in that you can say query text is

equal to uh you can say query is about

CH B now you might be wondering what

exactly is this Cho B is relevant to

Delhi okay in New Delhi which is the

capital of India people eat chur so I'm

not searching a document by saying New

York or delh I'm searching a document by

providing a food item that is popular in

delhi now that's when this Vector

database will come in play it will be

able bble to do semantic search so it

will be able to tie that query to Delhi

the other argument it has is number of

results okay and this is

results so look at that the first

document that I got is Delhi and this is

the distance so the distance of this

query and this Deli document is 1.5

whereas the this query and New York

document is 1.8 which means New York is

1.8 km away whereas Delhi is

1.55 it's not actually kilometer but

it's a distance ukan distance so 1.55

unit away it is more similar if I say

query is about Pisa then see New York

comes first not only that if I say quer

is about Brooklyn Bridge then New York

comes first so you can see it it is

doing a meaning based semantic search as

a human if I ask you okay this is my

query quer is about Brooklyn Bridge

which document is more near or more

related you will obviously say New York

because Brooklyn Bridge is in New York

you can also say Okay query is

about India C Delhi query is about air

pollution I know it it hurts but air

pollution in Delhi is higher see you can

say I have queries about Big Apple now

Big Apple is New York but you can see

that the words are not exactly matching

but it is doing the semantic search now

to delete all the document douments you

can simply say collection. delete and

then you know this way it can delete all

the documents you can also add documents

with metadata so I'm just copy pasting

same code the additional thing I have is

metadata which is for New York I have a

Wikipedia page for New York okay so we

can add that and when you ask the same

query it not only returns Delhi in the

metadata it will also return the URL or

the Wikipedia link of Delhi this is

useful many times when you are building

any llm Tool uh llm will generate an

answer but you want to know which source

it is referring to from where it got

that answer so having this metadata will

be useful we will see more about this as

we code more in this project I created

one more notebook called email generator

and there I have written the code for

basic llm now I want to go to the

website so let's say Nike has this

particular job posting I want to scrap

the data from this website

first if you look at the Lang chain

documentation it has this class called

Web B loader what this class will do is

it will uh accept the URL of that job

posting and it will extract data from

that so it's basically web scraping so

let me just uh copy this code

here and I will copy paste this

particular URL and you can then call

loader.

load.

pop. Page

content and that will be your page data

and I will

print page data now it's doing web

scraping so it will take some time but

you can see that it went to this website

the data will be in HTML format

internally it went through that HTML

structure and extracted all this key

information see apply for software

Engineers search job skip navigation I

mean you will see some search job skip

navigation you you'll see some

unnecessary details as well now we will

uh write a prompt for our llm for which

we will import this prompt template

class let me first do this thing in chat

GPD so I will just U you know copy paste

some small portion and I will say that

uh I will give you a job

posting from that

extract the skills role and description

in a Json format okay and you will say

job

posting and you will just copy paste the

whole thing see I just copy Pate the

whole thing when you hit enter chat GPT

will give you this kind of nice uh Json

object so this is the object that we

want to retrieve now you are seeing this

thing here is the extracted information

in Json format this is called Preamble

so if you don't want that

right I I just want the Json object I

don't want here is the extracted

information so let me go here edit

this no

Preamble and hit send no Preamble means

don't give me that initial text see it

just gave me a Json object so let me uh

import this particular class here then

create a prompt template from template

and we'll store this into a variable

called prompt extract because we are

using uh this for extraction purpose we

will later on use it for email writing

purpose as well and I did some iteration

sometimes you don't get your prompt

right in a very first shot so after some

iterations I created this nice looking

prompt where I'm saying that scrap text

from website this is my page data

instruction the scrap text is from CER

page whatever um extract it in Json

format these are the uh fields that I

need and no Preamble now we already have

this llm object so what we going to do

is use a pipe operation so prompt

extract let me I think close this

one prompt extract pipe llm and this

will form your chain okay so this is the

uh Lang chain chain basically you are

getting a prompt and you are passing it

to llm and here you can do chain extract

do invoke where the input

is Page data okay and this will

be um this page data here and I will

store it in a response object print

response. content control enter

see it extracted the skills the role

name experience so if you look at this

page right experience

experience see 2 plus years experience

this is the power of LM it can extract

uh the key information among the all the

Clutter and these are the skills which

are mentioned here see AWS Cloud devops

you can just verify those now this

content that we have is in a text format

so if you do

type see it is an st Str I want to

convert this text into a Json object for

which we can use Json parser so this is

the class that we are going to use from

Lang chain and uh we will say Json

parser and then Json parer do par what

do you want to pass rest.

content so then this becomes the Json

respon C now it is a dictionary object

so if you do type Json Json or

dictionary I mean they are same thing

see it's a dictionary now once this step

is over the next step is to prepare

chroma DB so I have this CSV

file where I have all the technologies

that my company is working on so atck is

working on all these companies okay TCS

infosis whatever that company is that

will have the capability of all the

Technologies and for each technology

they will have some portfolio portfolio

means okay these are the projects that

we have done in this particular

technology so if you go to at.com which

is a real company uh so we have done all

these projects for example uh let's say

moon tower tickets so if you go there

this is short of our portfolio okay here

it explains okay who was the client

which Tex Tech we used what was the

challenges solution and so on so if you

go to case studies here you'll see all

our clients and projects so this is

basically ATX portfolio similarly here

I'm using some dummy URL you you can

understand this is a learning project

but for each of the Technologies these

are the portfolio links and I need to

insert this this thing into chroma DB

then whenever there is a job posting it

should extract the skills from the job

and it will match one or multiple of

these Technologies and then it should

retrieve this portfolio URLs which I

will use in my email all right so I have

U put that CSV file here and I'm going

to just import that here so

import Panda as PD and I will create a

data frame so read CSV my portfolio. CSV

I think that's the name and see I have

this in my data frame now you will

iterate over your data frame and you

will insert those records one by one

into chroma DB so since we have written

this code before I'm going to save you

some time imported chroma DB instead of

client see in our sample notebook we use

client correct when you use client it

will create the chroma DB in memory

but I will use persistent client so it

will create DB on a disk so in your

current folder see right now in my

current folder there is no folder for

chrom ADB when I execute this code it

will create that database store it in a

folder so that next time when I want to

retrieve my data I have it on the

disk and here I'm saying that if not

collection. count means if let's say

collection doesn't have any count if it

is being created for the first time then

you iterate through all your data frame

rows folks this is very very easy simple

python code you just Hing over uh data

frame rows and for each of the rows you

are adding collection do add you are

adding a document metadata is your link

okay metadata is this links and text tag

is your main document so control enter

and that worked okay so if you look at

now your folder you see this Vector

store uh folder is created so internally

it is storing the data here you don't

need to look at these files it will be

gibberish for you but this shows that

when you use a persistent client it will

create a folder and store your records

inside that folder now I will do simple

collection query I will say Okay I I

need a person with experience in Python

experience in react and when I do that

see for the first one experience in

Python it is bringing these two which

are python related portfolio

for the second one it is bringing react

portfolio react native Port portfolio if

I do react native I think it just still

P see react native came first and then

react portfolio so for each of these

experiences I requested two results and

I requested metadata so which was the

portfolio link now my job is nothing but

Json response and in my job the

skills I have is this so what if we pass

all these skills so for example if I

pass these skills

here just for a demo

purpose then see um for my Nike position

it will give me for strong computer

science fundamental is giving this for

Java python goang it is see python it

doesn't have goang then for devops it is

is giving me my deos portfolio so it is

working okay all right so my links are

stored here now the way I created The

Prompt template for extracting the

content out of job posting I will create

another prompt template for forming an

email so once again I'm not going to

type the whole thing because it it will

waste your time so here my prompt

template says that this is my job

description whatever we got from Nike

website and the the instruction is you

are moan a business development

executive at attick atck is an AI and

software consulting company whatever uh

we have expertise whatever right like

usual stuff your job is to write a cold

email to the client see this is where we

are giving the clean instruction to LNM

saying that write a cold email to client

regarding job mention above and add the

portfolio links and whenever you see

something in a curly bracket that is

something we will give as an argument to

our prom

template so this link list okay this is

not a data structure link list but this

is uh this particular links that you

will give in a prom

template so you're giving two parameters

job description and the link list and

after that again usual copy paste stuff

so you are creating a chain of prompt

email and llm and you're invoking Your

Chain by passing these two parameters

okay job description and Link uh list uh

job description is nothing but the job

that we got right

so job if you do

job it is the Json object the dictionary

and the link list is these links so when

you control enter look at that that's my

final output it

generated very nice cold IM saying that

okay I came across your job description

blah blah blah we have expertise in AIML

software development and you can refer

to our portfolio our devops portfolio is

this our machine learning portfolio is

this so I will request you to download

this code and I hope you are coding

along with me and just look at the

description also folks very important

thing after 6 month don't come and say

that this code is not working which code

this code because Nike might take this

job posting away this is very temporary

so what you need to do is let a smart

work don't do dumb work okay don't just

download my notebook and run and say oh

it's not working and some people just

post it on YouTube comment oh it's not

working you need to provide a job

posting which is active which is

something you go copy paste in browser

you are able to access so make sure you

are changing this to some active job

posting which is available when you are

running this code so far we have built

the raw building blocks it's like we are

constructing a home and we have built

little pieces and now we will connect

all the pieces and create a project

structure in my folder I will create

another folder called app and in this

app folder I will create a proper

project structure we use um IPython

notebook or jupyter notebook for

experimentation to create our base to

create our code blocks that we can use

in the final project we have completed

that step now is the time to create a

proper project structure so I will open

this in my py Cham so let's go here and

what is the name of our folder well

called email generator right this guy

okay this

window

perfect here I will add a new file I

will call it main.py in that I will

create another folder called resource

remember we had this uh my portfolio.

CSV file we put that into resource so

any times you have text file Excel file

images it's better to put that in a

separate folder called resource now we

will uh create a basic UI using a

streamlit so if you not installed it

just go to your

terminal and run pip install streamlit

streamlet is a framework that allows you

to quickly build UI data scientist

Engineers use this to build very quick

prototypes so import okay I will

configure my python interpreter to be

this you can use Virtual environment as

well uh import streamlet as St then st.

title and I'm saying that URL input is

is uh this

so st. title Cod email generator then I

adding

the text input I'm saying enter a URL

this is my default value that I'm adding

okay then there is a button St do button

submit and if submit do button then st.

code now you can refer to stream lists

document ation it's a very easy

straightforward framework and what I do

is I take help of chat GPT I give my

requirement and usually chat GPT will

give you a very um nice base which will

uh save you a lot of time and then you

can make edits into that base

code so

hello um hiring

manager I'm from at like I'm just

putting some dummy code for now and then

language is a

markdown okay so let's run this code

first so here you can say streamlit run

then we have app folder correct and then

main.py so when you run this you get

this kind of UI when you hit submit

hello hiring manager I'm from blah blah

blah so my dumy UI with dummy response

is ready now I can write rest of my

classes so the first class that I want

to add is called chains. piy okay and in

that uh I will import first of all all

the necessary modules and then I will

use this load. environment now what is

this exactly so you can read the

documentation of do environment online

so just say python

load. EnV we have to install this okay

so pep install python. envance

and after you install it in your root

folder you can have a file called do EnV

so let me create that file so here I can

create a

file call it do EnV in that you can put

your API key it is a good idea to keep

the API key separate from your code it

is never a good idea to put it inside

the code because you can somehow protect

this okay so I have put my API key you

need to use your API key folks remember

so when I check in this code to GitHub

I'm going to remove it so once again

don't tell me that it is not working

it's not working because you did not put

your API ke and whenever uh you say

load. environment if you want to receive

any response you can do it or you can

just run it so what this method will do

is it will find EnV file

and it will set this as your environment

variable once your environment variable

is set you can use os. get environment

Grog API key to retrieve that particular

API key so if you want to taste anything

uh you can say if

name is equal
to main okay you can say print let's say

I just want to test this and when you

right click and run

it see it is printing that particular

API key now I'm going to create a class

called

chain and in Python this is how you

write the

Constructor so self. llm is nothing but

the chat grock object remember we

created that in our notebook so I'll go

to that notebook copy paste that thing

my model is Lama 3.1 70b is 70 billion

parameters and my API key is whatever I

got from the environment variable and

then I will write these two functions

these two functions are something that

we already wrote in our jupyter notebook

we wrote this uh basic code and now I

have converted that into a function so

if you look at it it's the same code uh

prompt

extract then you're using Json parer to

pass the response and you are just

returning that response back and another

uh chain that we had was chain email

okay so I had that raw code in my

notebook I took that code wrote this

function and my class chain is now

complete similarly I will create another

class called

portfolio so

portfolio. pi and here I will read that

CSV file and I will do the chroma DB

setup so just to save your learning time

I'm going to copy paste from my code so

here um you are initializing this

portfolio class with a file path which

is app resourc my portfolio. CSV so see

app resource my portfolio. CSV then you

are creating a data frame the same way

that we did it before and you are

creating chroma DB persistent client and

creating this particular collection and

when you call load portfolio it will go

through that data frame here and it will

uh add those records into collection and

when you do query links it will just

query I will also add utils.py where I'm

going to add a function which will just

clean some tag so if there are any HTML

tags Etc your you are getting from a web

page it will clean it up in our case

when we were doing that notebook we did

not have to do this because it was

straightforward but it is good to have

this code because see all these HTTP

tags all these special characters um

multiple spaces leading and trailing

wide spaces it will remove and it will

kind of massage your text and make it

good look and then I went to my main.py

file I put everything in a function so

this was not in a function so I created

a nice function called create streamlit

app and when submit button is hit I'm

using that same web based loader that we

used in that notebook and I'm calling

clean text this clean text function to

clean any unnecessary garbage then

portfolio. load portfolio will create my

chroma DB and llm do extract jobs

function will um call this function so

so it will create my Json objects for

the jobs and for every job so see here

let's say one one web page has multiple

jobs then I'm taking care of that

possibility for every job you will then

do portfolio. query links and then call

write email function so in my portfolio

once again in Chains actually my right

email is that same exit code and in the

main function you are creating chain and

portfolio and then you are just simply

calling this particular function now

here my app is already running so what I

need to do is go here and click on rerun

so when you rerun it will kind of reload

that code and rerun it and when you hit

submit it is now going to that Nike's

job

posting it is extracting the skills Etc

and it has now created this cold email

so a business development executive at

attick will just simply copy this and

send it to a hiring manager in Nike

saying that if you don't want to hire a

person full-time we are a service

provider software Services AI services

and we will uh give you the staff at a

cheaper cost um and you you can just

give your sales speech there this is the

tool folks that companies can use for uh

real okay this is the real tool that we

have built atle is my company like we

have been in business for four five

years we have more than 65 employees and

we use this kind of techniques so this

tool is useful in my own company so you

can imagine that this project was not

some toy project this is a real project

that any software company or any service

based companies can use for their uh

sales purpose all right that's it I'm

going to add requirement. txt file here

as well so make sure you install all the

necessary

dependencies and check the video

description all the code is given if you

like this video give it a thumbs up

share it with your friends if you have

any question there is a comment box

below

[Music]