A

Synopsis
on
(LLM Based AI agents)
Submitted in partial fulfilment of the requirements
for the award of the degree of

Bachelor of Technology
in
Computer Science and Design
By
Aman Dogra (2100970310024)
Prabal Sharma (2100970310110)
Abhinav Tripathi (2100970310003)
Semester – VII
Under the Supervision of
(Dr. Sambit Satpathy)

Galgotias College of Engineering & Technology

Greater Noida 201306

Affiliated to

Dr. APJ Abdul Kalam Technical University, Lucknow

(Session: 2024-2025)
ABSTRACT
In an era where digital tasks are increasingly complex and time-consuming, the integration of AI-driven solutions
has become essential. This project aims to explore the efficacy of large language model (LLM)-based AI agents
in automating two critical tasks: job search and automated email responses. Initially, the project will focus on
creating AI agents capable of searching for relevant job postings based on a user’s resume and generating
automated email responses after analysing received emails. By leveraging tools like CrewAI for agent
orchestration and open-source LLMs such as LLaMA-3 70B, this research will compare the performance, speed,
and accuracy of AI agents versus traditional methods. The findings are expected to demonstrate significant
improvements in automation, showcasing the potential for broader application in various digital environments.

Title: "Enhancing Task Automation with LLM-based AI Agents: A Comparative Study on Job Search and Email
Response Systems"

1. INTRODUCTION
In the rapidly evolving landscape of artificial intelligence, the ability to automate complex tasks through natural
language commands has emerged as a powerful tool. The development of Large Language Models (LLMs) has
opened up new possibilities for enhancing the efficiency and effectiveness of various digital operations. The
project at hand is deeply inspired by the foundational concepts introduced in the influential "World of Bits" paper.
The paper highlights the potential of AI agents to interact with the web and perform tasks traditionally handled by
humans, thereby revolutionizing digital task automation.

The primary objective of this research project is to explore the application of LLM-based AI agents in automating
two specific tasks: job search and email response generation. These tasks were selected due to their relevance in
professional and personal domains, where efficiency and accuracy are crucial. By integrating AI agents capable
of interpreting and acting on natural language commands, this project aims to demonstrate the transformative
potential of LLMs in automating everyday digital tasks.

The inspiration for this project stems from the "World of Bits" paper, which emphasizes the importance of
developing AI agents that can interact with various digital platforms through natural language. The paper envisions
a world where AI agents can browse the web, perform searches, fill out forms, and navigate websites—all through
human-like interaction. This concept resonates with the growing need for more intuitive and powerful automation
tools, particularly in areas like job searching and email management, where human effort can be significantly
reduced through intelligent automation.

Building on this foundation, the project seeks to investigate the feasibility and effectiveness of implementing
LLM-based AI agents to perform tasks that require comprehension, decision-making, and interaction with digital
environments. The focus on job search and email response generation is motivated by the potential impact these
tools could have on professional efficiency. For instance, job seekers spend a considerable amount of time
searching for relevant opportunities and crafting personalized responses to emails. Automating these processes
with AI could not only save time but also improve the quality of results by leveraging advanced language models.
How an Agent works?

Fig. 1.1

Role of Large Language Model (LLM) in improving Agentic workflow:

Fig.1.2
 2. LITERATURE SURVEY
The integration of AI agents in automating complex digital tasks has become an increasingly prominent area of
research, driven by advances in large language models (LLMs) and natural language processing (NLP). This
literature survey provides an in-depth exploration of the theoretical and practical underpinnings of this project,
which focuses on leveraging LLM-based AI agents for job search and email response automation. The survey
delves into key papers, methodologies, and technologies that have shaped this domain, with a particular emphasis
on the "World of Bits" paper by Karpathy et al., LLM advancements, AI agent architectures, and the current state
of task-specific automation.

2.1 . The Foundation: "World of Bits"

The "World of Bits" paper laid the groundwork for the vision of AI agents that can interact with the web as humans
do, using natural language commands to navigate and manipulate digital environments. This work was pioneering
in its approach to creating AI systems capable of performing tasks traditionally requiring human cognitive
abilities, such as browsing the web, filling out forms, and responding to emails.

In their study, Karpathy et al. explored the potential of deep reinforcement learning combined with imitation
learning to train agents that can generalize across a wide range of web tasks. They introduced the concept of
“World of Bits,” a framework where the web is treated as an environment that agents can learn to interact with
through direct pixel-level input and output. The research demonstrated that AI agents could achieve a significant
level of competence in performing web-based tasks autonomously, which inspired further exploration into the
automation of various digital tasks using LLMs and other AI technologies.

This project's inspiration stems from the "World of Bits" concept, seeking to extend the idea by incorporating
LLMs to enhance the agents' ability to understand and generate natural language, thus enabling more sophisticated
and context-aware task automation.

2.2 . Advances in Large Language Models

The evolution of LLMs has been a critical factor in enabling AI agents to perform complex tasks. Models like
GPT-3, BERT, and their successors have demonstrated remarkable capabilities in understanding and generating
human-like text, making them suitable for a wide range of applications, from conversational agents to content
creation. The recent development of LLaMA-3, a 70-billion parameter model with enhanced context length and
fine-tuning capabilities, represents a significant advancement in this field.

LLMs have been particularly effective in tasks involving language understanding and generation, such as
summarization, translation, and question answering. The scalability of these models, coupled with their ability to
learn from vast amounts of text data, has enabled them to generalize across different domains and tasks. This
generalization is crucial for the AI agents in this project, as it allows them to apply their language skills to varied
tasks like job searching and email response.

Recent studies have also highlighted the importance of fine-tuning LLMs for specific tasks to improve their
performance. Techniques such as reinforcement learning from human feedback (RLHF) and prompt engineering
have been employed to guide LLMs in generating more accurate and contextually relevant outputs. These
approaches will be essential in this project, where the agents need to interpret resumes, match job descriptions,
and craft professional email responses.

2.3 . AI Agent Architectures and Tool Integration

The architecture of AI agents plays a pivotal role in determining their effectiveness in performing tasks. Early AI
agents, such as those developed for virtual assistants like Siri and Alexa, relied on rule-based systems and
predefined workflows to execute tasks. However, these systems lacked the flexibility and adaptability required
for more complex and dynamic environments.

With the advent of LLMs and the integration of multi-tool AI agent frameworks, such as CrewAI, the architecture
of AI agents has evolved significantly. CrewAI, for instance, allows agents to access a variety of tools, including
web search, calculators, and interpreters, enabling them to perform a broader range of tasks. This modular
approach to agent design enhances their versatility and effectiveness in real-world applications.
The ability to integrate external tools with LLMs is particularly relevant to this project. For instance, the job search
agent could use web scraping and APIs from platforms like LinkedIn and Google Jobs to gather job postings,
while the email response agent could utilize knowledge bases or search engines to perform research before
generating responses. The combination of LLMs with specialized tools allows for a more holistic approach to task
automation, where the agent can gather, process, and act on information in a contextually appropriate manner.

2.4 . Task-Specific Automation: Job Search and Email Response

The automation of job search and email response tasks has been a subject of interest for researchers and developers
alike, given their relevance in both personal and professional settings. Traditional approaches to job search
automation have typically involved the use of keyword matching algorithms and APIs to filter and recommend
job postings based on user input. While effective to some extent, these methods often lack the nuance and
understanding required to match job seekers with the most suitable opportunities.

Recent studies have explored the use of AI to enhance job matching systems. For example, AI-driven platforms
like Pymetrics and Eightfold AI have employed machine learning algorithms to analyse resumes and job
descriptions, identifying patterns and recommending jobs that align with a candidate’s skills and experience.
However, these systems are often limited by their reliance on structured data and predefined criteria.

By integrating LLMs into the job search process, this project aims to overcome these limitations. LLMs can
interpret unstructured data, such as the natural language content of resumes and job postings, allowing for a more
sophisticated matching process. Furthermore, LLMs can consider contextual factors, such as job requirements and
candidate preferences, resulting in more personalized and relevant job recommendations.

Similarly, email response automation has been an area of active research, particularly in the context of customer
service and personal productivity tools. Early systems, such as Gmail’s Smart Reply and Outlook’s Suggested
Replies, utilized template-based approaches to generate responses. While these systems have been widely adopted,
their limitations in handling complex or nuanced email content have been noted.

Recent advancements in NLP, particularly with the rise of LLMs, have enabled more sophisticated email response
systems. Research by Google AI and OpenAI has demonstrated that LLMs can generate contextually appropriate
and personalized email responses, significantly reducing the time and effort required for email management. The
use of LLMs in this project’s email response agent is expected to build on these developments, providing a more
robust and flexible solution for automating email communications.

2.5 . Comparative Analysis of LLMs

One of the core objectives of this project is to conduct a comparative analysis of different open-source LLMs in
the context of task-specific automation. While proprietary models like GPT-4 have demonstrated exceptional
performance, the growing interest in open-source LLMs has spurred research into their capabilities and
limitations.

The literature on LLM comparison is still emerging, but several studies have provided valuable insights. For
example, a recent paper by Meta AI compared the performance of LLaMA-3 with other state-of-the-art models,
highlighting its strengths in tasks requiring long-term context understanding and its efficiency in resource-
constrained environments. Similarly, research by EleutherAI has focused on the potential of models like GPT-
NeoX to perform complex NLP tasks with fewer parameters, making them more accessible for a broader range of
applications.

In this project, the comparative analysis will involve evaluating the performance of different LLMs in the specific
tasks of job search and email response automation. Metrics such as accuracy, speed, and resource consumption
will be considered, along with qualitative assessments of the generated outputs. The goal is to identify the most
suitable LLMs for these tasks, as well as to uncover potential trade-offs between different models.

2.6 . Challenges and Ethical Considerations

Despite the significant progress in AI-driven task automation, several challenges and ethical considerations
remain. One of the primary concerns is the potential for bias in AI agents, particularly in tasks like job matching,
where biased algorithms could perpetuate discrimination. Research has shown that LLMs can inadvertently learn
and amplify biases present in their training data, leading to biased outcomes.

Addressing these challenges requires careful consideration of the data used to train AI agents, as well as the
implementation of bias mitigation techniques. This project will prioritize the use of diverse and representative
data sources, along with ongoing monitoring of the agents’ outputs to identify and correct any biases that may
arise.

Another challenge is the potential for over-reliance on AI agents, leading to a loss of human oversight in critical
decision-making processes. While AI can significantly enhance efficiency, it is essential to ensure that human
users remain involved in the loop, particularly in tasks that require judgment and discretion. This project will
explore mechanisms for maintaining human oversight, such as providing users with the ability to review and edit
AI-generated outputs before finalizing them.

Finally, the ethical implications of using AI agents to automate tasks that involve personal data, such as resume
analysis and email management, must be carefully considered. Ensuring data privacy and security will be
paramount, and the project will adhere to best practices in data handling and encryption to protect users’
information.

The integration of LLM-based AI agents into digital task automation represents a significant advancement in the
field of artificial intelligence. This literature survey has explored the foundational concepts, key developments,
and current challenges in this domain, providing a comprehensive context for the project’s objectives. By building
on the principles outlined in the "World of Bits" paper and leveraging the latest advancements in LLMs and AI
agent architectures, this project aims to contribute valuable insights into the potential of AI-driven automation in
job search and email response systems. The findings from this research will have broader implications for the
future of AI agents in automating a wide range of digital tasks, ultimately enhancing productivity and efficiency
in various domains.

3. PROBLEM FORMULATION
In today’s digital age, the volume of information and the complexity of tasks that individuals and organizations
manage daily have grown exponentially. Job seekers face the daunting challenge of sifting through countless job
postings to find opportunities that align with their skills and career goals. Similarly, professionals spend significant
time managing email communications, often requiring thoughtful responses that are both timely and relevant.
These tasks, though routine, are time-consuming and prone to human error or oversight. Traditional automation
methods, while helpful, often fall short in handling the nuanced and context-dependent nature of these tasks.

The proposed research work addresses the need for more sophisticated, intelligent automation tools that can
interpret and execute tasks based on natural language commands. The introduction of large language model
(LLM)-based AI agents presents a transformative opportunity to automate these complex tasks with a higher
degree of accuracy and contextual understanding. By leveraging LLMs, these AI agents can process unstructured
data, understand user intent, and generate outputs that closely mimic human decision-making processes.

This research is significant as it explores the integration of LLM-based AI agents into everyday professional tasks,
specifically job searching and email response generation. These tasks are critical for both individuals and
businesses, directly impacting productivity, efficiency, and career success. By automating these processes, the
research aims to demonstrate how AI can alleviate the burden of routine tasks, allowing users to focus on more
strategic and creative endeavours.

Furthermore, the comparative analysis of different open-source LLMs within this research provides valuable
insights into the most effective models for specific tasks, contributing to the broader field of AI research. The
outcomes of this study will inform the development of more advanced and accessible AI tools, paving the way for
their adoption in various digital environments. Ultimately, this work seeks to enhance the quality and efficiency
of digital task automation, underscoring the potential of AI agents to revolutionize how we interact with and
manage information in the digital world.
 4. OBJECTIVES
The objectives of this research project are centred around leveraging LLM-based AI agents to enhance automation
in job searching and email management. These objectives aim to explore the potential of AI in reducing the time
and effort required for these tasks while ensuring accuracy and contextual relevance. The specific objectives of
this study are:

4.1.Develop and Evaluate LLM-Based AI Agents for Job Search Automation:

The primary objective is to design AI agents that can autonomously search for job postings online based
on a user’s resume. This involves integrating LLMs to understand and match the nuanced language of
resumes with relevant job descriptions, and then comparing the effectiveness of these AI-driven systems
against traditional keyword-based search methods.

4.2.Automate Email Categorization and Response Generation Using LLMs:

The second objective is to create AI agents capable of categorizing incoming emails and generating
appropriate responses. This includes developing a system where LLMs can analyze the content of emails,
perform necessary research, and draft contextually relevant replies. The goal is to streamline email
management, allowing users to focus on more strategic tasks.

4.3.Compare the Performance of Different Open-Source LLMs in AI Agent Tasks:

The final objective is to conduct a comparative analysis of various open-source LLMs, assessing their
performance in both job search automation and email response generation. This comparison will help
identify the most effective models for these specific tasks and provide insights into how different LLMs
can be optimized for real-world applications.

These objectives are aligned with the broader aim of demonstrating the capabilities of LLM-based AI agents in
automating complex, language-dependent tasks, ultimately contributing to advancements in AI-driven digital
tools.

5. METHODOLOGY
The research project will be carried out over the 7th and 8th semesters, following a structured plan to develop,
implement, and evaluate AI agents using LLMs for job search and email management.

5.1. Study Design

 We’ll start by designing the AI agents, selecting the open source LLMs, and planning how these agents
will automate tasks like job searching and email responses.

5.2. Study Settings

 Initial work will be done in a controlled online environment, like Google Colab, where we’ll build and
test early prototypes of the AI agents.

5.3. Sampling
 We'll use sample data, such as resumes and job postings available online, along with sample emails, to
test the AI agents.
5.4. Variables
 Key factors to measure will include how accurately the AI matches jobs to resumes, the relevance of AI-
generated email responses, and how quickly the AI agents perform these tasks.

5.5. Controls
 We’ll compare the AI-driven methods with traditional approaches (like keyword-based job searches) to
see how much better (or worse) the AI performs.

5.6. Data Collection

 We’ll gather data on how well the AI agents are doing their tasks, like how accurate and fast they are,
and keep records of their outputs.

5.7. Data Analysis

 We’ll analyse this data to see which LLMs work best and how effective the AI agents are at automating
the tasks compared to traditional methods.

5.8. Ethical Clearance

 We’ll ensure that all data used is anonymous or generated only for the purpose of the project, especially
since we’re dealing with potentially sensitive data like resumes and emails.

5.9. Implementation and Refinement

 We’ll improve the AI agents based on the initial tests, making them more effective and expanding their
capabilities.

5.10. Expanded Testing and Deployment

 The AI agents will be tested in more realistic settings, possibly using live data, to see how well they
perform in real-world situations.

5.11. Comparative Analysis

 We’ll compare different LLMs to find out which one works best for each specific task the AI agents are
doing.

5.12. Final Data Collection and Analysis

 More data will be collected from expanded testing, and we’ll analyze it to draw final conclusions about
the project’s success.

5.13. Final Report and Presentation

 Finally, we’ll compile our findings into a report and present the project to faculty and peers, showcasing
the potential of using LLM-based AI agents for these tasks.
REFERENCES

[1.] Karpathy, A., Johnson, J., & Fei-Fei, L. (2016). The World of Bits: An Open-Domain Platform for Web-
Based Autonomous Agents. arXiv preprint arXiv:1603.04802.

[2.] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017).
Attention is All You Need. Advances in Neural Information Processing Systems, 30.

[3.] Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., Zheng, R.,
Fan, X., Wang, X., Xiong, L., Zhou, Y., Wang, W., Jiang, C., Zou, Y., Liu, X., Yin, Z., Dou, S., Weng, R.,
Cheng, W., Zhang, Q., Qin, W., Zheng, Y., Qiu, X., Huang, X., & Gui, T. (2023). The Rise and Potential of
Large Language Model Based Agents: A Survey. arXiv:2309.07864v3

[4.] Durante, Z., Huang, Q., Wake, N., Gong, R., Park, J. S., Sarkar, B., Taori, R., Noda, Y., Terzopoulos,
D., Choi, Y., Ikeuchi, K., Vo, H., Fei-Fei, L., & Gao, J. (2024). AGENT AI: Surveying the Horizons of
Multimodal Interaction. arXiv:2401.03568v2