FOA Project Report

Basic Conversational Chatbot – Robo

Submitted By
Vaishnavi Balodhi (04)
Aastha Lokhande (30)
Poornima Menon (31)
Divya Sharma (47)
Submitted on
17th of October, 2024.

1
Introduction
Chatbots are software programs designed to
simulate human conversation through voice or text
interactions. They are increasingly used across
various industries for customer service,
automation, and user engagement. This project
focuses on creating a basic chatbot capable of
holding a simple conversation with users using
natural language processing (NLP) techniques.
Our chatbot, "Robo," responds to users based on
their input by analyzing similarities between their
queries and the given dataset of conversational
text. We implemented the chatbot using Python
and several libraries for text processing and
vectorization.

2
Project Overview
Technology Stack

 Programming Language: Python

 Libraries:
o NLTK (Natural Language Toolkit) for

tokenization, text preprocessing, and

lemmatization.
o NumPy for numerical operations.

o Sklearn for vectorization (TF-IDF) and

similarity measures (cosine similarity).

 Tools:
o TF-IDF Vectorization to represent text in

numerical format.
o Cosine Similarity to compare the

similarity between user input and the

dataset.
Functionality Overview

The chatbot processes user inputs by tokenizing

and normalizing the text, using a pre-trained TF-
IDF model to vectorize the input. The chatbot then
finds the most relevant response by calculating
cosine similarity between the user’s input and
sentences from the dataset.
If the input matches any greeting keywords, the
chatbot responds with a predefined greeting. For
other inputs, it uses similarity measures to

3
generate an appropriate response based on the
dataset.
Dataset
The chatbot is built using a dataset that consists of
conversational dialogues covering everyday topics
like greetings, the weather, school, and social
activities. The dataset was formatted as plain text,
with dialogues separated by sentences.
Sample Dataset Content:
hi, how are you doing?
i'm fine. how about yourself?
i'm pretty good. thanks for asking.
The dataset includes a variety of casual
conversations that simulate real-life scenarios,
allowing the chatbot to provide relevant responses
based on user input(dialogs).

4
Implementation Details
Code Breakdown

 Importing Libraries:
We used Python's nltk and sklearn libraries for
natural language processing and machine
learning. The code begins by importing
necessary packages such as TfidfVectorizer for
vectorization and cosine_similarity for
comparing input text to the dataset.
 Tokenization and Preprocessing:
The text is split into sentences using NLTK's
sent_tokenize() function and into words using
word_tokenize(). We applied lemmatization to
reduce words to their base forms using
WordNetLemmatizer. Special characters and
punctuation were removed to ensure clean
input data.
 Lemmatization and Keyword Matching:
A function LemNormalize() was created to
preprocess user input by tokenizing,
lowercasing, and lemmatizing it. We defined a
set of greeting keywords (e.g., "hello", "hi",
"hey") and set the bot to respond with
predefined replies when these words were
detected.

5
  Response Generation:
For non-greeting inputs, the chatbot appends
the user’s input to the list of tokenized
sentences. Using TfidfVectorizer, the chatbot
converts the input into numerical vectors and
calculates cosine similarity between the input
and the dataset sentences. The chatbot then
responds with the sentence having the highest
similarity score.
Main Features

 Handling Greetings:
The chatbot recognizes greetings like "hello" or
"hi" and responds with a random greeting from
a list of predefined options.
 Conversation Simulation:
For more complex inputs, the chatbot uses
cosine similarity to retrieve the most
contextually relevant response based on the
dataset.

6
Challenges Faced
Some of the challenges encountered during the
project included:
 Dataset Preprocessing:
Cleaning and tokenizing the text dataset
required careful handling of punctuation and
special characters to ensure accurate text
comparison.
 Response Generation Accuracy:
The chatbot relies on cosine similarity and TF-
IDF scores, which work well for simple
conversation but may not always generate
responses that sound natural for more complex
inputs. Future work could involve integrating
more advanced NLP models, such as
transformer-based models like GPT.
 Error Handling:
When the chatbot does not find a relevant
match for the user input, it returns a default
response ("I am sorry! I don't understand
you"). Improving this fallback response
mechanism can be an area for further
development.

7
Conclusion
In this project, we successfully built a basic
conversational chatbot that simulates human-like
conversation using natural language processing
techniques. Although the chatbot is basic, it
demonstrates the foundational principles of how
modern chatbots work.
Future Enhancements:
 Expanding the dataset to include more diverse
conversations.
 Incorporating advanced NLP techniques like
deep learning for improved response
generation.
 Enhancing the error handling to make the
chatbot more interactive when it fails to
understand user input.

8
References
1.NLTK Documentation:
NLTK is a leading platform for building Python
programs to work with human language data. It
provides tools for tokenizing, parsing,
classification, and semantic reasoning.
NLTK Documentation link- https://www.nltk.org/
2.Scikit-learn Documentation:
Scikit-learn is a machine learning library in
Python, providing efficient tools for data mining
and data analysis. It was used in this project
for vectorization (TF-IDF) and calculating cosine
similarity between text samples.
Scikit-learn Documentation link- https://scikit-
learn.org/stable/
3.TF-IDF Explained:
TF-IDF (Term Frequency-Inverse Document
Frequency) is a numerical statistic that is
intended to reflect how important a word is to
a document in a collection or corpus. It is often
used as a weighting factor in information

9
 retrieval and text mining.
TF-IDF Article-https://en.wikipedia.org/wiki/Tf
%E2%80%93idf
4.Cosine Similarity Explained:
Cosine similarity is a metric used to determine
how similar two vectors are by measuring the
cosine of the angle between them. It is widely
used in natural language processing to
compare documents or sentences.
Cosine Similarity
Article-https://en.wikipedia.org/wiki/Cosine_simi
larity

10