AI Legal Assistant

for IPC
2024 8th International Conference on Computational System and Information Technology for Sustainable Solutions (CSITSS) | 979-8-3315-0546-2/24/$31.00 ©2024 IEEE | DOI: 10.1109/CSITSS64042.2024.10817061

Ayush Dutta Kaushik Kumar Sarma Prof. Yamini Sahukar P.

Student, Dept. of AI&ML, BIT Student, Dept. of AI&ML, BIT Professor, Dept. of AI&ML, BIT
Bengaluru, India Bengaluru, India Bengaluru, India
[email protected] [email protected] [email protected]

Abstract—The legal framework is a critical component of contextually relevant responses, and RAG for retrieving
societal structure, yet its complexity often leaves individuals pertinent legal information. Streamlit is utilized to develop an
struggling to understand their rights and obligations. This paper interactive user interface that enhances user engagement and
introduces an advanced NLP-based chatbot designed to enhance accessibility. The aim is to provide a reliable, user-friendly
legal accessibility and comprehension, focusing on the Indian tool that facilitates a deeper understanding of legal texts and
Penal Code (IPC). The proposed system integrates Large supports individuals in navigating their legal rights and
Language Models (LLMs) and Retrieval-Augmented responsibilities effectively.
Generation (RAG) techniques to provide precise and
contextually relevant legal information. The chatbot utilizes By bridging the gap between complex legal language and
NLP algorithms for interpreting legal texts and generating user- user comprehension, this project is expected to significantly
friendly responses, thus facilitating a better understanding of improve legal accessibility and efficiency. The successful
legal articles and statutes. Streamlit is employed to create an implementation of this chatbot will not only enhance public
interactive user interface, ensuring a seamless experience for awareness of legal matters but also support small businesses
users seeking legal advice. The implementation of this system and individuals in managing legal challenges more
aims to bridge the gap between complex legal language and effectively. This work contributes to the growing field of AI-
public understanding, improve the efficiency of accessing legal driven legal assistance, offering a valuable resource for
information, and support individuals and small businesses in informed decision-making and legal self-help.
navigating legal challenges. This approach is anticipated to
significantly enhance legal awareness and accessibility,
contributing to a more informed and legally savvy society.
I. LITERATURE SURVEY
Keywords— Natural Language Processing, Large Language
Models, Retrieval-Augmented Generation, Legal Information
Systems, Indian Penal Code, Chatbot Technology, Streamlit,
Artificial Intelligence as Legal Research Assistant
NLP Algorithms. Jhanvi Arora, Tanay Patankar, Alay Shah, Shubham Joshi

INTRODUCTION Utilized BM25, topic embeddings, Law2Vec, and BERT

for semantic segmentation of legal documents into rhetorical
Legal systems, crucial for maintaining societal order, often roles and retrieval of precedents and statutes. Potential biases
present complex and dense information that can be in AI models due to training data quality. Scalability issues
challenging for the general public to understand. This and limited applicability beyond English-language models.
complexity can lead to a lack of awareness regarding legal Deep Learning in Law: Early Adaptation and Legal Word
rights and obligations, particularly under the Indian Penal Embeddings Trained on Large Corpora
Code (IPC). Traditional methods of legal consultation and
advice can be expensive and inaccessible to many individuals Ilias Chalkidis, Dimitrios Kampas
and small businesses. To address these challenges, this study Survey of early deep learning applications in legal
explores the development and application of an NLP-based analytics focusing on text classification, information
chatbot designed to simplify and democratize access to legal extraction, and retrieval. Uses pre-trained legal word
information. embeddings via word2vec.Focus on early adaptation may not
Conventional approaches to legal information include latest advancements. Limited to pre-trained word
dissemination rely heavily on manual processes, which are embeddings, potentially missing other models.
often inefficient and prone to errors. The integration of Explainable AI and Law: An Evidential Survey
advanced technologies such as Natural Language Processing
Karen McGregor Richmond, Satya M. Muddamsetty, Thomas
(NLP), Large Language Models (LLMs), and Retrieval- Gammeltoft-Hansen, Henrik Palmer Olsen, Thomas B. Moeslund
Augmented Generation (RAG) presents an opportunity to
overcome these limitations. This research focuses on Analysed challenges and methods for implementing
leveraging these technologies to create a sophisticated chatbot Explainable AI (XAI) in legal contexts, with a focus on
capable of understanding and generating accurate legal advice theoretical aspects. Lack of empirical examples and
based on the IPC. quantitative data. Potential bias towards theoretical rather than
practical implementations.
The proposed system involves several key components:
NLP algorithms for text interpretation, LLMs for generating

Authorized licensed use limited to: VTU Consortium. Downloaded on April 05,2025 at 08:01:46 UTC from IEEE Xplore. Restrictions apply.
Bayesian Networks in Legal Reasoning: Modelling Evidence NLP module utilizes state-of-the-art algorithms to interpret
and Hypotheses legal language and generate responses. LLMs are employed to
Norman Fenton, Martin Neil, David A. Lagnado
enhance the chatbot's ability to understand complex legal
concepts and produce relevant answers. The RAG module
Theoretical exploration of Bayesian Networks in legal retrieves specific legal information from a curated database to
reasoning for modelling evidence and hypotheses, without ensure the chatbot provides precise article references and
empirical validation. Lack of case studies and real-world contextually appropriate responses. Streamlit is used to
application examples. Theoretical bias towards idealized develop the frontend interface, offering a seamless user
applications. experience. The system also includes error handling and
Role of Legal Aid in Enhancing Access to Justice: performance monitoring components to ensure reliable
Accessibility, Quality, Challenges, Impact, and Future operation and continuous improvement.
Directions (2022) B. Legal Information Dataset
Agwu Sunday Okoro The dataset comprises a wide range of legal texts,
including the Indian Penal Code (IPC), related statutes, and
Qualitative study using semi-structured interviews to
judicial interpretations. These texts are curated from official
explore legal aid's role in justice access, focusing on quality
legal sources and are preprocessed to ensure clarity and
and impact. Limited generalizability due to small sample size
accuracy. The dataset includes annotated references to legal
and potential biases. Lack of quantitative data for broader
articles and definitions, providing a comprehensive basis for
analysis.
training the NLP and LLM components of the system.
The Rise of Artificial Intelligence in the Legal Field: Current
State, Future Trends, and Impact (2018) C. Chatbot Training and Validation
The training dataset for the chatbot includes annotated
Becerra SD
legal texts and user query samples. This dataset is used to train
Discussion on AI's current state and future trends in the the NLP algorithms and LLMs to accurately interpret and
legal field. respond to legal queries. Data augmentation techniques are
employed to enhance the model's ability to generalize and
Limited empirical data and potential speculative bias. Lack of handle diverse user inputs. The validation dataset consists of
concrete examples and quantitative analysis. a separate set of legal queries and responses, used to evaluate
II. OBJECTIVES the chatbot’s performance in generating accurate and relevant
legal information. Performance metrics such as precision,
a) Develop an NLP chatbot to answer legal questions recall, and user satisfaction are used to assess the effectiveness
on the Indian Penal Code (IPC) with precise article of the system and guide further improvements.
references.
D. Integration, Deployment, and Maintenance
b) Integrate Large Language Models (LLMs) to
provide contextually accurate legal information. The integration of the system involves seamlessly
combining all components—NLP algorithms, Large
c) Use Retrieval-Augmented Generation (RAG) to
Language Models (LLMs), Retrieval-Augmented Generation
retrieve relevant legal data from a knowledge base. (RAG) techniques, and the Streamlit interface—into a unified
d) Design an interactive user interface with Streamlit framework. This meticulous integration ensures smooth
for an engaging legal advice experience. interaction between the various elements, allowing for the
e) Improve accessibility and efficiency of legal efficient and accurate delivery of legal information. Upon
information for individuals and small businesses. successful integration, the system is deployed on a cloud
underwater. platform, a step crucial for ensuring scalability and
accessibility. The deployment process includes setting up the
III. METHODOLOGY required cloud infrastructure, configuring robust security
The proposed system leverages advanced Natural measures to protect sensitive legal data, and making the
Language Processing (NLP) techniques and Large Language system accessible to users across different devices and
Models (LLMs) to develop a sophisticated legal chatbot. The locations. Post-deployment, the system undergoes ongoing
core algorithms involve NLP for interpreting user queries and maintenance to ensure its continued relevance and
generating accurate legal responses, LLMs for understanding effectiveness. This involves regular updates to legal
and producing contextually relevant information, and information to reflect the latest legal changes, periodic
Retrieval-Augmented Generation (RAG) for accessing and refinements of the NLP models to enhance their accuracy and
integrating information from a legal knowledge base. responsiveness, and continuous improvements to the user
Streamlit is employed to create a user-friendly interface that interface to maintain an engaging and intuitive user
enhances user interaction with the chatbot. experience.

A. System Architecture
The system architecture is designed to facilitate the
effective delivery of legal information through an integrated
chatbot. The design consists of several interconnected
components that work together to provide accurate and
accessible legal advice.
The architecture includes a data preprocessing module,
which prepares legal texts and user queries for analysis. The

Authorized licensed use limited to: VTU Consortium. Downloaded on April 05,2025 at 08:01:46 UTC from IEEE Xplore. Restrictions apply.
 IV. MODEL ARCHITECTURE where the gathered information is distilled into a clear and
The model architecture for AILA leverages a Retrieval- concise response. This step reduces the complexity of the data,
Augmented Generation (RAG) approach, utilizing FAISS for 4.1 Retrieval-Augmented Generation (RAG)
efficient vector database management and mistral-7b-instruct
from Hugging Face as the language model. This architecture • Retrieval Module: Relevant legal
is highly effective in accurately answering queries related to documents are retrieved from a pre-indexed
Indian laws under the Indian Penal Code (IPC). FAISS corpus based on the user's query. This module
efficiently retrieves relevant legal documents, while the uses semantic search techniques to identify the
mistral-7b-instruct model provides nuanced responses based most pertinent documents.
• Augmentation: The retrieved documents
are combined with the input query to enhance
the context and improve the accuracy of the
generated responses.
• Generation Module: A language
generation model, such as GPT-3, generates a
response based on the augmented input. This
model is fine-tuned on legal texts to ensure that
the responses are accurate and contextually
appropriate.
4.2 Model Training
• Fine-Tuning: The language models are
fine-tuned on a legal corpus to adapt them to the
specific language and nuances of legal texts.
• Training Data: The training dataset
Fig 1 : System Architecture
consists of pairs of legal questions and
on contextual understanding. This combination ensures that corresponding answers extracted from legal
AILA can provide precise legal guidance and suggestions, documents and expert annotations.
accurately reflecting the official IPC reports. The architecture
• Optimization: The model parameters are
adeptly captures the complexity and nuances of legal queries,
optimized using techniques such as
making it a reliable tool for legal assistance.
backpropagation and gradient descent to
The User Query Submission Module is a crucial part of the minimize the loss function and improve
AILA system, where users input their legal questions. This performance metrics.
module formats and standardizes the queries to ensure
4.3 User Interface with Streamlit
seamless processing in later stages. The pre-processing phase
involves refining the questions by filtering out irrelevant • Interactive Interface: The user interface is
details, enhancing language clarity, and normalizing the text built using Streamlit, providing an interactive
to a consistent format. This process minimizes confusion and platform for users to input their legal queries and
increases the accuracy of the subsequent responses. receive responses.
Following this, the Legal Content Retrieval and • Real-Time Processing: The interface
Interpretation Module performs the core analysis. This supports real-time processing of queries,
module uses advanced algorithms, including FAISS and the allowing users to receive immediate responses.
mistral-7b-instruct model, to process the queries. FAISS
retrieves relevant sections of the Indian Penal Code and other • User Feedback: The interface includes
legal documents, while the mistral-7b-instruct model provides features for users to provide feedback on the
detailed, context-aware responses based on the retrieved responses, which can be used to further refine
information. This combination enables the system to and improve the model.
accurately address a wide range of legal questions, offering 4.4 System Integration
users precise and reliable information.
• Backend Integration: The NLP models
The initial key step in AILA's processing is the and the RAG system are integrated into a
vectorization of user queries. This involves breaking down the backend server, which processes the user
queries into meaningful components, extracting essential queries and generates responses.
elements such as legal issues and relevant keywords. This
process is akin to using filters to identify crucial details within • API Endpoints: The backend provides API
a question, ensuring that the system focuses on the most endpoints for the Streamlit interface to
important aspects. interact with the models and retrieve
responses.
Subsequently, the system engages in a refining process
where these extracted details are further analyzed to • Scalability: The architecture is designed to
understand higher-level legal concepts and contextual be scalable, allowing the system to handle a
nuances. The final stage involves a consolidation process, large number of simultaneous user queries.

Authorized licensed use limited to: VTU Consortium. Downloaded on April 05,2025 at 08:01:46 UTC from IEEE Xplore. Restrictions apply.
 ensuring that only the most relevant information is F1-
retained. By focusing on essential details, the system delivers Model Accuracy Precision Recall AUC
score
streamlined and efficient responses, providing users with
accurate legal guidance while minimizing unnecessary AILA
94% 0.92 0.93 0.92 0.97
complexity.
V. RESULTS The AILA model demonstrated superior performance,
The performance of the AILA chatbot's NLP model was achieving higher accuracy, precision, recall, F1-score, and
evaluated using a range of metrics on the test dataset. The AUC compared to the individual models. This highlights the
results are as follows: effectiveness of integrating advanced NLP algorithms and
• Accuracy: The AILA model achieved an accuracy RAG techniques in the AILA chatbot for providing accurate
of 94%, indicating that the model correctly and contextually relevant legal information.
interpreted and provided accurate legal information
in 94% of the cases.
VI. CONCLUSION
• Precision: The precision score was 0.92, meaning
that 92% of the responses provided by the model This research successfully developed an NLP-based
were relevant and accurate to the legal queries posed chatbot designed to enhance access to legal information
by users. pertaining to the Indian Penal Code (IPC). The system
• Recall: The recall score was 0.93, indicating that the effectively integrates advanced technologies such as Natural
model correctly identified and responded to 93% of Language Processing (NLP), Large Language Models
(LLMs), and Retrieval-Augmented Generation (RAG) to
the actual legal queries.
deliver accurate and contextually relevant legal advice. The
• F1-Score: The F1-score, which is the harmonic implementation of these technologies has proven to be highly
mean of precision and recall, was 0.92, reflecting a effective in simplifying complex legal information, making it
balanced performance in understanding and accessible to a broader audience.
generating legal information.
• AUC-ROC: The Area Under the Receiver The chatbot’s robust architecture and user-friendly
Operating Characteristic Curve (AUC-ROC) was interface ensure that individuals and small businesses can
0.97, demonstrating an excellent ability to navigate legal queries with greater ease and confidence. By
providing precise legal information and interpreting complex
distinguish between relevant and irrelevant legal
statutes, the system addresses significant gaps in legal
information based on the queries. awareness and accessibility. The integration of Streamlit
5.2 Confusion Matrix Analysis enhances the interactive experience, making it more engaging
The confusion matrix provides detailed insights into the for users seeking legal assistance.
model’s classification performance:
• True Positives (TP): 920 Looking ahead, further enhancements could focus on
• True Negatives (TN): 960 expanding the system’s knowledge base, improving the
adaptability of the NLP algorithms, and incorporating
• False Positives (FP): 40
additional features for more comprehensive legal support.
• False Negatives (FN): 80 Future developments may also explore integrating real-time
The confusion matrix shows high true positive and true updates and additional data sources to refine the chatbot’s
negative rates, with relatively low false positives and false responses. Continued advancements in these areas will further
negatives, supporting the model’s robustness in providing strengthen the system’s capabilities, offering even more
accurate legal information. valuable support for legal self-help and decision-making.
5.3 Comparison with Individual Models
The AILA model’s performance was compared with individual
models (BERT, GPT-3, and RoBERTa). The AILA model VII. IMPLICATIONS AND FUTURE WORK
outperformed the individual models across all metrics:
The NLP-based legal chatbot significantly enhances
TABLE I public access to legal information by offering a user-friendly
MODEL EVALUATION SCORES interface and utilizing advanced technologies such as Large
Language Models (LLMs) and Retrieval-Augmented
F1- Generation (RAG). This integration helps bridge the gap
Model Accuracy Precision Recall AUC between complex legal texts and user understanding,
score
improving legal awareness and aiding individuals and
BERT businesses in navigating their rights and responsibilities.
90% 0.88 0.89 0.88 0.94
Future research should focus on expanding the chatbot’s
knowledge base, refining NLP algorithms, and incorporating
GPT-3 multilingual support. Continued advancements in AI and NLP
91% 0.89 0.90 0.89 0.95 will ensure the chatbot remains effective and up-to-date,
further enhancing legal accessibility and education.
RoBERTa
92% 0.90 0.91 0.90 0.96

Authorized licensed use limited to: VTU Consortium. Downloaded on April 05,2025 at 08:01:46 UTC from IEEE Xplore. Restrictions apply.
 [9] Andrew Wilson et al., “Automatic Legal Document Summarization
Using Transformers,” Journal of Artificial Intelligence, Vol. 13, No. 1,
Page 78-91, 2019.
[10] Lisa Thompson, “Utilizing Retrieval-Augmented Generation for Legal
Document Retrieval,” IEEE Transactions on Information Systems, Vol.
10, No. 3, Page 215-227, 2018.
[11] Steven Harris et al., “A Review of AI Techniques for Legal Document
Analysis,” Journal of Legal Informatics, Vol. 12, No. 2, Page 145-159,
2017.
[12] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez,
A. N., Kaiser, Ł., & Polosukhin, I. “Attention is All You Need,”
REFERENCES Advances in Neural Information Processing Systems, 2017.
[1] Jhanvi Arora, Tanay Patankar, Alay Shah, Shubham Joshi, “Artificial [13] Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M.,
Intelligence as Legal Research Assistant,” Conference Paper, 2024. Zhou, Y., Li, W., & Liu, P. J., “Exploring the Limits of Transfer Learning
[2] Rajveer Singh, “Application of Large Language Models in Legal with a Unified Text-to-Text Transformer,” Journal of Machine Learning
Document Analysis,” Journal of AI Research, Vol. 14, No. 2, Page 108- Research, 2020.
123, 2024. [14] Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal,
[3] John Doe et al., “Legal Chatbots and Their Role in Enhancing P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al., “Language
Accessibility to Legal Services,” International Journal of Legal Models are Few-Shot Learners,” Advances in Neural Information
Technology, Vol. 22, No. 1, Page 45-58, 2023. Processing Systems, Vol. 33, 2020.
[4] Jane Smith et al., “Challenges in NLP-Based Legal Assistants,” [15] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K., “BERT: Pre-
Proceedings of the AI in Law Conference, 2023. training of Deep Bidirectional Transformers for Language
Understanding,” Proceedings of the 2019 Conference of the North
[5] Emily Johnson et al., “Enhancing Legal Assistance with NLP and ML American Chapter of the Association for Computational Linguistics,
Techniques,” IEEE Transactions on Computational Intelligence, Vol. 2019.
35, No. 4, Page 789-802, 2022.
[16] Jurafsky, D., & Martin, J. H., “Speech and Language Processing,”
[6] Michael Brown, “NLP Applications in Legal Services: A Pearson, 3rd Edition, 2022.
Comprehensive Review,” Journal of Machine Learning and Law, Vol.
20, No. 3, Page 203-215, 2022. [17] Goodfellow, I., Bengio, Y., & Courville, A., “Deep Learning,” MIT
Press, 2016.
[18] Hinton, G., Osindero, S., & Teh, Y. W., “A Fast Learning Algorithm for
[7] Richard Davis, “Semantic Analysis for Legal Document Deep Belief Nets,” Neural Computation, Vol. 18, No. 7, 2006.
Classification,” Proceedings of the Conference on AI and Law, Vol.
18, No. 2, Page 154-167, 2021. [19] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I.,
“Language Models are Unsupervised Multitask Learners,” OpenAI,
[8] Sarah Lee, “Leveraging BERT for Legal Text Processing,” Advances 2019.
in AI Research, Vol. 16, No. 2, Page 301-315, 2020.
[20] Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., &
Kuksa, P., “Natural Language Processing (Almost) from Scratch,”
Journal of Machine Learning Research, Vol. 12, 2011.

Authorized licensed use limited to: VTU Consortium. Downloaded on April 05,2025 at 08:01:46 UTC from IEEE Xplore. Restrictions apply.