VISVESVARAYA TECHNOLOGICAL UNIVERSITY

“Jnana Sangama”, Belagavi-590018, Karnataka, India

PROJECT WORK REPORT ON

“AI Powered Medical Diagnosis”

Submitted in partial fulfilment of the requirements
For the Seventh Semester Bachelor of Engineering Degree
SUBMITTED BY
Mohammad Israr Hadagali (1IC21CD001)
Monica K (1IC21CD002)
Sanjay G R (1IC22CD400)
Muhammed Saad Khan (1IC21AI019)

Under the guidance of

Mr. Krishna Mehar
Assistant Professor
Department of AI & ML

IMPACT COLLEGE OF ENGINEERING AND APPLIED SCIENCES

Sahakarnagar, Banglore-560092

2024-2025
 IMPACT COLLEGE OF ENGINEERING AND APPLIED SCIENCES
Sahakarnagar, Banglore-560092

DEPARTMENT OF DATA SCIENCE

CERTIFICATE
This is to certify that the Project Work entitled "AI Powered Medical Diagnosis” carried out
by Monica K (1IC21CD002) is a bonafide student of Impact College of Engineering and
Applied Sciences Bangalore has been submitted in partial fulfilment of requirements of VII
semester Bachelor of Engineering degree in Computer Science & Engineering (Data
Science) as prescribed by VISVESVARAYA TECHNOLOGICAL UNIVERSITY during the
academic year of 2024-2025.

Signature of the Guide Signature of the HoD Signature of the Principal

Mr. Krishna Mehar Dr. Kaipa Sandhya Dr. Jalumedi Babu

Assistant Prof. Prof. & Head ICEAS, Bangalore
Dept. of AI & ML Dept. of CSE (CD)
ICEAS, Bangalore. ICEAS, Bangalore.

Name of Examiner Signature with date

1. 1.

2. 2.
 ACKNOWLEDGEMENT
The satisfaction and euphoria that accompany the successful completion of any task would
be incomplete without the mention of the people who made it possible and whose constant
encouragement and guidance crowned my efforts with success.

I consider proud to be part of Impact College of Engineering and Applied Sciences

family, the institution which stood by us in our endeavor.

I am grateful to our guide Mr. Krishna Mehar, Assistant Professor, Department of

Computer Science of Engineering (AI & ML), for his keen interest and encouragement
in our project, their guidance and cooperation helped in nurturing the project in reality.

I am grateful to Dr. Kaipa Sandhya, Head of Department Computer Science of

Engineering (Data Science), Impact College of Engineering and Applied Sciences
Bangalore who is source of inspiration and of invaluable help in channelizing our efforts
in right direction.

I express my deep and sincere thanks to our Management and Principal, Dr. Jalumedi
Babu for their continuous support.

Mohammad Israr Hadagali(1IC21CD001)

Monica K(1IC21CD002)
Sanjay G R(1IC22CD400)
Muhammed Saad Khan(1IC21AI019)

i
 ABSTRACT

The purpose of the AI-Powered Medical Diagnosis project is to use a combination

of patient symptoms, lab results, and medical images to make it simpler for
healthcare professionals to diagnose a disease using artificial intelligence. It
involves NLP for symptom analysis, Convolutional Neural Networks (CNNs) for
medical image classification, as well as machine learning on lab results. The tool
has made the diagnosis process automated by being developed using Python,
TensorFlow, Keras, and Scikit-learn. Predictions are done in real time through its
web interface. It's anticipated that the system, based on improved diagnosis
accuracy coupled with speed, will decrease the chances of medical errors, better
patient care, and solid support for doctors, mainly in under-resourced settings.

ii
 CONTENTS
ACKNOWLEDGEMENT i
ABSTRACT ii

CHAPTER No. TITLE PAGE NO

1 INTRODUCTION 1
1.1 The Evolution of Diagnostic Technology 2
1.2 How AI works in Diagnosing Conditions 2
1.3 Role of AI in Diagnosis 3
1.4 Benefits of AI in Healthcare 3
1.5 AI in Imaging and Radiology 3
1.6 AI in Enhancing Diagnostic Accuracy 3
1.7 Key Components 4
2 LITERATURE SURVEY 5
2.1 Existing System 6
2.2 Proposed System 7
2.3 Problem Statement 7
2.4 Objectives 8
3 SYSTEM REQUIREMENTS 9
3.1 Hardware Requirements 9
3.2 Software Requirements 11
4 SYSTEM DESIGN 12
4.1 Use Case Diagram 12
4.2 Activity Diagram 16
4.3 Architecture Diagram 18
5 IMPLEMENTATION 22
5.1 Overview of Project Modules 22
5.2 Tools and Technologies Used 27
6 TESTING 29
6.1 Types of Tests Performed 29
6.2 Results 32
CONCLUSION & FUTURE WORK 37

REFERENCES 39
 LIST OF FIGURES

FIGURE NO FIGURE NAME PAGE NO

1 Use case diagram for brain tumor detection 13

2 Activity Diagram 17
3 Activity Diagram for Text Generator 18
4 Architecture diagram 19
5 Home page 33
6 Home page of the modules 34
7 Detection of Brain Tumor 35
8 Detection of Bone Fracture 35
9 Detection of Lung Cancer 36
10 Text Generation 37

LIST OF TABLE

TABLE NO TABLE NAME PAGE NO

Table 1 Literature survey 5
AI Powered Medical Diagnosis

CHAPTER 1
INTRODUCTION

Artificial Intelligence (AI) is at the forefront of transforming healthcare, particularly in medical

diagnosis. The integration of AI technologies into the diagnostic process is redefining how
diseases are detected, monitored, and managed. AI-powered medical diagnosis leverages
advanced computational models, machine learning algorithms, and data analytics to enhance
the accuracy, speed, and reliability of identifying health conditions.
Artificial intelligence in medical diagnosis is revolutionizing the field of healthcare, offering
new levels of accuracy and efficiency. AI technologies, particularly in medical diagnostics, are
transforming how diseases are detected, analyzed, and treated. By leveraging machine learning
and deep learning algorithms, AI can process vast amounts of data swiftly and accurately,
providing healthcare providers with invaluable insights. These advancements are not only
enhancing the precision of diagnoses but also enabling early detection and personalized
treatment plans.
AI in medical diagnosis refers to the use of advanced computational methods and machine
learning algorithms to analyze complex medical data, interpret diagnostic tests, and assist
healthcare professionals in making more accurate and timely diagnoses. This technology has
the potential to revolutionize healthcare by enhancing diagnostic accuracy, enabling early
disease detection.
At the core of AI-driven diagnosis are sophisticated tools that process vast amounts of medical
data, including patient histories, medical imaging, genomic information, and real-time health
metrics. These systems use machine learning to recognize patterns and anomalies that might be
imperceptible to the lung cancer, bone fracture, brain tumor.
The evolution of AI in healthcare has been transformative, especially in the field of medical
diagnostics. Initially, AI was primarily used for administrative tasks, but its role has expanded
significantly. Now, AI and machine learning algorithms analyze vast amounts of data quickly
and accurately, assisting healthcare providers in making more informed decisions. These
technologies can process medical images, recognize patterns, and even predict disease
outcomes, revolutionizing the practice of medicine.
Moreover, AI systems employ techniques such as natural language processing (NLP) to analyze
unstructured medical texts like doctors' notes and clinical reports, while computer vision aids
in interpreting medical imaging data. This multifaceted approach allows AI to assist healthcare
professionals in making more informed decisions, reducing diagnostic errors, and improving
patient outcomes.
AI in medical diagnosis works by processing vast amounts of patient data, including electronic
health records, diagnostic imaging results, genetic information, and clinical profiles. By
comparing this information to thousands of other patient records, AI systems can identify
similarities, patterns, and trends that may not be immediately apparent to human clinicians. This
capability allows AI to provide valuable insights and support clinical decision-making.
The adoption of AI in medical diagnosis also addresses challenges in healthcare accessibility
and efficiency. By automating routine diagnostic tasks, AI-powered medical diagnosis is poised
to revolutionize the field of medicine, paving the way for a future where precision and
preventive care are more accessible than ever.

Dept. of CSE (CD) 2024-2025 1

AI Powered Medical Diagnosis
AI has the potential to revolutionize medical diagnosis by enhancing accuracy and efficiency.
Machine learning algorithms can analyze vast amounts of patient data, including medical
images, bio-signals, vital signs, and laboratory test results, to provide more accurate and timely
diagnoses. Studies have shown that AI systems can reduce false positives and false negatives in
mammogram interpretation, with one study reporting absolute reductions of 5.7% and 9.4%,
respectively.
The integration of AI in clinical laboratories has led to increased efficacy and precision.
Automated techniques in blood cultures, susceptibility testing, and molecular platforms have
become standard in numerous laboratories globally, contributing significantly to laboratory
efficiency. This automation allows for faster results, often within 24 to 48 hours, facilitating the
selection of suitable antibiotic treatments for patients with positive blood cultures.

1.1 The Evolution of Diagnostic Technology

The journey of AI in medical diagnosis has been marked by significant advancements over the
years:
Early Decision Support Systems: Decision support technologies have been around for decades,
with early systems like MYCIN developed in the 1970s for diagnosing blood-borne bacterial
infections. However, these rule-based systems, while promising, were not widely adopted in
clinical practice due to limitations in performance and integration with existing workflows.
Differential Diagnosis Generators: More recent tools, such as differential diagnosis generators,
have shown potential in assisting clinical diagnosis and education. Studies have found these
tools to be subjectively helpful, with varying degrees of accuracy in suggesting correct
diagnoses.
Modern AI and Machine Learning: The application of new computational methods, including
artificial intelligence and natural language processing, has significantly enhanced the
capabilities of diagnostic tools. These advanced systems can analyze large amounts of complex
patient data and identify trends in disease course and management.
Specialized AI Applications: Recent developments have seen AI being applied to specific areas
of diagnosis, such as:
• Analyzing blood samples to predict treatment responses in rheumatoid arthritis patients.

• Improving the diagnosis of leukemia through machine learning programs.

• Detecting underlying risk factors for future heart attacks.
• Enhancing radiological image analysis for various
conditions.
As AI continues to evolve, it is likely that the combined expertise of human clinicians and AI
algorithms will lead to more accurate diagnoses and improved patient outcomes. However, it is
important to note that while AI shows great promise, further research is needed to fully
understand and optimize its performance in real-world clinical settings.

1.2 How AI Works in Diagnosing Medical Conditions

AI in medical diagnosis relies on machine learning technologies to analyze complex medical
data and assist healthcare professionals in making accurate and timely diagnoses. These
advanced computational methods process vast amounts of patient information, including
electronic health records, diagnostic imaging results, and clinical profiles.

Dept. of CSE (CD) 2024-2025 2

AI Powered Medical Diagnosis
1.3 Role of AI in Diagnosis:
Enhanced Accuracy: AI algorithms improve diagnostic accuracy by analyzing complex
medical data, reducing human error.
Early Detection: Machine learning models can identify early signs of diseases such as cancer
or heart disease, allowing for timely intervention.
Efficiency: Automated systems speed up the diagnostic process, freeing up healthcare
providers to focus on patient care.

1.4 Benefits of AI in Healthcare:

Data Analysis: AI processes large datasets from electronic health records (EHRs), providing
insights that are difficult to achieve manually.
Imaging: Advanced AI tools enhance the interpretation of medical images, aiding radiologists
in identifying abnormalities.
Predictive Analytics: Predictive models forecast disease progression, helping in preventive
care and better resource allocation.
Clinical Decision Support: AI systems provide evidence-based recommendations, supporting
clinicians in making more informed decisions.

1.5 AI in Imaging and Radiology

Artificial intelligence has demonstrated remarkable progress in image-recognition tasks, particularly
in radiology. Deep learning algorithms, inspired by the human brain's neural network structure, can
analyze complex patterns in medical images with high accuracy. This capability allows AI to provide
quantitative assessments in an automated fashion, complementing the qualitative reasoning of
trained physicians.AI-powered tools have shown impressive results in several areas of medical
imaging:
Lung Nodule Detection: AI algorithms have been developed to detect pulmonary nodules in CT
scans, assisting in the early identification of potential lung cancer cases.
Brain Tumor Classification: AI accurately classifies brain tumors into grades with minimal
false positives or negatives, aiding in treatment planning and prognosis.
Bone Fracture Detection: AI algorithms have been developed to detect if the bone is fractured or not
through X-rays.

1.6 Artificial Intelligence in Enhancing Diagnostic Accuracy

Artificial intelligence (AI) is significantly enhancing diagnostic accuracy in the medical field, often
outperforming traditional methods. For instance, in radiology, AI-powered algorithms can analyze
medical images with remarkable precision. Studies have shown that AI systems detect breast cancer
in mammograms more accurately than human radiologists. Machine learning algorithms analyze
wound exudate and other clinical data to identify signs of infection before they become
clinically apparent. Early detection allows for prompt treatment, reducing the risk of severe
complications and promoting faster recovery. By leveraging machine learning and advanced
data analysis, AI tools provide healthcare providers with precise, timely, and actionable
insights. Integrating AI into clinical practice enhances the quality of medical care, ultimately
improving health outcomes for patients.

Dept. of CSE (CD) 2024-2025 3

AI Powered Medical Diagnosis
1.7 Key Components
Comprehension of Natural Language (CNL):
Objective: The main objective of CNL is to enhance the system's capacity to grasp and interpret
user inquiries with precision.
Methods: Employ sophisticated techniques in Natural Language Processing (NLP) to train the
model on recognizing and comprehending diverse linguistic subtleties such as context,
intonation, and informal expressions.
Significance: This enables seamless processing and comprehension of user-provided
information by the AI system, thus facilitating more precise diagnosis based on natural language
intricacies.

Utilization of Machine Learning in Diagnosis:

Goal: Constructing a resilient model based on Machine Learning (ML) to examine user inputs,
juxtapose them with an extensive repository encompassing medical records and expertise,
thereby generating plausible diagnoses.
Operation: The ML algorithm perpetually assimilates new information and hones its diagnostic
precision by incorporating feedback from users along with updated medical insights.
Benefits: By harnessing ML capabilities, the system becomes adaptable towards emerging
medical advancements. It can seamlessly integrate novel data while refining its diagnostic
aptitude through observing user outcomes and soliciting their valuable input.

User-Friendly Interface:
Goal: Develop a user-friendly interface that allows for smooth interaction between users and
the AI doctor, prioritizing ease of use and intuitiveness.
User-Centric Design: With a focus on the end-user, prioritize the development of an interface
that enhances user experience, thereby facilitating the widespread acceptance of the AI-driven
healthcare solution.
Seamless Incorporation with Telemedicine Platforms:
Aim: Streamline the incorporation of current telemedicine systems to allow individuals to
connect with human medical experts for additional consultation.
Role of AI Doctor: The AI doctor plays a crucial role in the healthcare industry by serving as
an initial diagnostic tool. Its purpose is to streamline the diagnostic process for both patients
and healthcare providers. It facilitates a seamless transition between AI-powered diagnostics
and human expertise, ensuring efficient and effective healthcare delivery.

Dept. of CSE (CD) 2024-2025 4

 AI Powered Medical Diagnosis
CHAPTER 2
LITERATURE SURVEY
Paper Title Author(s) Journal/Conference Year Summary of Relevance to
Findings Project

AI-Driven Various IEEE Transactions on 2021 Discusses CNN Relevant for

Diagnostic Authors Medical Imaging and GAN enhancing diagnostic
Systems for models accuracy in imaging
improving
Medical Imaging modalities like MRI
accuracy in
medical imaging and CT scans, vital
for cancer, for oncology and
cardiovascular neurology
diseases, and diagnostics.
neurology.
Natural K. Liu et al. IEEE Transactions on 2022 Examines the Crucial for projects
Language Information use of NLP for focused on
Processing in Technology in interpreting automating textbased
Medical Biomedicine clinical notes diagnostic systems,
Diagnostics and automating especially for
diagnosis of structured and
diseases like unstructured clinical
COVID-19 and data.
diabetes.
Artificial A. Patel et IEEE Journal of 2023 Highlights the Key for developing
Intelligence in al. Biomedical Health use of AI in ECG real-time monitoring
Cardiovascular Informatics and heart rate systems using AI,
Diagnostics variability especially in
analysis to detect wearable health
arrhythmia and technology for heart
other conditions.
cardiovascular
issues.

AI and M. Zhang, IEEE Computational 2024 Explores AI's Highly relevant for
Genomics: S. Gupta Biology role in analyzing integrating AI in
Predictive genomic data to genomic data
Models for predict disease analysis for
Disease Risk risk, particularly predictive
in personalized diagnostics.
medicine and
oncology.

Dept. of CSE (CD) 2024-2025 5

AI Powered Medical Diagnosis
2.1 Existing System
Based on the literature review presented in the papers, there are some key research gaps and
limitations in existing methods for implementing AI in healthcare:
Many AI models show strong performance in narrow academic datasets or benchmarks, but
generalization is difficult. the complexity and variability of real clinical conditions. More testing
on heterogeneous real-world data is needed. The accuracy of AI driven radiology depends on
the quality and diversity of the training data. NLP models require large, diverse datasets to train
on, and the data must be high-quality and appropriate. However, data sets can be limited by
research area or data type.
There are no frameworks or guidelines in place to ensure that NLP tools are working as
intended. Lack of interpretability and transparency has been identified as one the main barriers
to implementation of AI in clinical practice. AI-based algorithms can be superhuman in their
ability to interpret complex data. However, their power and complexity can also result in
spurious or even unethical and discriminatory conclusions when applied to human health data.
Without careful consideration of the methods and biases embedded in a trained AI system, the
practical utility of these systems in clinical diagnostics is limited.

• Generalizability: Many AI models show strong performance in narrow academic

datasets or benchmarks, but generalization is difficult. the complexity and variability
of real clinical conditions. More testing on heterogeneous real-world data is needed.
• Explainability: The transparency of the results of most AI models limits confidence
and adoption. Developing interpretable models and explanatory methods is critical.
• Model Validation: Many studies highlight the lack of standardized methods for
clinical validation of AI systems prior to deployment, which increases the potential
for undetected defects or failures.
• Customization: Most models use a one-size-fits-all approach, while medical
diagnosis and treatment planning require customization. Advancing personalized
and precision medicine with artificial intelligence is an open challenge.
• Integration into Workflows: There is a lack of substantial research to quantify the
effects and ease of integration into existing clinical workflows in a minimally
disruptive manner. More applied research is needed.
• GPU price and availability: As the models grow larger and larger, proprietary
high-performance GPUs are required to train and maintain them, which are very
expensive and of limited availability. This presents obstacles for many healthcare
systems with limited resources. Building efficient models and optimizing hardware
requirements is an open challenge.
• Lack of training recipes: There are no clear standard recipes for training robust and
reliable models that encode best practices for regularization, scaling, etc. This leads
to the proliferation of fragile designs that fail unexpectedly. Establishing strict
training protocols and benchmarks is important.
• Prompt Engineering: Appropriate prompts and examples are critical to encoding
intended behavior, but best practices remain unclear. Bad signals increase the
likelihood of unintended action. The development of rapid design techniques is
crucial to minimize these risks.
• Privacy: There are concerns about privacy, and regulations for data use and privacy
protections for NLP technologies have yet to be established.
• Lack of standardization: Different AI vendors may use different platforms and
methods.

Dept. of CSE (CD) 2024-2025 6

AI Powered Medical Diagnosis
2.2 Proposed System
Our AI-powered medical diagnosis initiative relies on diverse datasets that transcend mere
symptom records. our dataset is not only broad but also representative, strengthening the
efficiency and reliability of our AI model. The goal is to empower our model to be a valuable
tool supporting users in diverse healthcare situations. Our diagnostic system has a text generator
which predicts the next few words of the patient who are not able to complete the sentence due
to some medical conditions. It gives detailed information to the queries and suggestion that
should be followed by the user.
• Data Preprocessing: The foundation of our AI model is laid through thorough data
preprocessing. This involves cleaning and rectifying errors in our vast medical
records. Beyond technical aspects, privacy is paramount. We adhere strictly to data
protection laws, ensuring the integrity and accuracy of the data that forms the
backbone of our AI model.
• Model Building: We adopt a two-pronged strategy, utilizing natural language
processing (NLP) and machine learning (ML) in building our advanced AI model.
The integration of NLP and ML ensures our model comprehends intricacies in user
questions, improving diagnostic capabilities and enabling more profound
engagement in health-related conversations.
• AI Model Integration: User experience is at the forefront of our AI model
integration. We prioritize an intuitive and user-friendly interface, allowing seamless
input of symptoms. The system quickly generates initial diagnoses, providing users
with an overview of their health status. It also allows users to bridge the gap between
digital communication and personalized care.
• Scalability Planning: Scalability is a focal point in our strategic planning. We
design our AI-based healthcare system to be not only robust but also flexible enough
to adapt to a growing user base.
• Education and Awareness: We are recognizing that an informed user is an
empowered user. We communicate openly about the symptoms and treatment for
those symptoms encouraging users to view it as a complement to traditional
healthcare.
• Collaboration: Collaboration is ingrained in our approach to AI healthcare.
Actively involving text generator which predicts the next few words of the patient
who are not able to complete the sentence due to some medical conditions. This
collaborative approach aims to redefine healthcare by merging technology and
medicine, creating an innovative system deeply rooted in validated healthcare
knowledge.

2.3 Problem Statement

The accurate and timely diagnosis of medical conditions is critical for effective treatment and
improved patient outcomes. However, the complexity of medical data, coupled with the
increasing workload on healthcare professionals, can lead to diagnostic errors and delays. The
number of patients, intricacy and complexity, volumes of information, symptoms, lab results,
and medical images add up to a daunting challenge in the time frame as important as it is to get
disease diagnosis accurately as fast as possible. The lack of connectivity between the various
types of data used in the diagnosis process also adds up to the above issues. Traditional methods
of diagnosis rely on the manual interpretation of data, which delays and becomes inconsistent
at worst, and inaccurate at best.

Dept. of CSE (CD) 2024-2025 7

AI Powered Medical Diagnosis
2.3.1 Solution
This project developed an AI-powered system in an effort to integrate multiple data sources.
Advanced techniques in AI supported doctors in doing quicker and more accurate diagnoses
that not only improved patient care but also mitigated risks related to misdiagnosis. This project
aims to develop an AI-powered system leveraging Convolutional Neural Networks (CNN) and
machine learning techniques to automate and enhance the diagnosis of [specific condition(s),
e.g., lung cancer, brain tumor, bone fracture etc.] from medical images [e.g., X-rays, CT scans,
MRIs, etc.].
The system will focus on:
➢ Preprocessing medical images to ensure high-quality input.
➢ Building a robust CNN-based model for feature extraction and classification.
➢ Employing machine learning for additional analysis and diagnosis validation.
➢ A text generator to assist the patients who are unable to complete their sentences
due to certain medical conditions.
➢ By implementing this system, the project seeks to improve diagnostic accuracy,
reduce time to diagnosis, and assist healthcare providers in delivering better patient
care.

2.4 Objectives
The initial step in developing an AI-based diagnostic model is to design and implement a
reliable system capable of detecting common acute disorders. To accomplish this, use machine
learning techniques such as decision trees or neural networks that have been trained on a broad
collection of medical data and symptoms. Using supervised learning approaches, the model
should learn to spot patterns and connections between symptoms and diseases. To develop an
AI-powered web application that assists in diagnosing diseases using medical imaging and
patient data. The system includes modules for brain tumor detection, lung cancer detection,
bone fracture prediction and a text generator for real-time assistance.
• Improved Accuracy: Enhance diagnostic accuracy by reducing human errors and
providing precise symptoms.
• Enhanced Accessibility: Provide medical expertise to remote and underserved
areas.
• Model Education: To train the AI model, use supervised learning approaches. By
including labeled instances in the training dataset, you may emphasize the
relationship between presented symptoms and accurate diagnoses.
• Measures for Evaluation: Define and monitor performance measures such as
accuracy, sensitivity, and specificity to assess the model's effectiveness in disease
diagnosis. Create a baseline for future comparison and improvement.
• Create a User-Friendly Interface: Create an intuitive and user-friendly interface
for individuals to interact with the AI-based diagnostic system.

Dept. of CSE (CD) 2024-2025 8

AI Powered Medical Diagnosis
CHAPTER 3
SYSTEM REQUIREMENTS
3.1 Hardware Requirements
To support the efficient implementation and operation of the Natural Processing Language
(NLP), the following hardware infrastructure is required:
1. High-Performance Servers

• Purpose: To handle multiple user queries simultaneously and process large

datasets in real time.

• Specifications:

➢ High CPU power (multi-core processors) to manage backend services,

API requests, and database operations efficiently.
➢ Scalability to ensure performance remains optimal.

• Impact: To be faster, more accurate and scalable, making them a critical

component of modern healthcare systems.

2. GPU Support

• Purpose: To accelerate the training and inference of machine learning models,

particularly natural language processing (NLP) models.

• Specifications:

➢ Dedicated GPUs (e.g., NVIDIA Tesla or equivalent) for optimized

performance in processing large NLP models.

➢ Support for parallel computing to reduce the time required for model
training and inference.

• Impact: Enhances the usability, efficiency and trustworthiness of AI powered

medical diagnosis enabling seamless integration into workflow and improving
outcomes.

3. Storage Systems

• Purpose: To accommodate large-scale data processing needs, ensure database

performance, and allow for future data growth.

• Specifications:
➢ High-capacity storage systems for managing extensive datasets.
➢ Fast I/O (Input/Output) operations to ensure quick read/write access to
databases.

Dept. of CSE (CD) 2024-2025 9

AI Powered Medical Diagnosis
4. RAM

• Purpose: To handle large datasets and running multiple models simultaneously.

• Specifications:

➢ High-capacity storage systems for managing extensive datasets.

➢ Scalability: Increases or upgrades the memory capacity in a system to

meet growing performance demands.

• Impact: They influence performance, reliability, scalability and security. The

choice of storage architecture whether on cloud based or hybrid must align
specific needs of healthcare.

5. Internet Connection

• Purpose: To enable devices to communicate with each other globally providing

access to information, services and resources.

• Specifications:

➢ A good internet connection balances speed.

➢ Scalability: To grow and adapt to increasing demands such as higher data

usage, more connected devices or faster speed.

• Impact: They influence speed, accuracy, accessibility and overall quality of

care. It is essential for maximizing the potential of AI in healthcare.

6. Monitor

• Purpose: That serves as a primary interface between a user and a computer. It

displays visual information and provides means of interaction with software,
applications and data.

• Specifications:

➢ Allows users to view text, images, videos and other form of visual data.

➢ Scalability: It can adapt or grow with user needs in terms of size,

resolution, functionality.

• Impact: It acts as a bridge between complex AI algorithms and human

interpretation. High quality display enhances diagnostic accuracy.

Dept. of CSE (CD) 2024-2025 10

AI Powered Medical Diagnosis

3.2 Software Requirements

The software requirements define the foundational tools and frameworks necessary for the
successful development, deployment, and operation of the system. Each component plays a
vital role in ensuring efficiency, scalability, and security.
Operating Systems
The application is designed for cross-platform compatibility, ensuring it functions seamlessly
across various environments:
• Windows 10/11: A widely used OS that supports development tools and user
interaction.
• macOS: Provides a robust development environment with native support for Python
and containerization tools.
• Linux (Ubuntu preferred): Chosen for its reliability, security, and extensive support
for server-side deployments.
Programming Languages

• Python 3.8 or Later: Serves as the backbone for backend logic, machine learning model
implementation, and data preprocessing.
• JavaScript (ES6): Used for creating dynamic and interactive frontend elements.
Frameworks and Libraries
1. Backend Development:
• Django/Flask: Frameworks that streamline the creation of RESTful APIs, manage
routing, and handle server-side requests efficiently.
2. Frontend Development:
• JavaScript: A powerful library for building responsive, modular, and dynamic user
interfaces.
• HTML: Used to structure the contents of the web page.
• CSS: Is a style sheet language used for specifying the presentation and styling a
document written in HTML.
3. Machine Learning:
• Scikit-learn: Essential for implementing the Random Forest algorithm and other
supervised learning techniques.
• Pandas & NumPy: Core libraries for handling data preprocessing, manipulation,
and numerical computations.
• TensorFlow/Keras: For building and training deep learning (CNNs for medical
image analysis).
• OpenCV: For image processing and manipulation of medical images.
• NLTK / spaCy: For NLP tasks like symptom analysis from text data.
• Visualization: Matplotlib, Seaborn: For visualization data, model performance and
results.
By leveraging these tools and technologies, the system achieves a balance between
performance, usability, and maintainability, ensuring it meets the demands of healthcare.

Dept. of CSE (CD) 2024-2025 11

AI Powered Medical Diagnosis
CHAPTER 4
SYSTEM DESIGN

4.1 USE CASE DIAGRAM

Fig 4.1: Use case diagram for brain tumor detection

The Use Case Diagram for the AI-powered medical diagnosis illustrates the interactions
between the User and the Developer, showcasing the key functionalities and workflows that
define the system. It provides a visual representation of the system's design, highlighting how
the various components interact with the system to deliver the required output.

4.1.1 Developer Interactions

Developer
Developing AI-powered medical diagnosis systems is an exciting and impactful field that
involves combining knowledge from artificial intelligence, medicine, and software engineering.
To succeed in applying AI to the medical field, start by building a basic understanding of
medical terminologies, diseases, and diagnostic workflows to ensure your solutions are relevant
and effective.

Dept. of CSE (CD) 2024-2025 12

AI Powered Medical Diagnosis
Next, develop strong technical skills by learning key AI and machine learning (ML)
frameworks like TensorFlow, PyTorch, and Scikit-learn, along with techniques such as deep
learning, computer vision, and natural language processing (NLP). Understanding how to
handle medical data is crucial — work with imaging formats like DICOM for X-rays and MRIs,
analyze structured data from electronic health records (EHR), and apply NLP to unstructured
clinical notes. Securing high-quality datasets is essential; collaborate with hospitals for labeled
data or use public datasets like MIMIC-III and NIH Chest X-rays, while ensuring data privacy
and regulatory compliance. Focus on effective data preprocessing and use tools like SHAP or
LIME to ensure model explainability. Evaluate model performance using metrics like
sensitivity, specificity, and precision-recall, and improve your model with input from doctors.
When deploying AI solutions, integrate them with hospital systems, develop user-friendly
interfaces for medical professionals, and monitor performance regularly, retraining as necessary
to maintain accuracy. Continuous collaboration with healthcare experts and regulatory
consultants is key to building safe, effective, and compliant AI solutions.
Import MRI
Working with MRI (Magnetic Resonance Imaging) data involves handling specialized medical
file formats, preprocessing the data, and analyzing or visualizing it.
Data Loading
Efficient data loading is a critical step in medical AI applications, particularly when working
with imaging data such as MRI scans. Utilizing appropriate tools and libraries to load file
formats like DICOM and NIfTI ensures seamless integration with processing pipelines. It's
important to maintain data integrity by performing thorough checks for missing or corrupt
slices, as inconsistencies can significantly impact model performance and diagnostic
accuracy. Proper data handling at this stage helps establish a strong foundation for building
reliable and robust AI solutions in the medical domain.
Data preprocessing
Data preprocessing for MRI scans is essential to ensure high-quality data for analysis,
visualization, and model training. This process begins by loading the data and thoroughly
checking for missing slices in 3D or 4D datasets while handling corrupt or unreadable files by
excluding them from the pipeline. To enhance image clarity, noise reduction techniques such as
Gaussian or median filtering are applied. Skull stripping follows, where non-brain tissues are
removed to isolate the brain region using specialized algorithms or pre-trained models.
Consistency in image dimensions is achieved through cropping or padding to fit processing
requirements. For targeted analysis, optional segmentation can extract specific regions such as
tumors using intensity thresholds or advanced segmentation models. Finally, storing the
preprocessed data in a standardized format ensures smooth integration for downstream tasks,
including visualization and model training.
Model Building
Model building for MRI data analysis involves selecting the right machine learning (ML) or
deep learning (DL) approach and tailoring the model to specific objectives such as
classification, segmentation, or anomaly detection. The first step is defining the task:
classification models predict labels (e.g., tumor presence or disease types), while segmentation
models identify regions of interest like brain tumor boundaries. Proper dataset preparation is
crucial, involving the division of data into training, validation, and testing subsets, data
augmentation techniques to enhance diversity through flipping, rotation, or noise addition, and
Dept. of CSE (CD) 2024-2025 13
AI Powered Medical Diagnosis
normalization of image intensities for consistent inputs. Depending on the complexity and
nature of the task, model architecture choices range from traditional ML models like Random
Forest, SVM, or XGBoost for tabular or extracted features, to deep learning architectures such
as CNNs (e.g., ResNet or DenseNet) for raw MRI data analysis. Preprocessing for model input
often includes resizing MRI slices to a fixed shape, stacking slices for 3D models.
Training the Model
Training a model for MRI data analysis involves structured steps to ensure effective learning
and generalization. The process begins by defining the training objective, such as classification
to predict labels, segmentation to identify regions of interest, regression for continuous
predictions, or anomaly detection for identifying abnormalities. Preparing the data is crucial and
includes normalizing pixel intensities, resizing or resampling images to a fixed shape, and
converting 3D MRI volumes into slices or stacks if necessary. The dataset is typically divided
into training, validation, and test sets, often in a 70%-20%-10% split. Selecting a suitable model
architecture is task-dependent, with CNNs like ResNet or DenseNet for classification, U-Net or
SegNet for segmentation, and autoencoders or GANs for anomaly detection. Careful
consideration is given to ensure the model is neither too simple nor overly complex to prevent
overfitting.

Data augmentation plays a vital role in improving model generalization by applying techniques
such as flipping, rotation, scaling, and intensity shifts. Training the model involves setting
appropriate batch sizes and epoch counts, leveraging frameworks like TensorFlow, PyTorch, or
Keras, and using data loaders for efficient
processing. Continuous monitoring of validation metrics helps avoid overfitting. Model
evaluation on unseen test data is essential for measuring performance, using tools like confusion
matrices for classification tasks or visual overlays for segmentation. Fine-tuning through
hyperparameter adjustments or transfer learning from pre-trained models often enhances
performance. Lastly, saving the model's weights and architecture ensures reusability and
facilitates deployment.
Testing the Model
Testing a trained model on MRI data is essential to assess its generalization and suitability for
real-world applications. The process begins by preparing a separate test dataset that was not
used during training or validation to ensure an unbiased evaluation. The test data must undergo
the same preprocessing steps as the training data, including intensity normalization, resizing to
required dimensions, and ensuring consistency in orientation and voxel size. The trained model
is then loaded by restoring its saved weights and architecture using a compatible framework
like TensorFlow or PyTorch, ensuring the model and test data formats align seamlessly.

Evaluating model performance involves running forward passes on the test data to generate
predictions. For classification tasks, this includes outputting probabilities or class labels, while
segmentation tasks produce region masks. Error analysis follows, focusing on incorrect
predictions or poorly segmented regions to identify common failure cases such as
underrepresented classes or noise sensitivity. This analysis can reveal insights for improving
the model through refined preprocessing, enhanced augmentation strategies, or modifications
to the model architecture for better performance and robustness.

Dept. of CSE (CD) 2024-2025 14

AI Powered Medical Diagnosis
Data Augmentation
Data augmentation is a technique used to artificially expand the size of a dataset by creating
modified versions of the original data. For MRI images, augmentation improves model
generalization and robustness by simulating variability in the data.
1. Why Data Augmentation for MRI?
• Addresses overfitting by increasing dataset diversity.
• Simulates variations in acquisition conditions (e.g., orientation, noise).
• Enhances model robustness to unseen data.
CNN Model
A Convolutional Neural Network (CNN) is a type of deep learning architecture particularly
effective for image-related tasks, such as image classification, segmentation, and object
detection. CNNs are designed to automatically and adaptively learn spatial hierarchies of
features from images, making them highly effective for tasks involving medical imaging like
MRI scans.
Transfer Learning Models
Transfer learning is a powerful technique in deep learning where you leverage a pre-trained
model (trained on a large dataset) and fine-tune it on a new, smaller dataset. This is especially
beneficial for medical imaging tasks like MRI analysis, where annotated data can be limited.
Transfer learning can significantly speed up the training process and improve model
performance.

4.1.2 User Interactions

User
As the primary end-user of the application, the User is at the core of the system's functionality.
They have access to the following features:
Choose Image
User can choose the required image he wants to detect if the MRI scanned image is affected
with brain tumor or not.
Upload the Image
After choosing the image the user can upload the image selected and detect the brain tumor.

Dept. of CSE (CD) 2024-2025 15

AI Powered Medical Diagnosis
4.2 ACTIVITY DIAGRAM

Fig 4.2: Activity Diagram

The Activity Diagram represents the operational workflow of the AI-powered medical
diagnosis, breaking down the entire process from user input to generating actionable outputs. It
outlines how users interact with the system, how data is processed, and how the system delivers
outputs.
Below is an elaboration of the workflow steps:
1. Patient

A patient is the one who is receiving medical care or treatment from a health care provider.
2. Patient Medical Image

It is the image of patient’s medical report like MRI’s, X-rays, CT scans etc.
3. Medical Image Database

Medical image databases are essential resources for developing AI-powered diagnostic models.
They provide high-quality datasets of medical images, often annotated by experts, to facilitate
training, validation, and testing of algorithms.
4. Image Information

Image information provides information related to the patient’s report. It gives information
about the tumors, fracture. Image can be in the form of MRI’s, X-rays, CT scans
5. Preprocessing

Preprocessing in Medical Imaging is a crucial step that ensures the data is clean, standardized,
and suitable for training AI models. Medical images like MRI, CT, or X-rays often require
specialized techniques due to their complexity and the need for high accuracy in diagnosis.
6. Feature Extraction

Feature extraction in medical imaging is the process of identifying and isolating relevant
information from medical images to feed into machine learning models for tasks like
classification, segmentation, or diagnosis. It involves converting raw image data into
meaningful and discriminative features that represent patterns, structures, or abnormalities.

Dept. of CSE (CD) 2024-2025 16

AI Powered Medical Diagnosis
7. Classification

Classification in medical imaging involves categorizing medical images or their regions into
predefined classes based on features extracted from the data. It is a core task in AI-powered
medical diagnosis, used to identify conditions, detect abnormalities, and assist in decision-
making.
8. Output

Gives information related to the scanned reports and tells us if the patient is detected with brain
tumor, bone fracture, or lung cancer.

Fig 4.3: Activity Diagram for Text Generator

User
The user will ask the text generator to generate the texts that the patient will tell in the future
when the patient is unable to speak due to his medical conditions. The text will be generated
based on the patient’s previous documents like how patient speaks, what maybe the words
patients use, etc.,
Knowledge Base
A library which carries information about the patient’s previous records of medical conditions,
way of speech, what the patient might tell in the next few sentences. It stimulates human
conversation with a user, either through texts.
Server
Server also known as AI answers all the queries that are asked by the patient and helps the
patients.

Dept. of CSE (CD) 2024-2025 17

AI Powered Medical Diagnosis
4.3 ARCHITECTURE DIAGRAM

Fig 4.4 Architecture diagram

The Architecture Diagram outlines the complete operational framework of the AI- powered
medical diagnosis, detailing the data flow and interactions between various components. This
architecture ensures efficient data handling, predictive accuracy, and real- time responsiveness,
creating a robust and user-friendly system. Here’s an expanded explanation of each stage:
1. Medical Dataset (Disease symptoms)
A medical dataset for disease symptom analysis encompasses a comprehensive collection
of raw medical data essential for training AI models. This dataset typically includes various
types of patient information to capture the complexity of medical conditions. Patient data
comprises demographic details such as age and gender, along with medical history and
reported symptoms. Imaging data often includes modalities like MRI, CT, and X-ray scans,
providing visual insights into a patient’s condition. Lab results, such as blood tests and
biomarker measurements, contribute valuable quantitative data. Additionally, diagnosis
labels indicating whether a patient is healthy or affected by a specific disease are integral
for supervised learning, enabling the model to identify patterns and make accurate
predictions.

2. Pre-processing

• Data preprocessing is a critical step in preparing medical datasets for effective

analysis and improving model performance. This process ensures the data is clean,
standardized, and formatted appropriately for machine learning tasks. Key
components include data cleaning and data transformation.
• Data Cleaning involves resolving inconsistencies such as missing or erroneous
values, duplicate records, and irrelevant features. For instance, missing lab results
may be filled with average values, while outliers in medical imaging data might be
removed to maintain data integrity. Ensuring a consistent and accurate dataset helps
reduce noise and improve model learning.
• Data Transformation focuses on converting the data into suitable formats and
scales for machine learning algorithms. This includes normalizing numerical data,
such as scaling MRI scan intensity values to a 0–1 range for consistency, and
encoding categorical variables, such as converting "male" and "female" labels into
binary values (0 and 1). Additionally, data augmentation techniques, such as
rotating, flipping, or adding noise to MRI scans, enhance model generalization by
exposing the model to varied representations of the data.

Dept. of CSE (CD) 2024-2025 18

AI Powered Medical Diagnosis
3. Disease Symptoms Feature Vector
A Disease Symptoms Feature Vector represents the structured conversion of raw medical
data into a machine-readable numerical format, enabling machine learning models to
process and learn from the data effectively. The creation of this feature vector involves
several key steps:
• Feature Selection: This step focuses on identifying the most relevant features from
the dataset that contribute significantly to disease detection or diagnosis. Examples
include selecting specific symptoms or image regions indicative of a condition, such
as abnormal growths or lesions.
• Feature Engineering: New and meaningful features are derived from existing data
to enhance model performance. For instance, calculating the size, shape, or texture
of a tumor in medical imaging data can provide critical insights that improve model
predictions.
• Feature Representation:
o Tabular Data: Symptoms, test results, and demographic information are
transformed into numerical vectors suitable for ML models.
o Imaging Data: Pixel or voxel-level features such as edges, shapes, and
textures are extracted. Alternatively, deep learning approaches leverage
Convolutional Neural Networks (CNNs) to automatically extract high-level
features from raw image data, capturing complex patterns essential for
accurate disease detection and analysis.

4. Apply Desired Machine/Deep Learning Model

Apply Desired Machine/Deep Learning Model is the step where a suitable predictive model
is selected and trained on the feature vector created from the medical data. The choice of
model depends on the nature of the dataset and the problem being addressed.
• Machine Learning Models are typically employed for smaller datasets or tabular
data, where traditional algorithms such as Random Forest, Support Vector Machines
(SVM), and k-Nearest Neighbors (k-NN) are effective. These models are less
computationally intensive and work well when the relationships between features
are simpler or less complex.
• Deep Learning Models, on the other hand, excel in handling large datasets,
particularly in tasks involving complex data such as images or time-series.
Convolutional Neural Networks (CNNs) are commonly used for image data, like
MRI scans, as they are capable of extracting hierarchical features from raw pixel
data. Recurrent Neural Networks (RNNs) are suitable for sequential or time-series
data, such as ECG signals, where temporal dependencies play a crucial role.
Transformers, known for their ability to model complex relationships, are
increasingly used for multimodal data that combines different data types, such as
imaging and clinical text data, to achieve superior performance across diverse tasks.

5. Prediction Model
The Prediction Model is the final output of the training process, designed to predict disease
labels for new, unseen data based on the patterns learned during training. The structure of
this model typically consists of several layers:
• Input Layer: This layer receives the feature vectors, which are the numerical
representations of symptoms, medical images, or other relevant data. It serves as the
entry point for the data to be processed by the model.

Dept. of CSE (CD) 2024-2025 19

AI Powered Medical Diagnosis
• Hidden Layers: These layers consist of multiple nodes or neurons that process the
input data through learned patterns. In deep learning models, these layers help the
model extract hierarchical features and complex relationships from the data,
enabling it to make accurate predictions.
• Output Layer: The output layer generates the final prediction, such as a disease
label or probability, depending on the type of task (e.g., binary classification, multi-
class classification). This is where the model provides its inference based on the
processed data.
This structured approach allows the model to learn from historical data and make
predictions on new medical cases, assisting in tasks like disease diagnosis, symptom
analysis, or risk assessment.

6. Medical Test Data

Medical Test Data refers to the unseen data used to evaluate the performance and reliability
of a trained model. This dataset is similar to the training dataset in terms of content but is
not used during the model's training phase, ensuring that it serves as a genuine evaluation
tool. The primary purpose of using test data is to assess how well the model generalizes to
new, unseen cases, providing insights into its ability to make accurate predictions in real-
world scenarios. By testing the model on this data, we can evaluate its effectiveness in
diagnosing diseases or predicting medical outcomes, ensuring it performs reliably and
robustly outside the training environment.

7. Test Data Pre-Processing

Test Data Pre-Processing involves applying the same preprocessing techniques to the test
data as those used for the training data. This ensures that the model can process the test data
in the same way it handled the training data, maintaining consistency and compatibility.
Key steps in test data preprocessing include:
• Data Cleaning: Similar to the training data, the test data should be free from
inconsistencies such as missing values, outliers, or duplicates. This ensures that the
model evaluates clean, reliable data.
• Data Transformation: The test data must undergo the same transformations as the
training data. This includes normalizing numerical values, resizing images, or
encoding categorical variables in a consistent manner to align with the model's input
requirements.
The purpose of this preprocessing is to guarantee compatibility between the test data and
the trained model, ensuring that the evaluation is fair and the model performs accurately on
new, unseen data.

8. Test Data Feature Vector

Test Data Feature Vector is the process of converting the preprocessed test data into feature
vectors, following the same procedure used for the training data. This step ensures that the
test data is represented in a machine-readable numerical format, suitable for input into the
prediction model.
The purpose of creating a feature vector from the test data is to ensure consistency with the
training data's representation. This allows the model to process the test data in the same way
it processed the training data, enabling accurate evaluation and prediction. By converting
the test data into feature vectors, the model can make predictions on new cases, ensuring
that it generalizes well to unseen data and provides reliable results.
Dept. of CSE (CD) 2024-2025 20
AI Powered Medical Diagnosis
9. Disease Predicted
Disease Predicted refers to the final output generated by the model after processing the test
data, representing the model's prediction regarding a patient's condition. This output
indicates the model's assessment of whether a disease is present and, in some cases, the
specific disease diagnosed.
Examples of disease predictions include:
• Binary Classification: The model predicts whether a patient is healthy or diseased,
often used for tasks like detecting the presence or absence of a particular disease
(e.g., tumor vs. no tumor).
• Multi-Class Classification: The model identifies specific diseases, such as
classifying a patient’s condition into one of several possible diseases (e.g., types of
cancer, neurological disorders, or cardiac conditions).
These predictions form the core output of the system, helping healthcare professionals make
informed decisions based on the model's analysis.

Dept. of CSE (CD) 2024-2025 21

AI Powered Medical Diagnosis
CHAPTER 5
IMPEMENTATION
5.1 OVERVIEW OF PROJECT MODULES

The implementation phase is the core of the system's development, detailing the technical
realization of its features and functionality. The following sections elaborate on the system's
various modules and how they interconnect:

Brain Tumor Detection

Brain tumor detection is a critical application of artificial intelligence (AI) medical imaging. By
leveraging techniques like deep learning and transfer learning, AI models can assist radiologists
and medical professionals in diagnosing and classifying brain tumors with high accuracy and
efficiency.

The Brain Tumor Detection Workflow involves several key steps that ensure effective
identification and classification of brain tumors from medical images. Below is a breakdown of
the workflow:
1. Data Collection:
The first step in the workflow is to gather the necessary medical imaging data. The most
commonly used imaging techniques are:

• MRI (Magnetic Resonance Imaging), which provides detailed images of the

brain, commonly used for detecting brain tumors.

• CT (Computed Tomography) scans, which are useful for visualizing tumors,

especially in complex or acute cases.
2. Data Preprocessing:
Raw medical images are converted into a standardized format suitable for analysis. The
preprocessing steps include:
• Resizing: Standardize the image size (e.g., 256x256 pixels) to ensure
consistency across the dataset.

• Normalization: Scale pixel values (e.g., between 0 and 1) for better model
performance and faster convergence.

• Segmentation: Extract regions of interest (ROI), such as the tumor area, from
surrounding brain tissue to isolate the tumor.
• Augmentation: Apply transformations such as rotation, flipping, and noise
addition to artificially expand the dataset and improve model generalization.
3. Feature Extraction:
Extract meaningful patterns that represent the tumor’s properties, which are crucial for
classification. These features include:
• Shape: Tumor size, boundary irregularities.
• Texture: Intensity variations in the tumor, helping differentiate between benign
and malignant tumors.
Dept. of CSE (CD) 2024-2025 22
AI Powered Medical Diagnosis

4. Model Building:
AI models are created to classify and segment brain tumors. Common techniques
include:

• Convolutional Neural Networks (CNNs), which are highly effective for image
classification and feature extraction from medical images.

• Transfer Learning, which involves fine-tuning pre-trained models such as

VGG16, ResNet, or Inception on brain tumor datasets, saving time and
improving model performance by leveraging existing knowledge.
5. Model Training:
The AI model is trained using labeled datasets, which include images of tumors (labeled
as specific tumor types, e.g., gliomas, meningiomas, pituitary adenomas) and non-tumor
images. The process involves:
• Inputting the preprocessed MRI or CT images into the model.
• Using labeled data to train the model for both classification (tumor vs. non-
tumor) and segmentation (identifying and outlining tumor areas).
6. Model Testing:
Once trained, the model is validated on unseen data to evaluate its accuracy, robustness,
and generalization ability. This helps ensure that the model can accurately predict
outcomes on new, real-world data.
7. Classification:
The final step involves classifying images as either tumor or non-tumor. For tumor
images, the model might also classify the tumor into specific types, helping in diagnosis
and treatment planning. This stage is crucial for enabling healthcare professionals to
make informed decisions based on the model’s predictions.

Bone Fracture Detection

Bone fracture detection is an important application of AI in medical imaging, aiming to assist
radiologists and healthcare professionals in identifying fractures efficiently and accurately. By
analyzing X-rays, CT scans, or MRI images, AI models can detect fractures, classify their
severity, and recommend further action.
The workflow for bone fracture detection includes the following key steps:
1. Data Collection:
Different imaging modalities are used for bone fracture detection:

• X-rays are the most common and widely used for detecting bone fractures.

• CT scans provide detailed 3D views, particularly useful for complex fractures.

• MRI is used when soft tissue damage is also present along with fractures,
offering detailed insights into surrounding structures.

Dept. of CSE (CD) 2024-2025 23

AI Powered Medical Diagnosis
2. Data Preprocessing:
This step enhances the quality of images and ensures consistency across the dataset for
better model performance.

• Image Resizing: Standardize image sizes (e.g., 224x224 pixels) for consistency,
especially for CNN models.

• Normalization: Scale pixel values to a specific range (e.g., between 0 and 1) to

make training more efficient.

• Cropping: Focus on regions of interest (e.g., specific bones like the wrist,
elbow, or ankle) to reduce unnecessary background information.

• Image Augmentation: Apply transformations like rotation, flipping, and

contrast adjustments to artificially increase dataset diversity, helping the model
generalize better.
3. Feature Extraction:
AI models can automatically learn relevant features, but domain-specific features can
also be manually extracted:

• Edges and Contours: Detect sharp changes in bone structure that may indicate
fractures.

• Fracture Lines: Identify thin lines or discontinuities in the bone structure,

indicative of fractures.

• Bone Density: Assess variations in intensity, which could signal a fracture.

4. Model Building:
Several AI models are used for fracture detection:

• Convolutional Neural Networks (CNNs) are effective for extracting spatial

features from images for fracture detection.

• Transfer Learning: Leverage pre-trained models like ResNet, VGG16, or

Inception, which are fine-tuned on bone fracture datasets for improved
performance.

• Object Detection Models: Faster R-CNN or YOLO can be used to not only
detect fractures but also localize them within the image.
5. Model Training:
The AI model is trained using labeled datasets of bone images. The steps involved
include:

• Inputting preprocessed images into the model.

• Training the model with labels like "fracture" or "no fracture" and possibly
categorizing fractures into types (e.g., hairline, compound).

• Optimizing the model using appropriate loss functions, such as binary cross-
entropy for classification tasks.

Dept. of CSE (CD) 2024-2025 24

AI Powered Medical Diagnosis
6. Model Testing:
The model is validated on unseen test data to assess its generalization ability and
performance.

7. Classification:
The final step involves classifying the images:

• Fracture Detection: Determine if a fracture is present or absent in the image.

• The model may also classify the fracture type or its severity, aiding healthcare
professionals in
diagnosing and treating bone injuries.

Lung Cancer Detection

Lung cancer detection using AI is revolutionizing the medical field by enabling early diagnosis,
accurate classification, and efficient management of cancer cases. AI models analyze medical
images, such as CT scans or X-rays, to identify cancerous nodules and assess their malignancy.
The typical workflow for lung cancer detection includes the following steps:
1. Data Collection
Various imaging modalities are used to collect lung images for AI model training and
evaluation:

• CT-Scans are considered the gold standard for detecting lung nodules, offering
detailed 3D views of the lungs.

• X-rays are often used for initial screenings but are less sensitive than CT scans
in detecting small or early-stage nodules.
2. Data Preprocessing
The purpose of this step is to improve image quality, standardize formats, and prepare
data for use by AI models:

• Resizing: Standardize image dimensions (e.g., 224x224 pixels) for consistency

across the dataset.

• Normalization: Scale pixel intensities to a common range (e.g., 0-1) for better
model convergence and performance.
3. Feature Extraction
This step identifies key patterns in lung images that are indicative of cancer:

• Nodule Size: Categorize nodules into small (<3mm), medium (3–30mm), or

large (>30mm) to assist in determining their potential risk.

• Shape and Texture: Malignant nodules often have irregular shapes or

spiculated (star-like) textures, which are extracted as features.

• Location: Nodules located near the bronchial tree or lung periphery may suggest
malignancy.

Dept. of CSE (CD) 2024-2025 25

AI Powered Medical Diagnosis
4. AI Modelbuilding
AI models analyze the extracted features to classify or segment lung cancer nodules:

• Convolutional Neural Networks (CNNs) are commonly used for this task, as
they are effective at automatically extracting spatial features from medical
images, especially for tasks like classification or segmentation.
5. Training
AI models are trained using annotated datasets where nodules are labeled as benign or
malignant:

• The training process involves feeding labeled CT scans or X-rays into the model.

• The model is trained for classification tasks (distinguishing cancerous from non-
cancerous nodules) or segmentation tasks (identifying the exact boundaries of
the cancerous areas).
6. Testing and Validation
After training, the AI model is tested and validated using unseen data to assess its
generalization capability and performance. This step ensures the model’s reliability
when deployed for real-world use.
7. Classification
Once trained, the AI model classifies nodules into:

• Benign: Non-cancerous growths that do not require aggressive treatment.

• Adenocarcinoma: Cancerous tumors that require further investigation or

treatment, such as surgery, or radiation therapy.

• Squamous Carcinoma: Cancerous tumors that require further investigation or

treatment, such as surgery, or radiation therapy.

Text Generator
A text generator has been developed to assist patients who are unable to complete their
sentences due to certain medical conditions. It analyzes previous data to predict and generate
the next words in a sentence.
The typical workflow for text generation includes the following steps:
1. Data Collection from the Patient
Data collection ensures that the text generator adapts to the patient's communication
style, vocabulary, and medical context.

• Text Sources: Previous medical transcripts or conversation logs. Typed or

spoken words captured through assistive devices. Patient diaries or notes if
available.
2. Tokenization of the Data
It assigns the numbers to the words.

Dept. of CSE (CD) 2024-2025 26

AI Powered Medical Diagnosis
3. LSTM (Long Short-Term Memory) Model
LSTM models are widely used for text generation due to their ability to capture long-
range dependencies in text sequences.

• Memory Cell: Retains important information over time.

• Forget Gate: Discards irrelevant information.
• Input and Output Gates: Manage what new information is stored and used for
predictions.
• Input Layer: Accepts the tokenized data.
• LSTM Layers: Learn sequence patterns to predict the next word.
• Dense Layer: Outputs probabilities for the next word prediction.
4. Prediction

Once the LSTM model is trained, it can predict the next word or sequence of words
based on patient input.

• Input: The model receives a partially completed sentence from the patient.
• Processing: The LSTM generates probabilities for the possible next words.
• Output: The word with the highest probability is chosen and presented.

5.2 TOOLS AND TECHNOLOGIES USED

The implementation of the system relies on a robust set of tools and technologies, ensuring
functionality, scalability, and efficiency. These tools span various aspects of development, from
the frontend to backend, machine learning. Below is a detailed explanation of each:

Frontend
The frontend is designed to offer users an intuitive and responsive interface, ensuring smooth
navigation and interaction.

HTML/CSS:

➢ HTML: Provides the structural foundation of the web pages. It defines the layout and
organizes
➢ elements like forms, tables, and buttons for user input and interaction.
➢ CSS: Enhances the visual appeal of the application by enabling styling and formatting.
It ensures the interface is not only aesthetically pleasing but also adaptable to various
screen sizes, making it responsive for mobile and desktop users.

Django Templates:
➢ A powerful feature of the Django framework, templates dynamically render web pages
based on the system's backend logic.

Dept. of CSE (CD) 2024-2025 27

AI Powered Medical Diagnosis
Backend
The backend serves as the engine of the application, managing user requests, performing
required activities that the user wants, and connecting with the database.

Python:

➢ Python is chosen for its versatility, extensive libraries, and ease of integration with
machine learning workflows.
➢ It handles various backend tasks.

Django Framework:

➢ Django simplifies the development process by offering built-in tools for routing, request
handling, and middleware integration.
➢ It ensures that the backend is robust, secure, and capable of handling high traffic without
performance degradation.

Machine Learning
Machine learning drives the system's predictive capabilities, enabling personalized insights for
users.

• Scikit-Learn:
➢ This library is used to build and train the Random Forest model.
➢ It also supports data preprocessing.
• Pandas and NumPy:
➢ Pandas: Handles structured data.
➢ NumPy: Optimizes numerical computations.
Together, these libraries streamline the preparation of training and testing datasets.

This well-integrated stack of tools and technologies ensures that the AI powered medical
diagnosis system is robust, scalable, and efficient. By leveraging modern development
practices, the system delivers a user-centric experience while giving correct prediction to the
diseases.

Dept. of CSE (CD) 2024-2025 28

AI Powered Medical Diagnosis
CHAPTER 6
TESTING
6.1 TYPES OF TESTS PERFORMED

Testing is an essential part of the system development lifecycle, ensuring that the application is
robust, secure, and performs well under a variety of conditions. Below is an elaborated
explanation of the different types of tests conducted during the development and deployment of
the system:
1. Unit Testing
Unit testing involves testing individual components or functions of the system in isolation
to verify that each part of the system works correctly on its own.

• Purpose:
The primary goal of unit testing is to validate that each function, method, or component
produces the expected results given specific inputs, and that it handles edge cases or
error conditions properly. By isolating components, developers can identify issues at an
early stage, making them easier to fix.

• Implementation:
In this project, key functionalities, including the brain tumor detection, bone fracture
detection and lung cancer prediction, were subjected to unit tests. For example:

o Testing the system that stimulate tumors, fractures, nodules to create diverse test
cases. It also involves radiologists and medical professionals in test validation.
o Ensuring that the system handles invalid data inputs gracefully.

• Outcome:
This testing helped ensure that individual parts of the system were performing
correctly before they were integrated into the larger application.
2. Integration Testing
Integration testing focuses on verifying the interactions between different system
components to ensure that they work together seamlessly.

• Purpose:
This type of testing ensures that once the individual components of the system are
developed and unit tested, they can communicate and work together as expected. It
checks if APIs, databases, and the user interface are functioning as intended when
integrated.

• Implementation:
In this project, integration tests were performed to validate the communication between:
The frontend (user interface) and backend (logic layer), ensuring that data entered on
the frontend was correctly processed and reflected in the backend.
o The backend, confirming that data given by the user (e.g., images given for
detection) was properly saved, updated, and retrieved from the database.

Dept. of CSE (CD) 2024-2025 29

AI Powered Medical Diagnosis
• Outcome:
Integration testing confirmed that the different modules and components could interact
smoothly, providing a seamless user experience.
3. GUI Testing
Graphical User Interface (GUI) testing ensures that the application is user-friendly, visually
consistent, and responsive across different platforms and devices.

• Purpose:
GUI testing verifies that the application’s interface is easy to navigate, functional, and
consistent. It also ensures that users can access all features without encountering bugs
or design flaws.

• Implementation:
Tests focused on the following:

o Usability: Ensuring that users can easily navigate the application, upload data, and
find out the detection of the diseases and even medical chatbot.
o Responsiveness: Verifying that the interface adapts to various screen sizes and
devices, including desktops, tablets, and smartphones.
o Visual Consistency: Ensuring that elements like buttons, forms, and fonts are
consistently displayed across different browsers and screen resolutions.

• Outcome:
GUI testing confirmed that the application was visually appealing, user-friendly, and
compatible with multiple devices and screen sizes.
4. Regression Testing
Regression testing ensures that new code changes, such as feature additions or bug fixes, do
not unintentionally disrupt the functionality of existing features.

• Purpose:
This type of testing is crucial to maintaining the stability of the application. As new
features or bug fixes are implemented, regression tests verify that the changes do not
introduce new issues or break existing workflows.

Dept. of CSE (CD) 2024-2025 30

AI Powered Medical Diagnosis
• Implementation:
Automated scripts were used to perform regression tests on core functionalities,
including
o Data input validation: Ensuring that the system still correctly validates financial
data after updates.

• Outcome:
Regression testing helped ensure that recent updates did not inadvertently break critical
features, maintaining the overall integrity of the application.

By performing these different types of tests, the development team ensured that the system was
functional, scalable, user-friendly, and secure. Each type of testing addressed specific areas of
concern, from the correctness of individual functions to the overall performance under high
loads, and ensured that the system met both user expectations and security standards.

Dept. of CSE (CD) 2024-2025 31

AI Powered Medical Diagnosis
6.2 RESULTS
6.2.1 Home page
The home page of our project which allows the user to choose the required option that is: if the
user wants to detect brain tumor, bone fracture, lung cancer or even generation of texts.

Fig 6.1 Home page

6.2.2 Home page of modules

The home page of modules of our project which allows the user to choose the images that is: if
the user wants to detect brain tumor, bone fracture, lung cancer or even generation of texts. We
need to select the images from the files that is provided to detect the tumor, fracture and even
cancer. The last image generates the text where the user input the number of texts the user wants
and the text is generated.

Dept. of CSE (CD) 2024-2025 32

AI Powered Medical Diagnosis

Fig 6.2 Home page of the modules

Dept. of CSE (CD) 2024-2025 33

AI Powered Medical Diagnosis
6.2.3 Detection of Brain Tumor
This is the result of brain tumor which is been detected from the image we have chosen and it
gives a brief explanation of what has to be done next.

Fig 6.3 Detection of Brain Tumor

6.2.4 Detection of Bone Fracture

This is the result of fracture of the bone which is been detected from the image we have chosen.

Fig 6.4 Detection of Bone Fracture

Dept. of CSE (CD) 2024-2025 34

AI Powered Medical Diagnosis
6.2.5 Detection of Lung Cancer
This is the result of lung cancer which is been detected from the image we have chosen.

Fig 6.5 Detection of Lung Cancer

Dept. of CSE (CD) 2024-2025 35

AI Powered Medical Diagnosis
6.2.6 Text Generation
This is the result of the text generation where we had to input the number of texts to be
generated.

Fig 6.6 Text Generation

Dept. of CSE (CD) 2024-2025 36

AI Powered Medical Diagnosis
CONCLUSION & FUTURE WORK

CONCLUSION
An initiative to introduce an AI-based diagnostic tool in India has the potential to transform
healthcare access, especially in underserved areas that face ongoing challenges with limited
access to medical professionals. The intended outcomes include a broad set of improvements
that together will shape the healthcare landscape to be more inclusive and efficient.
Tackling the shortage of doctors and improving accessibility:
The main objective of the initiative is to improve access to health services, especially in remote
areas suffering from a lack of doctors. Offering a virtual "doctor”; The system aims to bridge
the gap in medical services by providing timely and accurate diagnostic knowledge to residents
of smaller towns and villages.
Quick and timely diagnosis for better health outcomes:
Quick and timely diagnosis of common illnesses such as colds and flu are a key component of
the initiative. This ensures that people receive prompt medical care that improves health
outcomes. Early intervention becomes a key preventive healthcare strategy that reduces disease
severity and reduces the overall burden on the healthcare system.
User-friendly user interfaces for comprehensive health communication:
User-friendly user interfaces for an AI-based diagnostic tool are crucial when health
communication is accessible to people at different levels. This inclusiveness ensures that a
broad user base can take advantage of the tool, encouraging widespread adoption and use. The
AI-Powered Medical Diagnosis System aims to simplify disease diagnosis and improve patient
care by leveraging AI.

Dept. of CSE (CD) 2024-2025 37

AI Powered Medical Diagnosis
FUTURE WORK
While the current version of the AI-powered medical diagnosis provides a solid foundation,
there are several opportunities for further enhancement and expansion to ensure its continued
relevance and efficiency.

Below are some key areas for future work:

1. Enhanced Machine Learning Models:

o Model Optimization: Future iterations could explore the use of more

advanced machine learning algorithms, such as deep learning models or
ensemble methods, to improve the accuracy and reliability.
o Personalization: Further improvements in personalization could be made by
incorporating additional factors.

2. User Experience Improvements:

o Mobile Application: Expanding the platform into a fully-fledged mobile

application (iOS/Android) to ensure users have easy access to their health
data and insights on the go.
o Voice Assistant Integration: Integration with voice assistants like Google
Assistant or Alexa could allow users to interact with the platform hands-free,
checking their health issues and get treatment based on it.

Dept. of CSE (CD) 2024-2025 38

AI Powered Medical Diagnosis
REFERENCES

[1]. S. Kaur et al., "Medical Diagnostic Systems Using Artificial Intelligence (AI) Algorithms:
Principles and Perspectives," in IEEE Access.

[2]. Vinod Kumar; Mohammed Ismail Iqbal; Rachna Rathore, "Natural Language Processing
(NLP) in Disease Detection.

[3]. C. L. Kok, Y. Y. Koh, C. K. Ho, N. T. C. Thanh and T. H. Teo, "Artificial Intelligence in

Cardiology," 2024 IEEE.

[4]. G. K. Thakur, N. Khan, H. Anush and A. Thakur, "AI-Driven Predictive Models for Early
Disease Detection and Prevention," 2024 International Conference on Knowledge Engineering
and Communication Systems (ICKECS).

[5]. Thanoon MA, Zulkifley MA, Mohd Zainuri MAA, Abdani SR. A Review of Deep
Learning Techniques for Lung Cancer Screening and Diagnosis Based on CT Images.

[6]. M. Li, L. Kuang, S. Xu and Z. Sha, "Brain Tumor Detection Based on Multimodal
Information Fusion and Convolutional Neural Network," in IEEE Access.

[7]. Su, Z.; Adam, A.; Nasrudin, M.F.; Ayob, M.; Punganan, G. Skeletal Fracture Detection
with Deep Learning: A Comprehensive Review. Diagnostics 2023.

[8] K. T. Putra et al., "A Review on the Application of Internet of Medical Things in Wearable
Personal Health Monitoring: A Cloud-Edge Artificial Intelligence Approach," in IEEE
Access, vol. 12, pp. 21437-21452, 2024.

[9] Cè M, Irmici G, Foschini C, Danesini GM, Falsitta LV, Serio ML, Fontana A, Martinenghi
C, Oliva G, Cellina M. Artificial Intelligence in Brain Tumor Imaging: A Step toward
Personalized Medicine. Curr Oncol. 2023.

[10] Kutbi M. Artificial Intelligence-Based Applications for Bone Fracture Detection Using
Medical Images: A Systematic Review. Diagnostics (Basel). 2024.

[11] B. O'Sullivan, J. Brierley, D. Byrd, et al., The TNM classification of malignant tumours-
towards common understanding and reasonable expectations, 2017.

[12] Y. Xu, A. Hosny, R. Zeleznik, et al., Deep learning predicts lung cancer treatment
response from serial medical imaging, 2019

Dept. of CSE (CD) 2024-2025 39