Bone Fracture Identification With Deep Learning Model Using Resnet50
Bone Fracture Identification With Deep Learning Model Using Resnet50
1
Bone Fracture Identification with Deep
2
3
Learning Model using Resnet50
#1 #2 #3 #4
4
Mr. Imran Khan M , Ms. Pavithra P , Ms. Arthi J ,Ms. Reshma J
5
Department of Computer Science and Engineering
6
Manakula Vinayagar Institute of Technology, Puducherry, India.
7
#1
8
[email protected]
9
10
Abstract: Since X-ray image interpretation being subjective, bone fractures present
11
substantial obstacles for medical diagnosis and can occasionally result in inaccurate
12
diagnoses and treatment delays. Our proposal involves using convolutional neural
13
networks like ResNet50 in a machine learning approach to tackle this problem. Through
14
the development of a reliable system for automated fracture identification and
15
classification, our method seeks to increase diagnostic accuracy and lessen reliance on
16
17
human diagnosis. Through the use of a dataset from the MURA collection to train our
18
deep learning model, we have created an effective tool that can accurately diagnose a
19
variety of bone fracture forms. Fast uploading of X-ray pictures is made possible by the
20
user-friendly interface, which enables quick predictions on the existence and
21
categorization of fractures. Additionally, our approach improves clinical decision-
22
making by offering customized therapy suggestions based on the examination of these
23
photos. Our model has performed exceptionally well in evaluations, with 95% accuracy
24
rate in fracture classification and identification. These results demonstrate the efficacy
25
of our approach in improving clinical diagnostic performance and patient outcomes.
26
Our ultimate objective is to optimize the diagnostic procedure, relieving the time-
27
consuming workload for healthcare providers and guaranteeing prompt and precise
28
patient care. In final analysis, the urgent demand for trustworthy automated systems for
29
30
bone fracture detection is addressed by our research. We want to transform medical
31
imaging and open the door to better patient outcomes and healthcare delivery by
32
utilizing AI and machine learning.
33
34
Keywords: Bone fracture, ResNet50, DL, Recommendation, Feature Extraction.
35
36
bones. In order to inform treatment
37
choices and guarantee the best possible
38
I. INTRODUCTION
39
care for patients, it is essential to
40
accurately identify and categorize
The human body consists of several
41
fractures in X-ray images. Small
42
types of bones that support the body's
fractures may still be Complex to detect
43
structural integrity and safeguard
because manual fracture detection is
44
important vital organs like the brain,
time consuming. Even though it can be
45
heart, and lungs. Due to their brittle
46
difficult to manually identify small
nature, these bones fracture easily in
47
fractures because of the time-consuming
situations Plunges and roadway
48
nature of the process and the high error,
49
accidents. Our body contains 206
doctors routinely use X-ray images to
50
different types of bones, each with its
evaluate bone fractures. Healthcare
51
own specific size, shape, and set of
52
practitioners require automated systems
properties giving medical professionals
53
to aid in fracture diagnosis because
vital insights into the internal workings
54
manual interpretation of these images is
of the human body in the context of
55
laborious and prone to errors. Much
56
contemporary healthcare. X-ray imaging
attention has been paid to the creation of
57
is a modality that is particularly useful
deep learning models specifically
60
for diagnosing a wide range of medical
61
designed for medical image analysis.
conditions, including fractures of the
62
These models have the potential to
63
64
65
1
improve diagnostic accuracy and Bone Fracture detection Using Deep
streamline workflows by automatically Learning in X-Ray images Leonardo
detecting and classifying abnormalities Tanzi [1]. The identification and
through the use of artificial intelligence categorization of bone fractures has
[15]. A potential application of deep received a lot of attention lately, and
learning techniques of bone fractures in several researchers have put up various
X-ray images. To tackle the problem of solutions to address this issue. In order
bone fracture detection and to identify the advantages of each
classification in X-ray images, a novel research and attempt to draw a
DL model is presented in this work [10]. generalized approach, we will assess and
We first go over the significance of examine a number of publications that
automated fracture detection systems in were selected based on their typical
healthcare settings as well as the methodology and in which the authors
function of medical image analysis. used various deep learning approaches
Utilizing the ResNet50 architecture for to categorize bone fractures. When it
feature extraction and classification, we comes to classifying bone fractures, DL
present our suggested system, which and CNN in particular has recently
builds upon this framework. shown outcomes that are on part with
human performance.
Apart from the precise identification and
categorization of bone fractures in X-ray Bone fracture detection using CNN irfan
pictures, our system has a module for khatik et al [2]. Digital x-rays that are
treatment advice that gives doctors specifically processed for bone fractures
useful information based on the kind and may result in lower diagnostic costs.
extent of fractures found. This module Additionally, this type of processing
improves clinical decision-making by might help a non-orthopedic or tiny
providing information on suitable clinician in a remote location detect and
medical procedures that are grounded in treat a bone fracture. It summarizes the
recognized guidelines and expert results with regard to certain bone
knowledge. A sizable dataset of X-ray fractures and evaluates the current CNN
pictures was used to assess the efficacy techniques employed in bone fracture
of our system, and the findings showed detection. Since there isn't yet a single,
an astounding 95% accuracy rate in universal method to detect fractures in
fracture diagnosis and categorization. various bone types, this review
These results highlight how trustworthy demonstrated the existence of several
and strong our method is in correctly methods for applying CNN and
detecting fractures and directing medical transfers learning to identify fractures in
interventions. Healthcare practitioners various bone kinds. There is currently no
can enhance clinical decision-making in general approach to cover all scenarios
the area of diagnosing and treating bone involving bone fractures in the ML
fractures by utilizing our technology. domain.
Improving the accuracy and efficiency
of patient care through the integration of Fracture Detection in X-ray using CNN
automated fracture diagnosis and Rinisha Bagaria et al [3]. This project is
treatment recommendation capabilities about a DL technique for detecting
eventually improves patient outcomes. various types of bone fractures and for
early detection of bone illnesses using
II. RELATED WORKS X-ray pictures. The convolutional neural
network model's efficiency in
differentiating between bone fractures
2
and healthy bones is employed. The Using artificial intelligence to identify
number of eras, batch quantity, kind of bone fractures Sultan Al Maskari et al
optimizer, and learning are among the [6]. Scientists, doctors, and business
important aspects that are taken into professionals are starting to see more
consideration while selecting the and more use of artificial intelligence
optimal model. With a specificity of 89, (AI), particularly in light of recent
it is therefore discovered that the advancements in deep learning (DL).
convolutional neural network model Recent published publications have
performs well. shown the value of DL for radiographic
assessment bone fracture identification.
Automatic Bone Fracture Prediction The current state of DL should be
Using Convolutional Neural Network known to practicing physicians because
Thaiyalnayaki et al [4]. Bone fractures it may soon have an impact on clinical
are common in humans and can happen operations. This article will give a
from a minor mishap or from extreme practicing clinician an idea of the
pressure being placed on the bone. current advancements in AI fracture
Because of this, a precise evaluation of a diagnosis by reviewing the most recent
fractured bone is essential in the medical research on the subject. Searching
field. Using information from CT and X- electronic databases, we located relevant
ray images, this research aims to create studies regarding AI's application in
an image processes-based system that bone fracture detection.
can quickly and effectively identifying Bone Fracture Segmentation in X-ray
fractured bones. Fuzzy borders and a lot Images Using a U-net Deep Learning by
of data in MR images make tumor Komal Ghoti et al. [7] Sophisticated
categorization and segmentation bone fracture segmentation technique
challenging. MR and CT scan data sets developed with deep learning is an
include much too much information for essential part of the medical imaging
human analysis and comprehension. The system. Bone fracture segmentation is
ability to precisely identify the location the process of identifying the various
and extent of a fractured bone is tissues that are fractured and those that
essential for making a fracture diagnosis. are not. Fractures can occur in the upper
The four steps of the diagnostic process extremities, including the elbow,
include feature extraction, classification, shoulder, fingers, wrist, hand, humerus,
and pre-processing of MR images. and forearm, to name a few. X-rays are
an imaging modality that is commonly
Analysis of Bone Fractures Using used to see and assess the bone
Machine Learning Techniques Ayesha architecture of the upper extremities. X-
Noureen et al[5]. Bone fractures are a rays are required for both the diagnosis
common condition in humans. Thus, this and the planning of treatment for a
study offered a practical method for fractured bone. Researchers have
treating bone fractures that incorporates concentrated on the subject of
cutting-edge technology. The utilization computational bone fracture
of a Deep Learning model is suggested segmentation over the previous ten years
as the answer. Google Colab was used to because of the broad. A multitude of
construct the suggested model. Several fully and partially automated methods
experiments were conducted in order to have been introduced, and their
train the suggested model. The accuracy advancement is steady. A promising
of the model was eighty-four percent. segmentation result is obtained using a
unique CNN-based deep learning
algorithm. This approach makes use of
3
the Musculoskeletal Radiographs both fractured and non-fractured bones.
(MURA) database. The CNN-based U- In the end, it was found that the SVM
Net model is trained using the MURA classification method works better than
Database. EBP-NN.
4
Fig:1 System Architecture
5
identify and analyze bone fractures in X- for bone fracture detection is provided
rays Fig.1.It is an essential below:
preprocessing step in the automated 1. Load the input images with the X-
fracture detection systems workflow or ray scans of the bones.
in the radiologists interpretation of 2. Using the Gaussian kernel formula,
medical Images. apply the Gaussian filter to every
image.
Formula for Gaussian filter 3. Feed the deep learning network
ResNet50 for bone fracture detection
with the Pre-processed images.
1) Noise reduction
A neural network's learning process may
be hampered by the frequent noise
present in medical images. If Gaussian
Here equation 1 represent the variables as, filtering is used to the images to
represent the intensity minimize noise, the network will have
of the pixel at position (x + i, y +j) in an easier time focusing on relevant
the original image. features related to fractures. Two
I and J represent variable are used as frequent types of noise seen in medical
indices in the summations that iterate images are speckle and Gaussian noise.
over the neighborhood around each
pixel. 2) Smoothing
X and Y variables represent the Gaussian filtering helps to improve the
coordinates of a pixel in the image. overall quality of the images by
K is typically an odd integer smoothing out the pixel intensities,
representing the size of the making them more suitable for network
neighborhood used for smoothing. analysis. By intensifying the contrast
Σ represents the standard deviation between the surrounding tissues and
of the Gaussian distribution. bones, the smoothing process might
enhance the visibility of fractures.
Here equation 2 represents the variables as,
G(x, y) is the value of the Gaussian 3) Important Features
kernel at position(x, y). When adjusting and reducing noise in
The natural algorithm’s base is e. the images, it's important to make sure
is the Gaussian distribution’s that important details like potential
Standard deviation (SD), which fractures Fig.1 and bone structures are
establishes how much smoothing is preserved. Gaussian filtering is useful
applied to the image. for preprocessing medical pictures
because it minimizes noise while
By convolving the picture with the preserving important edges and details
Gaussian kernel using methods like 2D in the image.
convolution, you can apply the Gaussian
filter to an image. Smoother and less D. Feature Extraction
noisy images are the end result, and this When extracting features for a variety of
can help with later processing stages, machine learning applications, including
including bone fracture identification deep learning, Principal Component
with ResNet50. A high-level summary Analysis (PCA) is a popular
of the procedures for using a Gaussian dimensionality reduction method.
filter as part of the data preprocessing Preprocessing the input data with PCA
6
can lower its dimensionality and identify required for the identification of bone
useful characteristics, which can then be fractures.
fed into a neural network for the purpose The features that have been modified
of detecting bone fractures through deep can be fed into the ResNet50 model after
learning. Using a technique called PCA has been used to extract features.
principal component analysis, or PCA, By providing a more condensed and
huge data sets can have their informative representation of the input
dimensionality reduced Fig.1. This is images, the reduced-dimensional feature
achieved by reducing the size of a large vectors have the potential to enhance the
set of variables while maintaining the neural network's performance and
majority of their information. efficiency. Utilizing an appropriate loss
Accuracy naturally suffers when a data function and optimization algorithm,
collection has fewer variables; however, train the ResNet50 model using the
the secret to dimensionality reduction is feature vectors that have been
to compromise a little on accuracy in preprocessed and PCA converted. Take
favor of simplicity. Because machine a look at the trained model's
learning algorithms can analyze data performance in identifying bone
points considerably more quickly and fractures on a different validation set. As
easily when dealing with smaller data necessary, adjust the model's parameters
sets because they are simpler to explore and make it more precise.
and visualize and don't require as many
irrelevant factors. E. ResNet50 Architecture
The ResNet50 network is used to
Formula for PCA organize the type of bones in the image.
To ascertain the bone is fractured, a
1) specific model from three different types
will be loaded after the bone type has
been predicted. Every model was trained
Here equation represent as variables as,
to identify fractures in different types of
is the mean of the data.
bones. This method uses a customized
Σ represents the covariance matrix.
model for every bone to determine if a
n is the number of images (data fracture is there or not, and it makes use
points) in the dataset. of ResNet50 powerful image
X represents the dataset containing classification skills to pinpoint the
the bone fracture images. precise kind of bone. When the results
of the fractured bone identification and
PCA for Feature Extraction: type of bone classification are displayed
Utilizing a Gaussian filter, apply PCA to to the user in the application, they will
the feature vectors of the previously be simple to understand. This approach
processed images. PCA will attempt to has a potential to significantly improve
retain as much of the variance in the the patient diagnosis and cared by
data as it can while converting the high helping medical practitioners identify
dimensional feature space into a lower- bone fractures Fig.2.It’s quick and
dimensional space. The most significant dependable image processing speeds up
directions of variation in the data are the diagnosis processes and helps ensure
represented by the modified features that patients get the care they need.
(principal components) that PCA
extracted. These elements can function
as a condensed version of the original
data, encapsulating the crucial details
7
F1 Score=2*
8
by the F1-Scores of 0.90 for "No between fracture and non-fracture
Fracture" and 0.97 for "Fracture”. The images, the model gives users precise
number of real instances of every class evaluations instantly. After identifying
in the dataset is referred to as support. In fractures, the system creates a
the dataset, there were 525 cases of personalized treatment recommendation
"Fracture" and 175 instances of "No based on the particular kind and degree
Fracture." The model's accuracy, which of the fractures found. This
measures how accurate the forecasts recommendation module suggests
were overall, is 0.95, meaning that 95% appropriate interventions, like surgery,
for each one separately and then taking based on best practices and medical
the average. The macro averages are knowledge.
0.94, 0.93, and 0.93. Giving more
weight to categories with more instances, IV. RESULT AND DISCUSSION
the weighted average first computes the
metrics for all classes and then A. Training and Testing
determines the weighted average The methodology comprised training
depending on the number of true and evaluating a picture dataset to
occurrences for each class. Here, we forecast the result. A subset of 1,451
have weighted averages of 0.95 for images was set aside for testing, out of
accuracy, 0.95 for recall, and 0.95 for the 19,048 images in the training
F1-score Fig.2. This shows the model's dataset. For the prediction challenge, the
overall performance while accounting researchers used a well-liked deep
for the dataset's class imbalance. learning architecture called ResNet50.
The model had a strong performance in
this specific challenge, as evidenced by
its remarkable 95% prediction accuracy.
This degree of accuracy indicates that
the model was very successful in
identifying patterns in the training data
and extrapolating them to the test data,
which was unknown, to provide precise
predictions. Notably, images of the
elbow, hand, and shoulder were included
in the training dataset Table III.
Likewise, images of these identical body
parts were included in the testing dataset
Table IV. The two dataset’s
compositional consistency guarantees
Fig:3 Evaluation Matrix that the model was tested and trained on
comparable kinds of data, which is
The project's primary objective is to essential for correctly evaluating the
create an intuitive user interface for model's performance. In summary, the
finding and classify the bone fractures in research effectively showcases the
medical field X-rays. Treatment
recommendations will then be made in
accordance with the fractures that are
found. Healthcare providers can upload
X-ray images to the user interface for
analysis. This model is probably based
on the ResNet50 architecture or
something similar. By differentiating
9
utilization of deep learning methods, F1-
specifically the ResNet50 model, to Model Accuracy% Precision% Recall%
Score%
forecast results by analyzing picture
data pertaining to distinct body sections. CrackNet 88.39% 89.09% 84.5% 86.73%
The approach's usefulness and
prospective utility in many practical Dilated
84.48% 87.50% 84.85% 86.15%
applications, such medical diagnostics CNN
or biomechanical analysis, are
highlighted by the high accuracy Inception 81.7% 76.2% 92.3% 83.4%
attained.
ResNet50 95% 93% 94.5% 93.5%
Parts Images
Elbow 5133
Hand 5835 Table: V Comparisons of various DL
Shoulder 8082 Models
Table: III Training Dataset The accuracy, precision, recall, and F1-
Score values of the various models were
Parts Images displayed in a graph along with the
Elbow 450 performance of the algorithm Fig.4.
Hand 485
Shoulder 516
B. DL algorithm performance :
In order to ascertain which algorithms
were most effective in precisely
forecasting fractures, the study entailed
evaluating photographs of bone
fractures. The deep learning model built
on top of ResNet50 proved to be the
most accurate of the algorithms that
were tested. A comparative investigation
showed that the ResNet50 model
consistently performed better than the
other models, reaching the greatest
accuracy level. In particular, the crack Fig:4 Performance metrics of DL model
Net model performed well but fell short
of the precision attained by ResNet50, When compared to other models, the
scoring an accuracy of 88.39%. In a algorithm's performance is highest when
similar vein, the Inception model compared to ResNet50.The following
obtained an accuracy of 81.7%, whilst model is the greatest and is followed by
the dilated CNN model reached an Inception and Dilated CNN. It has
accuracy level of 84.48%. The ResNet50 excellent accuracy and precision. Recall
model outperformed the dilated CNN for the Inception is higher than that of
and Inception models in terms of the others at 92.3%. Overall, the results
accuracy, despite their decent indicate that ResNet50 has the highest
performance Table V shows that below. accuracy level Fig.5.
10
makes the technology more useful by
improving accessibility and usability by
expediting the process of entering
photographs and obtaining predictions
and recommendations. Furthermore, the
effectiveness and precision of fracture
diagnosis can be greatly increased by
incorporating such a user interface into
healthcare procedures. Healthcare
practitioners can use artificial
intelligence (AI) and DL to get
objective, data-driven insights rather
than depending just on subjective
Fig:5 Performance Metrics interpretations Fig.7. Moreover, the
diagnostic procedure gains additional
In order to make it easier for users to value from the interface's capacity to
engage with the established model, the suggest treatments. Healthcare providers
project highlights how crucial it is to can more precisely customize their
create an intuitive and user-friendly treatment plans to the projected presence
interface Fig 6. This type of interface of a fracture and the recommendations
provides a means by which physicians that go along with it. This could result in
can upload pictures of bone fractures improved patient outcomes and a lower
with ease and obtain accurate risk of complications.
evaluations concerning the existence or
non-existence of fractures. The interface The emergence of user-friendly
facilitates faster and more dependable interfaces in medical technology signals
information retrieval for practitioners by a revolution in the identification and
streamlining the image upload and management of fractures. Predictive
analysis process. algorithms are seamlessly integrated into
these interfaces, allowing for quick
Furthermore, the interface is essential in image uploads and accurate fracture
providing treatment recommendations prediction. They also offer customized
based on the model's predictions, in therapy recommendations, enabling
addition to its fracture detection practitioners to confidently make well-
capabilities. By helping medical informed decisions. These interfaces
professionals create individualized have the potential to greatly enhance
treatment plans, this function not only patient outcomes and treatment by
helps the system detect fractures but also optimizing the diagnostic process and
improves its usefulness. The technology providing tailored insights. innovation in
enables healthcare professionals to make improving clinical practice as healthcare
well-informed decisions and enhance continues to change. Their powerful
patient care by including treatment machine learning capabilities and
recommendations straight into the intuitive functioning mark a paradigm
interface. For example, the ResNet50 shift in fracture therapy. As the
architecture's sophisticated deep healthcare industry continues to evolve,
learning model and the medical these interfaces show the revolutionary
practitioners who need its insights for potential of innovation in enhancing
clinical decision-making can clinical practice.
communicate with each other through
the interface's ease of use. The interface
11
Fig: 6 User interface to upload image
A. Accuracy
Accuracy in both training and to validate. The accuracy graph represents in Fig.8
The accuracy of the training exceeds in the below.
that of the validation. The model is
overfitting to the training set, which
explains this. When a model learns the
training data including the noise in the
data too well, it is said to have overfitted.
This implies that new, untested data will
not yield good results from the model.
Thus, the model is gaining knowledge
from the training set. But compared to
training accuracy, validation accuracy
does not improve as much. This
indicates that the model does not
perform well when applied to new data.
Fig:8 Accuracy Graph
12
B. Loss outperforms other models in the field,
The both training and validation losses demonstrating the efficacy of our
are plotted against the number of epochs methodology. Using ResNet50 for
training process iterations in the graph classification has shown to be especially
you sent me. The intention is for the successful since it uses its deep neural
validation loss to decrease as well, network design to reliably and precisely
signifying that the model is operating identify fractures. In addition, our
well on unseen data and generalizing interface is easy to use, making it
effectively, and for the training loss to quicker and easier for medical
decrease as the model gains knowledge professionals to upload photographs and
from the training set. receive precise, timely forecasts. In
Positively, the training loss does addition to increasing productivity, this
decrease as the number of epochs interface promotes teamwork among
increases in the graph you sent me. That medical professionals by allowing them
being said, the validation loss first to easily incorporate AI technology into
decreases relative to the training loss their daily procedures. Our solution's
before rising Fig.9. An indication of ultimate goal is to give medical
overfitting is this. When a model learns professionals all the help they need in
the training set too thoroughly including order to diagnose patients more
the noise in the data it is said to be accurately and give them the best care
overfit. possible.
VI. REFERENCES
13
Argenson, J. N., & Ollivier, M. [11] Jin, L. Yang, J., Kuang, K., Ni,
(2021). DL for the radiographic B., Gao, Y., Sun, Y., ... & Li,
diagnosis of proximal femur M. (2020). Deep-learning-assisted
fractures: Limitations and detection and segmentation of rib
programming issues. Orthopaedics fractures from CT
& Traumatology: Surgery scans:Development and validation
& Research, 107(2),102837 of FracNet. EBioMedicine, 62.
[6] Ghoti, K., Baid, U., & Talbar, S. [12] Nissinen, T., Suoranta, S.,
(2021, May). MURA: Bone Saavalainen, T., Sund, R.,
FractureSegmentation Using a U-net Hurskainen, O., Rikkonen,T., ...
DL in X-ray Images. In Techno- & Väänänen, S. P. (2021).
Societal 2020: Proceedings of the Detecting pathological features and
3rd International Conference on predicting fracture risk from dual-
Advanced Technologies forSocietal energy X-ray absorptiometry images
Applications—Volume 1 (pp. 519- using deep learning. Bone
531). Cham: Springer International reports, 14, 101070.Guan, B., Zhang,
Publishing. G., Yao, J., Wang, X., & Wang,
[7] Soft Computing Research M. (2020). Arm fracture detection in
Society.Bagaria, R., Wadhwani, S. X-rays based on improved deep
and Wadhwani, A. (2021). Bone CNN . Computers& Electrical
Fractures Detection using Support Engineering, 81, 106530.
Vector Machine and Error [13] Bagaria, R., Wadhwani, S., &
Backpropagation Neural Network. Wadhwani, A. K. (2020, April).
Optik, 168021. Different techniques for
[8] Pal, D., Reddy, P. B., & Roy, S. identification of a bone fracture in
(2022). Attention UW-Net: A Fully analysis of medical image. In 2020
connected model for automatic IEEE 9th International Conference
segmentation and annotation of chest on Communication Systems and
X-ray. Computers in Biology and Network Technologies (CSNT) (pp.
Medicine, 150, 106083. 327-332). IEEE
[9] Ma, Y., & Luo, Y. (2021). [14] Meena, T., & Roy, S.
Bone fracture detection through the (2022).Bone fracture detection using
two-stage system of crack-sensitive deep supervised learning from
CNN. Informatics in Medicine radiological images: A paradigm
Unlocked, 22, 100452. shift. Diagnostics, 12(10), 2420.
[10] Abbas, Waseem & Adnan, [15] AlGhaithi, A., & Al
Syed & Javid, Dr Maskari, S. (2021). Artificial
& Ahmad, Wakeel. intelligence application in bone
(2021).Analysis Of Tibia-Fibula fracture detection. Journal of
Bone Fracture Using DL Technique Musculoskeletal Surgery and
Of X-Ray Images. International Research, 5, 4.
Journal for Multiscale
Computational Engineering.
14