Multimedia Tools and Applications

https://doi.org/10.1007/s11042-023-16654-6

Advancements in hybrid approaches for brain tumor

segmentation in MRI: a comprehensive review of machine
learning and deep learning techniques

Ravikumar Sajjanar1 · Umesh D. Dixit1 · Vittalkumar K Vagga2

Received: 5 April 2023 / Revised: 29 June 2023 / Accepted: 23 August 2023

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023

Abstract
Magnetic resonance imaging (MRI) brain tumour segmentation is essential for the diag-
nosis, planning, and follow-up of patients with brain tumours. In an effort to increase effi-
ciency and accuracy, a number of machine learning and deep learning algorithms have
been developed over time to automate the segmentation process. Hybrid strategies, which
include the advantages of both machine learning and deep learning, have become more and
more popular as viable options. This in-depth analysis covers the developments in hybrid
techniques for MRI segmentation of brain tumours. The essential ideas of machine learn-
ing and deep learning approaches are then covered, with an emphasis on their individual
advantages and disadvantages. After that, the review explores the numerous hybrid strate-
gies put out in the literature. In hybrid approaches, various phases of the segmentation
pipeline are combined with machine learning and deep learning techniques. Pre-process-
ing, feature extraction, and post-processing are examples of these phases. The paper exam-
ines at various combinations of methods utilised at these phases, such as segmentation
using deep learning models and feature extraction utilising conventional machine learning
algorithms. The implementation of ensemble approaches, which integrate forecasts from
various models to improve segmentation accuracy, is also explored. The research study
also examines the properties of freely accessible brain tumour datasets, which are essential
for developing and testing hybrid models. To address the difficulties of generalisation and
robustness in brain tumour segmentation, it emphasises the necessity of vast, varied, and
annotated datasets. Additionally, by contrasting them with conventional machine learning
and deep learning techniques, the review analyses the effectiveness of hybrid approaches
reported in the literature. This comprehensive research provides information on recent
advancements in hybrid techniques for MRI segmenting brain tumours. It emphasises the
potential for merging deep learning and machine learning methods to enhance the preci-
sion and effectiveness of brain tumour segmentation, ultimately assisting in improving
patient diagnosis and treatment planning.

Keywords Segmentation · Deep learning · Brain tumor · Magnetic resonance imaging ·

Machine learning

Extended author information available on the last page of the article

13
Vol.:(0123456789)
 Multimedia Tools and Applications

1 Introduction

The brain seems to be a noteworthy organ that contains 100 billion neurons, often known
as nerve cells. Brain cancers are listed as the tenth prominent reason of demise in devel-
oped nations including both children and adults [1]. In the USA, 18,280 adult fatalities
from major brain tumors are projected to occur in 2022 [2]. Intracranial cancers, some-
times referred to as brain cancers, are malignant growths that originate in the brain’s intrac-
ranial structures and could vary in severity from mild to severe [3, 4]. As cell reproduction
rates rise and grow uncontrolled, brain tumors develop. Brain cancers can grow wherever
in the skull or brain, along with the skull base, nasal cavity, brainstem sinuses, and numer-
ous other locations. More than 150 different types of brain cancers exist. Brain cancers can
be divided into two categories: malignant and non-tumor [5, 6]. Moreover, three distinct
examinations and processes, involving imaging investigations, neurological examinations,
and biopsies, are required to identify a brain cancer. The use of MRI has become a popu-
lar and commonly used model for diagnosing brain cancers [7]. A dye might be inserted
into a vein while MRI scanning. On the basis of the MRI scan components, such as func-
tional MRI, and perfusion MRI, MR-spectroscopy (MRS), experts evaluate the cancer and
develop treatment recommendations. In certain cases, other imaging procedures like posi-
tron emission-tomography (PET) and computed tomography (CT) have been employed in
conjunction with MRI. Any issue may indicate brain’s which region is being damaged by
the cancer [8]. Neurological testing in this situation can aid the expert in making a more
accurate prognosis. The specialist performs a neurological assessment on the patient, test-
ing their balance, reflexes, hearing, strength, coordination, and vision. A biopsy has been
used to make a diagnosis that is more accurate. A sample of aberrant tissue is gathered and
evaluated under a microscopy during this operation [9].
A variety of MR scans, including post-contrast T1-weighted (T1ce), T2-weighted, fluid-
attenuated inversion recovery (FLAIR), and T1-weighted techniques [10] give further
information that aids in segmenting the brain cancer and its neighbouring tissues. The 3
tumor areas that are utilized in the actual clinical system have been: (i) enhancing region
(comprising of whole tumor’s enhancing part), (ii) peritumoral edema area (comprising of
whole tumor’s edema region) and (iii) necrosis region (comprising of tumor’s enhancing
and non-enhancing part). Gliomas, which are brain tumors, develop from glial cells [11].
The two types of it are high-grade glioma (HGG) that has been extremely aggressive and
possibly fatal, and low-grade glioma (LGG) that grows gradually. A patient’s LGG may
have a longer life time and be less intrusive due to the LGG’s apparent high frequency.
Sometimes HGG, more malignant gliomas, can survive for almost two years [12]. Even
with the use of cutting-edge imaging, radiation, and surgical methods, HGG is often incur-
able. Because of cell abnormalities, uneven tissue development, and the complicated and
varied character of HGG, segmenting HGG is a difficult process [13]. The fundamental
goal of digitalized brain cancer detection has been to gather critical clinical statistics on the
specific existence, kind, and location of the injury. Medical imaging results can direct and
control any following therapies, guaranteeing a good cancer identification and treatment
plan. The unregulated, irregular division and growth of bodily cells appears to be referred
to as cancer. A brain tumor forms if there is aberrant cell division and multiplication in the
brain tissue. Although their scarcity, brain cancers have historically been among the worst
malignancies. Brain cancers can either be intrinsic or invasive, based on where they ini-
tially manifested. Although the cells that make up original tumors come from brain tissues,
metastatic tumor may begin somewhere in the organ before extending to the brain.

13
Multimedia Tools and Applications

Furthermore, diagnostic testing significantly improves the efficacy of glioma therapy. It

might be tricky to detect and identify lesions using a lot of scanning and volumes since tumor
features differ from patient-to-patient. This makes treatment complicated. Such images also
include several artifacts [14], which were picked up by using numerous scanners and include
abnormalities, pixel fluctuations, and non-uniformity. In order to manage binary (low and
high) and numerous kinds of risk categories (e.g., weak, mild, medium, and high), a precise
and quick brain tumor segmentation (BTS) approach is required. This will encourage early
identification and cost-effective therapy. In addition, the standard technique was mostly used
to describe the initial generations of BTS [15]. The majority of the 2nd and 3rd generations
were made up of ML- and DL-predicated technology. In regards of how scientists extract char-
acteristics from examples, DL and ML are different. ML classification algorithms frequently
develop independently as analytical learning methods and make accurate forecasts employing
retrieved attributes. Medical imaging systems have demonstrated to benefit from the usage of
ML techniques for tissue characterisation. The effectiveness of ML approaches [16] totally
rests on the researcher or radiologist skill in identifying the most persuasive aspects, leading
to biased methodologies. Thus, DL methods [17] that use different statistical information lay-
ers and convolutional processes to acquire features and deploy forecasts [18] have attracted
the attention of academics. The present responsibilities of clinicians and radiologists may be
altered by DL-based medical image analysis, countenancing them to spend lesser time review-
ing clinical images and far more time establishing diagnostic and therapy decisions. Hence,
when applied to a large database, DL approaches give precise predictions while being more
costly than other methods in regards of space and computing time.
Many scholars have previously evaluated BTS and highlighted ML methods such as
K-Means Clustering, Support Vector Machine (SVM), or DL methods including CNN. For
those new to the topic, Magadza and Viriri’s [19] detailed discussion of DL techniques is help-
ful. Parvathy and Robert [20] present a good literature overview on DL algorithms and con-
ventional automated BTS procedures. The many ML and DL segmentation algorithms used
in brain tumour identification are not covered by these studies, though. A review paper on
BTS has yet to link the most recent ML and DL methods with the tumor diagnostic paradigm,
despite BTS being actively researched for recent years. This motivates to conduct a review on
hybrid segmentation methods employing DL and ML technologies for brain tumor diagnosis.
Over 100 academic research publications from the years 2015 to 2023 have been included in
this study. A statistics report predicated on the “Scopus” repository is supplied to determine
the number of studies on BTS conducted using ML and DL methods. The terms “BTS” and
“ML technique” or “BTS” and “DL method” or “brain cancer” and “technique name” have
been discovered in this repository.
Rest of the article is described as follows: Section 2 explains the BTS and commonly
employed methods; Section 3 reviews the articles related to BTS; Section 4 summarizes the
ML-based BTS; Section 5 summarizes the DL-based BTS; Section 6 reviews the hybrid
methods employed for BTS; Section 7 compares the performance of different hybrid and com-
monly utilized BTS methods; Section 8 details the discussion of review; and Section 9 con-
cludes this article.

13
 Multimedia Tools and Applications

2 Brain tumor segmentation (BTS)

Table 1 displays the various methods utilized for segmenting the image of a brain
tumor. Image segmentation is frequently utilized in both glioma therapy and detection.
In particular, an exact glioma segmentation mask might assist with surgery planning,
postpartum monitoring, and raising success rates [21, 22]. To determine the image seg-
mentation’s effectiveness, researchers explain the chore of segmenting brain tumors as
follows: By categorizing each input data voxel or pixels into a predetermined tumor
region categorization, the system tries to automatically separate the region containing
the tumor from the tissue that is healthy when given an input image from a variety of
imaging techniques (for instance, numerous MRI segments). The structure then returns
the associated data’s segmentation map.

2.1 Types of segmentation

BTS can be broadly categorized into three categories predicated on the extent of human
participation: manual segmentation, completely automated based segmentation, and
semi-automatic depending segmentation. Detailed explanations of these methods are
provided below:

• Manual segmentation

A personal operator utilizes complex tools to physically designate or outline around

tumor sites during manual segmentation. The segmentation outcome’s precision has
been significantly influenced by the training, experience, and knowledge of the human
operator’s analytical architecture. Regardless of being laborious and time-consuming;
moreover, manual segmentation was often employed as the standard for entirely com-
puterized and partially automated segmentation.

• Fully-automatic segmentation

The segmentation of brain tumors has been completely mechanized, requiring no

involvement from humans. Integrating artificial intelligence with previous information
helped to answer the segmentation problems. For entirely computerized segmentation
approaches, creative and discriminative approaches have been added as extra types. Dis-
criminating methods commonly employ supervised learning, which entails discovering
connections between an input image and the human-annotated information contained
in a large amount of data. The organization has employed traditional ML methods that
depend on hand generated characteristics extensively over the years. The complexity of
clinical images may prevent these methods from completely utilizing the training data
[29]. Due to their extraordinary effectiveness in computer-vision utilities and their capa-
bility to effectively derive attributes from images, DL methods have gained greater rec-
ognition recently. On the other hand, standard techniques depend on already-existing
information regarding the pattern of distribution in addition to manifestations of differ-
ent kinds of tissue.

• Semi-automatic based segmentation

13
 Table 1  Techniques for BTS and their views
References Input data Criteria Image segmentation Validation method Components analysed Results

[23] Utilizing MR images in Segmentation For boundary-based When 3DAC is applying BTS The precision as well as
virtual imaging object recognition, for segmentation, WE accuracy of the image
3DACWE was created is intended to identify segmentation are suc-
object boundaries cessfully attained
[24] Method for random paths Homogeneity-and The modelling of the When tumor-containing BTS Shows the growth’s inten-
Multimedia Tools and Applications

Object-characteristics- image makes use of a images have been iden- sity spread
based Random-Walks graphs tified and segmented,
algorithm the HORW method
is used
[25] MRI images Level Set Utilize Level Set for An adaptive method is Identification and Improvement in efficiency
form descriptions used for the depiction dissection of brain
of forms and tracks tumors
that have an effect on
displays
[26] Imaging functionality Application of Condi- MRI image segmentation Small regions referred BTS The splitting of images
with MRI tional Random Fields enhances in locating to as image patches problem has been
the damaged area have been retrieved resolved
from the brain image’s
segmented part
[27] Brain MRI initial data Clustering using ker- Utilizing Clustered The goal is optimized BTS Improved brain images
nelized fuzzy entropy Fuzzy Entropy, the by employing the differentiation
images are divided LHNPSO method
accurately by cluster-
ing
[28] MR structural images Development of the Edge-to-edge layers The BrainNetCNN struc- Utilizing neuroimaging Numerous variations of
BrainNetCNN are utilized for data ture in the illustration the BrainNetCNN design
filtering consists of layers of are being examined
convolution and layers
that are fully connected

13
 Multimedia Tools and Applications

This combines both machine and individual abilities. Users must be involved in start-
ing the classification process, getting input, and evaluating the results is shown in Table 2.
Semi-automatic methods of segmentation still require human intervention to produce the
intended outcomes, regardless of requiring a shorter amount of time than manually seg-
menting techniques.
In order to separate and categorise brain tumours using various types of medical imag-
ing data, the above table of references covers a variety of approaches, including saliency
identification, CNNs, 3D CNNs, MFCM algorithm, and deep learning frameworks.

2.2 Segmentation techniques

There are additional areas where division methods are thoroughly discussed. Figure 1 illus-
trates the BTS techniques utilized. Additionally, a number of well-known division algo-
rithms include the following:

• Method for thresholding

Voxels that are greater than a particular threshold were identified as being a component
of a tumor, as the titles suggest. Thresholding appears to be a useful segmentation tech-
nique that uses a number of threshold numbers for the corresponding histograms and the
grey level along with a set of parameters based on their strength. It was a previous method
for segmenting brain tumors [17]. Segmentation has been made feasible by applying inten-
sity thresholding to images taken for medicinal purposes. The segmentation process yields
certain common challenging activities in the domain of interest. Both local and worldwide
methods are utilized to properly position the tumors. In which it was possible to complete
the entire thresholding-based segmentation because there was a significant amount of con-
sistent strength among the contextual and the item.

• Methods determined by regions

The standardized behaviours of an image are employed as the foundation for the region-
based segmentation, and the image pixels are confirmed. In the evaluation method, the
neighbouring pixels produced by the various areas have been merged in accordance with
similar known occurrences. Region and Watershed development appear to be two distinct
kinds of segmentation depending on regions [39].

3 Technique for growing regions

The segmentation procedure begins with a seed voxel, from which similar voxels are deter-
mined to be a component of the tumor. The simplest and most popular technique of seg-
mentation depending on areas has resulted in region growth. The method, that utilized a
bonded region similar to an actual image pixel, only required one seed to get going. The
neighbouring pixels obtained from that are reinforced into the region-based identical cri-
terion in order to identify the connected regions. The seed utilized was chosen using either
a human or automatic process. To divide tumors and organs in MRI images for clinical
research, utilize region growth. The fundamental framework of the multispectral-image
recognition is unique and involves prior comprehension, fuzzy traits, and modifications

13
 Table 2  Techniques for BTS
References Input data Criteria Image segmentation Validation method Components analysed Results

[30] Brain tumor images Salient-map modifica- The colour difference The saliency identifica- Segmentation of brain The summary of the patch
(original data) tions between the sections tion method is used to tumors variations in the colour
has been specified identify one or several has been created
using the location of more noticeable por-
the salient tumor tions of an image
[31] Imaging functionality 3D- CNN The separation of brain The Dice similarity- Segmenting brain Multimodal MRI for
with MRI tumors with a 3D CNN coefficient can be tumors automatically the identification and
Multimedia Tools and Applications

utilized to assess the localization of brain

success of segmenta- metastases
tion
[32] Diverse tumor kinds as Features of clinical Comprehensive sequenc- The diagnosis selection Segmentation of brain Unique clinical and
examples practice ing of genomes was criteria for the patients tumors medicinal characteristics
done quickly were established were found
[33] CLE images (original Method for collecting CNN may help deter- A CLE imagery gather- Segmentation of brain The surgical CLE-image’s
data) surgical data mine the affected ing tool is utilized in tumors mapping for the biopsy’s
region combination with a site
surgical CLE imagery
technique to take CLE
image
[34] Imaging functionality Tissue BOLD structure The brain’s neuron activ- The ability to modulate neurosurgical excision Based on the realization
with MRI ity and spinal nerves is several factors that of brain tumors that oxygen’s mass is
measured by fMRI influence deoxygen- conserved
ated hemoglobin is
provided by BOLD
signal generation

13
 Table 2  (continued)
References Input data Criteria Image segmentation Validation method Components analysed Results

13
[35] MRI scans (original MFCM algorithm MFCM optimizes clas- The pre-processed Segmentation of brain A membership function
data) sification images have been tumors description is provided
organized using the for every point of data in
MFCM method to the collection
find the function of
membership
[36] 2D and 3D clinical visu- Creating the NiftyNet The tensorflow archi-
It is feasible to collect DL framework Automating the educa-
alization images the supporting struc- the required data tecture is utilized to tional environments
ture because of the innova-create an interface and to allow for further
tive internet tool visualize 2D and 3D investigation
images
[37] 2D MR images with a CNN programming DNN are used to DCNN is being Segmentation of brain Brain tumors are divided
patch-based technique separate various tumor researched for catego- tumors into various types
types rizing and segmenting
brain lesions
[38] MRI-derived multi- Categorization of images Characteristics have The effectiveness of the Segmentation of brain A substantial quantity of
modal brain images been recovered image’s categorization tumors guided learning
from the images by was used to evaluate
eliminating image the selection process
fragments
Multimedia Tools and Applications
Multimedia Tools and Applications

Fig. 1  Techniques used frequently for BTS

inside the fuzzy-region expansion. Utilizing FLAIR MRI, T1, and T2, quick multispectral
cancer segmentation was made feasible by deterministic intensity form.

4 Watershed algorithm

With this, the voxel concentrations or slopes have been signified by a topographical chart
comparable to those utilized in topography. The chart’s “steepness” determines a boundary.
Watershed appears to be a division method that considers both the geographic context and
the hydrographic intellectual structure. The water drop descends the gradients in the image
until it hits the close-by lower spot. The limits of neighbouring water drop in tributary
areas eventually look like an involuntary watershed assistant. Over-segmentation might be
prohibited by employing neither pre-processing nor post-processing techniques to enhance
an item’s architecture as an FCM clustering method. An innovative segmentation approach
was outlined by fusing a watershed method with a procedure for expectation maximization
(EM) [40]. When utilized in MRI, this produces effective gradient findings for separating
the CSF from the gray tissue.

• Other segmentation techniques

i. Atlas approach
Tumor tissue in the MRI is divided utilizing MRI technique; guidance Free of tumors.
The technique determines the outcomes of fragmentation by creating a solitary-to-solitary
correspondence between pre-segmented image and additional images using a registration-
based technique. The examined material will be divided into non-typical brains and nor-
mal, allowing the brain atlases to provide the various variants. This tactic is based on a
concept frequently used to describe brain regions.

13
 Multimedia Tools and Applications

ii. Segmentation based on Mask RCNN and U-Net

In recent years, UNet, culmination-to- culmination complete CNN for segmenting clini-
cal images, was introduced. The device consists of a decoder and an encoder. The encoder
was implemented to extract the spatial properties of the images, and the decoder was uti-
lized for organizing the encoded data into a map. The sequence of double 3 × 3 convolu-
tional algorithms that comprise up the encoder are followed by the max-pooling proce-
dure. An evolution of double 3 × 3 convolutional algorithms has been utilized to link the
decoder and encoder. Due to the robustness of the U-Net structure’s system, training takes
comparatively minimal duration and generates exact segmentation. It is possible to train
extremely large amounts of samples while still getting superior outcomes with a smaller
amount of data. The original development and application of UNet framework was in the
cell tracking domain. Currently, utilizing this architectural approach, the brain tumor [41]
can be separated from clinical imagery. To solve segmentation-related problems in the field
of healthcare imaging, researchers created the S3D U-Net [42], and H-Dense U-Net [43],
methods as enhanced variants of the U-Net approach.
To solve segmentation-related problems, a DNN based approach dubbed Mask-RCNN
[44] is created. To accurately identify instances, the Mask-RCNN instance segmentation
system has been created. The technique is utilized to identify various objects in a video
or an image. If an illustration was allotted to Mask-RCNN, it characterizes the images in
regards of masks, object-bounding boxes, and classes. Contrasted to U-Net, the Mask-
RCNN generates fewer precise segmentation outlines. Mask-RCNN uses the dual-stage
Regional-Proposal-Network and FastRCNN techniques. Characteristics were gathered
from each potential box employing RoIPool, and in the following phase, the categorization
process was carried out. Utilizing the Features Pyramid Network, Mask R-CNN improves
the conventional character regaining pyramids, which employ high-level traits and transfer
them to simpler levels. For object identification, the Region Proposal Network, that has
been employed to separate the image into parts and identify areas holding items of interest,
has been utilized. Bounding circles produced by RPN are then utilized to categorize ROIs.
Last but not least, Mask R-CNN generates filters by employing extra regions from the ROI
classification. These masks, which differ from binary masks in that they are defined by low
resolution fluctuating numbers, may hold more detailed characteristics.

5 Review on BTS

In the area of medical imaging methods, the BTS in MRI scans seems to be a prominent
issue. Finding the brain tumor’s precise location and size requires a precise identification
of the cancer. The K-means based clustering algorithm has been employed in the study
[45] to detect tumors based on categorization and morphological assessment. Typically, the
MRI scan picture is the initial pre-processed images. To retrieve the cancer from pre-pro-
cessed MRI identified images, the image has been first processed to K-means cluster gen-
eration. The final stage is determining how much of the tumor was removed. The procedure
in SciLab must be run several times, though, which proceeds more time. Recently, interest
in the new ML field of “DL” has increased substantially. It was widely employed in many
different applications and proved to be an efficient ML approach for many of the difficult
problems. The authors of the study [46] categorize a database of 66 brain MRIs employ-
ing a DNN classifier, among the DL structures. Principal component analysis (PCA), a
strong feature-based methodology, the wavelet decomposition, and the approach have all

13
Multimedia Tools and Applications

been employed to provide findings that were deemed to be quite acceptable across all pro-
cess variables.
A DNN-based fully automated BTS approach was employed in the study [10]. The net-
works are made for glioblastomas found in MRI, both LGG and HGG. These tumors may
grow anyplace in the brain and could take on almost any shape, structure, or severity due
to their unique architecture. These characteristics motivate research of an extremely power-
ful ML method that makes use of a very adaptive, large capacity DNN. Below is a break-
down of a number of design options that were shown to be crucial for obtaining depend-
able performance. Many CNN-based architectures, or DNNs modified to picture data, are
specifically explored in this paper. CNN architecture’s state-of-art differs from some of the
usual ones used in machine vision. In addition to local data, the network also employs extra
global background data. Contrary to most conventional CNN paradigm implementations,
systems employ an outer layer that seems to be a convolutional variant of a totally con-
nected layer, consequential in a 40-fold enhance in speed. To overcome difficulties related
to the disparity of tumour labels supplied, two-phase training technique is utilized. Last
but not least, a sequence designs where a subsequent CNN takes basic system’s output as a
supplementary stream of data. The dependability and accuracy in the most current national
whereas also being around 30 times quicker, according to findings from the 2013 BRATS
test database. Automatic segmentation seems to be a difficult procedure since brain tumors
have tentacles and distributed underlying addresses.
The study [47] utilized a variety of methods to raise the quality of the ingested hierar-
chical characteristics. Four different techniques have been implemented. In order to divide
data more precisely, the Multi-Level Deep Medic approach of the widely recognized Deep
Medic algorithm has been used. To enhance the standard of the multi-level characteris-
tic derived from DN, a novel which doubles activity and learning has been described as
well. It is an all-encompassing training tool that can be applied to a number of modern
systems, which includes U-Net and Deep Medic. Third, extra classifiers that support deep
networks’ high-level phases, which are employed to gather more data, employing a label
allocation loss function. An original MLP-based post-processing strategy was implemented
to enhance the deep network’s forecasting outcomes. Numerous studies have been per-
formed on the two common recently released BTS databases, the BRATS 2015 and 2017
collections of data. Results from the two datasets demonstrate that the suggested methods
increase the two widely recognized deep networks’ segmentation methods efficiency. It has
been tough to divide and category brain tumors because of their tentacles and scattered
underlying structures.
The automation of BTS remains problematic because of their significant anatomical
diversity. The study shows an automated BTS method predicated on DCN networks [48].
A patch-based method and a novel aspect have been employed to train the DN. It makes
utilization of the capacity to extract two co-centric fragments of various dimensions from
the original images. New Linear nexus design is developed using recent DNN events like
batch normalization, non-linear stimulus, inception components and dropouts. The tech-
nique utilizes a dropout regularize to handle the over-fitting issue caused on by a lack of
data. Images have been standardized and distortion field adjusted during pre-processing.
The next stage is to process the gathered patches by using DCNN that allocates a result
identifier to each patch’s centre pixel. Segmentation methods have been implemented in
post-processing to eradicate minimal FP at the borders. The BRATS 2015 and 2013 data-
sets are used to describe and evaluate a two-stage biased training technique that improves
the effectiveness metrics of other methods in analogous settings. However, the computation
duration has been long.

13
 Multimedia Tools and Applications

The precise division of brain tumors has been crucial for detection and therapy of
tumors. The study demonstrated a novel method for segmenting brain tumors employing
multicascaded CNN and completely linked conditional random fields. The greater part of
the segmentation procedure is comprised of the following two stages. The original con-
struction of the multi-cascaded complete network involved combining the intermediary
outputs of different connected elements for the purpose to keep into consideration the
regional reliance of tags and to benefit from multiple-scale characteristics for coarse seg-
mentation. Secondly, CRFs were implemented for the better segmentation to account for
the regional sources of data and remove some inaccurate findings. In addition, patches of
images from the sagittal, coronal, and axial views were utilized for training three distinct
segmentation simulations, which were subsequently merged to create the ultimate segmen-
tation outcome. The effectiveness of the presented technique was assessed on three freely
accessible databases. According to the trial findings, is more effective than existing tech-
niques. However, the framework works more difficult whenever the data are considerably
distinct [49].
For the development of trustworthy computer-aided diagnostic networks, collecting data
and visualization are essential stages. The process is carried out to divide brain tumors
in a systematic manner [50]. By combining OTSU and the customizable particle swarm
optimization method, the optimal interrupted value is found. The noise reduction filtering
is implemented in brain MRIs to minimize noise and improve picture clarity. The com-
ponents that were found were utilized to teach CNN and perform tasks related to catego-
rization. The study has a higher accuracy of 98% than analytical methods. Nevertheless,
by additionally tuning the algorithm for a variety of sources of data and utilizing novel
encountered heuristic methods, the diagnostic system’s effectiveness will be increased. The
study [51] utilized FLAIR Imaging methods for fully automatic major tumor segmenta-
tion and identification utilizing a universal DL design dubbed DeepSeg. The developed
DeepSeg is an adaptable, detaching design. Its two linked central elements are built on
the relationship between decoding and encoding. The transmission element is a CNN,
which is in the position of retrieving spatial characteristics. The produced semantic map
is delivered to the processing component, which integrates it to create the full-resolution
probabilistic map. The study incorporates a variety of neural network designs developed
around the modified U-Net design, which includes recurrent neural networks, NASNet, and
DCNet. Utilizing MRI data of the brain tumour segmentation problem, the DL frameworks
approach has been successfully investigated and tried online with 356 instances utilized
as training data and 136 instances utilized as cross validation. Hausdorff separation and
Dice values for the resulting segmentation methods vary from 0.82 to 0.85 and 9.7 to 18.7,
correspondingly. The study has effectively demonstrated the viability and relative effective-
ness of implementing numerous DL techniques in a novel DeepSeg structure to identify
brain tumors inside FLAIR images utilizing technology. Yet, the examination of the Deep-
Seg design needs to take into account fresh image samples from different MRI methods.
The Fig. 2 shows the literature matrix includes papers from 2020 to 2023, distributed as
12 papers in 2020, no papers in 2021, 13 papers in 2022, and 1 paper in 2023. The papers
from 2020 cover a range of methodologies, such as saliency identification, 3D CNNs, com-
prehensive genome sequencing, CNNs for confocal laser endomicroscopy (CLE) images,
fMRI and BOLD signal generation, MFCM algorithm for MRI scans, NiftyNet as an inter-
net tool, and CNN programming with DNNs for MR images. In 2022, the focus shifts to
evaluating and improving segmentation techniques, including graph-based methods, deep
learning models, and region-based approaches, with an emphasis on improved accuracy.
Multimodal data, transfer learning techniques, and feature extraction methods are explored,

13
Multimedia Tools and Applications

25

No of Paper 20

15

10

0
2020 2021 2022 2023
Year

Fig. 2  Literature Matrix

alongside the development of automated segmentation frameworks. Finally, in 2023, one

paper introduces a novel deep learning-based method for brain tumor segmentation, inte-
grating advanced architectures and attention mechanisms while utilizing large-scale data-
sets for training and validation.

6 ML‑based BTS

Image contraction in the type of segmentation offers several uses in the creation of CAD
systems that rely on radiographic images like MRI. Broadly speaking, unsupervised and
supervised image segmentation may be split into two groupings. Moreover, the borders
of the targeted object in the images can be defined using unsupervised segmentation tech-
niques such as edge detection, deformation, thresholding, and graph cutting. Contrarily,
supervised segmentation methods use training examples that also take into account past
knowledge of the image processing issue. The overview of unsupervised and supervised
learning was indicated in Table 2.

6.1 Unsupervised learning

Unsupervised learning, sometimes referred to as unsupervised ML, uses ML methods to

evaluate and organize unlabelled information. Without the assistance of a person, these
techniques group data and identify obscure patterns. In other terms, unsupervised learning
techniques detect correlations, spontaneously recurring tendencies and patterns, or compa-
rable features within a database. In contrary to supervised learning methods, unsupervised
learning techniques cannot be used to solve a prediction or categorization issue directly
since the parameters of the outcome have been unknown. More complicated processing
activities may be performed with unsupervised learning methods than with supervised
learning. This method also makes it simple to decrease dimensionality. Understanding raw
data could be aided by unsupervised learning. This learning becomes progressive and bal-
ances the outcome, similar to how human intellect functions. An unsupervised technique

13
 Multimedia Tools and Applications

called clustering divides the input into many groupings. In the medical imaging domain,
numerous clustering strategies were employed, particularly the grouping of healthy and
aggressive brain cancers. Compared to supervised learning methods, unsupervised learn-
ing techniques are less sophisticated and don’t need labeled dataset. The results, mean-
while, are frequently less accurate and can even be unexpected [52].
K-means clustering, morphology, and thresholding procedures were used by Khilkhal
et al. [53] for BTS in MRI. To improve the ultimate accuracy of the findings, non-brain
cells were eliminated during the morphological phase. The trials used LGG and HGG pic-
tures from the BRATS databases. In attempt to identify brain cancers effectively, Islam
et al. [54] suggested an enhanced design that combines Template-based K-means (TK) and
PCA + superpixel approach. With less computational time, this technique might produce
a usable segmentation outcome in MRI scans. For BTS in MR images, Kumar et al. [55]
presented a five-stage technique. This goal was accomplished by using a crude K-means
method. The results have been suggestive of the reality that the recommended technique
got superior ratings in assessment in contrast with earlier efforts. Furthermore, K-means
method has been straightforward, unpretentious to use, and ensures merging. Regrettably,
it’s indeed very reliant on the baseline numbers and grouping outliers. A technique for BTS
in MR images was put out by Sheela et al. [56] and was predicated on rotational triangular
portions with Fuzzy C-Means (FCM) method. Prior to thresholding, the image background
ought to be primarily removed using two-level morphological reconstructive procedures.
They used T1-a weighted contrast-improved image dataset—to assess the suggested struc-
ture’s effectiveness. The final evaluation of the suggested approach is shown. The FCM
model outperforms the k-means technique and yields the better consequence for example
points that overlap. However, a lot depends on the predetermined number of clusters.
Modified mean-shift-predicated FCM segmentation was used by Singh et al. [57] to rec-
ognize brain cancers in MR images. The consequences exhibited that the recommended
approach has a great extent of efficacy and precision. The number of clusters may be
determined via the Mean-Shift clustering method automatically with no issues brought
on by outliers. Nevertheless, when the number of clusters has been changed rapidly, this
technique fails to perform effectively (higher dimension). Chaddad [58] proposed a novel
method for retrieving Glioblastoma (GBM) characteristics from MR images using Gauss-
ian Mixture-Models (GMM) to examine the GMM’s efficacy in BTS. The T2-WI and
T1-WI had accuracy performances of 97.05% each. Furthermore, in FLAIR setting, the
accurateness diminished to 94.11%. Such experimental results offer hope for enhancing
heterogeneity characteristics and, in turn, earlier GBM therapy. The MRI-predicated brain
cancer was segmented by Pravitasari et al. [59] utilizing Reversible-Jump Markov Chain
Monte-Carlo and GMM technique. Moreover, consequences of the investigation confirmed
that the recommended model swiftly and effectively carried out the method. The GMM
method was less dependent on the quantity of variables. However, this approach does have
a sluggish rate of resolution and is delicate to starting settings.

6.2 Supervised learning

The term “supervised ML” refers to the popular subset of ML techniques known as super-
vised learning. This method uses labeled databases to teach computers how to accurately
categorize inputs or forecast events. The cross-validation procedure involves the model
changing its values till the structure is well fitted. In supervised learning, structures are
taught to give the best outcomes using a training dataset. This training database’s precise

13
Multimedia Tools and Applications

both output and input data (labels) allow the prototype to improve over time. The approach
updates the loss function after evaluating accuracy in order to reduce error. Methods for
supervised learning aid in locating comprehensive answers to problems encountered in
the actual world, particularly those related to medicine. To identify a brain tumor, several
supervised learning techniques have been used. An explanation of each approach is given
below. The ability to generate data outputs or gather information from previous experi-
ences is the main advantage of these strategies. These methods’ primary drawback is their
failure to appropriately categorize input statistics that didn’t correspond to the relevant cat-
egories inside the training set.
FLAIR as well as T1-weighted brain images can be used to recognize and segment brain
cancer, according to a technique proposed by Padlia et al. [60]. A fractional Sobel filter
has been used to improve pictures and reduce noise. For the purpose of identifying asym-
metries in brain images, mutual information and Bhattacharya coefficients were utilized.
SVM was employed to categorizing the statistical characteristics to distinguish the tumor
portion from the tumor zone after retrieving attributes of the targeted area via patches
and windows. Their approach has a 98.03% mean accuracy rate. Employing Kernel SVM
(KSVM)-Social Ski Driver (SSD) for far more precise categorization, Rao and Karunakara
[61] emphasized on effective segmentation and categorization. After being identified as
tumorous and benign using KSVM in this research, the malignant tumors have been fur-
ther classified as a moderate, high, and low employing SSD optimization approach. With
reliability scores of 99.15%, 99.36%, and 99.2% for the respective years of 2020, 2019,
and 2018 BRATS databases, it is demonstrated that the suggested KSVM-SSD approach
has been superior in terms of categorization accuracy measured on the BRATS databases.
However, since SVM can effectively handle unstructured and semi-structured input with
the use of the right kernel function, doing so is not always simple.
A discriminative Random Forest (RF) algorithm was created and improved by Lefko-
vits et al. [62] for BTS in multimodal MRI. The aim of tuning has been to identify the
best parameter readings and the relevant significant restrictions of the discriminative algo-
rithm. In regards of dice index, the suggested technique produced results for entire cancer
is 75–91% and for central region is 71–82%. An RDF-predicated BTS method that was
previously published by Ellwaa et al. [63] was extended. Instead of utilizing a randomly
chosen training database, the RDF was trained by an iterative procedure in which specific
patients have been exposed to the training examples employing heuristic methods. Approx-
imately 80% has been given as the method’s acquired dice score. Anitha and Raja [64]
developed methods for segmenting and identifying brain tumors predicated on RF filters
to divide the brain modalities into healthy and pathological. The specificity and sensitivity
rates of the suggested method have been 98% and 97%, correspondingly. Smaller Kernels
of Two-Path CNN (SK-TPCNN) with RF were used by Yang et al. [65] to develop an auto-
matic segmentation method. The central tumor, expanding tumor, and whole tumor, each
had sensitivity ratings of 92.2%, 83.2%, and 96%, correspondingly. Whereas RF performs
effectively with both discrete and on-going input and aids to increase accuracy by reducing
fitting problem in decision trees, it takes a long period to train.
Thayumanavan and Ramasamy [66] examined the effectiveness of RF Classifier (RFC),
Decision Tree (DT), and SVM in developing a paradigm for malignancies identification
and segmentation in brain-MR images. According to the research findings, RFC achieved
the best outcome with a precision of 98.37%. Additionally, RFC demonstrated 99.09%
specificity, accompanied by SVM and DT with 88.78%, and 95.68%, correspondingly.
While normalization and data scaling are not necessary for DT, learning takes a long time.
Moreover, a slight alteration in the source dataset causes a vast alteration in the method’s

13
 Multimedia Tools and Applications

architecture. One of the simplest and most effective classification techniques seems to be
the Naïve Bayes Classifier (NBC). It facilitates the creation of ML algorithms that can
anticipate outcomes accurately quickly. Many studies have used Naïve Bayes (NB) to
detect brain tumors in medical imaging. To find brain cancers on MR images, Kaur and
Oberoi [67] used an NBC, for instance. For BTS, the suggested method demonstrated
86% accuracy. Raju et al. [68] designed a Bayesian fuzz- clustering approach for BTS and
Multiple-SV-Neural Network (multi-SVNN) classifiers predicated on the Harmony-Crow
Search (HCS) evolutionary algorithms. The proposed approach, according to the research-
ers, has a 93% accuracy rate. The “zero-frequency issue” is when NB approaches provide
zero likelihood to a categorizing example whose category in the testing examples wasn’t
present in the training examples, despite the fact that they have been suitable for handling
multi-class forecasting issues and need far less training examples (Table 3).

7 BTS using DL methods

A kind of artificial intelligence (AI), and ML, known as DL imitates how people learn
specialized subjects. DL seems to be important in data analytics that also includes sta-
tistical and prediction modeling. Owing to DL approaches, a number of visual analytic
tasks, involving categorization, object identification, and monitoring, have shown con-
siderable performance increases [75]. Also, despite demonstrating state-of-the-art reli-
ability, DL approaches have had a substantial influence on medical imaging processing
activities mechanisation. Numerous DL approaches were created and used in a variety of
fields, especially the brain cancer’s identification, over the past few decades [76]. For BTS,
Nema et al. [77] created the RescueNet that has been trained using unpaired Generative-
Adversarial-Network (GAN). For the BRaTS 2017 and BRaTS 2015 databases, the find-
ings revealed Dice values of 94.63%, and 94.01%, correspondingly. Neelima et al. [78]
suggested a model for categorizing brain cancers utilizing MR images was based on Opti-
mal-DeepMRSeg technique, which was learned employing a developed Sailfish Political-
Optimizer (SPO) method for BTS. This approach produced improved accuracy, specific-
ity, segmentation accuracy, and sensitivity that have been, 91.7%, 92.5%, 90%, and 92.8%,
correspondingly. Besides that, when segmenting cancers employing 3D brain CT or MR
images, Rezaei et al. [79] presented an intelligent strategy predicated on an adversarial net-
work termed voxel-GAN for addressing unbalanced data issues. Once the recommended
strategy was examined on the ISLES database, the outcomes indicated 0.83 as Dice value,
9.3 as Hausdorff score, 0.81 as precision, and 0.78 as recall. The GAN networks have the
ability to increase data generation, decrease expenses, and improve data examples. How-
ever, such algorithms call for sophisticated databases and solid technical expertise.
Raju et al. [80] suggested a strategy for dividing and classifying brain cancers using
MR images used DBN and integrated active contour framework. The results showed that
the suggested model has 94.5% accuracy, 96.95% sensitivity, and 99.35% specificity. Even
with a massive quantity of input, the DBN algorithms may still deliver the greatest perfor-
mance outcomes. To be more effective, these models need a ton of data and highly special-
ized understanding. Relying on MRI scans, Badža and Barjaktarović [81] constructed a
convolutional autoencoder for BTS. According to the authors, the suggested strategy had
99.23% mean accuracy for pixel categorization and 99.28% mean accuracy for five-fold
cross-validation and single test outcome. Moreover, the use of autoencoders may drasti-
cally minimize the data dimensionality, offer a suitable method for significantly reducing

13
 Table 3  Summary of application of unsupervised and supervised techniques
Multimedia Tools and Applications

Reference Dataset Segmentation Demerits

Techniques

[69] BRATS 2015 Bayesian fuzzy clustering 1.Consumption of time

2.Extremely complicated system
[70] BRaTS 2017 lattice Boltzmann + superpixel fuzzy clustering 1.Extremely complicated system
2.Disregarding the hazy borders
[16] BRaTS 2013 Random Forest 1.Incredibly susceptible to sounds
2.New training sets are necessary
[71] Clinical MR Images Morphological Operation + Region of Interest + Region Grow- 1.Class disparities not taken into account
ing 2.Extremely complicated system
[72] MRI images FRFCM- BTS-ELM 1.Class disparities not taken into account
2.Employing only the T1 mode
[73] BRATS 2015 Region of Growing algorithm 1.Required to choose a starting location
[74] BRaTS 2015 VGG19 + k-means 1.Data is easily forgotten
2.There are several functions needed

13
 Multimedia Tools and Applications

source-data noise, and increase the developing DL system’s efficacy. Autoencoders can
only learn and replicate input attributes from the data they have been learned on; they can-
not learn or reproduce input attributes from fresh data. An advanced CNN for BTS in MR
images was presented by Sajid Iqbal et al. [82]. The segmentation challenging database for
BRATS, which consists of images collected via four distinct modalities, has been used by
the proposed scheme. In order to address the segmentation issue, an expanded version of
the current network was also shown. Several neural network structures are interconnected
in a consecutive manner in the system design, and Convolutional feature mappings are fed
at the peer stage. Hence, experimental findings on BRATS 2015 standard database demon-
strated the suggested approach’s accessibility and superiority to competing methodologies
in this field of study. For the purpose of segmenting gliomas within MRI images, Sun et al.
[83] presented a multi-pathway 3D design predicated on a Fully Convolutional Network
(FCN). Leveraging multi-modal MR images, the design uses 3D extended convolution
throughout every route to extract various receptive regions of feature maps that have been
subsequently spatially merged employing skip connections. This framework may make
it easier for FCN algorithms to define the borders of tumour regions. Besides that, with
the CNN methodology, Mohammad Havaei et al. [12] achieved a balance between perfor-
mance and computational efficiency that has previously been difficult for other approaches
to match. The CNN generates an attribute description instantly from the input because it is
learned on image inputs. Furthermore, a cascaded structure with two routes—one concen-
trating on glioma’s minute specifics and the other on the wider context—was suggested.
Moreover, we provide a two-stage patch-wise learning method that enables us to learn
methods in a matter of hours. By fully using the provided method’s convolutional features,
researchers can also segment whole brain imagery in 25 s to 180 s. The technique was
one of the best reliable in the research and was also extremely computationally intensive,
according to experimental findings on BTS challenges (BRATS 2013, and BRATS 2015).
The CNN algorithms may automatically discover hidden outlines within the source dataset
and distribute weights among stages. However, such frameworks are unable to represent
the object’s location and orientation.
Jalluri et al. [84] concentrated on the usages of several methods for brain tumor iden-
tification employing brain MRI. Moreover, an adaptive-bilateral filter (ABF) has been
used in research’s pre-processing to get rid of the sounds that have been prevalent in
MR images. Next, for accurate tumor region recognition, fuzzy recurrent neural net-
work (RNN) (FR-Net) and binary thresholding segmentation approaches were used.
Datasets for testing, validation, and training are employed. The machine-based provided
technique made a prediction about it or not the individual does have a brain cancer.
Specificity, accuracy, and sensitivity are only a few of the performance indicators that
were used to evaluate the final results. It has been hoped that the planned work would
operate more admirably than its competitors. Each example can be thought to be reliant
on earlier examples, and RNN algorithms can handle inputs of arbitrary length. Such
approaches do, however, run into problems including Vanishing or Explosive Gradi-
ent. Long Short-Term Memory (LSTM) systems have been RNN that may learn order
dependency in problems involving sequence prediction [85]. Among the most popular
RNN models to date seems to be the LSTM. The most effective method for simulat-
ing sequence information, it has been used to understand the intricate complexities of
individual action. Long-term memory is referred to as “cell state”. To classify cancers
employing multi-modal MRI, Xu et al. [86] presented an LSTM based Multi-modal-
UNet (LSTM-UNet). The suggested LSTM-UNet beat the conventional U-Net with less
model variables, according to empirical findings assessing the method’s effectiveness on

13
Multimedia Tools and Applications

the BRATS-2015 database. Long-term relationships can be learned via LSTM networks.
In order to capture both localised structural and worldwide contextual data, Cahall et al.
[87] suggested the DIU-Net paradigm for end-to-end BTS predicated on U-Net. This
architecture includes inception units with dilated convolutions along with widening
and contracting routes. Every dilated inception component in this design consists of 3
1 × 1 convolution processes accompanied by a l-dilated convolutional filters. In a further
work, Yang et al. [88] suggested a novel design (DCU-Net) predicated on the conven-
tional U-Net but with dilated convolutions for BTS. By substituting elongated spatially
pyramid pooling for max pooling towards the down-sampling procedure conclusion, the
feature reception field may be increased while maintaining image resolution. During
the training systems, both dilated convolutional residual cells and skip connections are
coupled to enhance the network’s capacity to detect tumour features more precisely. By
retrieving a weighted patch within the tumor’s boundary areas, Baid et al. [89] sug-
gested a 3D fully automatic BTS structure centred on the U-Net framework that divides
radiologically recognisable tumour subdivisions and tackles the class imbalance issue
among tumor and non-tumor spots. The outcomes demonstrated that the segmentation
based on weighted patch approach performs similarly to the pixel-based technique when
there exists a thin border between tumour subparts. Guan et al. [90] suggested a 3D
AGSE-VNet network incorporated segmentation layout predicated on V-net employing
the attention guide filter (AG) for every decoder and an incorporated component of the
squeeze and excites (SE) for every encoder, enhancing the helpful data consequently
employing the channel relationship. This framework can be used for BTS employing 3D
MR images. This algorithm’s effectiveness was evaluated using the BraTS 2020 data-
base, and it produced successful segmentation outcomes with the potential for clini-
cal trials. Such algorithms require a variety of assets, a large memory capacity, and a
long training period because they are susceptible to overfitting. Table 4 indicates the DL
methods summary in BTS.

Table 4  Overview of BTS employing DL techniques

Reference Dataset Segmentation Demerits
Techniques

[91] BRaTS 2020 CNN 1.New training sets are necessary

2. Interpretability is poor
[92] Hospital databases + DTCWT + DRL 1.Spatial coherence is lacking
BRaTS2018 + CQ500 2. Extremely complicated system
[93] BRaTS2013 FCRE + CNN 1.Utilizing only HGG sick individuals
2.New training sets are necessary
[94] BRaTS 2018,2015 and 2017 3D CNN 1.Extremely complicated system
2.Neglect to identify the subtle and tiny brain
tumor
[95] BRaTS 2017 + 2015 + 2013 DDM+ DPGM 1.Extremely complicated system
2.A significant amount of training data is
required
[96] Radiopaedia dataset VGG19-CNN 1.Employing only the T1 mode
2.Require to perform post-processing
[97] MRI dataset Faster R-CNN 1.Employing only the T1 mode
2.New training sets are necessary

13
 Multimedia Tools and Applications

8 Hybrid methods‑based BTS

There have been two kinds of brain cancers: benign and aggressive. A brain cancer could
be caused by an unchecked growth of aberrant cells within brain tissues. An innocuous
brain cancer does not harm the nearby normal and healthy cells; however, a malignant
cancer might affect the nearby brain cells and cause a person to pass away. To ensure the
patients’ life, a brain cancer may need to be found early. Often, MRI scanning has been
used to find brain tumors. Because of the tumors’ unusual shapes and locations inside the
brain, radiographers have been unable to effectively segment them in MRI images. A pre-
cise BTS has been required to find the cancer, to diagnose a patient appropriately, and to
offer the doctors the information necessary to carry out the patient’s operation. To dynami-
cally and effectively segment the cancer, Thillaikkarasi and Saravanan [98] introduced a
unique DL technique (kernel-based CNN with M-SVM). The processes in the study that
was being presented include pre-processing, attribute extraction, image categorization, and
BTS. The Contrast Limited Adaptive Histrogram-Equalization (CLAHE) and Laplacian of
Gaussian filtering method (LoG) are used to improve and flatten the MR image, and attrib-
utes may be derived from it depending on the tumor’s location, size, and surface character-
istics in the brain. In accordance with the chosen characteristics, M-SVM has been used to
classify the images. With the aid of the kernel-based CNN approach, the tumor has been
segmented from MR image. Moreover, the recommended method’s empirical outcomes
indicated that it can perform BTS with an accuracy of about 84% than existing strategies.
However, the method needs higher computation time [99].
To build intelligent clinics, a segmentation method for brain cancers was necessary.
Morphological diversity and major category mismatch need to be successfully addressed in
order to classify brain tumors intelligently. Because of these problems, conventional DNN
struggle to forecast higher-accuracy segmentation images. He-Xuan et al. [100] suggested
employing multimodal brain cancer images along with LSTM and UNET frameworks in
order to create new network architecture with a mixed gradient descent in order to address
sample disparity and define an intellectual segmentation procedure to detect brain cancer
as a solution to these issues. The open source BTS Challenge database has been utilized
to learn and validate the suggested network in order to confirm the viability of this tech-
nique. In 3 tumor regions—enhanced, centre, and whole—the study found DSCs of 0.80,
0.82, and 0.91, specificities of 0.98, 0.99, and 0.99, and sensitivities of 0.82, 0.85, and
0.93. Also, findings demonstrated that the suggested method could segment various tumor
regions more precisely than those of previous BTS approaches, demonstrating its possible
application usefulness in the medical identification of brain cancers. However, the study
was limited to a single tumor type, and this method requires a lot of computation time.
In order to achieve segmentation outcomes with visual and spatial coherence, Xiaomei
et al. [26] introduced a revolutionary BTS approach that is built by merging Conditional
Random-Fields (CRFs) and fully CNNs (FCNNs) in a coherent model. Employing 2D
image patching and image segments, this study learns a DL-predicated segmentation algo-
rithm as follows: FCNNs are trained employing image patches, CRFs are trained as RNN
(CRF-RNN) employing image slicing with FCNN variables fixed, and CRF-RNN and
FCNNs are fine-tuned employing image slices. In specifically, trained three segmentation
algorithms were used to separate brain tumors employing a voting-based fusion technique
employing 2D-image slice and patches collected in the radial, frontal, and sagittal per-
spectives, correspondingly. Slice-by-slice brain images segmentation using the proposed
approach was substantially quicker than using image patches. Using imagery data from the

13
Multimedia Tools and Applications

BRATS 2013, 2015, and 2016, the research tested the presented technique. The experimen-
tal findings have shown that the proposed technique is capable of creating segmentation
models using T1c, Flair, and T2 scans and achieving comparable outcomes to models cre-
ated using T1c, T2, Flair, and T1 scans. However, the method consumes more time.
A HTTU-Net model for BTS was proposed by Nagwa et al. [101]. This design makes
advantage of activation and normalization function. It features two loops, each of which
had a distinct kernel dimension and contains a distinct layer count. The ultimate seg-
mentation will then be created by merging those two tracks. Furthermore, to tackle the
issue of class imbalances, this study uses generalized Dice (GDL) loss mechanisms and
focal losses. The BraTS’2018 databases were used to assess the suggested segmentation
technique, and the consequences displayed mean Dice coefficients for 3 tumor regions-
enhancement, core, and whole of 0.745, 0.808, and 0.865, correspondingly. The BTS may
be done with the suggested HTTU-Net topology, and the findings are extremely accurate.
Together with the article, several both qualitative and quantitative assessments have been
discussed. It demonstrates that the findings of the provided model are extremely equivalent
to professional human-level competence and might aid professionals in cutting down on
diagnosis time. The study has limitations because of the numerous layers that make up its
intricate design.
Conventional deep CNN for fully autonomous BTS has two issues: inadequate multi-
scale tumor detection and spatial information loss due to recurrent pooling/striding. Zexun
et al. [102] employed a 3D atrous-convolution along with a solitary stride to substitute
striding/pooling as well as provide the framework for attribute learning to solve the first
issue. A AFPNet+3D CRF has been created and appended to the backbone’s end to address
the second issue. This framework enhances the general model’s capacity to discriminate
between cancers of different sizes by adding contextual data. As a last stage in the network
output’s post-processing, a 3D fully-connected CRF has been built to achieve hierarchical
segmentation of both spatial consistency and appearance. Several ablation tests performed
on MRI databases show that the technique’s multi-scale data acquisition and lossless attrib-
ute calculation are capable of solving the aforementioned issues. The suggested technique
attains superior results and may be successfully incorporated into clinical applications
when contrasted to state-of-the-art solutions on public standards. However, this model is
not tested on challenge databases.

9 Results and discussion

9.1 Performance evaluation

There are numerous techniques to assess the effectiveness of segmentation or cat-

egorization systems. To display their verified outcomes, researchers employ a variety of
approaches. The following commonly used performance metrics are examined in the pre-
sent investigation: Confusion matrix, Mean Square Error (MSE), Peak Signal to Noise
Ratio (PSNR), Jaccard Index (Tanimoto Co-efficient), Dice-overlap-index (DOI) or Dice
Similarity Coefficient (DSC), Accuracy metric, Specificity, Sensitivity, Recall, and Preci-
sion. The crucial data regarding the real outcome and the projected outcome given by seg-
mentation or categorization techniques is provided by confusion matrices. Table 5 provides
a demonstration of confusion matrix.

13
 Multimedia Tools and Applications

Table 5  Representation of Predicted Class 1 Pre-

Confusion Matrix dicted
Class 2

Actual Class 1 TP FN
Actual Class 2 FP TN

9.1.1 Mean squared error

Mean Squared Error (MSE) is a statistic that expressed in Eq. (1) the average squared vari-
ation among the temperature values anticipated and observed. Higher values denote worse
performance and provide an estimate of the entire prediction error.
1 ∑
MSE = ∗ (Actual − Forecast)2 (1)
N
The segmentation techniques are shown in Table 6 along with the relevant Mean
Squared Error (MSE) percentages. By evaluating the average squared difference between
the predicted segmentation and the actual segmentation, the MSE is a frequently used sta-
tistic to evaluate the effectiveness of a segmentation method. Each segmentation method
is represented by a row in the table, and the associated MSE percentage is given in the
adjacent column. Lower MSE percentages suggest fewer errors between the anticipated
and ground truth segmentations, indicating superior segmentation performance. Figure 3
depicts the Mean Square Error Ratio in graphic form.

9.1.2 Peak signal‑to‑noise ratio

A higher PSNR number denotes that the two pictures are more comparable or of greater
quality. PSNR is frequently employed to assess how well image compression methods
function or how faithfully reconstructed pictures compare to the originals. The PSNR
equation is as follows in Eq. (2)
( )
10Log10 peakval2
PSNR = (2)
MSE
Different segmentation techniques are shown in Table 7 together with the Peak Signal-
to-Noise Ratio (PSNR) values that correlate to them. The PSNR, which measures the ratio
of the highest possible power of a signal to the strength of corrupting noise, is a commonly

Table 6  Comparison assessment Reference Method used for segmentation MSE(%)

of MSE
[30] Salient-map modification 34
[37] CNN 28
[38] Image classification 58
[24] HORW 45
[25] Level Set 63
[27] Kernelized Fuzzy Entropy Clustering 78

13
Multimedia Tools and Applications

Mean Square Error

90
Error ratio (%) 80
70
60
50
40
30
20
10
0
Salient-map CNN Image HORW Level Set Kernelized
modification classification Fuzzy Entropy
Methods Clustering

Fig. 3  Comparison assessment of MSE

Table 7  Comparison assessment Reference Method used for segmentation PSNR (db)
of PSNR
[30] Salient-map modification 45.67
[37] CNN 56.32
[38] Image classification 62.15
[24] HORW 38.92
[25] Level Set 51.85
[27] Kernelized Fuzzy Entropy Clustering 68.78

used metric to assess the quality of a picture that has been segmented or rebuilt. The com-
parison of PSNR values across segmentation techniques is shown in Fig. 4. greater bars
denote greater PSNR values, which suggest segmentations of higher quality.
The effectiveness of frequently used BTS techniques is evaluated by taking each mod-
el’s accuracy into account. Accuracy seems to be the percentage of correctly identified
image pixels. This has been occasionally denoted to as complete pixel exactness. Whereas
being the common fundamental effectiveness indicator, if there was a class conflict, the
outcomes of detection process may be incorrect. When one of the specified categories per-
forms higher compared to the other, there is a category disparity. In this scenario, findings
would be skewed as a result of the superior accuracy of the dominant class outweighing the
poorer precision of the competing cohort. If there hadn’t been a class differences, the accu-
racy metric has been proposed for evaluating detection results utilizing images. Table 8 and
Fig. 5 represented the segmentation accuracy of a widely used model is displayed.
Accuracy
TPos + TNeg
Accuracy = (3)
TPos + TNeg + FPos + FNeg

The segmentation result indicated that Salient-map modification [30] method has
attained higher accuracy, while compared to other selected methods like CNN [37], Image

13
 Multimedia Tools and Applications

PSNR
80
70
60
50
Noise Ratio

40
30
20
10
0
Salient-map CNN Image HORW Level Set Kernelized
modification classification Fuzzy Entropy
Clustering
Methods

Fig. 4  Comparison assessment of PSNR

Table 8  Commonly used Reference Method used for segmentation Accuracy (%)
segmentation method’s
performance
[30] Salient-map modification 98
[37] CNN 95.1
[38] Image classification 96.48
[24] HORW 63
[25] Level Set 89.5
[27] Kernelized Fuzzy Entropy Clustering 90.19

Kernelized fuzzy entropy clustering

Level set
Methods

HORW

Image classification

Work of CNN

Salient-map modification

0 20 40 60 80 100
Accuracy

Fig. 5  Comparison of commonly used segmentation method’s accuracy

13
Multimedia Tools and Applications

classification [38], HORW [24], Level Set [25], and Kernelized Fuzzy Entropy Clustering
[27].
Moreover, the hybrid segmentation method’s performances are also compared that was
shown in Table 9. The hybrid BTS methods employed in different databases are selected
for this comparison process. The selected methods are lattice Boltzmann + super-pixel
fuzzy clustering [70], Morphological Operation + ROI + Region Growing [71], FRFCM-
BTS-ELM [72], VGG19 + k-means [74], FCRE+CNN [93], DDM + DPGM [95], VGG19-
CNN [96], kernel-based CNN + M-SVM [98], LSTM + UNET [100], CRFs + FCNNs
[26], HTTU-Net [101], and AFPNet+3D CRF [102].
Precision
TPos
P= (4)
TPos + FPos

Recall
TPos
R=
TPos + FNeg (5)

Specificity
TNeg
Specificity = (6)
TNeg + FPos

The comparison of dice similarity and accuracy of the hybrid BTS method is shown
in Fig. 6a and b, correspondingly. The result indicated that lattice Boltzmann + super-
pixel fuzzy clustering [70] method has attained higher accuracy of 99.4% compared
to other methods like VGG19 + k-means [74], VGG19-CNN [96], and kernel-based
CNN + M-SVM [98].
The overlapping pixel among the segmentation outcome (n) and the ground truth (m) is
provided by the Dice Similarity Coefficient (DSC).
2|m ∩ n|
DSC (m, n) = (7)
|m| + |n|

The result indicated that lattice Boltzmann + super-pixel fuzzy clustering [70] method
has attained higher dice of 93% compared to other methods like FRFCM- BTS-ELM [72],
DDM + DPGM [95], LSTM + UNET [100], CRFs + FCNNs [26], HTTU-Net [101], and
AFPNet+3D CRF [102].

9.1.3 Jaccard index

The Jaccard Index would demonstrate how closely the pixels in the segmented outcome (B)
and the surface of the truth (A) match up. More accurate results are obtained when the Jac-
card Index is high.
|m ∩ n|
JaccardIndex J (m, n) = (8)
|m ∪ n|
Based on their Jaccard Index scores, the segmentation techniques in Table 10 are com-
pared. A popular metric for assessing the similarity between two sets—in this instance,

13
13
Table 9  Performances of hybrid BTS methods
References Methods Accuracy (%) Specificity (%) Sensitivity (%) Dice (%)

[70] lattice Boltzmann + super-pixel fuzzy clustering 99.4 99.72 91.83 93

[71] Morphological Operation + ROI + Region Growing – 77.9 86.98 –
[72] FRFCM- BTS-ELM – – – 84.11
[74] VGG19 + k-means 94.06 – – –
[93] FCRE+CNN – – 72 71
[95] DDM + DPGM – – – 97.2
[96] VGG19-CNN 94.58 96.12 88.41 –
[98] kernel-based CNN + M-SVM 84 – – –
[100] LSTM + UNET – 99 93 91
[26] CRFs + FCNNs – – 82 84
[101] HTTU-Net – 99.9 80 80
[102] AFPNet+3D CRF – – – 82
Multimedia Tools and Applications
Multimedia Tools and Applications

Accuracy comparison
105
100
Accuracy 95
90
85
80
75
lattice Boltzmann VGG19+k-means VGG19-CNN kernel-based
+ super-pixel CNN+M-SVM
fuzzy clustering
Methods

(a)
Dice similarity comparison
120
100
80
60
Dice (%)

40
20
0
lattice
Boltzman
n+
FRFCM-
super- FCRE+C DDM+D LSTM + CRFs + HTTU- AFPNet+
BTS-
pixel NN PGM UNET FCNNs Net 3D CRF
ELM
fuzzy
clusterin
g
Dice (%) 93 84.11 71 97.2 91 84 80 82

(b)

Fig. 6  Hybrid BTS method performance comparison (a) Accuracy (b) Dice similarity

Table 10  Comparison of Jaccard Reference Method used for segmentation Jaccard Index
Index
[30] Salient-map modification 0.85
[37] CNN 0.92
[38] Image classification 0.78
[24] HORW 0.91
[25] Level Set 0.88
[27] Kernelized Fuzzy Entropy Clustering 0.80

the predicted and actual segmentations—is the Jaccard Index, commonly known as the
Intersection over Union (IoU). Improved segmentation accuracy and similarity to the
ground truth are shown by higher Jaccard Index values. The visual comparison of the

13
 Multimedia Tools and Applications

Jaccard Index
0.95

0.9
Index Ratio

0.85

0.8

0.75

0.7
Salient-map CNN Image HORW Level Set Kernelized
modification classification Fuzzy Entropy
Clustering
Methods

Fig. 7  Comparison of Jaccard Index

segmentation techniques based on their Jaccard Index values is presented in Fig. 7. It gives
users and investigators an immediate, intuitive knowledge of the relative performance of
each method, making it simpler for them to pinpoint the techniques that have the highest
segmentation accuracy.

10 Discussion

Brain tumors remain to be a hot topic for study in the medical-image processing domain.
The brain cancer is among the deadliest disease that arises when the proliferation of tis-
sues in the brain has been out of bounds. The death incidence of this malignancy moti-
vated experts examines ways for earlier brain cancer detection. MRI images remain among
the greatest methods to identify tumor by offering a view of soft matter inside the brain.
Numerous ML- and DL-based techniques were created during the past few decades.
But because so many articles used these strategies, it’s critical to review the most recent
research and techniques. In addition to reviewing frequently used BTS methodologies,
BTS articles, ML-based BTS techniques, DL-based BTS techniques, and hybrid BTS arti-
cles, this work compared the performance of the various approaches when used on data-
bases that were made accessible to the public. According to analyses of earlier studies,
MRI remains the most effective imaging method for diagnosing brain tumors (Tables 2 and
3). The primary rationale that MRI has been used so often is that it offers more informa-
tion than other imaging methods including CT scans. Furthermore, DL approaches have
been applied substantially more recently than ML methods. Nonetheless, the amount of
research that employed ML algorithms or hybrid strategies was still larger than DL-predi-
cated hybrid BTS strategies. The most popular ML and DL techniques for BTS were SVM
and CNN. Furthermore, BRATS databases have been used in the majority of studies. How-
ever, the CNN integrated with other methods have attained less accuracy for segmentation
process. Thus, the accuracy needs improvement in future work while integrating CNN with
other methods.

13
Multimedia Tools and Applications

11 Conclusion

BTS have benefited greatly from advances in AI. For example, pathology, anomaly
identification, brain cancer diagnosis, tissue density computation, therapy planning, and
computer-assisted surgery all have benefited from the use of ML and DL techniques
and methodologies by researchers. These methods have been efficient for tasks involv-
ing the BTS because their characteristics make it possible to tell aberrant tissues apart
from healthy ones. This essay provides an overview of approaches used in BTS. This
research covers a wide range of commonly employed BTS, ML-based BTS, and DL-
based BTS techniques for brain tumors identification. In order to assist scholars and
medical professionals in developing prospective research paths and, more crucially,
in identifying the most efficient and accurate tactics for BTS, the present work objec-
tively evaluates the existing methodologies based on numerous assessment parameters.
Despite CNN’s widespread popularity, it can be difficult to separate brain structures
using DL approaches. The efficiency of particular models is impacted by the images
captured by diverse scanners. Deep CNN performs poorly because to insufficient train-
ing data and resolution. Conventional strategies have been still applied in real-world cir-
cumstances, and training time remains an important consideration. This research makes
numerous suggestions for future research that might be used to improve the functional-
ity of the existing BTS processes: (i) Greater databases of 3D MRI should be gathered,
(ii) segmentation accuracy should be improved by creating a unique hybrid CNN-UNet
approach, and (iii) incorporated multi-head attention layers should be created for the
hybrid DL models.

Data availability Data sharing not applicable to this article as no datasets were generated during the current
study.

Declarations
Competing interests The authors declare that they have no known competing financial interests or personal
relationships that could have appeared to influence the work reported in this paper.

References
1. Kruchko C, Ostrom QT, Gittleman H, Barnholtz-Sloan JS (2018) The CBTRUS story: providing
accurate population-based statistics on brain and other central nervous system tumors for everyone.
Neuro-Oncol 20(3):295–298. https://​doi.​org/​10.​1093/​neuonc/​noy006
2. Tahir A, Asif M, Ahmad MB, Mahmood T, Khan MA, Ali M (2022) Brain tumor detection using
decision-based fusion empowered with fuzzy logic. Math Probl Eng 2022:1–13. https://​doi.​org/​10.​
1155/​2022/​27102​85
3. Iorgulescu JB et al (2022) Molecular biomarker-defined brain tumors: epidemiology, validity, and
completeness in the United States. Neuro-Oncol 24(11):1989–2000. https://​doi.​org/​10.​1093/​neuonc/​
noac1​13
4. Miller KD et al (2021) Brain and other central nervous system tumor statistics, 2021. CA A Cancer J
Clin 71(5):381–406. https://​doi.​org/​10.​3322/​caac.​21693
5. Tataei Sarshar N et al. (2023) “Glioma Brain Tumor Segmentation in Four MRI Modalities Using
a Convolutional Neural Network and Based on a Transfer Learning Method,” in Proceedings of the
7th Brazilian Technology Symposium (BTSym’21), Y. Iano, O. Saotome, G. L. Kemper Vásquez,
C. Cotrim Pezzuto, R. Arthur, and G. Gomes de Oliveira, Eds., in Smart Innovation, Systems and

13
 Multimedia Tools and Applications

Technologies, vol. 207. Cham: Springer International Publishing, pp. 386–402. https://​doi.​org/​10.​
1007/​978-3-​031-​04435-9_​39
6. Özyurt F, Sert E, Avcı D (2020) An expert system for brain tumor detection: fuzzy C-means with
super resolution and convolutional neural network with extreme learning machine. Med Hypotheses
134:109433. https://​doi.​org/​10.​1016/j.​mehy.​2019.​109433
7. Wadhwa A, Bhardwaj A, Singh Verma V (2019) A review on brain tumor segmentation of MRI
images. Magn Reson Imaging 61:247–259. https://​doi.​org/​10.​1016/j.​mri.​2019.​05.​043
8. Chahal PK, Pandey S, Goel S (2020) A survey on brain tumor detection techniques for MR images.
Multimed Tools Appl 79(29–30):21771–21814. https://​doi.​org/​10.​1007/​s11042-​020-​08898-3
9. Ranjbarzadeh R, Bagherian Kasgari A, Jafarzadeh Ghoushchi S, Anari S, Naseri M, Bendechache
M (2021) Brain tumor segmentation based on deep learning and an attention mechanism using MRI
multi-modalities brain images. Sci Rep 11(1):10930. https://​doi.​org/​10.​1038/​s41598-​021-​90428-8
10. Havaei M et al (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–
31. https://​doi.​org/​10.​1016/j.​media.​2016.​05.​004
11. Işın A, Direkoğlu C, Şah M (2016) Review of MRI-based brain tumor image segmentation using deep
learning methods. Procedia Comput Sci 102:317–324. https://​doi.​org/​10.​1016/j.​procs.​2016.​09.​407
12. Havaei M, Dutil F, Pal C, Larochelle H, Jodoin P-M (2016) “A Convolutional Neural Network
Approach to Brain Tumor Segmentation,” in Brainlesion: Glioma, Multiple Sclerosis, Stroke and
Traumatic Brain Injuries, A. Crimi, B. Menze, O. Maier, M. Reyes, and H. Handels, Eds., in Lecture
Notes in Computer Science. Cham: Springer International Publishing, pp. 195–208. https://​doi.​org/​
10.​1007/​978-3-​319-​30858-6_​17.
13. Chen B, Zhang L, Chen H, Liang K, Chen X (2021) A novel extended Kalman filter with support
vector machine based method for the automatic diagnosis and segmentation of brain tumors. Comput
Methods Prog Biomed 200:105797. https://​doi.​org/​10.​1016/j.​cmpb.​2020.​105797
14. Dhanachandra N, Manglem K, Chanu YJ (2015) Image segmentation using K -means clustering
algorithm and subtractive clustering algorithm. Procedia Comput Sci 54:764–771. https://​doi.​org/​10.​
1016/j.​procs.​2015.​06.​090
15. Abdel-Maksoud E, Elmogy M, Al-Awadi R (2015) Brain tumor segmentation based on a hybrid clus-
tering technique. Egypt Inf J 16(1):71–81. https://​doi.​org/​10.​1016/j.​eij.​2015.​01.​003
16. Bonte S, Goethals I, Van Holen R (2018) Machine learning based brain tumour segmentation on lim-
ited data using local texture and abnormality. Comput Biol Med 98:39–47. https://​doi.​org/​10.​1016/j.​
compb​iomed.​2018.​05.​005
17. Ilhan U, Ilhan A (2017) Brain tumor segmentation based on a new threshold approach. Procedia
Comput Sci 120:580–587. https://​doi.​org/​10.​1016/j.​procs.​2017.​11.​282
18. Karimi D, Warfield SK, Gholipour A (2021) Transfer learning in medical image segmentation: new
insights from analysis of the dynamics of model parameters and learned representations. Artif Intell
Med 116:102078. https://​doi.​org/​10.​1016/j.​artmed.​2021.​102078
19. Magadza T, Viriri S (2021) Deep learning for brain tumor segmentation: a survey of state-of-the-art. J
Imaging 7(2):19. https://​doi.​org/​10.​3390/​jimag​ing70​20019
20. Jyothi P, Singh AR (2023) Deep learning models and traditional automated techniques for brain
tumor segmentation in MRI: a review. Artif Intell Rev 56(4):2923–2969. https://​doi.​org/​10.​1007/​
s10462-​022-​10245-x
21. Baid U et al. (2021) “The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmen-
tation and Radiogenomic Classification,” https://​doi.​org/​10.​48550/​ARXIV.​2107.​02314
22. Bakas S et al (2017) Advancing the Cancer genome atlas glioma MRI collections with expert seg-
mentation labels and radiomic features. Sci Data 4(1):170117. https://​doi.​org/​10.​1038/​sdata.​2017.​117
23. Ibrahim RW, Hasan AM, Jalab HA (2018) A new deformable model based on fractional Wright
energy function for tumor segmentation of volumetric brain MRI scans. Comput Methods Prog
Biomed 163:21–28. https://​doi.​org/​10.​1016/j.​cmpb.​2018.​05.​031
24. Lim KY, Mandava R (2018) A multi-phase semi-automatic approach for multisequence brain tumor
image segmentation. Expert Syst Appl 112:288–300. https://​doi.​org/​10.​1016/j.​eswa.​2018.​06.​041
25. Farhi L, Yusuf A, Raza RH (2017) Adaptive stochastic segmentation via energy-convergence for
brain tumor in MR images. J Vis Commun Image Represent 46:303–311. https://​doi.​org/​10.​1016/j.​
jvcir.​2017.​04.​013
26. Zhao X, Wu Y, Song G, Li Z, Zhang Y, Fan Y (2018) A deep learning model integrating FCNNs and
CRFs for brain tumor segmentation. Med Image Anal 43:98–111. https://​doi.​org/​10.​1016/j.​media.​
2017.​10.​002
27. Pham TX, Siarry P, Oulhadj H (2018) Integrating fuzzy entropy clustering with an improved PSO for
MRI brain image segmentation. Appl Soft Comput 65:230–242. https://​doi.​org/​10.​1016/j.​asoc.​2018.​
01.​003

13
Multimedia Tools and Applications

28. Kawahara J et al (2017) BrainNetCNN: convolutional neural networks for brain networks; towards
predicting neurodevelopment. NeuroImage 146:1038–1049. https://​doi.​org/​10.​1016/j.​neuro​image.​
2016.​09.​046
29. Chen L, Bentley P, Mori K, Misawa K, Fujiwara M, Rueckert D (2018) DRINet for medical image
segmentation. IEEE Trans Med Imaging 37(11):2453–2462. https://​doi.​org/​10.​1109/​TMI.​2018.​28353​
03
30. Banerjee S, Mitra S, Uma Shankar B (2018) Automated 3D segmentation of brain tumor using visual
saliency. Inf Sci 424:337–353. https://​doi.​org/​10.​1016/j.​ins.​2017.​10.​011
31. Charron O, Lallement A, Jarnet D, Noblet V, Clavier J-B, Meyer P (2018) Automatic detection and
segmentation of brain metastases on multimodal MR images with a deep convolutional neural net-
work. Comput Biol Med 95:43–54. https://​doi.​org/​10.​1016/j.​compb​iomed.​2018.​02.​004
32. Cole BL, Pritchard CC, Anderson M, Leary SE (2018) Targeted sequencing of malignant Supratento-
rial pediatric brain tumors demonstrates a high frequency of clinically relevant mutations. Pediatr Dev
Pathol 21(4):380–388. https://​doi.​org/​10.​1177/​10935​26617​743905
33. Izadyyazdanabadi M et al (2018) Convolutional neural networks: ensemble modeling, fine-tuning and
unsupervised semantic localization for neurosurgical CLE images. J Vis Commun Image Represent
54:10–20. https://​doi.​org/​10.​1016/j.​jvcir.​2018.​04.​004
34. Pak RW et al (2017) Implications of neurovascular uncoupling in functional magnetic resonance
imaging (fMRI) of brain tumors. J Cereb Blood Flow Metab 37(11):3475–3487. https://​doi.​org/​10.​
1177/​02716​78X17​707398
35. Vishnuvarthanan A, Rajasekaran MP, Govindaraj V, Zhang Y, Thiyagarajan A (2018) Development
of a combinational framework to concurrently perform tissue segmentation and tumor identifica-
tion in T1 - W, T2 - W, FLAIR and MPR type magnetic resonance brain images. Expert Syst Appl
95:280–311. https://​doi.​org/​10.​1016/j.​eswa.​2017.​11.​040
36. Gibson E et al (2018) NiftyNet: a deep-learning platform for medical imaging. Comput Methods Prog
Biomed 158:113–122. https://​doi.​org/​10.​1016/j.​cmpb.​2018.​01.​025
37. Amin J, Sharif M, Yasmin M, Fernandes SL (2018) Big data analysis for brain tumor detection: deep
convolutional neural networks. Futur Gener Comput Syst 87:290–297. https://​doi.​org/​10.​1016/j.​
future.​2018.​04.​065
38. Ilunga-Mbuyamba E et al (2017) Localized active contour model with background intensity compen-
sation applied on automatic MR brain tumor segmentation. Neurocomputing 220:84–97. https://​doi.​
org/​10.​1016/j.​neucom.​2016.​07.​057
39. Ilunga-Mbuyamba E, Avina-Cervantes JG, Cepeda-Negrete J, Ibarra-Manzano MA, Chalopin C
(2017) Automatic selection of localized region-based active contour models using image content
analysis applied to brain tumor segmentation. Comput Biol Med 91:69–79. https://​doi.​org/​10.​1016/j.​
compb​iomed.​2017.​10.​003
40. Motagi AC, Malemath V (2018) Detection of Brain Tumor using Expectation Maximization (EM)
and Watershed. Int J Sci Res Comput Sci Eng 6:76–80. https://​doi.​org/​10.​26438/​ijsrc​se/​v6i3.​7680
41. Dong H, Yang G, Liu F, Mo Y, Guo Y (2017) “Automatic Brain Tumor Detection and Segmentation
Using U-Net Based Fully Convolutional Networks,” in Medical Image Understanding and Analysis,
M. Valdés Hernández and V. González-Castro, Eds., in Communications in Computer and Informa-
tion Science, vol. 723. Cham: Springer International Publishing, pp. 506–517. https://​doi.​org/​10.​
1007/​978-3-​319-​60964-5_​44.
42. Chen W, Liu B, Peng S, Sun J, Qiao X (2019) “S3D-UNet: Separable 3D U-Net for Brain Tumor
Segmentation,” in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, A.
Crimi, S. Bakas, H. Kuijf, F. Keyvan, M. Reyes, and T. van Walsum, Eds., in Lecture Notes in Com-
puter Science, vol. 11384. Cham: Springer International Publishing, pp. 358–368. https://​doi.​org/​10.​
1007/​978-3-​030-​11726-9_​32.
43. Li X, Chen H, Qi X, Dou Q, Fu C-W, Heng P-A (2018) H-DenseUNet: hybrid densely connected
UNet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging 37(12):2663–
2674. https://​doi.​org/​10.​1109/​TMI.​2018.​28459​18
44. Murali E, Meena K (2019) Stochastic gradient descent optimizer for segmentation of brain tumor
using mask R-CNN. JETIR 6(6). https://​www.​resea​rchga​te.​net/​profi​le/​Murali-​Erasa/​publi​cation/​
33597​0161_​Issue_6_​wwwje​tirorg_​ISSN-​2349-​5162/​links/​5d89c​8bc92​851ce​b793a​c159/​Issue-6-​
wwwje​tirorg-​ISSN-​2349-​5162.​pdf
45. Kurnar M, Sinha A, Bansode NV (2018) “Detection of Brain Tumor in MRI Images by Applying
Segmentation and Area Calculation Method Using SCILAB,” in 2018 Fourth International Confer-
ence on computing communication control and automation (ICCUBEA), pp. 1–5. https://​doi.​org/​10.​
1109/​ICCUB​EA.​2018.​86977​13

13
 Multimedia Tools and Applications

46. Mohsen H, El-Dahshan E-SA, El-Horbaty E-SM, Salem A-BM (2018) Classification using deep
learning neural networks for brain tumors. Futur Comput Inf J 3(1):68–71. https://​doi.​org/​10.​1016/j.​
fcij.​2017.​12.​001
47. Chen S, Ding C, Liu M (2019) Dual-force convolutional neural networks for accurate brain tumor
segmentation. Pattern Recogn 88:90–100. https://​doi.​org/​10.​1016/j.​patcog.​2018.​11.​009
48. Hussain S, Anwar SM, Majid M (2018) Segmentation of glioma tumors in brain using deep con-
volutional neural network. Neurocomputing 282:248–261. https://​doi.​org/​10.​1016/j.​neucom.​2017.​
12.​032
49. Hu K et al (2019) Brain tumor segmentation using multi-cascaded convolutional neural networks
and conditional random field. IEEE Access 7:92615–92629. https://​doi.​org/​10.​1109/​ACCESS.​
2019.​29274​33
50. Vijh S, Sharma S, Gaurav P (2020) “Brain Tumor Segmentation Using OTSU Embedded Adaptive
Particle Swarm Optimization Method and Convolutional Neural Network,” in Data Visualization
and Knowledge Engineering, J. Hemanth, M. Bhatia, and O. Geman, Eds., in Lecture Notes on
Data Engineering and Communications Technologies, vol. 32. Cham: Springer International Pub-
lishing, pp. 171–194. https://​doi.​org/​10.​1007/​978-3-​030-​25797-2_8
51. Zeineldin RA, Karar ME, Coburger J, Wirtz CR, Burgert O (2020) DeepSeg: deep neural network
framework for automatic brain tumor segmentation using magnetic resonance FLAIR images. Int J
CARS 15(6):909–920. https://​doi.​org/​10.​1007/​s11548-​020-​02186-z
52. Ahmed ST, Kumar SS, Guptha NS, AlShammari NK, Basha SM (2022) Improving medical image
pixel quality using micq unsupervised machine learning technique. Malaysian J Comput Sci:53–
64. https://​doi.​org/​10.​22452/​mjcs.​sp202​2no2.5
53. Khilkhal R, Ismael M (2022) Brain tumor segmentation utilizing thresholding and K-means clus-
tering. In: In 2022 Muthanna International Conference on engineering Science and technology
(MICEST). IEEE, Samawah, Iraq, pp 43–48. https://​doi.​org/​10.​1109/​MICES​T54286.​2022.​97901​
03
54. Islam MK, Ali MS, Miah MS, Rahman MM, Alam MS, Hossain MA (2021) Brain tumor detection
in MR image using superpixels, principal component analysis and template based K-means clus-
tering algorithm. Mach Learn Appl 5:100044. https://​doi.​org/​10.​1016/j.​mlwa.​2021.​100044
55. Kumar DM, Satyanarayana D, Prasad MNG (2021) An improved Gabor wavelet transform and
rough K-means clustering algorithm for MRI brain tumor image segmentation. Multimed Tools
Appl 80(5):6939–6957. https://​doi.​org/​10.​1007/​s11042-​020-​09635-6
56. Sheela CJJ, Suganthi G (2021) Accurate MRI brain tumor segmentation based on rotating tri-
angular section with fuzzy C- means optimization. Sādhanā 46(4):226. https://​doi.​org/​10.​1007/​
s12046-​021-​01744-8
57. Singh B, Aggarwal P (2017) Detection of brain tumor using modified mean-shift based fuzzy
c-mean segmentation from MRI images. In: In 2017 8th IEEE annual information technology,
electronics and Mobile communication Conference (IEMCON). IEEE, Vancouver, BC, pp 536–
545. https://​doi.​org/​10.​1109/​IEMCON.​2017.​81171​23
58. Chaddad A (2015) Automated feature extraction in brain tumor by magnetic resonance imaging using
Gaussian mixture models. Int J Biomed Imaging 2015:1–11. https://​doi.​org/​10.​1155/​2015/​868031
59. Pravitasari AA et al. (2019) “MRI-based brain tumor segmentation using Gaussian mixture model
with reversible jump Markov chain Monte Carlo algorithm,” presented at the The 2ND Interna-
tional Conference on Science, Mathematics, Environment, and Education, Surakarta, Indonesia, p.
020085. https://​doi.​org/​10.​1063/1.​51398​17
60. Padlia M, Sharma J (2019) “Fractional Sobel Filter Based Brain Tumor Detection and Segmenta-
tion Using Statistical Features and SVM,” in Nanoelectronics, Circuits and Communication Sys-
tems, V. Nath and J. K. Mandal, Eds., in Lecture Notes in Electrical Engineering, vol. 511. Singa-
pore: Springer Singapore, pp. 161–175. https://​doi.​org/​10.​1007/​978-​981-​13-​0776-8_​15
61. Rao CS, Karunakara K (2022) Efficient detection and classification of brain tumor using kernel based
SVM for MRI. Multimed Tools Appl 81(5):7393–7417. https://​doi.​org/​10.​1007/​s11042-​021-​11821-z
62. Lefkovits L, Lefkovits S, L. Szilágyi (2016) “Brain Tumor Segmentation with Optimized Ran-
dom Forest,” in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, A.
Crimi, B. Menze, O. Maier, M. Reyes, S. Winzeck, and H. Handels, Eds., in Lecture Notes in
Computer Science, vol. 10154. Cham: Springer International Publishing, pp. 88–99. https://​doi.​
org/​10.​1007/​978-3-​319-​55524-9_9
63. Ellwaa A et al (2016) Brain Tumor Segmantation Using Random Forest Trained on Iteratively
Selected Patients, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Inju-
ries. In: Crimi A, Menze B, Maier O, Reyes M, Winzeck S, Handels H (eds) Lecture notes in

13
Multimedia Tools and Applications

computer Science, vol 10154. Springer International Publishing, Cham, pp 129–137. https://​doi.​
org/​10.​1007/​978-3-​319-​55524-9_​13
64. Anitha R, Raja DSS (2018) Development of computer-aided approach for brain tumor detection using
random forest classifier. Int J Imaging Syst Technol 28(1):48–53. https://​doi.​org/​10.​1002/​ima.​22255
65. Yang T, Song J, Li L (2019) A deep learning model integrating SK-TPCNN and random forests for
brain tumor segmentation in MRI. Biocybernetics Biomed Eng 39(3):613–623. https://​doi.​org/​10.​
1016/j.​bbe.​2019.​06.​003
66. Thayumanavan M, Ramasamy A (2021) An efficient approach for brain tumor detection and seg-
mentation in MR brain images using random forest classifier. Concurr Eng 29(3):266–274. https://​
doi.​org/​10.​1177/​10632​93X21​10105​42
67. Kaur G, Oberoi A (2020) “Novel approach for brain tumor detection based on Naïve Bayes clas-
sification,” in Data Management, Analytics and Innovation, N. Sharma, A. Chakrabarti, and V.
E. Balas, Eds., in Advances in intelligent systems and computing, vol. 1042. Singapore: Springer
Singapore, pp. 451–462. https://​doi.​org/​10.​1007/​978-​981-​32-​9949-8_​31
68. Raju AR, Suresh P, Rao RR (2018) Bayesian HCS-based multi-SVNN: a classification approach
for brain tumor segmentation and classification using Bayesian fuzzy clustering. Biocybernetics
Biomed Eng 38(3):646–660. https://​doi.​org/​10.​1016/j.​bbe.​2018.​05.​001
69. Siva Raja PM, Rani AV (2020) Brain tumor classification using a hybrid deep autoencoder with
Bayesian fuzzy clustering-based segmentation approach. Biocybernetics Biomed Eng 40(1):440–
453. https://​doi.​org/​10.​1016/j.​bbe.​2020.​01.​006
70. Khosravanian A, Rahmanimanesh M, Keshavarzi P, Mozaffari S (2021) Fast level set method for
glioma brain tumor segmentation based on Superpixel fuzzy clustering and lattice Boltzmann
method. Comput Methods Prog Biomed 198:105809. https://​doi.​org/​10.​1016/j.​cmpb.​2020.​105809
71. Sheela CJJ, Suganthi G (2020) Morphological edge detection and brain tumor segmentation in
magnetic resonance (MR) images based on region growing and performance evaluation of modi-
fied fuzzy C-means (FCM) algorithm. Multimed Tools Appl 79(25–26):17483–17496. https://​doi.​
org/​10.​1007/​s11042-​020-​08636-9
72. SiSik F, Sert E (2020) Brain tumor segmentation approach based on the extreme learning machine
and significantly fast and robust fuzzy C-means clustering algorithms running on raspberry pi
hardware. Med Hypotheses 136:109507. https://​doi.​org/​10.​1016/j.​mehy.​2019.​109507
73. Srinivasa Reddy A, Chenna Reddy P (2021) MRI brain tumor segmentation and prediction using
modified region growing and adaptive SVM. Soft Comput 25(5):4135–4148. https://​doi.​org/​10.​
1007/​s00500-​020-​05493-4
74. Khan AR, Khan S, Harouni M, Abbasi R, Iqbal S, Mehmood Z (2021) Brain tumor segmentation
using K-means clustering and deep learning with synthetic data augmentation for classification.
Microsc Res Tech 84(7):1389–1399. https://​doi.​org/​10.​1002/​jemt.​23694
75. Suzuki K (2017) Overview of deep learning in medical imaging. Radiol Phys Technol 10(3):257–
273. https://​doi.​org/​10.​1007/​s12194-​017-​0406-5
76. Razzak MI, Naz S, Zaib A (2018) “Deep learning for medical image processing: overview, chal-
lenges and the future,” in Classification in BioApps, N. Dey, A. S. Ashour, and S. Borra, Eds., in
Lecture notes in computational vision and biomechanics, vol. 26. Cham: Springer International
Publishing, pp. 323–350. https://​doi.​org/​10.​1007/​978-3-​319-​65981-7_​12
77. Nema S, Dudhane A, Murala S, Naidu S (2020) RescueNet: an unpaired GAN for brain tumor seg-
mentation. Biomed Signal Process Control 55:101641. https://​doi.​org/​10.​1016/j.​bspc.​2019.​101641
78. Neelima G, Chigurukota DR, Maram B, Girirajan B (2022) Optimal DeepMRSeg based tumor
segmentation with GAN for brain tumor classification. Biomed Signal Process Control 74:103537.
https://​doi.​org/​10.​1016/j.​bspc.​2022.​103537
79. Rezaei M, Yang H, Meinel C (2019) “voxel-GAN: Adversarial Framework for Learning Imbal-
anced Brain Tumor Segmentation,” in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Trau-
matic Brain Injuries, A. Crimi, S. Bakas, H. Kuijf, F. Keyvan, M. Reyes, and T. van Walsum, Eds.,
in Lecture Notes in Computer Science, vol. 11384. Cham: Springer International Publishing, pp.
321–333. https://​doi.​org/​10.​1007/​978-3-​030-​11726-9_​29
80. Ratna Raju A, Pabboju S, Rajeswara Rao R (2019) Hybrid active contour model and deep belief
network based approach for brain tumor segmentation and classification. SR 39(4):473–487.
https://​doi.​org/​10.​1108/​SR-​01-​2018-​0008
81. Badža MM, Barjaktarović MČ (2021) Segmentation of brain tumors from MRI images using con-
volutional autoencoder. Appl Sci 11(9):4317. https://​doi.​org/​10.​3390/​app11​094317
82. Iqbal S, Ghani MU, Saba T, Rehman A (2018) Brain tumor segmentation in multi-spectral MRI
using convolutional neural networks (CNN). Microsc Res Tech 81(4):419–427. https://​doi.​org/​10.​
1002/​jemt.​22994

13
 Multimedia Tools and Applications

83. Sun J, Peng Y, Guo Y, Li D (2021) Segmentation of the multimodal brain tumor image used the
multi-pathway architecture method based on 3D FCN. Neurocomputing 423:34–45. https://​doi.​
org/​10.​1016/j.​neucom.​2020.​10.​031
84. SivaSai JG, Srinivasu PN, Sindhuri MN, Rohitha K, Deepika S (2021) “An Automated Segmentation
of Brain MR Image Through Fuzzy Recurrent Neural Network,” in Bio-inspired Neurocomputing, A.
K. Bhoi, P. K. Mallick, C.-M. Liu, and V. E. Balas, Eds., in Studies in Computational Intelligence,
vol. 903. Singapore: Springer Singapore, pp. 163–179. https://​doi.​org/​10.​1007/​978-​981-​15-​5495-7_9
85. Van Houdt G, Mosquera C, Nápoles G (2020) A review on the long short-term memory model. Artif
Intell Rev 53(8):5929–5955. https://​doi.​org/​10.​1007/​s10462-​020-​09838-1
86. Xu F, Ma H, Sun J, Wu R, Liu X, Kong Y (2019) LSTM multi-modal UNet for brain tumor seg-
mentation. In: In 2019 IEEE 4th International Conference on image, vision and computing (ICIVC).
IEEE, Xiamen, China, pp 236–240. https://​doi.​org/​10.​1109/​ICIVC​47709.​2019.​89810​27
87. Cahall DE, Rasool G, Bouaynaya NC, Fathallah-Shaykh HM (2021) “Dilated Inception U-Net (DIU-
Net) for Brain Tumor Segmentation,” https://​doi.​org/​10.​48550/​ARXIV.​2108.​06772.
88. Yang T, Zhou Y, Li L, Zhu C (2020) DCU-net: multi-scale U-net for brain tumor segmentation. XST
28(4):709–726. https://​doi.​org/​10.​3233/​XST-​200650
89. Baid U et al (2020) A novel approach for fully automatic intra-tumor segmentation with 3D U-net
architecture for gliomas. Front Comput Neurosci 14:10. https://​doi.​org/​10.​3389/​fncom.​2020.​00010
90. Guan X et al (2022) 3D AGSE-VNet: an automatic brain tumor MRI data segmentation framework.
BMC Med Imaging 22(1):6. https://​doi.​org/​10.​1186/​s12880-​021-​00728-8
91. Chattopadhyay A, Maitra M (2022) MRI-based brain tumour image detection using CNN based deep
learning method. Neurosci Inf 2(4):100060. https://​doi.​org/​10.​1016/j.​neuri.​2022.​100060
92. Liu G, Li X, Cai Y (2022) Segmentation for multimodal brain tumor images using dual-tree complex
wavelet transform and deep reinforcement learning. Comput Intell Neurosci 2022:1–11. https://​doi.​
org/​10.​1155/​2022/​53695​16
93. Chang J et al (2019) A mix-pooling CNN architecture with FCRF for brain tumor segmentation. J Vis
Commun Image Represent 58:316–322. https://​doi.​org/​10.​1016/j.​jvcir.​2018.​11.​047
94. Rehman A, Khan MA, Saba T, Mehmood Z, Tariq U, Ayesha N (2021) Microscopic brain tumor
detection and classification using 3D CNN and feature selection architecture. Microsc Res Tech
84(1):133–149. https://​doi.​org/​10.​1002/​jemt.​23597
95. Takrouni W, Douik A (2022) Improving geometric P-norm-based glioma segmentation through deep
convolutional autoencoder encapsulation. Biomed Signal Process Control 71:103232. https://​doi.​org/​
10.​1016/j.​bspc.​2021.​103232
96. Sajjad M, Khan S, Muhammad K, Wu W, Ullah A, Baik SW (2019) Multi-grade brain tumor classifi-
cation using deep CNN with extensive data augmentation. J Comput Sci 30:174–182. https://​doi.​org/​
10.​1016/j.​jocs.​2018.​12.​003
97. Kaldera HNTK, Gunasekara SR, Dissanayake MB (2019) Brain tumor classification and segmentation using
faster R-CNN. In: In 2019 advances in Science and engineering technology International conferences
(ASET). IEEE, Dubai, United Arab Emirates, pp 1–6. https://​doi.​org/​10.​1109/​ICASET.​2019.​87142​63
98. Thillaikkarasi R, Saravanan S (2019) An enhancement of deep learning algorithm for brain tumor
segmentation using kernel based CNN with M-SVM. J Med Syst 43(4):84. https://​doi.​org/​10.​1007/​
s10916-​019-​1223-7
99. Praveena HD, Guptha NS, Kazemzadeh A, Parameshachari BD, Hemalatha KL (2022) Effec-
tive CBMIR system using hybrid features-based independent condensed nearest neighbor model. J
Healthcare Eng 2022. https://​doi.​org/​10.​1155/​2022/​32973​16
100. Hu H-X, Mao W-J, Lin Z-Z, Hu Q, Zhang Y (2021) Multimodal brain tumor segmentation based on
an intelligent UNET-LSTM algorithm in smart hospitals. ACM Trans Internet Technol 21(3):1–14.
https://​doi.​org/​10.​1145/​34505​19
101. Aboelenein NM, Songhao P, Koubaa A, Noor A, Afifi A (2020) HTTU-net: hybrid two track U-net
for automatic brain tumor segmentation. IEEE Access 8:101406–101415. https://​doi.​org/​10.​1109/​
ACCESS.​2020.​29986​01
102. Zhou Z, He Z, Jia Y (2020) AFPNet: a 3D fully convolutional neural network with atrous-convolu-
tion feature pyramid for brain tumor segmentation via MRI images. Neurocomputing 402:235–244.
https://​doi.​org/​10.​1016/j.​neucom.​2020.​03.​097

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional affiliations.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a
publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript
version of this article is solely governed by the terms of such publishing agreement and applicable law.

13
Multimedia Tools and Applications

Authors and Affiliations

Ravikumar Sajjanar1 · Umesh D. Dixit1 · Vittalkumar K Vagga2

* Ravikumar Sajjanar
[email protected]
Umesh D. Dixit
[email protected]
Vittalkumar K Vagga
[email protected]
1
Department of Electronics and Communication Engineering, BLDEA’s V. P. Dr. P. G. Halakatti
College of Engineering and Technology (affiliated to Visvesvaraya Technological University,
Belagavi 590 018), Vijayapura 586 103, Karnataka, India
2
Department of Electronics and Communication Engineering, Government Polytechnic Koppal,
Koppal 583 231, Karnataka, India

13