AUTOMATIC BRAIN TUMOR SEGMENTATION

USING ARTIFICIAL INTELLIGENCE

Pakistan Institute of Engineering & Technology, Multan 1

 PROJECT SUPERVISOR
Engr. Saad Masood

PROJECT MEMBERS

Palwasha Munir 2kX6-EE-101

Romil Imtiaz 2kX6-EE-120
Waqar Mirza 2kX6-EE-132

Pakistan Institute of Engineering & Technology, Multan 2

 Presentation Flow
1. Introduction 13. Deep Learning
2. Problem Statement 14. U-net Model
3. Literature Review 15. U-net Segmentation Results
4. Research Gap 16. Progress
5. Solution Statement 17. Timeline/Schedule
6. Project Flow 18. References
7. Data Collection
8. Pre-Processing
9. Image Processing (Flow Diagram)
10. Image Processing
12. Automatic Segmentation Flow Diagram

Pakistan Institute of Engineering & Technology, Multan 3

 Introduction
• A brain tumor is a collection, or mass, of abnormal cells in the brain,
any growth of these abnormal cells inside brain can cause problems.
• Brain tumors can be cancerous (malignant) or noncancerous
(benign).

Pakistan Institute of Engineering & Technology, Multan 4

 Continue
• Brain tumor segmentation is a method of separating the
tumorous tissues from normal brain tissues.

Pakistan Institute of Engineering & Technology, Multan 5

Problem Statement
❑ The noninvasive magnetic
resonance imaging (MRI)
technique has arose as a front-
line diagnostic tool.
❑ Manual segmentation of
brain tumor with MRI images
performance is highly relied
on operator's experience.
❑ A total of 8401 radiologists in 47
states were required to provide
their malpractice history as part of
their credentialing application. Of
these, 2624 (31%) radiologists had
at least one claim in their career. [1]
Pakistan Institute of Engineering & Technology, Multan 6
 Literature Review
Sr.# Title Year Review
1 Research on feature extraction 2019 This paper proposes to use the local
of tumor image based on binary mode and convolutional neural
convolutional neural network. network based on image translation
[2] rotation invariance to cut and classify the
image, and then extract the feature to
achieve the purpose of classification.
2 An Automatic Brain Tumor 2018 A U-Net baseline model with optimal
Image Segmentation Method parameters structure is proposed which is
Based on the U-net.[3] suitable for segmentation. 1x1
convolutional layer, is added to the
baseline model to linearly recombine the
upper layer features, enrich extracted
features, reduce network parameters, and
improve the segmentation results.

Pakistan Institute of Engineering & Technology, Multan 7

 Literature Review
Sr.# Title Year Review
3 DRINet for Medical Image 2018 Convolutional neural networks (CNNs)
Segmentation.[4] has just changed the spectrum of medical
image segmentation in past few years. In
this paper DRINet novel CNN architecture
is proposed which minimize the problems
of learning the distinctive features in
images.

4 Interactive Medical Image 2018 An interactive framework is proposed in

Segmentation Using which a bounding box is made on the
Deep Learning With Image- tumor region with fine tuning based
Specific Fine Tuning.[5] segmentation, which further helps to
detect the region with tumor. 2015 Brats
Data Set has been used in this research.
BIFSeg achieves better result.

Pakistan Institute of Engineering & Technology, Multan 8

 Literature Review
Sr.# Title Year Review
5 Advanced Brain Tumor 2018 Different Segmentation methods has been
Segmentation from MRI discussed including Fuzzy C-means
Images.[6] method, segmentation using Region
Growing Method and through Genetic
Algorithm Based Method. Image
Segmentation through Computer Aid can
develop a more efficient way to detect
tumor.
6 Watershed Algorithm based 2018 In this research watershed segmentation
DAPP features for Brain Tumor algorithm is applied to images to segment
Segmentation and the tumor from the rest of the brain.
Classification.[7] Watershed Dynamic Angle
Projection technique is being introduced.
The brain image abnormality is observed, and 
research is conducted effectively through the 
BRATS dataset.

Pakistan Institute of Engineering & Technology, Multan 9

 Literature Review
Sr.# Title Year Review
7 A New Approach for 2018 The proposed approach optimally merges the 
Segmentation and Detection of segmentation results of the best-
Brain Tumor in 3D Brain MR known method and enhances the segmentati
Imaging. [8] on of the MR brain image.
The most widely used anisotropic filter is
applied. The results are tested on Flair Images
taken from MRI machine.

8 Concatenated and Connected 2018 A new methodology is being introduced

Random Forests With incorporating random forest and active
Multiscale Patch Driven Active contour model for segmentation.
Contour Model for Automated Competitive accuracy has been achieved with
Brain Tumor Segmentation of the proposed model.
MR Images.[9]

Pakistan Institute of Engineering & Technology, Multan 10

 Literature Review
Sr.# Title Year Review
9 An efficient non-invasive 2017 In order to identify the tumor grade (low /
method for brain tumor grade high), this paper suggested two different
analysis on MR images. [10] approaches for the study of MR images.
In the first technique Thresholding, edge
detection and filtering is applied. In
second method Histogram Equalization is
also done with segmentation with K-mean
Clustering.
10 Contrast enhanced brain 2017 Multi-Thresholding, and Active Contour (AC)
tumor segmentation based on based segmentation is applied. The method
Shannon's entropy and active is tested on Brats MRI Dataset. The
contour. [11] experimental results show a comprehensive
outcome in segmentation of tumor.

Pakistan Institute of Engineering & Technology, Multan 11

 Literature Review
Sr.# Title Year Review
11 Brain tumor classification using 2016 In this paper the eliminate the noise and strip
two-tier skull from MRI image and for feature
classifier with adaptive extraction they use discrete wavelet
segmentation transform (DWT) coefficients as feature vector
Technique[12]

12 Brain Tumor Segmentation 2016 In this paper they use of 2 small 3X3 kernels
Using Convolutional to obtain Deeper CNNs which have the same
Neural Networks in MRI effective receptive field of one 5X5 layer, but
Images[13] fewer weights.

Pakistan Institute of Engineering & Technology, Multan 12

 Literature Review
Sr.# Title Year Review
13 SEGMENTATION OF TUMOR 2015 In this they segment T1 T2 and flare image
AND EDEMA ALONG WITH with the help of non-linear anisotropic
HEALTHY TISSUES OF BRAIN diffusion filter. Then spatial filtering methods
USING WAVELETS AND are performed to obtain feature vector.
NEURAL NETWORKS[14]

14 Multi-Classification of Brain 2019 In this paper, a CNN architecture is proposed

Tumor Images to classify different types and grades of brain
Using Deep Neural tumors. The architecture of the network is
Network[15] evolved using different configuration to
acquire the most appropriate structure.

Pakistan Institute of Engineering & Technology, Multan 13

 Literature Review
Sr.# Title Year Review
15 Classification of Brain Tumors 2019 In this paper they talk about different accuracy
by Machine Learning values of other papers use SVM DVM method
Algorithms[16] and they increase the accuracy of the
segmentation.

16 Brain Tumor Segmentation 2019 The main goal is to develop a triple machine
Using Machine Learning learning classifier that can efficiently segment
Classifier[17] brain MRI images based on the accuracy
factor and error rate via Fuzzy C-media (FCM),
vector support machine (SVM) and
neighboring K-nearest. The proposed work is
an accurately segment the MR image and
classify the type of tumor.

Pakistan Institute of Engineering & Technology, Multan 14

 Literature Review
Sr.# Title Year Review
17 Brain Tumor Classification 2018 In this study, they attempted to train a
Using Convolutional Neural Convolutional Neural Network (CNN) to
Network[18] recognize the three most common types of
brain tumors, i.e. the Glioma, Meningioma,
and Pituitary. They implemented the simplest
possible architecture of CNN one each of
convolution, max-pooling, and flattening
layers, followed by a full connection from one
hidden layer.

Pakistan Institute of Engineering & Technology, Multan 15

 Research Gap
• Segmentation done manually is a time consuming task, and if
the person identifying the region where tumor is present is not
experienced and doesn’t have the right knowledge, it will
usually produce poor segmentation results. The research gap is
basically based on the maximum accuracy that can be attained
with any deep learning model.

Pakistan Institute of Engineering & Technology, Multan 16

 Solution Statement
• Automatic Segmentation is the solution, which has the ability to impact
positively on the clinical diagnoses by relieving doctors and physicians from
the strain of manual labelling of harmful tissues and by providing
sophisticated quantitative measurement to assist in diagnoses of the
disease.

Pakistan Institute of Engineering & Technology, Multan 17

Project Flow
MRI Data
Conversion from
From Aziz Image For Making Mask
Dicom to
Fatimah processing Images
readable format
Hospital
Dataset
Original +
Original
Mask
Images
Images

Testing Training
Data Data

U-net Model Segmented Image

Pakistan Institute of Engineering & Technology, Multan 18

Data Set Conversion from Ground Truth
Image
in Raw Dicom to other Images
Processing
format format

Data Set

Original
Original
& Ground
Images
Truth

Testing Training
Data Data

Output Segmented
Modified U-net Model
Image

19
Data Collection
• Data is collected from Aziz Fatimah Hospital,
Faisalabad[12]
• The name of MRI machine is Vantage Elan
(Toshiba)[13]
• Scans From Three Hundred Patients are
collected.
❑ Scans are analyzed and
segregation of collected data is
performed.
❑ Patients having tumor are
separated.
❑ Out of 300 patients scans 136
images are usable for training.
Pakistan Institute of Engineering & Technology, Multan 20
Pre - Processing
• Saved MRI Scans into DiCom
Format.
• Converted DiCom Files into
readable format for image
processing.

Pakistan Institute of Engineering & Technology, Multan 21

Image Processing ( Flow Diagram)

Pakistan Institute of Engineering & Technology, Multan 22

Image Processing ( Generating Mask Images)
❑ Sample Image

• Image of MRI Scan in

PNG format.

Pakistan Institute of Engineering & Technology, Multan 23

( Generating Mask Images)
Skull Striping

Output Image

Pakistan Institute of Engineering & Technology, Multan 24

( Generating Mask Images)
Contrast Stretching

Output Image

Pakistan Institute of Engineering & Technology, Multan 25

( Generating Mask Images)
Binary Opening

Output Image

Pakistan Institute of Engineering & Technology, Multan 26

( Generating Mask Images)
Binary Removing Small Objects

Output Image

Pakistan Institute of Engineering & Technology, Multan 27

( Generating Mask Images)
Binary Closing

Output Image

Pakistan Institute of Engineering & Technology, Multan 28

Automatic Segmentation Flow Diagram
Original
Images U-net
Training Data Model
Mask (Layers)
Images

Validation
Output Image
Image

Testing Data Test Image Output Image

Pakistan Institute of Engineering & Technology, Multan 29

Deep Learning
• Deep learning is part of a broader family of machine learning
methods based on artificial neural networks.
• For visual (image) data CNN (Convolutional Neural Networks) are
used.

Pakistan Institute of Engineering & Technology, Multan 30

U-net Model
• The u-net is convolutional neural network architecture for fast and
precise segmentation of images.
• It is mainly used for bio-medical images.
• It has a “U” shape structure. The architecture is symmetric and
consists of two major parts — the left part is called contracting path,
which is constituted by the general convolutional process; the right
part is expansive path which is organized by transposed convolutional
layers.

Pakistan Institute of Engineering & Technology, Multan 31

Pakistan Institute of Engineering & Technology, Multan 32
33
34
U-net Model Segmentation Results

Validation Image Validation Image

Result Through
U-net Model

Pakistan Institute of Engineering & Technology, Multan 35

U-net Model Segmentation Results

Test Image Test Image

Result Through
U-net Model

Pakistan Institute of Engineering & Technology, Multan 36

Progress
Project Background Image Processing Segregation of Data
Week 6-8 Week 16-18 Week 22-24

Overview of Getting Started with DL Improvement in Image Processing Technique

Methodologies Week 25 -26
Week 9-10 Week 19

Testing of Methodologies Draft Write-up and making dataset for U-net

Literature Review
Week 20 Week 26-28
Week 11-14

Data Collection More Data Collection Finalizing Draft & Implementation on U-net
Week 29-30
Week 15 Week 21

Pakistan Institute of Engineering & Technology, Multan 37

 Timeline/Schedule
Task/Week
5- 9-12 13-16 17- 21-24 25- 29-
1-4 8 20 28 32
Weeks Weeks Weeks
Weeks
Weeks Weeks Weeks Weeks
Literature Review

Learning of Programming
Language

Implementation

Documentation

Pakistan Institute of Engineering & Technology, Multan 38

References:
• [1] Whang, Jeremy S., et al. "The causes of medical malpractice suits
against radiologists in the United States." Radiology 266.2 (2013):
548-554.
• [2] Yang, Aimin, et al. "Research on feature extraction of tumor image
based on convolutional neural network." IEEE Access 7 (2019): 24204-
24213.
• [3] Yang, Tiejun, and Jikun Song. "An Automatic Brain Tumor Image
Segmentation Method Based on the U-net." 2018 IEEE 4th
International Conference on Computer and Communications (ICCC).
IEEE, 2018.

Pakistan Institute of Engineering & Technology, Multan 39

References:
• [4] Chen, Liang, et al. "DRINet for medical image segmentation." IEEE
transactions on medical imaging 37.11 (2018): 2453-2462.
• [5] Wang, Guotai, et al. "Interactive medical image segmentation
using deep learning with image-specific fine tuning." IEEE transactions
on medical imaging 37.7 (2018): 1562-1573.
• [6] Rajasekaran, Kavitha Angamuthu, and Chellamuthu Chinna
Gounder. "Advanced Brain Tumour Segmentation from MRI
Images." High-Resolution Neuroimaging: Basic Physical Principles and
Clinical Applications (2018): 83

Pakistan Institute of Engineering & Technology, Multan 40

References:
• [7] Jemimma, T. A., and Y. Jacob Vetharaj. "Watershed Algorithm based
DAPP features for Brain Tumor Segmentation and Classification." 2018
International Conference on Smart Systems and Inventive Technology
(ICSSIT). IEEE, 2018.
• [8] Jagan, A. "A New Approach for Segmentation and Detection of Brain
Tumor in 3D Brain MR Imaging." 2018 Second International Conference on
Electronics, Communication and Aerospace Technology (ICECA). IEEE, 2018.
• [9] Ma, Chao, Gongning Luo, and Kuanquan Wang. "Concatenated and
connected random forests with multiscale patch driven active contour
model for automated brain tumor segmentation of MR images." IEEE
transactions on medical imaging 37.8 (2018): 1943-1954
Pakistan Institute of Engineering & Technology, Multan 41
References:
• [10] Subashini, M. Monica, and V. Indra Gandhi. "An efficient non-invasive method for
brain tumor grade analysis on MR images." TENCON 2017-2017 IEEE Region 10
Conference. IEEE, 2017.
• [11] Priyadharshini, C., et al. "Contrast enhanced brain tumor segmentation based on
Shannon's entropy and active contour." 2017 Third International Conference on Biosignals,
Images and Instrumentation (ICBSII). IEEE, 2017
• [12] Anitha, V., and S. Murugavalli. "Brain tumour classification using two-tier classifier
with adaptive segmentation technique." IET computer vision 10.1 (2016): 9-17.
• [13]Pereira, Sérgio, et al. "Brain tumor segmentation using convolutional neural networks
in MRI images." IEEE transactions on medical imaging 35.5 (2016): 1240-1251.
• [14] Demirhan, Ayşe, Mustafa Törü, and İnan Güler. "Segmentation of tumor and edema
along with healthy tissues of brain using wavelets and neural networks." IEEE journal of
biomedical and health informatics 19.4 (2014): 1451-1458.

Pakistan Institute of Engineering & Technology, Multan 42

References:
• [15] Sultan, Hossam H., Nancy M. Salem, and Walid Al-Atabany. "Multi-
Classification of Brain Tumor Images Using Deep Neural Network." IEEE Access 7
(2019): 69215-69225.
• [16] Çınarer, Gökalp, and Bülent Gürsel Emiroğlu. "Classificatin of Brain Tumors by
Machine Learning Algorithms." 2019 3rd International Symposium on
Multidisciplinary Studies and Innovative Technologies (ISMSIT). IEEE, 2019.
• [17]  Kalaivani, A. S. Oliver, R. Pugalenthi, P. N. Jeipratha, A. A. S. Jeena and G.
Saranya, "Brain Tumor Segmentation Using Machine Learning Classifier," 2019
Fifth International Conference on Science Technology Engineering and
Mathematics (ICONSTEM), Chennai, India, 2019, pp.85-90
• [18] Abiwinanda, Nyoman, et al. "Brain tumor classification using convolutional
neural network." World Congress on Medical Physics and Biomedical Engineering
2018. Springer, Singapore, 2019.

Pakistan Institute of Engineering & Technology, Multan 43

References:
• [19] http://www.afh.com.pk
• [20] https://www.youtube.com/watch?v=oijmnoZ76Yk&t=1043s

Pakistan Institute of Engineering & Technology, Multan 44

45