Neural Networks 106 (2018) 79–95

Contents lists available at ScienceDirect

Neural Networks
journal homepage: www.elsevier.com/locate/neunet

An end to end Deep Neural Network for iris segmentation in

unconstrained scenarios
Shabab Bazrafkan a, *, Shejin Thavalengal b , Peter Corcoran a
a
Department of Electronic Engineering, College of Engineering, National University of Ireland Galway, University Road, Galway, Ireland
b
Xperi Galway, Cliona Building One, Parkmore East Business Park, Ballybrit, Galway, Ireland

article info a b s t r a c t
Article history: With the increasing imaging and processing capabilities of today’s mobile devices, user authentication
Received 8 December 2017 using iris biometrics has become feasible. However, as the acquisition conditions become more uncon-
Received in revised form 8 May 2018 strained and as image quality is typically lower than dedicated iris acquisition systems, the accurate
Accepted 21 June 2018
segmentation of iris regions is crucial for these devices. In this work, an end to end Fully Convolutional
Available online 30 June 2018
Deep Neural Network (FCDNN) design is proposed to perform the iris segmentation task for lower-
quality iris images. The network design process is explained in detail, and the resulting network is
Keywords:
Deep Neural Networks trained and tuned using several large public iris datasets. A set of methods to generate and augment
Data augmentation suitable lower quality iris images from the high-quality public databases are provided. The network is
Iris segmentation trained on Near InfraRed (NIR) images initially and later tuned on additional datasets derived from visible
images. Comprehensive inter-database comparisons are provided together with results from a selection
of experiments detailing the effects of different tunings of the network. Finally, the proposed model is
compared with SegNet-basic, and a near-optimal tuning of the network is compared to a selection of
other state-of-art iris segmentation algorithms. The results show very promising performance from the
optimized Deep Neural Networks design when compared with state-of-art techniques applied to the same
lower quality datasets.
© 2018 Elsevier Ltd. All rights reserved.

1. Introduction recognition for practical deployment was proposed and sub-

sequently developed by Daugman (1994) during the 1990s.
Biometric technology has become increasingly integrated into Such systems acquired the iris pattern using a dedicated
our daily life, from unlocking the smartphone to cash withdrawals imaging system that constrained the target eye and em-
from ATMs to shopping in the local supermarket (Pando, 2017). ployed near-infrared (NIR) imaging.
Various biometric modalities such as face, iris, retina, voice, finger- (ii) Systems supporting the acquisition of iris patterns from mo-
prints, palm prints, palm geometry are being used in a multitude bile persons, in unconstrained acquisition conditions, were
of applications including law enforcement, border crossing and developed during the 2000s, with the Iris On The Move
consumer applications (Corcoran & Costache, 2016; Thavalengal & system from Sarnoff being one of the better known of these
Corcoran, 2016). The iris of the human eye – the annular region (Matey et al., 2006). This system was designed for deploy-
between the pupil and sclera – is of particular interest as iris is ment in public spaces such as airports and requires people
a biometric modality with high distinctiveness, permanence, and
to walk along a specified path where multiple successive
performance (Prabhakar, Pankanti, & Jain, 2003).
iris images are acquired by a multi-camera systems under
The historical evolution of Iris recognition systems can be
controlled lighting conditions.
broadly summarized by a number of key stages, each presenting
(iii) Most recently iris recognition has been developed and
a new set of unique challenges over earlier implementations of the
deployed on handheld devices including smartphones
technology:
(Thavalengal & Corcoran, 2016). Such image acquisition
(i) The original proposal for the use of the iris as a biometric was is unsupervised and to a large extent unconstrained. This
made by the ophthalmologist Burch in 1936 (Irsch, Guyton & introduces new artifacts that are not found in earlier ac-
Johns, 2009) and the underlying technology to automate iris quisition environments including unwanted reflections, oc-
clusions, non-frontal iris images, low contrast and partially
blurred images. Isolating iris regions accurately in such an
* Corresponding author.
E-mail address: [email protected] (S. Bazrafkan). acquisition environment has proved to be more challenging

https://doi.org/10.1016/j.neunet.2018.06.011
0893-6080/© 2018 Elsevier Ltd. All rights reserved.
80 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95

and requires improvements to the authentication workflow a circular edge detector which searches over the image domain
as will be discussed shortly. for the best circle fit. Applying this operator twice, one can obtain
the two circular boundaries of iris. After this step, the occluding
The majority of existing iris recognition systems follow the au- eyelashes are detected with the help of curvilinear edge detection.
thentication workflow as (i) image acquisition: an eye image is There have been several similar techniques for iris segmentation
acquired using a camera, (ii) iris segmentation: eye/iris region is such as the algorithm proposed by Wildes et al. (1996), Kong and
located in this image followed by isolating the region representing Zhang (2001), Tisse et al. (1992) and Ma, Wang, and Tan (2002).
iris. (iii) Feature extraction: relevant features which represent the (All of these use circular Hough transform for finding the circles).
uniqueness of the iris pattern is extracted from the iris region and Another segmentation technique proposed by He, Tan, Sun, and
(iv) similarity of the two iris representation is evaluated by pattern Qiu (2009) uses an Adaboost-cascade iris detector and an elastic
matching techniques. model named ‘pulling and pushing method’.
The work presented in this paper focuses on successful seg- Further studies revealed that iris and pupil boundaries are not
mentation of non-ideal iris images, an essential element of the always circular, and modeling this accurately, improves the iris
authentication workflow if an unacceptably high level of failed recognition performance (Daugman, 2007). Daugman’s follow up
authentications is to be avoided. work (Daugman, 2007) incorporates active contours or snakes to
model the iris accurately. A similar approach proposed by Shah
1.1. Significance of iris segmentation
and Ross (2009) uses geodesic active contours for accurate iris
segmentation. Such techniques were shown to have high segmen-
Iris segmentation involves the detection and isolation the iris
tation accuracy in high quality images captured using dedicated iris
region from an eye image. The subsequent feature extraction and
cameras in the NIR region of electromagnetic spectrum. Proenca
pattern matching stages of any authentication workflow rely on
and Alexandre noted the poor performance of iris segmentation
the accurate segmentation of the iris and failed segmentations
techniques developed for high quality images when applied to
represent the single largest source of error in the iris authentica-
non-ideal images (Proença & Alexandre, 2006). A recent literature
tion workflow (Bigun, Alonso-Fernandez, Hofbauer, & Uhl, 2016;
survey on non-ideal iris image segmentation can be found in Jan
Erbilek, Da Costa-Abreu, & Fairhurst, 2012; Proença & Alexandre,
(2017). Among the literature, it is worth to be noted the efforts of
2010). For an accurate segmentation, the exact iris boundaries at
mobile iris challenge evaluation (MICHE) to provide a forum for
pupil and sclera have to be obtained, the occluding eyelids have
comparative research on the contributions to the mobile iris recog-
to be detected, and reflections have to be removed, or flagged.
nition field (De Marsico, Nappi, Riccio, & Wechsler, 2015; Marsico,
Errors at the segmentation stage are propagated to subsequent
Nappi, & Proença, 2017). Techniques based on various adaptive
processing stages (Bigun et al., 2016; Hofbauer, Alonso-Fernandez,
filtering and thresholding approaches are shown to be performing
Wild, Bigun, & Uhl, 2014). Detailed analysis of the impact of iris
well in these non-ideal scenarios (Gangwar, Joshi, Singh, Alonso-
segmentation is studied in Bigun et al. (2016), Erbilek et al. (2012)
and Proença and Alexandre (2010). Fernandez, & Bigun, 2016; Haindl & Krupička, 2015).
Numerous factors can introduce challenges in accurate iris
segmentation (Jillela & Ross, 2013) even on high-resolution iris 1.2.2. Applications of CNNs in iris recognition
systems. Examples include (i) occlusions caused by the anatomical In the last decade, deep learning techniques have become the
features of the eye; (ii) illumination conditions; (iii) user cooper- focus of intense research and the most successful approach in
ation; (iv) environmental factors; (v) noise & manufacturing vari- artificial intelligence and machine vision science. Deep learning
ations in image sensor technology; (vi) nature of the interacting based techniques are noted to provide state of the art results in
population. These factors apply to all iris acquisition systems. various applications such as object detection (Szegedy, Toshev, &
For mobile devices, in addition to these generic factors, there are Erhan, 2013), face recognition (Schroff, Kalenichenko, & Philbin,
additional concerns. Various image quality factors can also affect 2015), driver monitoring systems (Lemley, Bazrafkan, & Corcoran,
iris segmentation (Alonso-Fernandez & Bigun, 2013) and these 2017a, b, c), etc. In such deep learning based approaches, the input
become a limiting factor in consumer devices such as smartphones signal (image) is processed by consecutive signal processing units.
due to the challenging nature of acquiring suitable high-quality These units re-orient the input data to the most representative
images in a user-friendly smartphone use-case (Thavalengal, Bi- shape considering the target samples. The signal processing units
gioi, & Corcoran, 2015a, b). Hence, an iris segmentation technique are known as layers which could be convolutional or fully con-
which can accurately isolate the iris region in such low-quality nected. The fully convolutional models, such as the one presented
consumer images is important for the wider adoption constraint- in this work, are using only convolutional layers. These layers apply
free consumer iris recognition system. filters (known as kernels) to their input while the filter parameters
This work proposes to significantly improve the quality of iris are learned in the training step. In order to get better conver-
segmentation on lower quality images by introducing an end-to- gence, several techniques including drop-out (Srivastava, Hinton,
end deep neural network model accompanied by an augmenta- Krizhevsky, Sutskever, & Salakhutdinov, 2014) and batch normal-
tion technique. These improvements should enable improved iris ization (Ioffe & Szegedy, 2015) are presented in the literature. A
authentication systems for today’s mobile devices, encouraging a detailed introduction to CNNs (Convolutional Neural Networks)
broader adoption of iris recognition in day-to-day use cases. and its applications can be found in (Lemley et al., 2017a, b, c).
Recently, deep learning and convolutional neural networks are
1.2. Related literature & foundation methods applied in the domain of iris recognition. Minaee, Abdolrashidiy,
and Wang (2017) proposed the deep features extracted from VGG-
1.2.1. Iris segmentation Net for iris recognition. Authors in this work skipped iris segmen-
A detailed review of iris segmentation literature can be found in tation step in their framework, and hence it can be considered as
Bowyer, Hollingsworth, and Flynn (2008, 2013). Early work on iris a peri-ocular recognition more than iris recognition. Gangwar and
segmentation approximated the pupillary and limbic boundaries Joshi (2016) proposed a generalizable iris recognition architecture
as circles (Bowyer et al., 2008). An appropriate circular fitting for iris representation. An open source OSIRIS implementation for
method is incorporated for modeling these boundaries. Daugman’s iris segmentation is used. While authors note high generalizability
original work uses an integrodifferential operator for iris segmen- and cross-sensor performance, the segmentation errors generated
tation (Daugman, 2004). This integrodifferential operator acts as by OSIRIS could be affecting the result of this system. Liu et al.
 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95 81

proposed DeepIris for heterogeneous iris verification (Liu, Zhang, 1.2.4.2. Semi Parallel Deep Neural Network (SPDNN). The second
Li, Sun, & Tan, 2016). Also, deep learning based approaches for contribution of this work is the use of the recently introduced
spoof and contact lens detection can be found in Menotti et al. network design method called Semi Parallel Deep Neural Network
(2015) and Silva et al. (2015) (SPDNN) (Bazrafkan & Corcoran, 2017; Bazrafkan, Javidnia, Lemley,
& Corcoran, 2017) for generating iris maps from low quality iris
1.2.3. CNN for iris segmentation images. In an SPDNN, several deep neural networks are merged
Li et al. produced two CNN based models for iris segmentation into a single model to take advantage of every design. For a specific
(Liu, Li et al., 2016)-(i) hierarchical convolutional neural network task, one can design several DNN models each of them having
(HCNN) with three blocks of alternative convolutional and pooling advantages and shortcomings. The SPDNN method gives the possi-
layers fed directly in to a fully connected layer; and (ii) multi-scale bility of merging these networks in layer level using graph theory
fully convolutional network (MFCN) which contains six blocks of calculations. This approach maintains the order of the kernels from
interconnected alternative Conv and Pool layers fused through a the parent networks in the merged network. The convergence and
single multiplication layer followed by a Softmax layer. Jalilian and generalization of this method along with various application can
Uhl (2017) proposed three types of fully convolutional encoder– be found in Bazrafkan and Corcoran (2017) and Bazrafkan, Javidnia
decoder networks for iris segmentation. Arsalan et al. (2017) pro- et al. (2017). In the present work, the model is trained to generate
posed a two-stage iris segmentation based on CNNs for images an iris map from such low-quality images.
captured in visible light. Authors used circular Hough transform to
detect rough iris boundary in the first stage. A pre-trained VGG-
1.3. Contribution
face model is used in the second stage for the fine adjustment
of rough iris boundary obtained in the first stage. In order to
This work targets the iris segmentation in low-quality con-
overcome the requirement of large labeled data in the approaches
sumer images such as the images obtained from a smartphone. An
mentioned above, Jalilia, Uhl and Kwitt proposed a domain adap-
end to end deep neural network model is proposed to isolate iris
tion technique for CNN based iris segmentation (Jalilian, Uhl, &
region from the eye image. The proposed segmentation technique
Kwitt, 2017).
could be used with any existing state of the art feature extraction
1.2.4. Foundation methods and matching module without changing the whole authentica-
The two primary contribution of this work are (i) a novel iris tion workflow. Performance evaluation of the proposed technique
database augmentation and (ii) a semi parallel deep neural net- shows advantages over recent iris segmentation techniques pre-
work. sented in the literature. There are notably three primary contribu-
tions in this work.
1.2.4.1. Database augmentation. Since deep learning approaches
need a large number of samples to train a deep network, data aug- 1- An improved data augmentation technique optimized to gen-
mentation becomes a crucial step in the training process. Database erate diverse low-quality iris images. Such iris images are rep-
augmentation is the process of adding variation to the samples in resentative of unconstrained acquisition on a handheld mobile
order to expand the database and inject uncertainty to the training device from multiple established iris research databases.
set which help the network avoid overfitting and also generaliz- 2- A sophisticated iris segmentation network design derived us-
ing the results. Also, the augmentation step can introduce more ing Semi Parallel Deep Neural Network techniques; Design and
variations into the database and helps the network to generalize optimization methodologies are presented in detail.
its results. The most well-known augmentation techniques include 3- A detailed evaluation of the presented iris segmentation ap-
flipping, rotating and adding distortions to the image are widely proach is presented on various publically available databases.
used in expanding databases. Such techniques are usually used The presented method is compared with state of the art tech-
blindly and do not always guarantee any boost in the performance niques in iris segmentation.
(Lemley et al., 2017a, b, c).
In Lemley et al. (2017a, b, c), the authors proposed a smart
In the next section, the database and augmentation technique is ex-
augmentation method which combines two or more samples of
plained. The network design and Training is explained in Section 3
the same class and generates a new sample from that class. This
followed by results given in Section 4. The last section explains
method can give superior results compared to classical augmenta-
the numerical results, experiments on tuning, and comparisons to
tion techniques. Unfortunately, this method is only applicable to
state of the art segmentation methods.
classification problems, and unlike some types of augmentation
it cannot be used to manipulate the network results toward pre-
determined outcome or task. The ability to do this is another 2. Databases & augmentation methodology
important use of the augmentation, for example adding motion
blur to the samples can introduce the robustness to the motion blur In this work, four datasets are used for training and evaluation.
in the final results. Bath800 (Rakshit, 2007) and CASIA Thousand (‘‘CASIA Iris Image
The data augmentation technique employed in this work is Database’’, 2010) have been used in training and testing stages.
designed for the specific task of iris recognition. A high quality, ISO UBIRIS v2 (Proenca, Filipe, Santos, Oliveira, & Alexandre, 2010)
standard compliant iris image (Biometrics, 2007) is degraded to and MobBio (Sequeira, Monteiro, Rebelo, & Oliviera, 2014) are
make a reasonable representation of the real-world, low-quality taking part in tuning and also testing. Bath800 and CAISA thou-
consumer grade iris images. This degradation reduces iris–pupil sand has been augmented to represent more Real world consumer
and iris sclera contrast along with the introduction of various grade situations. 70% of the samples have been used for train-
noises. Iris maps obtained from state of the art iris segmentation ing/tuning, 20% for validation and 10% for testing. The network is
techniques are used to aid this process. This particular strategy trained initially on CASIA Thousand and Bath800 and tested on all
of augmentation is employed to harness the highly accurate seg- databases. For further observations, the original network has been
mentation capabilities of the state of the art iris segmentor. In this tuned on UBIRIS v2 and MobBio separately and also on a mixed
way, images which are a reliable representation of the low-quality UBIRIS+MobBio database. The experiments and discussions are
consumer images can be obtained along with the corresponding given in Section 5. Following is introducing databases used in this
iris map for the training of the neural network. work, followed by the ground truth generation and augmentation.
82 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95

Fig. 1. Eye socket samples from CASIA Thousand database.

Fig. 2. Eye socket samples from Bath800 database.

Fig. 3. Eye socket samples from UBIRIS database.

Fig. 4. Eye socket samples from MobBio database.

2.1. Datasets 2.2. Ground truth generation

2.2.1. Bath800 and CASIA Thousand

CASIA Thousand is a subset of CASIA Iris v4 database, and it con- Neither of Bath800 and CASIA Thousand databases are pro-
tains 20000 NIR images captured from 1000 individuals. Images vided with ground truth segmentation. As mentioned before these
databases contain very high-quality images taken in highly con-
are constrained high quality with high contrast. The resolution is
strained conditions. In this work, the segmentation from high-
[640 × 480] for all images. Samples of this database are shown quality images obtained using a commercial iris segmentation
in Fig. 1. Bath800 is a high-quality iris database taken under near solution (MIRLIN (‘‘MIRLIN’’, n.d.)) is considered as the ground
infrared illumination using a Pentax C-3516 M with 35 mm lens. truth for training stage. It can be noted that any commercial, high
Image resolution is [1280 × 960]. The database is made of 31997 performing segmentation technique could be used here as these
images taken from 800 individuals. The images are high quality high quality images could be segmented accurately using such
commercial systems. This specific choice of segmentor used in
and high contrast. Fig. 2 shows samples of this database. UBIRIS
this work is based on its availability and its performance on large
v2 database has 11102 images taken in visible wavelength with a scale iris evaluations (Quinn, Grother, Ngan, & Matey, 2013) Some
Canon EOS 5D which are relatively low-quality images captured segmentation examples are given in Figs. 5 and 6. The low reso-
in an unconstrained environment. The resolution of the database lution segmentation for Bath800 and CASIA Thousand is publicly
is 400 × 300. This database is not used in our training step. Some available.1
samples of this database are shown in Fig. 3. MobBio is a multi-
2.2.2. UBIRIS and MobBio
modal database including face, iris, and voice of 105 volunteers.
The manual segmentation of UBIRIS is available in IRISSEG-
The iris subset is used in the current work. The iris images are
EP database (Hofbauer et al., 2014) generated by WaveLab.2 The
taken in several orientations, with different levels of occlusion. 16 ground truth generation for this database is not completed. Only
images are taken from each individual and cropped and resized to segmentations for 2250 images from 50 individuals are given in
resolution 300 × 200. This database is highly unconstrained and
one of the most challenging sets in iris segmentation/recognition. 1 https://Goo.gl/JVkSyG.
Some samples of this database are shown in Fig. 4. 2 http://www.wavelab.at/sources/Hofbauer14b/.
 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95 83

Fig. 5. Bath800 automatic segmentation results.

Fig. 6. CASIA Thousand automatic segmentation results.

Fig. 7. UBIRIS manual segmentation samples in IRISSEG-EP.

IRISSEG-EP. Some samples for UBIRIS segmentation is shown in cases turns them into absolute zero intensity. This operation is
Fig. 7. not reversible as well. The blurring effect is applied in random
The manual segmentation for the MobBio database is included direction and random intensity, which makes it nontrivial to find
in IRISSEG-CC database (Hofbauer et al., 2014) generated by ISLAB.3 the deblurring filter.
The whole MobBio dataset has been segmented in IRISSEG-CC. All these operations are applied with precise observation on
Some samples for MobBio segmentation are given in Fig. 8. low quality consumer level iris images and the parameters are
set to simulate the wild conditions. These are next discussed in
2.3. Data augmentation turn with details of the augmentation approaches taken for each
independent factor. The code for augmenting the database is also
In this work two high-quality databases, Bath800 and CASIA available.4
Thousand have been used in training stage. In this section, the aug-
mentations applied to the high-quality iris images are explained. 2.3.1. Eye socket resolution
In order to find the best augmentations for the iris images, precise The resolution of the eye socket plays an essential role in how
observations have been done on low-quality iris images. The differ- much information one can extract from the image. In fact, while
ence between a high quality constrained iris images and consumer dealing with the front camera in a mobile phone, the resolution of
grade images depend on five different independent factors: 1- eye the camera is lower than the rear cameras. For example, Bazrafkan,
socket resolution, 2- image contrast, 3- shadows in the image, 4- Kar, and Costache (2015) observed that the number of the pixels in
image blurring, 5- noise level (Thavalengal et al., 2015a, b). In our the iris region for an image taken by a 5 MP front camera of a typical
observations, the noise level was low in unconstrained images. cell phone from the 45 cm distance was just 30 pixels. In order to
Noise is a well-studied phenomenon, and image de-noising can simulate the low resolution scenario, the high-quality eye socket
be done outside the network and also note that introducing high images and their corresponding ground truth have been resized
frequency noise into the dataset trains a low-pass filter inside using bilinear interpolation into smaller images [128 × 96]. This
the network; apply de-noising outside the network gives a higher can help the deep network to train faster as well.
chance to use the whole network potential to perform the seg-
mentation task. In addition, introducing Gaussian noise into the 2.3.2. Image contrast
dataset will cause underfitting as explained in Zheng, Song, Leung, In our observations of low quality iris images taken from hand-
and Goodfellow (2016). Therefore, in this work, the focus is on the held devices, the intensity properties of the image inside and out-
first four factors. side the iris region were different. In fact in the low-quality image
The augmentations are applied in a random manner. For exam- set, the region inside the iris was darker than the same region in
ple, the contrast, shadow and blurring are applied with random high-quality images. There was no specific brightness quality for
parameters. These operations are lossy. i.e., there is no inverse the regions outside the iris in low-quality images. They could be
operation which can regenerate original high quality images from customarily exposed or strongly bright or very dark. In the low-
low quality ones. Reducing the resolution of the iris image is a non- quality images, the regions outside the iris were suffering from the
reversible task. The contrast reduction contracts a large portion of low amount of contrast. In order to apply this transformation to
the histogram into a smaller region, and the quantization process the high-quality images, the contrast reduction inside and outside
turning the image into unit8 class results in information loss. of the iris region are targeted with different operations. For outside
Shadowing is reducing the intensity of the pixels and in some
4 https://github.com/C3Imaging/Deep-Learning-Techniques/blob/Iris_SegNet/
3 http://islab.hh.se/mediawiki/Iris_Segmentation_Groundtruth. DBaugmentation/DBaug.m.
84 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95

Fig. 8. MobBio manual segmentation samples in IRISSEG-CC.

Fig. 9. The histogram mapping for the outside region of the iris. Fig. 10. The histogram mapping for the region inside the iris.

of the iris region the following histogram transformation is applied.

This mapping can result is bright, dark, or normally exposed low
contrast outputs.

y = norm (tanh (3 × (x/255 − 0.5) + U (−0.3, 0.3))) × 255 (1)

where x is the input intensity in the range [0,255], y is the output
intensity in the same range, U (a, b) is the Uniform distribution
between a and b, and the norm function normalize the output
between 0 and 1. The Uniform distribution injects an amount of
uncertainty into the mapping which helps the generalization of the
model. The mean and standard deviation of the histogram mapping
curve is shown in Fig. 9.
The inside of the iris is mapped using following histogram
transformation.
Fig. 11. For inside the iris region, the contrast is reduced, and the region is getting
y = norm (tanh (3 × (x/255 − 0.5) − U (0, 0.8))) × 255 (2) darker. The outside of iris is just altered by decreasing the contrast.

The mean and standard deviation of the histogram curve is

shown in Fig. 10. This transformation is darkening the area while
decreasing the contrast at the same time. is shown in Fig. 12. Based on the value of the randSign function, the
An example of applied histogram mapping is showed in Fig. 11. shadowing direction is changed.
An example of shadowing is given in Fig. 13.

2.3.3. Shadows in the image 2.3.4. Image blurring

Low quality unconstrained iris images are profoundly altered by Bokeh effect caused by camera focus (Phillips & Komogortsev,
the direction of the illumination. In this work, shadowing is carried 2011), the hand jitter and unwanted head and hand movements
out by multiplying the image columns by the following function. are highly degrading the iris image quality in handheld devices.
One of the main challenge in iris segmentation in this scenario is
the blurring of the iris edges which affects the edge detection task.
y = norm (tanh (2 × randSign In majority of the iris segmentation methods, the gradient of the
× (x − 0.5 + U (−0.3, 0.3)))) + U (0, 0.1) (3) pixel intensity is used to find the iris region, and the blurring is
highly altering the gradient quality. The deep neural network is
where x is the dummy variable for image column number and y able to solve this problem if enough variation of motion blurring
is the coefficient for intensity, U (a, b) is the Uniform distribution is provided in the dataset. In order to include this effect in the
between a and b, and the norm function normalize the output training set, shadowed image is passed through a motion blur
between 0 and 1 and randSign generates a random coefficient in filter applying the linear camera motion by U (5, 10) pixels in the
the set {−1, 1}. The mean and standard deviation of this function direction U (−π, π), where U (a, b) is the Uniform distribution
 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95 85

Fig. 12. Mean and standard deviation for shadow coefficients.

The parent networks used in this work are fully convolutional

networks with different depths and kernel size each designed to
extract different levels of details. The network after merging these
networks is shown in Fig. 9 of Supplementary Material #1. The
model looks like a U-net (Ronneberger, Fischer, & Brox, 2015) with
the difference that there is no pooling applied in our proposed
network. The number of the channels in each layer is determined
Fig. 13. Shadowing applied to low contrast image. in Supplementary Material #2. The calculations guarantee that
the number of the parameters in the presented method is similar
to SegNet-Basic proposed in Badrinarayanan, Kendall, and Cipolla
(2015). Having the same number of parameters helps to obtain a
fair comparison between SegNet-Basic and proposed model.

3.2. Training

The proposed network is an end to end design which means that

Fig. 14. Applying motion blur in a random direction to the low contrast shadowed
image.
it accepts the eye socket image and gives the iris map. The network
has been trained using lasagna library (Dieleman et al., 2015) on
the top of the theano library (Al-Rfou et al., 2016) in python. The
loss function used in our work is the mean binary cross-entropy
between a and b. The final image after applying motion blur is between the output and target given by
shown in Fig. 14.
B N M
All the samples in Bath800 and CASIA Thousand databases are 1 ∑ ∑ ∑ ( ) ( ) ( )
degraded using this augmentation step. Some examples of the low- L= tij log pij − 1 − tij log 1 − pij (4)
M ×N ×B
quality samples and their corresponding ground truth is given in k=1 j=1 i=1

Fig. 15. wherein tij is the value of pixel (i, j) in the target image, pij is the
value of pixel (i, j) in the output image for the image of the size
3. Network design and training M × N and B is the batch size. The stochastic gradient descent with
momentum has been used to update the network parameters. The
3.1. Network design momentum term prevents the gradient descent to stick in the local
minimums, and also speeds up the convergence. In this approach,
Deep neural networks are capable of solving highly nonlinear the gradient decent uses the update value of the previous itera-
and challenging problems. In this work, four different end to end tion as the momentum in the current iteration. Suppose the loss
fully convolutional deep neural networks have been proposed to function is L (w) where w, is the set of network parameters. The
perform the iris segmentation on low quality images. These net- stochastic gradient method with momentum is given by
works are merged using SPDNN method, and the number of the
w := w − η∇ L (w) + α ∆w (5)
channels in each layer is selected in a way that the number of
the parameters in the proposed network is similar to the SegNet- wherein ∆w , is the update in the previous iteration, ∇ L (w) is the
basic. The network design and calculating the number of channels gradient value in the current iteration, η is the learning rate and,
in each layer are explained in detail in Supplementary Material #1 α is the momentum. In our training experiments, the learning rate
and #2 respectively. The SPDNN method is merging several deep and momentum are set to 0.001 and 0.9 respectively. The training
neural networks in the layer level using graph theory calculation method and learning parameters in training the proposed network
and graph contraction. This approach is preserving the order of the and SegNet-basic are same.
layers from the parent networks. The convergence and generaliza- The network is trained on an augmented version of Bath800 and
tion of SPDNN is discussed in Bazrafkan and Corcoran (2017) and CASIA1000 originally. Some experiments have been conducted on
other applications of this method is given in Bazrafkan, Javidnia et this original network given in Sections 5.2 and 5.3. These databases
al. (2017). are NIR databases. In order to provide a network, segmenting
86 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95

Fig. 15. Augmented samples and their corresponding segmentation map.

visible images, the original network has been tuned on UBIRIS and 1- Test on the original network: The proposed network is initially
MobBio databases. The same training method has been used in the trained on the augmented version of the Bath800 and CA-
tuning stage while learning rate and momentum are set to 0.001 SIA1000 databases. The first experiment compares the output
and 0.9 respectively. The train/tune has been done for 1000 epochs. of this network for different databases. The test set of Bath800
More details on tuning results is given in Supplementary Material and CASIA1000 and all the samples of UBIRIS and MobBio are
#3 and #4. used in the test stage. Section 5.2 discusses this experiment in
detail.
4. Results 2- Comparison with SegNet-Basic: This experiment discusses the
results of presented network compared the SegNet basic. Train-
In the test step, each eye socket image is given to the ing and testing for SegNet-Basic is done similar to the proposed
trained/tuned network, and the forward propagation is performed network. This experiment is presented in Section 5.3.
for this input. In the training stage, the output of the network is 3- Comparison to state of the art: In this experiment, the best
forced to converge to the iris segmentation map which is a binary results of the proposed method is compared with other meth-
image. The output of the network is a grayscale segmentation map,
ods in the literature. The numerical results are presented in
and the binary map is produced by thresholding technique, i.e., the
Section 5.4.
values bigger than a threshold are shifted to 1 and the others to 0.
The threshold value 0.45 has been used in our experiments. The Two side experiments has been conducted on tuning the net-
output of the proposed model for different databases are shown in work with visible datasets. These experiments are as follows:
Figs. 16 to 19.
Figs. 16 and 17, show the high-quality output for Bath800 and 1- Tuning; Network experiment: In this experiment, the original
CASIA1000 databases. These datasets are high quality constrained network trained on the augmented version of Bath800 and CA-
NIR sets, and their images follow a specific distribution which SIA1000 is tuned on UBIRIS and MobBio individually and also on
makes it easier for the DNN to perform the segmentation task.
a mixture of these two databases. In this way, the effectiveness
Figs. 18 and 19, show the output of the proposed network for more
of each database in boosting the performance is investigated.
difficult unconstrained UBIRIS and MobBio databases. These two
The network tuned on each database is tested on all databases.
figures show the results of the network tuned on these databases.
The results and discussions of this experiment is presented in
The results are not as good as Bath800 and CASIA1000, but one
Supplementary Material #3.
should note that these datasets are quite challenging and difficult
2- Tuning; Database experiment: This experiment is looking at
to segment. The numerical results are given in the following sec-
the results of previous experiment based on each database.
tion.
There are four networks trained and tuned which are presented
5. Evaluations in experiment 1 and 4 as follows: (i) Initially trained on Bath800
and CASIA1000. (ii) Tuned on UBIRIS. (iii) Tuned on MobBio.
Several metrics have been used to evaluate the network and (iv) Tuned on UBIRIS+MobBio. The output of each of these
investigate the tuning effect on the segmentation results. These networks for each database and also the average performance
metrics are presented in Table 1. In all equations True Positive is is investigated in this experiment. Supplementary Material #4
abbreviated as TP, True Negative as TN, False Positive as FP and is presenting this experiment in more detail.
False Negative as FN. Letter P stands for the number of all Positive
cases which is equal to TP+FN and N is the total number of negative In all experiments, µ stands for the average value for the given
cases equals to FP+TN. measure and σ is its standard deviation over all outputs.

5.1. Experimental results 5.2. Test on the original network

Three main experiments have been conducted to investigate In this experiment, the proposed trained network is tested over
the performance of the proposed network and the effect of the four databases (Bath800, CASIA1000, UBIRIS, and MobBio). The
tuning on the results. These experiments are as follows: reason for adding two more databases in the testing procedure is
 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95 87

Fig. 16. Output of the network for Bath800 test set. The results show high-quality output in this database.

Fig. 17. Output of the network for CASIA1000 test set. The results show high-quality output in this database.

Fig. 18. Output of the network for UBIRIS test set.

88 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95

Fig. 19. Output of the network for MobBio test set.

Table 1
Metrics used in the evaluation section.
Measure Description
Accuracy represents the ratio of all true results divided by the number
Accuracy
of all samples given by
Accuracy = TPP +TN
+N
This measure indicates the ability of the model to recall true positive
Sensitivity or True Positive Rate (TPR)
over all positive samples. i.e., a model with high sensitivity can rule out
negative samples more efficiently. Sensitivity is given by
Sensitiv ity = TPTP
+FN
= TPP
This measure indicates the ability of the model to recall true negative
Specificity or True Negative Rate
over all negative samples. i.e., a model with high specificity can find
positive samples more efficiently. Specificity is given by
Specificity = FPTN
+TN
= TN
N

Precision is the probability that the positive output is true positive in

Precision or Positive Predictive Value (PPV):
the space of all positive outcomes, which is given by
Precision = TPTP
+FP
NPV is the probability that the negative output is true negative in the
Negative Prediction Value (NPV)
space of all negative outcomes, which is given by
NPV = TNTN+FN
This measures the ability of the model to recall true positive cases and
F1 score
at the same time not missing positive cases. F1 score is given by
F 1 score = 2TP +2TP
FP +FN

Is a metric measuring the quality of binary classifiers. The critical

Matthew Correlation Coefficient (MCC)
property of MCC is its independence of the size of each class. It varies
in the range (−1,1) wherein 1 indicates the perfect model, and −1
declared that all output values are the inverse of the target value. MCC
is given by
MCC = √(TP +FP )(TPTP×+TN −FP ×FN
FN )(TN +FP )(TN +FN )

Is a metric showing the probability of informed decision given by the

Informedness
model. It ranges from −1 to 1; where 0 shows a random decision and 1
indicates no false outputs. Informedness is given by
Informedness = TPTP+FN
+ TNTN+FP
−1
Is a measure of the information content and information value of the
Markedness
model’s output (Battistella, 1990). Markedness is given by
Markedness = TPTP+FP
+ TNTN
+FN
−1

to observe the ability of the network in generalizing over other Since the network is trained on a merged version of the Bath800
databases. One of the main concerns in DNN community is to be and CASIA1000; only the test sub-set of these databases has been
able to generalize the trained network to unconstrained environ- used in testing stage. However, the network never saw the UBIRIS
ments. This is happening since the majority of Machine learning and MobBio set before. Therefore all samples of these databases
schemes are sharing the same database in train and test stage. The have been used for testing. The numerical results are shown in
neural networks learn the distribution of the data for the given Table 2. From this experiment, the network gives better results for
database, and since the test set follows the same distribution, it Bath800 and CASIA1000 which is expected.
gives promising results on the test set. Bazrafkan, Nedelcu, Fil- The network has already observed these two databases in
ipczuk, and Corcoran (2017) discusses this problem in more detail. training stage and learned their distribution which justifies the
 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95 89

Table 2
Testing on the original network. Metrics measured for different databases. Green
means higher performance and red declares lower quality results. A higher value of
µ and lower value for σ are desirable.

higher performance on these databases. Having a lower value second) which is comparable to the SegNet-basic 0.6144 Mp/s,
of sensitivity and precision in UBIRIS and MobBio databases de- while the proposed model have considerably better performance.
clares the amount of uncertainty of the model in giving back the Results show SegNet-basic has considerably better perfor-
positive cases. Moreover, the high value of specificity and NPV mance on Bath800 and CASIA1000 than other two databases;
shows that the trained model was able to rule out non-iris pixels which is expectable since the network is trained on former
in all databases. The value of F1-score, MCC, Informedness, and databases. In the following subsections, these results are compared
Markedness is high for Bath800 and CASIA1000 which indicate the to presented network in order to find the advantages and short-
ability of the network to produce consistent segmentations both comings of each design.
in finding iris and non-iris pixels for these databases. The same
measures return average values for UBIRIS database. This means 5.3.1. Comparing results on Bath800 and CASIA1000
that the network is generalized for semi-wild environments. More- Since both networks are trained on Bath800 and CASIA1000
over, the low value for MobBio indicates that the network is not databases, the numerical test results show the capability of each
much reliable to work in consumer level environments. In general, design in capturing the probability distribution of the training set.
the presented network is reliable in returning non-iris pixels in Figs. 20 and 21 illustrate the comparisons between the proposed
challenging scenarios. Note that both UBIRIS and MobBio are vis- method and SegNet-basic over Bath800 and CASIA100. These fig-
ible databases and that MobBio is a very challenging database. In ures show the mean value for each metric.
Section 5.4.2 the presented network is compared to other methods From these figures, it is concluded that the proposed method
on the MobBio database. is giving better results on the test set of Bath800 and CASIA1000.
Since these two datasets have been used to train both networks,
5.3. Comparison with SegNet-basic these comparisons show the higher capacity of the proposed
method in learning the data distribution in training stage. At the
SegNet (Badrinarayanan et al., 2015) is one of the most suc- same time, one can comment that learning the training distribution
cessful DNN approaches in semantic segmentation. SegNet-basic can be a sign of overfitting. In order to investigate this effect, both
is the small counterpart of the original SegNet. As explained in networks are tested on two other databases UBIRIS and MobBio
Supplementary Material #2, our proposed architecture contains explained in the next section.
almost the same number of parameters as SegNet-basic. This gives
us the opportunity to conduct fair comparisons between two mod- 5.3.2. Comparing results on UBIRIS and MobBio
els. The original SegNet-basic is trained on the road scenes for It is always important to investigate the performance of a model
object segmentation task. In order to conduct fair comparisons over databases which has not been used in the training stage in
with the proposed design, the SegNet-basic network is tuned for order to get better ideas on the model quality in unconstrained
iris segmentation by training it on the same data with same hyper- environments. In fact, the network is learning the samples which
parameters as our proposed model. Table 3 shows the results for are present in the training set, and a good model should be able to
SegNet-basic tested on four databases Bath800, CASIA1000, UBIRIS generalize the results for other samples specially unconstrained,
and MobBio. Note that the network is tested on the test set of consumer graded and difficult ones. In this section, the results for
Bath800 and CASIA1000 and all samples of UBIRIS and MobBio. the SegNet-basic network is compared with the proposed network
The SegNet-basic and proposed network contain almost the for UBIRIS and MobBio databases (which are not used in the train-
same number of parameters, so these two networks occupy the ing stage). The results are shown in Fig. 22 for UBIRIS and Fig. 23
same amount of memory. The proposed architecture has more for MobBio.
weight connections. On a Geforce 750Ti graphic card the process- The presented method is providing higher accuracy than
ing load for the proposed model was 0.77 Mp/s (Megapixel per SegNet-basic which implies the better quality in returning true
90 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95

Table 3
SegNet-basic. Metrics measured for different databases. Green means better quality and red declares
lower quality results. A Higher value of µ and lower value for σ are desirable.

Fig. 20. Comparisons between the presented network and SegNet-basic on Bath800. A higher value indicates better performance.

Fig. 21. Comparisons between the presented network and SegNet-basic on CASIA1000. Higher value indicates better performance.
 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95 91

Fig. 22. Comparisons between the presented network and SegNet-basic on UBIRIS. A higher value indicates better performance.

results in the space of all results. This is while SegNet-basic has overcome the problem of low contrast and noise interference in
higher sensitivity and NPV which it means that this architecture the eye socket image. Tan, He, and Sun (2010) proposes an inte-
is more efficient in ruling out non-iris pixels while the presented grodifferential constellation followed by a curvature fitting model
model has better performance in finding positive samples due to to find the iris area. In Radman, Zainal, and Suandi (2017) The
its higher specificity and precision. Lower FPR shows that the pro- Histogram of Oriented Gradients (HOG) is introduced as feature
posed model had lower probability in returning a negative decision and Support Vector Machine (SVM) is used to perform the auto-
and higher FNR shows that SegNet-basic is less probable in making matic segmentation of iris. The random walker algorithm is used
a mistake in returning positive decisions. However, in average the to generate the iris map in Tan and Kumar (2013). In Proenca
proposed method is more efficient since it has a higher value for F1- (2010) the sclera and iris regions are detected separately using
score which is the harmonic average of precision and sensitivity. neural networks as classifiers, and polynomial fitting is applied
Moreover, also higher MCC shows that the overall performance estimating the final iris region. Tan and Kumar (2012) proposes
of the presented network is better than SegNet-basic for UBIRIS a post-classification procedure including reflection and shadow
dataset. removal and several refinements on pupil and eyelid localizations
The numerical results of testing both networks on MobBio to get higher performance on iris segmentation task. From Fig. 24,
dataset are shown in Fig. 23. the proposed methods gives the best accuracy for UBIRIS database.
Results for MobBio database is in the same direction as UBIRIS. The main advantage of our work was by selecting the proper
The proposed model got higher accuracy which means it has better data augmentation described in Section 2.3. Augmentation step is
performance in finding iris pixels. Values for sensitivity and NPV essential in any model which is designed to work in unconstrained
shows better performance of SegNet-basic in ruling out non-iris conditions.
pixels. And observations on specificity and precision shows the The network is learning the distribution of the train set and
better performance of the proposed model in returning iris pixels. therefore, designing a set which is representative of the consumer
However, on average the proposed method has better performance level conditions is crucial in order to get reasonably high perfor-
due to higher numerical values for F1-score and MCC. mance, i.e., if one can mimic the real-life situations by introducing
The results of testing both networks on UBIRIS and MobBio enough variations to the training set, it is highly probable that the
datasets demonstrate the overall improved performance of pro- network is able to generalize the learning into non constrained
posed network over SegNet-basic. This shows that the model is not input test samples. Moreover, also tuning is an essential part of our
only more capable of learning the training data distribution, but approach to getting a better result for a pre-defined condition. The
also it has a better ability to generalize to unconstrained, consumer original network was trained on Bath800 and CASIA1000 databases
level environments. which are NIR iris images. At the same time, the UBIRIS database
is not big enough to train a DNN without encountering over-fitting
5.4. Comparison to state of the art condition. One of the best approached to train a network on such
a small database is to transfer the information from the original
In this next experiment, the proposed method is compared to network (which is trained on NIR images) and tune it on the new
the most advanced and state of the art segmentation methods in database.
the literature. In the first part, the accuracy of the proposed method The other advantage of the proposed method is the semi parallel
is compared to other methods over UBIRIS database. Moreover, design of the network. In this approach, one can take advantage
in the second part, the sensitivity, precision, and F1-score of the of several architectures at the same time. This is while the infor-
proposed method is compared with some other methods over mation flow inside the network is not limited to a single path but
UBIRIS, MobBio, and CASIA databases. The results presented here mixing and merging of several paths.
are the best results of our networks after tuning.
5.4.2. Experiments on sensitivity, precision, and F1-score
5.4.1. Accuracy on the UBIRIS database The sensitivity, precision, and F1-score of five iris segmenta-
The comparisons of the proposed method with the state of art tion methods (CAHT, GST, IFFP, Osiris, and WAHET) on several
methods over UBIRIS database is illustrated in Fig. 24. databases including UBIRIS, MobBio and CASIA are given in Hof-
The MFCN and HCNN (Liu, Li et al., 2016) methods are using a bauer et al. (2014). In our work, the comparisons with these meth-
22 layer, deep neural network to perform the iris segmentation. ods have been conducted on the same databases, and the results
Zhao and Ajay (2015) utilizes the Total Variation (TV) model to are given in Figs. 25 to 27. The results reported by Hofbauer et al.
92 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95

Fig. 23. Comparisons between the presented network and SegNet-basic on MobBio. A higher value indicates better performance.

Fig. 24. Accuracy of proposed method vs. other methods over UBIRIS database.

(2014) for UBIRIS database provides a detailed performance metric performing on low-quality consumer graded iris images as good as
including sensitivity, precision and f1-score. These metrics for each constrained high-quality samples. This is essential while the user
presented algorithm (CAHT, GST, IFFP, OSIRIS, and WAHET) are cal- tries to capture the iris information in handheld devices where
culated by comparing the algorithms results with the ground truth. there is hand shaking, sparse illumination and low-quality front
Hofbauer et al. (2014) work aim to streamline the development cameras. The proposed network shows that this conditions could
and analysis of newer iris segmentation algorithms by providing be compensated by augmenting the data and also merging several
a detailed baseline result. Hence the results presented there for designs into a single network and the numerical results show
CAHT, GST, IFFP, OSIRIS, and WAHET segmentation algorithms is promising performance of the proposed scheme.
expected to be the best results for these methods.
More recent work on UBIRIS, including the segmentation ap-
6. Conclusions
proach based on CNNs such as Liu, Li et al. (2016) provides just
the segmentation accuracy. A detailed comparison of segmentation
accuracy of these techniques to our proposed technique is shown In this work, a deep neural network framework has been pre-
in Fig. 24. As the algorithms used in these works are not available sented to segment low quality, consumer graded iris images.
for research community for reproducing their results, calculating There are three main contributions in this work.
new metrics which are not provided by the authors are not feasible.
Other recent works on iris segmentation such as Abdullah, (i) The data augmentation, wherein the high quality eye socket
Dlay, Woo, and Chambers (2017); Ehsaneddin (Jalilian & Uhl, 2017; images from Bath800 and CASIA1000 database are degraded
Morley & Foroosh, 2017) neither compare their results on UBIRIS and manipulated to give a proper approximation of the low
dataset nor make the segmentation technique public for a fair quality images. Four different factors have been considered
comparison of their techniques in the other databases. including image resolution, contrast, shadow and motion
Sensitivity and precision metrics measure the quality of the blurring. The augmented images give a close approximation
network in ruling out non-iris pixels and detection iris ones respec- of low quality unconstrained iris images.
tively, and F1-score is the harmonic average of these two metrics. (ii) The recently introduced Semi Parallel Deep Neural Network
The higher values correspond to better performance. As shown in method has been used to design a fully convolutional net-
Figs. 25 to 27, the proposed method gives superior results com- work by mixing and merging four parent networks. Each of
pared to other approaches on UBIRIS and MobBio databases. More- these networks take advantage of different kernel sizes and
over, on the high-quality CASIA database, the proposed method is depths which are extracting and processing different feature
still giving better results. This shows that the proposed method is levels. The final design is similar to U-Net without pooling.
 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95 93

Fig. 25. Sensitivity, Precision, and F1-score on UBIRIS database for proposed method vs. five other methods. Higher values indicate better performance.

Fig. 26. Sensitivity, Precision, and F1-score on MobBio database for proposed method vs. five other methods. Higher values indicate better performance.

Fig. 27. Sensitivity, Precision, and F1-score on CASIA database for proposed method vs. five other methods. Higher values indicate better performance.

(iii) Inter-database evaluations are giving a more realistic databases, and the effect of tuning was widely investigated. Our
overview of the network performance. Here a very essential experiments show that the tuning boosts the performance for the
problem in deep leaning community is addressed wherein database that the network is tuned on. This is expectable except for
the researchers are training a DNN on a specific database databases with very unspecific distributions which will decrease
and test it on the same database. In this work, every network the performance after tuning. Another conclusion is that tuning the
was tested on Bath800, CASIA1000, UBIRIS, and MobBio. model is only advisable when the end use conditions are known.
Employing this approach gives a realistic foresight of the But if the end use conditions are not known, such tuning is not
performance on real world situations. advised.
Since the presented network is a large model which is not easily
The proposed model has been initially trained on the aug- implementable on a low power handheld hardware, the future
mented version of the Bath800, and CASIA1000 databases and works include optimizing the network, training a smaller network,
further experiments were carried out by tuning the original net- or binarizing the model to reduce the number of calculations and
work on UBIRIS and MobBio. Tuned networks were tested on all the memory usage. Optimizing the network includes reducing the
94 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95

parameter precision down to binary or ternary (Li, Zhang, & Liu, Corcoran, P., & Costache, C. (2016). Smartphones, Biometrics, and a Brave New
2016). This will give, up to 32x memory compression and also World. IEEE Technology and Society Magazine, 35(3), 59–66. http://dx.doi.org/10.
reduces the calculation load extensively by eliminating most of 1109/MTS.2016.2593266.
Daugman, J. G. (1994, March). Biometric personal identification system based on iris
multiplications in the model. Another approach is to design a
analysis. Google Patents.
model with fewer parameters. Currently, our target is to reduce Daugman, J. (2004). How iris recognition works. IEEE Transactions on Circuits and
the number of parameters to the rate of 10x without causing Systems for Video Technology, 14(1), 21–30. http://dx.doi.org/10.1109/TCSVT.
considerable cutback in the performance. 2003.818350.
Daugman, J. (2007). New methods in iris recognition. IEEE Transactions on Systems,
Acknowledgments Man, and Cybernetics. Part B, Cybernetics: A Publication of the IEEE Systems, Man,
and Cybernetics Society, 37(5), 1167–1175.
De Marsico, M., Nappi, M., & Proença, H. (2017). Results from MICHE II–mobile iris
This research is funded under the SFI Strategic Partnership
challenge evaluation II. Pattern Recognition Letters, 91, 3–10. http://dx.doi.org/
Program by Science Foundation Ireland (SFI) and FotoNation Ltd. 10.1016/j.patrec.2016.12.013.
Project ID: 13/SPP/I2868 on Next Generation Imaging for Smart- De Marsico, M., Nappi, M., Riccio, D., & Wechsler, H. (2015). Mobile iris challenge
phone and Embedded Platforms. evaluation (MICHE)-I, biometric iris dataset and protocols. Pattern Recognition
We gratefully acknowledge the support of NVIDIA Corporation Letters, 57, 17–23. http://dx.doi.org/10.1016/j.patrec.2015.02.009.
with the donation of a Titan X GPU used for this research. Dieleman, S., Schluter, J., Raffel, C., Olson, E., Sonderby, S. K., & Nouri, D. et al., (2015,
Portions of the research in this paper use the CASIA-IrisV4 col- August). Lasagne: First release. http://dx.doi.org/10.5281/zenodo.27878.
Erbilek, M., Da Costa-Abreu, M. C., & Fairhurst, M. (2012). Optimal configuration
lected by the Chinese Academy of Sciences’ Institute of Automation
strategies for iris recognition processing. In IET conference on image processing
(CASIA). (pp. B2–B2). http://dx.doi.org/10.1049/cp.2012.0451.
Gangwar, A., & Joshi, A. (2016). DeepIrisNet: Deep iris representation with appli-
Appendix A. Supplementary data cations in iris recognition and cross-sensor iris recognition. In Proceedings -
international conference on image processing, Vol. 2016–Augus, (pp. 2301–2305).
Supplementary material related to this article can be found http://dx.doi.org/10.1109/ICIP.2016.7532769.
online at https://doi.org/10.1016/j.neunet.2018.06.011. Gangwar, A., Joshi, A., Singh, A., Alonso-Fernandez, F., & Bigun, J. (2016). IrisSeg:
A fast and robust iris segmentation framework for non-ideal iris images. In
2016 international conference on biometrics http://dx.doi.org/10.1109/ICB.2016.
References 7550096.
Haindl, M., & Krupička, M. (2015). Unsupervised detection of non-iris occlusions.
Abdullah, M. A. M., Dlay, S. S., Woo, W. L., & Chambers, J. A. (2017). Robust iris Pattern Recognition Letters, 57, 60–65. http://dx.doi.org/10.1016/j.patrec.2015.
segmentation method based on a new active contour force with a noncircular 02.012.
normalization. IEEE Transactions on Systems, Man, and Cybernetics: Systems, He, Z., Tan, T., Sun, Z., & Qiu, X. (2009). Toward accurate and fast iris segmentation
47(12), 3128–3141. for iris biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence,
Al-Rfou, R., Alain, G., Almahairi, A., Angermueller, C., Bahdanau, D., & Ballas, N., et al., 31(9), 1670–1684. http://dx.doi.org/10.1109/TPAMI.2008.183.
(2016). Theano: A {Python} framework for fast computation of mathematical Hofbauer, H., Alonso-Fernandez, F., Wild, P., Bigun, J., & Uhl, A. (2014). A ground
expressions. arXiv E-Prints, abs/1605.0. Retrieved from http://arxiv.org/abs/ truth for Iris segmentation. In Proceedings - international conference on pattern
1605.02688. recognition (pp. 527–532). http://dx.doi.org/10.1109/ICPR.2014.101.
Alonso-Fernandez, F., & Bigun, J. (2013). Quality factors affecting iris segmentation Ioffe, S., & Szegedy, C. 2015. Batch normalization: Accelerating deep network train-
and matching. In Proceedings - 2013 international conference on biometrics, http: ing by reducing internal covariate shift. In International conference on machine
//dx.doi.org/10.1109/ICB.2013.6613016. learning (pp. 448–456).
Arsalan, M., Hong, H. G., Naqvi, R. A., Lee, M. B., Kim, M. C., Kim, D. S., et al. Irsch, K., Guyton, D. L., & Johns, H. M. (2009). Anatomy of eyes. Encyclopedia of
(2017). Deep learning-based iris segmentation for iris recognition in visible light Biometrics,(April), 1212–1217. http://dx.doi.org/10.1007/978-0-387-73003-5_
environment. Symmetry, 9(11). 257.
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2015). SegNet: {A} Deep Convolutional Jalilian, E., & Uhl, A. (2017). Iris segmentation using fully convolutional encoder–
Encoder-Decoder Architecture for Image Segmentation. CoRR, abs/1511.0. Re- decoder networks. Advances in Computer Vision and Pattern Recognition, PartF1,
trieved from http://arxiv.org/abs/1511.00561. 133–155. http://dx.doi.org/10.1007/978-3-319-61657-5_6.
Battistella, E. L. (1990). Markedness: The evaluative superstructure of language. SUNY Jalilian, E., Uhl, A., & Kwitt, R. (2017). Domain adaptation for CNN based iris
Press. segmentation. In 2017 International conference of the biometrics special interest
Bazrafkan, S., & Corcoran, P. (2017). Semi-Parallel Deep Neural Networks (SPDNN), group (pp. 1–6). http://dx.doi.org/10.23919/BIOSIG.2017.8053502.
Convergence and Generalization. Retrieved from http://arxiv.org/abs/1711. Jan, F. (2017). Segmentation and localization schemes for non-ideal iris biometric
01963. systems. Signal Processing. http://dx.doi.org/10.1016/j.sigpro.2016.11.007.
Bazrafkan, S., Javidnia, H., Lemley, J., & Corcoran, P. (2017). Depth from Monocular Jillela, R., & Ross, A. A. (2013). Methods for iris segmentation. In M. J. Burge, & K.
Images using a Semi-Parallel Deep Neural Network (SPDNN) Hybrid Architec- W. Bowyer (Eds.), Handbook of iris recognition (pp. 239–279). London: Springer
ture. arXiv Preprint arXiv:1703.03867 (pp. 1–15). Retrieved from http://arxiv. London. http://dx.doi.org/10.1007/978-1-4471-4402-1_13.
org/abs/1703.03867. Kong, W. K., & Zhang, D. (2001). Accurate iris segmentation based on novel reflection
Bazrafkan, S., Kar, A., & Costache, C. (2015). Eye gaze for consumer electronics: and eyelash detection model. In Proceedings of 2001 international symposium on
Controlling and commanding intelligent systems. IEEE Consumer Electronics intelligent multimedia, video and speech processing (IEEE Cat. No.01EX489), (pp.
Magazine, 4(4), 65–71. http://dx.doi.org/10.1109/MCE.2015.2464852. 263–266), http://dx.doi.org/10.1109/ISIMP.2001.925384.
Bazrafkan, S., Nedelcu, T., Filipczuk, P., & Corcoran, P. (2017). Deep learning for facial Lemley, J., Bazrafkan, S., & Corcoran, P. (2017a). Deep learning for consumer devices
expression recognition: a step closer to a smartphone that knows your moods. and services: Pushing the limits for machine learning, artificial intelligence,
In IEEE international conference on consumer electronics. and computer vision. IEEE Consumer Electronics Magazine, 6(2), 48–56. http:
Bigun, J., Alonso-Fernandez, F., Hofbauer, H., & Uhl, A. (2016). Experimental analysis //dx.doi.org/10.1109/MCE.2016.2640698.
regarding the influence of iris segmentation on the recognition rate. IET Biomet- Lemley, J., Bazrafkan, S., & Corcoran, P. (2017b). Smart augmentation learning
rics, 5(3), 200–211. http://dx.doi.org/10.1049/iet-bmt.2015.0069. an optimal data augmentation strategy. IEEE Access. http://dx.doi.org/10.1109/
Biometrics, I. S. O. J. S. (2007). ISO 29794-1 biometric sample quality. Committee ACCESS.2017.2696121.
Draft, 1. Lemley, J., Bazrafkan, S., & Corcoran, P. (2017c). Transfer learning of temporal infor-
Bowyer, K. W., Hollingsworth, K., & Flynn, P. J. (2008). Image understanding for iris mation for driver action classification. In The 28th modern artificial intelligence
biometrics: A survey. Computer Vision and Image Understanding, 110, 281–307. and cognitive science conference.
http://dx.doi.org/10.1016/j.cviu.2007.08.005. Li, F., Zhang, B., & Liu, B. (2016). Ternary weight networks. arXiv Preprint arXiv:
Bowyer, K. W., Hollingsworth, K. P., & Flynn, P. J. (2013). A survey of iris biometrics 1605.04711.
research: 2008–2010. Handbook of Iris Recognition, (August), 15–54. http://dx. Liu, N., Li, H., Zhang, M., Liu, J., Sun, Z., & Tan, T. (2016). Accurate iris segmenta-
doi.org/10.1007/978-1-4471-4402-1_2. tion in non-cooperative environments using fully convolutional networks. In
CASIA Iris Image Database, (2010). Retrieved July 24, 2017, from http://biometrics. 2016 international conference on biometrics, http://dx.doi.org/10.1109/ICB.2016.
idealtest.org/. 7550055.
 S. Bazrafkan et al. / Neural Networks 106 (2018) 79–95 95

Liu, N., Zhang, M., Li, H., Sun, Z., & Tan, T. (2016). DeepIris: Learning pairwise filter MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015,
bank for heterogeneous iris verification. Pattern Recognition Letters, 82, 154– proceedings, Part III (pp. 234–241). Cham: Springer International Publishing.
161. http://dx.doi.org/10.1016/j.patrec.2015.09.016. http://dx.doi.org/10.1007/978-3-319-24574-4_28.
Ma, L., Wang, Y., & Tan, T. (2002). Iris recognition using circular symmetric filters. Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for
Object Recognition Supported By User Interaction for Service Robots, 2, 414–417. face recognition and clustering. In Proceedings of the IEEE conference on computer
http://dx.doi.org/10.1109/ICPR.2002.1048327. vision and pattern recognition (pp. 815–823).
Matey, J. R., Naroditsky, O., Hanna, K., Kolczynski, R., LoIacono, D. J., Mangru, S., Sequeira, A. F., Monteiro, J. C., Rebelo, A., & Oliviera, H. P. (2014). MobBIO: A multi-
et al. (2006). Iris on the move: Acquisition of images for iris recognition in modal database captured with a portable handheld device. In 9th international
less constrained environments. Proceedings of the IEEE, 94. http://dx.doi.org/10. conference on computer vision theory and applications (pp. 133–139). Lisbon.
1109/JPROC.2006.884091. Shah, S., & Ross, a. (2009). Iris segmentation using geodesic active contours. IEEE
Menotti, D., Chiachia, G., Pinto, A., Schwartz, W. R., Pedrini, H., Falcão, A. X., et al. Transactions on Information Forensics and Security, 4(4), 824–836. http://dx.doi.
(2015). Deep representations for Iris, face, and fingerprint spoofing detection.
org/10.1109/TIFS.2009.2033225.
IEEE Transactions on Information Forensics and Security, 10(4), 864–879. http:
Silva, P., Luz, E., Baeta, R., Pedrini, H., Falcao, A. X., & Menotti, D. (2015). An approach
//dx.doi.org/10.1109/TIFS.2015.2398817.
to iris contact lens detection based on deep image representations. In Brazilian
Minaee, S., Abdolrashidiy, A., & Wang, Y. (2017). An experimental study of deep
symposium of computer graphic and image processing, Vol. 2015–Octob (pp. 157–
convolutional features for iris recognition. In 2016 IEEE signal processing in
164). http://dx.doi.org/10.1109/SIBGRAPI.2015.16.
medicine and biology symposium, SPMB 2016 - proceedings http://dx.doi.org/10.
Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014).
1109/SPMB.2016.7846859.
MIRLIN. (n.d.). Retrieved from https://www.fotonation.com/products/biometrics/ Dropout: a simple way to prevent neural networks from overfitting. Journal of
iris-recognition/. Machine Learning Research (JMLR), 15(1), 1929–1958.
Morley, D., & Foroosh, H. (2017). Improving ransac-based segmentation through cnn Szegedy, C., Toshev, A., & Erhan, D. (2013). Deep neural networks for object detec-
encapsulation. In Proc. IEEE conf. on computer vision and pattern recognition. tion. In Advances in Neural Information Processing Systems (pp. 2553–2561).
Pando, A. (2017). Beyond Security: Biometrics Integration Into Everyday Life. Tan, T., He, Z., & Sun, Z. (2010). Efficient and robust segmentation of noisy iris images
Retrieved November 21, 2017, from https://www.forbes.com/sites/forbestech for non-cooperative iris recognition. Image and Vision Computing, 28(2), 223–
council/2017/08/04/beyond-security-biometrics-integration-into-everyday-lif 230.
e/#ce400be431fb. Tan, C.-W., & Kumar, A. (2012). Unified framework for automated iris segmentation
Phillips, C., & Komogortsev, O. V. (2011). Impact of Resolution and Blur on Iris using distantly acquired face images. IEEE Transactions on Image Processing,
Identification. 21(9), 4068–4079.
Prabhakar, S., Pankanti, S., & Jain, A. K. (2003). Biometric recognition: security and Tan, C.-W., & Kumar, A. (2013). Towards online iris and periocular recognition under
privacy concerns. IEEE Security & Privacy Magazine, 1(2), 33–42. http://dx.doi. relaxed imaging constraints. IEEE Transactions on Image Processing, 22(10),
org/10.1109/MSECP.2003.1193209. 3751–3765.
Proenca, H. (2010). Iris recognition: On the segmentation of degraded images Thavalengal, S., Bigioi, P., & Corcoran, P. (2015a). Evaluation of combined visible/NIR
acquired in the visible wavelength. IEEE Transactions on Pattern Analysis and camera for iris authentication on smartphones. In 2015 IEEE conference on
Machine Intelligence, 32(8), 1502–1516. computer vision and pattern recognition workshops, (pp. 42–49). http://dx.doi.
Proença, H., & Alexandre, L. A. (2006). Iris segmentation methodology for non- org/10.1109/CVPRW.2015.7301318.
cooperative recognition. IEE proceedings - vision, image, and signal processing.
Thavalengal, S., Bigioi, P., & Corcoran, P. (2015b). Iris authentication in hand-
http://dx.doi.org/10.1049/ip-vis:20050213.
held devices - considerations for constraint-free acquisition. IEEE Transactions
Proença, H., & Alexandre, L. A. (2010). Iris recognition: Analysis of the error rates
on Consumer Electronics, 61(2), 245–253. http://dx.doi.org/10.1109/TCE.2015.
regarding the accuracy of the segmentation stage. Image and Vision Computing,
7150600.
28(1), 202–206. http://dx.doi.org/10.1016/j.imavis.2009.03.003.
Thavalengal, S., & Corcoran, P. (2016). User authentication on smartphones: Focus-
Proenca, H., Filipe, S., Santos, R., Oliveira, J., & Alexandre, L. A. (2010). The
ing on iris biometrics. IEEE Consumer Electronics Magazine, 5(2), 87–93. http:
{UBIRIS.v2}: A database of visible wavelength images captured on-the-move
and at-a-distance. IEEE Transactions on Pattern Analysis Machine Intelligence, //dx.doi.org/10.1109/MCE.2016.2522018.
32(8), 1529–1535. http://dx.doi.org/10.1109/TPAMI.2009.66. Tisse, C.-, Martin, L., Torres, L., Robert, M., Zi, S., & Rousset, R. et al., (1992).
Quinn, G. W., Grother, P. J., Ngan, M. L., & Matey, J. R. (2013). IREX IV: part 1, Person identification technique using human iris recognition Advanced System
evaluation of iris identification algorithms. NIST Interagency/Internal Report- Technology, (i).
7949. Wildes, R. P., Asmuth, J. C., Hanna, K. J., Hsu, S. C., Kolczynski, R. J., & Matey, J. R.
Radman, A., Zainal, N., & Suandi, S. A. (2017). Automated segmentation of iris images et al., (1996). Automated, non-invasive iris recognition system and method. US
acquired in an unconstrained environment using HOG-SVM and GrowCut. Patent US5572596 A.
Digital Signal Processing, 64, 60–70. Zhao, Z., & Ajay, K. (2015). An accurate iris segmentation framework under relaxed
Rakshit, S. (2007). Novel methods for accurate human iris recognition. University of imaging constraints using total variation model. In Proceedings of the IEEE
Bath. international conference on computer vision (pp. 3828–3836).
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for Zheng, S., Song, Y., Leung, T., & Goodfellow, I. (2016). Improving the robustness of
biomedical image segmentation. In N. Navab, J. Hornegger, W. M. Wells, & A. deep neural networks via stability training. In Proceedings of the IEEE conference
F. Frangi (Eds.), Medical image computing and computer-assisted intervention– on computer vision and pattern recognition (pp. 4480–4488).