Download the Full Version of textbook for Fast Typing at textbookfull.

com

Roadside Video Data Analysis Deep Learning 1st

Edition Brijesh Verma

https://textbookfull.com/product/roadside-video-data-
analysis-deep-learning-1st-edition-brijesh-verma/

OR CLICK BUTTON

DOWNLOAD NOW

Download More textbook Instantly Today - Get Yours Now at textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Deep Learning with Structured Data 1st Edition Mark Ryan

https://textbookfull.com/product/deep-learning-with-structured-
data-1st-edition-mark-ryan/

textboxfull.com

Deep Learning with Structured Data 1st Edition Mark Ryan

https://textbookfull.com/product/deep-learning-with-structured-
data-1st-edition-mark-ryan-2/

textboxfull.com

Deep Learning for Medical Image Analysis 1st Edition S.

Kevin Zhou

https://textbookfull.com/product/deep-learning-for-medical-image-
analysis-1st-edition-s-kevin-zhou/

textboxfull.com

Deep Learning Innovations and Their Convergence With Big

Data 1st Edition S. Karthik

https://textbookfull.com/product/deep-learning-innovations-and-their-
convergence-with-big-data-1st-edition-s-karthik/

textboxfull.com
Design of Video Quality Metrics with Multi Way Data
Analysis A data driven approach 1st Edition Christian
Keimel (Auth.)
https://textbookfull.com/product/design-of-video-quality-metrics-with-
multi-way-data-analysis-a-data-driven-approach-1st-edition-christian-
keimel-auth/
textboxfull.com

Advanced Data Analytics Using Python: With Machine

Learning, Deep Learning and NLP Examples Mukhopadhyay

https://textbookfull.com/product/advanced-data-analytics-using-python-
with-machine-learning-deep-learning-and-nlp-examples-mukhopadhyay/

textboxfull.com

Deep Learning for Data Analytics: Foundations, Biomedical

Applications, and Challenges 1st Edition Himansu Das

https://textbookfull.com/product/deep-learning-for-data-analytics-
foundations-biomedical-applications-and-challenges-1st-edition-
himansu-das/
textboxfull.com

Deep Learning in Medical Image Analysis and Multimodal

Learning for Clinical Decision Support Danail Stoyanov

https://textbookfull.com/product/deep-learning-in-medical-image-
analysis-and-multimodal-learning-for-clinical-decision-support-danail-
stoyanov/
textboxfull.com

Programming PyTorch for Deep Learning Creating and

Deploying Deep Learning Applications 1st Edition Ian
Pointer
https://textbookfull.com/product/programming-pytorch-for-deep-
learning-creating-and-deploying-deep-learning-applications-1st-
edition-ian-pointer/
textboxfull.com
Studies in Computational Intelligence 711

Brijesh Verma
Ligang Zhang
David Stockwell

Roadside
Video Data
Analysis: Deep
Learning
Studies in Computational Intelligence

Volume 711

Series editor
Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland
e-mail: [email protected]
About this Series

The series “Studies in Computational Intelligence” (SCI) publishes new develop-

ments and advances in the various areas of computational intelligence—quickly and
with a high quality. The intent is to cover the theory, applications, and design
methods of computational intelligence, as embedded in the fields of engineering,
computer science, physics and life sciences, as well as the methodologies behind
them. The series contains monographs, lecture notes and edited volumes in
computational intelligence spanning the areas of neural networks, connectionist
systems, genetic algorithms, evolutionary computation, artificial intelligence,
cellular automata, self-organizing systems, soft computing, fuzzy systems, and
hybrid intelligent systems. Of particular value to both the contributors and the
readership are the short publication timeframe and the worldwide distribution,
which enable both wide and rapid dissemination of research output.

More information about this series at http://www.springer.com/series/7092

Brijesh Verma Ligang Zhang
•

David Stockwell

Roadside Video Data

Analysis: Deep Learning

123
Brijesh Verma David Stockwell
School of Engineering and Technology School of Engineering and Technology
Central Queensland University Central Queensland University
Brisbane, QLD Brisbane, QLD
Australia Australia

Ligang Zhang
School of Engineering and Technology
Central Queensland University
Brisbane, QLD
Australia

ISSN 1860-949X ISSN 1860-9503 (electronic)

Studies in Computational Intelligence
ISBN 978-981-10-4538-7 ISBN 978-981-10-4539-4 (eBook)
DOI 10.1007/978-981-10-4539-4
Library of Congress Control Number: 2017936905

© Springer Nature Singapore Pte Ltd. 2017

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained herein or
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.

Printed on acid-free paper

This Springer imprint is published by Springer Nature

The registered company is Springer Nature Singapore Pte Ltd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore
Preface

Video data analysis has become an increasingly important research area with
widespread applications in automatic surveillance of transportation infrastructure
including roads, rail and airports. As the amount of video data collected grows, so
does the opportunity for further processing with new artificial intelligence methods.
One type of useful video data that has seen little research is the roadside video data
that is collected using video-mounted vehicles. These video data may augment or
replace road-based surveys of the conditions of roadside objects such as trees,
grasses, roads and traffic signs and can be potentially used in many real-world
applications such as roadside vegetation growth condition monitoring, effective
roadside management to reduce the possible hazards to drivers and vehicles, and
developing automatic vehicles that are able to automatically sense roadside objects
and traffic signs.
Most existing studies on video data analysis are primarily focused on analyzing
generic object categories in the data content in public benchmark datasets. Very
limited research has focused on the analysis of roadside video data, although the
significance of developing smart techniques for roadside video data analysis has
been widely recognized. One of the main reasons is probably because there is a lack
of a comprehensive public dataset that was specifically created for roadside objects.
Another reason is the various types of variations and environmental conditions
encountered along road sides, which are still challenging issues in the computer
vision field. The great variability in the appearance and structure of objects as well
as the various types of environmental effects such as underexposure, overexposure,
shadows, and sunlight reflectance make accurate segmentation and recognition of
objects difficult. The current literature lacks a comprehensive review of existing
machine learning algorithms, particularly deep learning techniques, on roadside
data analysis.
This book highlights the methods and applications for roadside video data
analysis. It describes various system architectures and methodologies that are
specifically built upon different types of learning algorithms for roadside video data
processing, with detailed analysis of the segmentation, feature extraction and
classification. The use of deep learning to solve the roadside video data

v
vi Preface

segmentation and classification problems is one of the major highlights of this

book. Deep neural net learning has become popular in machine learning and data
mining areas. However, the benefits of a deep feature free approach must be bal-
anced against the considerations of accuracy and robustness, and most real-world
learning systems require some hand engineering of features and architectures. This
book examines via empirical testing the types of features and architectures that
contribute to the performance of multi-layer neural nets on real-world scene anal-
ysis. We then demonstrate novel architectures that perform scene classification with
equal or better accuracy to previous methods and investigate the feature engineering
into convolutional neural networks. Further, we provide an industrial perspective to
help align theoretical concerns with real-world results. Finally, as a case study of
roadside video data analysis, we demonstrate an application of vegetation biomass
estimation techniques for roadside fire risk assessment. Overall, this book compiles
the most useful strategies in the field of scene analysis to help researchers identify
the most appropriate features and architectures for their applications.

Brisbane, Australia Brijesh Verma

Ligang Zhang
David Stockwell
Acknowledgements

The authors express their gratitude for help from the Department of Transport and
Main Roads in Queensland, Australia for the creation of the data and assistance
with field resources for the fire risk survey. This work was supported under
Australian Research Council’s Linkage Projects funding scheme (project number
LP140100939).
Many students and research fellows in the Centre for Intelligent Systems at
Central Queensland University have contributed to research presented in this book.
The authors would like to thank the following researchers: Dr Sujan Chowdhury,
Dr Tejy Kinattukara Jobachan, Dr Peter Mc Leod, Dr M. Asafuddoula, Mrs. Toshi
Sinha and Mrs. Fatma Shaheen.

vii
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Collection of Roadside Video Data . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Industry Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.2 Benchmark Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Applications Using Roadside Video Data . . . . . . . . . . . . . . . . . . . 10
1.4 Outline of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2 Roadside Video Data Analysis Framework . . . . . . . . . . . . . . . . . . . . . 13
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.1 Pre-processing of Roadside Video Data . . . . . . . . . . . . . . . 14
2.2.2 Segmentation of Roadside Video Data into Objects . . . . . . 18
2.2.3 Feature Extraction from Objects . . . . . . . . . . . . . . . . . . . . . 19
2.2.4 Classification of Roadside Objects . . . . . . . . . . . . . . . . . . . 20
2.2.5 Applications of Classified Roadside Objects . . . . . . . . . . . . 21
2.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 Vegetation Segmentation and Classification . . . . . . . . . . . . 24
2.3.2 Generic Object Segmentation and Classification . . . . . . . . . 28
2.4 Matlab Code for Data Processing. . . . . . . . . . . . . . . . . . . . . . . . . . 32
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3 Non-deep Learning Techniques for Roadside Video Data
Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.1 Neural Network Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.1.2 Neural Network Learning Approach . . . . . . . . . . . . . . . . . . 42
3.1.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

ix
x Contents

3.2 Support Vector Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2.2 SVM Learning Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3 Clustering Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3.2 Clustering Learning Approach . . . . . . . . . . . . . . . . . . . . . . 57
3.3.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.4 Fuzzy C-Means Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.4.2 Fuzzy C-Means Learning Approach . . . . . . . . . . . . . . . . . . 75
3.4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.5 Ensemble Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.5.2 Ensemble Learning Approach . . . . . . . . . . . . . . . . . . . . . . . 80
3.5.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
3.5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.6 Majority Voting Based Hybrid Learning . . . . . . . . . . . . . . . . . . . . 86
3.6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.6.2 Majority Voting Approach . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.6.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.7 Region Merging Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.7.2 Region Merging Approach . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.7.3 Components of Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 100
3.7.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4 Deep Learning Techniques for Roadside Video Data Analysis . . . . . 119
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.3 Automatic Versus Manual Feature Extraction . . . . . . . . . . . . . . . . 122
4.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.3.2 Comparison Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.3.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
4.3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.4 Single Versus Ensemble Architectures . . . . . . . . . . . . . . . . . . . . . . 127
4.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.4.2 Comparison Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
Contents xi

4.4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

4.4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.5 Deep Learning Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.5.2 Deep Learning Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.5.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
4.5.4 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
5 Case Study: Roadside Video Data Analysis for Fire Risk
Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
5.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5.3 Proposed VOCGP Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
5.3.1 Problem Formulation and Motivations . . . . . . . . . . . . . . . . 162
5.3.2 Overview of Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
5.3.3 Grass Region Segmentation . . . . . . . . . . . . . . . . . . . . . . . . 164
5.3.4 Gabor Filter Voting for Dominant Vertical Orientation
Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 165
5.3.5 Vertical Orientation Connectivity of Grass Pixels
Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
5.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
5.4.1 Roadside Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . 171
5.4.2 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
5.4.3 Performance of Estimating Grass Biomass . . . . . . . . . . . . . 173
5.4.4 Performance of Predicting Grass Density . . . . . . . . . . . . . . 176
5.4.5 Fire-Prone Region Identification . . . . . . . . . . . . . . . . . . . . . 177
5.5 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
6 Conclusion and Future Insight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.1 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.2 New Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
6.3 New Opportunities and Applications . . . . . . . . . . . . . . . . . . . . . . . 188
Abbreviations

ANN Artificial Neural Network

ANOVA ANalysis Of VAriance
BG Brown Grass
CAV Context Adaptive Voting
CHM Contextual Hierarchical Model
CIVE Color Index of Vegetation Extraction
CNN Convolutional Neural Network
CRF Conditional Random Field
CRR Correct Recognition Rate
CSPM Contextual Superpixel Probability Map
CWT Continuous Wavelet Transform
DTM Digital Terrain Model
DTMR Department of Transport and Main Roads
ExG Excess Green
ExR Excess Red
FC Fully Connected layer
FCM Fuzzy C-Means algorithm
FRR False Recognition Rate
GG Green Grass
GLCM Gray-Level Co-occurrence Matrix
HOG Histograms of Oriented Gradients
KNN K-Nearest Neighbor
LBP Local Binary Pattern
LIDAR LIght Detection And Ranging
MLP Multi-Layer Perceptron
MNDVI Modification of NDVI
MNIST Mixed National Institute of Standards and Technology dataset
MO Morphological Opening
MRF Markov Random Field
MSRC MicroSoft Research Cambridge 21-class dataset

xiii
xiv Abbreviations

N/A Not Available

NDVI Normalized Difference Vegetation Index
NIR Near Infrared Ray
OCP Object Co-occurence Prior
PHOG Pyramid Histogram of Oriented Gradient
PID Pixel Intensity Difference
PPM Pixel Probability Map
PPS Pixel and Patch Selective
RBF Radial Basis Function
RELU REctified Linear Unit
ROI Region Of Interest
SCSM Spatial Contextual Superpixel Model
SIFT Scale-Invariant Feature Transform
SOM Self-Organizing Map
SVM Support Vector Machine
SVR Support Vector Regression
TL Tree Leaf
TS Tree Stem
VI Vegetation Index
VocANN VOCGP obtained using ANN
VocCNN VOCGP obtained using CNN
VOCGP Vertical Orientation Connectivity of Grass Pixel
VVI Visible Vegetation Index
List of Figures

Figure 1.1 Illustration of small regions cropped from DTMR video

frames. The white blocks in the figures give an
indication of the location, size and shape of the cropped
regions. Note that only a proportion of representative
cropped regions are included in the final cropped
roadside object dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Figure 1.2 Samples of cropped regions for seven roadside objects.
There are big variations in the appearance of the same
object and high similarity between objects (e.g. green
grass and tree leaf), which are challenges for accurate
object classification. These regions are rescaled to the
same shape and size only for illustration . . . . . . . . . . . . . . . . 4
Figure 1.3 Sample images for (top) sparse and (bottom) dense
grasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Figure 1.4 Sample images in the natural roadside object dataset
(left) with their corresponding pixel-wise ground truths
(right) for six types of roadside objects . . . . . . . . . . . . . . . . . 5
Figure 1.5 Sample images in the natural road object dataset.
The images were captured using a frontal view camera . . . . . 6
Figure 1.6 Sample images and their corresponding pixel-wise
ground truths in the Stanford background dataset. The
images were captured in a wide range of real-world
situations, and form one of the most challenging
datasets in the field of scene content understanding . . . . . . . . 7
Figure 1.7 Sample images and their corresponding pixel-wise
ground truths in the MSRC 21-class dataset. Note that
black pixels in the ground truth images indicate a void
label, which is excluded from the performance evaluation . . . 8

xv
xvi List of Figures

Figure 1.8 Sample images and their corresponding pixel-wise

ground truths in the SIFT flow dataset. Note that there
are 33 objects in total, but only a subset of object
categories is displayed in the sample images . . . . . . . . . . ... 8
Figure 1.9 Sample images and their corresponding pixel-wise
ground truths in the Croatia roadside grass dataset . . . . . . ... 9
Figure 1.10 Samples of handwritten digits in the MNIST dataset
(adopted from [4]) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 9
Figure 2.1 A general framework for roadside video data analysis . . . ... 14
Figure 2.2 An example of video to frame conversion. There are
500 frames in total in the example video, and every
10th frame is displayed. . . . . . . . . . . . . . . . . . . . . . . . . . . ... 15
Figure 2.3 An example of converting an RGB image (left)
to a grayscale image (right) . . . . . . . . . . . . . . . . . . . . . . . ... 15
Figure 2.4 Examples of cropped roadside regions resized into the
same resolution by applying an image resizing technique. ... 16
Figure 2.5 An example of noise removal (right) by applying a
median filter of 3
3 pixels to an original roadside
image (left) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 17
Figure 2.6 A sampling region indicated by a white plastic square in
field tests (left) and one possible corresponding sample
region indicated by a red rectangle in the image (right). . ... 17
Figure 2.7 Graphic illustration of object segmentation in roadside
frames. The frames are segmented into different object
categories, such as tree, sky, road, soil, brown grass,
and green grass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 18
Figure 2.8 Examples of roadside grasses with high and low fire risk ... 22
Figure 2.9 Examples of four levels of tree regrowth conditions,
including heavy, medium, light and zero . . . . . . . . . . . . . ... 23
Figure 2.10 Visual displays of segmented superpixels in roadside
images using a graph-based segmentation algorithm
[26]. Different colors indicate different superpixel regions ... 29
Figure 3.1 Graphic illustration of the neural network learning
approach. For each pixel in an image, O1 ; O2 ; O3 color
intensity features are extracted. Color moment texture
features are also extracted in a local patch centered at
the pixel. The color and texture features are combined
using a feature-level fusion and fed into an ANN for
classifying six object categories . . . . . . . . . . . . . . . . . . . . ... 42
Figure 3.2 Global accuracy versus number of hidden neurons
in the ANN classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 46
Figure 3.3 Classification results of six sample images from the
DTMR video data. One issue found is that pixels in
object boundaries are prone to be misclassified, which is
List of Figures xvii

a common problem for patch based feature extraction

techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 48
Figure 3.4 Overview of the SVM learning approach . . . . . . . . . . . . . ... 50
Figure 3.5 (Left) Pixel values in grass regions; (middle) pixel
values in soil regions; and (right) an input image
with target regions labelled . . . . . . . . . . . . . . . . . . . . . . . . ... 50
Figure 3.6 Illustration of training data acquisition. a Original image
with a ROI of 5
5 pixels; b index of pixels in the ROI;
c the training data collected from the ROI; and d the
final training data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Figure 3.7 Pixel value correction in the output images . . . . . . . . . . . . . . 53
Figure 3.8 Image samples used in experimental evaluations . . . . . . . . . . 54
Figure 3.9 Object segmentation results in sample images . . . . . . . . . . . . 54
Figure 3.10 Framework of the clustering learning approach. During
training, a set of regions is cropped to generate
class-semantic color and texture textons using K-means
clustering. During testing, features of all pixels within
each superpixel are projected to one of the learnt
textons, and further aggregated into texton occurrences.
A majority vote over texton occurrences in each
superpixel is used to segment objects from roadside
images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 58
Figure 3.11 Global accuracy versus the number of textons.
The number of color and texture textons
is the same, the combination weight for texture textons
w = 1, the size of Gaussian filters: 7
7 pixels,
and distance metric: Euclidean . . . . . . . . . . . . . . . . . . . . . ... 63
Figure 3.12 Computational performance versus the number of
textons on the cropped and natural roadside object
datasets. The time is the average seconds per image
(or region) required for calculating color and texture
features (i.e. Feature), and performing texton mapping
and classification (i.e. Classification) . . . . . . . . . . . . . . . . ... 64
Figure 3.13 Global accuracy versus the value of combination
weights. The weight of texture textons is a value relative
to a fixed value of 1 for color textons. Color & texture
texton no. = 60 (cropped dataset) and 30 (natural
dataset); the size of Gaussian filters: 7
7 pixels;
distance metric: Euclidean . . . . . . . . . . . . . . . . . . . . . . . . ... 65
Figure 3.14 Global accuracy versus the size of Gaussian filters.
The combination weight w = 1.2 (cropped dataset), and
1 (natural dataset); Color and texture texton no. = 60
(cropped dataset) and 30 (natural dataset); distance
metric: Euclidean (for both datasets). The performance
xviii List of Figures

of color textons remains the same using varied sizes of

Gaussian filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 66
Figure 3.15 Samples of segmentation results on the natural roadside
object dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 70
Figure 3.16 Segmentation results on frame samples from three
roadside videos. Frames in the three rows are selected
from the beginning, middle, and end sections of the
video, respectively. The frame number in each video
is shown below each figure . . . . . . . . . . . . . . . . . . . . . . . ... 71
Figure 3.17 Framework of the wavelet based FCM learning approach ... 75
Figure 3.18 First and second level decompositions in CWTs . . . . . . . ... 76
Figure 3.19 A sample set of pre-processed images in the natural road
image dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 78
Figure 3.20 A sample set of extracted road objects . . . . . . . . . . . . . . . ... 79
Figure 3.21 Framework of the ensemble learning approach . . . . . . . . . ... 81
Figure 3.22 Layering on the data which are clustered into a two
atomic clusters and one non-atomic cluster,
and b three clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 83
Figure 3.23 A sample set of extracted road objects using the
ensemble learning approach . . . . . . . . . . . . . . . . . . . . . . . ... 85
Figure 3.24 Comparative analysis of the ensemble learning
approach with three benchmark approaches . . . . . . . . . . . ... 86
Figure 3.25 Framework of the majority voting based hybrid approach ... 88
Figure 3.26 Examples misclassified as dense (left) and sparse (right)
grasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 93
Figure 3.27 Image examples illustrating the necessity of considering
local characteristics of objects (i.e. tree leaf) in a test
image. The two regions of tree leaves have substantially
contrasting differences in pixel intensities, and thus a
classification system trained using features extracted
from the left region may not work well
in the right region . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 96
Figure 3.28 Framework of the spatial contextual superpixel
model (SCSM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 97
Figure 3.29 Illustration of determining boundary versus
non-boundary pixels. Given a segmented superpixel,
the interior and exterior boundaries are determined by
taking a distance of a half of the height (or width)
of a patch away from the superpixel boundary. All
pixels within the region between interior and exterior
boundaries are treated as boundary pixels, while those
within interior boundary are non-boundary pixels. . . . . . . ... 101
List of Figures xix

Figure 3.30 Distribution of probability weights given to sky, road

and tree, depending on the (x, y) coordinates of pixels in
an image of Height
Width (H
W) pixels. The
coordinates start from ð0; 0Þ at the left-top corner
to the ðH; WÞ at the right-bottom corner . . . . . . . . . . . . . ... 102
Figure 3.31 Classification accuracy (%) versus number of hidden
neurons on the cropped roadside object dataset. Patch
based features outperform pixel based features
for all objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 110
Figure 3.32 Overall accuracy comparisons between approaches
on the natural roadside object dataset. The SCSM
outperforms all benchmark approaches. Patch based
features show higher accuracies than both PPS and pixel
based features for all classes . . . . . . . . . . . . . . . . . . . . . . . ... 112
Figure 3.33 Comparisons of classification results of the SCSM with
using pixel based features and PPS features. The SCSM
produces more consistent results than using pixel or
PPS features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 114
Figure 4.1 Architecture of the LeNet-5 CNN. There are 7 layers
in total excluding the input layer. The abstract feature
representations are extracted by progressively applying
convolutional and subsampling operations. The features
are then fed into two full connection layers for
classifying the input data into different object categories . ... 120
Figure 4.2 CNN research trend from 1970 to 2016. Source Scopus . ... 121
Figure 4.3 Framework for comparing between automatic feature
extraction in CNN and manual feature extraction using
the same MLP classifier . . . . . . . . . . . . . . . . . . . . . . . . . . ... 123
Figure 4.4 Comparisons of feature extraction in CNN, image-based
MLP and feature-based MLP . . . . . . . . . . . . . . . . . . . . . . ... 124
Figure 4.5 Two images in the cow heat sensor dataset showing
changed and unchanged colors . . . . . . . . . . . . . . . . . . . . . ... 125
Figure 4.6 Classification accuracy (%) obtained using CNN,
image-based MLP and feature-based MLP. The MLP
has 120 hidden neurons . . . . . . . . . . . . . . . . . . . . . . . . . . ... 126
Figure 4.7 Framework for comparing a single versus an ensemble
of CNNs and MLPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 128
Figure 4.8 Illustration of the architecture of an ensemble of CNNs
(or MLPs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 129
Figure 4.9 Framework of the three-layer deep learning network
architecture. The prediction layer takes superpixel-level
visual features as inputs and predicts class probabilities
of each superpixel using class-semantic classifiers. The
context voting layer calculates global and local Context
xx List of Figures

Adaptive Voting (CAV) features by casting votes from

superpixels in spatial blocks and from adjacent
superpixels respectively based on the most probable
class and Object Co-occurance Priors (OCPs). The
OCPs (shown in rectangles) are collected for all block
pairs from the training data. The integration layer
integrates visual feature based class probabilities, global
and local CAV features to produce a class probability
vector. Each superpixel is finally assigned to a class
label using a majority voting strategy . . . . . . . . . . . . . . . . ... 135
Figure 4.10 Illustration of the differences between relative location
prior, absolute location prior, and OCP in modelling
of three cases of a spatial correlation “left sky and right
building” (best viewed in color) . . . . . . . . . . . . . . . . . . . . ... 137
Figure 4.11 Illustration of the calculation of the global OCP in an
image from the Stanford background dataset. The pixel
distributions over all classes are calculated for each of
all spatial image blocks, and an object co-occurrence
matrix is then formed for every two-block pair
(#3 and #15 in the example) to reflect the long-range
label dependencies between objects within two blocks . . . ... 138
Figure 4.12 Qualitative results on the Stanford background dataset
(best viewed in color). Compared with using visual
features alone, the deep learning network shows more
robust scene parsing results by utilizing CAV features,
with many misclassifications successfully removed . . . . . ... 146
Figure 4.13 Qualitative results on the MSRC dataset. The deep
learning network using CAV features successfully
removes a significant portion of misclassification
in the results of visual feature based prediction . . . . . . . . ... 149
Figure 4.14 Qualitative results on the SIFT Flow dataset. The results
demonstrate the benefit of incorporating CAV features
to improve the classification accuracy and correct
misclassification error for complex objects
in natural scenes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 153
Figure 5.1 Graphic illustration of main processing steps in the
VOCGP approach. For a given image, the approach
outputs a VOCGP value for estimating the grass
biomass within a sampling window. The outputs
of grass region segmentation and dominant vertical
orientation detection are combined for VOCGP
calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 164
List of Figures xxi

Figure 5.2 Visual illustration of the real part of image responses

using five-scale (row) and four-orientation (column)
Gabor filters (left). Four orientations used in this case
study and their defined indices (right) . . . . . . . . . . . . . . . ... 167
Figure 5.3 Illustration of the difference in the connectivity along
the vertical orientation between high and low grasses.
The white and black in the right windows indicate the
dominant vertical and non-vertical orientations
respectively computed using Gabor filter votes. High
grasses have longer unbroken connectivity along the
vertical orientation than low grasses . . . . . . . . . . . . . . . . . ... 168
Figure 5.4 Location distribution of 61 sample sites for taking
images and collecting grass biomass data. These sites
(F001 to F061) are chosen from the Fitzroy region,
Queensland, Australia . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 171
Figure 5.5 Image samples for sparse, moderate and dense grasses . . . ... 172
Figure 5.6 Objective biomass and estimated VOCGP for all samples ... 175
Figure 5.7 Correlation between objective biomass and VOCGP for
all samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 175
Figure 5.8 Mean biomass and VOCGP for sparse, moderate and
dense grasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 176
Figure 5.9 Box and whisker plot of objective biomass and VOCGP
for sparse, moderate and dense grasses . . . . . . . . . . . . . . . ... 177
Figure 5.10 Best linear fit of objective biomass and VOCGP to three
grass categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 178
Figure 5.11 Distribution and classified results of 15 sampling grass
windows in image frames. a Ground truth: 2-sparse,
3-moderate, and 4-dense; b grass segmentation results:
white grass pixels and black non-grass pixels;
c estimated VOCGP, i.e. VocANN . . . . . . . . . . . . . . . . . . ... 179
Figure 5.12 Illustration of using the average VocANN over
15 sampling windows in each image for fire-prone
region identification. The frames are ordered according
to their locations on the state road No. 16A in the
Fitzroy region. The locally highest and lowest VocANN
are matched accurately with the corresponding grass
density levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 180
List of Tables

Table 2.1 Typical types of features used in existing studies

for video data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 20
Table 2.2 Summary of typical visible approaches for vegetation
segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 25
Table 2.3 Summary of typical invisible approaches for vegetation
segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 27
Table 3.1 Class accuracy (%) on the test and training data . . . . . . . . ... 46
Table 3.2 Performance (%) comparisons between classifiers . . . . . . . ... 46
Table 3.3 Performance (%) comparisons of color versus fused
color and texture features based on the ANN classifier . . . . . . 47
Table 3.4 A list of pixel characteristics features . . . . . . . . . . . . . . . . . . . 51
Table 3.5 Average pixel value for each object . . . . . . . . . . . . . . . . . . . . 52
Table 3.6 Confusion matrix for four objects . . . . . . . . . . . . . . . . . . . . . . 55
Table 3.7 Classification of grass regions using percentages
of grass pixels (%) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 56
Table 3.8 Global accuracy (% ± standard deviation) versus
distance metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 67
Table 3.9 Comparisons of class accuracy (%) between approaches . . ... 68
Table 3.10 Confusion matrix (%) for different classes using the
clustering learning approach . . . . . . . . . . . . . . . . . . . . . . . ... 69
Table 3.11 Global accuracy (%) of the LeNet-5 CNN on the
cropped roadside object dataset . . . . . . . . . . . . . . . . . . . . . ... 71
Table 3.12 Performance comparisons with existing approaches . . . . . . ... 72
Table 3.13 Performance (%) comparisons of the wavelet based
FCM approach with the FCM approach . . . . . . . . . . . . . . ... 79
Table 3.14 Performance comparisons of the ensemble learning
approach with three benchmark approaches . . . . . . . . . . . . ... 85
Table 3.15 Classification accuracy (%) using the SVM classifier . . . . ... 90

xxiii
xxiv List of Tables

Table 3.16 5-fold cross validation results (%) using a linear SVM
classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Table 3.17 Classification accuracy (%) using the ANN classifier . . . . . . . 91
Table 3.18 5-fold cross validation results using the ANN classifier. . . . . . 91
Table 3.19 Classification accuracy (%) using the KNN classifier . . . . . . . 92
Table 3.20 5-fold cross validation results using KNN with K = 7 . . . . . . 92
Table 3.21 Classification accuracy (%) using the majority voting
approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 92
Table 3.22 5-fold cross validation results using the majority voting
approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 92
Table 3.23 Comparisons of classification rates between classifiers
using 5-fold cross validations . . . . . . . . . . . . . . . . . . . . . . ... 93
Table 3.24 Summary of the single factor ANOVA test . . . . . . . . . . . . ... 94
Table 3.25 Results of the ANOVA test . . . . . . . . . . . . . . . . . . . . . . . . ... 94
Table 3.26 Class accuracy (%, ± standard deviation) of pixel versus
patch based features on the cropped roadside
object dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 111
Table 3.27 Confusion matrix of six classes obtained using the
SCSM on the natural roadside object data . . . . . . . . . . . . . ... 113
Table 3.28 Performance (%) comparisons with state-of-the-art
methods on the stanford background dataset . . . . . . . . . . . ... 114
Table 3.29 Confusion matrix of eight classes on the stanford
background dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Table 4.1 A brief review of typical types of CNNs . . . . . . . . . . . . . . . . . 121
Table 4.2 Accuracy (%) using an ensemble of CNNs . . . . . . . . . . . . . . . 129
Table 4.3 Accuracy (%) using an ensemble of MLPs . . . . . . . . . . . . . . . 130
Table 4.4 Accuracy (%) using a single CNN . . . . . . . . . . . . . . . . . . . . . 130
Table 4.5 Accuracy (%) using a single MLP . . . . . . . . . . . . . . . . . . . . . . 130
Table 4.6 Performance (%) comparisons with previous approaches
on the Stanford background dataset . . . . . . . . . . . . . . . . . . ... 145
Table 4.7 Confusion matrix for eight objects on the Stanford
background dataset (SVR, global accuracy 81.2%) . . . . . . ... 147
Table 4.8 Performance (%) comparisons with previous approaches
on the MSRC dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... 148
Table 4.9 Confusion matrix for 21 objects on the MSRC dataset
(SVR, global accuracy 85.5%) . . . . . . . . . . . . . . . . . . . . . ... 150
Table 4.10 Performance (%) comparisons with previous approaches
on the SIFT Flow dataset . . . . . . . . . . . . . . . . . . . . . . . . . ... 152
Table 5.1 Samples categorized into sparse, moderate, and dense
grasses based on human observation . . . . . . . . . . . . . . . . . ... 172
Table 5.2 Objective biomass and estimated VOCGP (i.e. VocANN
and VocCNN for using ANN and CNN respectively) for
sparse, moderate and dense grasses . . . . . . . . . . . . . . . . . . ... 174
List of Tables xxv

Table 5.3 Performance using images with different degrees

of rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Table 5.4 Performance comparisons of the VOCGP approach
with human observation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
Table 5.5 Confusion matrix of sparse, moderate and dense grasses
using estimated VocANN . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Chapter 1
Introduction

This chapter presents brief background information and datasets for roadside video
data analysis. It also shows some relevant applications of video data analysis and
presents a detailed outline of this book.

1.1 Background

The roadside is a specific region that is of great significance to many fields such as
agriculture, forestry, transportation, power and telecommunications, national
security and environmental authorities. Being able to acquire accurate information
about roadside conditions can be potentially useful for assisting a wide range of
practical applications, such as effective roadside management, vegetation growth
condition monitoring, and road hazard assessment. For instance, accurately
estimating the site-specific parameters of roadside vegetation such as biomass,
height, coverage, density and greenness can be critically important for monitoring
the real-time growth conditions of roadside grasses and trees. This supports the
design and implementation of effective vegetation management strategies by rele-
vant government authorities, preserving the road assets and improving the safety of
the driving public. Tracking the changes in these parameters is an effective way to
detect and quantify the possible effects on vegetation such as diseases, dryness, soil
nutrients and water stress. From the perspective of safety, roadside vegetation
(e.g. grass and tree) with high biomass can present a major fire hazard to the
security of drivers and vehicles, particularly in remote and low-population regions
without regular and frequent human checks on roadside grass growth conditions.
The current practice of monitoring roadside conditions by relevant authorities is
heavily dependent on manual measurements by humans, which suffer from high
investments in terms of labor, time, effort and cost. The necessity and importance of
developing automatic roadside video data analysis techniques have been widely

© Springer Nature Singapore Pte Ltd. 2017 1

B. Verma et al., Roadside Video Data Analysis: Deep Learning,
Studies in Computational Intelligence 711, DOI 10.1007/978-981-10-4539-4_1
2 1 Introduction

accepted by both government authorities and academics. Unfortunately, there has

been very limited progress in this direction. In fact, the vast majority of current
computer scientists working on video data analysis are focusing on developing
robust or real-time algorithms that can improve the performance on commonly used
benchmark datasets with general types of objects and scenes. Little attention to
roadside analysis may be due to the lack of such benchmark datasets, and also due
to its restriction to only a relatively small number of specific roadside objects.
In light of the current lack of work specifically focusing on the roadside, this
book aims to be one of the first studies that specifically focuses on roadside video
data analysis using various modern machine learning algorithms, particularly deep
learning techniques, with an emphasis on the use of these techniques for supporting
real-world roadside applications. We describe several industry datasets collected
from the state roads within the Fitzroy region, Central Queensland, Australia, as
well as several commonly used real-world benchmark datasets. We then present an
overview of a general systematic framework for roadside video analysis and
introduce the most widely used feature types and object classification algorithms in
present studies. Various types of non-deep and deep learning algorithms for video
analysis are further described in detail and evaluated using industry and benchmark
datasets. A case study is specifically introduced regarding the development of
automatic and efficient methods to estimating the biomass of roadside grasses and
identifying fire-prone road regions. Finally, we discuss several issues and chal-
lenges associated with analyzing roadside video data, and highlight possible future
directions.

1.2 Collection of Roadside Video Data

A robust and reliable machine learning technique, including deep learning algo-
rithms, is expected to be consistently effective on both local roadside datasets
collected for a specific purpose and general scene datasets taken from various
real-world scenarios. To enable direct and fair comparisons of the techniques
described in this book with other state-of-the-art approaches, this section introduces
four industry datasets and five widely used benchmark datasets.

1.2.1 Industry Data

The datasets used for evaluating machine learning techniques in this book come
from the roadside video data collected by the Department of Transport and Main
Roads (DTMR) using vehicle-mounted cameras in the Fitzroy region, Central
Queensland, Australia. Four cameras are mounted on the front, left, right and rear
directions of the vehicle, which runs across main state roads in Queensland each
1.2 Collection of Roadside Video Data 3

year. All video data is in an AVI format with a frame resolution of 1632  1248
pixels at approximately 10 meters apart, totaling more than 500 GB. The data from
the left-view camera primarily focuses on roadside vegetation regions, and thus can
be used to monitor vegetation growth conditions and find potential fire-prone areas,
whereby steps can be taken subsequently to eliminate the fire hazard.
Like similar real-world video datasets, the DTMR video data lacks pixel-wise
annotation of ground truths of all objects that are present in video frames, which
would require intensive labor efforts considering the huge number of all videos and
the high resolution of frames. However, ground truths are critically important for
designing and evaluating machine learning algorithms, as well as promoting future
research efforts in this field. In this book, we document four image datasets based
on the video captured using the left or frontal camera.

1.2.1.1 Cropped Roadside Object Dataset

We manually cropped a total of 650 small regions from 230 frames for seven types
of objects (100 regions per object except for sky which has 50 regions), including
brown grass, green grass, tree leaf, tree stem, soil, road, and sky as shown in
Figs. 1.1 and 1.2. All frames were selected from the video captured by the left
camera, and thus they primarily focus on the vegetation area rather than the road
area. To simulate realistic conditions as closely as possible, the frames were
selected to cover a wide range of vegetation types, and different scene content, day
times and locations, etc. A set of local regions was then cropped from these frames
and each region was ensured to belong to only one object. It should be noted that
the resulting rectangular regions have different resolution and shapes. The cropped
regions represent various types of appearance of roadside objects, and have very
challenging characteristics in terms of variations in the appearance and content
within the same object, as well as between objects, making them suitable for
evaluating the performance of machine learning algorithms in natural conditions.

Fig. 1.1 Illustration of small regions cropped from DTMR video frames. The white blocks in the
figures give an indication of the location, size and shape of the cropped regions. Note that only a
proportion of representative cropped regions are included in the final cropped roadside object
dataset
4 1 Introduction

Fig. 1.2 Samples of cropped regions for seven roadside objects. There are big variations in the
appearance of the same object and high similarity between objects (e.g. green grass and tree leaf),
which are challenges for accurate object classification. These regions are rescaled to the same
shape and size only for illustration

1.2.1.2 Cropped Grass Dataset

The cropped grass dataset was created to evaluate the performance of classifying
sparse versus dense grasses, and it comprises 110 color images including 60 for
dense grasses and 50 for sparse grasses. The images taken are color images under
natural lighting conditions. To cover various types of grasses in real-world situa-
tions, different canopy sizes were selected as shown in Fig. 1.3. All images were
stored in a JPEG format with a resolution of 900  500 pixels.
1.2 Collection of Roadside Video Data 5

Fig. 1.3 Sample images for (top) sparse and (bottom) dense grasses

1.2.1.3 Natural Roadside Object Dataset

We manually selected 50 images from left-view video data as shown in Fig. 1.4
(independent from those used for the cropped roadside object dataset). These
images were selected to be representative of roadside scenes in different real-world
cases, covering various types of vegetation and other objects, such as soil, road, and
sky. All pixels were manually annotated by a researcher in the computer vision field
into six categories of objects, including brown grass, green grass, tree, soil, road,

Fig. 1.4 Sample images in the natural roadside object dataset (left) with their corresponding
pixel-wise ground truths (right) for six types of roadside objects
6 1 Introduction

and sky, and they serve as ground truths for performance evaluations. The uncertain
regions were labeled as unknown. Unlike the cropped roadside object dataset, two
categories of tree leaf and stem were combined into one category of tree, due to
relatively small sizes of tree stems in the images.

1.2.1.4 Natural Road Object Dataset

The natural road object dataset was generated to evaluate the performance of object
detection in frontal view road conditions. More than 400 images were extracted
from DTMR video sequences that were captured using a frontal view camera. The
images were taken under different lighting conditions and natural settings, rescaled
to a resolution of 960  1280 pixels, and stored in a JPEG format. The images
primarily focus on road lanes, but also include other objects such as tree, grass, sky
and traffic sign. The presence of three objects, including road, sky and traffic sign,
was manually annotated to serve as ground truths for road object detection.
A sample set of images used is shown in Fig. 1.5.

Fig. 1.5 Sample images in the natural road object dataset. The images were captured using a
frontal view camera
Exploring the Variety of Random
Documents with Different Content
precaution, for an early night moth or two, lured by the prematurely
warm weather, had already made its appearance. Then he turned on
the light resolutely and started to undress. The suit he was wearing
was of dark blue serge with a white pin-head stripe, and as he
divested himself of it a new thought sprang up to his mind. Suppose
it, too, bore traces——? That head with its shattered, gaping wound
had rested against his knee. . . .
Seizing the garments he moved close to the bed-stand, and
beneath the powerful rays of the lamp he examined every thread
with straining eyes. No stain was visible; even his shirt cuffs by a
miracle had escaped contact, and with a sigh of relief he plunged his
hands in the various pockets to remove his keys and small change.
The first object his fingers touched was the burnt match with
which he had ignited the spirit lamp, and impatiently he filliped it out
of the window.
Everything was there in his pockets which he normally carried
except his handkerchief. That was gone, reduced to ashes and flung
to the winds of the night; but it would not have been had his
homecoming been as he must pretend even to himself.
Storm frowned. He would in all likelihood never wear that suit
again; even if it were not for the fact that mourning garb alone must
be his for many months to come, he could still never look upon it
again, remembering . . . .
He thrust that thought violently aside and continued with his
reasoning. Agnes always went through the pockets of the clothes he
had worn during the week for stray handkerchiefs when she was
collecting the laundry. Would she note that he had used one less
than usual?
It was not so much the fear of that, however, as the mental urge
to play up to his part, to make all things seem as though that hour in
the den had never been, that prompted Storm to go to his dresser
and take a fresh handkerchief from the drawer. Without effeminacy,
his fastidious taste inclined toward a dash of delicate scent, and
several varieties stood before him.
Tentatively he lifted a bottle of rose toilet water, but the first whiff
of fragrance made him replace it with a shudder. It brought back too
vividly the remembrance of that garden below where the opening
buds were even now scattered with filmy ashes. The lilac water he
also thrust aside—Leila had met him at the gate only a week before
with her arms filled with white lilac, and as she stood there, her head
looming fair and golden above them, he had thought her very like a
picture of the Annunciation . . . .
Finally he sprayed a few drops of eau de Cologne on the
handkerchief and stuffed it in the pocket of his coat lying across the
chair. Then, clad in his pajamas, he glanced at the clock on the
mantel.
Half-past twelve! It would be half-past six in the ordinary course
of events when Agnes would descend to dust the first floor and set
the breakfast table. Six hours to wait! Three hundred and sixty slow,
dragging minutes! How could he ever live through them? How did
the condemned spend the last hours before the end? He had read,
marveling, that some hardy criminals slept unconcernedly, some
raved, some prayed . . . .
God, why did such hideous thoughts intrude themselves now? He
would never stand in their shoes, no breath of suspicion would ever
approach him; he had laid his plans too well, had fortified himself
against any contingency which might arise. The scene had been
perfectly staged with not a detail missing to break the continuity of
the version of what had occurred which must impress itself upon
those who would view it.
Crossing the room, he seated himself on the side of the bed and
lighted a cigarette. He could keep the light going a little longer
without occasioning remark should one of the maids wake up, for he
had often read until past one. He must not overdo it, though; he must
not overdo anything. That would be the one great danger; he must
hold himself impervious against self-betrayal.
He smoked cigarette after cigarette until a single stroke tinkled
from the little clock, and, rousing himself from his reverie, Storm
reached over and extinguished the lamp. The darkness seemed
sinister, overwhelming, but as his eyes gradually accustomed
themselves to it he saw pale, silvery moonbeams creeping in
between the slats of the shutters, lying in shimmering bars across
the floor and lightening the gloom with a faint, almost spiritual
effulgence. The stillness of the night, too, was all at once broken by
a myriad sounds which had not penetrated his consciousness
before: strange creaks and groans in the walls as though something
invisible were abroad, sibilant whispers in the chimney and the liquid,
monotonous tap of water dripping from the faucet in the bathroom.
Outside, the wind blew gustily, and somewhere about the house a
loose shutter banged with a dismal, hollow sound.
What a hideous thing night was! There was something about it
which loosened a fellow’s thoughts, freed them from his stern control
and let them wander where they would, unrestrained. Only one other
such vigil had Storm kept: that of the crisis in Leila’s desperate
illness after their marriage. Every hour of it was branded on his brain,
every detail arose again to attack his senses: the pungent,
penetrating odor of carbolic, that strange, high voice which babbled
and was still, the white-clad nurse, gravely noncommittal, shutting
the door behind which he might not pass, the taste of his own blood
as he caught his lip between his teeth to keep back the groan of utter
despair.
The night then had seemed interminable, but in the end had
come the glorious promise that she would live! Now the dawn would
bring only tidings of death, but he would not call her back again if he
could; would not undo what he had done even if it lay in his power.
His Leila had never existed; the pedestal was empty, that was all!
Gad, if only he could smoke! His nerves shrieked for the solace
of nicotine, but he dared not light another cigarette. The smoke
curling up from his opened window to that of one of the maids
upstairs would tell her, should she also be awake, that her master
was keeping vigil there in the darkness alone. She would think
nothing of it now, perhaps, but later when the discovery was made
she might wonder. He must manage, somehow, to get through the
night without even the slight comfort that he craved!
With a whirring of soft wings, some tiny creature of the night
came and beat upon the shutter, and Storm started violently. The
bars of moonlight had traveled a barely perceptible inch or two
across the floor, and from the distance there came the crowning note
of desolation: the long-drawn, mournful howling of a dog.
Storm shivered. An old superstition which his Irish nurse had
instilled into his mind in the nursery days swept over him. A dog’s
howl was the sign of death! How could the beast know? In all that
sleeping countryside, there was one who shared his vigil, one who
raised his voice in warning and lament!
Storm rose and, tiptoeing to the window, opened the shutters
wide and fastened them noiselessly back against the house wall
while he strained his eyes in the direction from which the dismal
baying of the dog rose once more. Was it nearer now? Could it be
that the beast, led by some instinct more subtle and unerring than
man could fathom, had picked up the scent—the scent of the drifting
ashes? Bridget had told him that a dog could sense the presence of
death though it were miles away and would come to cry the news of
it. What if the creature were to appear suddenly there between the
trees and leap across the lawn to crouch beneath the curtained
window of the den downstairs and howl its dread message?
The next minute Storm’s tense attitude relaxed. What a fool he
was to be stirred by the idle superstition of an old-wives’ tale! His
nerves must be going back on him, what with that accursed howling
and the shifting shadows of the moonlight which were worse than
utter darkness could be. Would it never end?
As if in answer the mellow chime of the clock sounded upon his
ears. It must be three o’clock at least, possibly four——! He waited
breathlessly. A second note pealed forth softly to die away in a
vibrating echo, and then silence. Only two o’clock! Nearly five hours
more! God, could he endure it and keep his sanity? Doyle’s
gruesome story of Lady Sannox came to his mind. Would he be
found in the morning as the great physician had been after the night
of horror, a gibbering idiot trying to thrust both feet into one leg of his
trousers and babbling meaninglessly? Lady Sannox had been
unfaithful, too, but it was the lover, not the husband, who paid!
He forced the hideous picture from his thoughts and turned for
one final glance at the garden below. How still everything was! The
howling of the dog had ceased, and the wind had died down to a
mere rustling, whispering breeze. The moonlight, too, was paling,
and beneath its waning radiance the garden still slumbered
undisturbed as it had when he cast the ashes forth upon the air.
From these ashes would spring the phoenix, not of love, but of
murder; of hatred, vengeance and the lust to kill! What had he not
loosed upon the world!
He covered his eyes as if to shut out the scene of false peace, of
menacing, brooding calm before the crimson dawn; and staggering
back to the bed, he sank down upon it once more. The touch of the
smooth, cool linen beneath his fevered hand steadied him and
brought a moment of tranquility to his reeling senses, but he could
not stretch himself out upon it. The space beside him where Leila
had so often lain was blank and empty, yet oddly her presence
seemed near. He could almost hear her light tap upon the
connecting door, almost see it open and her slender, white-clad form
appear with the two heavy ropes of golden hair falling over her
shoulders. She would come to him swiftly, tenderly, and he would
take her in his arms and hold her close . . . .
But no tapping came upon the door, no form appeared, his arms
were empty! Great God, why could he not forget!
The clock struck three, the moonlight faded and vanished,
swallowed up in the darkest hour which comes before the dawn, and
still Storm crouched there at the bed’s foot sunk in a reverie of
retrospection.
In just such another springtime as this they had gone upon their
honeymoon. The awe and ecstasy of those days like a half-forgotten
fragrance stole again over his spirit and thrilled him anew. How
wonderful she had been, how wondrously sweet her shy
confidences, her little outbursts of tenderness, her bewitching,
bewildering changes of mood! How he reveled in each new phase of
her nature as it revealed itself to him; how he had worshipped her,
gloried in the possession of her! In the golden years that followed,
the first ecstasy had not faded; it had but stabilized, deepened into a
steady glow of unquestioning devotion, and the honeymoon had
never really ended until this hour!
Impotently he struck his forehead with his clenched fist. Why
must he go on thinking, thinking! The past was dead, buried beyond
hope of resurrection! Why must it come trooping back to rob him of
his strength and lull him to forgetfulness of the immediate future and
the crisis which impended? The night had been years long! Would it
never come to an end? Would this hideous darkness envelop him
forever?
Four o’clock! Thank God, he had missed an hour! Only two more
now, or three at the most, and then the cry of alarm would come
winging up from below and the curtain would rise!
A chill dampness as of the grave itself stole in at the window, and
Storm shivered although he was bathed in sweat. His pulse slowed
and weakness descended upon him, while a swift, unnerving fear
laid its clammy hands upon his throat.
He fought it off desperately. This was the dreaded hour before
the dawn, the hour of lowered vitality when life’s guard is down and
death stalks in upon those awaiting it, those whose time has come
and who slip out into the unknown quietly, peacefully. But for those
who are hurled into it suddenly, hideously, by shot or stab or crashing
blow——!
He dropped his wretched head upon his hands. This was
madness! He must not succumb to it, he must marshal his
resources, steady his brain, gather strength for the coming of day!
Slowly, almost imperceptibly, the darkness was changing from
black to gray. The eastern sky was unbroken, but a mist which could
rather be felt than seen was rising from the darker shadows, and the
wind had been succeeded by a dead calm. A hushed expectancy
seemed to brood over the world, and Storm waited, too, dreading yet
longing for the end of this prolonged suspense.
The clock ticked with maddening precision, and he tried to count
the minutes, to keep his traitorous thoughts from wandering into
dangerous, forbidden channels. His weakness had fallen from him,
his pulse quickened and a mounting excitement drove from him all
thought of fear. He would be ready when the time came to meet the
issue. But would the time ever come?
There were faint gray streaks in the sky now, the shadows had
sharpened and suddenly, piercingly, a cock crew. Storm welcomed
the strident sound with uplifted head and squared shoulders. The
dawn was coming at last!
He turned, crossed his arms on the foot of the bed and, resting
his chin upon them, stared out through the open window at the
lightening sky.
 Five liquid, mellow notes sounded from the mantel, and he smiled
grimly. One hour more and he could begin to listen for the maid’s
step upon the stair! His nerves were tingling in anticipation, and
without urging his thoughts leaped ahead. He must be ready when
the cry came, but not too obviously prepared. Surprise must come
before alarm, consternation before a show of grief. The maid herself
must lead him to her discovery. His face and manner must reveal no
slightest inkling of his knowledge of the truth.
Both of the servants were undeniably stupid. He had
anathematized them many a time for their crass density and
ignorance, but now he blessed it. They would suspect nothing, would
seize upon any explanation of the tragedy which was subtly planted
in their shallow brains and make it their own.
Of the outsiders, Carr must be called in first. He was a country
practitioner of the old-fashioned sort who had been established there
when Greenlea was known as Whigham’s Corners, and croup and
gout with their intermediate ills had been the range of his experience.
He, too, could be counted upon to see only what was placed before
him, and the details of the aftermath could safely be left in his hands.
A score of vague, anticipatory visions passed through Storm’s
brain. How shocked the crowd out here would be; and old George!
He was probably fast asleep now, filling the air with contented
snores. What would he say and do when the early edition of the
evening papers brought the tidings to him? Storm thanked heaven
that neither he nor Leila had relatives to come flocking with tears and
questions and advice. He would be free at least from prying eyes
beneath his own roof after the official medical inquiry had been
concluded.
Gray turned to rose in the eastern sky, the mist lifted, and the
world showed delicately green beneath it. The lone cock’s crowing
had been augmented by a chorus, and the birds stirred and twittered
in the trees. All life was waking to greet the new day, but Leila. . . .
What time was it? Storm rose weakly and tottered to the mantel.
The clock’s face was plainly visible in the half-light, and he drew his
breath sharply. Five minutes to six!
The pink glow deepened to crimson, and the sun in a blaze of
glory peeped over the low-lying hills, but Storm did not see the
spectacle for which he had waited through interminable tortured
hours. He had caught a glimpse of his reflection in the mirror and
stood gazing in incredulous dismay at the face which gazed back at
him. Could it be his own with that sickly, bluish pallor, unshaven jaw,
and haunted, sunken eyes which stared from dark-rimmed sockets?
Great heavens, if he appeared like that before even the servants
his guilt would be patent to all the world!—But would it? The maids
would surely be too agitated to note him in the shock of the
discovery; and later, when the doctor came, horror and natural grief
would account for the change in his appearance. His night of vigil
had provided him with that which would perhaps be an asset rather
than a danger.
What was that? He whirled about and stood listening. The floors
and ceilings were thick, and no ordinary sound could penetrate from
above; but had he not heard a step upon the stair? He waited in an
intensity of strained attention, for several moments. The silence
within the house remained unbroken.
With a sigh he glanced back at the clock. It must have struck the
hour while he stood glaring at the apparition the mirror had revealed,
for now the hands pointed to a quarter after six. Could it be that
perverse fate would ordain that the maids should over-sleep this day
of all days and prolong his agony?
Then his glance fell upon the bed. Its pillows were smooth and
untouched, its covers creased but not tumbled about. The veriest
child could see that it had not been slept in, the most casual glance
would reveal the secret of his night-long vigil!
In three strides he had reached it and thrown back the covers,
pommeling the pillows and crumpling the sheets. What a narrow
escape! He paused, breathless, when his task was completed and
gazed fearfully about him for other overlooked evidence.
His light had been burning until one o’clock. Hastily he picked up
a book at random from those on the table and opening it laid it face
downward upon the bed-stand. The stubs and ashes of the
cigarettes he had smoked would occasion no remark; and the most
painstakingly minute scrutiny failed to reveal any other incongruity in
the room.
 While he paused anew a sound came to his ears about which
there could be no doubt; cautious but naturally heavy footsteps were
descending the stairs from above. His heart leaped, and the blood
raced in his veins, but he stood motionless as the steps passed his
door and descended again.
It would come now, the cry for which he had waited! He held his
breath until his ear-drums seemed bursting, and the minutes
lengthened, but still the summons did not come. What could the girl
be doing? Would she set all the other rooms to rights before
approaching the den, or did she mean to shirk it altogether? Surely
that streak of artificial light burning in the daytime must catch her eye
as she passed along the hall! Was she gossiping with the milkman,
idling on the porch? The suspense was unbearable!
He had borne with it through the long watches of the night, but
now he could contain himself no longer. Every nerve was strained to
the breaking point, and his nails bit into the flesh of his clenched
hands. Was this agony to be stretched out interminably?
And then it came at last! A piercing, prolonged scream rang
suddenly through the quiet house, to break and rise again, echoing
back from the very walls.
Storm dropped his head in his hands, and an answering cry of
unconscious blasphemy trembled on his lips.
“Thank God!”
 Chapter V.
When Morning Dawned
While Storm hesitated, relaxed in that moment of utter abandon
to relief, it came again; a wild shriek mounting from below in a high
feminine voice and dying away in a quivering wail!
The long awaited discovery had come; now he must play his part.
One false move!——But he put that resolutely from his thoughts as
he flung his dressing gown about him and started for the door.
“What is it? What has happened?” He was leaning Over the stair-
rail now, and his voice, although subdued, held just the proper note
of sharp inquiry. Even as he spoke he heard a heavy foot along the
hall above and was conscious of the cook’s head peering down
affrightedly.
“Oh, Mr. Storm, sir! Mr. Storm!”
Agnes, the housemaid, sped along the lower hall and collapsed
at the foot of the stairs.
“Well, what is it?” Storm demanded peremptorily, but still in that
subdued tone. “Burglars here in the night? Don’t you know better
than to scream like that? You’ll frighten Mrs. Storm——”
He paused, and the girl’s shocked wail arose once more.
“Mrs. Storm! She’s down here, sir, in the den. Oh, come quick!”
“Down——!”
The word died in Storm’s throat, and still conscious of the cook’s
eyes he turned, dashed open the door of his wife’s empty room,
uttered a loud ejaculation and then plunged down the stairs.
“I thought she was asleep in her room!” he exclaimed. “Where
——?”
“In the den, sir!” Agnes scrambled to her feet and stood clinging
to the newel post as Storm passed her and rushed down the hall.
“Oh, may God have mercy——!”
 He heard a startled cry from above and lumbering feet hastily
descended the stairs as he burst into the den and then stopped
short. Leila’s body was lying face upward now upon the rug, her
waxen features clamped in the rigidity of marble, a hideous brown
clot enmeshing the soft gold of her hair and smeared across her
forehead.
The cry of horror which burst from Storm’s lips was not all
simulation, for anticipated as it was, the sight brought a sickening
qualm to him. He had conquered it the next moment, however, and
crossing to the body knelt and forced himself to touch it, to raise it
until it rested against his knee just as he had done the moment the
blow was struck. It was cold and stiff, the neck rigid, the eyes half
open and unwinking in their stare.
As the trembling servants appeared in the doorway he laid the
body gently back upon the rug and, rising, dashed his hand across
his eyes. He remembered that gesture; he had often seen a favorite
tragedian use it upon the stage.
“She is dead!” Horror, grief unutterable rang in his tones, and the
maids began to sob hysterically.
Without seeming to note their presence Storm staggered past
them to the telephone in the library.
“Greenlea 42 . . . . Dr. Carr, please . . . . Doctor, this is Storm,
Norman Storm. For God’s sake get over here as quickly as you
can! . . . . No, I can’t go into details, but it’s a matter of life and
death! . . . . All right, hurry, man!”
For a moment he sat there hunched over the silent instrument
while the sweat poured in rivulets down his face. So far, so good. His
shaking nerves were aiding him in the rôle he was playing, but he
must not let them get the upper hand.
The early morning sun streamed in at the long French windows
which opened on the veranda, and the twitter and chirp of birds
came to him from the lawn outside, mingling with the muffled wail
from the rear. He must go back. God! If only it were all over!
Agnes had collapsed again in a little heap in the den doorway,
but Ellen, the cook, knelt by the body, crooning pitifully over it as
Storm reentered. She made a grotesque figure clad only in the
blanket which she had thrown over her voluminous nightgown, her
iron-gray hair screwed back in a tight knob and tears streaming
down her round, honest face.
“Oh, sir!” She looked up, her eyes tragic with horror. “Who in the
world did it, sir?”
Storm started. A suspicion of murder already, and from the
source which he had least anticipated! If stupid, unimaginative Ellen
had leaped to such a conclusion could he hope after all that the truth
would not reveal itself to Dr. Carr and the authorities? He moistened
his lips with his tongue and stammered:
“She—she must have fallen—one of those fainting spells. It looks
as though she had struck her head on the fender, there.” He added
quickly, “When I came home late I supposed Mrs. Storm was asleep
in her room and did not disturb her. How did she come to be here?”
“Must have been waiting up for you, sir.” Agnes lifted her head
from her hands. “The mistress didn’t expect you home for dinner,
and I served her on the little table out on the veranda. She was
sitting out there still when Ellen and me went to bed, along about
nine. I asked her should I wait to lock up or see if you wanted a bit of
cold supper, sir, but she said no, that she would attend to it herself. If
only I’d known one of those attacks was coming on I wouldn’t have
left her for a minute! I’ll never forgive myself! But the mistress
seemed all right, as ever she was in her life, and I was that tired——”
Storm eyed her steadily:
“You would have heard Mrs. Storm had she called for help?”
“I don’t know, sir.” The girl twisted her hands. “I’m a pretty heavy
sleeper, and I never heard a thing during the night. I’ll never forget
the turn it gave me when I came down this morning and found the
light still on and her lying there on the floor——”
“God rest her soul!” Ellen ejaculated piously. “Sure we wouldn’t
have heard, away up there on the top floor at the back, unless she’d
screamed fit to wake the dead. I’d had a full day’s ironing, and I was
asleep the minute my head touched the pillow. The first I knew was
when Agnes here let that yell out of her awhile back. The best lady
ever I worked for and the kindest! She must have been took sudden
to fall over like that!”
Storm drew a breath of relief. It was evident that they were telling
the truth and that neither of them was aware of Brewster’s visit on
the previous night, nor had an inkling of its aftermath. He sank into a
chair and buried his face in his hands, the better to think. He must
get rid of them some way; their chatter and lamentations were
driving him mad!
“ ’Tis God’s will, sir,” Ellen ventured, in a hesitating effort at
consolation, though the tears still coursed unchecked down her
cheeks. “Couldn’t we move her, sir? ’Tis terrible to leave her lying
here, poor lady——”
“Not until the doctor comes.” Storm’s tones were hoarse and
muffled. “Please go away, both of you. I want to be alone. Mind you
say not a word of what has happened to the milkman or anyone else
who may come to the door until the doctor has taken charge. We
should have all the neighbors about our ears.”
“We won’t breathe a word.” Agnes scrambled to her feet. “You’ll
ring, sir, if you want anything? A cup of coffee, now——?”
“Nothing!” Storm waved aside the suggestion with a shudder of
disgust. “I only wish to be alone.”
When the maids had withdrawn and their sobs were cut off by the
closing of the pantry door, Storm’s hands dropped to his knees. They
had accepted his suggestion of the cause of death without question,
but would it be safe for him to volunteer that theory as a foregone
conclusion to the keener mind of the doctor? He knew the strength of
first impressions; were the circumstantial proofs of accidental death
obvious enough to preclude all suspicion of foul play? The evidence
which had seemed so impregnable to him when he first conceived it
crumbled before the wave of torturing doubt that assailed him. He
did not find it as easy as he had planned to put behind him forever
his secret knowledge of the truth. What would his thought processes
have been had he indeed believed his wife to be sleeping safe in her
room and come down to find her lying dead here?
The whirr of a light-running motor outside galvanized Storm into
action, and he sprang up from his chair and hurried down the hall,
flinging the front door wide just as the doctor mounted the veranda
steps. A fine, grizzled stubble adorned the latter’s usually clean-
shaven jaw, and his light ulster was buttoned close up about his neck
as though to conceal deficiencies in his hastily donned attire.
 “What is it, old man?” he began genially, and then at sight of the
other’s face he paused abruptly.
“Come.”
Without another word Storm turned and led the way to the den,
and the physician followed in silence. At the door the former, with a
gesture, stepped aside, and Dr. Carr’s glance fell upon the body.
Stifling an exclamation he advanced and made a brief, deft
examination. Then, shaken from his professional calm, he rose.
“There—is nothing I can do,” he announced jerkily. “She has
been dead for several hours—seven or eight, at least. Good God,
Storm, what does this mean?”
The gaze of the physician was filled with blank amazement and
horror, but to the other man it seemed sternly accusing, and he
stammered brokenly:
“I don’t know! She must have been here all night like this, while I
thought her safe in bed and asleep! It is horrible! Horrible!”
He hid his face in his hands to shut out those keen eyes bent
upon him, and Dr. Carr advanced and forced him gently down into a
chair.
“Here, man, don’t give way now! Pull yourself together! Do you
mean that you only just discovered——?”
“A minute before I telephoned to you. It was the housemaid who
found Leila like this when she came down to dust around, and her
screams awakened me.” Storm paused. A detailed explanation
would look too much like an attempt at an alibi; he must wait for the
other to drag the facts from him. “Oh, why didn’t I speak, why didn’t I
look in her room when I came home last night! But I was afraid of
disturbing her——”
He paused, and Dr. Carr asked quickly:
“You returned late and thought she had retired?”
“Yes. It was after eleven—I took the ten o’clock train from town—
and when I got here the house was all dark and silent, and Leila’s
bedroom and dressing-room doors were closed.” Storm’s hands
dropped to the arms of his chair, and he stared straight ahead of him
as he added deliberately: “I went to bed as quietly as I could so as
not to waken her, for she hadn’t been well; she was threatened with
one of these fainting attacks the night before last. I should never
have left her! But you know how it has been, Doctor; you never could
tell when they were coming on, and she had never done any real
harm to herself before——”
“ ‘Fainting attacks?’ ” the doctor repeated sharply. He wheeled and
approached the body once more and Storm watched him with bated
breath. “The right temple bone has been crushed in, as if with some
heavy, blunt instrument!”
“That knob on the corner of the fender——” Storm felt his way
carefully. “It—it’s all covered with blood! She must have fallen——”
The doctor glanced at it and then turned swiftly to him.
“Look here, Storm, have you questioned the servants? What do
they know of this?”
“Nothing. I’ve been too nearly crazed to question them
coherently, but from what I gathered they went to bed early and left
her sitting out on the veranda, and the housemaid said something
about Leila having told her that she would wait up for me, Think of it,
Doctor! She must have come in here——”
“Hold on a minute. Was the body lying just like this, face
upturned, when you saw it first?”
Storm nodded.
“Yes. I rushed to her and started to lift her up, but when I saw that
—that she was dead——”
He bowed his head on his breast as if unable to continue, but he
saw the physician measure with a swift eye the distance from the
chair to the body, and then stoop to examine the fender again.
Storm’s knuckles whitened as he gripped the chair-arms in an agony
of suspense. Would his implied suggestion bear fruit? Had he too
palpably ignored the other’s intimation that a blow had been struck?
Would it have been more natural for him to have presupposed
violence, murder, as the physician obviously had done? It was too
late now for him to question the wisdom of his course; he could
brace himself for the next step in the ghastly farce.
“Has anyone touched the body?” Dr. Carr spoke with professional
brusqueness.
“Yes; when I came back here after telephoning to you both the
servants were in the room, and Ellen was bending over my poor wife
—I can’t speak of it, Doctor; I can’t realize it! I feel as if I should go
mad! Leila——”
“I know, old man, but we’ve got to get at the bottom of this thing.
Try to collect yourself and think back. You said you were awakened
by the housemaid’s screams when she discovered the body. Do you
know if she touched it before you got down here?”
Storm shook his head.
“I never asked.” He kept his eyes lowered carefully to hide a glint
of triumph. When Carr discovered that Agnes had found the body
lying face down, the case he had manufactured would be complete.
“I wish you—you’d talk to them, Doctor. I—I can’t, just now. I’m all
in!”
“I will. Don’t think about them.” Dr. Carr glanced at the low light
on the desk which still glowed brazenly in the gloom of the curtained
room. “Who turned on that light instead of drawing aside the
curtains?”
“It must have been on all night. Agnes said that was the first thing
she noticed when she got to the door here; that the light was still
going. Then she saw the body——” he halted again and added in
studied ingenuousness: “I might have observed it when I came home
last night, I suppose, but it is scarcely perceptible from the front hall,
and finding it all dark there except for the lamp on the newel post
which Leila always leaves lighted for me, I went straight upstairs. It
never occurred to me that she would be waiting up for me, and in
here, although she has done so occasionally. I was pretty tired. My
God, Doctor, if I had seen the light and come in here, I might have
been able to save her! There might have been something I could do
—!”
“No, Storm, no one could have done anything for her. Death was
instantaneous. You heard nothing after you went upstairs? No sound
of a fall, or disturbance of any kind?”
“Nothing.” Storm started from his chair. “It couldn’t have
happened after I retired! Surely, if Leila had been alive when I
entered the house she would have heard me! The servants sleep
like logs, but I waken at the slightest sound. I would have known
——!”
 “That’s so. Of course, you would, old man.” Dr. Carr’s tone was
soothingly compassionate. “You’d better go upstairs now and put on
some clothes; you haven’t even slippers on your feet. I’ll have a word
with the servants while you’re gone——”
“But Leila!” Storm forced his shrinking eyes to turn yearningly
toward the still form. “I can’t bear to leave her lying like that! Ellen
wanted to lift her to the couch, but I thought we’d better wait until you
came——”
“Why?” The doctor shot the question at him, and Storm, realizing
his slip, swiftly countered:
“I didn’t know what to think! I tell you, Doctor, when I first came in
and saw her it looked almost like murder! My brain isn’t clear yet
from the shock of it, although when I saw the blood on the fender, of
course, I knew she must have fallen, and then I remembered her
condition—those fainting spells, and all that. There isn’t a soul in the
world who would harm a hair of Leila’s head!” He threw up his hands
with an impotent gesture. “I felt dazed, helpless! I had to depend on
you, and I wanted you to see everything just as it was.”
“You can rely on me, old man!” The doctor patted his arm and led
him to the door. “We must not move her yet, however. Under the
circumstances we’ll have to notify the authorities, merely as a matter
of form, and they may want to investigate for themselves. I’ll call
them up and then come and give you something to steady your
nerves. You’re bearing up splendidly, but we can’t have you going to
pieces until the formalities have been concluded. Is there anyone
you would like me to send for; any member of the family, or friend?”
“Yes!” Storm exclaimed in a sudden flash of inspiration. “Get old
George for me, will you, Doctor? George Holworthy, you know; 0328
Stuyvesant. Tell him to come out here on the first train, that I need
him. Don’t—don’t go into details, but make him understand that it’s
serious, desperate! I’m not a weakling, I won’t break down, but I’d
feel stronger if George were here. We’ve been friends for years.”
“I know; I’ll get him.” Dr. Carr drew out his handkerchief and
mopped his forehead. “Get into some clothes now. I’ll be right with
you.”
“I would rather stay here with Leila, alone with her——” Storm
murmured mendaciously. “I won’t touch her, Doctor; let me stay!”
 “No.” The physician transferred the key to the outside of the door
and locked it decisively as he spoke. “It wouldn’t do you any good,
Storm. You’ve got to brace up. You have put this affair in my hands
now, and I order you to pull yourself together. Get upstairs and take
a cold shower and then I’ll give you a sedative.”
With a last glance at the closed door, Storm stumbled to the
stairs and mounted, lurching against the banisters as though
overcome by weakness; but in reality his brain was seething with the
thought of the danger yet ahead.
He closed the door of his room softly behind him, and then
paused. What if Carr’s sympathetic, friendly manner had been
assumed to cloak a suspicion of the truth? The physician had
seemed to accept his theory, but he had not committed himself.
Suppose he were following, tiptoeing up the stairs now to peer in at
the keyhole——!
The thought was madness, yet Storm turned instinctively. The
key of his door had been mislaid long ago and never replaced, but a
heavy lounging robe hung from a peg on the center panel. Catching
a fold of it he drew it back over the door-knob so that it trailed before
the lock like a curtain, thick and impenetrable. His bathroom had no
entrance leading to the hall, and the only other door—that opening
on Leila’s dressing-room was protected by a cretonne portière.
He realized that he had no need for secrecy, there was nothing to
be done now which all the world might not safely see—and yet an
insane desire came to him to conceal himself from all eyes. He must
have a moment of respite from the rôle he was playing, a moment of
peace and calmness to gird himself anew for what the immediate
future might hold.
Did Carr accept the situation at its face value? The man whom in
the night he had half-scornfully dismissed from his mind as a simple
country practitioner now appeared in a vastly different light. For the
moment he held in his hands Storm’s immunity from suspicion, and
the latter’s disquietude increased.
There was his step upon the stair! What would his face reveal?
With a quick revulsion of feeling, Storm sprang to the door and
opened it.
 The smile with which the benign physician greeted him removed
all lingering doubt.
“Not taken your shower? Come, Storm, this won’t do! I’ve
’phoned, and the coroner’s assistant is on his way over from the
county seat.” He held out a small glass, and the other took it
mechanically. “Drink this and pull yourself together, for there are
some trying hours ahead.”
 Chapter VI.
The Verdict
“You were right, unquestionably, Storm,” Dr. Carr announced
twenty minutes later as the other joined him in the library. “I don’t
mind admitting that my thoughts—my sensations, rather—when first
I saw the body were identical with what yours had been, but there’s
only one possible conclusion. Mrs. Storm must have been seated in
that big chair by the hearth when she felt suddenly faint; and in trying
to rise, she must have fallen forward, striking her forehead with
crushing force against that solid brass knob on the fender. Agnes
tells me that she found her mistress lying face downward against it,
and thinking she had merely fainted, turned the body over. It was
only when she saw the wound that she screamed. Of one thing you
may be sure; Mrs. Storm didn’t suffer. She never knew what struck
her. Death came instantly.”
Storm sank into a chair, his twitching face turned from the light. If
Carr only knew!
“I must try to think that!” he murmured. “Did you get George
Holworthy on the ’phone, Doctor?”
“Yes. He is on his way out here by now. Agnes gave me some
coffee, and I told her to bring a tray for you—No protests!” as Storm
made a gesture of repugnance. “You are under my orders,
remember, and you’ve got to keep going.”
Storm drank the coffee obediently enough when it came,
conscious of a craving for its stimulus. The first and most hazardous
milestone was passed; Dr. Carr had fallen for his game, had been
completely hoodwinked by the circumstantial evidence he had
arranged. He had won an unconscious yet powerful ally, and the way
seemed clear before him, but the glow of elation was past.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com