See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/334410680

Online Incremental Machine Learning Platform for Big Data-Driven Smart

Traffic Management

Article in IEEE Transactions on Intelligent Transportation Systems · July 2019

DOI: 10.1109/TITS.2019.2924883

CITATIONS READS
115 2,023

9 authors, including:

Dinithi Nallaperuma Rashmika Nawaratne

La Trobe University La Trobe University
8 PUBLICATIONS 213 CITATIONS 27 PUBLICATIONS 435 CITATIONS

SEE PROFILE SEE PROFILE

Tharindu Bandaragoda Achini Adikari

La Trobe University La Trobe University
24 PUBLICATIONS 455 CITATIONS 29 PUBLICATIONS 425 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Communication Connect: Improving long term communication and mental health outcomes following stroke and brain injury View project

Brain Inspired Lifelong Machine Learning View project

All content following this page was uploaded by Tharindu Bandaragoda on 16 July 2019.

The user has requested enhancement of the downloaded file.

 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 1

Online Incremental Machine Learning Platform for

Big Data-Driven Smart Traffic Management
Dinithi Nallaperuma, Rashmika Nawaratne, Tharindu Bandaragoda , Achini Adikari, Su Nguyen, Member, IEEE,
Thimal Kempitiya, Daswin De Silva, Member, IEEE, Damminda Alahakoon, Member, IEEE,
and Dakshan Pothuhera

Abstract— The technological landscape of intelligent transport Things (IoT), sensor networks and social media has surpassed
systems (ITS) has been radically transformed by the emergence traditional means of collecting data, by creating voluminous
of the big data streams generated by the Internet of Things (IoT), and continuous streams of real-time data. Leveraging such big
smart sensors, surveillance feeds, social media, as well as growing
infrastructure needs. It is timely and pertinent that ITS harness data environments is a formidable issue, due to the intense vol-
the potential of an artificial intelligence (AI) to develop the ume and velocity at which data is generated by transportation
big data-driven smart traffic management solutions for effective and mobility systems [1]. Furthermore, the dynamic nature of
decision-making. The existing AI techniques that function in these environments makes the data generation volatile, which
isolation exhibit clear limitations in developing a comprehensive impedes the effectiveness of decision-making in ITS.
platform due to the dynamicity of big data streams, high-
frequency unlabeled data generation from the heterogeneous data The dynamicity of data generated by transportation systems
sources, and volatility of traffic conditions. In this paper, we pro- consists of continuously changing patterns and concept drifts.
pose an expansive smart traffic management platform (STMP) In a traffic context, concept drifts are the changes to the
based on the unsupervised online incremental machine learning, distributions of data in a traffic data stream over time [3].
deep learning, and deep reinforcement learning to address these Based on the nature of fluctuations in data streams, these
limitations. The STMP integrates the heterogeneous big data
streams, such as the IoT, smart sensors, and social media, changes are further classified as recurrent and non-recurrent
to detect concept drifts, distinguish between the recurrent and concept drifts. For example, traffic congestion changes due
non-recurrent traffic events, and impact propagation, traffic flow to peak/off-peak traffic is a recurrent concept drift whereas
forecasting, commuter sentiment analysis, and optimized traffic an accident or breakdown is a non-recurrent concept drift.
control decisions. The platform is successfully demonstrated Special importance should be placed into identifying non-
on 190 million records of smart sensor network traffic data
generated by 545,851 commuters and corresponding social media recurrent concept drifts as it could affect the entire road
data on the arterial road network of Victoria, Australia. network. Existing literature reports a number of supervised
machine learning algorithms that detect drifts and adapt to new
Index Terms— Smart traffic management, concept drift, unsu-
pervised incremental learning, deep learning, deep reinforcement concepts [3]–[6]. Although real-time concept drift detection is
learning, impact propagation, traffic optimization, traffic fore- crucial for effective decision making in transportation, feed-
casting, traffic control, social media analytics. back on the type of traffic incident is only received following
an unknown delay. This severely limits the applicability of
I. I NTRODUCTION the supervised learning nature of these algorithms. Therefore,
we postulate that concept drift detection in road traffic requires
R OAD traffic conditions and flow management con-
tinue to be an important area of research with many
practical implications. During the last decade, the techno-
unsupervised online incremental machine learning to address
the challenges of real-time, unlabeled, volatile data streams.
In this work, we distinguish online learning and incremen-
logical landscape of transportation has gradually integrated
tal learning. Online learning updates the model using each
disruptive technology paradigms into current transportation
incoming data point that arrives during the operation, without
management systems, leading to Intelligent Transportation
storing [7]. As such, online learning is utilized to handle
Systems (ITS) [1], [2]. The emergence of Internet of
large volumes of streaming data arriving at high velocity.
Manuscript received July 15, 2018; revised November 7, 2018 and Incremental learning is learning from batches of data at distinct
April 8, 2019; accepted May 31, 2019. This work was supported in part by the time intervals, and has the capability to stabilize the historical
Australian Government Research Training Program Scholarship and in part by
the Data to Decisions Cooperative Research Centre (D2D CRC) through the knowledge of the learning model over novel learnings [8].
Analytics and Decision Support Program. The Associate Editor for this paper Hence, the model is updated to any new data point that is
was J. Sanchez-Medina. (Corresponding author: Tharindu Bandaragoda.) received while keeping its existing knowledge intact. Further,
D. Nallaperuma, R. Nawaratne, T. Bandaragoda, A. Adikari, S. Nguyen,
T. Kempitiya, D. De Silva, and D. Alahakoon are with the Research Centre it is essential that non-recurrent concept drifts are identified
for Data Analytics and Cognition, La Trobe Business School, Bundoora, VIC and utilized for updated traffic propagation and traffic flow
3083, Australia (e-mail: [email protected]). prediction models in a real-time manner.
D. Pothuhera is with the Department of Information Management and
Technology, VicRoads, Kew, VIC 3101, Australia. To this end, we further address several key concerns which
Digital Object Identifier 10.1109/TITS.2019.2924883 are underexplored in current ITS, to support the development
This work is licensed under a Creative Commons Attribution 3.0 License. For more information, see http://creativecommons.org/licenses/by/3.0/
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

of a holistic traffic management platform. Following an exten-

sive review of current literature in ITS [1], [2], [9] we
identified the following four current challenges which have not
been sufficiently addressed. a) the development of real-time
machine learning algorithms and prediction schemes for non-
recurrent traffic incidents that impact an entire road network.
b) a majority of existing approaches focus on freeways and
highways, with very limited attention to arterial networks due
to the technical challenges of integrating multiple streams of
traffic data, thus fails when determining traffic propagation
in the entire network. c) current approaches do not account
for network level spatiotemporal variables that are expressed
as big data streams. d) the human element of road traffic,
commuter sentiment and emotions expressed regarding traffic
on social media channels, which are largely overlooked in
current ITS research. While social media is increasingly being
Fig. 1. Smart traffic management platform architecture.
used in emergency events [10]–[12], integrating such data with
other traffic-related data would provide a holistic view of the
situation from both road dynamics and commuter perspective. propagation estimation (Section II-C), traffic flow forecast-
Addressing these limitations, we designed and developed ing (Section II-D), optimization for intelligent traffic control
the ‘Smart Traffic Management Platform’ (STMP), an expan- (Section II-E) and commuter emotion analysis (Section II-F).
sive intelligent traffic data integration and analysis platform, L3 is also expansive as further modules can be linked to L2 for
that integrates heterogeneous traffic data sources. With the other traffic management activities.
proposed STMP platform we present the following research
contributions.
• a novel online incremental machine learning algorithm A. Data Transformation
to detect real-time concept drifts from big data streams The data transformation layer receives heterogeneous
• a deep learning approach for real-time network level sources of big data streams related to road traffic, such as
traffic flow prediction and impact propagation estimation IoT, sensor network data, social media data, video surveil-
in arterial road networks lance feeds, weather data, planned public events, and road
• a deep reinforcement learning approach to determine construction activities. These data sources are pre-processed,
optimal traffic control actions based on real-time mea- transformed and integrated into a computational format that
surements can be effectively ingested by the online incremental machine
• a social media data integration model to capture social learning algorithm in L2. Due to space limitations, we scope
behaviors during a non-recurrent traffic event, to deter- this study to trajectory data from Bluetooth Traffic Monitoring
mine commuter sentiment and emotion System (BTMS) and social media data.
• demonstrated the STMP platform on 190 million records 1) Traffic Flow Modeling: The flow of traffic in a road
of smart sensor network traffic data generated by network fluctuates due to recurrent events (e.g., peak, off-
545,851 commuters and corresponding social media data peak traffic), non-recurrent events (e.g., accidents, road work),
on the arterial road network of the State of Victoria in and the characteristics of the location (e.g., schools, shopping
Australia. centers). Therefore, the traffic flow needs to be modeled as a
Rest of the paper is organized as follows. The next section spatiotemporal function.
delineates the proposed STMP platform, followed by subsec- The traffic flow T (t, s) at location s and time t are a
tions for each research contribution. STMP is demonstrated directional measure determined separately for each direction
in Section III, and Section IV concludes the paper follow- of traffic at s. This is based on the number of vehicles that
ing a discussion on implications and potential for future pass through location s towards a particular direction at time
research. interval t to t + t [13]. In legacy approaches, traffic flow is
measured from several reference points and then extrapolated
II. P ROPOSED P LATFORM to determine the traffic flow of the road network based on the
The proposed STMP platform is illustrated in Fig. 1. density flow theories of hydrodynamics [13]–[16]. However,
It consists of three layers of functionality, L1-L3. It is expan- as the arrival of more comprehensive traffic data collections
sive in terms of the heterogeneous data sources that can systems, fully data-driven methods have been recently pro-
be transformed and integrated into the data transformation posed to estimate the traffic flow [17]–[19].
layer L1 (Section II-A). The middle layer L2 (Section II-B) In addition to point-based traffic flow, the availability of
consists of the online incremental machine learning algorithm vehicle trajectory data enables the traffic flow to be esti-
which detects concept drifts and classifies into recurrent/non- mated for road segments (between any two sensor locations).
recurrent traffic events. These are passed on as input to For example, the traffic flow of road segment AB can be
smart traffic management modules in layer L3 for impact determined by T (t, A → B) and T (t, B → A), which
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

NALLAPERUMA et al.: ONLINE INCREMENTAL MACHINE LEARNING PLATFORM FOR BIG DATA-DRIVEN SMART TRAFFIC MANAGEMENT 3

denotes directional traffic flow at time t from A to B and

B to A respectively.
2) Social Media Data Modeling: Social media data are user-
generated content streams with faster information dissemina-
tion. A social media stream can be regarded as a dynamic
content stream which is a continuous stream of messages over
time (m 1 , m 2 , . . . m n ) where each message consists of a short
text and a time-stamp. As the first step, the social media stream
is sampled using a fixed time interval (t) which produces Fig. 2. Proposed data driven unsupervised concept drift detection algorithm.
batches of social media messages (B1 , B2 , . . . Bi ) where each
batch is a collection of social media messages collected for
time t. Thus, for a given batch, the message collection for algorithm periodically transfers an aggregated knowledge to
t could be defined as, the offline learning algorithm. Transfer time window and initial
t =(i+1)×t
centroids for clustering are defined by the processing triggers.
B i = {m}t =i×t (1) The transfer time window is the time taken by the offline
The granularity of the time period t (e.g., hourly, daily, algorithm to learn. Learning time is shorter when the algorithm
weekly) can be customized to suit the velocity of the social learns a previously learnt concept whereas it takes longer time
media stream as well as to match other application specific to learn a new concept. Weight vectors of the initial centroids
requirements. Each batch is then taken for pre-processing. Dur- for clustering are assigned from the offline learning and this
ing the pre-processing tasks, duplicated values and stop words allows the algorithm to quickly adapt to known concepts.
are removed. Natural Language Processing (NLP) and regular
2) Incremental Learning: Incremental learning is based
expressions are used to remove URLs, symbols, and emojis.
on Incremental Knowledge Acquisition and Self Learn-
In addition, geolocation-based filtering is carried out to filter
ing (IKASL) algorithm [23]. The algorithm continues to learn
messages relevant to the context being studied. Subsequent
from new data based on learning and generalization of the
to the aforesaid transformation, the data are utilized for the
past learning outcomes. The functionality of the learning
analysis.
layer is based on Growing Self Organizing Maps (GSOM)
[24] that generates a dynamic feature map based on the
B. Unsupervised Concept Drift Detection Based on Online growing self-organizing process. For each output from the
Incremental Machine Learning online processing, Euclidean measurement is calculated to find
Following the data transformation process, feature vectors the closest node in the map. The winning node’s weight error
that are mostly unstructured with unlabeled target variables are rate is increased and the weight vector is updated using Eq. 2,
passed on to L2. In L2, a novel online incremental machine where w j (t) , w j (t + 1) are the weight vectors of node j
learning algorithm is proposed for real-time concept drift before and after adaptation, and Nn∗ is the neighborhood of
detection and adaptation (Fig. 2). The underlying base algo- the winning node.
rithm has been applied to examine context awareness in the ⎧
Aarhus city of Denmark motor traffic dataset [20] and to study ⎪
⎪ w j (t) ,
⎪
⎨
how the driver behavior change can affect the coordination j∈/ Nn∗ (t + 1)
w j (t + 1) =   (2)
between autonomous and human-driven vehicles [21], and the ⎪
⎪ w j (t) + L R (t) x k − w j (t) ,
⎪
⎩
algorithm has been extended in this work for recurrent and j ∈ Nn∗ (t + 1)
non-recurrent concept drift capture. The algorithm consists
of two forms of learning, online and offline. Online learning Node growth occurs when this error value exceeds a
addresses volume and velocity constraint. Data-driven triggers predefined growth threshold, GT . New nodes grow out of
are used to define the processing time window and level of the node with the highest accumulated error (Ne ) and are
abstraction. The offline algorithm consists of two learning initialized to reflect the neighborhood of Ne . Upon learning for
features: a predefined number of epochs, a calibrating phase smooths
• Incremental learning to learn from evolving new con- out irregularities in recent weight adaptations.
cepts, which effectively addresses both time and space In the dynamic feature map, weight vectors of the nodes
constraints. As the learning assumes the new incoming and its neighborhood represent the current knowledge of
data are similar to previously learnt concepts, incremen- the concepts. Hence the proximity matrix, S is calculated
tal learning can be used to detect drift between the new where skm (m-th node of the k-th neighborhood) represent
concept and old concept. the proximity of n k (Hi )m corresponding to hit node, Hi . Use
• Decremental learning to forget the concepts that are not of proximity matrix ensure the furthest neighborhood node
further relevant which allows the algorithm to adapt to will have the strongest impact on the aggregate weight vector.
the new concept. All such aggregated nodes form the x-th generalized layer G x .
1) Online Learning: Online learning is handled by an online Each node in the generalization layer has the potential to grow
adaptive clustering algorithm [22] where the model is updated into a feature map. Each subsequent learning phase, L x+1 ,
as the new data are presented. The online adaptive clustering is started with a winning node from the generalization layer.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

This further learning enables incremental learning allowing the

algorithm to preserve knowledge.
3) Decremental Learning: In the implementation of decre-
mental learning, generalization nodes that do not become Fig. 3. Sample road network.
a winner for any of the inputs are identified. These nodes
represent knowledge that is no longer relevant, hence will be simulations [25], which are often suboptimal as real-world
forgotten from the map. This allows the algorithm to adapt to traffic conditions deviate from the underlying assumptions.
the new concepts more efficiently. In contrast, there have been recent data-driven supervised
4) Concept Drift Detection: In concept drift detection, learning approaches which learn impact propagation models
the distance (d) between the generalized nodes in subsequent from road occupancy data from induction loop networks and
learning layers (GNx ) is calculated as, incident data from historic incident reports [26]. However,

D 2 such an approach can only be accurately applied to locations
d(x) = G N ix − G N ix−1 (3) with a substantial amount of historic incident data.
i=1
where D is the number of dimensions of the input and x is the This work proposes an unsupervised data-driven approach
learning phase (layer). The distance d(x) represents the new to predict incident impact propagation across the arterial road
knowledge acquired by GNx . Hence, a concept drift (CD) in network based on historical data on vehicle trajectories. It is
layer x can be identified as a concept drift when d(x) is a local based on the hypothesis that when the time and location of
maximum. the incident is s and t, the propagated impact of that incident
Among CDs, non-recurrent CDs result in relatively higher to a location Si is proportionate to the flow of traffic from Si
knowledge acquisition as they were not learned before, in con- to s at time t i.e., T (t, Si → s). It assumes that when there
trast, recurrent CDs results in lower knowledge acquisition is an incident at s, it is going to block/slow down the traffic
as they were being learned before. Hence, in a non-recurrent that flows Si → s, thus impacts the traffic on Si .
CD, d(x) should be significantly higher. To detect a non- Note that, since the availability of the historical vehicle
recurrent concept drift, the prominence of the distance change trajectory data, traffic flow between two points that are even
is calculated. A non-recurrent concept drift can be identified separated by several other junctions can be accurately esti-
when there is a significant difference in distance. mated.
Above mentions characteristics of CDs are being captured The relative incident impact to the location Si is determined
by f C D in Eq. 4 to identify whether a layer x represent a based on the traffic flow Si → s relative to the aggregated
non-recurrent CD, recurrent CD or neither. traffic flow goes through S at time t.
Let the n road segments connecting to S be {s1 →
f C D (x) s, . . . , sn → s}, the relative incident impact PI (t, Si → s)
⎧
⎪
⎪ if d(x) > d(x − 1) and, to the location Si from an incident in s at time t is defined as,
⎪
⎪
⎪
⎪ non-recurrent CD d(x) > d(x + 1) and, T (t, Si → s)
⎪
⎪ PI (t, Si → s) =   (5)
⎪
⎪ x n
T t, S j → s
⎪
⎪ j =1
⎪
⎪ d( j )
⎪
⎪
⎪
⎪ j =1 The relative incident impact can be determined across the
⎨ |d(x) − d(x + 1)| > road network for the reference points (e.g., junctions) that are
= x (4)
⎪
⎪ within a designated distance from the incident location. For
⎪
⎪
⎪
⎪ if d(x) > d(x − 1) and, example, let A, B, C, and D be the junctions of the road-
⎪
⎪
recurrent CD
⎪
⎪ network that connects to the incident location s as illustrated
⎪
⎪ d(x) > d(x + 1)
⎪
⎪ in Fig. 3, then the relative incident impacton each junction
⎪
⎪
⎪
⎪ will be PI (t, A → s), PI (t, B → s), PI (t, C → s), and
⎩no CD otherwise PI (t, D → s).
The relative incident impact decreases as the junctions are
further from the incident location. Junctions with a significant
C. Impact Propagation Estimation relative incident impact as well as the road network that
connects those junctions can be identified as the impacted
The previously described layer L2 captures concept drifts critical road segments of the network. Note that, an important
due to both recurrent and non-recurrent traffic incidents. advantage of this approach is that, with the availability of
Such incidents not only impact the location of occurrence historical data, it takes into account the temporality of the
but propagates through the road network. While the impact traffic flow when determining the relative incident impact,
propagation of recurrent incidents is known and accounted as the impacted road segments may vary depending on the
in traffic planning, the impact prediction on non-recurrent usual traffic conditions at the time t of the incident.
incidents in near-real times is crucial to remediate its impact.
Therefore, it is important to predict the impacted road segment
and the proportion of impact propagated. D. Traffic Forecasting
Due to the lack of historical traffic data, such impact pre- Critical road segments pose highly unpredictable traffic con-
dictions are often carried-out using mathematical model based ditions and they can be determined using impact propagation
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

NALLAPERUMA et al.: ONLINE INCREMENTAL MACHINE LEARNING PLATFORM FOR BIG DATA-DRIVEN SMART TRAFFIC MANAGEMENT 5

estimation. Providing suitable traffic control mechanisms for

such critical road segments is an important consideration in
ITS to enable a smooth traffic flow over the arterial road
network. In order to develop a suitable control mechanism, it is
essential to be able to forecast the traffic flow of the critical
road segments. In this section, we propose an approach for
traffic prediction based on the current traffic flow condition of
the surrounding using a Deep Neural Networks (DNN) model.
Previous efforts in road traffic forecasting range from traffic
flow modeling to recent advancements in data-driven models.
With the recent advancement in deep learning paradigm, Fig. 4. Deep reinforcement learning for adaptive traffic control.
the ITS research community has made efforts to extend the
ITS capabilities with deep learned modeling [17], [27], [28].
determine control actions based on recorded similar situations
In the proposed STMP platform, we introduce a new DNN
(from historical data or off-line simulation) or based on simple
based on LSTM [29] and enrich its prediction capabilities by
“if-then” rules. However, these techniques do not have a
incorporating the surrounding traffic flow of influential road
learning mechanism to automatically update their model. Also,
segments.
they do not have a strong inference power to deal with unseen
Standard DNN architectures based on LSTM attempt to pre-
situations.
dict time series data singularly considering a single time scale,
DNN combined with reinforcement learning usually
where time series data such as road traffic flow often have a
referred to as Deep Reinforcement Learning (DRL) [34], is a
temporal hierarchy with information spread out over multiple
generic and flexible way to develop intelligent and adaptive
time scales. With the introduction of cascade-connected layers
traffic control systems. Fig. 4 shows a DRL method for intel-
of LSTM networks, the abstraction of input observations
ligent traffic control in the proposed STMP. The goal of DRL
over multiple time scales can be incrementally extracted to
is to select the most suitable control program which decides
incorporate in modeling the input pattern [30], [31]. Therefore,
the duration for each time phase for each traffic light in the
due to the complex and stochastic nature of the road traffic,
network. In this method, the area of interest is first selected and
it is advantageous to model complex dependencies between the
the corresponding road infrastructure of this area is obtained
time series input from the traffic flow. This has been recently
using the data from OpenStreetMap (OSM) [35]. Because it is
attempted in [32], where correlations of the traffic flow of
very costly to test the algorithm on real environments, a virtual
targeted road segment and that of its upstream and downstream
environment is developed (via simulation) as a mean to vali-
road segments are considered for the prediction. However, it is
date the effectiveness of the evolved controller. In our imple-
imperative to capture the spatial correlation of the arterial road
mentation, we feed the road information and the traffic data
network including the connected road segments that affect the
into TraCI-SUMO [36] to generate simulation scenarios. Each
traffic flow of the targeted road segment.
simulation run is for two hours and the information (s, a, r, s  )
We designed a DNN based on hierarchically stacked three
layered LSTM network architecture that consists of 256,   the current state (s), action (a), reward (r ), and
of
a
new state
128 and 64 LSTM cells respectively, accommodating deeper s are recorded every 10 minutes in which s −→ r, s  .
abstraction of temporal hierarchy to be trained by the pre- The inputs for the DRL algorithm are the numbers of
diction model. The proposed DNN architecture captures the vehicles traveling through all lanes in the network (each road
correlation patterns of the targeted road segment and the segment can have many lanes). Based on the inputs, DRL will
surrounding road segments that impact the traffic flow of calculate the expected values for each action (e.g., a control
the targeted road segment, which was identified through the program) and the action with the highest expected values will
impact propagation analysis. This proposed DNN architec- be selected and applied to the traffic network. The proposed
ture has enabled the prediction model to incorporate longer DRL method has two fully connected hidden layers. The two
duration of current and previous traffic condition in order to hidden layers use Rectified Linear Unit (ReLU) as activators.
forecast the future traffic condition. The dimension of the output layer is the number of control
programs M (i.e the action a ∈ {C1 , , . . . , C M ) that we want
to select and linear activators are employed. At the beginning
E. Intelligent Traffic Control of each time interval, the most suitable control program is
Real-time concept drift detection and traffic forecasting are selected by,
useful inputs for intelligent traffic control to optimize network Cr r and < 
performance. Conventional control approaches to intelligent at =  (6)
argmax a Q(s , a) other wi se
traffic control such as static feedback control (SFC) and opti-
mal control and model predictive control (MPC) are developed where r and is a random number from 0 to 1,  is exploration
based on many assumptions and idealistic models [33]. As a factor which is decayed over time, Cr is the random control
result, these approaches have trouble coping with the dynamics program. In the proposed DRL algorithm, we apply Q-learning
of the traffic networks. Some traditional AI techniques such in which DNN is used to represent the Q function [39].
as case-based reasoning and rule-based systems are used to To improve the stability of Q function in the training process,
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

Fig. 5. High-level process of the proposed emotion analysis during an event.

we use the double learning and experience replay:

 
Q (s, a) = r + γ Q̃ s , argmax a Q s , a (7) Fig. 6. Bluetooth traffic monitoring system (BTMS).

where Q(s, a) is the primary Q network, Q̃(s, a) is the target the event. Twitter API is queried with re and te to collect the
network, and γ is the discount factor [50]. This trick has been relevant georeferenced tweets of the event. Note that, there are
shown to significantly improve the effectiveness of Deep Q- advanced methods of localizing the non-georeferenced tweets
learning. [41], however, such approaches are beyond the scope of this
The reward obtained for each time interval (e.g., 10 minutes) work. The radius re is set based on the impact propagation
is calculated as, analysis of the accident by drawing a bounding circle covering
the road segments with significant relative incident impact.
Wt − Wt −1 i f (Wt − Wt −1 ) < 0 The emotion extraction is based on Ekman’s model of
rt = (8)
−P ∗ (W t − Wt −1 ) other wi se emotions where it presents six basic emotions (“Joy”, “Anger”,
“Surprise”, “Disgust”, “Fear”, “Sadness”) which are known as
where Wt is the average waiting of vehicles traveling on the
universal emotions [42]. An emotion vocabulary was formed
traffic network in the time interval t, and P is the penalty for
by creating a dictionary of emotional expression per each
slowing down the traffic. Based on the recorded states, actions,
emotion category. Emotion expressions were obtained from
and rewards, DRL will update its parameters (i.e., connection
the LIWC dictionary [43] and other published research studies
weights) to optimize its decisions by using back-propagation.
[44]. Text mining and NLP techniques were used to extract
In this method, the traffic controller can be triggered via time-
the emotion expressions from social media contents. Having
based (e.g., every five minutes) or event-based mechanisms
identified the emotions of a particular tweet, the weight of
(e.g., by anomaly detection).
each emotion (we ) was calculated by taking the average
of emotional expressions present in each tweet in order to
F. Social Media Based Commuter Emotion Analysis determine the strength of the emotions expressed. Afterwards,
In the proposed architecture, once an event is detected using the accumulated emotion intensity level was calculated and
the concept drift module (Section II-B), the emotion analysis is used to determine if the emotional behavior was significant.
used to capture the emotional behaviors of commuters from the Emotion intensity (It ) can be defined based on the aggregation
live social media stream. The importance and the applicability of emotions over a specified time period, commencing at
of social media in transport sector have been stated in [38] time t, where n denotes the number of tweets and m denotes
where it mentions that social media data from commuters the number of emotions in the category, as expressed below.
potentially enrich other data sources by providing a different n m
view of the commuters and their behaviors which are not cap- It,t +t = wi j (9)
tured from other traffic data sources. This analysis is conducted i=0 j =0
as a supporting module to investigate commuter behaviors
during detected events, as social media platforms such as III. E XPERIMENTS
Twitter are widely used communication platforms to notify This section demonstrates the proposed STMP platform
about significant events such as accidents, traffic congestions using real traffic data from the arterial road network of the
and roadblocks [10]–[12] and in several studies, sentiment State of Victoria, Australia.
analysis using Twitter has been carried out as a measure of The information on traffic has been acquired from the
commuter satisfaction and to observe commuter behavior and Bluetooth Traffic Monitoring System (BTMS) that is used to
opinion patterns in order to improve the transportation services monitor the road traffic of arterial roads in Victoria. BTMS is
[39], [40]. This work extends the traditional sentiment analysis a type of automatic vehicle detectors that is used to estimate
by extracting deeper emotion from twitter, thus enabling a travel times in a road network [45], [46]. For a comprehensive
more granular analysis of user opinion and feeling. Fig. 5 is understanding of the Bluetooth sites refer [47].
a high-level illustration of the process. As shown in Fig. 6, BTMS consists of a network of
Once an event is identified by the concept drift detection, Bluetooth traffic scanners that are placed in the junctions of
the Twitter data stream is analyzed to collect tweets that are arterial roads. These Bluetooth scanners capture the Bluetooth
relevant to the event. Such tweets are defined as originating devices that transit the scanning zone, which are either Blue-
within a radius re of the event for a time interval te since tooth enabled vehicle stereo systems or the mobile devices
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

NALLAPERUMA et al.: ONLINE INCREMENTAL MACHINE LEARNING PLATFORM FOR BIG DATA-DRIVEN SMART TRAFFIC MANAGEMENT 7

Fig. 7. Schematic of the road network in the area of interest selected for
traffic analysis. Fig. 8. Recurrent and non-recurrent concept drift detection.

of the occupants. The scanners capture the unique electronic

in A at time t and subsequently detected in B, which can be
identifier (MAC address) of the Bluetooth devices that tran-
denoted as,
sit the scanning area and the transit timestamp [46]. Each  t +t
Bluetooth scanner periodically transmits those records to a
T (t, A → B) = I(v, A → B, t)dt (10)
central database. Since the electronic identifier is unique to t ∀v
each Bluetooth devices, its travel path can be traced across the
network of Bluetooth scanners placed in the road network. where t is the sampling interval for the traffic flow which
For this study, the dataset was obtained from Victoria can be adjusted to obtain the required granularity of the traffic
road authority (VicRoads) and comprises all vehicle records flow. I (v, A → B, t) is an indicator function which is active
for October 2017. This dataset consists of approximately if the vehicle v is first detected at A at time t and subsequently
190 million vehicle records obtained from 1,408 Bluetooth detected at B within a time threshold τ . It can be defined as,
scanners placed at the junctions of arterial roads. It contains I(v, A → B, t)
records from 545,851 unique MAC-IDs, which is assumed to ⎧
⎪
⎪ 1 if ∃ (v, A, t) ∈ D and ∃ (v, B, t  ) ∈ D,
be unique vehicles. ⎪
⎨
The capability of the proposed platform is demonstrated where t < t  ≤ t + τ
= (11)
by analyzing the traffic behavior around a key Shopping ⎪
⎪
⎪
⎩
Centre (SC), which is often a volatile traffic region in Vic- 0 otherwise
toria. It is one of the largest stand-alone shopping centers
in Australia with over 20 million annual visitor turnarounds. where τ is the trip threshold which is set enough for a single
Thus, it accounts for a large traffic footprint in surrounding trip in the road segment. The idea of setting this threshold
arterial roads. In addition, it is sandwiched between two large is to filter out noisy trips [48] such as pedestrians as well as
freeways (Princess Hwy and Monash Fwy) which are the vehicles that make a stop inside the segment.
key freeways that connect Melbourne Metropolitan Area to
Southeast Victorian suburbs. The combination of the above- B. Unsupervised Concept Drift Detection
mentioned factors yields unique and highly volatile traffic Transformed data were emulated as a real data stream to
patterns in the selected region. Moreover, the surrounding represent VicRoads live feed using an external application.
arterial roads are often operated in near saturation, thus, any This data stream is presented to the unsupervised concept drift
non-recurrent incident can result in significant congestion and detection algorithm explained in Section II-B. Fig. 8 illustrates
its impact often propagated significantly across the arterial the concept drifts detected in the selected traffic area. The
road network. Fig. 7 presents a schematic of the selected SC x-axis denotes execution timestamps of incremental learning
and the surrounding arterial roads. and distance measure calculations from Eq. 3 of the algorithm
are denoted on the y-axis. Fig. 8 demonstrated the recurrent
and non-recurrent concept drifts identified by Eq. 4 of the
algorithm.
A. Traffic Data Transformation The algorithm identified three recurrent concept changes
Each record in the dataset D = {(v, s, t)} can be denoted by throughout the data stream. These recurrent concept changes
(v, s, t) where v is the vehicle denoted by the MAC-ID, s is relate to the traffic flow changes due to longer shopping hours,
the location denoted by the site id and t is the time denoted by weekdays to weekend traffic flow and vice versa. A summary
the timestamp. The traffic flow of road segments was derived of the recurrent traffic flow changes is denoted in Table I.
from these traffic records. Moreover, the algorithm identified two non-recurrent con-
Based on the definition of traffic flow in Section II-A, cept drifts at execution timestamps [t22] and [t32], of which
the traffic flow T (t, A → B) of road segment AB can the concept drift at [t32] is employed to demonstrate
be defined as the number of vehicles that are first detected the functionality of L3 of the proposed STMP platform.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE I TABLE II
S UMMARY OF R ECURRENT C ONCEPT D RIFT D ETECTION I NCIDENT I MPACT P ROPAGATION FOR AN I NCIDENT AT
WARRIGAL R D -WAVERLEY R D I NTERSECTION

of detected concept drifts, 92.3% was in-line with the ground

truth validated by domain experts.

Fig. 9. Evidence for non-recurrent concept drift detection, left: usual traffic C. Impact Propagation Analysis
congestion is reduced in Warrigal Rd due to an accident in Monash Fwy, right:
accident on Warrigal Rd which has created traffic congestion on Warrigal Rd In Section II-D the impact propagation across the road
and Monash Fwy. network due to a non-recurrent incident was derived based on
the traffic flow towards that location from other road segments.
This selected incident has occurred at the Warrigal Rd-Waverly This capability is demonstrated based on the Warrigal Rd -
Rd intersection (see Fig. 7). Waverley Rd intersection (see Fig. 7) where the traffic incident
The tweets relevant to this incident were collected using was identified in the previous section (non-recurrent concept
the technique delineated in Section II-F, in which the two drift at timestamps [t32] in Fig. 8). Based on the Eq. 5
parameters are set as follows. The radius of the event (re ) (Sec. II-C) the relative incident impact was derived for the
is set to 3.7km which is the distance to the furthest location road segments that are close to the incident location.
(Warrigal Rd - Burwood Hwy intersection) with a significant Table II presents the road segments with a significant impact
relative incident impact identified by the impact propagation from the incident with its respective relative incident impact
analysis in the next section (see Table II). The event duration determined for the weekday 8 am traffic and weekday 8 pm
(te ) is set experimentally to 1 hour to be suitable for both major traffic. Note that two time-of-the-day values were selected to
and minor incidents. From the collected tweets, it was found compare and contrast differences in incident impact propaga-
that these non-recurrent traffic flow changes had occurred due tion due to different traffic behaviors.
to an accident (Fig. 9). Tweets in Fig. 9 (right) shows that As shown in Table II, the incident impact is mainly prop-
there was a communication gap of more than 45 minutes to agated along the Warrigal Rd in both directions (south and
gather information on the situation. Due to the data-driven northbound), compared to relatively less impact on Waverley
nature of the algorithm, concept drifts on traffic flow can be Rd (east and westbound). This is because Warrigal Rd carries
detected almost real time. Being able to provide a real-time a high traffic flow as it has entrance to the freeway. At 8 am,
notification on non-recurrent traffic flow changes will allow the highest impact on Warrigal Rd is for Batesford Rd to
better communications and effective optimizations. Waverley Rd traffic flow, as that traffic is bound to cross the
The proposed algorithm has the capability of detecting Warrigal Rd-Waverley Rd intersection and enter the Monash
concept drifts with different granularities such as daily or Fwy towards the City. In contrast, impact at 8 pm is highest
hourly. This can be achieved by changing the growth threshold on Monash Fwy to Waverley Rd, which consists of the traffic
and distance measure. With the demonstrated granularity, out coming from the city and exiting Monash Fwy to Warrigal Rd.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

NALLAPERUMA et al.: ONLINE INCREMENTAL MACHINE LEARNING PLATFORM FOR BIG DATA-DRIVEN SMART TRAFFIC MANAGEMENT 9

TABLE III
E VALUATION OF A CCURACY BASED ON THE T IME -L AG
S ELECTED FOR P REDICTION

Fig. 10. Traffic flow prediction performance comparison on the time period
Oct. 25 to Oct. 31, 2017.
The incident impact on Waverley Rd is relatively less, which
mostly impacts the eastbound traffic while the westbound
traffic is only impacted at 8 am. The DNN trained upon input of 3 time-lags (i.e., 45 minutes
prior traffic flow data) was selected for the examination
D. Traffic Forecasting in Fig. 10 to compare with the ground truth traffic flow
data. Based on the graphical representation of the prediction
The primary objective of the traffic forecasting experiment
performance, it is clear that the DNN model can successfully
is to evaluate the proposed DNN’s fitness in short-term traffic
model traffic flow with normal fluctuation, however, there are
flow forecasting. Based on the impact propagation analysis
limitations in predicting traffic flow that has severe fluctuations
conducted in the previous step, it was identified that the
which occur at a higher frequency. This limitation is expected
traffic flow of the road segment on Warrigal Rd (annotated
to be addressed when additional traffic data (e.g. roadworks,
as B to C in Fig. 7), in which heavy traffic congestion befall
incidents, events) is used for training the DNN model.
in frequent time periods, is critically impacted by the road
segments; A to B, C to D and C to E. Therefore, in this
experiment we have evaluated the effect of the identified road E. Intelligent Traffic Control
segments with respect to the traffic flow prediction of road To illustrate the effectiveness of DRL for adaptive traffic
segment B to C. In the experiment, the data horizon is taken control, we apply it to the traffic network around the selected
as 15 minutes, where the data extension was 24 days (1st to shopping center (Fig. 7) to control the traffic light phases.
24th October 2017). The selected area has 1805 lanes so the input dimension (i.e.,
Based on the Eq. 10, traffic flow is determined for the the state S) of the proposed DNN model is 1805. In our
selected road segments using 15-minute sampling intervals experiments, we perform 500 simulation runs (or epochs)
and utilized as the input sequences of the DNN. The data to learn the DNN model. Five master control programs are
was divided into training and testing subsets, in which the considered, i.e., M = 5. The first program is the default
first 24 days’ data were utilized for training of the model and program determined by the SUMO’s generator [36]. The sec-
the remaining 7 days’ data was utilized for testing the model ond program set all phase durations to 60 seconds. The
performance. To validate the effectiveness of the algorithm, third program doubles the phase durations for phases with
the model performance was measured by means of Mean durations greater than 40 seconds. The fourth program double
Squared Error (MSE) and Mean Absolute Error (MAE). the phase durations for all phases. The last program set the
Table III demonstrates the prediction performance with phase durations for all phases to 30 seconds. The parameters
respect to the measures selected. We evaluate the prediction of the proposed DRL algorithm are: discount factor γ = 0.95,
performance based on the multiple time-lags, which have initial exploration factor  = 1, and decay factor = 0.99.
presented to the DNN as input, ranging from 15 minutes to The network weights are updated using Adam optimizer with
2 hours. The time lag with the highest performance is marked the learning rate of 0.001 at the end of each simulation run
in bold. The findings reveal that with the time lag increases using the experience replayed from the agent’s memory. For
the model performance. However, after the time lag of 3, the reward function in Eq. 8, we use the penalty P = 5.
the model performance begins to decrease, and it starts to The results of DRL is shown in Fig. 11. It is easy to see
drastically reduce after the time lag of 5. Based on the internals that the average waiting time is roughly reduced by half after
of the algorithms and the findings, it is evident that the DNN the 500 epochs. It demonstrates the ability of the proposed
with three LSTM layers can successfully model dependencies DRL to learn effective control decisions even for a very large
between time series input (i.e., traffic flow data as input) for traffic network.
a time lag of 3. However, the current model is not capable The learning mechanism of the proposed DRL can be
enough to model complex dependencies over 3-time lags, adapted to different networks and other control problems
hence, the decline of the performance of the prediction model. in intelligent traffic systems. Instead of periodically rese-
Fig. 10 presents the traffic flow prediction performance lect the control program, we can modify the algorithm to
comparison on the time period Oct. 25 to Oct. 31, 2017, adjust the control program based on the signal from con-
in order to examine the prediction performance visually. cept drift or abnormally detection. Compared to conventional
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE IV
H IGHER E MOTION I NTENSITY T IMELINE

We found out that the association between emotional intensity

and congestion is statistically significant ( p < 0.05).
Depending on the required sensitivity to detect emotion
events, the threshold value could be changed, and human mon-
itoring could be used to validate the significant events captured
Fig. 11. Training performance of the proposed DRL. by the algorithm. Incorporating real-time monitoring of crowd-
sourced data on social media would enable authorities to take
proactive decisions and gain a better understanding of the
user’s perception.
This section demonstrated each module of the proposed
multi-layered STMP platform using a dataset of 190 million
records of smart sensor network traffic data generated by
545,851 commuters and corresponding social media data on
the arterial road network of Victoria, Australia. Data received
from heterogeneous sources is integrated and transformed in
L1, in L2, concept drifts, recurrent and non-recurrent events
are identified by the online incremental machine learning
Fig. 12. Emotion fluctuations over time (hourly). algorithm. Identified events are fed into the smart traffic
management modules in L3.
control methods, DRL is more suitable for high-dimensional IV. C ONCLUSION
streaming data because high-level features can be automati- This paper proposed a new smart traffic management plat-
cally extracted and DRL is data-driven and does not rely on form to capture dynamic patterns from traffic data streams
any explicit model. and to integrate AI modules for real-time traffic analysis and
adaptive traffic control. The main benefit of the proposed
platform is that its AI modules are designed to efficiently
F. Control Social Media Based Commuter Emotion Analysis cope with the key challenges of future transportation systems
Based on the proposed model in Section II-F, an experiment where IoT devices are widely adopted, analysis and control
was carried out to detect significant emotional behaviors of technologies must be more responsive and self-evolved, and
users who post about road conditions on social media. To con- social behaviors need to be taken into consideration. Moreover,
duct the experiment, Twitter data was collected on a day where the platform also overcomes the limitations of current algo-
a service disruption had occurred, which has consequently led rithms and technologies which rely heavily on limited labeled
to an increase in road traffic. Widely used hashtags (#VicTraf- data and strict assumptions about data and traffic behaviors.
fic, #VicRoads, #MelbTraffic, #GettrafficVIC) related to trans- To evaluate the feasibility and effectiveness of the proposed
portation in the selected region were used to filter the relevant platform, we have conducted a series of experiments based
social media content. Following Fig. 12 illustrates the emotion on real-time Bluetooth sensor network data and social media
expression variations overtime on an hourly scale on this day. data from the arterial road network in Victoria, Australia. The
Emotion intensity at each time frame was calculated and experimental results show that the platform can successfully
was compared with a 3-point moving average to determine if and in a timely manner detect recurrent and non-recurrent
the change in the behavior was significant or not. This will events, and those results are further validated using the insights
enable to identify abrupt peaks with high emotion intensities. automatically captured from social media. The experiments
Based on the average, Table IV shows the significant times also show that impact propagation and traffic flow prediction
which recorded a higher emotion intensity on Twitter. It can modules can efficiently predict short-term impacts of the
be seen that traffic peak hours demonstrated a higher nega- events. Finally, the simulation of a large-scale traffic network
tive emotion intensity when compared to other times of the shows that the proposed deep reinforcement learning can learn
day. We further explored the relationship between emotional to improve traffic signal control decision based on many real-
intensity and congestion using a Granger Causality test [49]. time data streams.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

NALLAPERUMA et al.: ONLINE INCREMENTAL MACHINE LEARNING PLATFORM FOR BIG DATA-DRIVEN SMART TRAFFIC MANAGEMENT 11

We acknowledge a number of potential areas for improve- [18] R. Yu, Y. Li, C. Shahabi, U. Demiryurek, and Y. Liu, “Deep learning:
ment. Further research directions involve fusing data from A generic approach for extreme condition traffic forecasting,” in Proc.
SIAM Int. Conf. Data Mining, Jun. 2017, pp. 777–785.
heterogeneous data sources such as security cameras, weather [19] G. Michau, A. Nantes, A. Bhaskar, E. Chung, P. Abry, and P. Borgnat,
information, and other transportation-related data sources. “Bluetooth data in an urban context: Retrieving vehicle trajectories,”
Also, the interpretability of AI modules, especially ones based IEEE Trans. Intell. Transp. Syst., vol. 18, no. 9, pp. 2377–2386,
Sep. 2017.
on complicated techniques such as deep neural networks, are [20] D. Nallaperuma, D. D. Silva, D. Alahakoon, and X. Yu, “A cognitive
worth investigating in the future to gain the acceptance of the data stream mining technique for context-aware IoT systems,” in Proc.
platforms. 43rd Annu. Conf. IEEE Ind. Electron. Soc., Nov. 2017, pp. 4777–4782.
[21] D. Nallaperuma, D. D. Silva, D. Alahakoon, and X. Yu, “Intelligent
ACKNOWLEDGMENT detection of driver behavior changes for effective coordination between
autonomous and human driven vehicles,” in Proc. 43rd Annu. Conf.
Authors would like to thank VicRoads–Victorian Road IEEE Ind. Electron. Soc., Oct. 2018, pp. 3120–3125.
Authority, Australia for their consultations in domain knowl- [22] A. Câmpan and G. Şerban, “Adaptive clustering algorithms,” in Proc.
Adv. Artif. Intell., Oct. 2006, pp. 407–418.
edge and providing us with real road traffic data. Authors [23] D. D. Silva and D. Alahakoon, “Incremental knowledge acquisition and
would also like to acknowledge the in-kind support from the self learning from text,” in Proc. Int. Joint Conf. Neural Netw. (IJCNN),
Data to Decisions Cooperative Research Centre (D2D CRC) Jul. 2010, pp. 1–8.
as part of their analytics and decision support Program. [24] D. Alahakoon, S. K. Halgamuge, and B. Srinivasan, “Dynamic self-
organizing maps with controlled growth for knowledge discovery,” IEEE
Trans. Neural Netw., vol. 11, no. 3, pp. 601–614, May 2000.
R EFERENCES
[25] C. S. Wirasinghe, “Determination of traffic delays from shock-wave
[1] I. Lana, J. D. Ser, M. Velez, and E. I. Vlahogianni, “Road traffic analysis,” Transp. Res., vol. 12, no. 5, pp. 343–348, Oct. 1978.
forecasting: Recent advances and new challenges,” IEEE Intell. Transp. [26] B. Pan, U. Demiryurek, C. Shahabi, and C. Gupta, “Forecasting spa-
Syst. Mag., vol. 10, no. 2, pp. 93–109, Apr. 2018. tiotemporal impact of traffic incidents on road networks,” in Proc. IEEE
[2] E. I. Vlahogianni, M. G. Karlaftis, and J. C. Golias, “Short-term 13th Int. Conf. Data Mining, Dec. 2013, pp. 587–596.
traffic forecasting: Where we are and where we’re going,” Transp. Res. [27] W. Huang, G. Song, H. Hong, and K. Xie, “Deep architecture for traffic
C, Emerg. Technol., vol. 43, pp. 3–19, Jun. 2014. flow prediction: Deep belief networks with multitask learning,” IEEE
[3] J. Gama, I. Žliobaite, A. Bifet, M. Pechenizkiy, and A. Bouchachia, Trans Intell. Transp. Syst., vol. 15, no. 5, pp. 2191–2201, Oct. 2014.
“A survey on concept drift adaptation,” ACM Comput. Surv., vol. 46, [28] X. Ma, Z. Tao, Y. Wang, H. Yu, and Y. Wang, “Long short-term memory
no. 4, p. 44, Apr. 2014. neural network for traffic speed prediction using remote microwave
[4] J. L. Lobo, J. D. Ser, M. N. Bilbao, C. Perfecto, and S. Salcedo-Sanz, sensor data,” Transp. Res. Part C, Emerg. Technol., vol. 54, pp. 187–197,
“DRED: An evolutionary diversity generation method for concept drift May 2015.
adaptation in online learning environments,” Appl. Soft Comput., vol. 68, [29] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural
pp. 693–709, Jul. 2018. Comput., vol. 9, no. 8, pp. 1–32, 1997.
[5] M. M. Masud et al., “Addressing concept-evolution in concept-drifting [30] M. Hermans and B. Schrauwen, “Training and analysing deep recurrent
data streams,” in Proc. IEEE Int. Conf. Data Mining, Dec. 2010, neural networks,” in Proc. Adv. Neural Inf. Process. Syst., Jun. 2013,
pp. 929–934. pp. 190–198.
[6] A. Bifet, G. Holmes, B. Pfahringer, R. Kirkby, and R. Gavaldà, [31] A. Graves, A.-R. Mohamed, and G. Hinton, “Speech recognition with
“New ensemble methods for evolving data streams,” in Proc. 15th deep recurrent neural networks,” in Proc. IEEE Int. Conf. Acoust.,
ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Jun. 2009, Speech Signal Process., May 2013, pp. 6645–6649.
pp. 139–148. [32] Z. Zhao et al., “LSTM network: A deep learning approach for short-
[7] M. Sayed-Mouchaweh, “Learning in dynamic environments,” in Learn- term traffic forecast,” IET Intell. Transp. Syst., vol. 11, no. 2, pp. 68–75,
ing from Data Streams in Dynamic Environments, Springer, New York, 2017.
NY, USA: 2016. [33] L. Baskar, B. De Schutter, J. Hellendoorn, and Z. Papp, “Traffic control
[8] M. A. Maloof and R. S. Michalski, “Incremental learning with par- and intelligent vehicle highway systems: A survey,” IET Intell. Transp.
tial instance memory,” Artif. Intell., vol. 154, nos. 1–2, pp. 95–126, Syst., vol. 5, no. 1, pp. 38–52, Mar. 2011.
Apr. 2004. [34] V. Mnih et al., “Human-level control through deep reinforcement learn-
[9] K. N. Qureshi and A. H. Abdullah, “A survey on intelligent transporta- ing,” Nature, vol. 518, pp. 529–533, Feb. 2015.
tion systems,” Middle-East J. Sci. Res., vol. 15, no. 5, pp. 629–642,
[35] M. Haklay and P. Weber, “OpenStreetMap: User-generated street maps,”
2013.
IEEE Pervasive Comput., vol. 7, no. 4, pp. 12–18, Oct. 2008.
[10] C. Khatri, “Real-time road traffic information detection through
social media,” Jan. 2018, arXiv:1801.05088. [Online]. Available: [36] A. Wegener, M. Piórkowski, M. Raya, H. Hellbrück, S. Fischer, and
https://arxiv.org/abs/1801.05088 J.-P. Hubaux, “TraCI: An interface for coupling road traffic and network
[11] D. Wang, A. Al-Rubaie, J. Davies, and S. S. Clarke, “Real time road simulators,” in Proc. 11th Commun. Netw. Simulation Symp., Apr. 2008,
traffic monitoring alert based on incremental learning from tweets,” pp. 155–163.
in Proc. IEEE Symp. Evolving Auton. Learn. Syst. (EALS), Dec. 2014, [37] H. V. Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with
pp. 50–57. double Q-Learning,” Sep. 2015, arXiv:1509.06461. [Online]. Available:
[12] P.-T. Chen, F. Chen, and Z. Qian, “Road traffic congestion monitoring https://arxiv.org/abs/1509.06461
in social media with hinge-loss Markov random fields,” in Proc. IEEE [38] S. M. Grant-Muller, A. Gal-Tzur, E. Minkov, S. Nocera, T. Kuflik,
Int. Conf. Data Mining, Dec. 2014, pp. 80–89. and I. Shoor, “Enhancing transport data collection through social
[13] M. J. Lighthill and G. B. Whitham, “On kinematic waves. II. A media sources: Methods, challenges and opportunities for textual data,”
theory of traffic flow on long crowded roads,” Proc. Roy. Soc. London, IET Intell. Transp. Syst., vol. 9, no. 4, pp. 407–417, Nov. 2014.
Ser. A, Math. Phys. Sci., vol. 229, pp. 317–345, May 1955. [39] F. Ali, D. Kwak, P. Khan, S. M. R. Islam, K. H. Kim, and K. S. Kwak,
[14] C. F. Daganzo, “The cell transmission model. Part I: A simple dynamic “Fuzzy ontology-based sentiment analysis of transportation and city
representation of highway traffic,” Transp. Res. Part B Methodol., feature reviews for safe traveling,” Transp. Res. C, Emerg. Technol.,
vol. 28, no. 4, pp. 269–287, 1994. vol. 77, pp. 33–48, Apr. 2017.
[15] P. I. Richards, “Shock waves on the highway,” Oper. Res., vol. 4, no. 1, [40] F. C. Pereira, A. L. C. Bazzan, and M. Ben-Akiva, “The role of context
pp. 42–51, 1956. in transport prediction,” IEEE Intell. Syst., vol. 29, no. 1, pp. 76–80,
[16] A. Aw, A. Klar, M. Rascle, and T. Materne, “Derivation of contin- Jan. 2014.
uum traffic flow models from microscopic follow-the-leader models,” [41] H. Abdelhaq, C. Sengstock, and M. Gertz, “EvenTweet: Online localized
SIAM J. Appl. Math., vol. 63, no. 1, pp. 259–278, 2002. event detection from twitter,” Proc. VLDB Endowment, vol. 6, no. 12,
[17] Y. Lv, Y. Duan, W. Kang, Z. Li, and F.-Y. Wang, “Traffic flow prediction pp. 1326–1329, Aug. 2013.
with big data: A deep learning approach,” IEEE Trans. Intell. Transp. [42] P. Ekman, “An argument for basic emotions,” Cognit. Emotion, vol. 6,
Syst., vol. 16, no. 2, pp. 865–873, Apr. 2015. nos. 3–4, pp. 169–200, 1992.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

[43] J. H. Kahn, R. M. Tobin, A. E. Massey, and J. A. Anderson, “Measuring Su Nguyen received the Ph.D. degree in artificial
emotional expression with the linguistic inquiry and word count,” intelligence and data analytics from the Victoria
Amer. J. Psychol., vol. 120, no. 2, pp. 263–286, Jul. 2007. University of Wellington, New Zealand, in 2013.
[44] R. Valitutti, “WordNet-affect: An affective extension of WordNet,” He is currently a Senior Research Fellow with the
in Proc. 4th Int. Conf. Lang. Resour. Eval., May 2004, pp. 1083–1086. Centre for Research in Data Analytics and Cogni-
[45] U. Mori, A. Mendiburu, M. Álvarez, and J. A. Lozano, “A review of tion, La Trobe University, Australia. His primary
travel time estimation and forecasting for advanced traveller information research interests include computational intelligence,
systems,” Transportmetrica A, Transp. Sci., vol. 11, no. 2, pp. 119–157, optimization, data analytics, large-scale simulation,
2015. and their applications in operations management and
[46] A. Bhaskar and E. Chung, “Fundamental understanding on the use of social media.
Bluetooth scanner as a complementary transport data,” Transp. Res. C,
Emerg. Technol., vol. 37, pp. 42–72, Dec. 2013.
[47] VicRoads. Bluetooth Sites–VicRoads Open Data. Accessed:
Feb. 26, 2019. [Online]. Available: https://vicroadsopendata-
vicroadsmaps.opendata.arcgis.com/datasets/48fd4d7e1127453ea
5f9bdc757ab00e7_0
[48] A. Nantes, M. P. Miska, A. Bhaskar, and E. Chung, “Noisy Bluetooth
traffic data?” Road Transp. Res., A J. Austral. New Zealand Res. Pract., Thimal Kempitiya received the bachelor’s degree
vol. 23, no. 1, pp. 33–43, Mar. 2014. (Hons.) from the Department of Computer Sci-
[49] C. Hiemstra and J. D. Jones, “Testing for linear and nonlinear Granger ence and Engineering, University of Moratuwa, Sri
causality in the stock price-volume relation, ” J. Finance, vol. 49, no. 5, Lanka, in 2015. He is currently a senior software
pp. 1639–1664, Dec. 1994. engineer in machine learning and digital human
simulation fields. His research interests include data
mining, business analytics, deep learning, incremen-
Dinithi Nallaperuma received the master’s degree tal learning, and human simulation.
in advanced software engineering from the Univer-
sity of Westminster, U.K., and the Ph.D. degree
from the Research Center for Data Analytics and
Cognition, La Trobe University, Australia. Prior to
commencing her Ph.D. degree, she was a Research
Engineer with the National University of Singapore.
Her research interests include the Internet of Things,
data stream mining, unsupervised learning, and cog-
Daswin De Silva received the Ph.D. degree in
nitive computing.
machine learning and artificial intelligence from
Monash University. He is currently a Senior Lecturer
with La Trobe University, Australia. Besides his
Rashmika Nawaratne received the bachelor’s academic and professional expertise in analytics, he
degree (Hons.) from the Department of Computer is also actively involved in industry research and
Science and Engineering, University of Moratuwa, engagement. His research interests include cognitive
Sri Lanka, in 2014. He is currently pursuing the computing, autonomous learning algorithms, incre-
Ph.D. degree with the Research Centre for Data Ana- mental knowledge acquisition, stream mining, social
lytics and Cognition, La Trobe University, Australia. media, and text mining.
Prior to commencing his Ph.D. degree, he was a
Technical Lead with the Software Product Devel-
opment Organization. His research interests include
incremental learning, video analytics, deep learning,
the Internet of Things, and data stream mining.

Damminda Alahakoon received the Ph.D. degree

Tharindu Bandaragoda received bachelor’s degree from Monash University, Australia, in 2002. He is
(Hons.) in engineering from the University of currently a Professor of business analytics with the
Moratuwa, Sri Lanka, and the M.Phil. degree in IT La Trobe University Business School, Melbourne,
from Monash University, Australia. He is currently Australia, and the Director of the Research Centre
pursuing the Ph.D. degree with the Research Centre for Data Analytics and Cognition. He has published
for Data Analytics and Cognition, La Trobe Univer- over 100 research articles in data mining, clustering,
sity, Australia. He is currently working on the D2D neural networks, machine learning, and cognitive
CRC Beat the News Project. His research interests systems. He received the Monash Artificial Intelli-
are in cognitive-inspired incremental learning, data gence Prize from Monash University.
mining, social media analytics, large-scale data min-
ing for anomaly detection, and mining insights from
social media.

Achini Adikari received bachelor’s degree (Hons.)

in information technology from the University of Dakshan Pothuhera received the M.B.A. and mas-
Moratuwa, Sri Lanka. She is currently pursuing the ter’s degrees in management information systems
Ph.D. degree with the Research Center for Data Ana- from Swinburne University, Australia. He is cur-
lytics and Cognition, La Trobe University, Australia. rently a Senior Project Manager with VicRoads,
Prior to commencing her Ph.D. degree, she was where he is responsible for delivering business and
an Associate Tech Lead with the Software Product IT projects for VicRoads on behalf of the Victorian
Development Organization. She is currently works government. He has over 20 years of experience in
on several research projects related to text analysis the IT industry, including over three years in USA
in public health forums and social media data, with and 11 years in Australia.
a keen interest in human emotions analysis using
self-learning artificial intelligence (AI).

View publication stats