Learning To Route With Sparse Trajectory Sets - Extended Version
Learning To Route With Sparse Trajectory Sets - Extended Version
—Extended Version
Chenjuan Guo, Bin Yang, Jilin Hu, Christian S. Jensen
Department of Computer Science, Aalborg University, Denmark
{cguo, byang, hujilin, csj}@cs.aau.dk
Abstract—Motivated by the increasing availability of vehicle We study a very different routing approach that relies on the
arXiv:1802.07980v1 [cs.LG] 22 Feb 2018
trajectory data, we propose learn-to-route, a comprehensive availability of trajectories from local drivers. Assuming that
trajectory-based routing solution. Specifically, we first construct local drivers implicitly take into account a multitude of factors,
a graph-like structure from trajectories as the routing infras-
tructure. Second, we enable trajectory-based routing given an such as traffic conditions, turns, travel time, fuel consumption,
arbitrary (source, destination) pair. road types, and traffic lights, when making routing decisions
In the first step, given a road network and a collection of and thus know best which paths are preferable, we propose a
trajectories, we propose a trajectory-based clustering method methodology that utilizes paths found in historical trajectories
that identifies regions in a road network. If a pair of regions are to construct new paths between arbitrary (source, destination)
connected by trajectories, we maintain the paths used by these
trajectories and learn a routing preference for travel between
pairs. We call this trajectory-based routing.
the regions. As trajectories are skewed and sparse, many region If historical trajectories show that many drivers traveling
pairs are not connected by trajectories. We thus transfer routing from a source s to a destination d follow a particular path, it
preferences from region pairs with sufficient trajectories to such is straightforward to recommend that path to drivers asking for
region pairs and then use the transferred preferences to identify directions from s to d. The big challenge now is how to benefit
paths between the regions. In the second step, we exploit the
above graph-like structure to achieve a comprehensive trajectory-
from historical trajectories when no historical trajectories
based routing solution. Empirical studies with two substantial capture paths from s to d. This is important because any set of
trajectory data sets offer insight into the proposed solution, historical trajectories is sparse in the sense that it is unlikely to
indicating that it is practical. A comparison with a leading provide paths for all s’s and d’s. For example, the road network
routing service offers evidence that the paper’s proposal is able of Denmark, a small country, contains some 1.6 million edges.
to enhance routing quality.
Thus, if all edges are candidate s’s and d’s, a minimum of 2.6
This is an extended version of “Learning to Route with Sparse
Trajectory Sets” [1], to appear in IEEE ICDE 2018. trillion (s, d) pairs are needed. Given that the distribution of
trajectories in a road network is skewed, an enormous set of
I. I NTRODUCTION trajectories (e.g., trillions for Denmark and quadrillions for
Germany) would be needed before routing could be done by
Vehicular transportation is an important aspect of the daily simply looking up paths of past trajectories for any (s, d) pair.
lives of many people and is essential to many businesses Figure 1 exemplifies the problem setting. The solid edges
as well as society as a whole [2], [3]. As a part of the and filled vertices are covered by a set of five trajectories,
continued digitization of societal processes, more and more while the dashed edges and unfilled vertices are not covered
data is becoming available in the form of trajectories that by any trajectories. For example, trajectory T1 visited A and
capture the movements of vehicles [4], [5]. This data offers then J, X, Y , and B3 before reaching B. If routing from A
a foundation for improving vehicular transportation, including to B is requested, the path A → J → X → Y → B3 → B,
vehicle routing. as captured by trajectory T1 , can be recommended directly.
Traditional routing is cost-centric and aims at returning The challenge is to enable routing for (s, d) pairs that are not
paths with minimal costs, e.g., distance, travel time, or fuel connected by trajectories, e.g., (A1 , B2 ) and (H, F ).
consumption. The cost of a path is computed from edge costs To enable trajectory-based routing with massive, but still
in edge-based cost modeling [6]–[11] or sub-path costs in sparse, sets of historical trajectories, we propose means that
path-based cost modeling [12]–[15]. In such routing, trajectory are able to generalize the cases where historical trajectories
data is often used for annotating the edges or sub-paths with can be utilized for routing. This includes three steps. In the
travel costs such as travel times; and routing services employ first step, we cluster vertices into regions and thus map a road
shortest path algorithms, e.g., Dijkstra’s algorithm or contrac- network graph into a region graph. Trajectories that originally
tion hierarchies [16], to return fastest, or simply shortest, paths. connect vertices in the road network graph now connect
However, an existing study [17] suggests that local drivers who regions in the region graph. This arrangement generalizes the
drive passenger vehicles follow paths that differ substantially cases where trajectories can be used for routing from being
from the paths computed using cost-centric routing and are between specific vertex pairs to being between region pairs. As
often neither fastest nor shortest. Our paper also focuses on regions include multiple vertices, this arrangement contributes
trajectory data that was generated from passenger vehicles. to solving the data sparseness problem.
D N region R2 pairs. Third, it presents a unified routing algorithm for the
B1 region graph. Fourth, it reports on an empirical evaluation
K B
region R1
B3 that offers insight into the proposed solution, indicating that it
Y B2
A1 A X is capable of efficiently computing paths that match those of
J local drivers better than do traditional routing services.
F1
A2 I Z F2 F Paper Outline: Section 2 covers related work. Section 3
region R4
covers preliminaries. Section 4 presents Step 1, region graph
E generation. Section 5 presents Step 2, preference learning and
G C Visited Vertex transfer. Section 6 presents Step 3, unified routing. Section 7
M Unvisited Vertex reports on empirical evaluations. Section 8 concludes.
region R3 Visited Edge
H Unvisited Edge
II. R ELATED W ORK
Trajectories: T1=<A, J, X, Y, B3, B> T2=<D, X, Z, C>
We first review studies on employing historical trajectories
T3=<E, Z, F2, F> T4=<G, H> T5=<D, K, Y, F1, F>
for path recommendation, considering three cases.
Fig. 1: Motivating Example Case 1: Given a source and a destination, complete tra-
For example, in Figure 1, A and J are clustered into region jectories exist that connect the source to the destination. For
R1 , and B3 and B are clustered into region R2 . Now, although example, given A and B in Figure 1, trajectory T1 went from
no trajectories connect A1 and B2 , T1 connects regions R1 and A to B. Then, the path of trajectory T1 is recommended.
R2 that are close to A1 and B2 . Thus, the path of T1 can be When multiple paths exist, the path with the highest popularity
used for recommending a path from A1 to B2 . For instance, is recommended, where the popularity can be defined using
a user may go from A1 to A, then follow the path used by T1 different strategies [18]–[20]. This is the simplest case, which
to reach B, and then go to B2 . This enables trajectory-based is also considered in our proposal.
routing between regions connected by trajectories. However, Case 2: Given a source and a destination, no complete
in the region graph, some region pairs are still not connected trajectories exist that connect the source to the destination,
by any trajectories, e.g., regions R3 and R4 in Figure 1. but trajectories exist that can be spliced such that the spliced
trajectories connect the source to the destination. In Figure 1,
In the second step, we learn routing preferences from
given A and F , sub-paths A → X from T1 , X → Z from T2 ,
available historical trajectories that connect some region pairs
and Z → F from T3 can be spliced to form a path from A
and then transfer these preferences to similar region pairs that
to F . Alternatively, T1 and T5 can also be spliced to enable a
are not connected by trajectories. Based on the transferred
different path from A to F . To determine which spliced path
preferences, we identify paths for the non-covered region pairs.
is “best”, absorbing Markov chains [18] and hidden Markov
Note that the routing preferences are learned for different
models [21] are employed to the probabilities that different
region pairs, not for different individual drivers. Assume that
spliced paths may occur based on historical trajectories. The
(R1 , R2 ) is similar to (R3 , R4 ), e.g., because both are from
spliced path with the highest probability is chosen. In contrast,
a residential area to a business district. Next, we extract a
we learn routing preference vectors from trajectories and apply
routing preference from the trajectories connecting R1 and R2
the preference vectors to identify best paths.
that explains the choice of paths from R1 to R2 . We transfer
Case 3: Neither complete nor spliced trajectories are able
this routing preference to driving from R3 to R4 and then
to connect a source to a destination. In the example, consider,
identify paths connecting R3 and R4 , upon which trajectory-
e.g., A1 to B2 , H to F , and M to N . Here, existing
based routing from H to F is possible.
methods [18]–[21] no longer work. In this paper, the use of
In the third step, we provide a unified routing solution, the proposed region graph, together with the mechanism of
called learn-to-route (L2R), which performs path finding on learning and transferring routing preferences captured by past
the region graph, thus enabling routing between arbitrary (s, d) trajectories, makes it possible to extend the situations where
pairs in the original road network graph. historical trajectories can be utilized to cover also Case 3.
To the best of our knowledge, this is the first solution Next, we review related work on road network clustering.
that learns routing preferences from historical trajectories and Gonzalez et al. [22] propose a graph partition method based on
transfers the learned preferences to the part of a road network prior knowledge of the road network hierarchy with l levels,
that is not covered by trajectories, thus supporting comprehen- which may vary from country to country. Wei et al. [23]
sive trajectory-based routing for arbitrary (s, d) pairs. propose a grid-based method for constructing regions using
The paper makes four contributions. First, it presents a trajectories, where two adjacent grid cells are merged if more
trajectory-based road network clustering algorithm that pro- than τ trajectories exist that passed through them. These
duces the data foundation—the region graph. Second, it studies rely heavily on “appropriate” parameters, e.g., l and
presents a general routing preference model, including an τ . Tuning such parameters is non-trivial. Based on recent
algorithm that extracts preferences from historical trajectories advances in modularity based graph clustering, we propose
and an algorithm that transfers preference to similar region a generic, parameter-free region generation method, where
parameters such as l and τ are not needed. Our proposal is routing from s to d when no trajectories capture paths from s
also different from POI clustering [24]. to d.
Finally, we consider learning of routing preferences [25]– Solution Overview. We propose a three-step procedure to
[28]. Methods [25], [26] compare the paths used by trajectories conquer the data sparseness problem, as outlined in Figure 2.
to skyline paths [9] to identify different users’ dominating Legend
factors when choosing paths, e.g., travel time, fuel consump- Road Network G Trajectories T Module
tion, or distance. TRIP [27] uses the ratios between individual Data
drivers’ travel time and average travel time to model person- Clustering
alized travel times. A recent study from Microsoft presents Both T-edges
Only T-edges
an algorithm that learns driver-specific parameters for Bing Region Graph GR and B-edges
$ S S H D U D Q F H 3 H U F H Q W D J H
3 U H I H U H Q F H 6 L P L O D U L W \
6 L P L O D U L W \ 3 H U F H Q W D J H
such that the constructed paths match the paths in Pij as much
as possible. Equivalently, we P aim at solving the optimization
problem V ∗ = arg maxV ∈V Pk ∈Pij pSim(Pk , PkV ), where DI
TT
FC
The intuition is two-fold: first, the more edges the constructed We also show the distribution of the learned routing pref-
path PkV shares with the ground-truth path Pk , the more erences as bars in Figure 6(a). We aggregate more than 200
similar the two paths are; second, the longer the shared edges unique routing preferences based on their travel cost features,
are, the more similar the two paths are. i.e., DI, TT, and FC. The bars show that the routing preferences
A naive way of solving the optimization problem is to search are distributed almost uniformly, indicating that T-edges do
the whole space, i.e., all combinations of features in the master have different routing preferences.
and slave dimensions. However, the search space can be very
large, thus rendering the learning algorithm inefficient. We B. Step 2: Transferring routing preferences
instead propose an efficient learning algorithm that is inspired So far, we have identified preference vectors for T-edges.
by coordinate descent. In short, we first identify the best travel The next step is to associate preference vectors with B-edges,
cost feature in the master dimension, and next, based on the which can then be used to identify appropriate paths for B-
chosen travel cost feature, we identify the best road condition edges. To this end, we transfer the routing preferences of T-
features in the slave dimension. edges to similar B-edges, which follows the intuition that when
Specifically, given the source and destination of each ground two region edges are similar, they also have similar routing
truth path Pk , we obtain a lowest-cost path using each cost preferences. For example, if most local drivers choose the
type. This yields three lowest-cost paths P̂kDI , P̂kTT , and P̂kFC , fastest paths with a preference for main roads to travel between
for distance, travel time, and fuel consumption [37], [38], a region in the city center and a northern suburb residential
respectively. We then measure the similarity between path Pk area, it is also likely that local drivers have this preference
when traveling between another region in the city center and few highly similar (e.g., similarity above 0.9) T-edges. This
a southern suburb residential area. makes it possible to transfer routing preferences among region
Based on the above intuition, we first introduce the similar- edges, and these observations indicate that the design choices
ity function that quantifies the similarity between two region are purposeful.
edges and then provide an algorithm that transfers routing Transferring preferences among similar region edges: We
preferences between similar region edges. adopt the idea of graph-based transduction learning [39], [40]
Similarity between two region edges: Any region edge, to transfer routing preferences from T-edges to similar B-
a T-edge or a B-edge, connects two regions. A region edge is edges. First, we build a undirected, weighted graph, where
described by the features of its two regions. In particular, we a vertex represents a region edge, which can be a T-edge or a
use two elements dis and F to describe a region edge re. B-edge. Given a total of n region edges, we use an adjacency
Element re.dis is a real value, indicating the Euclidean matrix M ∈ Rn×n to record the edge weights of the graph.
distance between the centroids of the two regions connected Specifically, M[i, j] equals to the similarity reSim(rei , rej )
by the region edge. The distance information is an influential between region edges rei and rej , where 1 6 i, j 6 n.
factor when drivers choose their paths. For example, drivers Next, we introduce an adjacency matrix reduction threshold
may prefer the fastest paths if they travel long distances, but amr . In the adjacency matrix, we only keep the values that
they may prefer the shortest paths when traveling at shorter exceed amr ; otherwise, we set the values to 0. This way, the
distances. adjacency matrix only captures “sufficiently” similar region
Next, element re.F describes the functionalities of the two edge pairs, which enables to control the accuracy of the
regions. Element re.F is also essential because, for example, transferred preferences. The less dense resulting matrix also
when traveling between two business districts and between a improves efficiency (see Figure 9(b) in experiments).
residential area and a city center, drivers may have different Figure 7 shows a graph with four vertices (i.e., n = 4)
preferences. In particular, we use a set of road types to describe representing two T-edges and two B-edges. The corresponding
the functionality of a region [31]. For each region, we consider matrix M is also shown. For example, M[1, 3] = 0.9 indicates
all edges that are incident to the vertices in the region and that the similarity between re1 and re3 is 0.9, and M[2, 3] = 0
select top-k road types of the edges as the region’s road type indicates that the similarity between re2 and re3 is smaller
set. For example, regions Ri and Rj have top-2 road type than threshold amr .
sets {TP1, TP2} and {TP3, TP4}, respectively. Then, region In the next step, we use a matrix Y ∈ Rn×p to denote
edge (Ri , Rj ) has element re.F that is the Cartesian product the initial routing preferences of different region edges. Here,
of the road type sets from both regions: re.F = {hTP1, TP3i, n is the total number of region edges, including T-edges
hTP1, TP4i, hTP2, TP3i, hTP2, TP4i}. and B-edges, and p is the total number of travel cost and
Based on the above, the similarity between two region road condition features that are used for modeling routing
edges rei and rej , quantified by the similarity of their feature preferences in Section V-A.
vectors, is defined as follows. To illustrate, we consider two travel cost features DI and
TT and three road condition features indicating preferences
min(rei .dis, rej .dis)
reSim(rei , rej ) = + J (rei .F, rej .F). on road type TP1, TP2, and both, i.e., TP1+2. In this setup,
max(rei .dis, rej .dis) matrix Y has p = 5 columns that represent features DI, TT,
The similarity function is the sum of distance similarity and TP1, TP2, and TP1+2.
region function similarity. For distance similarity, the more Each row in Y corresponds to a region edge’s routing
similar the two distances are, the larger the similarity is. preference. For a T-edge, the features corresponding to its
This captures the intuition that travels between equally far learned routing preference V ∗ are set to 1. For example,
∗
apart regions may tend to have similar routing preferences. assuming that T-edge re1 has preference vector Vre 1
= hDI,
For region function similarity, we use Jaccard similarity to TP1i, the first row of Y is set to (1, 0, 1, 0, 0). Similarly, if T-
∗
evaluate the similarity between the region functions. If the edge re2 has preference vector Vre 2
= hTT, TP2i, the second
two region edges share more function features, meaning that row is set to (0, 1, 0, 1, 0), as shown in Figure 7. Next, since
they connect similar region pairs, travels on the two region the routing preferences of the B-edges are unknown, the rows
edges are expected to have similar routing preferences. that represent B-edges are set to (0, 0, 0, 0, 0).
To justify design choices, that (i) similar region edges The transduction learning yields matrix Ŷ ∈ Rn×p that
have similar routing preferences and that (ii) the proposed records the transferred routing preferences for the B-edges.
region edge similarity function reSim(·, ·) is effective, we Specifically, Ŷ[i, j] indicates the probability of region edge
show the results of an experiment using preferences learned rei having the j-th routing preference feature. For B-edge
from T-edges in Figure 6(b). First, the “Similarity” bars show rei , the travel cost feature with the largest probability, i.e.,
that similar T-edges have similar routing preferences, while arg maxx∈{1,2} Ŷ[i, x], is used as the final travel cost feature.
dissimilar T-edges have dissimilar routing preferences. Second, In the example Ŷ in Figure 7, this is DI for re3 and TT for
the “Percentage” bars show the percentages of T-edge pairs re4 . The road type feature with the largest probability, i.e.,
that fall in a different T-edge similarity ranges. There are many arg maxx∈{3,4,5} Ŷ[i, x], is used as the final road type feature.
similar (e.g., similarity above 0.5) T-edges, although there are In the example Ŷ in Figure 7, this is TP1 for re3 and TP2
re1 re2 re3 re4
V*re1=<DI, TP1> V*re2=<TT, TP2>
re1 graph Laplacian matrix L in the second term of Equation 2. In
re1 re2 re2
re3
particular, L = D − M, where M is the adjacency
P matrix and
re4 D is a diagonal matrix where D[i, i] = k∈{1...n} M[i, k]
TP1+2
and D[i, j] = 0 if i =
6 j. In our example, we have
TP1
TP2
TT
DI
1.6 0 0 0
1.6 0 −0.9
−0.7
0 0.8 0 0 0 0.8 0
re1
re2
D= 0 0 1.6 0 L= −0.9 0 1.6
−0.8
−0.7
0 0 0 2.2 2.2
re3 re4 re3
−0.7 −0.8 −0.7
With the help of L, the second term actually computes the
? DI, TP1 ? TT, TP2
re4
re1 sum of the products of the similarities of two region edges and
re2
T-edge B-edge re3 the differences of the two region edges’ corresponding routing
re4 preferences. When the similarity of the two region edges is
Fig. 7: Transferring routing preferences high, a small difference between their routing preferences
make the product significant. Minimizing the second term in
for re4 . Finally, B-edges re3 and re4 obtain the transferred the objective function has the effect of smoothly spreading the
routing preferences Vre3 = hDI, TP1i and Vre4 = hTT, TP2i, routing preferences from T-edges to B-edges such that (1) two
respectively. region edges with high similarities have highly similar routing
Now the remaining question is how to obtain Ŷ, which is preferences; (2) two region edges with low similarities may
the core of the transduction learning. have dissimilar routing preferences.
Obtain matrix Ŷ: We obtain Ŷ by minimizing the following Third, we conduct L2 regularization [39], [40] to avoid over-
objective function fitting.
p h
X Next, we need to minimize the objective function. By
O(Ŷ) = (Y·x − Ŷ·x )T S(Y·x − Ŷ·x ) differentiating Equation 2 by Ŷ·x and then setting it to 0,
| {z } we get
x=1
Keeping T-edges’ preferences
(S + µ1 L + µ2 I)Ŷ·x = SY·x (3)
i
T
+ µ1 Ŷ·x LŶ·x + µ2 ||Ŷ·x ||22 , Using basic linear algebra practice, Equation 3 can be solved
| {z } | {z }
Transferring preferences to B-edges Regularization by iterative approximation algorithms [41], e.g., the Jacobi
(2) method [39] or the conjugate gradient method [42]. We need
where Y·x and Ŷ·x indicate the x-th column of matrices to solve Equation 3 p times; and each time, we obtain a Ŷ·x
Y and Ŷ, respectively. Hyper-parameters µ1 and µ2 control where x ∈ {1, 2, . . . , p}. Finally, we obtain Ŷ.
the relative influences of the second and third terms in the
objective function, respectively. C. Step 3: Applying Transferred Preferences
The intuition of each term of the objective function is as After step 2, each B-edge has a transferred preference vec-
follows. First, the T-edges should keep the routing preferences tor. For each B-edge, we now identify a few appropriate paths
that are learned in step 1. The T-edges’ learned routing according to its transferred preference vector. Consider B-
preferences serve as training data in the transduction learning edge (Ri , Rj ). Recall that a region has transfer centers where
process. trajectories enter and leave the region (see Section IV-B).
Matrix S ∈ Rn×n is an auxiliary matrix that indicates For each pair of a transfer center from Ri and a transfer
which region edges are T-edges. In particular, we organize center from Rj , we identify a path according to the preference
the region edges such that the first x edges are T-edges and vector. Finally, the identified paths are associated with B-edge
the remaining n − x edges are B-edges. Then, S is a diagonal (Ri , Rj ).
matrix, where the first x diagonal entries are set to 1 and the We proceed to modify Dijkstra’s algorithm to accommodate
remaining
1 diagonal entries are set to 0. Specifically, we have the preference, as shown in Algorithm 2. To ease the discus-
0 0 0
0 1 0 0
sion, we assume that a B-edge is associated with a transferred
S= in our example because re1 and re2 are
0
0
0
0
0 0
0 0
routing preference vector hDI, TP1i, meaning that minimizing
T-edges. travel distance and using road type TP1 are preferred. Recall
Based on S, the first term actually computes the sum of that the first dimension is master dimension and the second
the squared differences between Y·x and Ŷ·x of the rows that dimension is the slave dimension.
represents T-edges. By minimizing the first term, we try to The overall procedure is similar to the classical Dijkstra’s
identify a Ŷ that minimizes the difference. This means that algorithm. In the algorithm, each vertex is associated with two
the T-edges should try to keep their learned preferences from attributes—a cost attribute that records the cost of travel from
step 1. On the other hand, the first term does not pose any the source to the vertex and a parent attribute that records the
constraints between Y·x and Ŷ·x of the rows that represents parent vertex of the vertex. And we use a priority queue Q to
B-edges. control the order of visiting different vertices (lines 1–4).
Second, the T-edges’ routing preferences are transferred to Here, the cost value maintained in a vertex corresponds
B-edges. The transfer process ensures that the more similar to the specific cost type feature for the master dimension
the two region edges are, the more similar their routing of a given preference vector. For example, when considering
preferences are. This is realized by the use of the unnormalized preference vector hDI, TP1i, each vertex is associated with
a cost that equals to the distance (according to DI) from the prefer to follow a region edge that enables us to go to a region
source vertex to the vertex. that is geometrically close to the destination region. When a
The algorithm always chooses the vertex with the lowest region edge exists that directly connects Rs and Rd , we always
cost, say vertex u, to continue exploring (line 6). When use that region edge. Otherwise, we give higher priorities to
exploring from u, we differentiate two cases (lines 7–14): (i) at the region edges that lead to regions that are closer to the
least one edge (u, x) satisfies the slave preference, and (ii) no destination region Rd .
edge (u, x) exists that satisfies the slave preference. For case To illustrate, consider the region graph shown in Figure 5(b)
(i), only edges that satisfy slave preference are explored. For and assume the physical locations of the regions are also
case (ii), all u’s adjacent vertices are explored. This way, we represented in Figure 5(b). Assume that regions R1 and R4 are
make sure that the preferences on both the master and slave given as the source and destination regions. When exploring
dimensions are accommodated by the algorithm. from R1 , region R5 is preferred over regions R2 , R3 and R6
because R5 is much closer to destination region R4 . Finally,
Algorithm 2: ApplyingPreferencesModifiedDijkstra
the region path h(R1 , R5 ), (R5 , R4 )i is returned.
Input: Preference Vector: V = hmaster , slavei; Source and
destination vertices: vs , vd ; Road Network: G; Recall that each region edge corresponds to some paths in
Output: Path P that connects vs and vd the original road network graph. Based on this, a region path
1 for each vertex v ∈ G.V do can be mapped back to a path in the road network graph,
2 v.cost ← +∞; v.parent ← null; which is then returned as the result.
3 vs .cost ← 0; Case 2: At least one of vs and vd is not in a region. In
4 Initialize a priority queue Q and add all vertices to Q; this case, we find appropriate region vertices for vs or/and vd .
5 while vd is still in Q do Then, we apply the procedure from case 1.
6 vertex u ← Q.extractMin ();
7 Boolean noneSat ← false; To this end, we issue a fastest path finding algorithm from
8 if there does not exist a vertex x such that x is u’s adjacent vs to vd based on road network graph G. If a region is visited
vertex and edge (u, x)’s road type satisfies V.slave then by the algorithm, we consider it as a candidate region Rs .
9 noneSat ← true; Similarly, we can identify a candidate region Rd . Then we
10 for each vertex x that is adjacent u do apply the procedure from case 1 with source region Rs and
11 if edge (u, x)’s road type satisfies V.slave ∨noneSat destination region Rd to identify a path P . Finally, we return
then
a path that consists of three sub-paths—the fastest path from
12 if u.cost + wV .master (u, x) < x.cost then
13 x.cost ← u.cost + wV .master (u, x); vs to Rs , denoted as Ps , the path P that connects Rs and Rd ,
14 x.parent ← u; and the fastest path from Rd to vd , denoted as Pd , as shown
in Figure 8. In case there is only one or no candidate region,
15 Construct P from vd using the parent attributes and return P ; we simply return the fastest path, e.g., in the case of vs and
vd′ in Figure 8.
The three steps yield a region graph where each region edge
v d’ Path associated with
has a set of paths, meaning that the region graph can serve as R1 region edge (Rs, Rd)
a foundation for routing. Fastest path
vs Rd
VI. ROUTING ON R EGION G RAPHS
Ps P Pd vd
Given an arbitrary pair of a source vs and a destination vd
in the original road network graph G, we present a routing Rs
R2
algorithm that is able to recommend a path connecting them,
using the region graph. We distinguish two cases. Fig. 8: Routing, Case 2
Case 1: Vertex vs is in a region, say Rs , and vertex vd
is also in a region, say Rd . If both vertices are in the same VII. E MPIRICAL S TUDY
region, i.e., Rs = Rd , since we maintain inner-region paths
We conduct a comprehensive empirical study on two sub-
inside regions, we check if trajectories exist that traverse from
stantial GPS trajectory data sets.
vs to vd . If yes, we return a path with the largest number of
trajectory traversals; if no, we return the fastest path.
A. Experimental Setup
If the vertices are not in the same region, i.e., Rs 6= Rd , we
first identify a region path based on the region graph and then Road Network and GPS Trajectories: We use two road
map the region path to a path in the original road network. networks, N1 and N2 , both obtained from OpenStreetMap
Routing on the region graph: The intuition is to find (openstreetmap.org). N1 represents the road network of Den-
a region path that follows fewer region edges to reach the mark and includes 667,950 vertices and 1,636,040 edges,
destination region Rd . This is because if a region path consists which are contained in a 320 km × 370 km rectangular
of many region edges, it involves the stitching of many paths region. N2 represents the road network of Chengdu, China
from different trajectories, which may not represent coherent and includes 27,671 vertices and 77,444 edges, which are
routing preferences. Thus, in the routing procedure, we always contained in a 33 km × 25 km rectangular region.
We use two GPS data sets D1 and D2 from N1 and N2 , If neither source nor destination belongs to a region, the path
respectively. D1 consists of more than 180 million high- is in category OutRegion.
frequency GPS records that were collected by 183 vehicles Implementation Details: All algorithms are implemented in
at 1 Hz (i.e., one GPS record per second) in 2007 and 2008. Java using JDK 1.8. We conduct experiments on a server
D2 consists of 100 million low-frequency GPS records that with a 64-core AMD Opteron(tm) 2.24 GHZ CPU, 528
were collected by 10,864 taxis from August 3rd to 30th 2014. GB main memory under Ubuntu Linux. We use distance,
The sampling rate varies from 0.03 Hz to 0.1 Hz. We only travel time, and fuel consumption as the travel cost features,
use parts of trajectories where taxi have passengers on. We where the fuel consumption is computed based on speed
map match [29] the GPS records in D1 and D2 onto N1 and limits using vehicular environmental impact models [37]. We
N2 , respectively, obtaining 466,305 trajectories and 185,284 use six commonly used road types from OpenStreetMap as
trajectories, where the trajectories in D2 represent trips with road condition features: motorway, trunk, primary, secondary,
passengers. The travel distance distributions of the trajectories tertiary, and residential. The transduction learning algorithm
are shown in Table II. for transferring preferences (cf. Section V-B) is implemented
Distance (km) (0,10] (10,50] (50,100] (100, 500]
# Trajectories of D1 427,430 35,271 2,263 1,341
using the Junto library (github.com/parthatalukdar/junto).
Percentage (%) 91.6 7.6 0.5 0.3
Distance (km) (0,2] (2,5] (5,10] (10, 35] B. Evaluation of Design Choices
# Trajectories of D2 29,256 105,503 43,473 7,052
Percentage (%) 15.8 56.9 23.5 3.8
We evaluate the design choices chosen for L2R. In partic-
TABLE II: Statistics of Trajectories ular, we show the effect of important parameters by varying
Training and Testing Data: We partition the trajectories in them according to Table III where default values are shown
D1 and D2 into training data and testing data. Specifically, as bold. When we vary one parameter, we keep the remaining
trajectories that occurred during the first 18 months in D1 parameters at their default values. We show results for both
and the first 21 days in D2 are used as training data; and D1 and D2 in most empirical studies, but omit some results
trajectories that occurred in the last 6 months in D1 and the for D2 when they show little difference to those of D1 .
Parameters Values
last 7 days in D2 are used as testing data. # T-edges 1X, 2X, 3X, 4X, 5X
Evaluation Criteria: For each trajectory in the testing data Threshold amr 0.5, 0.6, 0.7, 0.8, 0.9
Test, we record its source and destination, departure time, TABLE III: Parameters of L2R
and the actual path used by the trajectory. Since the aim
Region Sizes: We report the sizes of the obtained regions by
of the paper is to reuse local drivers’ routing intelligence
computing their convex hulls and then reporting their areas
to recommend paths, the paths used by the local drivers are
(in km2 ) and maximum diameters (in km). Table IV reports
considered as the ground truth (GT) paths.
Size (km2 ) (0,2] (2,10] (10,100] >100
In the experiments, we run learn-to-route (L2R) on each pair D1 3,357 (78.6%) 539 (12.6%) 304 (7.12%) 70 (1.63%)
of source and destination in Test, and we compare the returned / 9.5 / 15.8 / 29.9 / 304.1
Size (km2 ) (0,2] (2,5] (5,10] >10
path with the GT path, using the path similarity function in D2 388 (72.1%) 127 (23.6%) 19 (3.53%) 4 (0.74%)
Section V-A. In addition, we also identify the shortest path, / 4.24 / 8.17 / 8.59 / 6.22
the fastest path, the paths returned by two personalized routing TABLE IV: Region Sizes
algorithms and by Google Maps, and compare them with the
GT path. The departure time is used when identifying the L2R the numbers of regions whose area falls in given ranges and
paths, the fastest paths and the Google Maps paths. the maximum diameter of the regions in each range. There
are a few large regions, but most regions have sizes less
We also report results w.r.t. the accuracy using a different
than 2 km2 . This indicates that the proposed modularity-based
but also popular path similarity function [26], [36] to evaluate
clustering is able to control the region size and avoids very
the similarity between the routing path and the ground truth
large regions. D1 has a few large regions, which represent
path.
P backbone highways. Since we maintain inner-region paths for
V e∈Pk ∩PkV len(e) regions, large regions do not affect the final routing quality.
pSim(Pk , Pk ) = P (4)
e∈Pk ∪P V len(e)
k
Transferring Preferences: We study the accuracy of trans-
ferring preferences from T-edges to B-edges in Step 2. As
It follows the intuition of the similarity function in Sec- we have no ground-truth preferences for B-edges, we cannot
tion V-A, but utilizes the union of segments in the constructed evaluate the accuracy of the transferred preferences in a
path PkV and the ground-truth path Pk as the denominator. straightforward manner. To evaluate the accuracy, we ran-
Results are categorized according to the lengths of the GT domly partition the preferences of T-edges into 5 partitions. We
paths and according to whether the source or destination of a reserve one partition as a ground truth. Next, we use partition
GT path belongs to a region in the obtained region graph. If 1; partitions 1 and 2; partitions 1, 2, and 3; and partitions 1, 2,
both the source and destination of a GT path are in regions, 3, and 4, to conduct the preference transfer. For each T-edge
the path is in category InRegion. If either the source or the in the reserved partition, we obtain a transferred preference,
destination is in a region, the path is in category InOutRegion. which we compare it with the ground truth preference. We
report the accuracy of the transferred preference against the We leave such performance improvements as an interesting
ground truth preference using Jaccard similarity. future research direction.
Figure 9(a) shows the accuracy when using 1, 2, 3, and 4 We also consider two personalized routing algorithms,
partitions, labeled as X, 2X, 3X, and 4X. The results indicate Dom [26] and TRIP [27], that are able to find personalized
that the more preferences of T-edges are used, the better the “shortest” paths between arbitrary source and destination for
accuracy we get. Therefore, we use all the preferences of T- individual drivers. The algorithms first learn a global routing
edges (i.e., 5X) in the remaining experiments. preference (rather than a routing preference for each region
Next, we consider the effect of the adjacency matrix reduc- pair in this paper) for each driver from the driver’s historical
tion parameter threshold amr on the transfer process. Since trajectories, then use the learned preference to obtain new,
Figure 6(b) already suggests that when the similarity between personalized weights for all edges, and finally apply shortest-
two region edges is low, their preference vectors are dissimilar, path finding using the new edge weights. Specifically, Dom uti-
we vary amr from 0.5 to 0.9 and ignore small values. lizes a routing preference that considers distance, travel time,
We use 4 partitions of T-edge preferences to build the and fuel consumption, whereas TRIP uses a routing preference
adjacency matrix and use the last partition as the ground that considers only travel time. In the experiment, we apply
truth preferences. We report the accuracy of the transferred each algorithm to learn a routing preference according to a
preferences against the ground truth measured using Jaccard driver’s trajectories in the training data. For each trajectory in
similarity, the null rate (N-rate), i.e., the percentage of trans- the testing data, we obtain the source, the destination, and
ferred preferences that get null values, and the run-time in the driver id. Then we apply Dom and TRIP to compute
Figure 9(b). the personalized, shortest path connecting the source and the
The accuracy of the transfer process increases slightly as destination according to the driver id. Other routing algorithms
amr increases and is not sensitive to the change of amr that use historical trajectories, e.g., [18], [19], [21], do not
values when amr exceeds 0.5. This is intuitive because a support routing between arbitrary source and destination, and
large amr enables transfer of routing preferences from T- thus are not comparable to L2R.
edges only to highly similar B-edges. However, as the amr Accuracy: The accuracies of L2R, Shortest, Fastest, Dom, and
value increases, the graph used in the graph-based transduc- TRIP are calculated using the path similarity functions (see
tion learning may become disconnected. Thus, some B-edges Equations 1 and 4), and are reported in Figures 10 and 11.
cannot be associated with transferred preferences and thus get Shortest’s accuracy drops as the travel distance increases.
a null preference vector. A smaller amr has the effect that the This is because Shortest tends to find a path that approximates
graph used in the graph-based transduction obtains many edges the straight line segment from a source to a destination. Such
and thus takes longer run-time. The setting amr = 0.7 gives paths are often not preferred by drivers. In D1 , when traveling
the best trade-off, i.e., relatively high accuracy and efficiency longer distances, highways are usually preferred. However,
and low null rate. We thus use this value in the remaining given the fact that using highways often yield longer travel
experiments. We simply associate fastest paths with B-edges distances, Shortest does not return such paths. Therefore, the
with null preference vectors. accuracy of Shortest is poor for longer distances.
The accuracy of Fastest is comparable to that of Shortest for
D1 D2 10 4
100 100 Accuracy small travel distances. However, Fastest achieves much higher
N-Rate Run-time
80 80 accuracy when travel distance is longer. When travelling
Accuracy (%)
10 3
Accuracy (%)
Run-time (s)
Accuracy (%)
Accuracy (%)
Accuracy (%)
Accuracy (%)
80 80
60 60
60 60
40 40 40 40
20 20 20 20
0 0 0 0
(0,10] (10,50] (50,100] (100,500] InRegion InOutRegion OutRegion (0,5] (5,10] (10,35] InRegion InOutRegion OutRegion
Distance (KM) Region Distance (KM) Region
L2R Dom L2R Dom 100 L2R Dom 100 L2R Dom
Shortest TRIP Shortest TRIP Shortest TRIP Shortest TRIP
100 Fastest 100 Fastest Fastest Fastest
80 80
Accuracy (%)
Accuracy (%)
Accuracy (%)
Accuracy (%)
80 80
60 60
60 60
40 40 40 40
20 20 20 20
0 0 0 0
(0,10] (10,50] (50,100] (100,500] InRegion InOutRegion OutRegion (0,5] (5,10] (10,35] InRegion InOutRegion OutRegion
Distance (KM) Region Distance (KM) Region
0.96
Running Time (s)
Running Time (s)
100 L2R Google 100 L2R Google 100 L2R Google 100 L2R Google
80 80 80 80
Accuracy (%)
Accuracy (%)
Accuracy (%)
Accuracy (%)
60 60 60 60
40 40 40 40
20 20 20 20
0 0 0 0
(0,10] (10,50] (50,100] (100,500] InRegion InOutRegion OutRegion (0,5] (5,10] (10,35] InRegion InOutRegion OutRegion
Distance (KM) Region Distance (KM) Region
L2R path simply coincides with the fastest path. However, The personalized routing Dom requires significantly more
when historical trajectories can be utilized, L2R improves the running time as it conducts an expensive multi-objective
accuracy of the fastest path (see InOutRegion and OutRegion skyline routing process. Next, Trip has a running time similar
in Figure 10(b)). to those of Shortest and Fastest as all three perform single-
Online Running Time: Run-times are reported in Figure 12. objective routing. Trip just uses personalized weights.
In all settings, L2R is most efficient. This is because the path Offline Processing Time for L2R: When using all training
finding process is conducted on the region graph, which is data and default parameters, the offline processing time for
much smaller than the original road network graph. When constructing the region graph (Section IV) and for executing
sources and (or) destinations are not in regions, the run-time steps 1–3 to learn and transfer routing preferences (Section V)
of L2R increases because it needs extra time to identify the for D1 are 21, 245, 106, and 7 minutes, respectively, and for
fastest paths from the source (destination) to a region. D2 are 9, 10, 29, and 0.06 minutes, respectively. Note that such
offline processing is parallelizable, e.g., by MapReduce [43], time-dependency, e.g., using a time-varying region graph, real-
[44]. time region graph updates when receiving new trajectories, and
the modeling of more than one preference for each T-edge.
D. Comparison with Google Maps
R EFERENCES
We also compare L2R with Google Maps. We query the
Google Directions API using a source, a destination, and the [1] C. Guo, B. Yang, J. Hu, and C. S. Jensen, “Learning to route with sparse
trajectory sets,” in ICDE, 2018, 12 pages.
departure time from the testing set as arguments to obtain [2] C. Guo, C. S. Jensen, and B. Yang, “Towards total traffic awareness,”
a Google path, which consists of a sequence of waypoints, SIGMOD Record, vol. 43, no. 3, pp. 18–23, 2014.
represented by longitude-latitude coordinates. [3] Z. Ding, B. Yang, Y. Chi, and L. Guo, “Enabling smart transportation
systems: A parallel spatio-temporal database approach,” IEEE Trans.
We follow an existing methodology [19] to compute the Computers, vol. 65, no. 5, pp. 1377–1391, 2016.
similarity between a Google path and a GT path. We first [4] J. Hu, B. Yang, C. Guo, and C. S. Jensen, “Risk-aware path selection
represent a GT path as a polyline in the longitude-latitude with time-varying, uncertain travel costsa time series approach,” VLDB
Journal, to appear, 2018.
coordinate system. We call this polyline a GT path polyline.
[5] Z. Ding, B. Yang, R. H. Güting, and Y. Li, “Network-matched trajectory-
Next, we introduce two polylines that are parallel to the GT based moving-object database: Models and applications,” IEEE Trans.
path polyline and are 10 meters away on each side. We thus Intelligent Transportation Systems, vol. 16, no. 4, pp. 1918–1928, 2015.
obtain a band around the GT path polyline. The solid line in [6] C. Guo, B. Yang, O. Andersen, C. S. Jensen, and K. Torp, “Ecosky: Re-
ducing vehicular environmental impact through eco-routing,” in ICDE,
Figure 14 shows the GT path polyline, and the two dashed 2015, pp. 1412–1415.
lines indicate the band. When a Google waypoint is within [7] M. Hua and J. Pei, “Probabilistic path queries in road networks: traffic
the band, it is a matched waypoint. We project each matched uncertainty aware path selection,” in EDBT, 2010, pp. 347–358.
[8] H. Liu, C. Jin, B. Yang, and A. Zhou, “Finding top-k shortest paths with
waypoint onto the GT path polyline to obtain a projection diversity,” IEEE Trans. Knowl. Data Eng., vol. 30, no. 3, pp. 488–502,
point. If two consecutive waypoints are matched waypoints, 2018.
we regard the edges between their projection points as the [9] B. Yang, C. Guo, C. S. Jensen, M. Kaul, and S. Shang, “Stochastic
skyline route planning under time-varying uncertainty,” in ICDE, 2014,
edges on which the Google path is matched to the GT path, pp. 136–147.
e.g., edges e1 , e2 , e3 , e4 , e5 , and e6 in Figure 14. The above [10] H. Liu, C. Jin, B. Yang, and A. Zhou, “Finding top-k optimal sequenced
enables us to use the path similarity function in the similarity routes,” in ICDE, 2018, p. 12 pages.
[11] B. Yang, C. Guo, and C. S. Jensen, “Travel cost inference from sparse,
function in Section V-A of the paper submission to measure spatio-temporally correlated time series using markov models,” PVLDB,
the similarity between Google and GT paths. vol. 6, no. 9, pp. 769–780, 2013.
Boundary Polyline
[12] J. Dai, B. Yang, C. Guo, C. S. Jensen, and J. Hu, “Path cost distribution
× estimation using trajectory data,” PVLDB, vol. 10, no. 3, pp. 85–96,
2016.
10m GT Path Polyline
× e5 e6 e7 [13] B. Yang, J. Dai, C. Guo, and C. S. Jensen, “PACE: A PAth-CEntric
e4 paradigm for stochastic path finding,” VLDB Journal, online first, 2017.
e3 10m Boundary Polyline
e1 e2 [14] S. Aljubayrin, B. Yang, C. S. Jensen, and R. Zhang, “Finding non-
× Google Waypoint Projection Point dominated paths in uncertain road networks,” in SIGSPATIAL, 2016,
pp. 15:1–15:10.
Fig. 14: Google Accuracy [15] J. Hu, B. Yang, C. S. Jensen, and Y. Ma, “Enabling time-dependent
uncertain eco-weights for road networks,” GeoInformatica, vol. 21,
We report the accuracy of Google vs. L2R paths in Fig- no. 1, pp. 57–88, 2017.
ure 13. In particular, the accuracy of Google paths lies between [16] R. Geisberger, P. Sanders, D. Schultes, and D. Delling, “Contraction
hierarchies: Faster and simpler hierarchical routing in road networks,”
60% and 85%, and the accuracy increases with the travel in WEA, 2008, pp. 319–333.
distance. However, Google paths show no pattern when we [17] V. Ceikute and C. S. Jensen, “Routing service quality - local driver
categorize according to whether the source and destination be- behavior versus routing services,” in MDM, 2013, pp. 97–106.
[18] Z. Chen, H. T. Shen, and X. Zhou, “Discovering popular routes from
long to regions. In all settings, L2R achieves higher accuracy, trajectories,” in ICDE, 2011, pp. 900–911.
indicating that L2R has the potential to improve the quality of [19] V. Ceikute and C. S. Jensen, “Vehicle routing with user-generated
state-of-the-art routing services. trajectory data,” in MDM, 2015, pp. 14–23.
[20] W. Luo, H. Tan, L. Chen, and L. M. Ni, “Finding time period-based most
frequent path in big trajectory data,” in SIGMOD, 2013, pp. 713–724.
VIII. C ONCLUSION AND O UTLOOK [21] J. Dai, B. Yang, C. Guo, and Z. Ding, “Personalized route recommen-
dation using big trajectory data,” in ICDE, 2015, pp. 543–554.
We propose a learn-to-route solution that enables compre- [22] H. Gonzalez, J. Han, X. Li, M. Myslinska, and J. P. Sondag, “Adaptive
fastest path computation on a road network: A traffic mining approach,”
hensive trajectory-based routing. The solution encompasses an in VLDB, 2007, pp. 794–805.
algorithm that clusters road intersections into regions, yielding [23] L. Wei, Y. Zheng, and W. Peng, “Constructing popular routes from
a derived region graph. It learns routing preferences for region uncertain trajectories,” in SIGKDD, 2012, pp. 195–203.
pairs with sufficient trajectories and transfers these preferences [24] S. Shang, K. Zheng, C. S. Jensen, B. Yang, P. Kalnis, G. Li, and J. Wen,
“Discovery of path nearby clusters in spatial networks,” IEEE Trans.
to region pairs with insufficiently many trajectories. It then Knowl. Data Eng., vol. 27, no. 6, pp. 1505–1518, 2015.
utilizes the learned and transferred preferences to enable [25] A. Balteanu, G. Jossé, and M. Schubert, “Mining driving preferences in
routing. Empirical studies offer evidence that the solution is multi-cost networks,” in SSTD, 2013, pp. 74–91.
[26] B. Yang, C. Guo, Y. Ma, and C. S. Jensen, “Toward personalized,
practical and is able to compute high-quality routes. In future context-aware routing,” VLDB Journal, vol. 24, no. 2, pp. 297–318,
work, it is of interest to consider finer granularity modeling of 2015.
[27] J. Letchner, J. Krumm, and E. Horvitz, “Trip router with individualized [37] C. Guo, B. Yang, O. Andersen, C. S. Jensen, and K. Torp, “Ecomark
preferences (TRIP): incorporating personalization into route planning,” 2.0: empowering eco-routing with vehicular environmental models and
in AAAI, 2006, pp. 1795–1800. actual vehicle fuel consumption data,” GeoInformatica, vol. 19, no. 3,
[28] D. Delling, A. V. Goldberg, M. Goldszmidt, J. Krumm, K. Talwar, and pp. 567–599, 2015.
R. F. Werneck, “Navigation made personal: inferring driving preferences [38] C. Guo, Y. Ma, B. Yang, C. S. Jensen, and M. Kaul, “Ecomark:
from GPS traces,” in SIGSPATIAL, 2015, pp. 31:1–31:9. evaluating models of vehicular environmental impact,” in SIGSPATIAL,
[29] P. Newson and J. Krumm, “Hidden Markov map matching through noise 2012, pp. 269–278.
and sparseness,” in SIGSPATIAL, 2009, pp. 336–343. [39] P. P. Talukdar and K. Crammer, “New regularized algorithms for
[30] X. Liang, J. Zhao, L. Dong, and K. Xu, “Unraveling the origin of transductive learning,” in ECML/PKDD, 2009, pp. 442–457.
exponential law in intra-urban human mobility,” arXiv:1305.6364, 2013.
[40] Y. Wang, B. Yang, L. Qu, M. Spaniol, and G. Weikum, “Harvesting facts
[31] G. Forbes, “Urban roadway classification,” in Urban Street Symposium,
from textual web sources by constrained label propagation,” in CIKM,
1999, pp. B-6/1–B-6/8.
2011, pp. 837–846.
[32] M. E. Newman and M. Girvan, “Finding and evaluating community
structure in networks,” Physical review E, vol. 69, no. 2, p. 026113, [41] G. H. Golub and C. F. Van Loan, Matrix computations. JHU Press,
2004. vol. 3, 2012.
[33] M. E. Newman, “Analysis of weighted networks,” Physical review E, [42] B. Yang, M. Kaul, and C. S. Jensen, “Using incomplete information
vol. 70, no. 5, p. 056131, 2004. for complete weight annotation of road networks,” IEEE Trans. Knowl.
[34] H. Shiokawa, Y. Fujiwara, and M. Onizuka, “Fast algorithm for Data Eng., vol. 26, no. 5, pp. 1267–1279, 2014.
modularity-based graph clustering,” in AAAI, 2013, pp. 1170–1176. [43] B. Yang, Q. Ma, W. Qian, and A. Zhou, “TRUSTER: trajectory data
[35] J. Han, J. Pei, and M. Kamber, Data mining: concepts and techniques. processing on clusters,” in DASFAA, 2009, pp. 768–771.
Elsevier, 2011. [44] P. Yuan, C. Sha, X. Wang, B. Yang, A. Zhou, and S. Yang, “XML
[36] E. Erkut and V. Verter, “Modeling of transport risk for hazardous structural similarity search using mapreduce,” in WAIM, 2010, pp. 169–
materials,” Operations Research, vol. 46, no. 5, pp. 625–642, 1998. 181.