Distance Landmarks Revisited for Road Graphs

0.36 MB·

by  Shuai Ma

Distance Landmarks Revisited for Road Graphs
Shuai Ma1 Kaiyu Feng1 Haixun Wang2 Jianxin Li1 Jinpeng Huai1
1 SKLSDELab, Beihang University, China 2 Google Research, USA
{mashuai, fengky, lijx, huaijp}@buaa.edu.cn [email protected]
Abstract—Computing shortest distances is one of the funda- large graphs (e.g., with tens of millions of nodes and edges)
mental problems on graphs, and remains a challenging task for online applications [24]. Therefore, a lot of optimization
today.Distancelandmarkshavebeenrecentlystudiedforshortest
techniques have been recently developed to speed up the
distance queries with an auxiliary data structure, referred to
4 computation [5], [13], [20], [24], [25], [27], [28], [33], [34].
as landmark covers. This paper studies how to apply distance
1
landmarks for fast exact shortest distance query answering on Distance landmarks (a.k.a. distance oracles, see Section
0
large road graphs. However, the direct application of distance II-Bfordetails)aredatastructuresthatsupportefficientshort-
2
landmarksisimpractical duetothehighspaceandtimecost. To est distance query answering, and have been recently studied
n rectify thisproblem, we investigate novel techniquesthat can be
inboththeory[23],[30]andpractice[24],[26],[28].Ann×n
a seamlesslycombinedwithdistancelandmarks.Wefirstproposea
J notion of hybrid landmark covers, a revision of landmark covers. triangular matrix of size n2/2 for all-pair shortest distances
3 Second, we proposea notion of agents, each of which represents can be computed in O(n2logn+ mn) time, using Dijkstra’s
1 a small subgraph and holds good properties for fast distance algorithm[10], where n and m are the numbersof nodesand
query answering. We also show that agents can be computed in edges,respectively.Withthedistancematrix,shortestdistance

] lineartime.Third,weintroducegraphpartitionstodealwiththe
queriescanbeansweredinO(1)time.Thissolution,however,
B remaining subgraph that cannot be captured by agents. Fourth,
is not practicalon large graphs:the preprocessingtime is too
D we develop a unified framework that seamlessly integrates our
proposed techniques and existing optimization techniques, for long,andevenifoneiswillingtowaitthatlong,thematrixis
.
s fastshortestdistancequeryanswering.Finally,weexperimentally toolargeto be storedeffectively.Forinstance,the matrixof a
c verify that our techniquessignificantly improve the efficiency of graphwith one million nodesneedsabout1,862GB memory
[ shortest distance queries, using real-life road graphs.
(here the distance entries are stored as 4-byte integers).

1
Distance landmarks aim at striking a balance between the
v I. INTRODUCTION
efficiency benefits of answering shortest distance queries and
0
We study the node-to-node shortest distance problem on
9 the time and space cost of computing and storing them. And
6 largegraphs:givenaweightedundirectedgraphG(V,E)with distance landmarks have already been adopted for answering
2 non-negative edge weights and two nodes of G, the source approximate shortest distances [24], [26], [28], [30], and for
1. s and the target t, find the shortest distance from s to t answering exact shortest distances on directed graphs [14],
0 in G. We allow the usage of auxiliary structures generated [23].However,howtoapplydistancelandmarksforanswering
4 by preprocessing, but restrict them to have a moderate size exactshortestdistancesonundirectedgraphsismainlylimited
1 (compared with the input graph). In this work, we are only
to pure theoretical analyses [30].
: interested in exact shortest distances on large graphs.
v
Contributions&Roadmap.Toourknowledge,weareamong
i Findingshortest distances, a twin problemof findingshort-
X thefirsttostudytheapplicationofdistancelandmarksforfast
est paths, is one of the fundamental problems on graphs, and
r exact shortest distance queries on large undirected graphs.
hasfounditsusageasabuildingblockinvariousapplications,
a
e.g., measuring the closeness of nodes in social networks and (1) We develop an approximation algorithm with a constant
Webgraphs[18],[24],[28],andfindingthedistancesbetween factor 2 to analyze distance landmarks by establishing con-
physical locations in road networks [34]. nections with vertex covers (Section III), based on which
Algorithms for shortest distances have been studied since we show that the direct application of distance landmarks is
1950’sandstillremainanactiveareaofresearch.Theclassical not practical for large-scale graphs. We then propose hybrid
one is Dijkstra’s algorithm [6] due to Edsger Dijkstra. Dijk- landmark covers, a revised notion of traditional landmark
stra’s original algorithm runs in O(n2) [7], and the enhanced covers, to reduce the space cost (Section III).
implementation with Fibonacci heaps runs in O(nlogn+m) (2) We propose a notion of agents such that each agent rep-
due to Fredman & Tarjan [10], where n and m denote the resents a small subgraph, referred to as deterministic routing
numbersofnodesandedgesinagraph,respectively.Thelatter areas (DRAs) (Section IV). Then landmarksare only built for
remainsasymptoticallythefastestknownsolutiononarbitrary agents, instead of the entire graph. Hence, both space and
undirected graphs with non-negativeedge weights [30]. time cost are reduced. We give an analysis of agents and
However, computing shortest distances remains a challeng- DRAs, based on which we developa linear time algorithmfor
ing problem, in terms of both time and space cost, for large- computing DRAs along with their maximal agents. As shown
scale graphs such as Web graphs, social networks and road in the experimental study, on average about 1/3 nodes of a
networks. The Dijkstra’s algorithm [10] is not acceptable on graph are captured by agents and their DRAs.
(3) We introduce the bounded graph partitioning problem integrated the CH [13] and ARCFLAG [22] techniques with
(BGP) to deal with the remaining subgraph that cannot be distance landmarks into our framework.
capturedbythe DRAs of agents, and show thatthe problemis (4) Graph partitioning has been extensively studied since
NP-complete(SectionV).We thenproposeanotionof SUPER 1970’s [16], [17], [35], and has been used in various applica-
graphs that combine graph partitions with hybrid landmark tions,e.g.,circuitplacement,parallelcomputingandscientific
coverstosupportefficientshortestdistanceanswering.Wealso simulation [35]. The graph partitioning problem considered
build connections between the traditional graph partitioning in this work differs from the traditional one that it concerns
problemandtheBGP problem,andutilizethetraditionalgraph more on the number of nodes with edges across different
partitioningapproaches,e.g.,METIS,tosolvetheproblem.As partitions, instead of the number of edges with endpoints
shown by the experiments, METIS works well. across differentpartitions.Nevertheless, we build connections
(4)WeproposeaunifiedframeworkDISLANDforfastshortest between these two problems, and make use of the existing
distance query answering (Section VI), which seamlessly approaches, e.g., METIS [16], to solve the graph partitioning
combines distance landmarks with agents, graphs partitions problem considered in this work. It is also worth mentioning
(SUPER graphs), and existing speed-up techniques [32], [34]. that graph partitioning has already been used to speed-up
(5) Using real-life largeroadgraphs,we conductan extensive Dijkstra’s algorithm [22].
experimental study (Section VII). We find that our DISLAND (5) Agents and deterministic routing areas proposed in this
scaleswellwithlargegraphs,e.g.,ittakes0.28×10−3seconds study (Section IV) are significantly different(fromdefinitions
on graphs with 2.4 × 107 nodes and 5.7 × 107 edges. toanalysestoalgorithms)fromthe1-dominatorsets proposed
Moreover, DISLAND is 9.4, 134.9, and 14,540.1 times faster in [29]. Moreover, the latter are for shortest path queries on
than CH [13], ARCFLAG [22], and bidirectionalDijkstra [20], nearly acyclic directed graphs, which is not appropriate for
respectively. Moreover,the auxiliary structures occupy only a real-lifelargegraphs,asthese graphstypicallycontainalarge
moderate size of space (about 1/2 of the input graphs), and strongly connected components [2].
can be pre-computed efficiently.
Due to the space constraint,we deferall the proofsto [11]. II. PRELIMINARY
Related work. (1) Algorithms for node-to-node shortest dis- In this section, we first present basic notations of graphs.
tanceshavebeenextensivelystudiedsince1950’s,andfallinto We then introduce the notion of distance landmarks.
different categories in terms of different criteria:
• exact distances [4], [5], [7], [10], [13], [14], [20], [23], A. Graph Notions
[23], [25], [27], [29], [32], [34] and approximate dis-
We first introduce graphs and the related concepts.
tances [24], [26], [28], [30];
Graphs. A weighted undirected graph (or simply a graph) is
• memory-based [7], [10], [13], [20], [23]–[30], [32]–[34]
defined as G(V, E, w), where (1) V is a finite set of nodes;
and disk-based algorithms [4], [5];
(2) E ⊆ V ×V is a finite set of edges, in which (u,v) or
• for unweighted [24], [28], [33] and weighted graphs [4],
(v,u) ∈ E denotes an undirected edge between nodes u and
[5], [7], [10], [13], [14], [20], [23], [23], [25]–[27], [29],
v; and (3) w is a total weight function that maps each edge
[30], [32], [34]; and
in E to a positive rational number.
• for directed [14], [23], [29] and undirected graphs [4],
We simply denote G(V, E, w) as G(V,E) when it is clear
[5], [7], [10], [13], [20], [23]–[28], [30], [32]–[34].
from the context.
In this work, we study the memory-based exact shortest
distance problem on weighted undirected large real-world Subgraphs.GraphH(Vs,Es,ws)isasubgraphofgraphG(V,
graphs.Noneofthe previousworkhasexperimentallystudied E,w) if (1) for each node u∈Vs, u∈V, and, moreover,(2)
for each edge e ∈ E , e ∈ E and w (e) = w(e). That is, H
how to apply distance landmarks for solving this problem. s s
contains a subset of nodes and a subset of edges of G.
(2)Distancelandmarkshavebeenrecentlyinvestigatedforap-
We also denote subgraph H as G[V ] if E is exactly the
proximate shortest distance queries [24], [26], [28], [30], and s s
set of edges appearing in G over V .
foransweringexactshortestdistancesondirectedgraphs[14], s
[23].However,howtoapplydistancelandmarksforanswering Paths and cycles. A simple path (or simply a path) ρ is a
exactshortestdistancesonundirectedgraphsismainlylimited sequence of nodes v /.../v with no repeated nodes, and,
1 n
to pure theoretical analyses [30]. Nevertheless, in this work, moreover, for each i∈[1,n−1], (vi, vi+1) is an edge in G.
we investigate how to utilize distance landmarks to speed-up A simple cycle(orsimply a cycle)ρ is a sequenceofnodes
shortest distance queries on real-life large road graphs. v /.../v with v = v and no other repeated nodes, and,
1 n 1 n
(3) There has recently been extensive work on speed-uptech- moreover, for each i∈[1,n−1], (vi, vi+1) is an edge in G.
niques for shortest distance queries: bidirectionalsearch [20], The length of a path or cycle ρ is the sum of the weights
hierarchicalapproaches[13],nodeandedgelabeling[22],[27] of its constituent edges, i.e., Pin=−11w(vi,vi+1).
and shortcuts [25] (see [32], [34] for two recent surveys). We say thatv (resp. v ) is a neighborof v (resp. v ).
i+1 i i i+1
These techniquesare complementaryto our work, and can be We also say thata nodeisreachabletoanotheroneif there
incorporated into our approach. We have indeed seamlessly exists a path between these two nodes.
2
Shortest paths and distances. A shortest path from one node Input:A weighted undirected graph G(V,E,w).
Output:A landmark cover D of G.
utoanothernodevisapathwhoselengthisminimumamong
1. Remove redundant edges from G;
all the paths from u to v.
2. Compute a vertex cover D of G;
The shortest distance between nodes u and v, denoted by 3. return D.
dist(u,v), is the length of a shortest path from u to v. Figure1. 2-approximation algorithm forcomputinglandmarkcovers
Connected components. A connected component (or simply
approximationfactor of O(logn). To do this, we first present
a CC) of a graph is a subgraph in which any two nodes
a notion of redundant-edge-free (REF) graphs. We then build
are connected by a path, and is connected to no additional
the relationship between the LMC problem and the clasical
nodes. A graph is connected if it has exactly one connected
vectex cover (VC) problem on REF graphs, which leads to a
component, consisting of the entire graph.
2-approximation algorithm. Finally, we evaluate the cost of
Cut-nodes and bi-connected components. A cut-node of a landmark covers with the approximation algorithm.
graph is a node whose removal increases the number of A vertex cover of a graph is a set of nodes such that each
connected components in the graph. edge of the graph is incident to at least one node of the set.
A bi-connectedcomponent(or simply a BCC) of a graph is The VC problem is to find a minimum set of vertex covers, a
a subgraphconsistingofa maximalsetofedgessuchthatany classicaloptimizationproblemknowntobeNP-complete[12].
two edges in the set must lie on a common simple cycle. Graphsoftencontainredundantedgeswhendistancequeries
B. Distance Landmarks areconcerned.GraphGisredundant-edge-free(REF)ifitcon-
tainsnoredundantedges,whereanedge(u,v)isredundantif
We next introduce the notion of distance landmarks [24].
its removal has no effects on the shortest distance dist(u,v).
Consider an ordered set of l vertices D = < x ,...,x >
1 l By the definition of REF graphs above, it is trivial to see
such that for each i∈[1,l], x is a distinct node in graph G.
i that REF graphs preserve shortest distances, and that a graph
We say that D is a landmark cover of graph G if and only
mayhave multiple REF graphs.We nextbuild the relationship
if for any node pair (u,v) in G with u reachable to v, there
betweenlandmarkcoversandvertexcovers,statedasfollows.
exists a landmark x (1 ≤ i ≤ l) in D such that the shortest
i
distance dist(u,v) = dist(u,x )+dist(x ,v). This is achieved Theorem 2: For any REF graph G, a set S of nodes is a
i i
byrepresentingeachnodeinGasavectorofshortestdistances landmark cover of G iff S is a vertex cover of G. ✷
to the set of landmarks in D. More specifically, each node As a consequence, the LMC problem is identical to the VC
u∈V is represented as an l-dimensional vector distVec(u): problem on REF graphs.
distVec(u) = <dist(x ,u),...,dist(x ,u)>.
1 l Approximation algorithm. It is well-known that the VC prob-
The LMC problem is to find a landmark cover with a lem has a 2-approximation algorithm [31], which basically
minimum number of landmarks in a graph. The problem is
computesa maximal matching of a graph by greedily picking
unfortunately intractable, as shown below.
edges and removing all endpoints of the picked edges [6].
Proposition 1: The LMC problem is NP-complete [24]. ✷ Following from Theorem 2, we obtain a 2-approximation
To reduce its computational complexity, an O(logn)- algorithm for the LMC problem, presented in Fig. 1.
Given a graph G(V,E), the algorithm first computes an
approximation algorithm was proposed by using the approx-
imation algorithms for the set cover (SC) problem [24]. This REF graphof Gby removingredundantedges(line 1).Itthen
algorithm,however,runsin cubictime,andcannotbedirectly computes a vertex cover D of the REF graph (line 2), and
simply returns D as a landmark cover of G (line 3).
used for large graphs, as already been observed in [24].
Note that testing whether an edge (u,v) is redundant
Remarks. (1) With a landmark cover D, the exact shortest in a graph G(V,E,w) is typically efficient. When comput-
distancedist(u,v)foranynodepair(u,v)canbecomputedin ing dist(u,v) using Dijkstra’s algorithm on graph G(V,E \
O(|D|)time,where|D|isthenumberoflandmarksinD.This {(u,v)},w), if dist(u,v′) > w(u,v) for any node v′ before
is obvious as dist(u,v) = min{dist(u,xi) + dist(xi,v) | xi ∈ reaching v, it is easy to verify that (u,v) is not a redundant
D}.(2)Asa landmarkcoverD occupies|D| (|V|−1)space, edge.Moreover,for a largeportionof edges(u,v), its weight
itssize |D| mustbesmallinorderto applyitonlargegraphs. w(u,v) is exactly the shortest distance dist(u,v) in real-life
III. DISTANCE LANDMARKS REVISITED graphssuchasroadnetworks.Hence,our VC basedalgorithm
is typically much faster than the SC based algorithm [24],
Inthissection,wefirstshowthatitisnotpracticaltodirectly
though they have the same time complexity.
utilize landmark covers due to the high space cost. We then
Remarks. The 2-approximation algorithm allows us to have
propose a notion of hybrid landmark covers to alleviate this
both lower and upper bounds for the sizes and space cost of
problem. Here we consider a graph G(V,E,w).
landmarkcovers.IfthealgorithmreturnsalandmarkcoverD,
A. Landmark Covers then the lower and upper bounds for the size of the optimal
To give a more accurate estimation of landmarkcovers,we landmark cover are |D|/2 and |D|, respectively.
develop an approximation algorithm with a constant factor 2. Findings on landmark covers. We next experimentally test
Recall that the SC based algorithm(SectionII-B, [24]) has an theoverheadoflandmarkcoverswithourapproximationalgo-
3
TableI
OVERHEADOFLANDMARKCOVERSVS.ORIGINALGRAPHS
GraphsG(V,E) LandmarkcoversD
name size(MB) ≤|D|≤ ≤ ||DV|| ≤(%) ≤size≤(GB) ≤ ssiizzee((DG)) ≤ time(s)
CO 9.62 [181,276, 362,552] [41.6,83.2] [588.42, 1,176.83] [6.27×104, 1.25×105] 34.1
FL 24.59 [447,486, 894,972] [41.8,83.6] [3,568.67, 7,137.33] [1.49×105, 2.97×105] 391.8
CA 42.54 [761,662, 1,523,324] [40.3,80.6] [10,730.05, 21,460.09] [2.58×105, 5.17×105] 1,205.6
E-US 80.17 [1,450,115, 2,900,230] [40.3,80.6] [38,880.24, 77,760.48] [4.97×105, 9.93×105] 4,315.3
W-US 139.24 [2,545,995, 5,091,990] [40.7,81.3] [118,786.74, 237,573.47] [8.74×105, 1.75×106] 12,984.3
C-US 312.10 [5,811,428, 1,1622,856] [41.3,82.5] [609,721.69, 1,219,443.38] [2.00×106, 4.00×106] 66,996.9
US 531.63 [9,737,381, 19,474,762] [40.7,81.3] [1,737,359.48,3,474,718.95] [3.35×106, 6.69×106] 196,194.6
rithm.We tested sevenreal-lifedatasetsfrom[8] (pleaserefer space cost of making x a landmark, denoted by spaceL(x), is
to Section VII for details about the datasets and experimental exactly |N |. Alternatively, the naive approach incurs a space
x
settings). We adopted the adjacency-listrepresentation [6] for cost of |Px|, denoted by spaceN(x), by storing the shortest
graphswhencountingtheirspacecost,andassumedthatnodes distances for each node pair in P .
x
and distances were stored as 4-byte integers. Consider an ordered set of l vertices D = < x ,...,x >
1 l
The experimental results shown in Table 1 tell us that: such that (a) for each i∈[1,l], x is a node in graph G, and
i
(1) The size of an optimal landmark cover is large, and (b) Pxi ∩Pxj =∅ for any i6=j ∈[1,l].
typically 40%–80% of the nodes in a graph are landmarks. Hybrid landmark covers.WesaythatD˜ =(D,E−)isahybrid
D
(2)Thespacecostofalandmarkcoverishuge,andistypically landmark cover of graph G if and only if:
more than 104–106 times of the graphitself. For instance, the (1) for each xi (i∈[1,l]), spaceL(xi)≤spaceN(xi),
landmarkcoveroftheUSgraphwith1/2GBspacemayincur (2) there exist no other nodes x in G, but x 6∈ D, such that
a space cost of more than 1.74×106 GB. spaceL(x)≤spaceN(x), and
(3) Computing landmark covers of large graphs is inefficient. (3) E− is a set edges, denoting all the node pairs of G such
D
It took our algorithm more than 2 days 6 hours on the US that no landmarks in D lie on their shortest paths.
dataset. It is worth mentioning that here we only compute We also call E = {(u,x) | u∈N ,x∈D}∪E− the set
D˜ x D
the landmarks nodes, not including computing the shortest of edges enforced by a hybrid landmark cover D˜.
distances between graph nodes and landmarks. Furthermore,
Remark. (1) Essentially, D consists of a maximal set of
the directly usage of SC based algorithm [24] is even worse,
landmarks such that the space cost of each landmark in the
duetoitshighspaceandtimecost(itevenrunsoutofmemory
set is not larger than the corresponding naive cost.
–16GB– for the smallest CO dataset on our testing machine).
Hence, it suffices to conclude that the direct application of (2) A hybrid landmark cover D˜ of a graph can be treated as
anothergraphwith the same set of nodes,but with a different
distance landmarks as [24] is impractical for large graphs.
set of edges, i.e., the set E of enforcededges. Similarly,the
D˜
B. Hybrid Landmark Covers naiveapproachtransformsagraphintoacompletegraph.This
provides a unified view for these two approaches.
The naive matrix approachstores the pre-computedall-pair
shortestdistancesofagraphG(V,E),andtakes|V|(|V|−1)/2 (3) Computing hybrid landmark covers on large graphs re-
space.Andthelandmarkapproachwasproposedtoreducethe mainsverychallenging.Indeed,theycannotbedirectlyusedin
spacecostto|V||D|,where|D|isthesizeofalandmarkcover. practiceaswell.AswillbeseeninSectionV,webuildhybrid
One might believe that the landmark approach always incurs landmarkcoversw.r.t.a(small)subsetofnodesingraphG.In
the following, we will explore techniques to support efficient
less space than the matrix approach. It is, however, not the
case as shown by the following example. shortest distance queries on large real-life road graphs.
Example 1: Consider nodex in a landmarkcoverD that lies IV. USING REPRESENTATIVESFORLANDMARKS
on the shortest paths of a set {(u ,u ), ..., (u , u )} of
1 2 2k−1 2k As illustrated and analyzed in Section III, the direct appli-
k node pairs in a graph, where nodes u 6= u for any i 6= j
i j cation of distance landmarks is not practical for large graphs.
∈ [1,2k]. Then node x takes k space in the naive approach,
A straightforward approach is to use representatives, each
by directly adding edges to connectthose k node pairs, while
of which captures a set of nodes in a graph. The distance
it takes 2k space in the landmark approach, by adding edges
landmarksarefortherepresentativesonly,insteadoftheentire
between x and each of the 2k nodes. ✷
graph, which reduces both space and time cost.
This motivates us to propose a hybrid approach combining Thetasktofindaproperformofrepresentativesis,however,
thenaiveapproachwiththe landmarkone.Todothis, wefirst nontrivial. Intuitively, we expect representatives to have the
define the following notions. following properties. (1) A small number of representatives
Consider a node x in a graph G. Let P be a set of node canrepresentalargenumberofnodesin agraph;(2)Shortest
x
pairs such that x lies on their shortest paths, and let N be distances involved within the set of nodes being represented
x
the set of distinct nodes in P . For a landmark node x, we by the same representative can be answered efficiently; And,
x
only store the shortest distances between x and the node in moreover, (3) the representatives and the set of nodes being
N , instead of all the nodes in the graph as [24]. Hence, the represented can be computed efficiently.
x
4
(4)Nodev isamaximalagent,whilenodeuisnotamaximal
agent since A+ ⊂A+.
u v
We then consider graph G (V ,E ) in Fig. 3, and let c·
2 2 2
⌊p|V |⌋ = 2·⌊p5|⌋ = 4, where c=2 and |V |=5.
2 2
(1) Nodes u,v and w are three maximal agents, whose DRAs
are all the entire graph G , and, hence,
2
(2) u,v and w are three equivalent agents. ✷
Figure2. Usingagentsforlandmarks Figure3. ExampleagentsandDRAs Remarks. (1) As illustrated by the above examples, a DRA
of graph G(V,E) may have a size larger than c ·⌊p|V|⌋,
In this section, we first propose agents and deterministic
and multiple equivalent agents. (2) Trivial agents can only
routing areas (DRAs) to capture representativesand the set of
represent themselves. Hence, we are only interested in non-
nodesbeingrepresented,respectively.Wethengiveananalysis
trivial agents (or simply called agents) in the sequel.
ofthepropertiesof DRAs andtheiragents,andshowthatthey
are indeed what we want. Finally, we present a linear-time
B. Properties of Agents and DRAs
algorithm for computing agents and their DRAs. The idea of
using agents and DRAs is illustrated in Fig. 2. WenextgiveananalysisofagentsandDRAs,andshowthat
We consider a graph G(V,E,w). they hold good properties for shortest distance queries.
Proposition 3: Any agent in a graph has a unique DRA. ✷
A. Agents and Deterministic Routing Areas
We first present agents and their DRAs. This shows that agents and DRAs are well defined notions.
Agents. Givena nodeu in graphG(V,E), we say thatuis an Proposition 4: Without the size restriction, any node u in
agent of a set of nodes, denoted by A , if and only if: graph G is a maximal agent, and its DRA G[A+] is exactly
u u
(1) node u∈A is reachable to any node of A in G, the connected component (CC) to which u belongs. ✷
u u
(2) all neighbors of any node v ∈Au\{u} are in Au, and Thisjustifies the necessity of the size restriction foragents.
(3) the size |Au| of Au is equal or less than c·⌊p|V|⌋, Otherwise, DRAs are simply CCs, and are mostly useless.
where c is a small constant number, such as 2 or 3.
Proposition 5: For any two nodes v,v′ in the DRA G[A+] of
Here condition (1) guarantees the connectivity of subgraph u
agent u in graph G,
G[A ], condition (2) implies that not all neighbors of agent
u (1) the shortest distance dist(v,v′) in DRA G[A+] is exactly
u are necessarily in A ; and condition (3), referred to as size u
u the one in the entire graph G; and
restriction, limits the size of Au of agent u. (2) it can be computed in linear time in the size of G. ✷
Notethatanodeumaybeanagentofmultiplesetsofnodes
A1,...,Ak such that Ai ∩Aj = {u} for any i6= j ∈ [1,k]. The size restriction guarantees that the shortest distance
u u u u
AndwedenoteasA+ theunionofallthesetsofnodeswhose computation within a DRA can be evaluated efficiently.
u
agent is u , i.e., A+u = A1u ∪...∪ Aku. Proposition 6: Given a node v in the DRA G[A+u] of agent u
Maximal agents. We say that an agent u is maximal if there in graph G, and another node v′ in G, but not in G[A+], the
u
exist no other agents u′ such that A+ ⊂A+. shortest distance dist(v,v′) = dist(v,u) + dist(u,v′). ✷
u u′
Trivial agents. We say thata maximalagentu is trivial if A+ Propositions 5 and 6 together guarantee that the shortest
u
contains itself only, i.e., A+u = {u}. distancesbetweenthenodesintheDRAsoftwodistinctagents
Equivalent agents. We say that two agents u and u′ are can be answered correctly and efficiently.
equivalent, denoted by u≡u′, if A+ =A+.
u u′ Proposition 7: Any agent in a CC H(Vs,Es) of graph G(V,
Deterministic routing areas (DRAs). We referto the subgraph E) with |V |>c·⌊p|V|⌋ mustbe a cut-nodeofgraphG. ✷
s
G[A+] with nodes A+ as a DRA of agent u.
u u This motivates us to identify maximal agents by utilizing
Intuitively, DRA G[A+] is a maximal connected subgraph
u the cut-nodes and BCCs, which will be seen immediately.
connecting to the rest of graph G through agent u only.
We next illustrate these notions with an example below. Proposition 8: Any node in a bi-connectedcomponent(BCC)
with size largerthanc·⌊p|V|⌋ ofgraphG(V, E) isa trivial
Example 2: First consider graph G (V ,E ) in Fig. 3, and
1 1 1 agent. ✷
let c·⌊p|V |⌋ = 2·⌊p|16⌋ = 8, where c=2 and |V |=16.
1 1
(1)Nodeuisanagent,andits DRA isthesubgraphintheleft As we are interested in non-trivial agents only, those large
hand side of the vertical line across u; BCCs could be simply ignored with any side effects.
(2)Nodev isanagent,andits DRA isthesubgraphintheleft Theorem 9: Given any two agents u and u′,
hand side of the vertical line across v; (1) if u∈A+, then A+ ⊆A+;
u′ u u′
(3) Node w is not an agent since it can not find a DRA with (2) if u′ ∈A+, then A+ ⊆A+; and
u u′ u
size less or equal than 8; (3) A+∩A+ = ∅, otherwise. ✷
u u′
5
Input:Graph G(V,E) and constant c.
Output:The DRAs associated with their maximal agents.
1. Find all cut-nodes Vc and BCC nodes Vbc of G;
2. Build the BC-SKETCHgraph G(V,E,ω) with V = Vc∪Vbc;
3. Identify and return the DRAsand their maximal agents of G.
ProcedureextractDRAs
Input:BC-SKETCHgraph G(V,E,ω) of graph G and constant c.
Output:The DRAs and their maximal agents of G.
Figure4. Cut-nodes andbi-connected components 1. let F be the set of cut-nodes with leaf neighbors in G;
/* note that a leaf node must be a BCC node */
2. while F is not empty do
3. pick a cut-node v from F; let X be the neighbors of v;
/* note that there is at most one non-leaf node in X */
4. let α := Py′∈Xω(y′) - |X| + 1;
5. if α≤c·⌊p|V|⌋ then
Figure5. BC-SKETCHgraphG3 ofgraphG3 76.. lmeterωge(yanl)l B:=CCα;nodes in X and v into one BCC node yn;
Corollary 10: Given any two maximal agents u and u′, then 8. if there is a non-leaf node in X then replace it with yn;
either A+ =A+ or A+∩A+ = ∅ holds. ✷ 9. F := F \{v};
u u′ u u′ 10.let F′ be the set of new cut-nodes with leaf neighbors;
This says when maximalagents are concerned,there exists 11.for each cut-node v in F′ do
a unique set of non-overlapping DRAs. 12. let X′ be a set of leaf neighbors of v′ such that
13. for each y′∈X′, ω(y′) ≤ c·⌊p|V|⌋;
C. Computing DRAs and Maximal Agents 14. mark X′ as the DRA A+ of agent v′;
v′
15.return all DRAs with their maximal agents.
In this section, we first present a notion of BC-SKETCH
graphs, based on which we then propose an algorithm for Figure6. Computing DRAsandmaximalagents
computing DRAs and their maximal agents. (1) Finding cut-nodes and BCCs. The algorithm starts with
The main result here is stated as follows. computingallcut-nodesandbi-connectedcomponents(line1),
byusingthelinear-timealgorithmdevelopedbyJohnHopcroft
Theorem 11: Finding all DRAs, each associated with one
maximal agent, in a graph can be done in linear time. ✷ and Robert Tarjan [6], [15].
(2) Constructing BC-SKETCH graphs. After all the cut-nodes
We shall prove this by providing a linear time algorithm
and BCCs are identified, the BC-SKETCH graph G(V,E,ω)
thatcomputesDRAsandmaximalagents.We firstpresentBC-
can be easily built (line 2). To see this can be done in linear
SKETCH graphs, a key notion employed by the algorithm. time, the key observation is that the number |E| of edges in
A BC-SKETCH graph G(V,E,ω) of a graph G(V,E) is a G is exactly |V|−1 since G is a tree.
bipartite graph, in which (1) V = V ∪V such that V is
c bc c (3) Identifying DRAs and their maximal agents. Finally, the
the set of cut-nodes in G, and Vbc is the set of BCCs in G; algorithm identifies and returns the DRAs and their maximal
(2) for each cut-node v ∈ Vc and each BCC yb ∈ Vbc, there agents (line 3), using Procedure extractDRAs in Fig. 6.
(e3x)isωtsiasnaedwgeeig(hvt,fyubn)c∈tioEnisfufcvhitshaatcfuotr-neoadche onfodBeCyC y∈b;Vand, Procedure extractDRAs takes as inputthe BC-SKETCH graph
b bc G of graph G and constant c, and outputs the DRAs and their
ω(yb) is the number of nodes of G in BCC yb. maximal agents, by repeatedly merging BCCs with size less
Example 3: ConsidergraphG3 in Fig. 4(1),in whichlabeled than c·⌊p|V|⌋. More specifically, the procedure starts with
nodes u,v,w,x,y are the cut-nodes of G3, and the corre- the set F of cut-nodes with leaf neighbors (line 1). It then
sponding BCCs of G3 are BC1,BC2,BC3,BC4,BC5, and recursivelymergestheneighboringBCC nodesofcut-nodesto
BC6, and are shown in Fig. 4(2). generate new BCC nodes (lines 2-9). For a node v ∈ F with
The BC-SKETCH graph G3(V,E,ω) of graph G3 is shown neighborsX,ifP ω(y′)-|X|+1≤c·⌊p|V|⌋,theycan
inFig.5,inwhichω(BC )=4,ω(BC )=ω(BC )=ω(BC ) y′∈X
1 2 3 4 be merged into a new BCC node (lines 3-8). Intuitively, this
= ω(BC ) = 2, and ω(BC ) = 5. ✷
6 5 says cut-node v is not a maximal agent, and it is combined
One may noticethat thereare no cyclesin the BC-SKETCH into the DRAs of maximal agents. A key observation here is
graph G . This is not a coincidence, as shown below. thatthereisatmostonenon-leafnodeinX.Ifthereissucha
3
non-leafneighbor,thenitisreplacedbythenew BCC nodeyn
Proposition 12: BC-SKETCH graphs have no cycles, which
(line 8), by which the merging processing is made possible.
implies that they are simply trees. ✷
Once a cut-node is considered, it is never considered again
Proposition 12 indicates that we can employ the good (line 9). After no merging can be made, we have found all
properties of trees for computing DRAs and maximal agents. maximal agents, i.e., all the cut-nodes in the updated BC-
We are now ready to present algorithm compDRAs shown SKETCH graph. We then identify DRAs for these maximal
inFig.6.IttakesasinputgraphGandconstantc,andoutputs agents(lines10-14).Foranyleafneighbory′ ofacut-nodev′,
the DRAs of G, each associated with a maximal agent. if ω(y′) ≤ c·⌊p|V|⌋, then y′ is an Av′ of agentv′. All these
6
together constitute the A+ of agent v′ (lines 12-14). Finally, SUPER graphs.WenextintroduceSUPER graphsthatcombine
v′
all DRAs with their maximal agents are returned (line 15). graph partitions with hybrid landmark covers.
We now explain the algorithm with an example as follows. Consider a partition (V ,...,V ) of graph G. For each
1 k
fragment V (i ∈ [1,k]), let (1) B be the set of boundary
Example 4: Consider graph G in Fig. 4(1) again. Here we i i
3 nodes of V , and (2) D˜ = (D ,E− ) be a hybrid landmark
let c=2, and c·⌊p|V|⌋ = 6. Firstly, cut-nodesand BCCs are i i i Di
cover for the set B of boundary nodes of V .
computed as shown in Fig. 4(2). Secondly, the BC-SKETCH i i
graph G of G is constructed as shown in Fig. 5. After the The SUPER graph of graph partition (V1,...,Vk) is a
3 3 weighted undirected graph G(V,E,Υ) such that:
merging step stops, the updated BC-SKETCH graph consists
(1) V = B ∪...∪B ∪D ∪...∪D , i.e., the union of all
ofthree BCC nodes:BC1′ = {BC1,BC2,BC3},BC4,BC2′ = boundary n1odes and dkistanc1e landmarkks on each fragment;
{BC5,BC6} and two cut-nodes: w and x. Finally, the DRAs
(2) E = E ∪E ∪...∪E , where E ⊆E is the set of
and their maximal agents are identified: agent w with DRA B D˜1 D˜k B
BC′ and agent x with DRA BC′. ✷ edgeswithbothendpointsbelongingtoB1∪...∪Bk, andfor
1 2
eachi∈[1,k],E isthe setofedgesenforcedbythe hybrid
Correctness & Complexity. The correctness of algorithm landmark cover D˜D˜i; and
compDRAs can be readily verified based on the analyses i
(3)Foreachedge(u,v)∈E ,Υ(u,v)isexactlyequaltothe
in Section IV-B. To show that algorithm compDRAs runs in B
edge weight w(u,v) in graph G, and for each edge (u,v) ∈
linear time, it suffices to show that procedure extractDRAs
E (i∈[1,k]),Υ(u,v)isthelocalshortestdistancebetween
can be done in linear time. It is easy to see that each node in D˜i
u and v in the fragment V only.
the BC-SKETCH graph is visited at most twice in procedure i
That is, a SUPER graph G of graph G(V,E) only consists
extractDRAs, and hence the procedure runs in linear time.
of the landmarks and boundary nodes. Hence, the size of G
This completes the proof of Theorem 11.
is typically much smaller than graph G. Intuitively, SUPER
Summary.(1)WehaveproposedanotionofagentsandDRAs graphsuseasmallsetofnodesinafragment,i.e.,theboundary
aiming at reducingthe size of graphssuch that landmarksare nodes and distance landmarks, to represent a large number of
onlyforagents,insteadoftheentiregraph.(2)We havegiven nodes, i.e., all the nodes in the fragment.
a theoreticalanalysisof agentsand DRAs, based on whichwe
B. Bounded Graph Decompositions
have developed a linear time algorithm for computing DRAs
and their maximal agents. (3) As shown in our experimental As the landmarks are for the boundary nodes, the number
study, on average about 1/3 nodes of a graph are captured by of boundary nodes has a key impact on the size of SUPER
non-trivial agents and their DRAs. graphs.In addition, the size of a fragmentshould be bounded
in order to efficiently compute its hybrid landmark cover.
V. INTRODUCING GRAPH PARTITIONS FORLANDMARKS This motivates us to study the following problem.
The bounded graph partitioning problem is to find a partition
Web graphs contain a large strongly connected compo-
nents[2],and,similarly,thereisusuallyalargeBCCinreal-life (V1,...,Vk)ofgraphG(V,E),denotedbyBGP,suchthat(1)
graphssuchasthecollaborationandsocialnetworks[9],[19]. |Vi|≤ΓforeachfragmentVi (i∈[1,k]),and(2)|B|≤ǫ·|V|,
AspointedoutinSectionIV,fortheBCCsinagraphG(V,E) where Γ≤|V| is a positive integer,ǫ∈[0.0,1.0] is a rational
with a size larger than ⌊p|V|⌋, each node in those BCCs is number, and |B| is the total number of boundary nodes.
The problem is, however, nontrivial, as expected.
a trivial agent that can only represent itself. This motivates
us to introduce the graph partitioning techniques for distance Proposition 13: The BGP problem is NP-complete. ✷
landmarks,basedonwhichweuseasmallsetofnodes,instead
Traditional graph partitioning is to find a partition (V ,
of a single agent node, to represent a large set of nodes. 1
..., V ) of a graph such that (1) the k fragments have a
k
In this section, we first introduce graph partitions. We then
roughly equal number of nodes, and (2) the number of edges
propose a notion of SUPER graphs which combine graph
connecting nodes in different fragments is minimized. The
partitionswith hybridlandmarkcovers.We finallypresentthe
problem has been extensively studied since 1970’s [16], [17],
bounded graph partition problem and its solution.
[35], and has been used in various applications, e.g., circuit
We consider a graph G(V,E).
placement, parallel computing and scientific simulation [35].
Large-scalegraphpartitioningtoolsareavailablesuchasthe
A. Graph Partitions and Super Graphs
best-known METIS [16]. Hence, this study is not to propose
We first introduce graph partitions and SUPER graphs. a new graph partitioning algorithm. Instead, it builds rela-
Graph partitions. We say that (V1,...,Vk) is a partition of tionships between the BGP problem and the traditional graph
graph G(V,E) if and only if (1) Ski=1Vi = V, and (2) for partitioning problem, and makes use of existing approaches
any i 6= j ∈ [1,k], V ∩V = ∅, in which we refer to a V for solving the BGP problem.
i j i
(i∈[1,k]) as a fragment of the partition. Key observations. Foranypartition(V , ..., V ), the setB of
1 k
We also say that node u in V (1 ≤ i ≤ k) is a boundary boundarynodeswith edgesacrossdifferentfragmentsandthe
i
node if there exists an edge (u,v) in G from nodes u to v set E of all edges connecting nodes in different fragments
B
such that v ∈V and j 6=i (1≤j ≤k). satisfy: |B|≤2|E |.
j B
7
This is, minimizing |E | essentially reduces the upper
B
bound of |B|. Moreover, those edges in E are part of the
B
Figure7. Thepreprocessing module
SUPER graph. Hence, minimizing |EB| also reduces the size
of the SUPER graph. This observation inspires us to adopt (6) Finally, it builds a SUPER graph G(V,E,Υ) of graph G.
existing approaches,e.g., METIS [16], to partition graphsand The entire process is illustrated in Fig. 7.
generateSUPERgraphs.Aswillbeseenininourexperiments,
B. A Bi-level Query Answering Approach
smaller SUPER graphs help answer shortest distance queries.
We next present the query answering module.
Summary. (1) We have introduceda notion of SUPER graphs
Given a source node s and a target node t, this module
thatcombinegraphpartitionswithdistancelandmarks.(2)We
finds the shortest distance from s to t, by making use of the
haveproposedthe BGP problem,andshownitis NP-complete.
auxiliary structures produced by the preprocessing module.
(3) We have also built connections between the BGP problem
Morespecifically,givennodessandt,thequeryanswering
and the traditional graph partitioning problem, which makes
module executes the following processes:
it possible to use the existing approaches, e.g., METIS [16],
to solve our problem. As will be seen in our experiments, (1) When nodess and t belong to the same DRA G[A+u] with
METIS works well for the BGP problem, and the produced agent u such that A+u = A1u∪...Ahu.
SUPER graphsaretypicalsmall, whichonlyhave2–4%nodes If s and t further fall into the same Aiu, then it invokes
and 10–15% edges compared with the original graphs. Dijkstra’s algorithm on the subgraph G[Aiu]. Otherwise, it
simply returns w(s,u) + w(u,t) in constant time.
VI. A UNIFIED FRAMEWORK FORANSWERING SHORTEST (2) When s and t belong to two DRAs G[A+ ] and G[A+]
DISTANCE QUERIES with agentsu and u , respectively.As dist(s,uts) = dist(s,uut)
s t s
In this section, we propose a unified framework, referred +dist(us,ut)+dist(ut,t),inwhichdist(s,us)anddist(ut,t)
to as DISLAND, for fast shortest distance query answering, are already known, we only need to compute dist(us,ut).
which consists of two modules: preprocessing and query Let Vs and Vt be the fragments to which agents us and ut
answering. We combine distance landmarks with agents and belong, respectively.As observed in [4], fragmentsVs and Vt
graph partitions (SUPER graphs), and seamlessly integrate andthe SUPER graphtogethersuffice to answer exactshortest
existing speed-up techniques [13], [22] into the framework. distance queries. Hence, the algorithm invokes the Dijkstra’s
Consider a graphG(V,E) with non-negativeedge weights. algorithm on the union of subgraphs G[Vs], G[Vt] and the
SUPER graph G(V,E,Υ) to compute dist(us,ut).
A. Preprocessing for Query Answering Following the analysis above, we have the following.
We first present the preprocessing module. Proposition 14: Framework DISLAND correctly answers
Given graph G(V,E), the module seamlessly combines shortest distance queries. ✷
agents and graph partitions with hybrid landmark covers, and
it produces (a) maximal agents along with their DRAs, (b) C. Optimization Techniques
graph partitions, and (c) a SUPER graph G(V,E).
There exist quite a few speed-up techniques for shortest
More specifically, given graph G(V,E), the module exe-
distance computations [32], [34]. DISLAND is very flexi-
cutes the following processes:
ble such that most of these techniques, if not all, can be
(1)Itfirstcomputesthe DRAsandtheirmaximalagents,using
seamlessly incorporated to further speed-up shortest distance
algorithm compDRAs proposed in Section IV-C.
query answering. In this study we have adopted bidirectional
(2)ForeachDRAwithanon-trivialmaximalagentu,itfurther search [20], contraction hierarchies (CH) [13], and Arc-Flags
(a) computes all the shortest distances dist(u,v) for all nodes
(ARCFLAG) [22] due to their effectiveness and generality.
vinitsDRA,and(b)addsanedge(u,v)withweightdist(u,v) We first introduce the three optimization techniques.
for each node v in the DRA. (1) Bidirectional search(BSEARCH,[20])simultaneouslyper-
(3) It then generates a shrink graph, the subgraph G[A] of G forms two searches: forward and backward, starting at the
in which A is the set of agent nodes, including both trivial sourceandtargetnodes,respectively[20],[32].Itinvokestwo
andnon-trivialagents.Foreach DRA with a maximalagentu, instances of the Dijkstra’s algorithm simultaneously, and has
only u is kept in G[A]. thesametime complexityasthe(singledirectional)Dijkstra’s
(4) It next calls METIS [16] to produce a graph partition algorithm. However, BSEARCH is usually more efficient than
(V1,...,Vk) for the shrink graph G[A] such that for each the Dijkstra’s algorithm in practice.
i ∈ [1,k], |Vi| is roughly equal to c ·⌊p|V|⌋. Here c is a (2) Contraction hierarchies (CH, [13]) first imposes a total
small constant number, such as 2 or 3. order O on the nodes of a graph, in ascending order of
(5) For each fragment V (i ∈ [1,k]), it computes a (local) their relative ‘importance’,and then constructsa hierarchyby
i
hybridlandmarkcoverD˜ for the boundarynodesof V only, contractingall the nodes in this order. A node v is contracted
i i
by calling the SC based algorithm (Section II-B, [24]). Note by removing it from the graph such that shortest paths in the
that here we did not use the VC based algorithm, which was remaining graph are preserved, achieved by replacing paths
proposed for estimating of the size of landmark covers only. of the form u/v/w by a shortcut edge (u,w). Note that the
8
TableII
shortcut (u,w) is only required if u/v/w is the only shortest
REAL-WORLDGRAPHS
path from u to w. After all the nodes are contracted, all the
Name Regions #ofNodes #ofEdges
shortcuts are appended into the graph. CO Colorado 435,666 1,042,400
CH uses BSEARCH with minor revisions for query answer- FL Florida 1,070,376 2,687,902
CA California&Nevada 1,890,815 4,630,444
ing. Give two nodes u and w with O(u) < O(w), CH only E-US EasternUS 3,598,623 8,708,058
visits two kinds of paths u/···/v /···/w in the process: (a) W-US WesternUS 6,262,104 1,5119,284
i
O(u) < ··· < O(w) or (b) there is a uniquev with O(u) < C-US CentralUS 14,081,816 33,866,826
i US EntireUS 23,947,347 57,708,624
··· < O(vi)and O(vi) ¿ ···>O(w). Inthis way, CH avoids their DRAs and the SUPER graph G(V,E,Υ).
visiting the nodes with an order lower than u and w in the
(1) Let U = {u ,...,u } be the set of non-trivial maximal
forward and backward searches, respectively, which makes it 1 h
agents identified. The extra space of U and their DRAs is the
much more efficient than BSEARCH alone in practice.
extraedgesfromthoseagentstothesetofnodesintheirDRAs,
(3) Arc-Flags (ARCFLAG, [22]) is a partition-based edge la- which is exactly equal to Ph |A+| - h.
belingapproach,anditdividesagraphG(V,E)intopartitions i=1 ui
(2) Each fragment in the partition (V ,...,V ) roughly has
(V ,...,V )andgathersinformationforeachedgee∈E and 1 k
1 k the same size of c·⌊p|V|⌋. We set c = 2 or 3 in practice.
for each fragment V (i ∈ [1,k]) on whether the edge e lies
i Hence, the number of fragments is less than ⌊p|V|⌋.
on a shortest path into the fragmentV . To do this, each edge
i ForeachfragmentV (i∈[1,k]),letE bethesetofedges
e is associated with a flag vector fe with k bits (the number enforced by the hybridi landmark coverDD˜˜i for the boundary
of fragments) such that the vector f contains a flag 1 or 0 i
e
for V indicatingwhether or note is usefulfor a shortestpath nodes of Vi. Hence, the number of extra edges in the SUPER
queryitonodesinVi. Itiseasyto verifythat ARCFLAG incurs graph G is bounded by Pki=1|ED˜i|.
(3) The remaining extra space is incurred by the shortcuts
k|E| bits of extra space.
added by CH and the Arc-Flags added by ARCFLAG.
We next show how to seamlessly incorporate these three
As will be shown in our experiments, all these auxiliary
optimization techniques into our framework DISLAND.
structuresonlyincurasmallspacecost,andtheentireprepro-
(1) The shrink graph G[A] of graph G is appended with
cessing can be finished in a reasonably fast way.
shortcuts, by using the CH approach.
(2) We build a hybrid landmark cover for each fragment, by VII. EXPERIMENTALSTUDY
incorporating the CH searching process. WenextpresentanextensiveexperimentalstudyoftheDIS-
We only consider the shortest paths ρ = u/···/vi/···/w LANDframeworkforshortestdistancequeryanswering.Using
such that (a) O(u) < ··· < O(w), in which case ρ is real-liferoadnetworks,we conductedfive setsofexperiments
called order rising, or (b) O(u) ¡ ··· < O(vi) and O(vi) to evaluate: (1) the impacts of agents, graph partitions, and
¿ ··· > O(w), in which case ρ is called order turning. hybridlandmarkcovers;(2) the preprocessingtime and space
When computing landmarks for a fragment, we cover a node overhead of bidirectional Dijkstra [20], CH [13], ARCFLAG
pair (u,v) only if (1) there exists an order rising or turning [22], their counterpartsusing agents (Agent+ Dijkstra, Agent
path between u and v, and (2) their (local) shortest distance + CH, Agent+ ARCFLAG), and DISLAND; and (3) the perfor-
in the fragment is equal to their (global) shortest distance mance of all these approaches.
in the entire shrink graph. Moreover, (a) for these node
pairs (u,w) connected by order turning paths, we select the A. Experimental Settings
nodes with highest order as landmarks; and (b) for these We first introduce the settings of our experimental study.
remaining node pairs (u,w) connected by order rising paths, Real-life graphs. We chose seven datasets of various sizes
we use the cost model to greedily select landmarks or build
from the Ninth DIMACS Implementation Challenge [8],
direct edges, following the hybrid landmark approach. As the
shown in Table 2. Each dataset is an undirected graph that
searchingspaceisreduced,thisbothimprovestheefficiencyof
representsapartoftheroadnetworkintheUnitedStates(US),
computing hybrid landmark covers, and, of course, the query
where each edge weight is the distance (integers) required to
answering.Moreover,weadoptthequeryansweringapproach
travel between the two endpoints of the edge.
for CH [13], instead of the bidirectional Dijkstra’s algorithm,
Distance queries.Weadoptedthequerygeneratorin[34].Our
in the query answering module of DISLAND.
distance queries were generated as following. On each road
(3) We compute edge labeling, by using the ARCFLAG ap- network,we generatedeightsets Q , Q , ... , Q ofqueries.
1 2 8
proach.Todothis,wefurthercallMETIS todoasecondlevel (1) We first imposed a 256×256 grid on the road network
partition of the SUPER graph, where each fragment is treated and computedthe side length ℓ of each grid cell. (2) We then
as a single node, and the edge weight between fragments are
randomlychosetenthousandnodepairsfromtheroadnetwork
the number of edges connecting them. When building Arc-
to compose Q (i ∈ [1,8]), such that the grid distance of all
i
Flags, we again incorporate CH, by considering order rising nodepairsinQ isin[2i−1·ℓ,2i·ℓ).Notethatthegriddistance
i
or turning shortest paths only, to speed-up the processing.
oftwonodesu,vinaquerysetisthedistanceofthecellsinto
Extra space analysis. This module produces two kinds of which u and v fall, respectively. Moreover, the grid distance
auxiliarystructures:thenon-trivialmaximalagentsalongwith of any node pair in Q is larger than the grid distance of all
i
9
TableIII TableV
EFFECTIVENESSOFAGENTSANDDRAS EFFECTIVENESSOFHYBRIDLANDMARKCOVERS
Graphs Agents(#,%) Nodes(#,%)inDRAs time(s) Graph Withcostmodel Withoutcostmodel
CO (56,277,12.9%) (156,329,35.9%) 1.1 fragments |D˜| |ED˜| time(s) |D| |ED| time(s)
FL (140,379,13.1%) (378,937,35.4%) 3.7 CO 32.1 537.8 0.1 49.8 549.4 0.1
CA (273,191,14.4%) (623,811,33.0%) 11.3 FL 39.5 689.3 0.2 61.7 705.7 0.2
E-US (546,481,15.2%) (1,228,876,34.1%) 34.3 CA 51.3 1,021.9 0.4 78.4 1,045.1 0.4
W-US (869,904,13.9%) (2,116,339,33.8%) 100.4 E-US 71.1 1617.1 0.9 107.0 1651.8 0.8
C-US (2,034,358,14.4%) (4,583,413,32.5%) 402.4 W-US 68.9 1,541.6 0.9 104.6 1,576.8 0.9
US (3,452,222,14.4%) (7,927,453,33.1%) 1153.7 C-US 116.4 3,251.3 4.0 169.4 3,329.8 3.9
US 124.9 3,584.3 4.9 183.1 3,673.4 4.8
TableIV
EFFECTIVENESSOFGRAPHPARTITIONS TableVI
SIZESOFSUPERGRAPHS
Shrink fragments avg#of avg(#,%)of
graphs (#) nodes boundarynodes time(s) G CO FL CA E-US W-US C-US US
CO 220 1,269.7 (76.1,5.99%) 1.1 |Vc|/|V| 3.9% 3.0% 2.9% 2.8% 2.1% 2.3% 1.8%
FL 340 2,033.6 (92.5,4.55%) 3.1 |Ec|/|E| 14.5% 10.9% 12.7% 14.2% 10.3% 14.5% 12.0%
CA 470 2,695.8 (114.9,4.26%) 6.6 |V|/|V| 3.9% 3.0% 2.9% 2.8% 2.1% 2.3% 1.8%
E-US 630 3,761.5 (156.4,4.16%) 13.8 |E|/|E| 14.8% 11.1% 13.0% 14.5% 10.5% 14.5% 12.3%
W-US 840 4,935.4 (151.9,3.08%) 26.2
C-US 1,280 7,420.6 (241.4,3.25%) 85.5 Exp-2: Impacts of graph partitions. In the second set of ex-
US 1,650 9,709.0 (260.2,2.68%) 126.7 periments, we justified that the BGP problem could be solved
node pairs in Q . For each query set Q (i ∈ [1,8]), we well by METIS, originally for traditional graph partitioning
i−1 i
problems. Using the shrink graphs generated at Exp-1, we
report the average running time of over all the ten thousand
queries in the set. evaluatedtheeffectivenessandefficiencyofMETIS.Toensure
the query efficiency of DISLAND, each fragment has at most
Algorithms. We adopted the latest version 5.0.2 of METIS
c·⌊p|V|⌋ numberof nodes.We used the multilevelbisection
[21], implementedwith ANSI C. We also re-implementedthe
method of METIS with the balance factor fixed to 1.003. The
original CH [3] from its inventors of using Microsoft Visual
results are reported in Table 4.
C++. Bidirectional Dijkstra, ARCFLAG and their counterparts
The results tell us that there are only about (up to) 6%
using agents were also written in Microsoft Visual C++. All
of nodes are boundary nodes, and the largest graph can be
thesealgorithmsusedcommondatastructuresandprocedures,
finishedin127seconds.Thisclearlyjustifiedouranalysisand
borrowed from CH [3], for similar tasks.
choicetoattacktheBGP problembyusingexistingapproaches
All experiments were run on a PC with an Intel Core i5-
to traditional graph partitioning problems.
2400 [email protected] and 16GB of memory. Each test was
Exp-3: Impacts of hybrid landmark covers. In the third set of
repeated over 5 times, and the average is reported here. We
experiments,usingthegraphfragmentsgeneratedatExp-2,we
compare algorithms running on general commercialPCs with
evaluated(1)theaveragenumberofnodesandedgesenforced
a16GBmemorylimitation,andhence,algorithmsusinglarger
bythehybridlandmarkscoverswithorwithoutthecostmodel,
memory, e.g., [1], are not in our consideration.
and (2) their average efficiency on a single fragment. The
results are reported in Table 5.
B. Experimental Results
The results tell us that the usage of the cost modelboth re-
We next present our findings. In all experiments, we tested ducesthenumberoflandmarksandenforcededges,moreover,
the datasets in Table 2, and fixed the constant c = 2 when it only incurs little extra time cost.
computing agents and graph partitions on graphs G(V,E). We also report the SUPER graphs in Table 6. The SUPER
Exp-1: Impacts of agents. In the first set of experiments, we graphs G are quite small, typically have 2–4% nodes and
evaluated(1)thenumberofnon-trivialagents,(2)the number 10–15% edges compared with the original graphs G(V,E).
andpercentageofthenodesrepresentedbytheagents(exclud- Usinghybridlandmarkcoverswiththecostmodel,the SUPER
ing the agents themselves from DRAs), and (3) the efficiency graphsG(Vc,Ec)furtherreduce0.2–0.3%edges.Thisjustified
of our algorithm compDRAs for computing agents and their the effectiveness of agents and graph partitions, and the
DRAs. The results are reported in Table 3. introduction of the cost model for hybrid landmark covers.
Therearearound1/7nodesarenon-trivialagents,andabout Exp-4: Preprocessing time and space overhead. In the fourth
1/3nodesarecapturedbyagentsinthesegraphs,whichmeans setofexperiments,wetestedthespacecostandpreprocessing
basicallytheshrinkgraphisonlyabout2/3oftheinputgraph. time of Dijkstra, Agent + Dijkstra, CH, Agents + CH, AR-
Moreover,althoughthe size restriction is ≤2·⌊p|V|⌋, DRAs CFLAG, Agents + ARCFLAG, and DISLAND. For DISLAND,
are typically small in these graphs, and each agent represents we did a second level partition on the SUPER graphs into k
2 or 3 other nodes on average. Algorithm compDRAs also fragments, determined as follows: k = ⌊ m ⌋·100 if m ¿
1000
scales well, and it can be done in less than half an hour for 1000, and k =⌊ m ⌋·10, otherwise, where m is the number
100
the largest graph in the preprocessing. offragmentsoftheshrinkgraphs,showninTable4.ARCFLAG
As will be seen in the following experiments, this makes called METIS topartitionthegraphsintok fragmentsaswell.
agents a light-weight optimization techniques, which benefits The results are reported in Figure 8.
most, if not all, existing shortest distance algorithms. The results tell us that (1) the space cost follows the order:
10