Table Of ContentMulti-resource Energy-efficient Routing in Cloud
Data Centers with Networks-as-a-Service
Lin Wang∗§, Antonio Ferna´ndez Anta‡, Fa Zhang†, Jie Wu¶ Zhiyong Liu∗(cid:107)
∗Beijing Key Laboratory of Mobile Computing and Pervasive Device, ICT, CAS, China
†Key Lab of Intelligent Information Processing, ICT, CAS, China
‡IMDEA Networks Institute, Spain
§University of Chinese Academy of Sciences, China
¶Department of Computer and Information Sciences, Temple University, USA
(cid:107)State Key Laboratory for Computer Architecture, ICT, CAS, China
6
1
0
Abstract—With the rapid development of software defined
2
networkingandnetworkfunctionvirtualization,researchershave SDN Controller Application-Specified
r proposed a new cloud networking model called Network-as-a- Processing Functions
a Service(NaaS)whichenablesbothin-networkpacketprocessing
M
andapplication-specificnetworkcontrol.Inthispaper,werevisit
Network Function
the problem of achieving network energy efficiency in data Virtualization
2 centersandidentifysomenewoptimizationchallengesunderthe
1 NaaS model. Particularly, we extend the energy-efficient routing
optimization from single-resource to multi-resource settings. We Commodity General-
Purpose Server
] characterize the problem through a detailed model and provide
I
aformalproblemdefinition.Duetothehighcomplexityofdirect
N
solutions, we propose a greedy routing scheme to approximate
s. the optimum, where flows are selected progressively to exhaust Figure1. AnoverviewofageneralNaaSimplementationinatypicalcloud
c residualcapacitiesofactivenodes,androutingpathsareassigned data center. The control plane is separated from the data plane by the SDN
[ based on the distributions of both node residual capacities and whilethedataplanecanbecustomizedthroughNFV.
flow demands. By leveraging the structural regularity of data
2 hardware- and software-end modifications. This is due to the
center networks, we also provide a fast topology-aware heuristic
v method based on hierarchically solving a series of vector bin specific deployment settings designed for the routing and
6
packing instances. Our simulations show that the proposed forwarding protocols used in current data centers. As a result,
8
routing scheme can achieve significant gain on energy savings incremental implementations are usually not achievable and
0
and the topology-aware heuristic can produce comparably good
significant effort has to be made for every single design.
5
results while reducing the computation time to a large extent.
0 The situation has been changed with the evolution of
1. I. INTRODUCTION network architecture. On the one hand, researchers proposed
0 With the widespread adoption of cloud computing, enor- to decouple control plane from data plane, which enables
5 rapid innovation in network control. This idea then naturally
mous large-scale data centers have been deployed for compa-
1 led to the advent of Software Defined Networking (SDN).
nies like Google, Microsoft, and Amazon, to provide online
:
v services including searching and social networking. Generally Insteadofhavingallnetworknodestoruntheroutingprotocol
i in a distributed manner, SDN abstracts the network control
X speaking, data centers are consolidated facilities holding tens
functionality to a logically centralized controller. The routing
of thousands servers that are connected by a well-structured
r
a network termed data center network (DCN). Despite some decisions are then made by the controller and will be pushed
tonetworknodesthroughawell-definedApplicationProgram-
designs that rely on specialized hardware and communication
ming Interface (API) such as OpenFlow [3]. As a result, the
protocols,mostoftheDCNarchitecturesleveragecommodity
optimizationworkcanbetotallyimplementedinthecontroller,
Ethernetswitchesandrouterstointerconnectservers,andthus
whichneedsonlyverybasicsoftwaremodificationintheevent
are compatible with TCP/IP applications.
of network changes. On the other hand, the innovation in
As inter-node communication bandwidth is the principal
data plane has also been sped up by the technology called
bottleneck in data centers, there has been a large body of
Network Function Virtualization (NFV) where packets are
work on optimizing the performance of DCNs (e.g., [1], [2]).
handled by software-based entities on general-purpose servers
However, in order to apply these proposals to production
with network functions virtualized.
DCNs, a lot of effort has to be undertaken, including both
Taking advantage of the advancement of both control plane
ThisworkwassupportedinpartbyMinisteriodeEconomiayCompetitivi- and data plane in networks, a new cloud networking model
dad grant TEC2014- 55713-R, Regional Government of Madrid (CM) grant called Network-as-a-Service (NaaS) has been recently pro-
Cloud4BigData (S2013/ICE-2894, co-funded by FSE & FEDER), NSF of
posed[4],[5].AnoverviewofageneralNaaSimplementation
Chinagrant61520106005,andEuropeanCommissionH2020grantsReCred
andNOTRE. can be found in Fig. 1. In the NaaS model, packet forwarding
2
decisionsareimplementedbasedonspecificapplicationneeds. rithms by simulations. Section VIII concludes the paper.
Moreover,NaaSenablesthedesignofin-networkpacketmodi-
ficationandthusin-networkservices,suchasdataaggregation, II. RELATEDWORK
stream processing or caching can be specified by upper-layer In this section, we revise from several viewpoints the
applications. Based on this new networking model, several existing work related to our study.
working examples have been studied, including in-network Software defined networking. The high-level coupling of
aggregation for big data applications [6]. the control plane and the data plane in traditional networks
Although there is still a range of research challenges for bringsveryhighcomplexitytonetworkmanagementandleads
a widespread adoption of NaaS, we observe that some fun- to a very slow pace of development and evolution of network
damental problems have emerged from this new model. It is functionalities due to the reliance on proprietary hardware.
recognizedthatnomatterwhatnetworkingmodelisemployed, SDNisforcedtosolvetheseproblemsbychangingthedesign
problems such as load balancing and energy saving always and management of a network in the following two ways:
possesstheirimportance.However,comparedtothetraditional in an SDN, the control plane and the data plane are clearly
packet-forwarding-centricmodel,NaaSbringsnewchallenges separated; the control plane is logically consolidated. The
to these problems by allowing in-network packet operations, control plane thus exercises a full view of the network and
making existing solutions not efficient or even not applicable can be implemented with a single software control program.
to these problems any more. Through a well-defined API, the control plane carries out
Particularly, we revisit the problem of achieving network direct control to push decisions over multiple data-plane
energy efficiency in data centers and identify some new chal- elements in the network. The logically centralized control
lenges under the NaaS model. With packets being processed can facilitate most network applications including network
bygeneral-purposeservers,energy-relatedissuesbecomemore virtualization [11], server load balancing [12], and energy-
prominent. The energy saving problem in DCNs has been efficient networking [7], [10], which would require enormous
widely studied and most proposals are based on the general effortstobeimplementedinatotallydistributedenvironment.
approach of consolidating network flows and turning off Software packet processing. Recently, researchers have
unused network elements (e.g., [7], [8], [9], [10]). In packet argued for building evolvable networks, whose functionality
forwarding networks, link utilization is the most important changes with the needs of its users and is not tied to par-
criterion for flow consolidation. However, this is no longer ticular hardware vendors [13]. This is called general-purpose
valid under the NaaS model, where a network node can be networking, where a network-programming framework is run
congested not only by data communications, but also by the on top of commodity general-purpose hardware. On the one
overloading of other resources such as processing units or hand, there have been several research prototypes demon-
memory.Withoutconsideringotherresources,alinkutilization strating that general-purpose hardware is capable of high-
oriented consolidation of flows may lead to very bad resource performance packet processing when packets are subjected to
saturation at some network nodes and to serious network single particular type of processing, such as IP forwarding
instability. Therefore, it is essential to take into account [13] or cryptographic operations [14]. It has also been shown
multiple resources when making routing decisions under the in [15] that such a software packet processing platform can
NaaS model. To the best of our knowledge, this is the first achievepredictableperformancewhilerunningvariouspacket-
research attempt towards multi-resource traffic engineering. processing applications and serving multiple clients with dif-
The main contributions of this paper are as follows: i) we ferentneeds.Ontheotherhand,Niccolinietal.[16]developed
identifynewresearchchallengesforconventionaloptimization software mechanisms that exploit the underlying hardware’s
problemsundertheNaaSmodel,andcharacterizethenetwork power management features for more energy-efficient packet
energy saving problem through a detailed model with com- processing in software routers.
plexity analyzed; ii) we propose a greedy routing scheme Network-as-a-Service. With the development of SDN and
where path selection is done based on the distributions of software packet processing, a new networking model called
node residual capacities and flow demands; iii) by utilizing NaaS has been recently proposed [4], [5]. Under this model,
thestructuralpropertyofDCNs,weprovideatopology-aware the network is conducted based on an SDN implementation
heuristic which can accelerate problem solving while produc- where a centralized controller takes charge of flow man-
ing comparably good results; iv) we validate the efficiency of agement, including routing paths assignment and network
the proposed algorithms by extensive simulations and show functions interposition. The network is comprised of general-
that significant energy efficiency gain in NaaS-enabled DCNs purpose servers with multiple network ports connected by
can be achieved by the techniques proposed in this paper. high-speed links. In each node in the network, network func-
The remainder of this paper is organized as follows. Sec- tions for packet processing are virtualized and can be invoked
tion II summarizes the related work. Section III gives the throughthecontrollerbyupper-layerapplicationsaccordingto
model and the definition of the problem, as well as some theirneeds.Severalresearchattemptshavealreadybeenmade
complexity analysis. Section IV proposes a greedy routing to adopt NaaS in real date centers [6].
scheme and Section V presents a topology-aware heuristic. Energy-efficient DCN. The topic of achieving energy-
Section VI examines the performance of the proposed algo- efficientdatacenternetworkshasbeenextensivelystudied.Re-
3
searcheffortsareconcentratedonthefollowingtwocategories. commoditygeneral-purposeservers,itisreasonabletoassume
One is designing new architectures with less network devices that all the nodes in V are identical. Thus, for all v ∈ V, we
involvedwhileprovidingsimilarend-to-endconnectivity[17]. assume C =C for all k ∈{1,2,...,K}.
v,k k
Theotherisapplyingtrafficengineeringtechniquestoconsol- We define a flow as a sequence of data packets that possess
idate network flows and switch off unused network elements thesameentitiesinthepacketheaderssuchasthesamesource
(switches or links). The seminal work in this category is the and destination IP addresses. Suppose we are given a set of
concept of ElasticTree proposed by Heller et al. [7], which M flow demands D = {d ,d ,...,d }. The packets from
1 2 M
is a network-wide power manager that dynamically adjusts the same flow d will be routed following a single path in
m
the set of active network devices to satisfy changing traffic order to avoid packet reordering at the destination. For all the
loads in DCNs. Shang et al. [18] discussed how energy can packetsfromagivenflow,aprocessingprocedureisdefinedon
besavedbyenergy-awareroutingwithnegligibleperformance every node on the flow’s routing path, which is used to carry
degradation in high-density DCNs. Some follow-up works out some per-flow computation to the payloads of packets,
include REsPoNse [8], CARPO [9] and GreenDCN [10]. e.g., intercepting packets on-path to implement opportunistic
Multi-resource allocation. Resource allocation in comput- caching strategies [5]. Due to the fact that the data carried by
ing systems has been widely discussed under the limit of a the packets from the same flow generally possess the same
single resource, such as CPU time and link bandwidth. While structure (e.g., same packet size), we assume that (nearly) the
multi-resource allocation is considered in cloud computing same amount of computation will be applied to the packets
systems, it is usually carried out with a slot-based single from the same flow. As a result, we have to keep (almost) the
resource abstraction. Recently, researches have made some same reservation across each type of resource on every node
efforts (e.g., [19], [20]) towards multi-resource fair sharing on the path for each flow. Each flow d is represented by
m
undertheDominantResourceFairness(DRF)modelproposed a three-tuple (vs ,vt ,R(cid:126) ) where vs and vt are the source
m m m m m
by Ghodsi et al. [21]. On the network side, DRF-based ap- and destination respectively, while R(cid:126) is a K-dimensional
m
proaches (e.g., [22], [23]) have also been proposed to achieve vector (r ,...,r ) describing the amounts of resources
m,1 m,K
multi-resource fair queueing in software packet processors. in all types required (and reserved) for a node to process
Compared to these studies, our work possesses its unique- the packets from flow d . These resource demands can be
m
ness in the sense that we are the first to target the problem obtainedbyapplyingthesametechniqueusedin[22].Forthe
of achieving network-wide energy efficiency under multi- sake of simplicity and without loss of generality, we assume
resource settings based on the NaaS model. that the r for m∈{1,2,...,M} are normalized by C for
m,k k
any k ∈{1,2,...,K}, i.e. R(cid:126) ∈[0,1]K.
m
III. PROBLEMSTATEMENT To quantify the performance of approximations, we term γ
In the server end, the problem of finding the minimum as the performance ratio of an algorithm for a minimization
number of servers to accommodate a given set of tasks problem if the objective values in the solutions provided by
whose resource requirements are characterized by vectors is the algorithm are upper-bounded by γ times the optimal.
defined as the Vector Bin Packing (VBP) problem. The best
B. Problem Formulation
known solution for the general form of the problem is a
Using the introduced notations, the energy-efficient multi-
(lnK) + 1 + (cid:15) approximation for any (cid:15) > 0, provided by
resource routing problem can be formally defined as follows.
Basaletal.,[24]whereK isthenumberofdimensionsofeach
For a vector (cid:126)x, we denote by ||(cid:126)x|| the standard (cid:96) norm.
item. While the single-resource network energy optimization ∞ ∞
problem has been well-studied, very little attention has been Definition 1 (ENERGY-EFFICIENT MULTI-RESOURCE
received by the energy-efficient routing problem in networks ROUTING (EEMR)). Given a network G = (V,E) and a
with multiple resources. With an emerging trend of software set of M flows d ,...,d whose demands are characterized
1 M
packet processing in networks, this problem has raised its byR(cid:126) ,...,R(cid:126) from[0,1]K,findapathP fromvs tovt for
1 M m m m
significance. In the following, we provide a formal modeling each flow d such that ||A(cid:126) || ≤ 1 for v ∈ V where A(cid:126) =
m v ∞ v
of the problem and examine its complexity. (cid:80) R(cid:126) is the aggregation of the resource requirement
m:v∈Pm m
vectors of flows that are routed through node v. The objective
A. Preliminary Notations
is to minimize |Q| where Q = {v | v ∈ V ∧A(cid:126) (cid:54)= (0,...,0)}
i
We abstract a given software packet-processing network as is the set of nodes that are used to carry flows.
graph G = {V,E}, where V is the set of N nodes, each
The EEMR problem can be formulated as a Mixed Integer
of which represents a general-purpose server with software
Program(MIP)inthefollowingway.Weintroducetwobinary
packetprocessingfunctionalities,andE isthesetofundirected
variables x and y . The binary variable x indicates
edges representing the network links. Each node v ∈ V has m,v v m,v
whether flow d is routed through node v and y indicates
limited amounts of K different types of hardware resources, m v
whether node v is active or not. Our objective is to minimize
namely CPU, memory, and network bandwidth, to name a
the number of active nodes.1 Note that we have an implicit
few. The total amount of type-k resource is constrained by
a positive capacity C (k ∈ {1,2,...,K}). Due to the fact
v,k 1Asthestaticpowerconsumptionofanodeisdominant,weonlyconsider
that packet-processing networks are usually constructed using usingpower-downbasedstrategyasthemainenergysavingmechanism.
4
assumption that feasible solutions are always achievable, that solveVBPoptimally,whichcontradictswiththefactthatVBP
is, the network with the designed capability is able to handle is NP-hard.
the given traffic demands.
Theorem 2. There is no asymptotic PTAS for the EEMR
(cid:88)
(P ) minimize y problem unless P=NP.
1 v
v∈V
This is directly applied from the fact that VBP with K ≥
subject to
2 is know to be APX-hard which implies that there is no
|| (cid:88) R(cid:126) ·x || ≤1 v ∈V asymptotic PTAS for it [27]. From the above reduction we
m m,v ∞
m∈{1,2,...,M} know that actually VBP is a special case for EEMR, meaning
x ≤y v ∈V,1≤m≤M that EEMR has at least the same complexity as VBP.
m,v v
xm,v,yv ∈{0,1} v ∈V,1≤m≤M IV. ENERGY-EFFICIENTMULTI-RESOURCEROUTING
x : flow conservation
m,v The complexity analysis results show that the EEMR prob-
The constraints of program P are as follows: the first con- lemisNP-hard,forwhichnoexistingexactsolutionscanscale
1
straint states that the flows routed through the same node tothesizeofcurrentdatacenternetworks.Therefore,weresort
do not exceed the node resource dimensions; the second to an intuitive approach that can provide suboptimal solutions
constraint tells whether a node is active or not; the third very quickly. We detail our design in this section.
constraint ensures that each flow can only follow a single
A. Key Observations
path. Flow conservation on x forces that the nodes that
m,v
flow demand d is routed through form a path between vs We propose a greedy routing scheme to solve the energy-
m m
and vt in the network. efficientmulti-resourceroutingproblem.Thegeneralideaisto
m
Note that when K = 1, P corresponds to the general ca- useasfewaspossiblenodestocarryallthetrafficflowswhile
1
pacitatednetworkdesignproblemwhichhasbeenwidelystud- maintaing the capacity constraints in all resource dimensions.
ied. For the uniform link capacitated version of the problem, More specifically, our design is based on the following two
Andrews, Antonakopoulos and Zhang [25] provided a poly- observations: i) flows preferably follow paths that consists of
logarithmic approximation when the capacity on each link is moreactivenodes(thatalreadycarrysometraffic)asthiswill
allowed to be exceeded by a polylogarithmic factor. Recently, introduce less extra energy consumption to the network; ii) it
[26] explored the multicommodity node-capacitated network is important to allocate routes for flows on the active nodes
designproblemandprovidedaO(log5n)-approximationwith such that all dimensions of the resources in every active node
O(log12n) congestion. However, none of the studies can can be fully utilized.
provide high-quality approximations with capacity constraints Thesecondobservationisanewconcernsteamingfromthe
that are inviolable. This is mainly because with strict capacity multi-resource context. In the single-resource case, the only
constraints,findingoutwhetherthereisafeasiblesolutionfor criterionfortheefficiencyofanodeisitsresourceutilization,
the problem is already NP-hard. i.e.,thecarriedtrafficdividedbythetotalcapacity.Asaresult,
steeringflowstothosenodeswithlowutilizationswillleadto
C. Complexity Analysis
an energy-efficient routing solution. However, this approach
Incontrastwith thetraditionalenergy-efficientrouting(i.e., is not applicable to the multi-resource case. With multiple
capacitatednetworkdesign)problem,EEMRextendsthecon- dimensions of resources, it is not clear how to define the
cept of “load” from single-dimensional to multidimensional, resourceutilizationofanode,thuswewillnotbeabletomake
which makes the problem even computationally harder. In routing decisions based on node utilizations. For instance,
general, we have the following complexity results. given two load vectors (cid:104)0.6,0.4,0.1(cid:105) and (cid:104)0.4,0.4,0.3(cid:105) of a
node, and a flow demand vector (cid:104)0.1,0.3,0.4(cid:105), the resulted
Theorem 1. Solving the EEMR problem is NP-hard.
loads of this node when routing the given flow under the
Proof. Theproofisconductedonapolynomial-timereduction two different load levels are (cid:104)0.7,0.7,0.5(cid:105) and (cid:104)0.5,0.7,0.7(cid:105),
from the VBP problem which is known to be NP-hard. which are not directly comparable. In order to step over this
Assume we are given an arbitrary instance of VBP and now obstacle, we will provide a measuring method based on the
we reduce it to the EEMR problem in the following way: distributions of node residual capacities and flow demands.
each bin in the VBP instance is a node in the network for
B. The Routing Scheme
EEMR and each node is connected with two extra nodes src
and dst. Each item in the VBP instance represents a flow The pseudocode of the routing scheme is shown in Algo-
which originates from src and ends at dst and has resource rithm 1. The algorithm runs progressively. In each iteration,
requirements characterized by the vector for the item. Then, it first tries to use only the set of active nodes. By searching
straightforwardly,ifweobtainanoptimalsolutionforEEMR, the flow demand list, it tries to find out a candidate flow to
this solution will correspond to an optimal solution to VBP routeonthesubnetworkG composedbytheactivenodesand
a
with the same structure. As a result, any polynomial-time thecorrespondingnetworklinksconnectingthesenodes.Note
algorithm that optimally solves EEMR can also be used to thatitisnecessarytoremovethenodesthatarenotcapableof
5
Algorithm 1 Multi-Resource Green (MRG) routing Inversion-based node weight assignment. We now de-
1: Va =∅; /*set of active nodes*/ scribethefunctionInvCountforassigningweightstonetwork
2: for each (v ∈V) S(cid:126)v ={0}K; /*residual resources*/ nodes.Thesecondobservationwementionedatthebeginning
3: Ea (cid:44){(v1,v2)∈E | ∀v1,v2 ∈Va}; Ga (cid:44){Va,Ea}; ofthissectionsuggeststhatoncewehaveobtainedacandidate
4: while (D is not empty) flow to route on the subnetwork comprised of the active
5: dc == none; capable nodes, it is important to decide which nodes are
6: for each (dm ∈D) /*Search for a candidate flow*/ preferable to carry the candidate flow. We provide a measure
7: Gcm =Ga\{v |S(cid:126)v(cid:108)R(cid:126)m};/*removeincapablenodes*/ basedonthedistributionsoftheloadvectorsofboththenode
8: if (IsConn(Gm,vs ,vt ) == true) residualcapacitiesandtheflowdemand.Thegeneralnotionis
c m m
9: dc =dm; Gc =Gcm; thatiftheresourcedimensionsofanodeareallkeptbalanced,
10: break; thenmoreflowswilllikelyfitintothenode.Asaconsequence,
11: if (dc == none) /*candidate flow not found*/ the number of nodes that need to be active will be reduced.
12: dc =RandSelect(D); To clarify, we first introduce the concept of inversion.
13: Gc =G\{v | S(cid:126)v(cid:108)R(cid:126)c}; Definition 2. Given two vectors X(cid:126) = (cid:104)x ,...,x (cid:105) and Y(cid:126) =
14: for each (v ∈V) /*node weight assignment*/ 1 n
15: if (v ∈Va) wv =InvCount(S(cid:126)v,R(cid:126)m); (cid:104)y1,...,yn(cid:105), an inversion is defined as the condition xi > xj
and y <y , 1≤i,j ≤n.
16: else wv =K(K−1)/2+1; i j
17: for each ((v1,v2)∈Ec) we =(wv1 +wv2)/2; Property 1. Given two vectors in n dimensions, the total
18: Pc =SPath(Gc,dc); /*shortest path routing*/ number of inversions is upper bounded by n(n−1)/2.
19: Va =Va∪Pc; D =D\dc;
20: for each (v ∈Pc) S(cid:126)v =S(cid:126)v−R(cid:126)m; Aswearefocusingonthedistributionsofthenoderesidual
capacitiesandtheflowresourcesdemands,itisstraightforward
thataninversioncanleadtomuchheavierresourcedimensions
imbalance on a node as the scarce resource is demanded
more and the abundant resource is demanded less. Therefore,
carryingtheflow,thatis,whentheflowiscarriedbythenodes, in order to keep all the dimensions of resources balanced,
at least one dimension of the resource capacities of the nodes the number of inversions has to be minimized. Based on
willbeviolated,leadingtonodecongestion.WedenotebyGm this principle, the inversion-based node weight assignment
c
the residual network after removing the incapable nodes and procedure assigns weights for nodes that are already active
the links attached to these nodes. We then carry out function according to the number of inversions shared by the node
IsConn, a depth-first search procedure, to verify if the source residual capacity vector and the flow demand vector. The
and the destination of the current flow are connected in Gm. weights of the inactive nodes are set to be one unit larger
c
If a candidate flow d that can be routed on Gm is found, than the maximum number of inversions that can be shared
c c
we stop the search procedure; otherwise we pick up a flow by any residual capacity vector and flow demand vector. As
demanduniformlyatrandom(functionRandSelect)fromthe a result, if possible, the nodes that are active and with less
flow demand list. At this time, the residual capacity of the numbers of inversions will be preferably chosen to carry the
subnetwork formed by current active nodes is not sufficient candidate flow and the inactive nodes have the lowest priority
for carrying any new flow, thus more nodes are needed to be to be used.
activated so that routing demands for the newly selected flow Path selection.Theguidelineforselectingtherouteforthe
canbesatisfied.Onceacandidateflowd hasbeendetermined, candidate flow is to choose a path that connects the source
c
we remove the incapable nodes (those that satisfy S(cid:126) (cid:108)R(cid:126) and the destination of the candidate flow while minimizing
v c
which means that these exists at least one dimension k such the total weights of nodes that are on the path. This is
that S(cid:126) (k) > R(cid:126) (k)) according to the resources demand of actually equivalent to solving a node-weighted single-source
v c
the candidate flow and we denote by G the resulted network. shortestpathroutingproblem.Wenoticethatthisproblemcan
c
Then, we apply a weight assignment process where we assign be transformed into a traditional link-weighted single-source
weights to the active nodes in V by invoking procedure shortest path routing problem by setting the weight of each
a
InvCount (see below), and the weights for other nodes to link to be the half of the sum of the weights of the endpoints
be (K(K −1)/2+1). In order to facilitate path selection, of this link. Denote by R1 and R2 the node-weighted and
we carry out a node-link transformation procedure to assign the transformed link-weighted shortest path routing problems
weights for links based on the weights for nodes. The design respectively. We have the following property.
of node weight assignment and node-link transformation will Property 2. Solving R is equivalent to soving R .
1 2
be detailed later in this section. At last, the candidate flow
will be routed by involving a shortest-path-based algorithm This is because as long as a flow is routed through an in-
such as Dijkstra algorithm on the weighted network G and termediate node (nodes except the source and the destination)
c
will be removed from the demands list. The above process is onapath,theweightonthisnodewillbesharedbytwolinks
repeated until the route for every flow has been assigned. (i.e., the ingress and egress links). Thus, if we let all the links
6
Algorithm 2 Hierarchical Green Routing (HGR)
1: function VBP(D) /*vector bin packing algorithm*/ Core ……
2: idx=1; dc =none; Sidx ={0}K;
34:: αwkhi=le(cid:80)(Ddmis∈DnoRt mem(kp)ty/)(cid:80)dm∈D(cid:80)Kk=1Rm(k); Aggr. ……
5: Dc =D\{dm ∈D | Sidx(cid:108)Rm}; Edge
6: dc =argmindm∈Dc(cid:80)Kk=1αk(Sidx(k)−Rm(k))2; Pod #0 Pod #1
7: if (dc == none) idx++; /*open a new bin*/
8: else D =D\{dc}; /*pack the current item*/ Figure 2. An example to show the topology-aware heuristic in a fat-tree
topology,wherethesetofactivenodesaredeterminedlayerbylayer.
9: return idx
10: for (0≤i≤z−1) /*# of aggr. nodes in each pod*/ new topology-aware heuristic for the most common tree-like
11: Niagg =VBP({dm | vms or vmt in pod i}) data center network topologies.
12: for (0≤j ≤z/2−1) /*# of core nodes*/ The key observation we have from tree-like topologies is
13: Njcore =VBP({dm | (vms or vmt mod (z2/4))/2=j}) that the number of active nodes can be determined layer
by layer. We take a typical fat-tree topology (as shown in
Fig. 2) as an example. The number of edge nodes cannot be
that are attached to an intermediate node to share half of the optimized since edge nodes are also responsible for inter-host
nodeweight,itisalwaystruethatthetotalweightsonthelinks communication in the same rack. In each pod, the number
on this path will be equal to the total weights on the internal of aggregation nodes can be determined according to the
nodesonthispath.Asaresult,solvingthecorrespondinglink- flow demands that flow out of and into the pod. This is
weightedsingle-sourceshortestpathroutingproblemwillalso actually to solve a vector bin packing problem as we have
givesolutionstothepathselectionforthecandidateflow.Itis introduced previously. The core layer is a bit different from
well-known that the link-weighted single-source shortest path the aggregation layer; for a z-ary fat-tree, all the cores nodes
routingproblemcanbesolvedefficientlybyusingtheDijkstra thatsharecongruencewithrespectto(z/2)willberesponsible
algorithm. for carrying the flow demands from the aggregation nodes in
the same positions in every pod. Thus for these core nodes,
C. Time Complexity
solving a vector bin packing can give the right number of
WenowanalyzethetimecomplexityoftheproposedMRG nodesthatneedtostayactive.Inspiredbythisobservation,we
algorithm. The algorithm runs iteratively and in each iteration proposeHGR,ahierarchicalenergy-efficientroutingalgorithm
exactly one flow will be chosen as the candidate and will be based on solving a set of vector bin packing problems. The
routed.Asaresult,themaximumnumberofiterationswillbe pseudocode of HGR is shown in Algorithm 2.
upper bounded by the number of flow demands M. In each Vector bin packing. The function VBP we adopted for
iteration, the algorithm first searches in the flow demand list solving the vector bin packing problem is a norm-based
tofindoutacandidateflowandthemosttimeconsumingpart greedy algorithm [28]. The algorithm is bin-centric which
inthiscandidateflowsearchingprocedureisdepth-firstsearch means that it focuses on one bin idx and always places
which can be accomplished in O(|E|) time where |E|≤N2 the most suitable remaining item that fits in the bin. To
is the total number of edges in the network (N is the total find out the most suitable item, the algorithm looks at the
number of nodes). The total searching time in one iteration difference between the demand vector R and the residual
m
then will be in O(M·|E|). Once the candidate flow is found, capacity vector S under a certain norm. We choose the (cid:96) -
idx 2
the time complexity will be dominated by the shortest path norm and from all unassigned items, we choose the item that
routing algorithm which can be done in O(|E|+N ·logN) minimizes (cid:80)K α (S (k)−R (k))2 where α represents
k=1 k idx m k
time as the weight assignment procedure can be finished in the importance of dimension k among all dimensions and is
time O(N·K·logK). Combining all these, we have that the given by
(cid:80)
MRG algorithm can be finished in O(|E|M2) time. α = dm∈DRm(k) .
k (cid:80) (cid:80)K R (k)
V. TOPOLOGY-AWAREHEURISTIC dm∈D k=1 m
TheproposedMRGalgorithmcanleveragethecoordination Ifnoitemcanbefoundtofitintothecurrentbinidx,weopen
of the flow demands in multiple dimensions and minimize the a new bin and repeat the above procedure.
numberofactivenetworknodesefficiently.However,MRGis Time complexity. The HGR algorithm replies on solving
generally conducted without taking into account the topology several instances of the vector bin packing problem. In the
features of the network. We notice that topologies of the worst case, the sizes of the vector bin packing instances can
networks commonly used in data center networks such as fat- be as large as O(M) and thus it will take O(M2) time to be
tree or VL2 have very high level of symmetry and they are solvedbyVBPalgorithm.Asaresult,thetotaltimecomplexity
usually well structured in layers. Therefore, we argue that the of HGR can be given by O(M2). Compared to the MRG
routingalgorithmcanbefurtherimprovedbytakingadvantage algorithm, HGR can provide a speedup of Ω(|E|). We will
of the topology characteristics. In this section, we provide a validate this speedup by simulations.
7
VI. EVALUATION
SRSP SRG MRSP MRG SRSP SRG MRSP MRG
We carried out extensive simulations to evaluate the perfor-
maAC.PadWnUeSctieamewidluoeeiltfdpahlttoshitouyewnmopomuSrorepaptahrtolyiygnssoegoidcrsfiataolhlumgcroossrroiietmnshmuaalnslaa.dtpiIotn8onGpthfiBwinsditDsihneRcgatAsiC.oMonr,.ewAi5ell2p.ro6ofGvitHdheez Number of Inactive Nodes (%)000000......463512 Number of Incomplete Flows11486202000000
algorithms are implemented in Python. 0.0 20 40 N6u0mber8 o0f Flo1w0s0 200 300 0 20 40 N6u0mber8 o0f Flo1w0s0 200 300
Wechoosefat-treesofdifferentsizesasthedatacenternet- (a) Energy savings under different (b) Numberofincompleteflows
work topologies. This is because fat-tree is a typical topology numbersofflows
used in DCNs, and can provide equal-length parallel paths
SRSP SRG MRSP MRG SRSP SRG MRSP MRG
between any pair of end hosts, which is very beneficial for 0.50
sfdguuoeennftnitafcewoitrliraasom.rtneelsTdyhpiraeanatctnokflrdaeoonttwhmdepolrydmor:eocmutfehrtsaoiesnnmidgnesgntphdwapepteaohsriseanutdtsrsieoegdgfomaferinndetdloaeocshuhseormsflotsbsfoie.mwdtThuehalparerettoiorocepcneqhossuolsoisiarngereygne- mber of Congested Nodes4352 Energy Savings (%)00000.....3423400555
Nu1 0.20
ment of each resource dimension of each flow is generated
following a normal distribution (in the positive side) where 0 20 40 N6u0mber8 o0f Flo1w0s0 200 300 0.15 2 3Num4ber of5 Reso6urce D7imen8sions9 10
the mean and the variation are all set to be 0.02 to provide (c) Numberofcongestednodes (d) Energy savings under different
large resource demand diversity. The node capacity of each numbersofresourcedimensions
resource dimension is assumed to be normalized to 1. Figure3. PerformancecomparisonforMRGunderthescenarioswherethe
We carry out two groups of simulations for validating networktopologyisgivenbyan8-aryfat-treewith208nodes(128end-hosts
and80packetprocessors).
MRG and HGR respectively: i) For evaluating the perfor-
mance of algorithm MRG, we compare it with three other
is reasonable because MRSP and MRG take into account
algorithms of interest: Single-Resource Shortest Path (SRSP),
more resource dimensions and it is likely that node capacities
Single-Resource Green (SRG), Multi-Resource Shortest Path
are violated more easily than with single-resource solutions.
(MRSP). The efficiency of energy saving of the four algo-
However, when considering only one resource dimension,
rithms are examined on two fat-tree topologies in different
some nodes will be congested due to the neglect of other
scales under different numbers of flow demands. We also
resource dimensions, although more flow demands are likely
exploretheimpactofthenumberofresourcedimensionsunder
tobeassigned.Thenumbersofnodesthatarecongestedunder
certain scenarios. ii) The performance of algorithm HGR is
different numbers of flows are shown in Fig. 3(c).
compared with that of algorithm MRG. We first study the
Impact of the number of resource dimensions. Fig. 3(d)
impact of the number of resource dimensions. Then, under
depicts the simulation results for examining scalability of
certain scenarios, we examine the efficiency of energy saving
MRG with respect to the number of resource dimensions. It
and the running time of both MRG and HGR under different
can be obviously noticed that the energy saving performance
numbers of flow demands. All the results are averaged among
of MRG has a very significant improvement with the increase
20independenttestsandallthefiguresshowwiththeaverage
of the number of resource dimensions and converges to a
and the standard deviation.
high level. This is because with more resource dimensions,
B. Performance of Algorithm MRG the proposed inversion-based node weight assignment can
distinguish nodes from one another more accurately and thus
Energy savings. The simulation results for evaluating the
the path chosen for each flow will be more effective in terms
energy saving performance of MRG are depicted in Fig. 3(a,
of energy saving.
b, c). The energy saving ratio is represented by the number
of inactive nodes divided by the total number of nodes. It can
C. Performance of Algorithm HGR
be seen from Fig. 3(a) that MRG outperforms the other three
algorithms with respect to energy savings under all scenarios. We first compare the scalability of MRG and HGR with
SRSP and MRSP converge to very low energy saving ratios respect to the number of resource dimensions. The simulation
very quickly while SRG and MRG can exploit more energy results are shown in Fig. 4(a). It can be observed that HGR
saving potentials by carefully steering traffic flows. We also outperforms MRG when the number of resource dimensions
comparetheperformanceofallthealgorithmsunderextremely is very small. However, with the increase of the number of
heavy load scenarios. When the number of flow demands resource dimensions, the energy saving performance of HGR
exceedsthecapabilityofthenetwork(andcongestionhappens drops dramatically with a constant rate, while MRG performs
at some critical nodes), MRSP and MRG will block more better and better and converges finally as we have discussed
flows than SRSP and SRG as can be seen in Fig. 3(b). This before. This is mainly because HGR is largely based on the
8
flow using this path. Otherwise, we include also the inactive
0.6 MRG(30) HGR(30) MRG(60) HGR(60) MRG HGR nodeswithweightsassignedandfindoutapathintheresulting
%)0.6 network.Whenaflowcompletesitstransmission,wefocuson
Energy Savings (%)0000....4352 mber of Inactive Nodes (0000....4352 tlwiwinfieoltl-httiebymepneeecss,towomwfoperfllkeol,etweawdsv:eeifnobthureaftmfheserhaoetshrxteitshtteinimynegwea;lflryefoowiarnrsrtithtvhheeeadtlnohflenatogwvw-eolsirvvkeeurdnaytsiflslthohtwoheryest
0.1 Nu0.1 existingshort-livedflowsaregoneandwecarryouttheMRG
0.0 2 3 4 5 6 7 8 9 10 0.0 20 40 60 80 100 120 algorithm to reroute those flows that have long life-times in
Number of Resource Dimensions Number of Flows
(a) Energy savings under different (b) Energy savings under different order to achieve energy efficiency. After this rerouting, the
numbersofresourcedimension numbersofflows routesforthebufferednewflowswillbeassignedaswell.The
node-weight assignment and path selection procedures have
Figure 4. Performance comparison for HGR under the scenarios where the
networktopologyisgivenbyan8-aryfat-treewith208nodes(128end-hosts low complexity. Thus they can be applied to online scenarios
and80packetprocessors). conveniently. Nevertheless, the centralized environment also
TableI
enables parallel acceleration to ensure realtime optimization.
RUNNINGTIMESTATISTICSOFTHEALGORITHMS(UNIT:SECS)
Heterogeneity. For the sake of tractability, we assumed in
#of flows 20 40 60 80 100 120 our model that the resources required for processing all the
packets from a given flow on every node of the flow’s routing
alg.MRG 5.37 16.63 37.00 58.26 92.63 101.89
pathareveryrelatedtothesizeofthepackets,andthushomo-
alg.HGR 0.026 0.078 0.192 0.400 0.647 0.681
geneitycanbeassumed.However,thismaynotbetruewhena
flowrequiresdifferentprocessingfunctionsondifferentnodes.
vector bin packing heuristic which performs well when the
Our model can be extended to the heterogeneous case by
number of dimensions is small due to the greedy manner of
treating each node on the routing path independently, e.g.,
item assignment, but it has very poor scalability with respect
in the MRG algorithm, the weights on nodes can be assigned
to the number of dimensions.
in a hop by hop manner; in the HGR algorithm, each vector
We then choose a fair number of resource dimensions
bin packing instance is solved by having different resource
(K = 3) and compare both the energy saving ratio and the
requirementsfromtheflowdemands.Weleavemoreelaborate
runningtimeofMRGandHGR.Theenergysavingresultsare
solutions for future work.
depictedinFig.4(b).Weobservethatwhenthenumberofflow
demands is not very large, MRG and HGR are comparable
B. Practical Application Scenarios
in terms of energy savings, but HGR suffers from some
Named data networking. To generalize the role of thin
performance degradation when the number of flows is very
waist of the IP architecture, Named Data Networking (NDN)
large. However, HGR compensates this slight loss of energy
was proposed, where packets can name objects other than
efficiency by a very significant reduction on the running time.
communication endpoints. NDN routes and forwards packets
As can be seen from Table I, for a fat-tree with 80 packet
based on names, which requires high-performance processing
processing nodes (i.e., |E| = 192), the running time of HGR
(e.g., prefix matching) capability at network nodes. Moreover,
is around 0.5 percent of that of MRG, which confirms the
it is also useful for a network node to cache the received
lower bound on the speedup Ω(|E|).
data packets in its content store and use them to satisfy future
VII. DISCUSSION requests. These properties make NDN a good application sce-
We discuss in this section some practical issues that are nariofortheNaaSmodel.Asaresult,theproposedGreenNaaS
related to the application of the proposed technique. solutionwillhavethepotentialtobeusedforachievingenergy
efficiency in NDN.
A. Model Extension
Server-centric data center network architecture. In tra-
Dynamic flow joining and leaving. The problem we have ditional switch-centric networks, data packets are transmitted
discussed is for scenarios where a static set of flow demands through only proprietary network devices such as switches or
isgivenaprioriandtheproposedMRGalgorithmisdedicated routes.Withthepossibilityofhavingmulti-portnetworkcards
to solving this problem. However, the reality differs from the onservers,severalserver-centricnetworkarchitecturessuchas
static case by having flows joining and leaving the network BCube [29], SWCube and SWKautz [30] have been recently
dynamically. We observe that although we did not take into proposed for data center networks. In these server-centric
account the dynamic property of the set of flow demands, network architectures, servers are also involved in packet
the MRG algorithm can be extended to the online case due forwarding. By applying more application-specific processing
to its progressive fashion. When a new flow arrives in the functions on packets, these architectures are very easy to be
network, we first check whether the subnetwork formed by extended to adopt the NaaS model and thus GreenNaas can
only the active nodes is capable of carrying this flow. If it be used to save energy in those server-centric data center
is true, we carry out the node-weight assignment and path networks where energy issue is more prominent than in
selection procedures to find out a routing path and route the traditional switch-centric data center networks.
9
Middleboxorchestration.Middleboxesarespecialnetwork [10] L. Wang, F. Zhang, J. A. Aroca, A. V. Vasilakos, K. Zheng, C. Hou,
devices that are responsible for packet processing to provide D.Li,andZ.Liu,“Greendcn:Ageneralframeworkforachievingenergy
efficiencyindatacenternetworks,”IEEEJournalonSelectedAreasin
functionality such as NATs, firewalls or WAN optimizers.
Communications,vol.32,no.1,pp.4–15,2014.
Traditionally, these devices are proprietary and have been [11] R.Sherwood,G.Gibb,K.-K.Yap,G.Appenzeller,M.Casado,N.McK-
implemented on closed hardware platforms which are usually eown,andG.M.Parulkar,“Cantheproductionnetworkbethetestbed?”
inOSDI,2010,pp.365–378.
hard to be extended. To change this situation, recently many
[12] R. Wang, D. Butnariu, and J. Rexford, “Openflow-based server load
proposals have been provided to make middlebox function- balancinggonewild,”inHot-ICE,2011,pp.12–17.
alities software-centric (e.g., [31], [32]). NaaS incorporates
[13] K. R. Fall, G. Iannaccone, M. Manesh, S. Ratnasamy, K. J. Argyraki,
these by leveraging application-specific in-network packet M. Dobrescu, and N. Egi, “Routebricks: enabling general purpose
processing. Although there are still many critical problems, network infrastructure,” Operating Systems Review, vol. 45, no. 1, pp.
112–125,2011.
such as service chain design, needing more research efforts,
[14] K.Jang,S.Han,S.Han,S.B.Moon,andK.Park,“Sslshader:Cheap
GreenNaaS provides a clear insight on how to achieve energy sslaccelerationwithcommodityprocessors,”inNSDI,2011.
efficiency in NaaS systems which is definitely an important [15] M. Dobrescu, K. J. Argyraki, and S. Ratnasamy, “Toward predictable
performance in software packet-processing platforms,” in NSDI, 2012,
issue in the near future.
pp.141–154.
[16] L. Niccolini, G. Iannaccone, S. Ratnasamy, J. Chandrashekar, and
VIII. CONCLUSION L. Rizzo, “Building a power-proportional software router,” in USENIX
AnnualTechnicalConference,2012,pp.89–100.
Westudytheenergy-efficiencymulti-resourceroutingprob-
[17] D. Abts, M. R. Marty, P. M. Wells, P. Klausler, and H. Liu, “Energy
lemwhicharisesfromtherecentlyproposedcloudnetworking proportionaldatacenternetworks,”inISCA,2010,pp.338–347.
model NaaS. This optimization problem differs from the [18] Y. Shang, D. Li, and M. Xu, “Energy-aware routing in data center
network,”inGreenNetworking,2010,pp.1–8.
traditional energy-efficient routing problem by having node
[19] C.Joe-Wong,S.Sen,T.Lan,andM.Chiang,“Multi-resourceallocation:
capacitiesandflowdemandsrepresentedbyvectorsinmultiple Fairness-efficiency tradeoffs in a unifying framework,” in INFOCOM,
dimensions. We provide a simple iterative routing scheme 2012,pp.1206–1214.
[20] W. Wang, B. Li, and B. Liang, “Dominant resource fairness in cloud
which selects flows iteratively to exhaust the residual capac-
computing systems with heterogeneous servers,” in INFOCOM, 2014,
ities in active nodes and assign routes to flows based on the pp.583–591.
distributions of node residual capacities and flow demands. [21] A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker, and
I. Stoica, “Dominant resource fairness: Fair allocation of multiple
To leverage the structural property of data center network
resourcetypes,”inNSDI,2011.
topologies, we also provide a topology-aware heuristic desig- [22] A. Ghodsi, V. Sekar, M. Zaharia, and I. Stoica, “Multi-resource fair
nated to fat-trees, which can provide comparably good energy queueingforpacketprocessing,”inSIGCOMM,2012,pp.1–12.
[23] W. Wang, B. Liang, and B. Li, “Low complexity multi-resource fair
efficiency while significantly reducing the computation time.
queueingwithboundeddelay,”inINFOCOM,2014,pp.1914–1922.
[24] N. Bansal, A. Caprara, and M. Sviridenko, “Improved approximation
REFERENCES algorithmsformultidimensionalbinpackingproblems,”inFOCS,2006,
pp.697–708.
[1] M.Al-Fares,S.Radhakrishnan,B.Raghavan,N.Huang,andA.Vahdat,
“Hedera:Dynamicflowschedulingfordatacenternetworks,”inNSDI, [25] M. Andrews, S. Antonakopoulos, and L. Zhang, “Minimum-cost net-
work design with (dis)economies of scale,” in FOCS, 2010, pp. 585–
2010,pp.281–296.
592.
[2] T.Benson,A.Anand,A.Akella,andM.Zhang,“Microte:finegrained
trafficengineeringfordatacenters,”inCoNEXT,2011,p.8. [26] R.Krishnaswamy,V.Nagarajan,K.Pruhs,andC.Stein,“Clusterbefore
you hallucinate: approximating node-capacitated network design and
[3] OpenFlow.https://www.opennetworking.org/images/stories/downloads/sdn-
energyefficientrouting,”inSTOC,2014,pp.734–743.
resources/white-papers/wp-sdn-newnorm.pdf.
[27] G. J. Woeginger, “There is no asymptotic ptas for two-dimensional
[4] T. Benson, A. Akella, A. Shaikh, and S. Sahu, “Cloudnaas: a cloud
networkingplatformforenterpriseapplications,”inSoCC,2011,p.8. vectorpacking,”Inf.Process.Lett.,vol.64,no.6,pp.293–297,1997.
[28] R.Panigrahy,K.Talwar,L.Uyeda,andI.Wieder,“Heuristicsforvector
[5] P.Costa,M.Migliavacca,P.Pietzuch,andA.L.Wolf,“Naas:Network-
as-a-serviceinthecloud,”inHot-ICE,2012. binpacking,”inESA,2011.
[29] C.Guo,G.Lu,D.Li,H.Wu,X.Zhang,Y.Shi,C.Tian,Y.Zhang,and
[6] P. Costa, A. Donnelly, A. I. T. Rowstron, and G. O’Shea, “Camdoop:
Exploiting in-network aggregation for big data applications,” in NSDI, S.Lu,“Bcube:ahighperformance,server-centricnetworkarchitecture
formodulardatacenters,”inSIGCOMM,2009,pp.63–74.
2012,pp.29–42.
[30] D. Li and J. Wu, “On the design and analysis of data center network
[7] B. Heller, S. Seetharaman, P. Mahadevan, Y. Yiakoumis, P. Sharma,
architecturesforinterconnectingdual-portservers,”inINFOCOM,2014,
S.Banerjee,andN.McKeown,“Elastictree:Savingenergyindatacenter
networks,”inNSDI,2010,pp.249–264. pp.1851–1859.
[31] V. Sekar, N. Egi, S. Ratnasamy, M. K. Reiter, and G. Shi, “Design
[8] N. Vasic, P. Bhurat, D. M. Novakovic, M. Canini, S. Shekhar, and
D. Kostic, “Identifying and using energy-critical paths,” in CoNEXT, andimplementationofaconsolidatedmiddleboxarchitecture,”inNSDI,
2012,pp.323–336.
2011,p.18.
[32] A.Gember-Jacobson,R.Viswanathan,C.Prakash,R.Grandl,J.Khalid,
[9] X. Wang, Y. Yao, X. Wang, K. Lu, and Q. Cao, “Carpo: Correlation-
awarepoweroptimizationindatacenternetworks,”inINFOCOM,2012, S.Das,andA.Akella,“Opennf:enablinginnovationinnetworkfunction
control,”inSIGCOMM,2014.
pp.1125–1133.
