Dynamic Environments for Virtual Machine Placement considering Elasticity and Overbooking

0.34 MB·

by  Jammily Ortigoza

Dynamic Environments for Virtual Machine
Placement considering Elasticity and Overbooking
Jammily Ortigoza Fabio Lo´pez-Pires Benjam´ın Bara´n
Science and Technology School Itaipu Technological Park National University of Asuncio´n
Catholic University of Asuncio´n National University of Asuncio´n Catholic University of Asuncio´n
[email protected] [email protected] [email protected]
Paraguay Paraguay Paraguay
6
Abstract—Cloud computing datacenters provide millions of For virtualized datacenters with deployments of VMs that
1
0 virtualmachinesinactualcloudmarkets.Inthiscontext,Virtual rarely change its configuration over time, a static (offline)
Machine Placement (VMP) is one of the most challenging
2 formulation of the VMP problem may be appropriate [50].
problems in cloud infrastructure management, considering the
Additionally, in virtualized datacenters where a small number
n large number of possible optimization criteria and different
a formulations that could be studied. Considering the on-demand of VMs are created and destroyed, a semi-static formulation
J model of cloud computing, the VMP problem should be solved of the VMP problem could be acceptable (e.g. consolidating

8 dynamically to efficiently attend typical workload of modern VMs every day at midnight). On the other hand, considering
applications. This work proposes a taxonomy in order to un-
thetodaymorerealisticon-demandmodelofcloudcomputing
] derstand possible challenges for Cloud Service Providers (CSPs)
C with dynamic resource provisioning, a static (or semi-static)
in dynamic environments, based on the most relevant dynamic
D parameters studied so far in the VMP literature. Based on formulation of the VMP problem can result in under-optimal
the proposed taxonomy, several unexplored environments have solutions after a short period of time. Clearly, the VMP pro-
.
s been identified. To further study those research opportunities, blem for cloud computing environments must be formulated
c sample workload traces for each particular environment are
asapuredynamic(online)optimizationproblemtoefficiently
[
required;therefore,basicexamplesillustrateapreliminarywork
on dynamic workload trace generation. attend dynamic workload of modern applications [49].
1
v A. Background and Motivation
1 I. INTRODUCTION
8 TheVMPproblemhasbeenextensivelystudiedandseveral
8 The rapid demand growth for computational resources in surveys have already been presented in the VMP literature.
1 modern business and scientific applications presents several Existing surveys focus on specific issues such as: (1) energy-
0 challenges for design, implementation and management of efficient techniques applied to the problem [4], [60], (2) parti-
.
1 scalable datacenters to meet the requirements of customers culararchitectureswheretheVMPproblemisapplied,asfede-
0 in a competitive and efficient way [67]. ratedclouds[22],and(3)methodsforcomparingperformance
6
Considering the evolution of resource provisioning, three ofplacementalgorithmsinlargeon-demandclouds[52].None
1
main models could be identified: (1) traditional provisioning of the mentioned surveys presented a general and extensive
:
v of resources with independent physical hardware, (2) modern study of a large part of the VMP literature. In consequence,
i
X provisioning of shared resources through virtualized hardware Lo´pez-Pires and Bara´n presented in [49] an extensive up-to-
and (3) trending dynamic provisioning of resources through date survey of the most relevant VMP literature and proposed
r
a a cloud computing model [6]. The traditional provisioning a novel taxonomy in order to identify research opportunities
environment has mostly evolved to a virtualized provisioning defining a general vision on this problem.
of resources in current datacenters, considering its advantages According to [49], the VMP problem is mostly formulated
for management and resource utilization. as an online optimization problem, where live migration
Virtualization in modern datacenters introduces complex techniques allow VMs to be dynamically consolidated on ne-
management decisions related to the placement of virtual cessary PMs according to dynamic requirements of resources.
machines (VMs) into the available physical machines (PMs). The most studied environment for online formulations of the
In this context, Virtual Machine Placement (VMP) represents VMPproblemconsidersthatVMsaredynamicallycreatedand
the process of selecting which VMs should be executed in a destroyed [49]. To the best of the authors’ knowledge, there
given set of PMs of a datacenter [47]. The VMP problem is is no published work presenting a detailed characterization of
mostly formulated as a combinatorial optimization problem, possible dynamic environments for the VMP problem.
representing one of the most challenging problems in virtua- Clearly, a deeper research of possible dynamic parameters
lized datacenters infrastructure management, considering the in cloud computing is necessary in order to propose holistic
large number of possible optimization criteria and different and more realistic environments for the formulation of the
formulations that could be studied [49]. VMP problem for cloud computing datacenters.
Consequently, this work complements the taxonomy pre- III. CLASSIFICATIONCRITERIA
sented by the authors in [49] focusing specifically on dy-
This work identified the following dynamic parameters:
namic formulations of the VMP problem from the providers’
• resource capacities of VMs (vertical elasticity);
perspective, proposing a taxonomy in order to understand
• number of VMs of a service (horizontal elasticity);
possible challenges for Cloud Service Providers (CSPs) in
• utilization of resources of VMs (related to overbooking).
dynamicenvironmentstoefficientlyattendcustomers’requests
Consequently, dynamic environments for online formula-
for virtual resources, based on the most relevant dynamic
tions of the provider-oriented VMP problem may be classi-
parametersstudiedsofarintheVMPliterature.Thetaxonomy
fied by one or more of the following classification criteria:
proposed in this work must be jointly studied with the taxo-
(1) elasticity and (2) overbooking.
nomy first proposed in [49] in order to represent a complete
VMP problem. A. Elasticity
The remainder of this paper is organized as follows:
Considering the dynamic workload of modern cloud appli-
Section II details the literature selection process considered
cations,proactiveelasticityisaveryimportantissuetoaddress
in this work, while Section III introduces the classification
for CSPs in order to deal with under-provisioning (saturation)
criteria of the proposed taxonomy. Section IV presents the
and over-provisioning (under-utilization) of cloud resources
proposed taxonomy detailing the mathematical notation and
[3]. Under-provisioning can cause SLA violations, impacting
basicexamplesoftheidentifieddynamicenvironmentsforthe
directly on economical revenue while over-provisioning can
VMP problem. Based on the proposed taxonomy, Section V
cause inefficient utilization of resources, directly impacting
presents a preliminary work on generation of workload traces
on resource utilization and energy consumption.
for the identified dynamic environments. Finally, conclusions
Research articles considering online formulations of the
and future work are left to Section VI.
provider-oriented VMP literature have already studied two
II. REVIEWEDLITERATURE types of elasticity: vertical and horizontal (see Figure 1).
Vertical elasticity can be defined as the ability of cloud
A. Keywords Search
services to dynamically change capacities of virtual resources
The selection process of relevant articles started with a
(e.g. CPU and RAM memory) inside a VM, while horizontal
search for research articles from Google Scholar database
elasticity can be defined as the ability of cloud services to
[scholar.google.com]withatleastoneofthefollowingselected
dynamically adjust the number of VMs [73].
keywords in the article title: (1) virtual machine placement,
It should be noted that from a CSP perspective, cloud
(2) vm placement, (3) virtual machine consolidation, (4) vm
services considering elasticity should be more important (i.e.
consolidation or (5) server consolidation.
higherlevelofSLA)thanothernon-elasticcloudservices.An
B. Publisher Filtering elastic cloud application could request additional resources to
scale up the applications’ resources and a CSP must consider
Considering the large number of results from keywords
more important the mentioned request than a request from a
searchstep,theliteratureselectionprocessfocusedonresearch
not elastic cloud service.
articles from the following well-known publishers: (1) ACM,
Implementing vertical elasticity requires shorter time of
(2) IEEE, (3) Elsevier and (4) Springer. This filtering step
service reconfiguration than horizontal elasticity, but with a
resulted in a reduction from 446 to 172 research articles. A
higher migration cost. On the other hand, horizontal elasticity
detailed list of the 172 resulting articles can be found in [48].
enables stronger high availability than vertical elasticity, but a
C. Abstract Reading
coordination overhead is required and infrastructure comple-
Considering the 172 resulting articles from the publisher xity increases [73].
filteringstep,areadingoftheabstractswasperformedinorder
to identify the most relevant articles that specifically study
the VMP problem. Additionally, short papers (i.e. research
articles with less than 6 pages) were removed from the V V
1 Horizontal 2
selectedliterature,resultingin84selectedarticlesoftheVMP y Elasticity
literature. A detailed list of the 84 resulting articles can be tic
found in [48]. itsa
lE
la
D. Online Formulations for Provider-oriented VMP Problem citreV
Basedonthe84studiedarticlesaddressedin[49],thiswork
selected the 64 articles that proposed online formulations for
theVMPproblemfromtheproviders’perspective,considering
H H
1 2
the relevance of this type of environments for actual cloud
computing providers. An in-depth reading of this universe of
Figure1. Verticalandhorizontalelasticity.Verticalelasticitydynamicallyad-
64articleswasperformedwiththeaimofidentifyingthemost juststhecapacitiesofvirtualresourcesinsideaVMwhilehorizontalelasticity
relevant dynamic parameters. dynamicallyadjuststhenumberofVMs(e.g.indistributedapplications).
B. Overbooking It should be mentioned that all the identified environments
in this work consider that VMs are dynamically created and
Resources overbooking can make cloud services more
destroyed. According to the identified dynamic environments,
profitable for CSPs, overlaying requested virtual resources
thesimplestenvironmentisEnvironment(0,0),whilethemost
onto physical resources at a higher ratio than 1:1 [29]. Online
complex environment is Environment (3,3). Additionally, the
formulations of the provider-oriented VMP considering over-
proposed taxonomy showed that 50% of the articles studied
booking include particular considerations to efficiently attend
Environment(0,1)while39%ofthestudiedarticlesconsidered
customers’ requirements, enforcing SLAs.
Environment(0,0),representingthemoststudiedenvironments
Considering the dynamic workload of cloud applications
in the considered literature. Several unexplored environments
and services, virtual resources of VMs are also dynamically
were also identified, as detailed in the following subsections.
used giving space to re-utilization of idle resources that
were already reserved. Research articles considering online
formulations of the provider-oriented VMP literature have A. Cloud Service and Environment Notation
already studied two types of overbooking: server and network
CSPs dynamically receive requests for the placement of
resources overbooking.
cloud services with different characteristics according to the
In this context, CSPs should measure the utilization of
classification criteria presented in Section III, representing
resources of VMs in order to correctly manage the over-
real-world environments and generalizing the deployment of
booking with the available physical resources, minimizing
cloud services in several possible cloud architectures (e.g.
SLA violations. Monitoring utilization of virtual resources
single-cloud, distributed-cloud or federated-cloud). Cloud ser-
and workload of cloud applications and services also helps
vices may represent simple services such as Domain Name
CSPs to consider forecasting techniques for approximating in
Service (DNS) or complex multi-tier elastic applications.
advance the required management actions (e.g. migrations of
A cloud service is composed by a set of VMs, where each
VMs) for the consolidation process, to reduce resource under-
VM of a cloud service could be located for its execution in
provisioning [19], [77].
different cloud datacenters according to the customers prefe-
IV. PROPOSEDTAXONOMY rences or requirements (e.g. legal issues or high-availability).
Based on the universe of 64 studied research articles, Configuration of VMs of a cloud service changes dy-
dynamicenvironmentsforonlineformulationsoftheprovider- namically when elasticity is considered. On the other hand,
oriented VMP problem may be classified by one or more utilization of virtual resources change dynamically according
of the following classification criteria: (1) elasticity and (2) tothedemandwhenoverbookingisconsidered;otherwise,the
overbooking, as presented in Section III. utilization of each virtual resource is considered at 100%.
The proposed taxonomy is presented in Figure 3 as a two- Formally, a cloud service Sb can be distributed across
dimensional coordinate axis where each dimension represents different possible cloud datacenters. Each cloud datacenter
a classification criteria (elasticity and overbooking). DCc hosts VMs Vc(cid:48)j associated to different cloud services. A
First, dynamic environments could be formulated conside- VM Vc(cid:48)j associated to a service Sb is denoted as Vb(cid:48)c(cid:48)j.
ring one of the following elasticity values:
where:
• elasticity=0: no elasticity;
• elasticity=1: horizontal elasticity; S : Cloud service b;
b
• elasticity=2: vertical elasticity; DC : Cloud datacenter c;
c
• elasticity=3: horizontal and vertical elasticity. mDC : Number of VMs V in DC ;
c j c
Additionally, identified dynamic environments may also mS : Number of VMs V in S ;
b j b
consider one of the following overbooking values: V(cid:48) : V in DC ;
cj j c
• overbooking=0: no overbooking; Vb(cid:48)c(cid:48)j: Vj in DCc from service Sb.
• overbooking=1: server resources overbooking;
• overbooking=2: network resources overbooking;
Figure 2 presents a basic example of a cloud service S ,
• overbooking=3: server and network overbooking. 1
distributed across 2 cloud datacenters DC and DC and
Based on the combinations of the possible values of the 1 2
using 4 VMs: V(cid:48)(cid:48) , V(cid:48)(cid:48) , V(cid:48)(cid:48) , V(cid:48)(cid:48) . These cloud datacenters
classification criteria (elasticity, overbooking), the proposed 111 112 121 122
couldrepresentgeo-distributeddatacentersownedbyoneCSP
taxonomy identified 16 different possible environments (see
or a federated-cloud with two different CSPs. Each cloud
Figure 3). Considering this representation, each identified
datacenter hosts 2 VMs of S : V(cid:48) and V(cid:48) represent V and
dynamic environment is denoted by its elasticity and over- 1 11 12 1
V in DC respectively (analogously DC hosts 2 VMs).
booking coordinates. For example, Environment (0,0) denotes 2 1 2
Complementing the above notation, each cloud datacenter
adynamicenvironmentthatdoesnotconsiderneitheranytype
DC may be represented as:
of elasticity nor any type of overbooking, while Environment c
(1,3)denotesadynamicenvironmentthatconsidershorizontal
elasticity with both types of overbooking. DC ={V(cid:48) },j ∈{1,2,...,mDC } (1)
c cj c
For simplicity, this work considers only processing, me- B. Dynamic Environment Classification
mory and network resources for a VM, but the notation is
Based on the notation previously presented in Section
general enough for considering any set of virtual resources.
IV-A, the 16 identified environments (see Figure 3) present
V(cid:48)(cid:48) ={Vcpu(cid:48)(cid:48) ,Vram(cid:48)(cid:48) ,Vnet(cid:48)(cid:48) ,R(cid:48)(cid:48) , particular considerations and different time variables may be
bcj bcj bcj bcj bcj
(2) defined for a correct characterization. A summary of the time
SLA(cid:48)(cid:48) ,t ,t }
bcj init end variables is presented in Table I. Next, the identified dynamic
where: environments are presented, enumerated by its elasticity and
overbooking coordinates.
V(cid:48)(cid:48) : V in DC from service S ;
Vbccpju(cid:48)(cid:48) : Prjocessingcrequirements ofbV(cid:48)(cid:48) in [ECU]; • (0,0) No Elasticity (0), No Overbooking (0): It repre-
bcj bcj sents the most basic dynamic environment identified for
Vram(cid:48)(cid:48) : Memory requirements of V(cid:48)(cid:48) in [GB];
bcj bcj solving the provider-oriented VMP problem. The CSP
Vnet(cid:48)(cid:48) : Network requirements of V(cid:48)(cid:48) in [Mbps];
bcj bcj have to attend the placement of cloud services that are
R(cid:48)(cid:48) : Economical revenue for locating V(cid:48)(cid:48) in [$];
bcj bcj dynamically created and destroyed in function of time t.
SLA(cid:48)(cid:48) : SLA of V(cid:48)(cid:48) . SLA(cid:48)(cid:48) ∈ {1,..,s};
bcj bcj bcj From the studied universe of 64 articles, 39% proposed
s: Highest priority level of SLAs;
formulations of the VMP problem considering this basic
t : Initial discrete time when V(cid:48)(cid:48) is executed;
init bcj environment (Figure 3).
tend: Final discrete time when Vb(cid:48)c(cid:48)j is executed. • (0,1) No Elasticity (0), Server Resources Overbooking
(1): According to the studied articles, the provider-
Utilization of the resources of each V(cid:48)(cid:48) is represented by: oriented VMP problem is mostly formulated considering
bcj overbookingofserverresources(i.e.processing,memory
and storage) without considering neither horizontal nor
U(cid:48)(cid:48) ={Ucpu(cid:48)(cid:48) ,Uram(cid:48)(cid:48) ,Unet(cid:48)(cid:48) } (3)
bcj bcj bcj bcj vertical elasticity. This environment represents 50% of
the studied universe of 64 articles (see Figure 3). For
where:
this particular environment, CSPs must monitor the dy-
U(cid:48)(cid:48) : Utilization of requirement V(cid:48)(cid:48) ; namic utilization of virtual server resources for a safe
bcj bcj
Ucpu(cid:48)(cid:48) : Utilization of Vcpu(cid:48)(cid:48) in [ECU]; overbooking. The following variables must be defined as
bcj bcj
Uram(cid:48)(cid:48) : Utilization of Vram(cid:48)(cid:48) in [GB]; a function of time: Ucpu(cid:48)(cid:48) (t) and Uram(cid:48)(cid:48) (t).
bcj bcj bcj bcj
Unet(cid:48)(cid:48) : Utilization of Vnet(cid:48)(cid:48) in [Mbps]. • (0,2) No Elasticity (0), Network Resources Overbooking
bcj bcj
(2):Overbookingcouldbealsoconsideredexclusivelyfor
virtual network resources. Analogously to the Environ-
Note that in practical applications Ucpu(cid:48)(cid:48) is lower than
bcj ment (0,1), CSPs must monitor the dynamic utilization
Vcpu(cid:48)(cid:48) (Ucpu(cid:48)(cid:48) ≤ Vcpu(cid:48)(cid:48) ), giving place to CPU over-
bcj bcj bcj of virtual network resources for a safe overbooking.
booking of resources. The same overbooking situation may
Consequently, the utilization of network resources is
occur for other resources. defined as a time variable: Unet(cid:48)(cid:48) (t). This environment
bcj
Each of the 16 identified environments (see Figure 3)
represents only 5% of the studied articles (see Figure 3).
considers different parameters that dynamically change as a
• (0,3) No Elasticity (0), Server and Network Resources
function of time t, giving place to possible different notations
Overbooking (3): Representing the most complex envi-
for each environment. The following subsections detail each
ronment for overbooking, this dynamic environment is
of the identified dynamic environment, presenting particular
identified as a research opportunity, considering that no
notation for its characterization.
studied article proposed a formulation of the provider-
orientedVMPproblemthatjointlyconsidersoverbooking
of server and network resources without elasticity (see
Cloud Service (S)
1 1 Figure 3). For this particular environment, CSPs must

S1={V’11,V’12,V’21,V'22} monitorthedynamicutilizationofbothvirtualserverand

network resources for a safe overbooking. Consequently,
={V’ ’ ,V’ ’ ,V’ ’ ,V'’ }
111 112 121 122 Ucpu(cid:48)(cid:48) (t), Uram(cid:48)(cid:48) (t) and Unet(cid:48)(cid:48) (t) are defined as
bcj bcj bcj
a function of time.
(VV’111 ) (VV’211 ) • (1,0) Horizontal Elasticity (1), No Overbooking (0):
Elasticity could be considered in order to efficiently
V2 V2 attend the dynamic demand of resources according to
(V’ ) (V’ )
12 22 a SLA associated to a given cloud service. A dynamic
Cloud Datacenter Cloud Datacenter environmentthatconsidershorizontalelasticityrepresents
1 2
(DC1) (DC2) particular considerations associated to scaling up and
down the number of requested VMs that composes a
Figure2. Exampleofacloudserviceconsideredinthiswork. cloud service.
3 UnexploredEnvironment UnexploredEnvironment UnexploredEnvironment UnexploredEnvironment
2 [5],[39],[66] UnexploredEnvironment UnexploredEnvironment UnexploredEnvironment
g
n [8], [77], [36], [34], [54], [10],
oki [38], [57], [1], [71], [58], [51],
bo 1 [65], [16], [2], [19], [33], [17], UnexploredEnvironment [21] UnexploredEnvironment
r [75], [37], [24], [59], [13], [27],
ve [32], [20], [14], [41], [63], [70],
O [68],[56]
[72], [9], [76], [31], [40], [64],
0 [61], [45], [15], [23], [26], [43], UnexploredEnvironment [42] [73],[12]
[18], [69], [62], [30], [25], [11],
[46],[35],[44],[7],[74],[53],[28]
0 1 2 3
Elasticity
Figure3. Taxonomyondynamicenvironmentsforauniverseof64selectedpapers.
Determining when to scale is not considered a responsi- Itshouldbementionedthatthenotationpresentedinthis
bility of the CSPs and it is out of the scope of this work. sectionisgeneralenoughtoconsideranyotherresources
In this context, the number of VMs of cloud services for vertical elasticity such as Vnet(cid:48)(cid:48) (t) just to cite one.
bcj
is a time variable: mS ≤ mS (t) ≤ mS . This Accordingtothestudiedarticles,only[42](1.5%)studied
bmin b bmax
particular environment is also identified as a research this environment (see Figure 3).
opportunity, representing the most basic environment for • (2,1) Vertical Elasticity (2), Server Resources Overboo-
solving problems considering horizontal elasticity for king (1): Additionally to vertical elasticity, a dynamic
parallel applications such as MapReduce jobs. environment could also include overbooking of server
• (1,1) Horizontal Elasticity (1), Server Resources Over- resources. For these particular environment, CSPs must
booking (1): Additionally to horizontal elasticity, a dy- monitorthedynamicutilizationofvirtualserverresources
namic environment could also include overbooking of forasafeoverbooking.Consequently,thefollowingvari-
server resources. The following variables are defined as ables are defined as a function of time: Ucpu(cid:48)(cid:48) (t),
bcj
afunctionoftime:Ucpu(cid:48)(cid:48) (t),Uram(cid:48)(cid:48) (t)andmS (t). Uram(cid:48)(cid:48) (t), Vcpu(cid:48)(cid:48) (t) and Vram(cid:48)(cid:48) (t). According to
bcj bcj b bcj bcj bcj
This environment also represents a research opportunity. the studied articles, only [21] (1.5%) studied this envi-
• (1,2) Horizontal Elasticity (1), Network Resources Over- ronment (see Figure 3).
booking(2):AnalogouslytotheEnvironment(1,1),CSPs • (2,2) Vertical Elasticity (2), Network Resources Over-
must monitor the dynamic utilization of virtual net- booking (2): Analogously to the Environment (2,1),
work resources for a safe overbooking. Consequently, Unet(cid:48)(cid:48) (t) is defined as a time variable, additionally to
bcj
Unet(cid:48)(cid:48) (t) is defined as a time variable, additionally Vcpu(cid:48)(cid:48) (t)andVram(cid:48)(cid:48) (t).Thisparticularenvironment
bcj bcj bcj
to mS (t). This environment is identified as a research represents a research opportunity, considering that no
b
opportunity, considering that no studied article considers studied article proposed a formulation of the provider-
overbooking of network resources with horizontal elasti- oriented VMP in this particular environment.
city. • (2,3) Vertical Elasticity (2), Server and Network Re-
• (1,3) Horizontal Elasticity (1), Server and Network Re- sources Overbooking (3): No studied article proposed
sources Overbooking (3): No studied article proposed a formulation of the provider-oriented VMP problem
a formulation of the provider-oriented VMP problem that jointly considers overbooking of server and net-
that jointly considers overbooking of server and network work resources with vertical elasticity. For this particular
resources with horizontal elasticity, representing a re- environment, the following variables are defined as a
search opportunity. For this particular environment, the function of time: Ucpu(cid:48)(cid:48) (t), Uram(cid:48)(cid:48) (t), Unet(cid:48)(cid:48) (t),
bcj bcj bcj
following variables are defined as a function of time: Vcpu(cid:48)(cid:48) (t) and Vram(cid:48)(cid:48) (t).
bcj bcj
Ucpu(cid:48)b(cid:48)cj(t), Uram(cid:48)b(cid:48)cj(t), Unet(cid:48)b(cid:48)cj(t) and mSb(t). • (3,0) Horizontal and Vertical Elasticity (3), No Over-
• (2,0) Vertical Elasticity (2), No Overbooking (0): As booking (0): Both types of elasticity lead to different
mentioned before, elasticity could be considered in order impacts for cloud datacenters infrastructure management
to efficiently attend the dynamic demand of resources and respond to different requirements of customers’
according to a SLA associated to a cloud service. A applications.Definitely,inrealworldenvironmentsCSPs
dynamicenvironmentthatconsidersverticalelasticityre- should be able to solve the VMP problem considering
presentsparticularconsiderationsassociatedtothevirtual formulations that jointly implement both horizontal and
resources capacities of requested VMs that composes vertical elasticity for cloud services. In this context, [73]
a cloud service. This work considers processing and and [12] proposed different approaches for dealing with
memory requirements as time variables: Vcpu(cid:48)(cid:48) (t) and thesechallenges,representing3%ofthestudieduniverse
bcj
Vram(cid:48)(cid:48) (t). (see Figure 3).
bcj
TableI
In this environment of both mixed elasticity types, the
SUMMARYOFTIMEVARIABLESFORTHE16IDENTIFIEDDYNAMIC
followingtimevariablesaredefined:mSb(t),Vcpu(cid:48)b(cid:48)cj(t) ENVIRONMENTS.
and Vram(cid:48)(cid:48) (t).
bcj Env. ElasticityType OverbookingType TimeVariables
• (3,1) Horizontal and Vertical Elasticity (3), Server Re-
(0,0) NotConsidered NotConsidered -
sources Overbooking (1): Additionally to horizontal and -Ucpu(cid:48)(cid:48) (t)
vertical elasticity, a dynamic environment could also (0,1) NotConsidered Server -Uramb(cid:48)c(cid:48)j (t)
bcj
include overbooking of server resources. For these par- (0,2) NotConsidered Network -Unet(cid:48)(cid:48) (t)
bcj
ticular environment, CSPs must monitor the dynamic -Ucpu(cid:48)b(cid:48)cj(t)
utilization of virtual server resources for a safe overboo- (0,3) NotConsidered ServerandNetwork -Uram(cid:48)b(cid:48)cj(t)
-Unet(cid:48)(cid:48) (t)
king. Consequently, the following variables are defined bcj
as a function of time: Ucpu(cid:48)b(cid:48)cj(t), Uram(cid:48)b(cid:48)cj(t), mSb(t), (1,0) Horizontal NotConsidered --mmSSbb((tt))
Vcpu(cid:48)b(cid:48)cj(t) and Vram(cid:48)b(cid:48)cj(t). (1,1) Horizontal Server -Ucpu(cid:48)b(cid:48)cj(t)
• (3,2) Horizontal and Vertical Elasticity (3), Network -Uram(cid:48)b(cid:48)cj(t)
Resources Overbooking (2): Analogously to the Envi- (1,2) Horizontal Network --mUnSebt((cid:48)t(cid:48)) (t)
ronment (3,1), the following variables are defined in bcj
-mSb(t)
Ufunncetti(cid:48)o(cid:48)n(to)f.tTimhies: pmaSrtbic(ut)la,rVecnpvuir(cid:48)bo(cid:48)cjn(mt)e,nVt rreapmre(cid:48)bs(cid:48)cejn(tt)aarned- (1,3) Horizontal ServerandNetwork --UUcrpaum(cid:48)b(cid:48)(cid:48)c(cid:48)j((t)t)
bcj bcj
search opportunity, considering that no studied article -Unet(cid:48)b(cid:48)cj(t)
-Vcpu(cid:48)(cid:48) (t)
proposed a formulation of the provider-oriented VMP in (2,0) Vertical NotConsidered -Vramb(cid:48)c(cid:48)j (t)
this particular environment. bcj
-Vcpu(cid:48)(cid:48) (t)
• (3,3) Horizontal and Vertical Elasticity (3), Server and -Vramb(cid:48)c(cid:48)j (t)
Network Resources Overbooking (3): Considering both (2,1) Vertical Server -Ucpu(cid:48)(cid:48)bcj(t)
bcj
types of elasticity and both types of overbooking re- -Uram(cid:48)(cid:48) (t)
bcj
present the most complex environment identified in this -Vcpu(cid:48)b(cid:48)cj(t)
(2,2) Vertical Network -Vram(cid:48)(cid:48) (t)
work. CSPs efficiently solving formulations of the VMP bcj
-Unet(cid:48)(cid:48) (t)
problem in this complex dynamic environment will re- bcj
-Vcpu(cid:48)(cid:48) (t)
bcj
present a considerable advance on this research area and -Vram(cid:48)(cid:48) (t)
bcj
its cloud datacenters will be able to scale according to (2,3) Vertical ServerandNetwork -Ucpu(cid:48)(cid:48) (t)
bcj
trending types of requirements with sufficient flexibility. -Uram(cid:48)(cid:48) (t)
bcj
Asthemostgeneralenvironment,thefollowingvariables -Unet(cid:48)b(cid:48)cj(t)
are defined in function of time for characterizing this -mSb(t)
complexenvironment:mSb(t),Vcpu(cid:48)b(cid:48)cj(t),Vram(cid:48)b(cid:48)cj(t), (3,0) HorizontalandVertical NotConsidered --VVcrpaum(cid:48)b(cid:48)(cid:48)bc(cid:48)cjj((t)t)
Ucpu(cid:48)b(cid:48)cj(t), Uram(cid:48)b(cid:48)cj(t) and Unet(cid:48)b(cid:48)cj(t). A recommen- -mSb(t)
dedpathforresearchisexploringandaddressingchallen- -Vcpu(cid:48)b(cid:48)cj(t)
ges of particular environments identified as research op- (3,1) HorizontalandVertical Server -Vram(cid:48)b(cid:48)cj(t)
-Ucpu(cid:48)(cid:48) (t)
portunitiesbeforeconsideringthisadvancedandcomplete bcj
-Uram(cid:48)(cid:48) (t)
dynamic environment for solving the provider-oriented bcj
-mSb(t)
VMP problem. -Vcpu(cid:48)(cid:48) (t)
(3,2) HorizontalandVertical Network -Vramb(cid:48)c(cid:48)j (t)
bcj
C. Dynamic Environment Examples -Unet(cid:48)(cid:48) (t)
bcj
Duetospacelimitations,thisworkfocusonarepresentative -mSb(t)
-Vcpu(cid:48)(cid:48) (t)
bcj
example of the Environment (1,0), representing an elastic -Vram(cid:48)(cid:48) (t)
application implementing horizontal elasticity (see Figure 4). (3,3) HorizontalandVertical ServerandNetwork -Ucpu(cid:48)(cid:48)bcj(t)
bcj
Interested readers can refer to [55] for a complete set of -Uram(cid:48)b(cid:48)cj(t)
-Unet(cid:48)(cid:48) (t)
examples of the 16 identified dynamic environments. bcj
ItisimportanttorememberthatEnvironment(1,0)includes
onlyhorizontalelasticity,butverticalelasticityaswellasboth
types of overbooking (server and network resources) can be The horizontal elasticity of the example in Figure 4 can be
observed in the workload trace presented in Section V-A. observed considering that initially, cloud service S starts in
1
Figure 4 presents different levels of detail for the envi- t = 0 requesting 2 VMs (in blue) across cloud datacenters
ronment. The CSP level (CSP ) represents the requests that DC and DC from t=0 to t=5. Assuming an increasing
1 1 2
CSPs receive for the placement of cloud services (or VMs) in demandforresources,S scalesupthenumberofVMsadding
1
thePMsoftheavailableclouddatacenters.Next,cloudservice 1 VM hosted at cloud datacenter DC (in brown) at t=1. In
1
level(S )detailsrequestedresourcesofcloudservicesateach t = 2, S scales up the number of VMs adding 1 more VM
1 1
discrete time. Cloud datacenter levels (DC and DC ) detail hostedatclouddatacenterDC (inbrown)resultingin4VMs
1 2 2
resources of cloud services for each cloud datacenter. for attending the demand for resources from t = 2 to t = 3.
CSP1 V1(cid:48)1(cid:48)1,V1(cid:48)2(cid:48)1 V1(cid:48)1(cid:48)1,V1(cid:48)2(cid:48)1,V1(cid:48)1(cid:48)2 V1(cid:48)1(cid:48)1,V1(cid:48)2(cid:48)1,V1(cid:48)1(cid:48)2,V1(cid:48)2(cid:48)2 V1(cid:48)1(cid:48)1,V1(cid:48)2(cid:48)1,V1(cid:48)1(cid:48)2,V1(cid:48)2(cid:48)2 V1(cid:48)1(cid:48)1,V1(cid:48)2(cid:48)1
S1 V1(cid:48)1,V2(cid:48)1 V1(cid:48)1,V2(cid:48)1,V1(cid:48)2 V1(cid:48)1,V2(cid:48)1,V1(cid:48)2,V2(cid:48)2 V1(cid:48)1,V2(cid:48)1,V1(cid:48)2,V2(cid:48)2 V1(cid:48)1,V2(cid:48)1
DC1 V1 V1,V2 V1,V2 V1,V2 V1
DC2 V1 V1 V1,V2 V1,V2 V1
Events +S1 S1:+V1(cid:48)1(cid:48)2 S1:+V1(cid:48)2(cid:48)2 S1:−V1(cid:48)1(cid:48)2−V1(cid:48)2(cid:48)2 −S1
0 1 2 3 4 5 (t)
Figure4. BasicexampleofEnvironment(1,0)
ThecloudserviceS returnstoitsinitialconfigurationscaling It is important to mention that additionally, an user of the
1
down to 2 VMs at t = 4, assuming a decreasing demand for generator is also able to include real-world workload traces,
resources, finishing the requests for resources at t=5. extending or reducing the trace to specific requirements for
the experiments. In this case, the workload generator could
V. WORKLOADTRACESFORDYNAMICENVIRONMENTS generate synthetic workloads based on real-world workloads
adjusting the workload to a specific size of the problem
Based on the proposed taxonomy, several research oppor-
instance (e.g. workload trace duration), mantaining character-
tunities have been identified (see Section IV-B). Therefore,
istics of the real-world trace (e.g. probability distribution).
sample workload traces for each particular environment are
required in order to: (1) explore the challenges associated
A. Workload Trace Generation Example
to each environment, (2) propose formulations and test al-
As an example of utilization of the workload trace ge-
gorithms that solve these challenges with different workload
nerator, Table II presents the input data considered for the
types and (3) effectively compare performance and quality of
generation of the workload trace presented in Table III. Due
different algorithms with reproducible experiments.
to space limitation as well as similarity of the structure of
As identified in [48], there is no existing testbed problem
the workload traces of the different environments, Table III
instances for the VMP that can today be used as a world
representsonlyabasicexampleofthemostcomplexdynamic
accepted benchmark. Consequently, the authors are working
environment identified in this work, Environment (3,3). This
on a workload trace generator for the VMP problem to be
example includes all possible dynamic parameters (resource
abletogeneratedifferentinstancesforexperimentaltestsbased
capacities of VMs, number of VMs of a cloud service and
on the dynamic environments proposed in this work. A brief
utilization of resources of VMs). Interested readers can refer
introductionofpreliminaryresultsispresentedinthissection.
to [55] for more detailed examples of workload traces for the
The proposed workload trace generator for the VMP pro-
16 different dynamic environments.
blem considers the following input data (see Table II):
AccordingtoTableII,alltheconsideredvaluesareselected
• workload trace duration, randomly from the specified values considering that for this
• range of values for virtual resources of VMs, particular example the probability distribution is uniform. It
• rangeofvaluesforutilizationofvirtualresourcesofVMs, is important to mention that several other data probability
• range of revenue values for executing VMs, distributions could be considered.
• range of SLA values of VMs, The generated workload trace of Table III could have only
• range of number of VMs for cloud services, a duration of t=4 considering that both minimum and maxi-
• number of cloud services, mumvalueswere4(seeTableII).Forthisparticularexample,
• probability distribution. the number of cloud services was fixed to 1 considering that
both minimum and maximum values were 1 (see Table II).
TableII Additionally, the number of VMs of the cloud services in
INPUTDATAFOREXAMPLEWORKLOADTRACEFROMTABLEIII. the example of Table III can be adjusted from 1 to 6. The
revenue for executing VMs can vary from 0.1$ to 1.5$ and
InputData Value(Min-Max)
its corresponding SLAs can vary from 0 to 2. Analogously,
Workloadtraceduration(t) (4-4)
RangeofCPUvaluesforresources (4-10) the values for each virtual resources of VMs as well as its
RangeofMemoryvaluesforresources (2-16) utilization can vary from its specified values (see Table II).
RangeofNetworkvaluesforresources (100-1000)
Horizontal elasticity is considered in Table III in order to
RangeofCPUvaluesforutilization (2-10)
RangeofMemoryvaluesforutilization (1-16) efficiently attend the increasing demand of resources scaling
RangeofNetworkvaluesforutilization (0-1000) up the number of VMs of S from 2 (at t = 0) to 3 (at
1
RangeofrevenuevaluesforexecutingVMs (0.1-1.5) t = 1) and from 3 (at t = 1) to 4 (at t = 2). The number
RangeofSLAvaluesofVMs (0-2)
of VMs scales down from 4 (at t = 3) to 2 (at t = 4),
RangeofnumberofVMsforcloudservices (1-6)
Numberofcloudservices (1-1) assuming a decreasing demand of resources. Additionally,
Probabilitydistribution Random vertical elasticity for processing and memory resources could
TableIII
EXAMPLEOFWORKLOADTRACEFORVMPPROBLEMINENVIRONMENT(3,3)
t Sb Dc Vj Vcpu(cid:48)b(cid:48)cj Vram(cid:48)b(cid:48)cj Vnet(cid:48)b(cid:48)cj Rb(cid:48)(cid:48)cj SLA(cid:48)b(cid:48)cj Ucpu(cid:48)b(cid:48)cj Uram(cid:48)b(cid:48)cj Unet(cid:48)b(cid:48)cj
0 1 1 1 8 16 1000 0.5 1 8 14 150
0 1 2 1 8 16 1000 0.5 1 8 9 50
1 1 1 1 8 16 1000 0.5 1 7 10 160
1 1 2 1 8 16 1000 0.5 1 7 10 100
1 1 1 2 8 16 1000 0.5 1 7 7 70
2 1 1 1 8 16 1000 0.5 1 6 11 200
2 1 2 1 8 16 1000 0.5 1 6 11 150
2 1 1 2 8 16 1000 0.5 1 6 9 50
2 1 2 2 8 16 1000 0.5 1 6 12 60
3 1 1 1 8 16 1000 0.5 1 4 12 180
3 1 2 1 8 16 1000 0.5 1 4 12 150
3 1 1 2 8 16 1000 0.5 1 1 9 60
3 1 2 2 8 16 1000 0.5 1 1 8 60
4 1 1 1 4 8 1000 0.5 1 2 6 200
4 1 2 1 4 8 1000 0.5 1 2 6 100
be observed in the workload trace of Table III from t=3 to elasticity values: no elasticity, horizontal elasticity, vertical
t = 4, where Vcpu(cid:48)(cid:48) and Vcpu(cid:48)(cid:48) decrease from 8 [ECU] elasticity or both horizontal and vertical elasticity. Additio-
111 121
to 4 [ECU] and Vram(cid:48)(cid:48) and Vram(cid:48)(cid:48) decrease from 16 nally, identified dynamic environments may also consider one
111 121
[GB] to 8 [GB], assuming a decreasing demand of resources. of the following overbooking values: no overbooking, server
Vertical elasticity could also be applied to other resources as resourcesoverbooking,networkresourcesoverbookingorboth
described in Section IV-B. server and network overbooking.
Finally, server and network resources utilization change Based on the combinations of the possible values of the
dynamically in VMs from t = 0 to t = 4, representing classification criteria (elasticity and overbooking), the pro-
important data for CSPs in order to apply a safe overbooking posed taxonomy identified 16 different possible environments
ofbothserverandnetworkresources(seeTableIII).Att=0, (seeFigure3),characterizingeachenvironmentwithparticular
itcanbeseenahighutilizationofbothprocessingandmemory mathematical notation of time variables (see Table I).
resource, representing a possible alarm for scaling up the
The proposed taxonomy showed that research of online
number of VMs (horizontal elasticity) as can be observed in
formulations of the provider-oriented VMP problem has been
t = 1. Low utilization of resources can be seen at t = 3,
mainly studied in Environment (0,0) and (0,1) with 39% and
representing an alarm for scaling down both the number of
50% of the studied articles respectively. Other briefly studied
VMs (horizontal elasticity) as well as hardware configuration
environments are Environment (0,2), (2,0), (2,1) and (3,0).
of each VM (vertical elasticity) as can be seen at t=4.
Several research opportunities for unexplored environments
were identified (see Figure 3). For example, no paper was
VI. CONCLUSIONSANDFUTUREWORK
found studying horizontal elasticity alone, even more, joint
Based on an universe of 64 studied publications carefully network and server overbooking is still a field with no publi-
chosen as explained in Section II, this work extended the shedpaper.Consideringbothtypesofelasticityandbothtypes
taxonomy presented in [49] focusing on dynamic (online) ofoverbookingrepresentthemostadvancedenvironmentiden-
formulationsoftheVMPproblemfromtheproviders’perspec- tifiedinthiswork:Environment(3,3).CSPsefficientlysolving
tive complementing a previous work of the authors [49] and formulations of the VMP problem in this complex (3,3)
proposed a novel taxonomy in order to understand possible dynamicenvironmentwillrepresentaconsiderableadvanceon
challenges for Cloud Service Providers (CSPs) in dynamic thisresearchareaanditsclouddatacenterswillbeabletoscale
environments to efficiently attend customers’ request for vir- according to trending types of requirements with sufficient
tualresources,basedonthemostrelevantdynamicparameters flexibility. A recommended path for future work is exploring
studied so far in the VMP literature. andaddressingchallengesofparticularenvironmentsidentified
ThisworkidentifiedthatresourcecapacitiesofVMs(asso- asresearchopportunitiesbeforeconsideringthisadvancedand
ciatedtoverticalelasticity),numberofVMsofacloudservice complete (3,3) dynamic environment for solving the provider-
(associatedtohorizontalelasticity)andutilizationofresources oriented VMP problem.
ofVMs(relatedtooverbooking)arethemostrelevantdynamic Atthetimeofthiswriting,theauthorsarealreadyworking
parameters in literature. Consequently, dynamic environments on extending identified environments to consider dynamic
foronlineformulationsoftheprovider-orientedVMPproblem level of SLAs and dynamic revenue for executing VMs, just
were classified by one of the following classification criteria: tociteafewcharacteristicstobeincluded.Additionally,other
(1) elasticity and (2) overbooking. First, dynamic environ- environmentscouldbestudiedconsideringdynamicelectricity
ments could be formulated considering one of the following costs or pricing schemes in federated clouds, among others.
REFERENCES [19] W.Fang,X.Liang,S.Li,L.Chiaraviglio,andN.Xiong,“Vmplanner:
Optimizingvirtualmachineplacementandtrafficflowroutingtoreduce
[1] M. Alicherry and T. Lakshman, “Optimizing data access latencies in networkpowercostsinclouddatacenters,”ComputerNetworks,vol.57,
cloudsystemsbyintelligentvirtualmachineplacement,”inINFOCOM, no.1,pp.179–196,2013.
2013ProceedingsIEEE. IEEE,2013,pp.647–655. [20] T.Ferreto,C.A.DeRose,andH.-U.Heiss,“Maximummigrationtime
[2] A.Anand,J.Lakshmi,andS.Nandy,“Virtualmachineplacementopti- guaranteesindynamicserverconsolidationforvirtualizeddatacenters,”
mizationsupportingperformanceslas,”inCloudComputingTechnology inEuro-Par2011ParallelProcessing. Springer,2011,pp.443–454.
andScience(CloudCom),2013IEEE5thInternationalConferenceon, [21] T.C.Ferreto,M.A.Netto,R.N.Calheiros,andC.A.DeRose,“Server
vol.1. IEEE,2013,pp.298–305. consolidationwithmigrationcontrolforvirtualizeddatacenters,”Future
[3] M.Armbrust,O.Fox,R.Griffith,A.D.Joseph,Y.Katz,A.Konwinski, GenerationComputerSystems,vol.27,no.8,pp.1027–1034,2011.
G.Lee,D.Patterson,A.Rabkin,I.Stoicaetal.,“Abovetheclouds:A [22] M. Gahlawat and P. Sharma, “Survey of virtual machine placement
berkeleyviewofcloudcomputing,”UniversityofCalifornia,Berkeley, in federated clouds,” in Advance Computing Conference (IACC), 2014
Tech.Rep,2009. IEEEInternational. IEEE,2014,pp.735–738.
[4] A. Beloglazov, J. Abawajy, and R. Buyya, “Energy-aware resource [23] S.Georgiou,K.Tsakalozos,andA.Delis,“Exploitingnetwork-topology
allocationheuristicsforefficientmanagementofdatacentersforcloud awareness for vm placement in iaas clouds,” in Cloud and Green
computing,” Future Generation Computer Systems, vol. 28, no. 5, pp. Computing (CGC), 2013 Third International Conference on. IEEE,
755–768,2012. 2013,pp.151–158.
[5] O. Biran, A. Corradi, M. Fanelli, L. Foschini, A. Nus, D. Raz, and [24] H.GoudarziandM.Pedram,“Energy-efficientvirtualmachinereplica-
E.Silvera,“Astablenetwork-awarevmplacementforcloudsystems,” tionandplacementinacloudcomputingsystem,”inCloudComputing
in Proceedings of the 2012 12th IEEE/ACM International Symposium (CLOUD),2012IEEE5thInternationalConferenceon. IEEE,2012,
onCluster,CloudandGridComputing(ccgrid2012). IEEEComputer pp.750–757.
Society,2012,pp.498–506.
[25] Y. Guo, A. L. Stolyar, and A. Walid, “Shadow-routing based dynamic
[6] R. Buyya, C. S. Yeo, and S. Venugopal, “Market-oriented cloud com-
algorithms for virtual machine placement in a network cloud,” in
puting:Vision,hype,andrealityfordeliveringitservicesascomputing
INFOCOM,2013ProceedingsIEEE. IEEE,2013,pp.620–628.
utilities,”inHighPerformanceComputingandCommunications,2008.
[26] A.Gupta,L.V.Kale´,D.Milojicic,P.Faraboschi,andS.M.Balle,“Hpc-
HPCC’08.10thIEEEInternationalConferenceon,2008.
aware vm placement in infrastructure clouds,” in Cloud Engineering
[7] N.M.Calcavecchia,O.Biran,E.Hadad,andY.Moatti,“Vmplacement
(IC2E),2013IEEEInternationalConferenceon. IEEE,2013,pp.11–
strategies for cloud scenarios,” in Cloud Computing (CLOUD), 2012
20.
IEEE5thInternationalConferenceon. IEEE,2012,pp.852–859.
[27] A. Gupta, D. Milojicic, and L. V. Kale´, “Optimizing vm placement
[8] Z.CaoandS.Dong,“Anenergy-awareheuristicframeworkforvirtual
for hpc in the cloud,” in Proceedings of the 2012 workshop on Cloud
machineconsolidationincloudcomputing,”TheJournalofSupercom-
services,federation,andthe8thopencirrussummit. ACM,2012,pp.
puting,pp.1–23,2014.
1–6.
[9] D. Chang, G. Xu, L. Hu, and K. Yang, “A network-aware virtual
[28] Y. Ho, P. Liu, and J.-J. Wu, “Server consolidation algorithms with
machineplacementalgorithminmobilecloudcomputingenvironment,”
bounded migration cost and performance guarantees in cloud com-
in Wireless Communications and Networking Conference Workshops
puting,” in Utility and Cloud Computing (UCC), 2011 Fourth IEEE
(WCNCW),2013IEEE. IEEE,2013,pp.117–122.
InternationalConferenceon. IEEE,2011,pp.154–161.
[10] K.-y.Chen,Y.Xu,K.Xi,andH.J.Chao,“Intelligentvirtualmachine
[29] D. Hoeflin and P. Reeser, “Quantifying the performance impact of
placement for cost efficiency in geo-distributed cloud systems,” in
overbooking virtualized resources,” in Communications (ICC), 2012
Communications(ICC),2013IEEEInternationalConferenceon. IEEE,
IEEEInternationalConferenceon. IEEE,2012,pp.5523–5527.
2013,pp.3498–3503.
[30] H.-J.Hong,D.-Y.Chen,C.-Y.Huang,K.-T.Chen,andC.-H.Hsu,“Qoe-
[11] A.Dalvandi,M.Gurusamy,andK.C.Chua,“Time-awarevm-placement
aware virtual machine placement for cloud games,” in Network and
androutingwithbandwidthguaranteesingreenclouddatacenters,”in
Systems Support for Games (NetGames), 2013 12th Annual Workshop
CloudComputingTechnologyandScience(CloudCom),2013IEEE5th
on. IEEE,2013,pp.1–2.
InternationalConferenceon,vol.1. IEEE,2013,pp.212–217.
[31] W. Huang, X. Li, and Z. Qian, “An energy efficient virtual machine
[12] H. T. Dang and F. Hermenier, “Higher sla satisfaction in datacenters
placement algorithm with balanced resource utilization,” in Innovative
with continuous vm placement constraints,” in Proceedings of the 9th
Mobile and Internet Services in Ubiquitous Computing (IMIS), 2013
WorkshoponHotTopicsinDependableSystems. ACM,2013,p.1.
SeventhInternationalConferenceon. IEEE,2013,pp.313–319.
[13] D. S. Dias and L. H. M. Costa, “Online traffic-aware virtual machine
placementindatacenternetworks,”inGlobalInformationInfrastructure [32] Z. Huang and D. H. Tsang, “Sla guaranteed virtual machine consol-
andNetworkingSymposium(GIIS),2012. IEEE,2012,pp.1–8. idation for computing clouds,” in Communications (ICC), 2012 IEEE
InternationalConferenceon. IEEE,2012,pp.1314–1319.
[14] A. V. Do, J. Chen, C. Wang, Y. C. Lee, A. Y. Zomaya, and B. B.
Zhou,“Profilingapplicationsforvirtualmachineplacementinclouds,” [33] Z. Huang, D. H. Tsang, and J. She, “A virtual machine consolidation
in Cloud Computing (CLOUD), 2011 IEEE International Conference framework for mapreduce enabled computing clouds,” in Proceedings
on. IEEE,2011,pp.660–667. ofthe24thInternationalTeletrafficCongress. InternationalTeletraffic
[15] D.DongandJ.Herbert,“Energyefficientvmplacementsupportedby Congress,2012,p.26.
dataanalyticservice,”inCluster,CloudandGridComputing(CCGrid), [34] I. Hwang and M. Pedram, “Hierarchical virtual machine consolidation
2013 13th IEEE/ACM International Symposium on. IEEE, 2013, pp. in a cloud computing system,” in Cloud Computing (CLOUD), 2013
648–655. IEEESixthInternationalConferenceon. IEEE,2013,pp.196–203.
[16] J. Dong, H. Wang, X. Jin, Y. Li, P. Zhang, and S. Cheng, “Virtual [35] J.W.Jiang,T.Lan,S.Ha,M.Chen,andM.Chiang,“Jointvmplacement
machine placement for improving energy efficiency and network per- and routing for data center traffic engineering,” in INFOCOM, 2012
formanceiniaascloud,”inDistributedComputingSystemsWorkshops ProceedingsIEEE. IEEE,2012,pp.2876–2880.
(ICDCSW),2013IEEE33rdInternationalConferenceon. IEEE,2013, [36] H. Jin, H. Qin, S. Wu, and X. Guo, “Ccap: A cache contention-
pp.238–243. awarevirtualmachineplacementapproachforhpccloud,”International
[17] C.Dupont,G.Giuliani,F.Hermenier,T.Schulze,andA.Somov,“Anen- JournalofParallelProgramming,pp.1–18,2013.
ergyawareframeworkforvirtualmachineplacementincloudfederated [37] H. Jin, D. Pan, J. Xu, and N. Pissinou, “Efficient vm placement with
datacentres,”inFutureEnergySystems:WhereEnergy,Computingand multiple deterministic and stochastic resources in data centers,” in
CommunicationMeet(e-Energy),2012ThirdInternationalConference GlobalCommunicationsConference(GLOBECOM),2012IEEE. IEEE,
on. IEEE,2012,pp.1–10. 2012,pp.2505–2510.
[18] S. Fang, R. Kanagavelu, B.-S. Lee, C. H. Foh, and K. M. M. Aung, [38] D.Kakadia,N.Kopri,andV.Varma,“Network-awarevirtualmachine
“Power-efficient virtual machine placement and migration in data cen- consolidation for large data centers,” in Proceedings of the Third
ters,” in Green Computing and Communications (GreenCom), 2013 InternationalWorkshoponNetwork-AwareDataManagement. ACM,
IEEE and Internet of Things (iThings/CPSCom), IEEE International 2013,p.6.
ConferenceonandIEEECyber,PhysicalandSocialComputing. IEEE, [39] B. Kantarci, L. Foschini, A. Corradi, and H. T. Mouftah, “Inter-and-
2013,pp.1408–1413. intra data center vm-placement for energy-efficient large-scale cloud
systems,” in Globecom Workshops (GC Wkshps), 2012 IEEE. IEEE, [59] B.C.Ribas,R.M.Suguimoto,R.A.Montano,F.Silva,L.deBona,and
2012,pp.708–713. M.A.Castilho,“Onmodellingvirtualmachineconsolidationtopseudo-
[40] N. Kord and H. Haghighi, “An energy-efficient approach for virtual boolean constraints,” in Advances in Artificial Intelligence–IBERAMIA
machine placement in cloud based data centers,” in Information and 2012. Springer,2012,pp.361–370.
Knowledge Technology (IKT), 2013 5th Conference on. IEEE, 2013, [60] L.SalimianandF.Safi,“Surveyofenergyefficientdatacentersincloud
pp.44–49. computing,” in Proceedings of the 2013 IEEE/ACM 6th International
[41] K.Le,R.Bianchini,J.Zhang,Y.Jaluria,J.Meng,andT.D.Nguyen, ConferenceonUtilityandCloudComputing. IEEEComputerSociety,
“Reducing electricity cost through virtual machine placement in high 2013,pp.369–374.
performance computing clouds,” in Proceedings of 2011 International [61] K. Sato, M. Samejima, and N. Komoda, “Dynamic optimization of
ConferenceforHighPerformanceComputing,Networking,Storageand virtual machine placement by resource usage prediction,” in Industrial
Analysis. ACM,2011,p.22. Informatics (INDIN), 2013 11th IEEE International Conference on.
[42] K. Li, J. Wu, and A. Blaisse, “Elasticity-aware virtual machine place- IEEE,2013,pp.86–91.
ment for cloud datacenters,” in Cloud Networking (CloudNet), 2013 [62] L. Shi, B. Butler, D. Botvich, and B. Jennings, “Provisioning of
IEEE2ndInternationalConferenceon. IEEE,2013,pp.99–107. requests for virtual machine sets with placement constraints in iaas
[43] K.Li,H.Zheng,andJ.Wu,“Migration-basedvirtualmachineplacement clouds,”inIntegratedNetworkManagement(IM2013),2013IFIP/IEEE
in cloud systems,” in Cloud Networking (CloudNet), 2013 IEEE 2nd InternationalSymposiumon. IEEE,2013,pp.499–505.
InternationalConferenceon. IEEE,2013,pp.83–90. [63] W.ShiandB.Hong,“Towardsprofitablevirtualmachineplacementin
the data center,” in Utility and Cloud Computing (UCC), 2011 Fourth
[44] W. Li, J. Tordsson, and E. Elmroth, “Virtual machine placement for
predictable and time-constrained peak loads,” in Economics of Grids, IEEEInternationalConferenceon. IEEE,2011,pp.138–145.
Clouds,Systems,andServices. Springer,2012,pp.120–134. [64] S. Shigeta, H. Yamashima, T. Doi, T. Kawai, and K. Fukui, “Design
and implementation of a multi-objective optimization mechanism for
[45] X. Li, Z. Qian, S. Lu, and J. Wu, “Energy efficient virtual machine
virtual machine placement in cloud computing data center,” in Cloud
placementalgorithmwithbalancedandimprovedresourceutilizationin
Computing. Springer,2013,pp.21–31.
a data center,” Mathematical and Computer Modelling, vol. 58, no. 5,
[65] N.A.SinghandM.Hemalatha,“Reduceenergyconsumptionthrough
pp.1222–1235,2013.
virtualmachineplacementinclouddatacentre,”inMiningIntelligence
[46] J.-W. Lin and C.-H. Chen, “Interference-aware virtual machine place-
andKnowledgeExploration. Springer,2013,pp.466–474.
mentincloudcomputingsystems,”inComputer&InformationScience
[66] F.Song,D.Huang,H.Zhou,H.Zhang,andI.You,“Anoptimization-
(ICCIS), 2012 International Conference on, vol. 2. IEEE, 2012, pp.
based scheme for efficient virtual machine placement,” International
598–603.
JournalofParallelProgramming,vol.42,no.5,pp.853–872,2014.
[47] F.Lo´pez-PiresandB.Bara´n,“Amany-objectiveoptimizationframework
[67] V.SoundararajanandK.Govil,“Challengesinbuildingscalablevirtuali-
forvirtualizeddatacenters,”inProceedingsofthe20155thInternational
zeddatacentermanagement,”ACMSIGOPSOperatingSystemsReview,
Conference on Cloud Computing and Service Science, 2015, pp. 439–
vol.44,no.4,pp.95–102,2010.
450.
[68] B.SpeitkampandM.Bichler,“Amathematicalprogrammingapproach
[48] F. Lo´pez-Pires and B. Bara´n, “Virtual machine placement literature
forserverconsolidationproblemsinvirtualizeddatacenters,”Services
review,” Polytechnic School, National University of Asuncio´n, Tech.
Computing,IEEETransactionson,vol.3,no.4,pp.266–278,2010.
Rep.,2015.[Online].Available:http://arxiv.org/abs/1506.01509
[69] M.-H.Tsai,J.Chou,andJ.Chen,“Preventvmmigrationinvirtualized
[49] F.Lo´pez-PiresandB.Bara´n,“Avirtualmachineplacementtaxonomy,” clusters via deadline driven placement policy,” in Cloud Computing
in Proceedings of the 2015 IEEE/ACM 15th International Symposium Technology and Science (CloudCom), 2013 IEEE 5th International
onCluster,CloudandGridComputing. IEEEComputerSociety,2015. Conferenceon,vol.1. IEEE,2013,pp.599–606.
[50] F.Lo´pez-Pires,E.Melgarejo,andB.Bara´n,“Virtualmachineplacement. [70] K.Tsakalozos,M.Roussopoulos,andA.Delis,“Vmplacementinnon-
a multi-objective approach,” in Computing Conference (CLEI), 2013 homogeneousiaas-clouds,”inService-OrientedComputing. Springer,
XXXIXLatinAmerican. IEEE,2013,pp.1–8. 2011,pp.172–187.
[51] K. Lu, R. Yahyapour, P. Wieder, C. Kotsokalis, E. Yaqub, and A. I. [71] S. Wang, Z. Liu, Z. Zheng, Q. Sun, and F. Yang, “Particle swarm
Jehangiri, “Qos-aware vm placement in multi-domain service level optimizationforenergy-awarevirtualmachineplacementoptimizationin
agreementsscenarios,”inCloudComputing(CLOUD),2013IEEESixth virtualizeddatacenters,”inParallelandDistributedSystems(ICPADS),
InternationalConferenceon. IEEE,2013,pp.661–668. 2013InternationalConferenceon. IEEE,2013,pp.102–109.
[52] K. Mills, J. Filliben, and C. Dabrowski, “Comparing vm-placement [72] S.-H.Wang,P.P.-W.Huang,C.H.-P.Wen,andL.-C.Wang,“Eqvmp:
algorithmsforon-demandclouds,”inCloudComputingTechnologyand Energy-efficientandqos-awarevirtualmachineplacementforsoftware
Science (CloudCom), 2011 IEEE Third International Conference on. defineddatacenternetworks,”inInformationNetworking(ICOIN),2014
IEEE,2011,pp.91–98. InternationalConferenceon. IEEE,2014,pp.220–225.
[53] M. Mishra and A. Sahoo, “On theory of vm placement: Anomalies in [73] W.Wang,H.Chen,andX.Chen,“Anavailability-awarevirtualmachine
existingmethodologiesandtheirmitigationusinganovelvectorbased placement approach for dynamic scaling of cloud applications,” in
approach,” in Cloud Computing (CLOUD), 2011 IEEE International UbiquitousIntelligence&Computingand9thInternationalConference
Conferenceon. IEEE,2011,pp.275–282. onAutonomic&TrustedComputing(UIC/ATC),20129thInternational
[54] I.S.Moreno,R.Yang,J.Xu,andT.Wo,“Improvedenergy-efficiencyin Conferenceon. IEEE,2012,pp.509–516.
cloud datacenters with interference-aware virtual machine placement,” [74] J.-J.Wu,P.Liu,andJ.-S.Yang,“Workloadcharacteristics-awarevirtual
in Autonomous Decentralized Systems (ISADS), 2013 IEEE Eleventh machineconsolidationalgorithms,”inProceedingsofthe2012IEEE4th
InternationalSymposiumon. IEEE,2013,pp.1–8. InternationalConferenceonCloudComputingTechnologyandScience
[55] J. Ortigoza, F. Lo´pez-Pires, and B. Bara´n, “Workload trace generation (CloudCom). IEEEComputerSociety,2012,pp.42–49.
for dynamic environments in cloud computing,” Polytechnic School, [75] K. Zamanifar, N. Nasri, and M. Nadimi-Shahraki, “Data-aware vir-
NationalUniversityofAsuncio´n,Tech.Rep.,2015.[Online].Available: tual machine placement and rate allocation in cloud environment,” in
http://arxiv.org/abs/1507.00090 Advanced Computing & Communication Technologies (ACCT), 2012
[56] J. T. Piao and J. Yan, “A network-aware virtual machine placement SecondInternationalConferenceon. IEEE,2012,pp.357–360.
andmigrationapproachincloudcomputing,”inGridandCooperative [76] X. Zhang, Y. Zhang, X. Chen, K. Liu, G. Huang, and J. Zhan, “A
Computing(GCC),20109thInternationalConferenceon. IEEE,2010, relationship-based vm placement framework of cloud environment,” in
pp.87–92. Proceedings of the 2013 IEEE 37th Annual Computer Software and
[57] J.J.Prevost,K.Nagothu,B.Kelley,andM.Jamshidi,“Optimalupdate ApplicationsConference. IEEEComputerSociety,2013,pp.124–133.
frequencymodelforphysicalmachinestatechangeandvirtualmachine [77] X.Zhang,Q.Yue,andZ.He,“Dynamicenergy-efficientvirtualmachine
placementinthecloud,”inSystemofSystemsEngineering(SoSE),2013 placement optimization for virtualized clouds,” in Proceedings of the
8thInternationalConferenceon. IEEE,2013,pp.159–164. 2013InternationalConferenceonElectricalandInformationTechnolo-
[58] B.C.Ribas,R.M.Suguimoto,R.A.Montano,F.Silva,andM.Castilho, giesfor RailTransportation (EITRT2013)-VolumeII. Springer, 2014,
“Pbfvmc:Anewpseudo-booleanformulationtovirtual-machineconsol- pp.439–448.
idation,”inIntelligentSystems(BRACIS),2013BrazilianConferenceon.
IEEE,2013,pp.201–206.