Job Scheduling Strategies for Parallel Processing: 15th International Workshop, JSSPP 2010, Atlanta, GA, USA, April 23, 2010, Revised Selected Papers

231 Pages·2010·3.84 MB·English

by  Kaiqi Xiong

Lecture Notes in Computer Science 6253
CommencedPublicationin1973
FoundingandFormerSeriesEditors:
GerhardGoos,JurisHartmanis,andJanvanLeeuwen
EditorialBoard
DavidHutchison
LancasterUniversity,UK
TakeoKanade
CarnegieMellonUniversity,Pittsburgh,PA,USA
JosefKittler
UniversityofSurrey,Guildford,UK
JonM.Kleinberg
CornellUniversity,Ithaca,NY,USA
AlfredKobsa
UniversityofCalifornia,Irvine,CA,USA
FriedemannMattern
ETHZurich,Switzerland
JohnC.Mitchell
StanfordUniversity,CA,USA
MoniNaor
WeizmannInstituteofScience,Rehovot,Israel
OscarNierstrasz
UniversityofBern,Switzerland
C.PanduRangan
IndianInstituteofTechnology,Madras,India
BernhardSteffen
TUDortmundUniversity,Germany
MadhuSudan
MicrosoftResearch,Cambridge,MA,USA
DemetriTerzopoulos
UniversityofCalifornia,LosAngeles,CA,USA
DougTygar
UniversityofCalifornia,Berkeley,CA,USA
GerhardWeikum
MaxPlanckInstituteforInformatics,Saarbruecken,Germany
Eitan Frachtenberg Uwe Schwiegelshohn (Eds.)
Job Scheduling
Strategies
for Parallel Processing
15th International Workshop, JSSPP 2010
Atlanta, GA, USA, April 23, 2010
Revised Selected Papers
1 3
VolumeEditors
EitanFrachtenberg
Facebook,475BrannanSt.
SanFrancisco,CA,94107,USA
E-mail:[email protected]
UweSchwiegelshohn
RoboticsResearchInstitute
SectionInformationTechnology
TUDortmundUniversity
Otto-Hahn-Str.8
44227Dortmund,Germany
E-mail:[email protected]
LibraryofCongressControlNumber:2010937281
CRSubjectClassification(1998):C.2.4,D.4,C.2,D.2,H.3,I.6
LNCSSublibrary:SL1–TheoreticalComputerScienceandGeneralIssues
ISSN 0302-9743
ISBN-10 3-642-16504-4SpringerBerlinHeidelbergNewYork
ISBN-13 978-3-642-16504-7SpringerBerlinHeidelbergNewYork
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting,
reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965,
initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsareliable
toprosecutionundertheGermanCopyrightLaw.
springer.com
©Springer-VerlagBerlinHeidelberg2010
PrintedinGermany
Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India
Printedonacid-freepaper 06/3180
Preface
Thisvolumecontainsthepaperspresentedatthe15thworkshoponJobSchedul-
ing Strategies for Parallel Processing that was held in Atlanta (GA), USA, on
April 23, 2010 in conjunction with the IEEE International Parallel Processing
Symposium 2010.
This year 18 papers were submitted to the workshop. All submitted papers
went through a complete review process, with the full version being read and
evaluated by an average of four reviewers. We would like to especially thank
the programcommittee members and additionalreferees for their willingness to
participate in this effort and their excellent, detailed reviews:
Henri Casanova, Peter A. Chronz, Walfredo Cirne, Julita Corbalan, Arash
Deshmeh,DickEpema,DrorG.Feitelson,AllanGottlieb,RajkumarKettimuthu,
Virginia Lo, Kuan Lu, Vicent Matossian, Jose E. Moreira,Bill Nitzberg, Elizeu
Santos-Neto,AngelaC.Sodan,MarkS.Squillante,DanTsafrir,PhilippWieder,
and Ramin Yahyapour.
Thepapersinthis volumeshowaprolificgrowthinthe areasofapplicability
forparallelscheduling.Togetherwiththemorecommonschedulingaspects(such
asclusterandGridscheduling,workloadanalysis,metrics,qualityofservice,and
task scheduling), these papers increasingly discuss more recent problems and
applications, such as virtualized environments, many-core processors, DNA se-
quencing,andHadoop.This volume alsoincludes a paper thatsummarizes Dan
Tsafrir’sworkonunderstandingtheroleofuserestimatesinjobschedulingeval-
uations.Hisinsights,whichwerepresentedinthis workshop’skeynote,arequite
instructive and lead to the conclusion that accurate user estimates are indeed
betterforefficientscheduling.Althoughthisconclusionmaysoundintuitive,itis
actuallycontradictorytopreviousstudiesthatfoundinaccurateestimatestoim-
proveschedulerperformance.Followinghis analysis,Danalsosuggestspractical
ways to deal with estimate inaccuracy for realistic job scheduler evaluations.
One of the stated goals of the JSSPP workshop is to explore the applica-
tion of traditional scheduling topics to novel scenarios. A good example of such
topics is task and graph scheduling, which until last year was largely outside
the scope of JSSPP. Recent technologies are reviving interest in this topic, as
was discussed last year in the context of workflow jobs in a Grid environment
(in a paper by Gong et al.), and this year in the context of the emerging multi-
core/multi-threadedprocessors.Xia,Prasanna,andLiapplyideasfromdynamic
load balancing and hierarchical thread grouping to the contemporary architec-
ture of the Sun Niagara processor with its 64 hardware threads. A different
thread-level scheduling aspect for many-core architectures is introduced in the
paper by Yamada and Kusakabe. Here, a modification to the operating sys-
tem is suggested to permit multi-threaded applications to aggregate time and
space resources for improved cache efficiency. In another example of applying
VI Preface
traditional scheduling to newer problems, the paper by Saule, Bozdag, and
Catalyurek shows how a contemporaryDNA sequencing workloadcan benefit –
in terms of reduced slowdown– from the application of earliest-deadline-firstto
moldable-job scheduling.
Two other papers combine traditional scheduling techniques with modern
technologicalchallengesandcapabilities.For the former,the paper by Klusa´ˇcek
andRudova´?addressesa deficiency in many ofthe workloadtracesused for job
scheduling simulations: the lack of complete system and workload information,
includingmachinecharacteristicsandfailures.Addingthesedatatoatrace(syn-
theticallyorwithactualcollecteddata,inthecaseofMetaCentrum’strace)can
significantly alter the results of a scheduler evaluation.For the latter, the paper
by Verboven, Vanmechelen, and Broeckhoveintroduces a scheduling scheme for
virtualmachines, where the workloadhas mixed high and low quality-of-service
(QoS)requirements.Theschedulertakesadvantageofthe relativeeasethatvir-
tualizationoffersforjobpreemptiontoletoverbookedlow-priorityjobsfillinfor
underutilized resources,without interrupting high-priority jobs. Overbooking is
alsoexploredinthepaperbyBirkenheuer,Brinkmann,andKarl,extendingtheir
work from the previous workshop. Here, the authors integrate a statistical risk
assessmentmoduletotheirGrid/Cloudschedulerthatusesautomatedrun-time
predictions to overbook backfilled “gaps”, while still minimizing the economic
penalties to the system owner from missed deadlines.
The economics of Grid scheduling and QoS were in fact a major theme in
this year’s papers, and played a minor part in several other papers. The paper
by Sandholm and Lai explores a dynamic resource allocation scheduler for the
popular Hadoop environment, in which users receive a share of computational
resources that is proportional to their bid. On a related vein, in Ding’s con-
tribution, a greedy double-auction mechanism to dynamically price resources is
simulated,backedbyatheoreticalanalysis.Xiong’spaperonthe otherhandas-
sumes static pricing per QoS level, and proposes a resource allocation approach
to minimize the total cost to the application, again offering a theoretical treat-
ment of the optimization problem.Similarly, in the paper by Takefusa,Nakada,
Kudoh,andTanaka,alinear-programmingmethodisusedtoco-allocateprocess-
ing and networking resources to Grid applications, based on an actual system’s
problem statement. And last but not least, in the metascheduler proposed by
Fo¨lling, Grimme, Lepping, and Papaspyrou, different sites can “lease;; under-
utilized resources to heavily loaded sites that request them.
A few years ago, we described the transition of the classic job scheduling
paradigm for parallel processing based on the proliferation of new technolo-
gies like many-core architectures and Grid or Cloud computing. Today, we can
observethat these technologicaladvances produce significantchangesin the us-
age patterns. Due to many-core architectures, the exploitation of parallelism is
not restricted any more to a few high performance applications. Similar to the
situation during the last decade of the previous century when every application
exploitedsuperscalarprocessors,applicationsarenowexpectedtomakeefficient
useofmultiple cores.This ofcourseresultsinnew challengesforjobscheduling.
Preface VII
Furthermore, Grid and Cloud computing provides affordable processing power
tomanyusers,leadingtonewapplicationsifthereareefficientresourcemanage-
mentsystems.Theseresourcemanagementsystemswillconsistofmultiplelayers
relatedto users,resourceproviders,anddomainmanagers.However,the alloca-
tion of tasks to these layers is still under investigation. Therefore, we strongly
believe that research in the field of this workshop will remain interesting and
challenging in the years to come.
The proceedings of previous workshopsare availablefrom Springer as LNCS
volumes 949, 1162, 1291, 1459, 1659, 1911, 2221, 2537, 2862, 3277, 3834, 4376,
4942, and 5798. Since 1995 these volumes have also been available online.
June 2010 Eitan Frachtenberg
Uwe Schwiegelshohn
Organization
Workshop Organizers
Eitan Frachtenberg Facebook
Uwe Schwiegelshohn TU Dortmund University
Program Committee
Henri Casanova University of Hawaii at Manoa
Walfredo Cirne Google
Julita Corbalan Technical University of Catalunya
Dick Epema Delft University of Technology
Dror Feitelson The Hebrew University
Allan Gottlieb New York University
Rajkumar Kettimuthu Argonne National Lab
Virginia Lo University of Oregon
Jose Moreira IBM Thomas J. Watson Research Center
Bill Nitzberg Altair Engineering, Inc.
Angela Sodan University of Windsor
Mark Squillante IBM Thomas J. Watson Research Center
Dan Tsafrir Technion
Ramin Yahyapour TU Dortmund University
Table of Contents
Resource Provisioning in SLA-Based Cluster Computing .............. 1
Kaiqi Xiong and Sang Suh
AnAdvanceReservation-BasedCo-allocationAlgorithmforDistributed
Computers and Network Bandwidth on QoS-GuaranteedGrids ........ 16
Atsuko Takefusa, Hidemoto Nakada, Tomohiro Kudoh, and
Yoshio Tanaka
A Greedy Double Auction Mechanism for Grid Resource Allocation .... 35
Ding Ding, Siwei Luo, and Zhan Gao
Risk Aware Overbooking for Commercial Grids ...................... 51
Georg Birkenheuer, Andr´e Brinkmann, and Holger Karl
The Gain of Resource Delegation in Distributed Computing
Environments ................................................... 77
Alexander Fo¨lling, Christian Grimme, Joachim Lepping, and
Alexander Papaspyrou
A Moldable Online Scheduling Algorithm and Its Application to
ParallelShort Sequence Mapping .................................. 93
Erik Saule, Doruk Bozda˘g, and Umit V. Catalyurek
Dynamic ProportionalShare Scheduling in Hadoop................... 110
Thomas Sandholm and Kevin Lai
The Importance of Complete Data Sets for Job Scheduling
Simulations ..................................................... 132
Dalibor Klusa´ˇcek and Hana Rudov´a
HierarchicalSchedulingofDAGStructuredComputations onManycore
Processorswith Dynamic Thread Grouping ......................... 154
Yinglong Xia, Viktor K. Prasanna, and James Li
Multiplexing Low and High QoS Workloads in Virtual Environments ... 175
Sam Verboven, Kurt Vanmechelen, and Jan Broeckhove
Proposal and Evaluation of APIs for Utilizing Inter-Core Time
AggregationScheduler............................................ 191
Satoshi Yamada and Shigeru Kusakabe
Using Inaccurate Estimates Accurately ............................. 208
Dan Tsafrir
Author Index.................................................. 223
Resource Provisioning in SLA-Based
Cluster Computing
Kaiqi Xiong and Sang Suh
Department of Computer Science, Texas A&MUniversity,
Commerce, TX 75429, USA
kaiqi [email protected]
Abstract. Cluster computing is excellent for parallel computation. It
has become increasingly popular. In cluster computing, a service level
agreement (SLA) is a set of quality of services (QoS) and a fee agreed
between a customerand an application serviceprovider. Itplaysan im-
portantroleinane-businessapplication.Anapplicationserviceprovider
uses a set of cluster computing resources to support e-business appli-
cations subject to an SLA. In this paper, the QoS includes percentile
response time and cluster utilization. We present an approach for re-
sourceprovisioninginsuchanenvironmentthatminimizesthetotalcost
of cluster computing resources used by an application service provider
forane-businessapplicationthatoftenrequiresparallelcomputationfor
high service performance, availability, and reliability while satisfying a
QoS and a fee negotiated between a customer and the application ser-
vice provider. Simulation experiments demonstrate the applicability of
theapproach.
Keywords:Clustercomputing,schedulingtheory,resourceprovisioning,
service level agreement, and percentile response time.
1 Introduction
In computer science,scheduling theory is concernedwith the optimal allocation
ofscarceresourcessuchasservers,processorsandnetworklinkstocomputerser-
viceactivitiesovertime,withtheobjectiveofoptimizingoneorseveralcomputer
performance measures (e.g., see Levner [13]). Cluster computing is excellent for
parallel computation. It has become increasingly popular. The management of
computing service resources is fundamental to cluster computing. The increas-
ing pervasiveness of network connectivity and the proliferationof on demand e-
businessapplicationsandservicesinpublicdomains,corporatenetworks,aswell
as home environmentsgive rise to the need for the designof appropriateservice
management solutions in cluster computing. Accurately predicting e-business
application and scientific computation performance based on service statistics
and a customer’s perceived quality allows an application service provider (sim-
ply called a service provider) not only to assure quality of services but also to
avoid over provisioning to meet a service level agreement (SLA).
E.FrachtenbergandU.Schwiegelshohn(Eds.):JSSPP2010,LNCS6253,pp.1–15,2010.
(cid:2)c Springer-VerlagBerlinHeidelberg2010