GDS: an Architecture Proposal for a Grid Data-Sharing Service

23 Pages·2016·0.62 MB·English

View metadata, citation and similar papers at core.ac.uk brought to you by CORE
provided by HAL-Rennes 1
GDS: an Architecture Proposal for a Grid Data-Sharing
Service
Gabriel Antoniu, Marin Bertier, Luc Boug´e, Eddy Caron, Fr´ed´eric Desprez,
Mathieu Jan, S´ebastien Monnet, Pierre Sens
To cite this version:
Gabriel Antoniu, Marin Bertier, Luc Boug´e, Eddy Caron, Fr´ed´eric Desprez, et al.. GDS: an
Architecture Proposal for a Grid Data-Sharing Service. [Research Report] RR-5593, INRIA.
2005. <inria-00071239>
HAL Id: inria-00071239
https://hal.inria.fr/inria-00071239
Submitted on 23 May 2006
HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est
archive for the deposit and dissemination of sci- destin´ee au d´epˆot et `a la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publi´es ou non,
lished or not. The documents may come from ´emanant des ´etablissements d’enseignement et de
teaching and research institutions in France or recherche fran¸cais ou ´etrangers, des laboratoires
abroad, or from public or private research centers. publics ou priv´es.
INSTITUTNATIONALDERECHERCHEENINFORMATIQUEETENAUTOMATIQUE
GDS: an Architecture Proposal for a Grid
Data-Sharing Service
GabrielAntoniu—MarinBertier—LucBouge´ —EddyCaron—
Fre´de´ricDesprez—MathieuJan—Se´bastienMonnet—PierreSens
N° 5593
June2005
The`meNUM
(cid:13)
G
N
E apport
+
R
F
3-- de recherche(cid:13)
9 (cid:13)
5
5
R--
R
A/
RI
N
N I
R
S
9 I
9
3
6
9-
4
2
0
N
S
S
I

GDS: an Architecture Proposal for a Grid Data-Sharing Service
GabrielAntoniu,MarinBertier,LucBougé,EddyCaron,
FrédéricDesprez,MathieuJan,SébastienMonnet,PierreSens
ThèmeNUM—Systèmesnumériques
ProjetGRAAL,PARIS,REGAL
Rapportderecherche n°5593 —June2005— 19pages
Abstract: Gridcomputinghasrecentlyemergedasaresponsetothegrowingdemandforresources
(processing power, storage, etc.) exhibited by scientific applications. We address the challenge of
sharing large amounts of data on such infrastructures, typically consisting of a federation of node
clusters. We claim that storing, accessing, updating and sharing such data should be considered
by applications as an external service. We propose an architecture for such a service, whose goal
istoprovidetransparentaccesstomutabledata,whileenhancingdatapersistenceandconsistency
despitenodedisconnectionsorfailures.Ourapproachleveragesonweavingtogetherpreviousresults
intheareasofdistributedsharedmemorysystems,peer-to-peersystems,andfault-tolerantsystems.
Key-words: Datasharing,gridcomputing,transparentaccess,mutabledata,peer-to-peersystems,
faulttolerance,consistencyprotocols
This text is also available as a research report of the Laboratoire de l’Informatique du Parallélisme
http://www.ens-lyon.fr/LIP.
Unite´ derechercheINRIARhoˆne-Alpes
655,avenuedel’Europe,38334MontbonnotSaintIsmier(France)
Te´le´phone:[phone]—Te´le´copie+33476615252
GDS: une Architecture pour le Partage de Données sur la Grille
Résumé: Lecalculsurlesgrillesarécemmentémergécommeuneréponseauxdemandescrois-
santesderessources(puissancedetraitement,destockage,etc.)issuesdesapplicationsscientifiques.
Nousnousintéressonsiciaudéfidupartagededonnéesdansunetelleinfrastructure,quiestconsti-
tuéeclassiquementd’unefédérationdenœudsprovenantdediversesgrappes. Noussouhaitonsque
lestockage,l’accèsetlesmisesàjourdecesdonnéessoientconsidéréesparlesapplicationscomme
un service externe. Nous proposons une architecture qui fournit un tel service, dont l’objectif est
d’offrir un accès transparent aux données modifiables, disposant de mécanismes de persistance et
decohérencededonnéesmêmeencasdedéconnexionoudepannes. Notreapprochevientfédérer
nosprécédentsrésultatsdanslesdomainesdessystèmesdistribuésàmémoirepartagée,dessystèmes
pairsàpairsetdessystèmestolérantsauxpannes.
Mots-clés : Partage de données, Calcul sur grille, Accès transparent, Données modifiables, Sys-
tèmespairsàpairs,Toléranceauxpannes,protocolesdecohérence
GDS:anArchitectureProposalforaGridData-SharingService 3
1 Introduction
Data management in grid environments Data management in grid environments is currently
a topic of major interest to the grid computing community. However, as of today, no approach
has been widely established for transparent data sharing on grid infrastructures. Currently, the
most widely-usedapproachto data managementfor distributed grid computationrelies on explicit
data transfers between clients and computing servers: the client has to specify where the input
data is located and to which server it has to be transferred. Then, at the end of the computation,
the results are eventually transferred back to the client. As an example, the Globus [16] platform
providesdataaccessmechanismsbasedontheGridFTPprotocol[1]. Thoughthisprotocolprovides
authentication, paralleltransfers, checkpoint/restartmechanisms, etc., itstillrequires explicit data
localization.
Ithasbeenshownthatprovidingdatawithsomedegreeofpersistencemayconsiderablyimprove
the performance of series of successive computations. Therefore, Globus has proposed to provide
so-calleddatacatalogs[1]ontopofGridFTP,whichallowmultiplecopiesofthesamedatatobe
manuallyrecordedonvarioussites. However,theconsistencyofthesereplicasremainstheburden
oftheuser.
Inanotherdirection,alarge-scaledatastoragesystemisprovidedbyIBP[5],asasetofso-called
buffersdistributedoverInternet. Theusercan“rent”thesestorageareasandusethemastemporary
buffersforoptimizingdatatransfersacrossawide-areanetwork. Transfermanagementstillremains
theburdenoftheuser,andnoconsistencymechanismisprovidedformanagingmultiplecopiesof
the same data. Finally, Stork [18] is another recent example of system providing mechanisms to
explicitlylocate, moveandreplicatedataaccordingtotheneedsofasequenceofcomputations. It
providestheuserwithanintegratedinterfacetoscheduledatamovementactionsjustlikecomputa-
tionaljobs. Again,datalocationandtransferhavetobeexplicitlyhandledbytheuser.
Ourapproach:transparentaccesstodata Agrowingnumberofapplicationsmakeuseoflarger
andlargeramountsofdistributeddata. Weclaimthatexplicitmanagementofdatalocationsbythe
programmerarisesasamajorlimitationwithrespecttotheefficientuseofmodern,large-scalecom-
putationalgrids. Suchalow-levelapproachmakesgridprogrammingextremelyhardtomanage. In
contrast,theconceptofadata-sharingserviceforgridcomputing[2]opensanalternativeapproach
to the problem of grid data management. Its ultimate goal is to provide the user with transparent
accesstodata. IthasbeenillustratedbytheexperimentalJUXMEM[2]softwareplatform: theuser
onlyaccessesdataviaglobalhandles.Theservicetakescareofdatalocalizationandtransferwithout
anyhelpfromtheexternaluser.Theservicealsotransparentlyappliesadequatereplicationstrategies
andconsistencyprotocolstoensuredatapersistenceandconsistencyinspiteofnodefailures. These
mechanismstargetalarge-scale,dynamicgridarchitecture,wherenodesmayunexpectedlyfailand
recover.
Required properties The target applications under consideration are scientific simulations, typ-
icallyinvolvingmultipleweakly-coupledcodesrunningondifferentsites, andcooperatingviape-
RR n°5593
4 G.Antoniu,M.Bertier,L.Bougé,E.Caron,F.Desprez,M.Jan,S.Monnet,P.Sens
riodic data exchanges. Transparent access to remote data through an external data-sharing service
arisesasamajorfeatureinthiscontext. Suchaserviceshouldprovidethefollowingproperties.
Persistence. Sincegridapplicationscanhandlelargemassesofdata,datatransferamongsitescan
be costly, in terms of both latency and bandwidth. In order to limit these data exchanges,
thedata-sharingservicehastoprovidepersistentdatastorage,soastosavedatatransfers. It
shouldrelyonstrategiesableto:1)reusepreviouslyproduceddata,byavoidingrepeateddata
transfersbetweenthedifferentcomponentsofthegrid;2)trigger“smart”pre-fetchingactions
to anticipate future accesses; and 3) provide useful information on data location to the task
scheduler,inordertooptimizetheglobalexecutioncost.
Faulttolerance. The data-sharing service must match the dynamic character of the grid infras-
tructure. In particular, the service has to support events such as storage resources joining
and leaving, or unexpectedly failing. Replication techniques and failure detection mecha-
nismsarethusnecessary. Basedonsuchmechanisms,sophisticatedfault-tolerantdistributed
data-management algorithms can be designed, in order to enhance data availability despite
disconnectionsandfailures.
Dataconsistency. In the general case, shared data manipulated by grid applications are mutable:
they can be read, but also updated by the various nodes. When accessed on multiple sites,
data are often replicated to enhance access locality. To ensure the consistency of the differ-
entreplicas,theservicereliesonconsistencymodels,implementedbyconsistencyprotocols.
However,previousworkonthistopic(e.g.,inthecontextofDistributedSharedMemorysys-
tems, DSM)generallyassumesasmall-scaled, stablephysicalarchitecture, withoutfailures.
Itisclearthatsuchassumptionsarenotrelevantwithrespecttoourcontext. Therefore,build-
ing data-sharing service for the grid requires a new approach to the design of consistency
protocols.
In this paper, we address these issues by proposing an architecture for a data-sharing service
providinggridapplicationswithtransparentaccesstodata. Weconsiderthegeneralcaseofadis-
tributedenvironmentinwhichClientssubmitjobstoaJobManager,anentityinchargeofselecting
theComputingServerswherejobexecutionshalltakeplace. Whenthesamedataaresharedbyjobs
scheduledondifferentservers,theData-SharingServicecanbeusedtostoreandretrievethemina
transparentway. ThisgeneralorganizationschemeisillustratedonFigure1.
Thepaperisorganizedasfollows. Section2firstdescribesafewmotivatingscenariosthatillus-
tratethethreerequiredpropertiesmentionedabove. Section 3presentsanoverviewofaparticular
gridcomputingenvironmentcalledDIET,whosearchitectureimplementsthegenericorganization
schemeillustratedonFigure1. Morespecifically,wediscusstheneedsofsuchanenvironmentwith
respecttodatamanagement. InSection4,the JUXMEMsoftwaredatamanagementplatformisin-
troducedandweshowhowitcanbeusedasabasistofulfilltheseneeds. Severalaspectsrelatedto
faulttoleranceandconsistencyarediscussedindetail. Section5providesanoverviewoftheglobal
architecture. Finally,Section6concludesanddiscussesfuturedirections.
INRIA
GDS:anArchitectureProposalforaGridData-SharingService 5
Grid Computing Environment
Client Client
Job manager
Computing Computing Computing
server server server
Data Sharing Service
Persistence Fault tolerance Consistency
Figure1: Overviewofadata-sharingservice.
2 Application scenarios
Ourapproachcanbebestmotivatedbyagridapplicationmanaginglargedatasetsandneedingdata
persistence. OnesuchprojectiscalledGrid-TLSE[11]andissupportedbytheFrenchACIGRID
ResearchProgram. ItaimsatdesigningaWebportalexposingthebest-levelexpertiseaboutsparse
matrixmanipulation. Throughthisportal,theusermaygatheractualstatisticsfromrunsofvarious
sophisticated sparse matrix algorithms on his/her specific data. The Web portal provides an easy
access to a variety of sparse solvers, and it assists the comparative analysis of their behavior. The
inputdataareeitherproblemssubmittedbytheuser,orrepresentativeexamplespickedupfromthe
matrixcollectionavailableonthesite. Thesesolversareexecutedonagridplatform. Sincemany
sparsematricesofinterestareverylarge,avoidinguselessdatamovementisofuttermostimportance.
Asophisticateddatamanagementstrategyisthusneeded.
The process for solving a sparse symmetric positive definite linear system, Ax = b, can be
divided into four stages as follows: ordering, symbolic factorization, numerical factorization and
triangularsystemsolution. Wefocusonorderinginthisscenario. Theaimoforderingistofinda
suitablepermutationP ofmatrixA.BecausethechoiceofpermutationP willdirectlydeterminethe
numberoffill-inelements,theorderinghasasignificantimpactonthememoryandcomputational
requirementsforthelatterstages.
RR n°5593
6 G.Antoniu,M.Bertier,L.Bougé,E.Caron,F.Desprez,M.Jan,S.Monnet,P.Sens
Let us consider a typical scenario to illustrate the need for threefold requirement for data per-
sistence, fault tolerance and consistency. It is concerned with the determination of the ordering
sensitivityofaclassofsolverssuchasMUMPS,SuperLUorUMFPACK,thatis,howperformance
isimpactedbythematrixtraversalorder. Itconsistsofthreephases. Phase1exercisesallpossible
internalorderingsinturn. Phase2computesasuitablemetricreflectingtheperformanceparameters
understudyforeachrun: effectiveFLOPS,effectivememoryusage,overallcomputationtime,etc.
Phase3collectsthemetricforallcombinationsofsolvers/orderingsandreportsthefinalrankingto
theuser.
If Phase 1 requires exercising n different kinds of orders with m different kinds of solvers,
thenm(cid:2)nexecutionsaretobeperformed. Withoutpersistence,thematrixhastobesentm(cid:2)n
times. If the server provided persistent storage, the data would be sent only once. If the various
pairssolvers/orderingsarehandledbydifferentserversinPhase2and3,thenconsistencyanddata
movements between servers should be provided by the data management service. Finally, as the
numberofsolvers/orderingsispotentiallylarge,manynodesareused.Thisincreasestheprobability
forfaultstooccur,whichmakestheuseofsophisticatedfault-tolerancealgorithmsmandatory.
Anotherclassofapplicationsthatcanbenefitfromthefeaturesprovidedbyadata-sharingservice
iscode-couplingapplications. Suchapplicationsarestructuredasasetof(generallydistributed)au-
tonomouscodeswhichattimesneedtoexchangedata. ThisschemeisillustratedbytheEPSN[10]
project,alsosupportedbytheFrenchACIGRIDResearchProgram. Thisprojectfocusesonsteer-
ingdistributednumericalsimulationsbasedonvisualization.Itreliesonasoftwareenvironmentthat
combines the facilities of virtual reality with the capabilities of existing high performancesimula-
tions. Thegoalistomakethetypicalwork-flow(modeling, computing, analyzing)moreefficient,
thanks to on-line visualization and interactive steering of the intermediate results. Possible errors
can thus be detected and the the researcher can correct them on-the-fly, by tuning the simulation
parameters. The application consists in a visualization code coupled with one or more simulation
codes. Eachsimulationcodemaybeparallelandmaymanipulatedataaccordingtosomespecific
distribution. As in the case of the Grid-TLSE application described above, a data-sharing service
providingpersistent,transparentaccesstodistributeddatacansimplifythedatamovementschemes
between the coupled codes. Moreover, in the case of code coupling, the basic operations consist
in extracting and modifying the simulation data. As the data status alternates from consistent to
inconsistent during the simulation, it is important for the visualization code to be able to obtain a
consistentviewofthedata. Thiscanbeensuredthankstotheconsistencyprotocolsprovidedbythe
dataservice.
3 Overview of a grid computing environment:
the DIET platform
TheGridRPCapproach[22]isagoodcandidatetobuildProblemSolvingEnvironments(PSE)on
thecomputationalgrid. ItdefinesanAPIandamodeltoperformremotecomputationonservers. In
suchaparadigm,aclientcansubmitproblemtoanagentthatselectsthebestserveramongalargeset
ofcandidates,giveninformationabouttheperformanceoftheplatformgatheredbyaninformation
INRIA
GDS:anArchitectureProposalforaGridData-SharingService 7
service. Thegoalistofindasuitable(ifnotthebest!) trade-offbetweenthecomputationalpower
of the selected server and the cost of moving input data forth and back to this very server. The
choiceismadefromstaticanddynamicinformationaboutsoftwareandhardwareresources,aswell
asthelocationofinputdata,whichmaybestoredanywherewithinthesystembecauseofprevious
computations. Requestscanbethenprocessedbysequentialorparallelservers.
TheGridRPCAPIisthegridformoftheclassicalUnixRPC(RemoteProcedureCall)approach.
It has been designed by a team of researchers within the Global Grid Forum (GGF). It defines a
standard client API to send requests to a Network Enabled Server (NES) system [23], therefore
promoting portability and interoperability between the various NES systems. Requests are sent
through synchronous or asynchronous calls. Asynchronous calls allow a non-blocking execution,
thereby providing another level of parallelism between servers. A function handle represents a
bindingbetweenaproblemnameandaninstanceofsuchfunctionavailableonagivenserver. Of
course several servers can provide the same function (or service) and load-balancing can be done
at the agent level beforethe binding. A sessionID is associatedto each non-blockingrequestand
allowstoretrieveinformationaboutthestatusoftherequest. Waitfunctionsarealsoprovidedfora
clienttowaitforspecificrequesttocomplete. ThisAPIisinstantiatedbyseveralmiddlewaresuch
asDIET[7],Ninf[20],NetSolve[4],andXtremWeb[15].
The paradigm used in the GridRPC model is thus two-level, mixed parallelism, with different
(potentially parallel) requests executed on different servers. However, the server-level parallelism
remainshiddentotheclient.
3.1 Overallarchitectureof DIET
Inthissection,wefocusonourGridRPC-basedmiddleware: theDIETplatform. Thevariousparts
oftheDIETarchitecturearedisplayedonFigure2.
TheClient isanapplicationwhichusesDIETtosolveproblems. Differenttypesofclientsshould
beabletouseDIET,asproblemscanbesubmittedfromawebpage,aspecificPSEsuchas
Scilab,ordirectlyfromacompiledprogram.
TheMasterAgent(MA) receives computation requests from clients. A request is a generic de-
scriptionoftheproblemtobesolved. TheMAcollectsthecomputationalcapabilitiesofthe
availableservers,andselectsthebestoneaccordingtothegivenrequest.Eventually,therefer-
enceoftheselectedserverisreturnedtotheclient,whichcanthendirectlysubmititsrequest
tothisserver.
TheLocalAgent(LA) transmits requests and information between a given MA and the locally
availableservers. Notethat,dependingontheunderlyingnetworkarchitecture,ahierarchyof
LAsmaybedeployedbetweenaMAandtheserversitmanages,sothateachLAistheroota
ofsubtreemadeofitssonLAsandleafservers. EachLAstoresthelistofpendingrequests,
togetherwiththenumberofserversthatcanhandleagivenrequestinitssubtree.Finally,each
LAincludesinformationaboutthedatastoredwithinthenodesofitssubtree.
RR n°5593