Table Of ContentR M A
“This book combines the unique strengths of the authors to create a beautiful coverage of the interplay iga an bit Web Data
bmeatnwaegeenm tehnet .u Int dise rdlyeisntgin deede tpo tbheecoormy, ep trhaec taicuathl oarligtaortiivthem sso,u racned f osro tfhtwisa troep tico oflosr isnt uudseen ttso daanyd fporra Wcteitbio dnaetras ux olesc ebou
u l
alike.”
S R
– Susan B. Davidson, University of Pennsylvania e o
n u
e s
lla se Management
“Web data management is a broad field, and this text manages to cover it all while tying the material r t
t
together brilliantly, conveying it as a single field rather than just a collection of independent topics. It suc-
ceeds in explaining both theory and practice – a difficult task that you do not often see accomplished in any
W
field of computer science. It is a unique book that fills a pressing need.”
– Michael Benedikt, University of Oxford
e
The Internet and World Wide Web have revolutionized access to information. Users now store information across b
multiple platforms from personal computers, to smartphones, to Web sites such as YouTube and Picasa. As a
consequence, data management concepts, methods, and techniques are increasingly focused on distribution con- D
cerns. That information largely resides in the network, as do the tools that process this information.
a
This book explains the foundations of XML, the Web standard for data management, with a focus on data distribution.
t
It covers the many facets of distributed data management on the Web, such as description logics, that are already
a
emerging in today’s data integration applications and herald tomorrow’s semantic Web. It also introduces the ma-
chinery used to manipulate the unprecedented amount of data collected on the Web. Several “Putting into Practice”
M
chapters describe detailed practical applications of the technologies and techniques.
Striking a balance between the conceptual and the practical, the book will serve as an introduction to the new global
a
information systems for Web professionals as well as for master’s level courses.
n
Serge Abiteboul is a researcher at INRIA Saclay and ENS Cachan and cofounder of the start-up Xyleme. His previous
books include the textbook Foundations of Databases. a
g
Ioana Manolescu is a researcher at INRIA Saclay, and the scientific leader of the LEO team, joint between INRIA and
University Paris XI. e
M
Philippe Rigaux is a Professor of Computer Science at the Conservatoire National des Arts et Métiers. He has
co-authored six books, including Spatial Databases.
e
Marie-Christine Rousset is a Professor of Computer Science at the University of Grenoble.
n
Pierre Senellart is an Associate Professor in the DBWeb team at Télécom ParisTech, the leading French engineering
t
school specializing in information technology.
Serge Abiteboul, Ioana Manolescu, Philippe Rigaux,
Marie-Christine Rousset, Pierre Senellart
Cover image: © kalachakra / Photocase
Cover design by David Levy
Web Data Management
TheInternet and World WideWeb have revolutionized access to infor-
mation. Users now store information across multiple platforms from
personalcomputers,tosmartphones,toWebsitessuchasYouTubeand
Picasa. As a consequence, data management concepts, methods, and
techniquesareincreasinglyfocusedondistributionconcerns.Thatinfor-
mation largely resides in the network, as do the tools that process this
information.
This book explains the foundations of XML, the Web standard for
data management, with afocus on data distribution. Itcovers the many
facets of distributed data management on the Web, such as description
logics, thatarealreadyemergingintoday’sdataintegrationapplications
and herald tomorrow’s semantic Web. It also introduces the machinery
used to manipulate the unprecedented amount of data collected on the
Web.Several“PuttingintoPractice”chaptersdescribedetailedpractical
applicationsofthetechnologiesandtechniques.
Strikingabalancebetweentheconceptualandthepractical,thebook
will serve as an introduction to the new global information systems for
Webprofessionalsaswellasformaster’slevelcourses.
Serge Abiteboul is a researcher at INRIA Saclay and ENS Cachan
and cofounder of the start-up Xyleme. His previous books include the
textbookFoundationsofDatabases.
Ioana Manolescu is a researcher at INRIA Saclay, and the scientific
leaderoftheLEOteam,jointbetweenINRIAandUniversityParisXI.
PhilippeRigauxisaProfessorofComputerScienceattheConservatoire
National des Arts et Métiers. He has coauthored six books, including
SpatialDatabases(2001).
Marie-Christine Rousset is a Professor of Computer Science at the
UniversityofGrenoble.
Pierre Senellart is an Associate Professor in the DBWeb team at Télé-
com ParisTech, the leading French engineering school specializing in
informationtechnology.
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE i — #1
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE ii — #2
WEB DATA
MANAGEMENT
Serge Abiteboul
ˆ
INRIASaclay–Ile-de-France
Ioana Manolescu
ˆ
INRIASaclay–Ile-de-France
Philippe Rigaux
CNAM, France
Marie-Christine Rousset
Universityof Grenoble,France
Pierre Senellart
TélécomParisTech,France
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE iii — #3
CAMBRIDGE UNIVERSITY PRESS
Cambridge,NewYork,Melbourne,Madrid,CapeTown,
Singapore,SãoPaulo,Delhi,Tokyo,MexicoCity
CambridgeUniversityPress
32AvenueoftheAmericas,NewYork,NY10013-2473,USA
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781107012431
©SergeAbiteboul,IoanaManolescu,PhilippeRigaux,
Marie-ChristineRousset,andPierreSenellart2012
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2012
PrintedintheUnitedStatesofAmerica
AcatalogrecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloginginPublicationData
Webdatamanagement/SergeAbiteboul...[etal.].
p. cm.
Includesbibliographicalreferencesandindex.
ISBN978-1-107-01243-1(hardback)
1. Webdatabases. 2. Databasemanagement. 3. Electroniccommerce.
I. Abiteboul,S.(Serge)
QA76.9.W43.W41542011
005.7402854678–dc23 2011037996
ISBN978-1-107-01243-1Hardback
Additionalresourcesforthispublicationathttp://webdam.inria.fr/Jorge/
Cambridge University Press has no responsibility for the persistence or accuracy of
URLsforexternalorthird-partyInternetWebsitesreferredtointhispublicationand
does not guarantee that any content on such Web sites is, or will remain, accurate
orappropriate.
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE iv — #4
Contents
Introduction pageix
Part 1 Modeling Web Data
1 Data Model 3
1.1 SemistructuredData 3
1.2 XML 5
1.3 WebDataManagementwithXML 16
1.4 TheXMLWorld 18
1.5 FurtherReading 28
1.6 Exercises 29
2 XPath and XQuery 32
2.1 Introduction 32
2.2 Basics 32
2.3 XPath 42
2.4 FLWORExpressionsinXQuery 54
2.5 XPathFoundations 62
2.6 FurtherReading 67
2.7 Exercises 69
3 Typing 72
3.1 MotivatingTyping 72
3.2 Automata 75
3.3 SchemaLanguagesforXML 80
3.4 TypingGraphData 89
3.5 FurtherReading 91
3.6 Exercises 92
v
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE v — #5
vi Contents
4 XML Query Evaluation 95
4.1 FragmentingXMLDocumentsonDisk 97
4.2 XMLNodeIdentifiers 99
4.3 XMLQueryEvaluationTechniques 103
4.4 FurtherReading 112
4.5 Exercises 113
5 Putting into Practice: Managing an XML Database with EXIST 116
5.1 Prerequisites 116
5.2 InstallingEXIST 117
5.3 GettingStartedwithEXIST 118
5.4 RunningXPathandXQueryQuerieswiththeSandbox 120
5.5 ProgrammingwithEXIST 123
5.6 Projects 127
6 Putting into Practice: Tree Pattern Evaluation Using SAX 131
6.1 Tree-PatternDialects 131
6.2 CTPEvaluation 134
6.3 ExtensionstoRicherTreePatterns 138
Part 2 Web Data Semantics and Integration
7 Ontologies, RDF, and OWL 143
7.1 Introduction 143
7.2 OntologiesbyExample 145
7.3 RDF,RDFS,andOWL 148
7.4 Ontologiesand(Description)Logics 159
7.5 FurtherReading 169
7.6 Exercises 170
8 Querying Data Through Ontologies 171
8.1 Introduction 171
8.2 QueryingRDFData:NotationandSemantics 172
8.3 QueryingThroughRDFSOntologies 176
8.4 AnsweringQueriesThroughDL-LITEOntologies 179
8.5 FurtherReading 194
8.6 Exercises 195
9 Data Integration 196
9.1 Introduction 196
9.2 ContainmentofConjunctiveQueries 199
9.3 Global-as-ViewMediation 200
9.4 Local-as-ViewMediation 204
9.5 Ontology-BasedMediators 215
9.6 Peer-to-PeerDataManagementSystems 222
9.7 FurtherReading 229
9.8 Exercises 229
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE vi — #6
Contents vii
10 Putting into Practice: Wrappers and Data Extraction
with XSLT 231
10.1 ExtractingDatafromWebPages 232
10.2 RestructuringData 234
11 Putting into Practice: Ontologies in Practice (by Fabian
M. Suchanek) 236
11.1 ExploringandInstallingYago 236
11.2 QueryingYago 237
11.3 WebAccesstoOntologies 238
12 Putting into Practice: Mashups with YAHOO! PIPES and XProc 240
12.1 YAHOO!PIPES:AGraphicalMashupEditor 240
12.2 XProc:AnXMLPipelineLanguage 241
Part 3 Building Web Scale Applications
13 Web Search 247
13.1 TheWorldWideWeb 248
13.2 ParsingtheWeb 250
13.3 WebInformationRetrieval 257
13.4 WebGraphMining 272
13.5 HotTopicsinWebSearch 280
13.6 FurtherReading 281
13.7 Exercises 283
14 An Introduction to Distributed Systems 287
14.1 BasicsofDistributedSystems 288
14.2 FailureManagement 295
14.3 RequiredPropertiesofaDistributedSystem 299
14.4 ParticularitiesofP2PNetworks 303
14.5 CaseStudy:ADistributedFileSystemforVeryLargeFiles 305
14.6 FurtherReading 308
15 Distributed Access Structures 310
15.1 Hash-BasedStructures 311
15.2 DistributedIndexing:SearchTrees 325
15.3 FurtherReading 336
15.4 Exercises 337
16 Distributed Computing with MAPREDUCE and PIG 339
16.1 MAPREDUCE 341
16.2 PIG 348
16.3 FurtherReading 359
16.4 Exercises 361
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE vii — #7
viii Contents
17 Putting into Practice: Full-Text Indexing with LUCENE (by
Nicolas Travers) 364
17.1 Preliminary:ALUCENESandbox 364
17.2 IndexingPlainTextwithLUCENE–AFullExample 364
17.3 PutItintoPractice! 371
17.4 LUCENE–TuningtheScoring(Project) 372
18 Putting into Practice: Recommendation Methodologies (by Alban
Galland) 374
18.1 IntroductiontoRecommendationSystems 374
18.2 Prerequisites 375
18.3 DataAnalysis 377
18.4 GeneratingSomeRecommendations 380
18.5 Projects 385
19 Putting into Practice: Large-Scale Data Management with
HADOOP 387
19.1 InstallingandRunningHADOOP 388
19.2 RunningMAPREDUCEJobs 391
19.3 PIGLATINScripts 395
19.4 RunninginClusterMode(Optional) 395
19.5 Exercises 397
20 Putting into Practice: COUCHDB, a JSON Semistructured
Database 400
20.1 IntroductiontotheCOUCHDBDocumentDatabase 401
20.2 PuttingCOUCHDBintoPractice! 417
20.3 FurtherReading 419
Bibliography 421
Index 431
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE viii — #8
Introduction
The Internet and the Web have revolutionized access to information. Individuals
are depending more and more on the Web to find or publish information, down-
load music and movies, and interact with friends in social networking Web sites.
Following a paralleltrend, companiesgomore andmore towardWebsolutions in
theirdailyactivitybyusing Webservices(e.g., agenda)aswellasbymovingsome
applications into the cloud (e.g., with Amazon Web services). The growth of this
immense information source is witnessed by the number of newly connected peo-
ple,bytheinteractionsamongthemfacilitatedbythesocialnetworkingplatforms,
and above all by the huge amount of data covering all aspects of human activity.
WiththeWeb,informationhasmovedfromdataisolatedinveryprotectedislands
(typically relational databases) to information freely available to any machine or
anyindividualconnectedtotheInternet.
Perhaps the best illustration comes from a typical modern Web user. She has
informationstoredonPCs,apersonallaptop,andaprofessionalcomputer,butalso
possiblyonsomeserveratwork,onhersmartphone,inane-book,andsoon.Also,
she maintains information in personal Web sites or social network Web sites. She
maystore picturesinPicasa, moviesinYouTube, bookmarks inFirefoxSync, and
the like. So, even an individual is now facing the management of a complex dis-
tributedcollectionofdata.Onadifferentscale,publicorprivateorganizationsalso
havetodealwithinformationproducedandstoredindifferentplaces,orcollected
ontheWeb,eitherasasideeffectoftheiractivity(e.g.,worldwidee-commerceor
auctionsites)orbecausetheydirectlyattempttounderstand,organizeandanalyze
datacollectedontheWeb(e.g.,searchengines,digitallibraries,orWebintelligence
companies).
Asaconsequence,amajortrendintheevolutionofdatamanagementconcepts,
methods, and techniques is their increasing focus on distribution concerns: Since
information now mostly resides in the network, so do the tools that process this
information to make sense of it. Consider for instance the management of inter-
nalreportsinacompany.Typically,manycollectionsofreportsmaybemaintained
in different local branches. To offer a unique company-wide query access to the
ix
Abiteboul: “Prelims” — 2011/10/28 — 14:16 — PAGE ix — #9
