High quality Linked Data generation from heterogeneous data sources

195 Pages·2017·8.5 MB·English

by  Anastasia Dimou

High Quality Linked Data Generation
from Heterogeneous Data
De generatie van kwaliteitsvolle gelinkte data
uit heterogene gegevens
Anastasia Dimou
31 oktober 2017
Table of Contents
ListofFigures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
ListofTables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1 Introduction 1-1
1.1 SemanticWeb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3
1.2 LinkedData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4
1.3 ResourceDescriptionFramework. . . . . . . . . . . . . . . . . . . . . . 1-7
1.4 KnowledgeRepresentation . . . . . . . . . . . . . . . . . . . . . . . . .1-10
1.5 LinkedDatalifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-12
1.5.1 Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-12
1.5.2 Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-13
1.5.3 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-13
1.5.4 Provenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-13
1.5.5 Publication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-14
1.6 ProblemStatement . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-15
1.6.1 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-17
1.6.2 ResearchQuestions . . . . . . . . . . . . . . . . . . . . . . . . .1-17
1.6.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-18
1.6.4 Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-19
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-19
2 Declaration 2-1
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3
2.2 StateoftheArt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
2.2.1 MappingLanguages . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
2.2.2 R2RML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6
2.2.3 Editors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8
2.3 Limitationsandrequirements . . . . . . . . . . . . . . . . . . . . . . . . 2-8
2.3.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8
2.3.2 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9
2.3.3 R2RMLgeneralization . . . . . . . . . . . . . . . . . . . . . . . .2-10
2.4 RMLLanguage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2-11
2.5 RMLExtensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2-15
2.5.1 GracefulDegradation . . . . . . . . . . . . . . . . . . . . . . . . .2-15
2.5.2 DataTransformations . . . . . . . . . . . . . . . . . . . . . . . . .2-15
2.6 RMLEditor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2-18
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2-21
c TableofContents
3 Execution 3-1
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3
3.2 StateoftheArt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3
3.2.1 Structure-andformat-specifictools . . . . . . . . . . . . . . . . . 3-4
3.2.2 R2RMLprocessors . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8
3.3 ExecutionFactors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-9
3.3.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-10
3.3.2 Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-10
3.3.3 Materialization . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-11
3.3.4 Location . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-12
3.3.5 Drivingforce . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-12
3.3.6 Trigger. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-13
3.3.7 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-13
3.3.8 Dynamicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-14
3.4 RMLMapper. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-14
3.4.1 Phases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-15
3.4.2 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-16
3.4.3 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-16
3.4.4 Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-19
3.4.5 GraphicalInterfaces . . . . . . . . . . . . . . . . . . . . . . . . .3-21
3.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-21
3.5.1 Formal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-21
3.5.2 Experimental . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-22
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-26
4 Quality 4-1
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3
4.2 StateoftheArt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-4
4.3 QualityAssessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5
4.3.1 LinkedDataQualityAssessment . . . . . . . . . . . . . . . . . . . 4-6
4.3.2 MappingRulesQualityAssessment . . . . . . . . . . . . . . . . . 4-7
4.4 MappingRulesRefinements . . . . . . . . . . . . . . . . . . . . . . . .4-10
4.5 RMLValidator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4-12
4.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4-13
4.6.1 Usecases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4-13
4.6.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4-15
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4-17
5 Workflow 5-1
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
5.2 StateoftheArt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
5.2.1 DatasetandServiceDescriptions . . . . . . . . . . . . . . . . . . 5-5
5.2.2 LinkedDatapublishingcycle . . . . . . . . . . . . . . . . . . . . . 5-6
5.2.3 ProvenanceandMetadataVocabularies . . . . . . . . . . . . . . . 5-7
5.2.4 ApproachesfortracingPROV&metadata . . . . . . . . . . . . . . 5-8
5.2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-10
5.3 GenerationWorkflow . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-10
5.3.1 AccessInterfaces . . . . . . . . . . . . . . . . . . . . . . . . . . .5-12
TableofContents d
5.3.2 DataSource . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-14
5.3.3 BindingCondition . . . . . . . . . . . . . . . . . . . . . . . . . . .5-16
5.4 RefinementWorkflow . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-17
5.5 MetadataandProvenance . . . . . . . . . . . . . . . . . . . . . . . . .5-18
5.5.1 HighLevelWorkflow . . . . . . . . . . . . . . . . . . . . . . . . .5-21
5.5.2 MetadataDetailsLevels . . . . . . . . . . . . . . . . . . . . . . .5-22
5.6 RMLWorkbench. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-24
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5-27
6 UseCases 6-1
6.1 OpenData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-4
6.1.1 EWI OpenDataDataModel . . . . . . . . . . . . . . . . . . . . . . 6-4
6.1.2 EWI OpenDataVocabularies . . . . . . . . . . . . . . . . . . . . . 6-5
6.1.3 EWI OpenDataWorkflow . . . . . . . . . . . . . . . . . . . . . . . 6-5
6.2 iLastic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-6
6.2.1 iLasticModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
6.2.2 iLasticVocabulary . . . . . . . . . . . . . . . . . . . . . . . . . . 6-8
6.2.3 iLasticLinkedDataset . . . . . . . . . . . . . . . . . . . . . . . . 6-9
6.2.4 iLasticWorkflow . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-9
6.3 COMBUST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-12
6.3.1 COMBUSTModel. . . . . . . . . . . . . . . . . . . . . . . . . . .6-14
6.3.2 COMBUSTVocabulary . . . . . . . . . . . . . . . . . . . . . . . .6-15
6.3.3 COMBUSTLinkedDataset . . . . . . . . . . . . . . . . . . . . .6-15
6.3.4 COMBUSTWorkflow . . . . . . . . . . . . . . . . . . . . . . . . .6-17
6.4 DBpedia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-17
6.4.1 DBpediaGenerationLimitations . . . . . . . . . . . . . . . . . . .6-18
6.4.2 DBpediaWorkflow . . . . . . . . . . . . . . . . . . . . . . . . . .6-19
6.4.3 DBpediaLinkedDatasubsets . . . . . . . . . . . . . . . . . . . .6-21
6.5 CEUR-WS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-22
6.5.1 SemanticPublishingChallenge . . . . . . . . . . . . . . . . . . .6-23
6.5.2 RML-basedWorkflow . . . . . . . . . . . . . . . . . . . . . . . . .6-24
6.5.3 SolutionsDescription . . . . . . . . . . . . . . . . . . . . . . . . .6-26
6.5.4 SolutionsComparison . . . . . . . . . . . . . . . . . . . . . . . .6-27
6.5.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-30
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-31
7 Conclusions 7-1
List of Figures
1.1 LinkedOpenDatacloudin2007 . . . . . . . . . . . . . . . . . . . . . . 1-5
1.2 LinkedOpenDatacloudin2017 . . . . . . . . . . . . . . . . . . . . . . 1-6
1.3 AnRDFtripleconsistsofasubject,predicate,andobject . . . . . . . . . 1-7
1.4 OverviewofthisPhDthesis . . . . . . . . . . . . . . . . . . . . . . . . .1-19
2.1 R2RMLextensibleelements . . . . . . . . . . . . . . . . . . . . . . . .2-11
2.2 Diagram with RML elements. In light gray the R2RML elements which
weregeneralizedaredepicted. Indarkgray, theextensionoverR2RML
aredepicted.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2-12
2.3 MappingsourceswithoutandwithRML . . . . . . . . . . . . . . . . . .2-13
2.4 RMLEditorGraphicalUserInterface . . . . . . . . . . . . . . . . . . . .2-19
3.1 RMLMapper’scomponentsandoverallarchitecture . . . . . . . . . . . .3-16
3.2 LinkedDatagenerationfromdatainCSV,JSONandXMLformat(logarith-
micseconds) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-24
3.3 LinkedDatagenerationfromdatainJSONformatwithdiversedepth(from
1to5levels–logarithmicseconds) . . . . . . . . . . . . . . . . . . . . .3-25
3.4 Performancecomparison: MappingsExtractorvsRMLExtractor . . . . .3-25
4.1 SimilarlyoccursfortheSubjectMap. . . . . . . . . . . . . . . . . . . . . 4-7
4.2 RMLValidatorarchitecture . . . . . . . . . . . . . . . . . . . . . . . . . .4-13
5.1 DataretrievalandLinkedDatageneration. AtripleconsistsofRDFTerms
which are generated by Term Maps (4). Those Term Maps are defined
with mapping rules and are instantiated with data fractions referred to
usingareferenceformulation relevanttothecorrespondingdataformat.
Thosefractionsarederivedfromdataextractedatacertainiteration(3)
from a logical source (2). Such a logical source is formed by data re-
trievedfromarepository(1)whichisaccessedasdefinedusingthecor-
respondingdatasetorservicedescriptionvocabulary. . . . . . . . . . . .5-10
5.2 QualityAssessmentenabledLinkedDatamappingandpublishingworkflow5-17
5.3 AcoordinatedviewofLinkedDatapublishingworkflowactivities. . . . . .5-20
6.1 LinkedDatasetGenerationandPublishingWorkflowfortheiLasticproject.6-9
6.2 A publication as presented at iLastic user interface with information de-
rived both from iMinds data warehouse and the analyzed and enriched
publication’scontent. . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-11
6.3 ThegraphexplorerviewforErikMannens. . . . . . . . . . . . . . . . . .6-12
ListofFigures f
6.4 COMBUSTintegration. . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-13
6.5 Screenshotofaviolationslistpresentedtothe DBpediacommunity. For
everyviolatingmappingrule,thepredicatewiththeexistingRDFterm,ac-
cordingtothecorrespondingDBpediamapping,andtheexpectedvalue,
accordingtotheDBpediaontologyarepresented. . . . . . . . . . . . . .6-20
List of Tables
1.1 AmappingofLinkedDatainterfacestoappropriatestoragesolutions. . . .1-15
2.1 R2RMLVsRML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2-12
3.1 Mappingrulesdefinitionandsupporteddataformatsperimplementation. . 3-8
3.2 FactorsaffectingLinkedDatagenerationanddetermininghowcorrespond-
ing algorithms should be designed and tools should be implemented re-
spectively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-10
3.3 RMLMapper’scomponents . . . . . . . . . . . . . . . . . . . . . . . . . .3-17
3.4 Execution time in seconds for Linked Data generation from data sources
indifferentformatsandwhosenumberofrecordsrangesbetween10and
100,000records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-23
3.5 Execution time in seconds for Linked Data generation from data sources
indifferentformatsandwhosenumberofrecordsrangesbetween10and
100,000records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-24
4.1 Semantic violations detected after assessing the mapping rules’ validity.
Thefirstcolumndescribesthetypeofviolation,theseconditslevel(Warn-
ing or Error), the third the expected RDF term according to the schema
(ontology or vocabulary), while the fourth the RML term map which define
howtheRDFtermisgenerated. Thelastspecifiespotentialrefinementthat
canbeapplied. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8
4.2 Semantic violations detected after assessing the mapping rules’ validity.
Thefirstcolumndescribesthetypeofviolation,whilethesecondspecifies
potentialrefinementthatcanbeapplied. . . . . . . . . . . . . . . . . . . .4-11
4.3 Evaluationresultssummary. Inthe DatasetAssessment part, weprovide
the number of triples (Size) and test-cases (TC), evaluation Time (Time),
Failedtest-cases(Fail.) andtotalindividualViolations(Viol.). IntheMap-
pingAssessmentpart,weprovidethemappingdocumentnumberoftriples(Size),
evaluationTime(Time), Failed TCs(Fail.) andViolationinstances(Viol.).
Finally,weprovidethenumberofLinkedDatasetviolationsthatcanbead-
dressedbyrefiningthemappingrules(Ref.) andestimatedcorresponding
LinkedDatasetviolationsthatareresolved(Affect. triples). . . . . . . . .4-17
5.1 Tableofpropertiesrequiredforeachentity . . . . . . . . . . . . . . . . . .5-18
5.2 Tableofpropertiesrequiredforeachentity . . . . . . . . . . . . . . . . . .5-19
6.1 UseCasessummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-36
ListofTables h
6.2 InputDatausedintheEWI OpenDatausecase . . . . . . . . . . . . . . .6-37
6.3 Classes&propertiesusedtoannotatetheEWI OpenData . . . . . . . .6-37
6.4 Classes&propertiesusedtosemanticallyannotatetheiLasticdatamodel6-37
6.5 Classes & properties used to annotate the COMBUST partners Linked
Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-38
6.6 Classes&propertiesusedtoannotateCEUR-WS.orgworkshopproceed-
ingsvolumes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-39
6.7 Task1solutionsforSemanticPublishingChallenge: theirprimaryanalysis
methods,methodologies,implementationsbasisandevaluationresults. . .6-39
6.8 Task1and2solutions: thevocabulariesusedtoannotatethedata. . . . .6-40
6.9 Statisticsabouttheproduceddataset(Task1–2014and2015editions) . .6-40
6.10 Numberofentitiesperconceptforeachsolution(Task1–2014and2015
editions) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-40
6.11 Statisticsaboutthemodel(Task1–2014and2015editions) . . . . . . . .6-41
List of Acronyms
API ApplicationProgrammingInterface
ADMS AssetDescriptionMetadataSchema
B2B Business-to-business
BiBO BiBliographicOntology
CSV CommaSeparatedValues
CSS CascadingStyleSheets
CDFLGD ContactDetailsofFlemishLocalGovernmentsDataset
CERIF CommonEuropeanResearchInformationFormatOntology
COMBUST CombiningMulti-trUstBusinesSdaTa
CSV CommaSeparatedValues
CSVW CSVontheWeb
CWA ClosedWorldAssumption
DBLP DigitalBibliography&LibraryProject
DBO DBpediaontology
DCAT DtaCATalogvocabulary
DCMI DublinCoreMetadataInitiative
DQA DataQualityAssessment
DQTP DataQualityTestPattern
DWH DataWareHouse
EBNF ExtendedBackus-NaurForm
ESWC European/ExtendedSemanticWebConference
ETL Extract,Transform,Load
EWI Economie,WetenschapenInnovatie
FaBiO FRBR-alignedBibliographicOntology
FOAF TheFriendOfAFriendontology
FnO FunctionOntology
GAV Global-As-View
GIM GeographicInformationManagement
GML GeographyMarkupLanguage
GRDL GleaningResourceDescriptionsfromDialectsofLanguages
GREL GeneralRefineExpressionLanguage
GR GoodRelationsontology
GSoC GoogleSummerofCode
GUI GraphicalUserInterface
HDT Header-Dictionary-Triples
HTML HyperTextMarkupLanguage
HTTP HyperTextTransferProtocol
ListofAcronyms j
IRI InternationalizedResourceIdentifier
ISO InternationalOrganizationforStandardization
ISWC InternationalSemanticWebConference
JAPE JavaAnnotationPatternsEngine
JDBC JavaDataBaseConnectivity
JSON JavaScriptObjectNotation
JSON-LD JSONforLinkedData
KML KeyholeMarkupLanguage
LAV Local-As-View
LDOW LinkedDataOntheWeb
LD LinkedData
LOD LinkedOpenData
LIMES LInkdiscoveryframeworkforMEtricSpaces
LOV LinkedOpenVocabularies
LOCN LOCatioNCoreVocabulary
MQA MappingQualityAssessment
N3 Notation3
NER NamedEntityRecognition
ODBC OpenDataBaseConnectivity
ORG ORGanizationontology
OSLO OpenStandardsforLocalAdministrations
OWL WebOntologyLanguage
PDF PortableDocumentFormat
PROV-O PROVOntology
RDF ResourceDescriptionFramework
RDFa RDFAnnotations
RDFS RDFSchema
RSS ReallySimpleSyndication
SHACL ShapesConstraintLanguage
SKOS SimpleKnowledgeOrganizationSystem
SML SparqlificationMappingLanguage
SPAR SemanticPublishingAndReferencingontologies
SPARQL SPARQLProtocolandRDFQueryLanguage
SPARQL-SD SPARQLServiceDescription
SPC SemanticPublishingChallenge
SPIN SPARQLInferencingNotation
SQL StructuredQueryLanguage
SSD SolidStateDisk
SWC SemanticWebConference
SWDF SemanticWebDogFood
SWRC SemanticWebforResearchCommunities
SVM SupportVectorMachine
TDT TheDataTank
TPF TriplePatternFragments
Turtle TerseRDFTripleLanguage
URI UniversalResourceIdentifier
URL UniversalResourceLocator