Linking Library Metadata to the Web: the German Experiences

0.23 MB·English

by  Gabriele Meßmer

Linking library metadata to the web:
the German experience
GabrieleMeßmer
«Whatisthevalueofacatalogueofmorethan23millionrecords?»was
oneofthequestionswediscussedwhenstartingthelinkedopen
data project at the Bavarian State Library. Many generations of
librarianshavebeendoingagoodjobcreatingmachine-readable
cataloguerecords,nowadayscalledmetadata,withthepurposeof
describingbooks,printedmusic,manuscriptsandmaps,ofbuilding
upauthorityfiles,listingholdingsandmore.Toincreasethevalueof
theseexpensivedata,itisessentialtodaynotonlytooffercatalogue
interfaces to retrieve information, but also to open the catalogue
databasesandtogivefreeaccesstotherecords.
InGermanysomelibrariesoftheNorth-Rhine-Westphalianlibrary
networkwerethefirsttopublishtheirrecordsaslinkedopendata
(LOD) in March 2010. At the same time, the hbz created a LOD
website. Sincethenmoreandmorelibrarieshavestartedtodiscuss
the issue of open and linked open data as well as the question
of making their data freely accessible. In Germany there are six
librarynetworksrunningfivedifferentunioncatalogs. Although
the German cataloging networks and the libraries have worked
closelytogetherformanyyears,sharingcommonauthorityfilesas
wellasexchangingrecordsforre-use,therearestillinmanycases
differentrecordsforoneandthesamebookinthevariousunion
JLIS.it.Vol.4,n.1(Gennaio/January2013).
DOI:10.4403/jlis.it-5507
G.Meßmer,Linkinglibrarymetadatatotheweb
catalogs.
Becauseofthespecialsituationoffivedifferentcoexistinglibrary
networkcatalogs,thereisnouniformidentifierforcataloguerecords
in Germany. Many records already have an OCLC number that
servesalmostassuchanidentifier,butthisisbynomeansthecase
forallrecords. Sometimesrecordsdescribingthesamebookmay
havetwoorevenmoredifferentidentifiers. AfterstartingtheLOD
projectsinGermany,itbecameimmediatelyobviousthatacommon
andpersistentidentifierwasneededforeverytitlerecordandideally
onlyonesingleidentifierfordifferentrecordsdescribingoneand
thesameresource. SoCulturegraph1wasborn,alinkedopendata
service with the aim to generate a specific identifier for all kinds
ofobjectsheldbylibrariesinGermany. Thisidentifiershouldbe
usedtoreferencethedescriptionofvariousobjects. Itshouldhavea
definedsyntax,mustbeuniqueandpersistent.
In the first step, the German library networks provided records
formonographsandmulti-partworkspublishedafter1945tobe
ingested into the Culturegraph database. Then the records were
comparedandbundles(clusters)werecreatedwithrecordswhich–
althoughslightlydiffering–weresupposedtobelongtogetherand
describedthesameobject. Anumberofidentifiersservedasmatch
criteriasuchasISBN,ISMN,OCLCnumberandothers. Bynowa
resolving and look-up service is available to retrieve these single
titlesorbundlesinCulturegraph. Thenextstepwillbetoestablisha
largerdatabasewithallrecordsheldintheGermanlibrarynetworks,
ataskwhichisespeciallychallengingwhenitcomestoearlyprinted
bookswithlongbaroquetitles.
TheNorth-Rhine-WestphalianLibraryServiceCenter(hbz)commis-
sionedtheBerlinlawyerDr.TillKreutzertoanalysethelegalaspects
ofopenlibrarydata. In2011hepublishedaguideline(OpenData
1http://www.culturegraph.org.
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.392
JLIS.it.Vol.4,n.1(Gennaio/January2013)
–FreigabevonDatenausBibliothekskatalogen)whichisagoodbasis
for all legal questions around open data. The guideline contains
chaptersabouttheprotectabilityofdata,especiallyofsingledata
fields. Itexaminesdatabasesandcollectionsofdatasuchascatalogs
respectivelyanditdiscussesthepossibilitytoreleasecatalogsand
publishrecordsasopendata.
ThefirstlinkedopendataserviceoftheGermanNationalLibrary
was the publication of the authority file for personal names and
theauthorityfileforsubjectheadingsin2010. Today,theGerman
NationalLibrarymakesavailablethreedatasets: titlerecordsofthe
main collection (without records for printed music), the German
UnionCatalogueofSerials(Zeitschriftendatenbank,ZDB)andthe
completeGermanintegratedauthorityfile(GemeinsameNormdatei,
GND). The data model for the bibliographic data is documented
in a paper (The Linked Data Service of the German National Library:
Modellingofbibliographicdata), whichisavailableonthewebalso
inEnglish. AlldatasetsarepublishedunderaCreativeCommons
Zero(CC0)license.
The B3Kat project – open data
In2010theCatalogingandMetadataCommissionoftheBavarian
andtheBerlin-Brandenburglibrarynetworksalsostartedtodiscuss
openandlinkedopendata. TheresearchlibrariesofBavaria,Berlin
andBrandenburguseacommoncataloguedatabasecalledB3Kat.
Thiscataloguecontainsmorethan23millionrecordsof180member
libraries. The records are held in MAB, the special German data
exchangeformat,andarelinkedtorecordsoftheGermanauthority
file. A small working group was established in order to achieve
quick results. At first the working group identified reasons for
havingopencataloguemetadata.
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.393
G.Meßmer,Linkinglibrarymetadatatotheweb
• tomakelocalandregionaldatavisibleworldwide,i.e. tohave
themnolongerhiddeninthedeepweb;
• tocompleteandincreasethevalueofalreadyestablishedweb
presencessuchasWikipediabygivinglinkstoauthoritative
resources;
• toprovidedatafornewlydevelopedweb-basedservices;
• tointegratedataintothesemanticwebwiththepossibilityto
re-usethecompletedandenricheddataone’sowncatalogue
environment;
• tocontributetoandtoactivelypromotetheopenaccessmove-
ment.
IntheprojectpartAanOAIPMHrepositorywasestablishedinthe
Aleph500environmentofB3Kat. ThetitlerecordsheldintheAleph
systemandstructuredaccordingtotheGermanexchangeformat
MABwereconvertedtoMARCXMLandthenprovidedasopendata.
TheparticularchallengewasontheonehandtomapasmanyMAB
fieldsaspossibletoMARCXML,inordertoincludeamaximumof
information,andontheotherhandtoincludethebasicinformation
abouttheowninglibraries. Asthedatapoolcontainsmorethan23
millionrecords–ahugeamountofdata–twodifferentmethods
are offered to pick up the records: there is the complete data set,
splitinthreeparts,frozenatacertaindate,andanOAIrepository
comprisingallcontinuouslyongoingupdatesofrecordsornewly
createdrecords.2Itispossibletodownloadthewholedatasetorto
selectonlytherecordsofaspecificlibrary,toselectasinglerecord,if
theB3KatIDnumberisknown,ortoobtaindefinedsetsofrecords
2Informationabouttheopendatapoolcanbefoundhere: https://opacplus.
bib-bvb.de/TouchPoint_touchpoint/help.do?helpContext=opendata_en.
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.394
JLIS.it.Vol.4,n.1(Gennaio/January2013)
whichcanbeidentifiedbecausetheyincludecertaincodesorfields.
Thecompletedatasetwillbepublishedtwiceayear. AsofMarch
2012thecompletesetorpartsofthesethavebeendownloadedby
thisapproachmorethan400times.
The B3Kat Project – linked open data
This MARCXML based open data pool serves as a basis for the
linkedopendatapool(partBoftheproject). Thiswastheeasiest
wayforthenextstep,thetransformationtoRDF,becausethereareal-
readytoolsforthisprocess.ManyfieldsprovidedintheMARCXML
formathadtobemappedtotheRDFdatamodelandpublishedas
RDFdata. WhereverpossibleURIsarebeingused. Thereforefor
everytitlerecordauniformresourceidentifier(URI)wascreated,
basedontheB3Katidentitynumber(startingwithBV)andthename
spacereservedespeciallyforB3Katlod.b3kat.de. Thisnamespace
wasregisteredatDENIC,aregistryforGermandomainsunderthe
topleveldomain.de.
Tolinkthedatatootherdataaswellaspossiblemanymorelinks
wereimplementedwhichwereprovidedbytheparticularcontent
ofourrecords: forexamplelinkstotheGermanintegratedauthority
file,toWorldCat,tothelanguagecodeISO639-2,totheLibraryof
CongresssubjectheadingsandtotheDeweyDecimalClassification.
Currently, the RDF data pool consists of about 600 million RDF
triples. As of March 2012 the pool has been downloaded more
than680timesandhadmorethan7600visits. Forthetimebeing
the linked open data set is published only bi-annually, while the
updatesfortheopendataarecontinuouslyprovided. Information
abouttheRDFdataset,thedatamodel,theusedontologiesandthe
SPARQLendpointcanbefoundontheprojectwebpage.3
3http://lod.b3kat.de.CurrentlythispageisonlyinGerman.
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.395
G.Meßmer,Linkinglibrarymetadatatotheweb
Figure1:EsempiodiunrecordinB3kat.
BoththeopendataandthelinkedopendatapoolofB3Katwentlive
inthefirstdaysofDecember2011. Atthemomentitisthelargest
bibliographicrecordortitlesetavailableinGermany.
Legal aspects
Animportanttopicinthediscussionaboutopendataisthelegal
aspect. Theworkinggrouphadtoconsiderquestionssuchas:
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.396
JLIS.it.Vol.4,n.1(Gennaio/January2013)
• Will service providers agree with their records being pub-
lished?
• Willalllibrariesacceptthepublicationoftheirrecords?
• Aretherefields/tagsthatshouldnotbepublished,e.g. subject
headingsorURLs?
• What about catalogue enrichment, e.g. abstracts or table of
contentsintegratedintherecords?
• Underwhichlicenseshouldopendatabeprovided?
Some German libraries decided not to publish the full records as
openorlinkedopendata,buttoomitfromtheopendatasomefields
suchasURLs. Somelibrariansbelievethatparticularlyexpensive
parts of a record, like subject headings, should not be published
forfree. TheBavarian-Berlin-Brandenburgworkinggrouphowever
recommended to publish the records as completely and fully as
possible–inordertomakethemreallymeaningfulforallinterested
partiesandtomakesurethatthisserviceisalsobeneficialforthe
generaldataexchangebetweenlibrariesandnetworks. Forthetime
being only the URLs linking to table of contents purchased from
commercial service providers cannot be published for copyright
reasons.
Beforepublishingmetadataitisalsonecessarytoconsiderandto
definethelegalconditionsfortheirreuse. Therearetwomodels:
• topublishdataunderaspeciallicenseor
• towaiveanyrights.
ToreallycomplywiththeconceptandprerequisitesofLinkedOpen
Data (LOD)it is necessary to provide these data without any re-
strictions under a completely free license. The B3Kat records are
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.397
G.Meßmer,Linkinglibrarymetadatatotheweb
thereforepublishedunderCC0UniversalPublicDomainDedica-
tion,whichisalsousedforthemetadataintheEuropeanacontext.
Thisallowsthemaximumuseoftherecordsandtheproviderhas
noadministrativeoverheadtocontrolthelicensesoftheusers.
Conclusions
The project has come to an end, but it is not finished. There are
stillthingstodo: bothprojectwebpagesmustalsobepublishedin
English. Thelicenseinformationmustbeintegratedintotherecords,
theMARCXMLrecordsaswellasintheRDFones. Furthermore,an
updateprocessisneededtokeepthelinkedopendataup-to-date
aswell,whichimpliestohandlecorrectedanddeletedrecords. A
requestoftenarticulatedistopublishthecompletesetof23million
recordsintheMARCXMLbasedOAIPMHrepository,notsplitit
asitisinthreeparts. Uptonowthiscouldnotberealizedbecause
ofperformanceandhardwareissues. Bothpartsoftheprojectwere
successful. Theimplementationwasrealizedquickly,thesetsarefre-
quentlyindemandandwelearnedalotaboutpublishingmetadata
inMARCXML(ourexperiencesofarhasbeenpredominantlywith
theGermanMABformat)andaboutdoingthisinanOAIrepository.
Techniciansandcatalogershadtoworkhandinhandtogetthebest
outoftheexistingdata. Publishingopendataisnojobtodoonthe
fly,alongsidethedailywork. Itneedstimeandmoney,becausea
soundcalculationisnecessarytoaccountforstaffcost,forthehard-
andsoftwareneededandaboveallforthetimerequiredtoholdthe
dataup-to-date.
Providingopendatamayalsomeanashiftindatamanagement
insidethelibrarycommunity. Untilnow,deliveryofdatafordiffer-
entpurposeswasuptothedatabaseproviderandalotofworkhad
tobedonefordifferentfiledefinitionsandtransfers,alwaysupto
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.398
JLIS.it.Vol.4,n.1(Gennaio/January2013)
thesender,nottherecipient.WithZ39.50andevenmorewithOAIit
wasstilluptothedataprovidertodefinethemethodforthedelivery
ofdataandthusthestructureandthefieldstobesupported. With
opendataandevenmorelinkedopendataitisnowuptotheuserto
maketherelevantchoicesandselectionsandevenare-modellingof
recordsontheirside. Neverthelessthelearningcurveinanalyzing
the data provided in order to make best use of them is still to be
followedandalotofstandardizationandharmonizationofformats
andcontentsisstilltobedoneinordertomaketheuseofopendata
asmoothmethodoflibrarycooperationandrecordreuse.
Until late summer 2012 yet another OAI repository will be estab-
lishedinthecontextoftheEuropeanaLibrariesprojectinwhichthe
BavarianStateLibraryisoneofthepartners. Oneoftheoutcomesof
theEuropeanaLibrariesprojectistheReportonthealignmentoflibrary
metadatawiththeEuropeanaDataModel(EDM). Thisrepositorywill
alsousetheopenMARCXMLdata,butitwillonlycontainmetadata
ofdigitizedobjects. Thesemetadatawillbeenrichedwithlinksto
thumbnails. Notleastthisrepository–alsoinMARCXML–will
serveasadatapoolforGermanandEuropeanportalswhichpresent
metadataforallkindsofdigitalmaterials. Theexpectedadvantage
ofEDMistoenrichtherecordsinEuropeanaandthusmakethem
interoperableandfitforthesemanticweb.WithEDMforEuropeana
andCIDOCCRM,thecommondataformatforthefutureGerman
DigitalLibrarywecanclearlyseewhichfurtherrequirementscome
upwhendealingwithdigitizedinformation. Itisnotonlyathumb-
nail–itmaybeallimages(surelynotintheoriginalhighresolution,
butinalowerone)andthestructuralmetadataastransportedin
METS/MODSfilesandfinallythefulltextinformation.
Linkedopendatawillleadtonewserviceswhichwillbedeveloped
inthenearfuture. Theyheavilyrelyonalotofbasicknowledge
oflibrarians: metadata,structures,normalization,qualitycontrol,
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.399
G.Meßmer,Linkinglibrarymetadatatotheweb
standardnumbersetc.Thelibrarianoftodayisnolongeracataloger,
but a metadata specialist and a manager – working on how the
richinformationcontainedinlibraryrecordscanbemostusefully
exploitedandintegratedintotheweb.
References
Kreutzer,Till.OpenData–FreigabevonDatenausBibliothekskatalogen.Köln:Hochschul-
bibliothekszentrumdesLandesNordrhein-Westfalen,2011.http://www.hbz-
nrw.de/dokumentencenter/veroeffentlichungen/open-data-leitfaden.pdf.
(Cit.onp.392).
ReportonthealignmentoflibrarymetadatawiththeEuropeanaDataModel.2011.http:
//www.europeana-libraries.eu/documents/868553/1eade085-34ac-487f-82af-
d5cd2545e619.(Cit.onp.399).
TheLinkedDataServiceoftheGermanNationalLibrary:Modellingofbibliographicdata.
Leipzig:DeutscheNationalBibliothek,2012.http://www.dnb.de/SharedDocs/
Downloads/EN/DNB/service/linkedDataModellierungTiteldaten.pdf?__
blob=publicationFile.(Cit.onp.393).
JLIS.it. Vol.4,n.1(Gennaio/January2013).Art.#5507 p.400