Discovering Structural Similarities in Narrative Texts using Event Alignment Algorithms

132 Pages·2014·0.81 MB·English

by  Nils Reiter

Discovering Structural Similarities in
Narrative Texts using Event Alignment
Algorithms
Dissertation zur Erlangung der Doktorwürde
im Fach Computerlinguistik
der Neuphilologischen Fakultät
der Ruprecht-Karls-Universität Heidelberg
vorgelegtvon
Nils Reiter
Publication June2014
Commissionchair Prof. Dr. EkkehardFelder
GermanistischesSeminar
UniversitätHeidelberg
Supervisorandfirstreviewer Prof. Dr. AnetteFrank
InstitutfürComputerlinguistik
UniversitätHeidelberg
Secondaryreviewer Prof. Dr. SebastianPadó
InstitutfürMaschinelleSprachverarbeitung
UniversitätStuttgart
ii
Acknowledgements
I would like to thank my supervisor, Anette Frank, for supervising and guiding this
thesis. In particular, I am thankful for the opportunity to delve into details, for the
continuous support, the engaging discussions and the critical yet fruitful feedback. It
wasapleasuretoworkinandwithherentiregroup. Inparticular,Iwouldliketothank
myyear-longofficemateEvaSourjikováforthepleasantofficeatmosphere,myfriend
andcolleagueMichaelRothforallthefunwehadandMatthiasHartungfor...Idon’t
knowwheretostart.
Furthermore, I thank “my” ritual experts, Oliver Hellwig and Christof Zotter, for an-
sweringquestionsanddoingannotationwork. IalsothankThomasBögel,IrinaGoss-
mann, Mareike Hartmann, Borayin Larios, Julio Cezar Rodrigues and Britta Zeller for
working in this ritual research project and doing implementation and annotation. All
this would not have been possible without the Sonderforschungsbereich 619 and the
fundingbytheGermanResearchFoundation,forwhichIamequallythankful.
FollowingPropp,folktaleshavemagicalhelperagents,andsodoesthetaleofworking
onthisthesis. Iamverythankfultomyhelperagentsandfortheencouragementthey
gaveme.
NilsReiter
iii
Abstract
This thesis is about the discovery of structural similarities across narrative texts. We
willdescribeamethodthatisbasedoneventalignmentscreatedautomaticallyonau-
tomatically preprocessed texts. This opens up a path to large-scale empirical research
onstructuralsimilaritiesacrosstexts.
Structuralsimilaritiesareofinterestformanyareasinthehumanitiesandsocialsci-
ences. We will focus on folkloristics and research of rituals as application scenarios.
Folkloristicsresearchesfolktales,i.e.,talesthathavebeenpasseddownorallyforalong
time. Similaritiesacrossdifferentfolktaleshavebeenobserved,bothatthelevelofindi-
vidualevents(beingabandonedinthewoods)orparticipants(thegingerbreadhouse)
andstructurally: Eventsdonothappenatrandom,butinacertainorder. Ritualsarean
omnipresentpartofhumanbehaviorandarestudiedinethnology,socialsciencesand
history. Similaritiesacrosstypesofritualshavebeenobservedandsparkedadiscussion
aboutstructuralprinciplesthatgovernthecombinationofindividualritualelementsto
rituals.
Asdescriptionsofritualsfeaturealotofuncommonlanguageconstructions,wewill
alsodiscussmethodsofdomainadaptationinordertoadaptexistingNLPcomponents
to the domain of rituals. We will mainly use supervised methods and employ retrain-
ingasameansforadaptation. Thispresupposesannotatingsmallamountsofdomain
data. We will be discussing the following linguistic levels: Part of speech, chunking,
dependency parsing, word sense disambiguation, semantic role labeling and corefer-
ence resolution. On all levels, we have achieved improvements. We will also describe
howtheseannotationlevelsarebroughttogetherinasingle, integrateddiscourserep-
resentationthatisthebasisforfurtherexperiments.
Inordertodiscoverstructuralsimilarities,weemploythreedifferentalignmentalgo-
rithmsandusethemtoalignsemanticallysimilarevents. Sequencealignment(Needle-
man-Wunsch) is a classic algorithm with limited capabilities. A graph-based event
alignment system that has been developed for newspaper texts will be used in com-
parison. As a third algorithm, we employ Bayesian model merging, which induces a
hiddenMarkovmodel,fromwhichweextractanalignment. Wewillevaluatethealgo-
rithmsintwoexperiments. Inthefirstexperiment,weevaluateagainstagoldstandard
of aligned descriptions of rituals. Bayesian model merging and predicate alignment
achievethebestresults,measuredusingtheBlancmetric. Duetodifficultiesincreating
aneventalignmentgoldstandard,thesecondexperimentisbasedonclusterinduction.
Although this is not a strict evaluation of structural similarities, it gives some insight
intothebehaviorofthealgorithms.
We induce a document similarity measure from the generated alignments and use
this measure to cluster the documents. The clustering is then compared against a
iv
goldstandardclassificationofdocumentsfrombothscenarios. Inthisexperiment, the
lemma alignment baseline achieves the best numerical performance on folktales (but
as it aligns lemmas instead of event representations, its expressiveness is limited), fol-
lowed by predicate alignment, Needleman-Wunsch and Bayesian model merging. On
descriptionsofrituals,thepredicatealignmentalgorithmoutperformsallbaselinesand
theotheralgorithms. Shallowmeasuresofsemanticsimilaritiesoftextsoutperformthe
alignment-basedalgorithmsonfolktales,buttheydonotallowtheexactlocalizationof
similarities.
Finally,wepresentagraph-basedalgorithmthatrankseventsaccordingtotheirpar-
ticipation in structurally similar regions across documents. This allows us to direct
researchers from humanities to interesting cases, which are worth manual inspection.
Because in digital humanities scenarios, the accessibility of results to researchers from
humanities is of utmost importance, we close the thesis with a showcase scenario in
which we analyze descriptions of rituals using the alignment, clustering and event
ranking algorithms we have described before. We will show in this showcase how
resultscanbevisualizedandinterpretedbyresearchersofrituals.
v
Contents
1 Introduction 1
2 Digital Humanities 4
2.1 ExistingComputationalLinguisticsResearchwithinDigitalHumanities 6
2.2 ChallengesforComputationalLinguistics . . . . . . . . . . . . . . . . . . . 7
2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3 Related Work 10
3.1 DomainAdaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 ComputationalNarrativeAnalysis . . . . . . . . . . . . . . . . . . . . . . . . 16
4 Application Scenarios 33
4.1 Folktales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.2 Rituals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5 Automatic Semantic Annotation and Domain Adaptation 47
5.1 SystemArchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.2 AdaptationtotheRitualDomain . . . . . . . . . . . . . . . . . . . . . . . . . 50
5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
6 Discovering Structural Similarities 63
6.1 DiscoveringStorySimilaritiesthroughEventAlignments . . . . . . . . . . 63
6.2 EventAlignmentAlgorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.3 GoldStandardandEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
6.4 Experiment1: ComparisonagainstanAlignmentGoldStandard . . . . . 78
6.5 Experiment2: Alignment-basedClusteringEvaluation . . . . . . . . . . . 83
6.6 Graph-basedDetectionofStructuralSimilarities . . . . . . . . . . . . . . . 88
6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
7 Analyzing and Exploiting Structural Similarities in Digital Humanities 91
7.1 InspectingStorySimilaritiesGlobally . . . . . . . . . . . . . . . . . . . . . . 91
7.2 UncoveringStructuralSimilarities . . . . . . . . . . . . . . . . . . . . . . . . 92
7.3 Fine-grainedAnalysisofStructuralSimilarities . . . . . . . . . . . . . . . . 96
8 Conclusions 98
8.1 ChallengesforComputationalLinguistics . . . . . . . . . . . . . . . . . . . 98
vi
Contents
8.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
8.3 OutlookandFutureWork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Appendix 103
1 Folktale: Bearskin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
2 ProppianEventFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
3 DescriptionofaCu¯d.a¯karan.aRitual . . . . . . . . . . . . . . . . . . . . . . . 104
4 MathematicalNotationOverview. . . . . . . . . . . . . . . . . . . . . . . . . 107
5 DiscourseRepresentationFileFormat . . . . . . . . . . . . . . . . . . . . . . 107
vii
List of Tables
2.1 Pastdigitalhumanitiesresearchincomputationallinguistics . . . . . . . . 5
3.1 Approachesforstatisticaldomainadaptation . . . . . . . . . . . . . . . . . 13
3.2 Collectionswithannotatedstoryintentiongraphs . . . . . . . . . . . . . . 22
3.3 Storymodelingapproaches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.4 Storyaggregationapproaches . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.1 Overviewofsomekeycharacteristicsofthefolktalecorpus . . . . . . . . 37
4.2 Coverageofresourcesonthefolktalecorpus . . . . . . . . . . . . . . . . . . 38
4.3 Keycharacteristicsofthecorpusofdescriptionsofritualscorpus . . . . . 43
5.1 Preprocessingcomponents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.2 PackageversionsandURLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.3 Ritualdomainphenomenaandhowtheyareaddressed . . . . . . . . . . . 50
5.4 Datasetsforpartofspeechtaggingandchunking . . . . . . . . . . . . . . 51
5.5 Trainingsetsforpartofspeechtaggingandchunking . . . . . . . . . . . . 52
5.6 Resultsforadaptationofpartofspeechtagging . . . . . . . . . . . . . . . . 53
5.7 Resultsforadaptationofchunking . . . . . . . . . . . . . . . . . . . . . . . . 53
5.8 Resultsforadaptationofdependencyparsing . . . . . . . . . . . . . . . . . 55
5.9 Resultsforadaptationofwordsensedisambiguation . . . . . . . . . . . . 56
5.10 Resultsforadaptationofframelabeling . . . . . . . . . . . . . . . . . . . . . 58
5.11 Resultsforadaptationofframeelementlabeling . . . . . . . . . . . . . . . 59
5.12 Resultsforcoreferenceresolutionwithadaptedpreprocessing . . . . . . . 60
5.13 Resultsforadaptationofcoreferenceresolution . . . . . . . . . . . . . . . . 61
5.14 Overviewofadaptationimprovements . . . . . . . . . . . . . . . . . . . . . 62
6.1 Overviewofexperiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
6.2 Algorithmoverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6.3 Alignmentgoldstandard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
6.4 Overviewofclusteringdatasets . . . . . . . . . . . . . . . . . . . . . . . . . 75
6.5 ResultsforExperiment1: Comparisonwithagoldstandard . . . . . . . . 80
6.6 QuantifiederroranalysisofprecisionerrorsinExperiment1 . . . . . . . . 83
6.7 ResultsforExperiment2: Clusterinductionondescriptionsofrituals . . 85
6.8 ResultsforExperiment2: Clusterinductiononfolktales . . . . . . . . . . . 87
7.1 Mostconnectedeventsacrossdescriptionsofrituals . . . . . . . . . . . . . 93
viii
List of Figures
3.1 Primitiveandcomplexunits(Lehnert,1981) . . . . . . . . . . . . . . . . . . 19
3.2 Gametreefortheengagementstory(Mani,2012) . . . . . . . . . . . . . . . 20
3.3 Narrativechainsandschema(Chambers,2011) . . . . . . . . . . . . . . . . 23
3.4 Anexampleglobfordynamicanalogydetection(Elson,2012b) . . . . . . 26
4.1 Top-levelcategoriesintheAarne-Thompson-Utherindex . . . . . . . . . . 34
4.2 (Sub-)structureofafireritualaccordingtoStaal(1989) . . . . . . . . . . . 40
5.1 Classdiagramfordiscourserepresentation . . . . . . . . . . . . . . . . . . . 49
6.1 Exampleforstructurallysimilarsequences . . . . . . . . . . . . . . . . . . . 64
6.2 AlignmentproducedbyNeedleman-Wunsch . . . . . . . . . . . . . . . . . 67
6.3 Runningpredicatealignmentonthesequences⟨a,b,a⟩and⟨b,a⟩ . . . . . 68
6.4 Alignmentproducedbypredicatealignment . . . . . . . . . . . . . . . . . . 69
6.5 RunningBayesianmodelmergingonthesequences⟨a,b,a⟩and⟨b,a⟩ . . 71
6.6 AlignmentproducedbyBayesianmodelmerging. . . . . . . . . . . . . . . 71
6.7 Undirectedgraphg createdfromanalignment . . . . . . . . . . . . . . . . 89
6.8 VisualisationofconnectivityscoresforalignmentinFigure6.7 . . . . . . 90
7.1 Heatmapwithdocumentsimilarities . . . . . . . . . . . . . . . . . . . . . . 92
7.2 Screenshotofthealignmentvisualization . . . . . . . . . . . . . . . . . . . . 93
ix
1 Introduction
Narrative texts are important textual sources in the humanities and social sciences. In
particular, many research questions revolve around similarities, parallels or overlaps
acrossnarrativetexts,e.g.,discoveryofintersectionsinbiographiesorsimilardevelop-
mentsofcharactersinfictionaltales. Withintheemergingparadigmofdigitalhumani-
ties,wedevelopmethodsandasystemfortheautomaticdiscoveryofstorysimilarities
innarrativetexts. Thesystemusesadvancedcomputationallinguisticstechniquesand
isdesignedforthespecificneedsandpremissesofdigitalhumanities.
Westudy twoapplication scenariosin detail: Researchof folktalesand ofrituals. In
both areas, the detection of similarities across stories plays a major role. For folktales,
theclassificationoftalesfromdifferenterasandculturesbasedoncommonelementsin
theirstorylineshasbeenstudiedsincetheearlytwentiethcentury. Thediscoveryofre-
occurringelementsinritualshassparkedadiscussionabouttheexistenceofstructural
principles that rule the combination of these elements. For these principles, the term
“ritualgrammar”hasbeencoined1.
Bothapplicationscenariossuggestquantitative-empiricalresearchapproachesasthey
aim at analyzing more than a single textual source. In addition, the expressiveness of
the analysis increases the more sources have been studied. However, traditional re-
search approaches prevalent in the humanities do not scale easily, as they rely on hu-
man scholars (close-)reading texts. A system that automatically detects similarities in
narratives isa keycomponent for enabling large-scale empiricalresearch inthis direc-
tion2.
The system we describe detects story similarities using event alignment algorithms.
Thealignmentalgorithmsworkondenselyconnecteddiscourserepresentations,which
contain representations for events, characters and the linguistically analyzed textual
source data. Using measures of semantic similarity for events, the alignment methods
detect story similarities across discourses that can be visualized and thus made acces-
sibletohumanitiesresearchers.
Thediscourserepresentationsthemselvesarecreatedfullyautomaticallyusingstate
of the art techniques on a variety of different linguistic analysis levels. Given the pe-
culiar text characteristics of the descriptions of rituals, the linguistic processing tools
are adapted to the domain of rituals. Not all processing tools for the different linguis-
tic annotation levels employ the same basic methodology. Consequently, the adapta-
tion of these tools must employ different adaptation strategies. Supervised linguistic
1Thisthesiswaswrittenwithinthecontextoftheresearchproject“Ontologymodelingforritualstructure
research”, in the collaborative research center “Ritual Dynamics”, funded by the German Research
Foundation(Sonderforschungsbereich619,Ritualdynamik).
2InthewordsofMoretti(2000),wearedescribingadistant-readingapproach.
1