Argumentative Zoning: Information Extraction from Scientific Text

352 Pages·2012·2.28 MB·English

Argumentative Zoning:
Information Extraction from Scienti(cid:2)c Text
Simone Teufel
U N IVER
E S
H IT
T Y
O H
G
F
R
E U
DIN B
PhD
University of Edinburgh
1999

3
Acknowledgements
Let me tell you, writing a thesis is not always a barrel of laughs(cid:151)and strange
things can happen, too. For example, at the height of my thesis paranoia, I had a re-
current dream in which my cat Amy gave me detailed advice on how to restructure
the thesis chapters, which was awfully nice of her. But I also had a lot of human help
throughoutthistime,whetherthingswere going(cid:2)ne orbeserk.
Mostofall,IwanttothankMarcMoens:Icouldnothavehadabetterormore
knowledgable supervisor. He always took time for me, however busy he might have
been, reading chapters thoroughly in two days. He both had the calmness of mind to
givemelotsoffreedominresearch,andtherightjudgementtoguidemeaway,tactfully
but determinedly, from the occasional catastrophe or other waiting along the way. He
wasgreatfuntoworkwithandalsobecameagoodfriend.
My work has pro(cid:2)tted from the interdisciplinary, interactive and enlightened
atmosphereattheHumanCommunicationCentreandtheCentreforCognitiveScience
(which is now called something else). The Language Technology Group was a great
place to work in, as my research was grounded in practical applications developed
there.
Jean Carletta helped me design the annotation experiment and interpret the
results.Herkeeneyeandsharpmindhelpedmenipmanyerrorsinthebud.Ihavealso
enjoyedmanyexcitingdiscussionswithChrisBrew,aboutstatistics,scienti(cid:2)cwriting
and many other things. Katja Markert and Michael Strube read and commented on
versionsofchapters.
Myannotators,VasilisKaraiskos,AnneWilsonandAndersBowers,didmetic-
ulouswork; their critical commentson the task, the guidelines,and their observations
about the articles were extremely valuable. Thanks also to the subjects who partici-
patedinthesmallerannotationexperiment.
DavidMcKelvieand Claire Groverfrom theLanguage TechnologyGroupex-
pertly helped me with problems to do with XML encoding and transformation, fsg-
match grammar debugging, cascading style sheets and the like. Andrei Mikheev also
helpedmewithstatisticalandpracticalproblems.Iwasoftenthankfulforhisprograms
whichhaveprovenveryusefulformytask.FrankKellerandDavidSterratsinglehand-
edlysolvedtheoneortwoLATEXproblemsImighthavehad:-).
I was also blessed with the (cid:2)nest crowd of friends in Edinburgh: Frank and
4
Mirella,Katja,Vasilis,Janet,Claire, Marc,Scott,Elina,Merce(cid:30), Karen,Ash,Line,Ze-
lal, Massimo,Roberto, Ira and Jesse. My youngest friends in Edinburgh,Thomas and
Catrin,haveoccupiedaspecialplaceinmyheart.Richard,forreadingthedamnthing
twice, for staying up (remotely, in Berlin!) during the frantic night of last corrections
andprinting,forsortingoutmymisbehavin’linux(cid:151)butevenmoresofordraggingme
away from my desk from time to time to lovely places like Gullane beach et al., and
forbeingthere.
Last, and most importantly, I want to thank the poor people who are con-
tractually bound not to give up on me. Liebe Eltern, ich danke euch fu¤r euren uner-
schu¤tterlichenGlaubenanmich,fu¤reureEngelsgeduld,eureLiebeundUnterstu¤tzung,
undeureToleranzinallenDingen.Ohneeuchha¤tteichdasniegeschafft.
5
Abstract
Wepresentanewtypeofanalysisforscienti(cid:2)ctextwhichwecallArgumenta-
tiveZoning.
We demonstratethatthistype of textanalysiscan be usedfor generating user-
tailored and task-tailored summaries and for performing more informative citation
analyses.
We also demonstrate that our type of analysis can be applied to unrestricted
text,bothautomaticallyandbyhumans.Thecorpusweusefortheanalysis(80confer-
encepapersincomputationallinguistics)isadif(cid:2)culttestbed;itshowsgreatvariation
withrespecttosubdomain,writingstyle,registerandlinguisticexpression.Wepresent
reliabilitystudieswhich we performed on thiscorpus and for which we use two unre-
latedtrainedannotators.
The de(cid:2)nition of our seven categories (argumentative zones) is not speci(cid:2)c to
the domain, only to the text type; it is based on the typical argumentation to be found
inscienti(cid:2)carticles. Itre(cid:3)ects theattributionofintellectualownershipinscienti(cid:2)car-
ticles,expressionsofauthors’stancetowardsotherwork,andtypicalstatementsabout
problem-solvingprocesses.
On the basis of sentential features, we use two statistical models (a Naive
Bayesianmodelandanngrammodeloperatingoversentences)toestimateasentence’s
argumentative status, taking the hand-annotated corpus as training material. An alter-
native,symbolicsystemusesthefeaturesinarule-basedway.
Thegeneralworkinghypothesisofthisthesisisthatempiricaldiscoursestudies
can contribute to practical document management problems: the analysis of a signif-
icant amount of naturally occurring text is essential for discourse linguistic theories,
and the application of a robust discourse and argumentation analysis can make text
understandingtechniquesforpracticaldocumentmanagementmorerobust.

Declaration
I declare that this thesiswas composedby myself and that the work contained therein
ismyown,exceptwhereexplicitlystatedotherwiseinthetext.
(SimoneTeufel)
7

Contents
1 Introduction 13
1.1 InformationForaginginScience . . . . . . . . . . . . . . . . . . . . 13
1.2 Scienti(cid:2)c Articles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.3 EmpiricalNaturalLanguageResearch . . . . . . . . . . . . . . . . . 18
1.4 GoalandOutlineofthisThesis . . . . . . . . . . . . . . . . . . . . . 21
2 Motivation 25
2.1 ManualAbstracting . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.1.1 SummaryTailoring . . . . . . . . . . . . . . . . . . . . . . . 26
2.1.2 CitationInformation . . . . . . . . . . . . . . . . . . . . . . 31
2.2 AutomaticAbstracting . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.2.1 TextExtraction . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.2.2 FactExtraction . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.3 ANewApproach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.3.1 DesignoftheApproach . . . . . . . . . . . . . . . . . . . . 48
2.3.2 RhetoricalDocumentPro(cid:2)les (RDPs) . . . . . . . . . . . . . 57
2.3.3 RDPs forTailoredSummaries . . . . . . . . . . . . . . . . . 64
2.3.4 RDPs forCitationMaps . . . . . . . . . . . . . . . . . . . . 70
2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3 Argumentative Zoning 75
3.1 FixedSectionStructure . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.2 AModelofPrototypicalScienti(cid:2)cArgumentation . . . . . . . . . . . 83
3.2.1 ArgumentativeMoves:Swales(1990) . . . . . . . . . . . . . 83
3.2.2 CitationsandAuthorStance . . . . . . . . . . . . . . . . . . 88
3.2.3 AttributionofIntellectualOwnership . . . . . . . . . . . . . 93
3.2.4 StatementsaboutProblem-SolvingProcesses . . . . . . . . . 97
3.2.5 Scienti(cid:2)c Meta-Discourse . . . . . . . . . . . . . . . . . . . 99
9
10 Contents
3.2.6 StrategiesofScienti(cid:2)cArgumentation . . . . . . . . . . . . . 102
3.3 AnAnnotationSchemeforArgumentativeZones . . . . . . . . . . . 106
3.4 ArgumentativeZonesandRDP Slots . . . . . . . . . . . . . . . . . . 114
3.5 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4 AGoldStandard forArgumentative Zoning 129
4.1 EvaluationStrategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.1.1 EvaluationofFactExtraction . . . . . . . . . . . . . . . . . 130
4.1.2 EvaluationofTextExtraction . . . . . . . . . . . . . . . . . 131
4.1.3 OurEvaluationStrategyforArgumentativeZoning . . . . . . 138
4.2 EvaluationMeasures . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.3 ReliabilityStudies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.3.1 ExperimentalDesign . . . . . . . . . . . . . . . . . . . . . . 145
4.3.2 StudyI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
4.3.3 StudyII . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.3.4 StudyIII . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
4.3.5 Signi(cid:2)canceofReliabilityResults . . . . . . . . . . . . . . . 161
4.4 Post-Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
4.4.1 ArgumentativeStructureofAuthorAbstracts . . . . . . . . . 162
4.4.2 ReductionofAnnotationAreas . . . . . . . . . . . . . . . . 164
4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
5 AutomaticArgumentative Zoning 171
5.1 OverviewofAutomaticArgumentativeZoning . . . . . . . . . . . . 171
5.2 CorrelatesofArgumentativeStatus . . . . . . . . . . . . . . . . . . . 174
5.2.1 TraditionalFeatures . . . . . . . . . . . . . . . . . . . . . . 178
5.2.2 Meta-DiscourseFeatures . . . . . . . . . . . . . . . . . . . . 186
5.3 APrototypeSystem . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
5.3.1 CorpusEncoding . . . . . . . . . . . . . . . . . . . . . . . . 196
5.3.2 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . 199
5.3.3 Feature Determination . . . . . . . . . . . . . . . . . . . . . 200
5.3.4 StatisticalClassi(cid:2)ers . . . . . . . . . . . . . . . . . . . . . . 214
5.3.5 SymbolicRules . . . . . . . . . . . . . . . . . . . . . . . . . 219
5.4 IntrinsicEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
5.4.1 NaiveBayesModel . . . . . . . . . . . . . . . . . . . . . . . 220