Paraphrase Pattern Acquisition by Diversifiable Bootstrapping

109 Pages·2014·1.11 MB·English

2014/08/17 3:39am
DRAFT
Paraphrase Pattern Acquisition by
Diversifiable Bootstrapping (Thesis Draft)
Hideki Shima
Last updated: Aug 2014
LanguageTechnologiesInstitute
SchoolofComputerScience
CarnegieMellonUniversity
Pittsburgh,PA15213,USA
ThesisCommittee:
TerukoMitamura(chair),CarnegieMellonUniversity
EricNyberg,CarnegieMellonUniversity
EduardHovy,CarnegieMellonUniversity
PatrickPantel,MicrosoftResearch
Submittedinpartialfulfillmentoftherequirements
forthedegreeofDoctorofPhilosophy
inLanguageandInformationTechnologies.
2014/08/17 3:39am
DRAFT
Abstract
Texts that convey the same or close meaning can be written in many different
ways. Because of this, computer programs are not good at recognizing meaning
similarity between short texts. Toward solving this problem, researchers have been
investigatingmethodsforautomaticallyacquiringparaphrasetemplates(paraphrase
extraction)fromacorpus.
State-of-the-artapproachesinparaphraseextractionhavelimitedabilitytodetect
variation (e.g. “X died ofY”, “X has died ofY”, “X was dying ofY”, “X died from
Y”, “X was killed in Y”). Considering practical usage, for instance in Information
Extraction, a paraphrase resource should ideally have higher coverage so that it can
recognizemorewaystoconveythesamemeaningintext(e.g. “X succumbedtoY”,
“X fell victim toY”, “X suffered a fatalY”, “X was terminally ill withY”, “X lost
hislongbattlewithY”,“X(writer)wrotehisfinalchapterY”),withoutaddingnoisy
patternsorinstancesthatconveyadifferentmeaningthantheoriginalseedmeaning
(semanticdrift).
The goal of this thesis work is to develop a paraphrase extraction algorithm that
canacquirelexically-diversebinary-relationparaphrasetemplates,givenarelatively
small number of seed instances for a certain relation and an unstructured monolin-
gual corpus. The proposed algorithm runs in an iterative fashion, where the seed
instances are used to extract paraphrase patterns, and then these patterns are used to
extractmoreseedinstancestobeusedinthenextiteration,andsoon.
The proposed work is unique in a sense that lexical diversity of resulting para-
phrasepatternscanbecontrolledwithaparameter,andthatsemanticdriftisdeferred
by identifying erroneous instances using a distributional type model. We also pro-
poseanewmetricDIMPLEwhichcanmeasurequalityofparaphrases,takinglexical
diversityintoconsideration.
Ourhypothesisisthatsuchamodelthatexplicitlycontrolsdiversityandincludes
a distributional type constraint will outperform the state-of-the-art as measured by
precision, recall, and DIMPLE. We also present experimental results to support this
hypothesis.
2014/08/17 3:39am
DRAFT
Acknowledgments
ThisresearchwassupportedinpartbyDARPAgrantFA8750-12-2-0342funded
undertheDEFTprogram.
ThispublicationwasmadepossiblebygrantNPRP-09-873-1-129fromtheQatar
National Research Fund (a member of the Qatar Foundation). The statements made
hereinaresolelytheresponsibilityoftheauthors.
2014/08/17 3:39am
DRAFT
Contents
1 Introduction 1
1.1 FundamentalProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.1 DiversifiableBootstrappingforParaphraseAcquisition . . . . . . . . . . 9
1.2.2 Measuringopen-domaintypesimilaritybydistributionaltypemodel . . . 10
1.2.3 Comparisonwiththestate-of-the-art . . . . . . . . . . . . . . . . . . . . 11
1.3 TheGoalandContributionofthisDissertation . . . . . . . . . . . . . . . . . . . 11
1.4 ThesisOutline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 RelatedWorksofParaphraseExtraction 14
2.1 ParaphraseExtractionbyDistributionalApproaches . . . . . . . . . . . . . . . . 14
2.1.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 ParaphraseExtractionbyAlignment . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2.1 FromMonolingualParallel/ComparableCorpora . . . . . . . . . . . . . 19
2.2.2 FromBilingualParallelCorpora . . . . . . . . . . . . . . . . . . . . . . 19
2.2.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3 ReviewofBootstrapLearning . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.1 BootstrapApproaches . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.3.2 SemanticDriftPrevention . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.3.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4 ParaphraseResourcesbyHumanLexicographerortheCrowd . . . . . . . . . . 23
2.4.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.5 ParaphraseEvaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.6 SummaryofProblemsinPreviousWorks . . . . . . . . . . . . . . . . . . . . . 27
3 BootstrapParaphraseAcquisitionFramework 28
3.1 ParaphraseAcquisitionbyBootstrapping . . . . . . . . . . . . . . . . . . . . . 28
3.1.1 BaseFramework: Espresso . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.1.2 TheoryofDistributionalSimilarity . . . . . . . . . . . . . . . . . . . . 31
3.2 ExtendingEspresso . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3 Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.3.1 PatternRepresentation . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.3.2 HumanEffort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
iv
2014/08/17 3:39am
DRAFT
4 DistributionalTypeConstraint 38
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2 ProposedMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2.1 CreatingaVectorofTypes . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2.2 VectorSpaceModelofTypes . . . . . . . . . . . . . . . . . . . . . . . 42
4.2.3 TypeKnowledgeBase: YAGO2 . . . . . . . . . . . . . . . . . . . . . . 42
4.3 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3.1 BuildingEvaluationDataset . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3.2 EvaluationResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5 DiversifiableBootstrappingModel 48
5.1 DiversifiableBootstrapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.1.1 Diversityfunction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.1.2 EspressoDiversification . . . . . . . . . . . . . . . . . . . . . . . . . . 52
5.1.3 DiversificationwithDifferentλ Values . . . . . . . . . . . . . . . . . . 53
5.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2.1 ComparisonwithMMR . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2.2 DiversificationandSemanticDrift . . . . . . . . . . . . . . . . . . . . . 54
5.2.3 LimitationsandWeakness . . . . . . . . . . . . . . . . . . . . . . . . . 54
5.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
6 Diversity-awareEvaluationMetricforParaphrasePatterns 55
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
6.2 DIMPLEMetric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.2.1 CumulativeGain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.2.2 DIMPLEAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
6.3 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
6.3.1 AnnotatedParaphraseResource . . . . . . . . . . . . . . . . . . . . . . 58
6.3.2 IntrinsicParaphraseMetrics . . . . . . . . . . . . . . . . . . . . . . . . 58
6.3.3 ExtrinsicEvaluationDatasets . . . . . . . . . . . . . . . . . . . . . . . 59
6.3.4 ExtrinsicPerformanceMetric . . . . . . . . . . . . . . . . . . . . . . . 60
6.3.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
7 Evaluation 62
7.1 EvaluationMetrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
7.2 ExperimentSettings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.2.1 RelationsandOriginalSeed . . . . . . . . . . . . . . . . . . . . . . . . 63
7.2.2 SourceCorpus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
7.2.3 ParaphraseExtractorConfigurations . . . . . . . . . . . . . . . . . . . . 64
7.2.4 GoldStandardAnnotation . . . . . . . . . . . . . . . . . . . . . . . . . 65
7.3 Result: EffectofDiversification . . . . . . . . . . . . . . . . . . . . . . . . . . 65
v
2014/08/17 3:39am
DRAFT
7.4 Result: EffectofType-basedInstanceFiltering . . . . . . . . . . . . . . . . . . 70
8 Conclusions 72
8.1 FutureWorks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
A ExampleParaphrases 75
A.1 Relation: LEADER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
A.2 Relation: person graduated school . . . . . . . . . . . . . . . . . . . . 78
A.3 Relation: person death date . . . . . . . . . . . . . . . . . . . . . . . . . 81
B ParaphraseAnnotationGuideline 83
B.1 TaskDescription . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
B.2 LabelDefinition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
B.3 ExamplesbyLabel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
B.3.1 LabelM(Matched) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
B.3.2 LabelO(Matchedandout-of-dictionary) . . . . . . . . . . . . . . . . . 84
B.3.3 LabelR(Related) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
B.3.4 LabelI(Inconclusive) . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
B.3.5 LabelA(Antonym) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
B.3.6 LabelW(Wrong) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
B.4 Examplesofborderlinecases . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
B.4.1 Argumentorderdifference . . . . . . . . . . . . . . . . . . . . . . . . . 87
B.4.2 SlightDifferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
B.4.3 Labelscomparedbycertaintylevel . . . . . . . . . . . . . . . . . . . . 88
B.4.4 PresentTense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Bibliography 90
vi
2014/08/17 3:39am
DRAFT
List of Figures
1.1 Granularityofmeaningequivalence. . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 ParaphrasingtemplateusedtohelpidentifyanswercandidateinQA. . . . . . . . 4
1.3 Diverse set of paraphrasing templates help accumulating evidence for answer
candidatesinQA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Contrasting differences in paraphrases (binary-argument templates for died-of
relation)withsyntacticandlexicaldiversity . . . . . . . . . . . . . . . . . . . . 7
1.5 AninvalidinstanceisdetectedusingaNERsystem. . . . . . . . . . . . . . . . . 8
1.6 Thebootstrapparaphraseacquisitionframework. . . . . . . . . . . . . . . . . . 9
1.7 AninvalidinstanceisdetectedusingaNERsystem. . . . . . . . . . . . . . . . . 10
2.1 Patterns extracted by a vanilla Espresso given Ephyra seed data Schlaefer et al.
(2006)andaWikipediacorpus. . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2 Thenumberofmostfrequenttemplatesnecessarytoreachdifferentrecalllevels.
WeplottedthischartfromthedataatTable5in(Romanoetal.,2006). . . . . . . 24
3.1 Overview of the Espresso algorithm. Many bootstrapping learning algorithms
workmoreorlessinthesamewayasdescribedhere. . . . . . . . . . . . . . . . 29
3.2 Patternextraction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1 Detecting an invalid instance using type information: traditional and proposed
models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2 Hypernym tree from WordNet where higher-level types are abstract and shared
among multiple entities, while lower-level ones are more concrete and distinc-
tive. Notice that “heart attack” and “cancer” are located far away in the taxon-
omy, even though they are similar in terms of possibility to become a cause of
death. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7.1 Paraphrases extracted by diversifiable bootstrap paraphrase learner (D-BPL) is
compared against the baselines, using different metric scores (y-axis; macro-
averagedoverrelations)periteration(x-axis). . . . . . . . . . . . . . . . . . . . 67
7.2 Effect of diversification (D-BPL) is compared to BPL, using different metric
scores(y-axis;macro-averagedoverrelations)periteration(x-axis). . . . . . . . 71
vii
2014/08/17 3:39am
DRAFT
List of Tables
1.1 Sub-areasofparaphrasingresearch . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Diverseparaphrasesfordie. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Uniqueness of Proposed Work in contrast with other data-driven paraphrase ac-
quisitionmethods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.4 Summaryofcontributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1 Comparison of paraphrases acquired for “killed”, from the dataset by Metzler
etal.(2011). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 Syntacticvariabilityexamplesforaprotein-proteininteractiontemplate“X acti-
vateY”,fromTable1in(Romanoetal.,2006) . . . . . . . . . . . . . . . . . . . 25
2.3 Synonymsof“lead(v)”inWordNet. . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1 Corpuspreprocessing: symbolescape . . . . . . . . . . . . . . . . . . . . . . . 33
3.2 Exampleoftoospecificpatterns . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3 Exampleoftoogeneralpatterns . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.1 Seed entity type vector where elements are sorted by two different weighting
algorithms: (a)typefrequency(b)pseudo-TFIDF.Onlytop15elementsareshown. 42
4.2 Exhaustivesetoftypesassociatedwith“heartattack”inYAGO2. . . . . . . . . . 43
4.3 Exhaustive list of seed entities used in the experiment. Each pair of X andY are
incause-of-deathrelationwherethefollowingholds: X diedfromacauseY. . . 44
4.4 ExperimentalresultsmeasuredinPrecision@N.*Statisticalsignificancehasbeen
observedfortheProposedmethodagainsttheBaselinemethodwithaconfidence
thresholdp<0.01. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.5 Top-10 ranked entities by the Baseline and the Proposed B algorithm. Note that
similarityscoresarenotdirectlycomparablebetween(a)and(b). . . . . . . . . . 46
5.1 Theexclusivelistofseedinstancesforeachrelation. . . . . . . . . . . . . . . . 50
5.2 Top 15 (out of hundreds or thousands) ranked list of paraphrases acquired by
Diversifiable Bootstrapping are shown, after the 5th iteration. When a smaller
λ was specified, the method preferred a pattern that gave more lexical diversity.
When the lexical diversification was disabled (λ = 1.0), the patterns tended to
havesyntacticandmorphologicaldiversity. . . . . . . . . . . . . . . . . . . . . 51
6.1 10mostfrequentlyoccurringsourceverbsinthreedatasets. Numbersarepositive-
labeledpairswheretheverbappearsinatleastonesideofapair. . . . . . . . . . 59
viii
2014/08/17 3:39am
DRAFT
6.2 Correlationbetweenintrinsicparaphrasemetricsandextrinsicparaphraserecog-
nition task metrics where DIMPLE’s Q score is based on lenient judgment.
Bold figures indicate statistical significance of the correlation statistics (null-
hypothesistested: “thereisnocorrelation”,p-value<0.01). . . . . . . . . . . . . 60
6.3 SameastheTable6.2,exceptthattheQscoreisbasedonstrict judgment. . . . . 60
7.1 Listofseedrelationswithargumenttypesandseedsource . . . . . . . . . . . . 64
7.2 CandidateSourcecorporaforexperiments. . . . . . . . . . . . . . . . . . . . . . 64
7.3 Goldstandardlabeldescription. . . . . . . . . . . . . . . . . . . . . . . . . . . 66
7.4 Distributionofgold-standardlabelsannotatedontheextractedparaphrases. . . . 66
7.5 D-BPLresult: Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
7.6 D-BPLresult: Recall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
7.7 D-BPLresult: Dimple . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
7.8 D-BPLresult: numberofdistinctkeyword . . . . . . . . . . . . . . . . . . . . . 69
8.1 Summaryofcontributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
A.1 OriginalseedandtypesextractedforLEADER. . . . . . . . . . . . . . . . . . . 75
A.2 Rankedlistofextractedparaphrasesbybootstrapping. . . . . . . . . . . . . . . 76
A.3 Pooledparaphrasesannotatedwith“O”label. . . . . . . . . . . . . . . . . . . . 77
A.4 Originalseedandtypesextractedforperson graduated school. . . . . . . . . . . 78
A.5 PatternsbyCPL(first100) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
A.6 Rankedlistofextractedparaphrasesbybootstrapping. . . . . . . . . . . . . . . 80
A.7 Originalseedandtypesextractedforperson death date. . . . . . . . . . . . . . . 81
A.8 PatternsbyCPL(exhaustive) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
A.9 Rankedlistofextractedparaphrasesbybootstrapping. . . . . . . . . . . . . . . 82
ix
2014/08/17 3:39am
DRAFT
Chapter 1
Introduction
WhatisParaphrase?
Paraphrasing is one of disciplines in Natural Language Processing which concerns about para-
phrases,ornaturallanguageexpressionswithmeaningequivalenceorsimilarity.
Table1.1: Sub-areasofparaphrasingresearch
Classification of Paraphrase Research Usage / Application
(1) Paraphrase Recognition
• Question Answering
(cid:5)(cid:13)(cid:2)(cid:20)(cid:20)(cid:9)(cid:4)(cid:23)(cid:17)(cid:24)(cid:1)(cid:3)(cid:24)(cid:18)(cid:4)(cid:1) (cid:27)(cid:28)(cid:9)(cid:4)(cid:29)(cid:30) (word/phrase-level) • Text Summarization
(cid:5)(cid:1)(cid:1)(cid:9)(cid:4)(cid:1)(cid:2)(cid:18)(cid:4)(cid:4)(cid:1) (cid:27)(cid:28)(cid:9)(cid:4)(cid:29)(cid:30) (sentence-level) • Automatic Grading
(cid:5)(cid:2)(cid:1)(cid:9)(cid:4)(cid:2)(cid:2)(cid:18)(cid:4)(cid:1) (cid:27)(cid:28)(cid:9)(cid:4)(cid:29)(cid:30) (document-level) • Plagiarism Detection
(2) Paraphrase Generation • Query Expansion
• (cid:1)(cid:2)(cid:3)(cid:4)(cid:1)(cid:5)(cid:1)(cid:3)(cid:6)(cid:3)(cid:7)(cid:8)(cid:3)(cid:9)(cid:4)(cid:10)(cid:7)(cid:8)(cid:8)(cid:4)(cid:7)(cid:11)(cid:7)(cid:12)(cid:9)(cid:4)(cid:13)(cid:2)(cid:6)(cid:13)(cid:4)(cid:14)(cid:15)(cid:3)(cid:4)(cid:16)(cid:17)(cid:6)(cid:13)(cid:3)(cid:14)(cid:18) • Reference Expansion in
• (cid:19)(cid:3)(cid:4)(cid:15)(cid:7)(cid:1)(cid:4)(cid:7)(cid:4)(cid:20)(cid:21)(cid:14)(cid:4)(cid:21)(cid:22)(cid:7)(cid:1)(cid:23)(cid:2)(cid:24)(cid:7)(cid:14)(cid:2)(cid:21)(cid:25)(cid:4)(cid:22)(cid:21)(cid:24)(cid:4)(cid:15)(cid:2)(cid:8)(cid:4)(cid:26)(cid:21)(cid:16) Automatic Evaluation
(cid:1)(cid:19)(cid:3)(cid:4)(cid:15)(cid:7)(cid:1)(cid:4)(cid:10)(cid:20)(cid:3)(cid:25)(cid:14)(cid:12)(cid:4)(cid:21)(cid:22)(cid:7)(cid:1)(cid:23)(cid:2)(cid:24)(cid:7)(cid:14)(cid:2)(cid:21)(cid:25)(cid:4)(cid:22)(cid:21)(cid:24)(cid:4)(cid:15)(cid:2)(cid:8)(cid:4)(cid:26)(cid:21)(cid:16)
(3) Paraphrase Extraction • Resource for (1) and (2)
(cid:1) {word, phrase, sentence} (cid:2) Paraphrase dictionary
-levelparaphrases (cid:2) Sentence-aligned paraphrase
• with/withoutvariables corpus
• with/withoutstructure
(cid:1)(cid:4)(cid:5)(cid:11)(cid:7)(cid:8)(cid:5)(cid:10)(cid:3)(cid:3)(cid:16)(cid:17)(cid:7)(cid:13)(cid:6)(cid:5)(cid:15)
(cid:1)(cid:2)(cid:3)(cid:4) (cid:5)(cid:6)(cid:7)(cid:8) (cid:1)(cid:2)(cid:3)(cid:4) (cid:9)(cid:7)
(cid:1)(cid:16)(cid:3)(cid:18)(cid:6)(cid:7)(cid:3)(cid:6)(cid:14)(cid:3)(cid:2)(cid:10)(cid:3)(cid:19)(cid:18)(cid:8)(cid:20)(cid:7)(cid:21)(cid:3)(cid:6)(cid:14)(cid:3)(cid:2)(cid:15)
(cid:1)(cid:2)(cid:3)(cid:22)(cid:17)(cid:21)(cid:3)(cid:9)(cid:10)(cid:3)(cid:3)(cid:3)(cid:9)(cid:3)(cid:3)(cid:12)(cid:8)(cid:18)(cid:18)(cid:3)(cid:3)(cid:2)(cid:15)
(cid:1)(cid:2)(cid:3)(cid:4)(cid:5)(cid:6)(cid:7)(cid:8)(cid:3)(cid:9)(cid:10)(cid:3)
(cid:2)(cid:3)(cid:11)(cid:12)(cid:3)(cid:7)(cid:13)(cid:8)(cid:3)(cid:4)(cid:5)(cid:11)(cid:7)(cid:8)(cid:5)(cid:3)(cid:6)(cid:14)(cid:3)(cid:9)(cid:15) (cid:1)(cid:1)(cid:1)(cid:10)(cid:3)(cid:3)(cid:1)(cid:2)(cid:15)
Paraphrasingresearchcanberoughlyclassifiedintothreesubareas,namelyparaphraserecog-
nition, generation, and extraction (see Table 1.1). The table illustrates the variations of a para-
phrase unit: words, phrases, templates (text fragment with variables), syntactically structured
templates, sentences, and even longer texts. The three subareas of paraphrasing are as follows.
Paraphrase recognition is a binary classification problem deciding whether given two texts are
1