Investigations in Entity Relationship Extraction

156 Pages·2022·2.729 MB·English

by  Sachin Sharad Pawar, Pushpak Bhattacharyya, Girish Keshav Palshikar

Studies in Computational Intelligence 1058
Sachin Sharad Pawar
Pushpak Bhattacharyya
Girish Keshav Palshikar
Investigations
in Entity
Relationship
Extraction
Studies in Computational Intelligence
Volume 1058
SeriesEditor
JanuszKacprzyk,PolishAcademyofSciences,Warsaw,Poland
The series “Studies in Computational Intelligence” (SCI) publishes new develop-
mentsandadvancesinthevariousareasofcomputationalintelligence—quicklyand
withahighquality.Theintentistocoverthetheory,applications,anddesignmethods
of computational intelligence, as embedded in the fields of engineering, computer
science, physics and life sciences, as well as the methodologies behind them. The
series contains monographs, lecture notes and edited volumes in computational
intelligence spanning the areas of neural networks, connectionist systems, genetic
algorithms,evolutionarycomputation,artificialintelligence,cellularautomata,self-
organizingsystems,softcomputing,fuzzysystems,andhybridintelligentsystems.
Ofparticularvaluetoboththecontributorsandthereadershiparetheshortpublica-
tiontimeframeandtheworld-widedistribution,whichenablebothwideandrapid
disseminationofresearchoutput.
This series also publishes Open Access books. A recent example is the book
Swan,Nivel,Kant,Hedges,Atkinson,Steunebrink:TheRoadtoGeneralIntelligence
https://link.springer.com/book/10.1007/978-3-031-08020-3
IndexedbySCOPUS,DBLP,WTIFrankfurteG,zbMATH,SCImago.
AllbookspublishedintheseriesaresubmittedforconsiderationinWebofScience.
· ·
Sachin Sharad Pawar Pushpak Bhattacharyya
Girish Keshav Palshikar
Investigations in Entity
Relationship Extraction
SachinSharadPawar PushpakBhattacharyya
TCSResearch DepartmentofComputerScience
Pune,India andEngineering
IndianInstituteofTechnologyBombay
GirishKeshavPalshikar Mumbai,India
TCSResearch
Pune,India
ISSN 1860-949X ISSN 1860-9503 (electronic)
StudiesinComputationalIntelligence
ISBN 978-981-19-5390-3 ISBN 978-981-19-5391-0 (eBook)
https://doi.org/10.1007/978-981-19-5391-0
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature
SingaporePteLtd.2023
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse
ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Preface
With the advent of the Internet, a large amount of digital text is generated every
day,suchasnewsarticles,researchpublications,blogs,socialmedia,andquestion
answeringforums.Asalotofvaluableinformationishiddenwithinsuchtext,itis
essentialtodeveloptechniquesforautomaticallyextractinginformationfromthese
documents.SeveralapplicationssuchasNaturalLanguageUnderstanding,Question
Answering,andInformationRetrievalwouldbenefitfromthisinformation.Themost
basicunitofinformationisanentity,andgenerally,theseentitiesoccurinsentences
withvarioussemanticrelationsamongthem.Theentitiesandrelationsareasmall
butimportantsteptowardunderstandingthemeaningofthattext.
This monograph is a result of a Ph.D. thesis investigating various techniques
for entity and relation extraction. It provides an extensive overview of traditional
entity and relation extraction techniques as well as the recent deep learning-based
techniques. The book also focuses on two important types of relation extraction
techniques—(i) joint extraction techniques where entities and relations are jointly
extractedwhereboththetasksofentityandrelationextractionhelpeachother,and(ii)
extractionofcomplexrelationswheretherelationtypesmaybeN-ary(i.e.,having
more than two entity mention arguments) and cross-sentence (i.e., entity mention
argumentsmaybespreadacrossmultiplesentences).
Commercially, the tasks of entity and relation extraction are quite important as
they serve as key prerequisites for several downstream NLP applications such as
KnowledgeGraphCreation,QuestionAnswering,IntelligentInformationRetrieval,
and Summarization. The knowledge of entity mentions present in a text along
with semantic relations among them is an important step toward natural language
understanding.
Pune,India SachinSharadPawar
July2022 PushpakBhattacharyya
GirishKeshavPalshikar
v
Acknowledgements
This monograph is based on my Ph.D. thesis work jointly carried out at Indian
InstituteofTechnologyBombayandTCSResearch,underthesupervisionofProf.
PushpakBhattacharyya(IITBombay)andGirishKeshavPalshikar(TCSResearch).
Sofirstly,Iwouldliketoexpress mygratitude towardbothmysupervisorswhose
supportandguidancewereimmenselyimportantthroughoutmyPh.D.work.Ialso
thankmycommitteemembers,Prof.MalharKulkarni,Prof.OmDamani,andProf.
Saketha Nath, for their invaluable feedback. For transitioning the thesis into this
monograph, I would like to thank Prof. Pushpak for his invaluable comments and
suggestions.
Iamalsogratefultomyemployer,TataConsultancyServices,forgivingmethe
opportunity to pursue Ph.D. while working with them. I was fortunate to have a
strong group of fellow Ph.D. students from IITB with whom I had several useful
technical and non-technical discussions. I would especially like to thank Girish,
Rudra,Joe,Raksha,Kevin,Sandeep,Diptesh,Anoop,andAaditya.Iamalsothankful
tomycolleagues-cum-friendsinTCSPune—Swapnil,Nitin,Sangam,Kripa,Sapan,
Avadhut,Vikrant,Shailesh,Manoj,andRajiv.Ialwaysdiscussedmyresearchwork
indetailwithSwapnilandNitin.Theyhaveprovidedmewithhelpfulsuggestions
throughout my Ph.D. journey. I would also like to thank Sandeep and Kedar, my
long-timefriendssincemymaster’sdaysinIITB,forourregularget-togethersthat
helpedmerefreshmymind.
Finally, I would like to express my gratitude toward my family—my parents
(SharadandSarita),mywifeAmeya,myin-laws(SadanandandVijayalaxmi).They
gave me strength and support to persistently carry on my research throughout my
Ph.D.journey.
July2022 SachinSharadPawar
vii
Contents
1 Introduction ................................................... 1
1.1 Entities ................................................... 1
1.2 Relations .................................................. 2
1.2.1 Global-versusMention-levelRelations ................. 3
1.3 Motivation ................................................ 4
1.4 ResearchGapsandObjectives ................................ 5
1.4.1 End-to-endRelationExtraction ........................ 6
1.4.2 N-aryCross-sentenceRelationExtraction ............... 7
1.5 OrganizationoftheMonograph ............................... 8
References ..................................................... 8
2 LiteratureSurvey .............................................. 11
2.1 RelationExtraction ......................................... 11
2.1.1 Feature-basedMethods ............................... 11
2.1.2 KernelMethods ...................................... 14
2.1.3 NeuralApproaches ................................... 23
2.1.4 Datasets ............................................ 23
2.1.5 Evaluation .......................................... 24
2.2 JointEntityandRelationExtraction ........................... 25
2.2.1 MotivatingExample .................................. 26
2.2.2 OverviewofTechniques .............................. 26
2.2.3 JointInferenceTechniques ............................ 27
2.2.4 JointModels ........................................ 30
2.2.5 ExperimentalEvaluation .............................. 33
2.3 N-aryCross-sentenceRelationExtraction ...................... 37
2.3.1 ExtractingCross-sentenceRelations .................... 37
2.3.2 ExtractingN-aryandCross-sentenceRelations ........... 38
References ..................................................... 40
ix
x Contents
3 JointInferenceforEnd-to-endRelationExtraction ................ 45
3.1 Introduction ............................................... 45
3.1.1 ProblemDefinition ................................... 46
3.1.2 MotivationforJointExtraction ......................... 46
3.2 Background:MarkovLogicNetworks(MLN) .................. 47
3.2.1 BasicsofFirst-orderLogic ............................ 47
3.2.2 BasicsofMLNs ..................................... 48
3.2.3 FormalDefinition .................................... 49
3.2.4 InferenceinMLNs ................................... 49
3.3 BuildingBlocksforOurApproach ............................ 50
3.3.1 IdentifyingEntityMentionCandidates .................. 50
3.3.2 EntityTypeClassifier ................................. 51
3.3.3 EntityTypeAgnosticRelationClassifier ................ 52
3.3.4 PipelineRelationClassifier ............................ 53
3.4 JointExtractionusingInferenceinMarkovLogicNetworks
(MLN) .................................................... 53
3.4.1 Motivation .......................................... 53
3.4.2 DomainsandPredicates ............................... 54
3.4.3 GenericRules ....................................... 55
3.4.4 Sentence-specificRules ............................... 56
3.4.5 AdditionalSemanticRules ............................ 59
3.4.6 JointInference ....................................... 60
3.5 Example .................................................. 60
3.6 ExperimentalAnalysis ...................................... 62
3.6.1 LimitationsofOurApproach .......................... 65
References ..................................................... 65
4 JointModelforEnd-to-EndRelationExtraction .................. 67
4.1 Motivation ................................................ 67
4.2 AllWordPairsModel(AWP-NN) ............................ 68
4.2.1 FeaturesfortheAWP-NNModel ....................... 69
4.2.2 ArchitectureoftheAWP-NNModel .................... 71
4.3 InferenceUsingMarkovLogicNetworks ...................... 73
4.4 ExperimentalAnalysis ...................................... 76
4.4.1 Datasets ............................................ 76
4.4.2 ImplementationDetails ............................... 77
4.4.3 Results ............................................. 77
4.4.4 AnalysisofResults ................................... 80
4.5 Domain-SpecificEntitiesandRelations ........................ 82
4.5.1 AdverseDrugReactions .............................. 82
4.5.2 TAC2017:ADRExtractionTask ....................... 84
References ..................................................... 87
Contents xi
5 N-aryCross-SentenceRelationExtraction ........................ 89
5.1 Introduction ............................................... 89
5.2 ProblemDefinition ......................................... 92
5.2.1 ComparisonwithRelevantPastWork ................... 93
5.3 ProposedApproach ......................................... 93
5.3.1 ConstructingSequenceRepresentations ................. 95
5.3.2 ConstrainedSubsequenceKernel(CSK) ................. 96
5.3.3 FormalDefinitionofCSK ............................. 97
5.3.4 ClassifyingCandidateRelationInstances ................ 98
5.4 ExperimentalAnalysis ...................................... 102
5.4.1 Datasets ............................................ 102
5.4.2 ImplementationDetails ............................... 106
5.4.3 AnalysisofResultsandErrors ......................... 107
5.5 DiscussiononDecompositionofN-aryRelations ............... 109
5.5.1 ExamplesofVariousRelationTypes .................... 110
5.5.2 GeneralizedTheorem ................................. 112
References ..................................................... 113
6 RecentAdvancesinEntityandRelationExtraction ................ 115
6.1 JointEntityandRelationExtraction ........................... 115
6.1.1 UsingSpan-BasedRepresentationforEntityMentions .... 115
6.1.2 UsingBERTEmbeddings ............................. 118
6.2 N-aryCross-SentenceRelationExtraction ..................... 118
6.2.1 StandardDataset ..................................... 118
6.2.2 GraphNeuralNetworks ............................... 119
6.2.3 UsingBERTEmbeddings ............................. 119
References ..................................................... 120
7 Conclusions .................................................... 123
7.1 SummaryoftheMonograph ................................. 123
7.2 FutureDirections ........................................... 125
References ..................................................... 127
AppendixA:Foundations .......................................... 129