Table Of ContentP1:JZZ
0521836573pre CB1028/Feldman 0521836573 October12,2006 18:37
The Text Mining Handbook
Text mining is a new and exciting area of computer science research that tries to
solve the crisis of information overload by combining techniques from data mining,
machine learning, natural language processing, information retrieval, and knowledge
management. Similarly, link detection – a rapidly evolving approach to the analysis of
textthatsharesandbuildsonmanyofthekeyelementsoftextmining–alsoprovides
new tools for people to better leverage their burgeoning textual data resources. Link
detectionreliesonaprocessofbuildingupnetworksofinterconnectedobjectsthrough
various relationships in order to discover patterns and trends. The main tasks of link
detectionaretoextract,discover,andlinktogethersparseevidencefromvastamounts
ofdatasources,torepresentandevaluatethesignificanceoftherelatedevidence,and
tolearnpatternstoguidetheextraction,discovery,andlinkageofentities.
TheTextMiningHandbookpresentsacomprehensivediscussionofthestateoftheart
intextminingandlinkdetection.Inadditiontoprovidinganin-depthexaminationofcore
textminingandlinkdetectionalgorithmsandoperations,theworkexaminesadvanced
preprocessing techniques, knowledge representation considerations, and visualization
approaches. Finally, the book explores current real-world, mission-critical applications
of text mining and link detection in such varied fields as corporate finance business
intelligence,genomicsresearch,andcounterterrorismactivities.
Dr. Ronen Feldman is a Senior Lecturer in the Mathematics and Computer Science
DepartmentofBar-IlanUniversityandDirectoroftheDataandTextMiningLaboratory.
Dr.Feldmaniscofounder,ChiefScientist,andPresidentofClearForest,Ltd.,aleader
in developing next-generation text mining applications for corporate and government
clients.HealsorecentlyservedasanAdjunctProfessoratNewYorkUniversity’sStern
SchoolofBusiness.Apioneerintheareasofmachinelearning,datamining,andunstruc-
tureddatamanagement,hehasauthoredorcoauthoredmorethan70publishedarticles
andconferencepapersintheseareas.
JamesSangerisaventurecapitalist,appliedtechnologist,andrecognizedindustryexpert
intheareasofcommercialdatasolutions,Internetapplications,andITsecurityproducts.
He is a partner at ABS Ventures, an independent venture firm founded in 1982 and
originallyassociatedwithtechnologybankingleaderAlex.BrownandSons.Immediately
before joining ABS Ventures, Mr. Sanger was a Managing Director in the New York
officesofDBCapitalVenturePartners,theglobalventurecapitalarmofDeutscheBank.
Mr.Sangerhasbeenaboardmemberofseveralthought-leadingtechnologycompanies,
includingInxightSoftware,GomezInc.,andClearForest,Inc.;hehasalsoservedasan
officialobservertotheboardsofAlphaBlox(acquiredbyIBMin2004),Intralinks,and
ImagineSoftwareandasamemberoftheTechnicalAdvisoryBoardofQualys,Inc.
i
P1:JZZ
0521836573pre CB1028/Feldman 0521836573 October12,2006 18:37
ii
P1:JZZ
0521836573pre CB1028/Feldman 0521836573 October12,2006 18:37
THE TEXT
MINING HANDBOOK
Advanced Approaches in
Analyzing Unstructured Data
Ronen Feldman
Bar-IlanUniversity,Israel
James Sanger
ABSVentures,Waltham,Massachusetts
iii
CAMBRIDGEUNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
The Edinburgh Building, Cambridge CB2 8RU, UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521836579
© Ronen Feldman and James Sanger 2007
This publication is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format 2006
ISBN-13 978-0-511-33507-5 eBook (NetLibrary)
ISBN-10 0-511-33507-5 eBook (NetLibrary)
ISBN-13 978-0-521-83657-9 hardback
ISBN-10 0-521-83657-3 hardback
Cambridge University Press has no responsibility for the persistence or accuracy of urls
for external or third-party internet websites referred to in this publication, and does not
guarantee that any content on such websites is, or will remain, accurate or appropriate.
P1:JZZ
0521836573pre CB1028/Feldman 0521836573 October12,2006 18:37
Inlovingmemoryofmyfather,IssacFeldman
v
P1:JZZ
0521836573pre CB1028/Feldman 0521836573 October12,2006 18:37
vi
P1:JZZ
0521836573pre CB1028/Feldman 0521836573 October12,2006 18:37
Contents
Preface page x
I. Introduction to Text Mining 1
I.1 DefiningTextMining 1
I.2 GeneralArchitectureofTextMiningSystems 13
II. Core Text Mining Operations 19
II.1 CoreTextMiningOperations 19
II.2 UsingBackgroundKnowledgeforTextMining 41
II.3 TextMiningQueryLanguages 51
III. Text Mining Preprocessing Techniques 57
III.1 Task-OrientedApproaches 58
III.2 FurtherReading 62
IV. Categorization 64
IV.1 ApplicationsofTextCategorization 65
IV.2 DefinitionoftheProblem 66
IV.3 DocumentRepresentation 68
IV.4 KnowledgeEngineeringApproachtoTC 70
IV.5 MachineLearningApproachtoTC 70
IV.6 UsingUnlabeledDatatoImproveClassification 78
IV.7 EvaluationofTextClassifiers 79
IV.8 CitationsandNotes 80
V. Clustering 82
V.1 ClusteringTasksinTextAnalysis 82
V.2 TheGeneralClusteringProblem 84
V.3 ClusteringAlgorithms 85
V.4 ClusteringofTextualData 88
V.5 CitationsandNotes 92
vii
P1:JZZ
0521836573pre CB1028/Feldman 0521836573 October12,2006 18:37
viii Contents
VI. Information Extraction 94
VI.1 IntroductiontoInformationExtraction 94
VI.2 HistoricalEvolutionofIE:TheMessageUnderstanding
ConferencesandTipster 96
VI.3 IEExamples 101
VI.4 ArchitectureofIESystems 104
VI.5 AnaphoraResolution 109
VI.6 InductiveAlgorithmsforIE 119
VI.7 StructuralIE 122
VI.8 FurtherReading 129
VII. Probabilistic Models for Information Extraction 131
VII.1 HiddenMarkovModels 131
VII.2 StochasticContext-FreeGrammars 137
VII.3 MaximalEntropyModeling 138
VII.4 MaximalEntropyMarkovModels 140
VII.5 ConditionalRandomFields 142
VII.6 FurtherReading 145
VIII. Preprocessing Applications Using Probabilistic
and Hybrid Approaches 146
VIII.1 ApplicationsofHMMtoTextualAnalysis 146
VIII.2 UsingMEMMforInformationExtraction 152
VIII.3 ApplicationsofCRFstoTextualAnalysis 153
VIII.4 TEG:UsingSCFGRulesforHybrid
Statistical–Knowledge-BasedIE 155
VIII.5 Bootstrapping 166
VIII.6 FurtherReading 175
IX. Presentation-Layer Considerations for Browsing
and Query Refinement 177
IX.1 Browsing 177
IX.2 AccessingConstraintsandSimpleSpecificationFilters
atthePresentationLayer 185
IX.3 AccessingtheUnderlyingQueryLanguage 186
IX.4 CitationsandNotes 187
X. Visualization Approaches 189
X.1 Introduction 189
X.2 ArchitecturalConsiderations 192
X.3 CommonVisualizationApproachesforTextMining 194
X.4 VisualizationTechniquesinLinkAnalysis 225
X.5 Real-WorldExample:TheDocumentExplorerSystem 235
XI. Link Analysis 244
XI.1 Preliminaries 244
P1:JZZ
0521836573pre CB1028/Feldman 0521836573 October12,2006 18:37
Contents ix
XI.2 AutomaticLayoutofNetworks 246
XI.3 PathsandCyclesinGraphs 250
XI.4 Centrality 251
XI.5 PartitioningofNetworks 259
XI.6 PatternMatchinginNetworks 272
XI.7 SoftwarePackagesforLinkAnalysis 273
XI.8 CitationsandNotes 274
XII. Text Mining Applications 275
XII.1 GeneralConsiderations 276
XII.2 CorporateFinance:MiningIndustryLiteraturefor
BusinessIntelligence 281
XII.3 A“Horizontal”TextMiningApplication:PatentAnalysis
SolutionLeveragingaCommercialTextAnalytics
Platform 297
XII.4 LifeSciencesResearch:MiningBiologicalPathway
InformationwithGeneWays 309
Appendix A: DIAL: A Dedicated Information Extraction Language for
Text Mining 317
A.1 WhatIstheDIALLanguage? 317
A.2 InformationExtractionintheDIALEnvironment 318
A.3 TextTokenization 320
A.4 ConceptandRuleStructure 320
A.5 PatternMatching 322
A.6 PatternElements 323
A.7 RuleConstraints 327
A.8 ConceptGuards 328
A.9 CompleteDIALExamples 329
Bibliography 337
Index 391
