Deep Learning Techniques for Biomedical and Health Informatics

352 Pages·2020·36.35 MB·English

by  Basant Agarwal (editor), Valentina Emilia Balas (editor), Lakhmi C. Jain (editor), Ramesh Chandra Poonia (editor), Manisha Sharma (editor)

Deep Learning Techniques
for Biomedical and Health
Informatics
Deep Learning Techniques
for Biomedical and Health
Informatics
Edited by
Basant Agarwal
Valentina Emilia Balas
Lakhmi C. Jain
Ramesh Chandra Poonia
Manisha
AcademicPressisanimprintofElsevier
125LondonWall,LondonEC2Y5AS,UnitedKingdom
525BStreet,Suite1650,SanDiego,CA92101,UnitedStates
50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom
©2020ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicor
mechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem,without
permissioninwritingfromthepublisher.Detailsonhowtoseekpermission,furtherinformationaboutthe
Publisher’spermissionspoliciesandourarrangementswithorganizationssuchastheCopyrightClearance
CenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(other
thanasmaybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenour
understanding,changesinresearchmethods,professionalpractices,ormedicaltreatmentmaybecome
necessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingandusing
anyinformation,methods,compounds,orexperimentsdescribedherein.Inusingsuchinformationormethods
theyshouldbemindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhavea
professionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliability
foranyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,or
fromanyuseoroperationofanymethods,products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-819061-6
ForinformationonallAcademicPresspublications
visitourwebsiteathttps://www.elsevier.com/books-and-journals
Publisher:MaraConner
AcquisitionEditor:ChrisKatsaropoulos
EditorialProjectManager:GabrielaD.Capille
ProductionProjectManager:PunithavathyGovindaradjane
CoverDesigner:MarkRogers
TypesetbySPiGlobal,India
Contributors
SelamAhderomElectronScienceResearchInstitute,EdithCowanUniversity,Joondalup,
WA, Australia
Kamal Alameh Electron Science Research Institute, Edith Cowan University, Joondalup,
WA, Australia
Salah Alheejawi Department ofElectrical and Computer Engineering, University of
Alberta,Edmonton, AB, Canada
Ashish Anand Department ofComputer Science and Engineering,Indian Institute of
Technology Guwahati, Guwahati, India
Rangel ArthurSchool of Technology (FT), State University of Campinas(UNICAMP),
Limeira, Brazil
MuhammadWaseem Ashraf GC University Lahore, Lahore, Pakistan
ValentinaEmiliaBalasDepartmentofAutomationandAppliedInformatics,AurelVlaicu
University of Arad, Arad, Romania
RichardBerendt Cross CancerInstitute, Edmonton,AB,Canada
AnimeshBiswasDepartment ofMathematics, University ofKalyani, Kalyani,India
Mou De Netaji Subhash Engineering College, Kolkata; Computer Innovative Research
Society, Howrah, India
Vijaypal SinghDhakaManipalUniversity Jaipur, Jaipur, India
Reinaldo Padilha Franc¸aSchool ofElectrical Engineering and Computing(FEEC),
State University of Campinas(UNICAMP),Campinas, Brazil
Bappaditya GhoshDepartment of Mathematics, University of Kalyani, Kalyani,India
E.A. Gopalakrishnan Centerfor Computational Engineering and Networking (CEN),
AmritaSchool ofEngineering,Amrita Vishwa Vidyapeetham,Coimbatore,India
P.GopikaCenterforComputationalEngineeringandNetworking(CEN),AmritaSchoolof
Engineering, AmritaVishwa Vidyapeetham, Coimbatore,India
Yuzo Iano School of ElectricalEngineering and Computing (FEEC), State University of
Campinas(UNICAMP), Campinas, Brazil
xiii
xiv Contributors
VijayJeyakumarDepartmentofBiomedical Engineering, SSN CollegeofEngineering,
Chennai,India
Naresh Jha Cross CancerInstitute, Edmonton,AB,Canada
Anirban KunduNetaji Subhash Engineering College, Kolkata; Computer Innovative
Research Society, Howrah,India
PreethiKurian Department of Biomedical Engineering, SSN Collegeof Engineering,
Chennai,India
ChengLuCASE Western Reserve University, Cleveland, OH,United States
Swanirbhar Majumder Department of Information Technology,Tripura University,
Agartala, India
MrinalMandal Department of Electricaland Computer Engineering, University of
Alberta, Edmonton, AB,Canada
NavidMavaddatElectronScienceResearchInstitute,EdithCowanUniversity,Joondalup,
WA,Australia
D.A. Meedeniya University of Moratuwa, Moratuwa, Sri Lanka
Takhellambam Gautam Meitei Department of Electronics and Communication
Engineering,North Eastern Regional Institute of Scienceand Technology,Nirjuli, India
AnaCarolinaBorgesMonteiroSchoolofElectricalEngineeringandComputing(FEEC),
State University ofCampinas (UNICAMP), Campinas, Brazil
P.Naga Srinivasu Department ofCSE, GIT, GITAM Deemed to be University,
Visakhapatnam, India
Ramesh Chandra Poonia Norwegian University of Scienceand Technology (NTNU),
Alesund, Norway
Nitesh PradhanManipal University Jaipur, Jaipur, India
Geeta Rani Manipal University Jaipur,Jaipur,India
Nivedita Ray De Sarkar Netaji SubhashEngineering College,Kolkata; Computer
InnovativeResearch Society, Howrah, India
I.D. Rubasinghe University ofMoratuwa, Moratuwa, Sri Lanka
Subhashis SahuDepartment of Physiology,University ofKalyani, Kalyani, India
SunilKumarSahuDepartmentofComputerScienceandEngineering,IndianInstituteof
Technology Guwahati, Guwahati,India
Contributors xv
SinamAjitkumarSinghDepartment ofElectronics and Communication Engineering,
North EasternRegional Institute ofScience and Technology, Nirjuli, India
K.P.SomanCenterforComputationalEngineeringandNetworking(CEN),AmritaSchool
ofEngineering,Amrita Vishwa Vidyapeetham, Coimbatore,India
V.SowmyaCenterforComputationalEngineeringandNetworking(CEN),AmritaSchool
ofEngineering,Amrita Vishwa Vidyapeetham, Coimbatore,India
T.SrinivasaRaoDepartment ofCSE, GIT, GITAM Deemed to be University,
Visakhapatnam, India
MuhammadImranTariqThe superior University,Lahore, Pakistan
ShahzadiTayyabaThe University ofLahore,Lahore, Pakistan
Valentina TiporliniElectron Science Research Institute, Edith Cowan University,
Joondalup, WA,Australia
HongmingXuCleveland Clinic, Cleveland,OH,United States
1
Unified neural architecture
for drug, disease, and clinical
entity recognition
Sunil Kumar Sahu, Ashish Anand
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING, INDIAN INSTITUTE
OF TECHNOLOGY GUWAHATI, GUWAHATI, INDIA
1.1 Introduction
Biomedicalandclinicalnamedentityrecognition(NER)inthetextisanimportantstepin
several biomedical and clinical information extraction tasks [1–3]. State-of-the-art
methods formulate an NER task as a sequence labeling problem where each word is
labeled with a tag and, based on the tag sequence, entities of interest are identified. In
comparison to the generic domain, recognizing entities in the biomedical and clinical
domainsaredifficultduetoseveralreasons,includingtheuseofnonstandardabbrevia-
tionsoracronyms,multiplevariationsofthesameentities,etc.[3,4].Furthermore,clin-
ical notes often contain shorter, incomplete, and grammatically incorrect sentences [3],
thus making it difficult for models to extract rich context. Most widely used models,
including conditional random field (CRF), maximum entropy Markov model (MEMM),
orsupportvectormachine,usemanuallydesignedrulestoobtainmorphological,syntac-
tic,semantic,andcontextualinformationofawordorpieceoftextsurroundingaword,
and then use them asfeaturesfor identifying correct labels [5–10].Performance of such
models is limited by the choice of explicitly designed features specific to the task and
itscorrespondingdomain.Forexample,ChowdhuryandLavelli[6]explainedseveralrea-
sonswhyfeaturesdesignedforbiologicalentitiessuchasproteinsorgenesarenotequally
importantfordisease name recognition.
Deeplearning-basedmodelshavebeenusedtoreducemanualeffortsforexplicitfea-
turedesign[11].Here,distributionalfeaturesareusedinplaceofmanuallydesignedfea-
turesandamultilayerneuralnetworkisusedinplaceofalinearmodeltoovercomethe
needsoftask-specificmeticulousfeatureengineering.Although proposed methodsout-
performseveralgenericdomainsequencelabelingtasks,itfailstoovercomethestateof
artinabiomedicaldomain[12].Therearetwoplausiblereasonsbehindthis:First,itlearns
featuresonlyfromword-levelembeddingand,second,ittakesintoaccountonlyafixed
lengthcontextoftheword.Ithasbeenobservedthatword-levelembeddingspreservethe
DeepLearningTechniquesforBiomedicalandHealthInformatics.https://doi.org/10.1016/B978-0-12-819061-6.00001-X 1
©2020ElsevierInc.Allrightsreserved.
2 Deep Learning Techniques for Biomedical and Health Informatics
syntacticandsemanticpropertiesofawordbutmayfailtopreservemorphologicalinfor-
mationthatcanalsoplayanimportantroleinNER[6,13–16].Forinstance,drugnames
Cefaclor,Cefdinir,Cefixime,Cefprozil,andCephalexinhaveacommonprefix,andDoxy-
cycline,Minocycline,andTetracyclinehaveacommonsuffix.Thesecommonprefixes/suf-
fixes are often sufficient to predict entity types. Furthermore, window-based neural
architecture can only consider words that appear within the user-decided window size
as context and thus is likely to fail in picking up vital clues lying outside the window.
Thisworkaimstoovercomethetwopreviouslymentionedissues.Toobtainbothmor-
phologicallyaswellassyntacticallyandsemanticallyrichembedding,twobi-directional
longshort-termmemorynetworks(BLSTMs)areusedinahierarchy.First,BLSTMworks
oneachcharacterofthewordsandaccumulatesmorphologicallyrichwordembedding.
Second, BLSTM works at the word level of a sentence to learn contextually rich feature
vectors.Tomakesureallcontextlyinganywhereinthesentenceisutilized,weconsider
theentiresentenceasinputanduseafirst-orderlinearchainCRFinthefinalprediction
layer.The CRFlayer accommodates dependency information about the tags.
We evaluated the proposed model on three standard biomedical entity recognition
tasks,namelyDiseaseNER,DrugNER,andClinicalNER.Thisstudydistinguishedfeatures
compared in several other studies [15, 17–19]. Ma and Hovy [15] focused on sequence
labeling tasks in the generic domain and used a convolutional neural network (CNN)
for learning character-based embedding in contrast to bi-directional LSTMused in this
study. Luo et al. [17] used a similar architecture as ours for the BioCreative V.5.BeCalm
Tasksfocusedonpatents.Luoetal.[19]usedattention-basedBLSTMwithCRFforchem-
icalNER,whereasZengetal.[18]usedasimilararchitectureonlyfordrugNER.However,
ourworkstillhasalottooffertoreaders.First,weevaluateaunifiedmodelonthedifferent
genre of texts (clinical notes vs. biomedical research articles) for multiple entity types.
None of the previously mentioned studies evaluate the different genre of texts. Second,
extensiveanalysesareperformedto understandthe significanceofvarious components
ofthemodelarchitecture,includingCRF(sentence-levellikelihoodforaccountingfortag
dependency)versusword-levellikelihood(treatingeachtagindependently);featureabla-
tionstudytounderstandtheimportanceofeachfeaturetype;andthesignificanceofword
and character embedding. Lastly, error analysis is also performed to gain insight as to
wherenewmodelsshouldfocustofurtherimprovetheperformance.Wecomparethepro-
posed model with the existing state-of-the-art models for each task and show that it
outperforms them. Further analysis of the model indicates the importance of using
character-basedwordembeddingalongwithwordembeddingandCRFlayerinthefinal
outputlayer.
1.2 Method
1.2.1 Bi-directional long short-term memory
Recurrent neural network (RNN) is a variant of neural networks that utilizes sequential
information and maintains history through its recurrent connection [20, 21]. RNN can
Chapter1(cid:129)Unifiedneuralarchitecturefordrug,disease,andclinicalentityrecognition 3
beusedforasequenceofanylength;however,inpractice,itfailstomaintainlong-term
dependencyduetovanishingandexplodinggradientproblems[22,23].Longshort-term
memory(LSTM)network[24]isavariantofRNNthattakescareoftheissuesassociated
with vanillaRNN byusing three gates (input,output, and forget)and amemory cell.
WeformallydescribethebasicequationspertainingtotheLSTMmodel.Leth(t(cid:1)1)and
c(t(cid:1)1)behiddenandcellstatesofLSTM,respectively,attimet(cid:1)1,thencomputationof
currenthidden state at time tcan be given as:
iðtÞ¼σðUðiÞxðtÞ+WðiÞhðt(cid:1)1Þ+biÞ
fðtÞ¼σðUðfÞxðtÞ+WðfÞhðt(cid:1)1Þ+bfÞ
oðtÞ¼σðUðoÞxðtÞ+WðoÞhðt(cid:1)1Þ+boÞ
gðtÞ¼tanhðUðgÞxðtÞ+WðgÞhðt(cid:1)1Þ+bgÞ
l
cðtÞ¼cðt(cid:1)1Þ*fðtÞ+gðtÞ*iðtÞ
hðtÞ¼tanhðcðtÞÞ*oðtÞ
whereσissigmoidactivationfunction,*isanelement-wiseproduct,xðtÞ2d istheinput
vectorattimet,U(i),U(f),U(o),UðgÞ2N(cid:3)d,W(i),W(o),W(f),WðgÞ2N(cid:3)N,bi,bf,bo,bg2N,
h(0),cð0Þ2N arelearningparametersforLSTM.Here,disdimensionofinputfeaturevec-
tor,Nis hidden layer size, and h(t) is outputofLSTMat time step t.
IthasbecomecommonpracticetouseLSTMinbothforwardandbackwarddirections
to capture both past and future contexts, respectively. First, LSTM computes its hidden
states in the forward direction of the input sequence and then does it in the backward
direction. This way of using two LSTMs is referred to as bi-directional LSTM or simply
BLSTM. We use bi-directional LSTM in our model. The final output of BLSTM at time t
is given as:
!
hðtÞ¼hðtÞ(cid:4)hðtÞ (1.1)
!
where (cid:4) is concatenation operation and hðtÞ and hðtÞ are hidden states of forward and
backwardLSTMat timet.
1.2.2 Model architecture
SimilartoanyNERtask,weformulatethebiomedicalentityrecognitiontaskasatoken-
levelsequencetaggingproblem.Weusebeginning-inside-outside(BIO)taggingscheme
in our experiments [25]. Architecture of the proposed model is present in Fig. 1.1. Our
model takes the whole sentence as input and computes a label sequence as output.
The first layer of the model learns local feature vectors for each word in the sentence.
Weuseconcatenationofwordembedding,PoStagembedding,andcharacter-basedword
embeddingasalocalfeatureforeveryword.Character-basedwordembeddingislearned
through applying a BLSTM on the character vectors of a word. We call this layer Char
BLSTM(Section1.2.3.1).Thesubsequentlayer,calledWordBLSTM(Section1.2.5),incor-
poratescontextualinformationthroughaseparateBLSTMnetwork.Finally,weuseaCRF
to encode the correct label sequence on the output of Word BLSTM (Section 1.2.5).
Now onward, the proposed framework will be referred to as CWBLSTM. Entire network