Bioinformatics for Evolutionary Biologists: A Problems Approach

410 Pages·2023·3.337 MB·English

by  Bernhard Haubold, Angelika Börsch-Haubold

Bernhard Haubold
Angelika Börsch-Haubold
Bioinformatics
for Evolutionary
Biologists
A Problems Approach
Second Edition
Bioinformatics for Evolutionary Biologists
ö
Bernhard Haubold Angelika B rsch-Haubold
(cid:129)
Bioinformatics
for Evolutionary Biologists
A Problems Approach
Second Edition
123
Bernhard Haubold Angelika Börsch-Haubold
Department ofEvolutionary Genetics Plön,Schleswig-Holstein, Germany
Max-Planck-Institute
for EvolutionaryBiology
Plön,Schleswig-Holstein, Germany
ISBN978-3-031-20413-5 ISBN978-3-031-20414-2 (eBook)
https://doi.org/10.1007/978-3-031-20414-2
1stedition:©SpringerInternationalPublishingAG2017
2ndedition:©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringer
NatureSwitzerlandAG2023
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseof
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors, and the editorsare safeto assume that the adviceand informationin this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
hereinorforanyerrorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregard
tojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Like all of biology, evolutionary biology is rapidly turning digital as we are sur-
rounded by more and more cheap computers and cheap computerized machines
for gathering data, especially DNA sequence data. This book is designed to help
biologistsanalyzethesedatausingclassicalbioinformaticstechniques.
We teach bioinformatics as a practical skill focused on the analysis of DNA
sequencesontheUnixcommandline.Wedothisbyposingaseriesofgradedand
solvedProblems,mostofwhicharemeanttobetackledontheUnixcommandline.
The Unix commandline is our interfaceof choice, becauseit is malleable like no
graphicalinterface.
The malleability of the Unix commandline takes some practice to unlock,and
opportunities for getting this practice have grown in leaps and bounds since the
publicationofthefirsteditionofBioinformaticsforEvolutionaryBiologistsin2017.
In2019MicrosoftreleasedanimprovedversionoftheirLinuxsubsystemthatruns
onWindows10,whichwasinturnimprovedforWindows11twoyearslater.In2019
Google’schromeOSjoinedtheclubofUnix-capablesystemswiththerelaseofLinux
forChromebooks.GiventhattheUnixcommandlinehasbeenpartofApple’smacOS
since2001,thismeansthatallfourmajorcomputeroperatingsystems—Windows,
macOS,chromeOS,andLinux—nowallowuserseasyaccesstothecommandline.
Outofthesefour,wehavetestedontwo,Windows11withUbuntu20.04andmacOS
BigSur.
ThisbookisintendedforbiologistswithnopriorknowledgeofUnix.Wecover
thefundamentalsofinteractingwiththecomputeronthecommandlineinChapter
1.InChapter2wealignpairsofproteinandDNAsequencesandscoretheresulting
alignments.Thealignmenttechniquesexploredherecanbespedupsubstantiallyby
usingexactmatching,thetopicofChapter3.InChapter4,wemixexactmatching
withthealignmentmethodsofChapter2toarriveatthefastalgorithmsofmodern
molecularbiologyunderlyingsuchpopulartoolsasBlast.Uptothispoint,wehave
only aligned pairs of sequences, but at the end of Chapter 4 we generalize this to
align multiple sequences. Multiple sequence alignments are the starting point for
phylogenies,thetopicofChapter5.Phylogeniesdescribetheevolutionofmultiple
species. If we are interested in the evolution of individual species, we can use
v
vi Preface
concepts from population genetics, the topic of Chapter 6. A central model of
populationgeneticstracesasampleofgenesfromthepresentbackintimetotheir
mostrecentcommonancestor.Thisresultsinatreecalledcoalescent,whichlooks
similartothephylogeniesofthepreviouschapter.Inthelastchapter,Chapter7,we
covertwotopicsnotnecessarilybasedonsequences,butstillubiquitousinmodern
biology,thestatisticsofmultipletestingandthefundamentalsofdatabases.
Thefirsteditionofthisbookwaswellreceivedandwehaveuseditregularlyin
coursestaughtattheMax-Planck-InstituteinPlo¨nandatLu¨beckUniversity.Some
of our students were so kind as to point out errors—thankyou again!—which we
havetriedourbestto fix.Someerrorsprobablyremain,andweareboundtohave
introducednewones,soifyoucatchabug,pleasedropBHalineat
[email protected]
Wewilladdittotheerratalistmaintainedat
guanine.evolbio.mpg.de/beb2
and,wehope,fixitinafutureedition.Whilefixingerrors,wealsostrovetoclarify
and in places expand the narrative. As a result, the total number of Problems has
grownfromunder500toover800.
However,thebiggestchangeconcernstheUnixtoolswrittenforthecourse.We’ve
rewrittentheminGoanddistributethemasabiologicaltoolbox,a“biobox”,at
sn.pub/dy6S42
Whilerewritingtheprograms,wealsoaddedafewnewones,forexampleplotLine,
toquicklyconvertpairsof
푥,
푦valuesintolinegraphs.
Finally, let us reiterate a point from the first edition. Half of this book consists
ofAnswers,whichyoufindintheyellowpagesintheback.Thesearedesignedto
bereadevenifyoudofindyourownsolutiontoaparticularProblem.AnAnswer
mightexpandontheProblem,mightinspireyoutoimproveyouranswer,andrarely,
wehope,mightgiveyouthesatisfactionofhavingcomeupwithabetterone.Inthat
casewe’dbeparticularlykeentohearfromyou.
Plo¨n, BernhardHaubold
August2022 AngelikaBo¨rsch-Haubold
Contents
PartI Problems
1 TheUnixCommandLine ....................................... 3
1.1 GettingStarted............................................. 4
1.2 Files,Directories,andPrograms .............................. 11
1.3 Scripts.................................................... 18
2 OptimalAlignment............................................. 29
2.1 KeepingScore ............................................. 29
2.2 Construction............................................... 39
2.3 Application................................................ 51
3 ExactMatching................................................ 57
3.1 KeywordTrees............................................. 57
3.2 SuffixTrees ............................................... 64
3.3 SuffixArrays .............................................. 70
3.4 TextCompression .......................................... 81
4 FastAlignment ................................................ 93
4.1 Global.................................................... 93
4.2 Local..................................................... 98
4.3 Glocal....................................................110
4.4 Assembly .................................................122
4.5 MultipleSequences.........................................132
5 EvolutionBetweenSpecies:Phylogeny............................141
5.1 TreesofLife...............................................141
5.2 RootedTrees ..............................................148
5.3 UnrootedTrees ............................................154
vii
viii Contents
6 EvolutionwithinPopulations ....................................161
6.1 DescentfromOneorTwoParents.............................161
6.2 TheCoalescent ............................................172
7 InterrogatingandStoringData ..................................183
7.1 Statistics ..................................................183
7.2 RelationalDatabases........................................190
PartII Answers
1 TheUnixCommandLine .......................................207
1.1 GettingStarted.............................................207
1.2 Files,Directories,andPrograms ..............................211
1.3 Scripts....................................................217
2 OptimalAlignment.............................................229
2.1 KeepingScore .............................................229
2.2 Construction...............................................239
2.3 Application................................................250
3 ExactMatching................................................257
3.1 KeywordTrees.............................................257
3.2 SuffixTrees ...............................................262
3.3 SuffixArrays ..............................................266
3.4 TextCompression ..........................................274
4 FastAlignment ................................................287
4.1 Global....................................................287
4.2 Local.....................................................291
4.3 Glocal....................................................303
4.4 Assembly .................................................312
4.5 MultipleSequences.........................................320
5 EvolutionBetweenSpecies:Phylogeny............................329
5.1 TreesofLife...............................................329
5.2 RootedTrees ..............................................333
5.3 UnrootedTrees ............................................339
6 EvolutionwithinPopulations ....................................345
6.1 DescentfromOneorTwoParents.............................345
6.2 TheCoalescent ............................................351
7 InterrogatingandStoringData ..................................361
7.1 Statistics ..................................................361
7.2 RelationalDatabases........................................367
A UnixGuide ....................................................377
Contents ix
B Programs .....................................................397
B.1 Own .....................................................397
B.2 Biobox ...................................................398
B.3 Third-Party................................................398
References.........................................................401
Index .............................................................405
Part I
Problems