Sound Analysis and Synthesis with R

676 Pages·2018·21.536 MB·English

by  Jérôme Sueur

Jérôme Sueur
Sound Analysis and
Synthesis with R
123
JérômeSueur
MuséumNationald’Histoirenaturelle
Paris,France
ElectronicSupplementaryMaterial Theonlineversionofthisarticle(https://doi.org/10.
1007/978-3-319-77647-7)containssupplementarymaterial,whichisavailabletoauthorized
users.
ISSN2197-5736 ISSN2197-5744 (electronic)
UseR!
ISBN978-3-319-77645-3 ISBN978-3-319-77647-7 (eBook)
https://doi.org/10.1007/978-3-319-77647-7
LibraryofCongressControlNumber:2018939906
©SpringerInternationalPublishingAG,partofSpringerNature2018
Preface
Soundisvirtuallyalwaysaroundus,everywhere,allthetime.Thismorning,myday
startedonaratherunpleasantone:thatoftherepeatedbuzzingofmyalarmclock.
The nighthad been quietdespite the purrof the centralheating,some motorbikes
racingdownthestreet,andblackbirdssinginginoneofourgarden’streesatdawn.
NowthatIgodowntothekitchenIcanhearmyclothesrubagainstmybodyandthe
woodenstepscrackundermyfeet.AssoonasImove,IrealizethatImyselfgenerate
sound.Quicklythehousewakesupinanexplosionofsurroundingsoundsseeping
from the flush, the kettle, the toaster, the fridge, ventilation, and other domestic
appliances.Musicplaysontheradio,butitdoesnotcoverthecallofahungrycat
andfamilyvoicesthatsooninvestanddominateintheacousticspace.Myworking
dayisalong,andsometimesexhausting,suiteofsounds:metallictrainscreeches,
mobilephoneringtones,officebabbles,sirenblares,streetworkroar,andradiotunes
butalsosomeamazingtropicalsoundsIrecordedinaremoteforestthatIplayback
onmycomputertoescapethatcitysoundscape.Soundisubiquitous.Itconstantly
reachesmybody,beingabsorbedorbouncedback,receivedandprocessedthrough
myears.Butmybodyisalso asoundsource.Myheart,myblood,mybreath,my
bones,andmyvocalchordsgeneratesound.I am soundin a worldofsounds.No
air,nolife,nosound.
ThesoundscapeIgothroughanygivenworkdayismainlyacitysoundscapewith
verylittleenjoyablesound.Mostofthissoundcanbeconsideredasnoise,whichis
actuallysoundconveyingnoinformationoroverlappingothermeaningfulsounds.
Nicesoundscapesarecertainlytobefoundinnature,inthemiddleofadarkforestor
inthedepthofanevendarkerocean.Wildlifesoundcanbeabirdsong,afrogcall,
aninsecthummer,adeergrunting,theexplodingsoundofasmallpistolshrimp,or
theamazingwhistleofagiantwhale.Weoftenrefertotheextraordinarydiversityof
lifeformsandcolours,butlifediversityisalsotobefoundinanimalvocalizations.
Animals can produce rhythmic or continuous, pure-tone or polyphonic,harmonic
ordysharmornic,synchronizedorcacophonicsounds.Animalscanalmostplayany
instrumentinanyorchestra.Whatevertheirproperties,thesoundsemanatingfrom
animals are never exactly the same from one song bout to another one, from one
individualtoanotherone,andfromonespeciestoanotherone.Thevarietyofanimal
soundissohighthataudiorobotsthatcanidentifyandinterpretahumanvoiceand
chatwithyouonthephonecanhardlydiscriminatethesoundofadolphinfroma
whale’s.Thisanimalacousticdiversitymaybea challengeforsoundanalysisand
synthesis,butmoreimportantly,theyarealivingtreasurethathastobeenjoyedand
preserved.
Animalsoundvarietyisthematterofbioacousticsandecoacoustics,twoclosely
relatedlifesciencesdisciplines.AsabioacousticianorecoacousticianIoftenhaveto
facethenaivebutessentialquestionaboutmyresearch:“Comeon,Jérôme,what’s
your job all about? Is there really a point in listening to cicadas?”. Addressing
thisquestionis almostthe same issue as wonderingwhywe needto name insects
and flowers, scrutinize the sky to discover new stars, analyse the old century
playwrights style, or understand the physics of a golf ball. Such fundamental
researchparticipatesintheworld’sknowledgeandbioacoustics.Ecoacousticsareno
exception.Itisessentialto describeandto understandthe patternsandprocessing
that determine natural acoustic environments.To me, it is as important a thing to
know how a pigeon call is produced as it is to know how a financial index is
computed. However, it would be unfair to say that bioacoustics and ecoacoustics
haveno applicationin ourdaily lives. Nexttime youfly, thinkthat the enginesof
the plane you are comfortablyseated in will not fail after sucking up flying birds
owingtotheloudspeakersattheendoftherunwaythatplayspecificalarmsounds
and scare them away. These specific sounds were designed by Thierry Aubin, a
renownedbioacoustician.
Oneofthemostimportantsoundmarkofmycityworkingdayisthesubtlenoise
ofmyfingersonthecomputerkeyboard.AssoonasIhavesettledadministration,
teaching,curation,andsupervisiondutiesIopenRandplaywith.ButwhydoesR
hassuchanimportantroleinmyprofessionallife?
I have been reluctant to programming for ages. As a schoolboy and later as a
university student I always disliked programmingcourses as I was confused with
FOR, WHILE, IF, THEN, DO, and other mysterious instructions. I surely have
always been a software user for work, but I never thought that one day, I would
have written a command rather than just clicked on a mouse. It actually took me
a long time to get into R and, eventually, to love R. I was introduced to R by a
colleagueofmine,MichelBaylac,whoisanexpertinmorphometrics.Thiswasat
amorninglabcoffeebreakandherefollowsthediscussionwehadsometenyears
ago.ConsiderthattheoriginaldialoguewasinFrench:
Jérôme: Michel,howdidyoudoyourellipticFourieranalysisinyourlastpaper?
Icouldnotfindanystatisticalsoftwarethatdoesit.
Michel: IusedR.
Jérôme: Sorry?
Michel: IprogrammedtheEFAwithR.
Jérôme: Air?Idonotknowthissoftware.Whatisit?
Michel: WellitisaprogramminglanguagederivingfromS.
Jérôme: Ace? I do not know that one either. So you can do your own analysis.
Soundsgreat!
Michel: Itis.Anditisfree.Nolicencetopurchase.
A few coffee breaks and some explanations about software names later, I
successfullyinstalledRandgotstartedwithitbutIquicklygaveupsousedIwas
to graphical user interface. Fortunately, Michel gave an R-based statistics Master
course a few weeks later so I went back to school and followed his instructions
to run R multivariateanalysis. ButI had no data to analyse at thattime anda few
monthslaterIhadforgottenalmosteverything.Ayearafter,Ijoinedmyfuturewife
CarolineSimonis,whoisaco-authorofseewave,forasecondsessionofMichel’s
course and I joined her again for a course on linear models with R organized by
Emmanuel Paradis, who wrote the best-seller Rfor beginners and the wonderful
phylogenetics ape package. I was probably more motivated to be with Caroline
thantolearnR,butCarolineunderstoodtheinterestofRmuchfasterthanIdidand
she talkedme intostarting to write morecoderatherthansimplylm(y~x).This
timeIhaddatatolookintoathandandfreetimetoplaywithR.Icanstillremember
that winter evening when I could plot my very first pure-tone spectrogram. The
image was incomplete and inaccurate, but to me it was a wonderful and shiny
plot that motivated me to keep on with R. I was so amazed at being able to run
such an analysis by myself knowing all the production steps perfectly that in the
followingweeksIcouldnotstopwritingbasicsound-analysisdedicatedfunctions.I
wasluckytobegreatlyhelpedbyCarolineandalsobyThierryAubin,mymentorin
bioacousticsandtheauthoroftheSyntanasofwarethatinspiredthemainseewave
functions.
Rdefinitelychangedmyresearch.Iwasnomorelimitedtotheutilitiesprovided
byprohibitiveclosedsourcesoftwaresthatmydepartmentcouldnotafford.Icould
do almost everything by myself: I could draw fieldwork observation maps, read,
analyse,andchangemysoundsamples.Icouldcollectqualitativeandquantitative
data, run batch processes, apply multivariate analysis, plot high-quality graphics,
andeventuallyproduceapapercombiningRandLATEXinanicelayout.Themost
important thing was that I was not only a user but also a designer. I was able to
create,imagine,andsharenewtoolswithothers.
I’mtellingthisverypersonalstoryasIthinkthiscouldbethefutureofanyone
who is a bit afraid of software programming.Learning R is not that difficult with
a little help—which a nice woman like Caroline can do but this is not absolutely
necessary. Just keep in mind that it is worth an effort and that the reward will be
tremendous.
This book was written for students who are interested in bioacoustics and
ecoacoustics, but I really hope that it can help anyone who is willing to dive into
the fantastic area of acoustics and into the endless land of R. A book and a cake
are not that different: they both require time and energy to be made, but they are
consumedapace.Ireallyhopethisbookhasanicetaste,andIwishyouverynice
readingandprogrammingnights!
Paris,France JérômeSueur
Contents
1 Introduction................................................................. 1
1.1 SoundasaScienceMaterial........................................ 1
1.2 Layout............................................................... 2
1.3 ConventionforNotationandCode................................. 5
1.4 BookCompilation .................................................. 6
2 WhatIsSound?............................................................. 7
2.1 ADebateUnderaDangerousTree................................. 7
2.2 SoundasaMechanicalWave....................................... 9
2.2.1 AirParticleMotion........................................ 9
2.2.2 AirPressureVariation..................................... 10
2.2.3 Amplitude ................................................. 12
2.2.4 Phase....................................................... 20
2.2.5 Duration ................................................... 20
2.2.6 Frequency.................................................. 21
2.2.7 WritingSoundwithaSimpleEquation.................. 25
2.2.8 AmplitudeandFrequencyModulations.................. 26
2.3 SoundasaTimeSeries............................................. 29
2.4 SoundasaDigitalObject........................................... 30
2.4.1 Sampling................................................... 30
2.4.2 Quantization............................................... 30
2.4.3 IssuesinSamplingandQuantization..................... 32
2.4.4 FileFormat ................................................ 33
2.5 SoundasaSupportofInformation................................. 34
3 WhatIsR?................................................................... 37
3.1 ABriefIntroductiontoanOceanofTools ........................ 37
3.2 HowtoGetR........................................................ 39
3.3 DoYouSpeakR? ................................................... 40
3.3.1 WhereAmI? .............................................. 40
3.3.2 Objects..................................................... 41
3.3.3 Operators .................................................. 46
3.3.4 Functions .................................................. 47
3.3.5 ControllingFlow .......................................... 50
3.3.6 ManipulatingObjects ..................................... 54
3.3.7 VectorizationandRecycling.............................. 62
3.3.8 HandlingCharacterStrings............................... 64
3.3.9 DrawingaGraphic........................................ 65
3.3.10 Scripting................................................... 73
3.3.11 CallingExternalSoftware ................................ 74
3.4 RandSound......................................................... 75
3.4.1 ToUseorNottoUseRforSoundAnalysis?............ 75
3.4.2 MainPackages ............................................ 76
3.4.3 HowtoInstallseewave................................. 79
4 PlayingwithSound......................................................... 81
4.1 ObjectClasses....................................................... 81
4.1.1 vector,matrix,data.frameClasses............. 81
4.1.2 tsandmtsClasses....................................... 82
4.1.3 audioSampleClassofthePackageaudio .......... 85
4.1.4 soundClassofthePackagephonTools.............. 86
4.1.5 WaveClassofthePackagetuneR...................... 87
4.2 HowtoRead(Load)aSound ...................................... 90
4.2.1 .wavFiles................................................. 90
4.2.2 .mp3Files................................................. 92
4.2.3 From.mp3to.wavFiles................................ 93
4.2.4 .flacFiles............................................... 94
4.2.5 LocalFiles................................................. 94
4.2.6 OnlineFiles................................................ 95
4.2.7 SongMeter©Files ........................................ 99
4.3 HowtoListentoaSound........................................... 100
4.3.1 WiththePackageaudio................................. 101
4.3.2 WiththePackagephonTools .......................... 105
4.3.3 WiththePackagetuneR................................. 105
4.3.4 WiththePackageseewave.............................. 106
4.4 HowtoRecordaSound ............................................ 107
4.5 HowtoWrite(Save)aSound ...................................... 108
4.6 TuningR............................................................. 110
5 DisplayoftheWave ........................................................ 111
5.1 Oscillogram ......................................................... 112
5.1.1 SimpleOscillogram....................................... 112
5.1.2 Axes........................................................ 114
5.1.3 Colors...................................................... 116
5.1.4 DecorationandAnnotation............................... 119
5.1.5 ZoomIn.................................................... 121
5.1.6 ABitofInteractivity...................................... 123
5.1.7 MultipleOscillogram ..................................... 123
5.2 AmplitudeEnvelope................................................ 125
5.2.1 Principle ................................................... 125
5.2.2 InPracticewithseewave ............................... 128
5.2.3 Smoothing................................................. 129
5.2.4 InPracticewithphonTools............................ 136
5.3 CombiningOscillogramandEnvelope ............................ 138
6 Edition ....................................................................... 139
6.1 Resampling.......................................................... 139
6.2 ChannelsManaging................................................. 142
6.3 ManipulatingSoundSections...................................... 146
6.3.1 Extract ..................................................... 146
6.3.2 Delete...................................................... 149
6.3.3 Paste ....................................................... 150
6.3.4 Repeat...................................................... 154
6.3.5 Reverse .................................................... 155
6.4 RemovingandInsertingSilenceSections ......................... 155
6.5 ChangingAmplitude................................................ 159
6.5.1 Offset ...................................................... 159
6.5.2 AmplitudeLevel........................................... 161
6.5.3 Fade-InandFade-Out..................................... 163
7 AmplitudeParametrization ............................................... 167
7.1 LinearRelativeScale ............................................... 167
7.2 LogarithmRelativeScale........................................... 173
7.2.1 Signal-to-NoiseRatio..................................... 173
7.2.2 dBWeightings............................................. 174
7.2.3 dBArithmetic ............................................. 175
7.2.4 SoundAttenuationThroughSpreadingLosses.......... 177
7.3 AbsoluteScale ...................................................... 181
8 Time-AmplitudeParametrization ........................................ 185
8.1 WhatandHowtoMeasure?........................................ 185
8.2 ManualMeasurements.............................................. 186
8.3 AutomaticMeasurements .......................................... 191
8.3.1 TheCicadaCase........................................... 193
8.3.2 TheFrogCase............................................. 198
8.4 AmplitudeModulationAnalysis................................... 205
8.4.1 TheCicadaCase........................................... 205
8.4.2 TheFrogCase............................................. 209
9 IntroductiontoFrequencyAnalysis:TheFourierTransformation... 213
9.1 FromTimetoFrequencyandBack ................................ 213
9.2 FourierSeries ....................................................... 214
9.2.1 Periodicity................................................. 214
9.2.2 TrigonometricFourierSeries............................. 217
9.2.3 CompactFourierSeries................................... 219
9.2.4 ExponentialFourierSeries................................ 222
9.3 FourierTransform................................................... 224
9.4 FrequencyScales.................................................... 229
9.4.1 BarkandMelScales ...................................... 229
9.4.2 MusicalScale.............................................. 231
9.5 AmplitudeScales ................................................... 235
9.6 FourierWindows.................................................... 236
9.7 InverseFourierTransform.......................................... 240
9.8 Cepstrum............................................................ 241
10 Frequency,Quefrency,andPhaseinPractice........................... 247
10.1 FrequencySpectrum................................................ 247
10.1.1 FunctionsofthePackagetuneR ........................ 248
10.1.2 FunctionsofthePackageseewave..................... 249
10.1.3 IdentificationofPeaks .................................... 265
10.1.4 ProfileAnalysis ........................................... 275
10.1.5 SymbolicAnalysis ........................................ 286
10.1.6 Parametrization............................................ 293
10.2 QuefrencyCepstrum................................................ 302
10.3 PhasePortrait........................................................ 303
11 SpectrographicVisualization.............................................. 309
11.1 Short-TimeFourierTransform..................................... 309
11.1.1 Principle ................................................... 309
11.1.2 TheUncertaintyPrinciple................................. 312
11.2 ComputationandDisplayoftheSpectrogram..................... 315
11.3 FunctionofthePackagesignal ................................. 319
11.4 FunctionsofthePackagetuneR.................................. 320
11.5 FunctionofthePackagephonTools............................. 324
11.6 FunctionofthePackagesoundgen.............................. 325
11.7 FunctionsofthePackageseewave............................... 326
11.7.1 2DSpectrogram........................................... 326
11.7.2 ExternalComputingoftheShort-TimeFourier
Transform.................................................. 349
11.7.3 InverseShort-TimeFourierTransform................... 351
11.8 MeasurementsandAnnotationsontheSpectrogram.............. 353
11.8.1 SimpleMeasure ........................................... 353
11.8.2 FancyMeasureandAnnotation........................... 353
11.8.3 AutomaticParametrization ............................... 358
11.9 ComplexDisplayandPrinting ..................................... 362
11.9.1 Multi-SpectrogramGraphic .............................. 362
11.9.2 PrintinginaFile........................................... 364
11.9.3 LongSpectrogramGraphic............................... 365
11.10 DynamicSpectrogram.............................................. 366
11.11 Movie................................................................ 368