Music, Mathematics and Language: The New Horizon of Computational Musicology Opened by Information Science

264 Pages·2022·8.379 MB·

by  Keiji Hirata, Satoshi Tojo, Masatoshi Hamanaka

Keiji Hirata
Satoshi Tojo
Masatoshi Hamanaka
Music,
Mathematics
and Language
The New Horizon of Computational
Musicology Opened by Information
Science
Music, Mathematics and Language
· ·
Keiji Hirata Satoshi Tojo Masatoshi Hamanaka
Music, Mathematics
and Language
The New Horizon of Computational
Musicology Opened by Information Science
KeijiHirata SatoshiTojo
DepartmentofComplexandIntelligent GraduateSchoolofInformationScience
Systems JapanAdvancedInstituteofScience
FutureUniversityHakodate andTechnology
Hakodate,Hokkaido,Japan Ishikawa,Japan
MasatoshiHamanaka
MusicInformationIntelligenceTeam
RIKENCenterforAdvancedIntelligence
Project
Chuo,Tokyo,Japan
ISBN 978-981-19-5165-7 ISBN 978-981-19-5166-4 (eBook)
https://doi.org/10.1007/978-981-19-5166-4
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNature
SingaporePteLtd.2022
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsofreprinting,reuseofillustrations,
recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor
informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Preface
Wearelivinginaveryrichcultureofmusic.Nowadays,suchgenredistinctionsas
jazz,rock,pop,andclassicalaremeaningless.Fusionhasoccurredin1970s;Miles
Davis introduced electric instruments, as well as church modes used in medieval
times,andhastriedtoopenanewhorizonforthefuturejazz.Evenaprofessional
musiciantrainedinonegenrelookenjoyingothergenres.ToruTakemitsu,oneofthe
best-knowncomposersinexperimentalmusic,issaidtohavehadastrongcompassion
withSergeantPepper’sLonelyHeartsClubBandoftheBeatles.Suchatendencyof
mixturewouldproceedfurtherinfuture.
Atthesametime,theauthenticclassicalmusicseemslosingitspopularity.Those
faithfulguardiansoftraditionmaystillthinkthattheclassicalmusicisabranchof
liberalartsfortheeducatedclass,andthustheymayconsiderthatthefaithfulrepli-
cationofwrittenscoresissignificant.However,archivingtheclassicsandenjoying
music are two different things. Classical music was written when there were no
recordingtechnology.Fromthecurrentviewpoint,therepriseofthesamephrasesis
nomoreneeded,andthuswecouldshortentheirplayingtimetoavoidbeingbored.
In the beginning of twentieth century, many composers tried to break an old
style and to innovate music. One such representative is Arnold Schoenberg who
triedtodestroy thetonalitywhich had governed thetraditionalmusic,introducing
dodecaphony.However,thistrialwasnotsuccessfulatacquiringthepopularity.On
thecontrary,so-calledpopularmusicseemsthemostfaithfulsuccessoroftherather
oldstyleoftonalmusic,consistingofamelodyaccompaniedbyharmonicchords,
andthestylehasfunctionedtoevokeouremotions.
Still,wewantnewmusicpieces.Thereappearnewpiecesonthewebeveryday,
and we try to catch them by digital methods like Apple Music, Amazon Prime,
YouTube, and so on. We say a new hit song is new; surely, there may be a new
constituentinachord,ormaybeanewprogressionofchordswhichhasnotbeen
heardbefore,andsuchaninventionwouldappearfurthermoreaslongasthereare
anexplosivenumberoftheircombinations.However,pleasedonotforgetthatthey
are still written in do-re-mi-fa-sol-la-ti-do, with chords of do-mi-sol or sol-ti-re.
Whatarealternatives?Thetemperamentofscaleshasalonghistoryinselectinga
diatonic set of notes and in fixing the pitches of these notes in an octave. We feel
v
vi Preface
harmonizedwhentheratiooftwofrequenciesisasimplerationalnumber,butanytwo
notesarenotrigidlyharmonicinthecurrentlyprevalent12-toneequaltemperament,
whichisaproductofmanycompromises.Yet,weappearsufficientlyadaptedtothis
temperament.
Do we need to abandon the temperament for exploring the new possibilities in
music? Then, where is music headed? Who wants the change? Or, will songs are
createdcontinuouslyinthesameformalism?Willmusicforeverprogressandevolve,
orwillitsaturateatsomepoint?Ifmusicisgoingtosaturatesomewhere,istherea
limittohumancreativity?Whatismusicinthefirstplace?Composers,performers,
andmusicologistsofthepastmusthavebeenraisingthesamequestionsandinves-
tigating music. They must have taken different approaches and set different goals,
wonderingwhattechniques,skills,andknowledgetheywouldneedtoacquireinorder
tocreatenewmusic.Theresultsincludeactualmusicalmasterpieces,performances,
andmusictheories.
Inthelasthalf-century,weobtainedanapparatustomirrorourcognitiveprocesses
andintelligentfunctions—acomputer.Nowadays,weareabletosimulatetosome
extentourbrainfunctionsandintelligentactivitiesinsidethecomputersinisolated
environments from the real world, and to even conduct as many experiments that
would not be allowed in reality as required. We call a virtual person who owns
itsindependentandsubjectivecognitionanagent.Thenotionofcognitionmaybe
different among various psychologists; some requires a perfect ego, that is, being
conscious of ‘self,’ and others are rather tolerant of the definition and may call
a robot with a simple inference mechanism with independent knowledge base an
agent. Naturally, implementing a cognitive and emotional ego agent in computer
with a rigorous description in formal language is difficult. In the early days of an
electroniccomputer,suchanoptimisticpredictionthathumanintelligencecouldbe
realizedinacomputerwithin10or20yearswasprevalent.Itwasin1948whenthe
precursoroftheideafortheTuringtestwaspublished.Sincethen,therehavebeen
severalcyclesofenthusiasmanddisappointmentalongwithexcessiveexpectations
andtirelesstrialsanderrors.WearenowinthemidstofanotherAIboom,ineffect
a deep learning boom; these ebbs and flows can be also regarded as the history of
developmenttodescribetheintelligentactivitiesandfunctions.
If music is one of the intellectual activities of human beings, we believe that
tackling the seemingly unanswered question “what is intelligence in music?” or
“what is beauty in music?” using the powerful tool of a computer can provide us
withsomeusefulnovelinsights.Itwouldbemeaningfultoconstructacomputational
musictheorybasedontraditionalmusictheory,justascomputationallinguisticshas
beenconstructedandhasproducedvaluableresultsbasedontraditionallinguistics.
Oneofthepurposesofwritingthisbookwastodemonstrateanapproachtorealizing
thiscomputationalmusictheory.
Now,tofindthekeytorealizingcomputationalmusictheory,letusgobackand
thinkabouttheclaimthattheoriginofmusicandlanguageisoneandthesame.As
wecannotobtainfossilsofoldlanguages,wecannotprovethisintheevolutionary
linguistics. However, observing various facts, we can convince ourselves that the
claimseemssomewhatplausible.Apartfromsuchlinguisticaspect,thelanguageis
Preface vii
themostusefulmethodtoexternalizethenotionofbeauty.Toexpressthebeautyin
language means that we symbolize and find structure of music as language. Jean-
JacquesNattiez,followinghismasterJeanMolino,consideredthatmusiccouldbe
structured and analyzed with regard to semiology, and thus the meaning of music
could be externalized. However, in Nattiez’s book, he confessed his own remark
includedacontradiction1.Inhisinterviewwithvirtualhimself,hehonestlyadmitted
thatitwasimpossibletofullyexpressthemusicbeautyinsymbols.AsNattiezfinally
claimedthatthereshouldbeanadequatetheoryoflinguisticstoconstructameaning
ofmusic,webelievethatclarifyingthegrammarstructureofmusicisanessential
andmoststeadywaytoacomputationalmusictheory.
Inthisbook,wefirstconsiderthemeaningofmusic,regardingcognitivereality.
Tothispurpose,weintroducethetheoryofHeinrichSchenker2,whereheclaimed
that there was a fundamental skeleton common in every music piece, and we can
obtain this via the procedure called reduction and how these skeleton worked on
our cognition. Next, we consider why the 12-tone equal temperament came to be
prevalent in Chap. 2 titled ‘The mathematics of ebony and ivory keys.’ The equal
temperament,aswehavetold,isaresultofcompromises;however,wecaneasilyget
accustomedtoitandcanenjoymusicwithit.Thischapterfunctionsasanintroduction
tomathematicaldefinitionofharmonythatconcernstheratiosoffrequencyintonic
waves.Chapter3‘Musicandlanguage’explainsthefundamentalsofgrammartheory
aswellasthecompositionalityprinciple,thatis,thesemanticsofasentencecanbe
composedinparalleltosyntacticstructure.Wewillcontendthatmusicalsopossesses
thesamecontext-freegrammarasnaturallanguagethoughsomewhatinaweaksense.
In Chap. 4, we explain the most prevalent score notation called Berklee method,
which was initiated from Berklee School of Music in New York. Nowadays, we
mostlyemploychordnamesaccordingtothemethodinpopularmusicandconsider
howefficientthismethodis.InChaps.5and6,wegivetwotheoriesoncomputational
musicology.WefirstintroduceImplication-Realizationmodel(IRmodel),thatisa
theorytoreflectour human intuitiononmelody progression.Next,weexplain the
GenerativeTheoryofTonalMusic(GTTM)withitscomplementarytheoryofTonal
PitchSpace(TPS).Chapters7,8,and9aredevotedtotheapplicationsofGTTM.In
Chap.7,wediscussalgebraicoperationsontrees.InChap.8,wereportoureffortsto
implementGTTManalyzerwhichretrievesatreefromagivenpiece.InChap.9,we
illustrateourworkonmusicmorph-ing,togenerateanewmusicpiecefromgiven
two other pieces. Finally, Epilogue, we discuss the future of music with regard to
computationandAI,posingHilbert-stylequestions.
Thisbookaimsatageneralreaderofmusic,inadditiontoprofessionalscientists
of music information processing. Therefore, we try to make each chapter be as
1Jean-Jacques Nattiez. “Je vous prends en flagrant délit de contradiction!” in La Musique, La
Recherche,etLaVie,LeméacEditeurInc.,Montréal,(1999/2004).
2HeinrichSchenker,DerFreieSatz.NeuemusikalischeTheorienundPhantasien,Margada,Liège,
Belgium(1935).
viii Preface
independentofotherchaptersaspossible.Thus,wewouldliketoshareacommon
interestindiscussing‘musicologyincomputerage’inbroadercommunity.
Hakodate,Japan KeijiHirata
Ishikawa,Japan SatoshiTojo
Tokyo,Japan MasatoshiHamanaka
March2022
Acknowledgements
Asauthors,wewouldliketothankeveryonewhohasbeeninvolvedinourresearch;
research collaborations, discussions and chats at conferences, laboratory visits,
hostingresidencies,emailexchanges,events,andsoonhaveallcontributedtothe
resultsofourresearch.
We would like to thank Tory Honor for English rewriting, corrections, and
translation.
WewouldliketothankMioSuginoofSpringer.Infact,wewouldnothavebeen
able to publish this book without the patient but kind encouragement and careful
supportofMioSuginothroughoutthisproject.
MuchoftheworkpresentedinthisbookwassupportedbyJSPSKAKENHIGrant
Numbers20300035,23500145,25330434,26280089,16H01744,and21H03572.
ix
Contents
1 MachinethatComputestheMeaningofMusic .................... 1
1.1 MusicTheorythatEnjoystheBenefitsofComputers ........... 1
1.2 AShortHistoryofProgramMusic ........................... 3
1.3 UnderstandingMusicfromaGestaltPointofView ............. 5
1.4 AttributionofMeaningtoMelodybySemiotics ............... 8
1.5 CategorizationofMusicalMeaning .......................... 11
1.6 ReductionHypothesisandSchenkerianAnalysis ............... 13
1.7 AnalysisofRetrograde ..................................... 15
1.8 UnderstandingMusicthatImitatesUnderstandingLanguage ..... 19
1.9 CognitiveReality .......................................... 23
References ..................................................... 25
2 TheMathematicsofEbonyandIvoryKeys ....................... 27
2.1 Scalewith2and3 ......................................... 27
2.2 RevisedPythagorasScale ................................... 31
2.3 JustIntonation—Introductionof5 ........................... 33
2.4 Eitz’sNotation ............................................ 34
2.5 WhatIsOvertone? ......................................... 36
2.6 Meantone ................................................ 37
2.7 IssueonModulation ....................................... 38
2.8 EqualTemperament ....................................... 39
2.9 Scale,Mode,andKey ...................................... 43
2.10 Historyof12-ToneEqualTemperament ...................... 44
2.11 Symmetry ................................................ 46
References ..................................................... 49
3 MusicasFormalLanguage ...................................... 51
3.1 ChomskyandGenerativeGrammar .......................... 52
3.2 FormalLanguageandAutomata ............................. 55
3.3 HumanLanguageintheHierarchy ........................... 61
3.4 LanguageClassofChordProgression ........................ 63
3.4.1 Context-FreeSyntaxofCadence ...................... 63
xi