music complexity: a multi-faceted description of audio content

123 Pages·2006·1.74 MB·English

MUSICCOMPLEXITY:AMULTI-FACETEDDESCRIPTIONOFAUDIOCONTENT
ADISSERTATIONSUBMITTEDTOTHEDEPARTMENTOFTECHNOLOGYOFTHE
UNIVERSITATPOMPEUFABRAFORTHEPROGRAMINCOMPUTERSCIENCEANDDIGITAL
COMMUNICATIONINPARTIALFULFILMENTOFTHEREQUIREMENTSFORTHEDEGREEOF
–
DOCTORPERLAUNIVERSITATPOMPEUFABRA
SebastianStreich
2006
©CopyrightbySebastianStreich2006
AllRightsReserved
ii
DOCTORALDISSERTATIONDIRECTION
Dr. XavierSerra
DepartmentofTechnology
UniversitatPompeuFabra,Barcelona
ThisresearchwasperformedattheMusicTechnologyGroupoftheUniversitatPompeuFabrainBarcelona,
Spain.PrimarysupportwasprovidedbytheEUprojectsFP6-507142SIMAChttp://www.semanticaudio.org.
iii
iv
Abstract
Thecomplexityofmusicisoneofthelessintensivelyresearchedareasinmusicinformationretrievalsofar.
Althoughveryinterestingfindingshavebeenreportedovertheyears,thereisalackofaunifiedapproachto
thematter. Relevantpublicationsmostlyconcentrateonsingleaspectsonlyandarescatteredacrossdifferent
disciplines. Especiallyanautomatedestimationbasedontheaudiomaterialitselfhashardlybeenaddressed
in the past. However, it is not only an interesting and challenging topic, it also allows for very practical
applications.
Themotivationforthepresentedresearchliesintheenhancementofhumaninteractionwithdigitalmusic
collections. Aswewilldiscuss,thereisavarietyoftaskstobeconsidered,suchascollectionvisualization,
play-list generation, or the automatic recommendation of music. While this thesis doesn’t deal with any
of these problems in deep detail it aims to provide a useful contribution to their solution in form of a set
ofmusiccomplexitydescriptors. Therelevanceofmusiccomplexityinthiscontextwillbeemphasizedby
an extensive review of studies and scientific publications from related disciplines, like music psychology,
musicology,informationtheory,ormusicinformationretrieval.
This thesis proposes a set of algorithms that can be used to compute estimates of music complexity
facetsfrommusicalaudiosignals. Theyfocusonaspectsofacoustics, rhythm, timbre, andtonality. Music
complexity is thereby considered on the coarse level of common agreement among human listeners. The
target is to obtain complexity judgements through automatic computation that resemble a na¨ıve listener’s
point of view. Expert knowledge of specialists in particular musical domains is therefore out of the scope
of the proposed algorithms. While it is not claimed that this set of algorithms is complete or final, we will
see a selection of evaluations that gives evidence to the usefulness and relevance of the proposed methods
of computation. We will finally also take a look at possible future extensions and continuations for further
improvement,liketheconsiderationofcomplexityonthelevelofmusicalstructure.
v
vi
Acknowledgments
DuringmyworkonthisthesisIlearnedmanythingsandthatisnotonlyintheprofessional,butalsointhe
personalsense. DuringmystayattheMusicTechnologyGroupIhadthelucktoexperienceauniqueclimate
ofencouragement,support,andopencommunicationthatwasessentialforbeingabletoproceedsofastand
smoothlywithmyresearchactivities. Itgoeswithoutsayingthatsuchaclimatedoesn’tcomebyitself,but
isduetoeveryindividual’sefforttocreateandmaintainthisfruitfulenvironment. Ithereforewanttoexpress
mygratitudetothewholeMTGasthegreatteamithasbeenformeduringmytimeinBarcelona.
In particular and at the first place I would like to thank Dr. Xavier Serra, my supervisor, for giving me
theopportunitytoworkonthisveryinterestingtopicasamemberoftheMTG.Also,andspecially,Iwantto
thankPerfectoHerreraforprovidingcountlesssuggestionsandconstantsupportformyworkinthisresearch
project. WithouthimIcertainlywouldnotbeatthispointnow.
Further thanks go to my colleagues from Office 316, Bee Suan Ong, Emilia Go´mez, and Enric Guaus
formanyfruitfuldiscussionsandimportantfeedback. Ialsowanttothankalltheotherpeople–directlyor
indirectly–involvedinthesuccessoftheSIMACresearchprojectfortheirappreciatedcooperation.
This research was funded by a scholarship from Universitat Pompeu Fabra and by the EU-FP6-IST-
507142projectSIMAC.Iamthankfulforthissupport,whichwasacrucialeconomicbasisformyresearch.
On the personal side I also want to thank my wife and my family, who provided me with the support,
encouragement,andsometimesalsowiththenecessarydistractionthathelpedmetokeepgoing.
Finally,aspecialthankyounotehastobeincludedformy“sisterinlaw”forhelpingoutwiththeprinting
andeverythingelse. Ioweyouacoffee. ;-)
Hamamatsu,Japan SebastianStreich
November16,2006
vii
viii
List of Figures
1.1 Illustrationofthreedifferentviewsondigitalmusic. . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Screenshotfromamusicbrowserinterfacedisplayingpartofasongcollectionorganizedby
danceabilityanddynamiccomplexity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 TheWundtcurvefortherelationbetweenmusiccomplexityandpreference. . . . . . . . . . 9
2.1 Completeorder,chaos,andcompletedisorder[fromEdmonds(1995)].. . . . . . . . . . . . 15
2.2 Possibleinclusions[fromEdmonds(1995)]. . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 ExampleformodelselectionwithMinimumDescriptionLength[fromGru¨nwald(2005)]. . 23
3.1 IllustrationoftheImplication-RealizationProcess[fromSchellenbergetal.(2002)].. . . . . 30
3.2 Treeobtainedwithalgorithmicclusteringof12pianopieces[fromCilibrasietal.(2004)]. . 36
3.3 PlotsfromVossandClarke(1975). Left: log (loudnessfluctuation)againstlog (f)fora)
10 10
ScottJoplinpianorags,b)Classicalradiostation,c)Rockstation,d)Newsandtalkstation;
Right: log (pitchfluctuation) against log (f) for a) Classical radio station, b) Jazz and
10 10
Bluesstation,c)Rockstation,d)Newsandtalkstation. . . . . . . . . . . . . . . . . . . . . 41
3.4 AverageDFAexponentαfordifferentmusicgenres[fromJenningsetal.(2004)] . . . . . . 43
4.1 Instantaneousloudnessasineq. 4.3(dots),globalloudnessasineq. 4.4(greysolidline),and
averagedistancemarginasineq. 4.5(dashedlines)forfourexampletracks. . . . . . . . . . 49
4.2 Frequencyweightingcurvesfortheouterear. . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.3 Instantaneoustotalloudnessvalues(eq. 4.17)onlogarithmicscaleforthesamefourtracks
showninfigure4.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.4 Snapshots of energy distributions on 180°horizontal plane for two example tracks (track1=
modernelectronicmusic,track2=historicjazzrecording). . . . . . . . . . . . . . . . . . . 57
4.5 SchemafortimbrecomplexityestimationwithunsupervisedHMMtraining . . . . . . . . . 60
4.6 Timbre symbol sequences for excerpts of a) Baroque cembalo music, b) a modern Musical
songwithorchestra. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.7 Blockdiagramoftimbralcomplexitycomputationbasedonspectralenvelopematching. . . 65
ix
4.8 Bandloudnesssimilarities(eq.4.28)forthreemusicexcerpts: a)bagpipeensemble,b)clas-
sicalsymphony, c)rap(thethicklineatthebottommarksthesectionsthatareidentifiedas
voicecontentandthereforeleftoutinthecomplexitycomputation). Thetimbralcomplexity
iscalculatedasthepercentageofframeswherethesimilarityisequaltozero. . . . . . . . . 66
4.9 Distance measures for tonal complexity computation: a) inverted correlation D[i], b) city
1
blockdistanceD[i],c)linearcombination,d)correspondingHPCPdata(beforemovingav-
2
erage). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.10 Excerptsfromthetimeseriess(n)forthreeexamplepiecesfromdifferentmusicalgenres. . 72
4.11 Doublelogarithmicplotsofmeanresidualovertimescales. . . . . . . . . . . . . . . . . . . 73
4.12 DFAexponentfunctionsforthethreeexampletracksfromfigure4.10. . . . . . . . . . . . . 74
5.1 Screenshotfromthesecondpartofthewebsurvey.. . . . . . . . . . . . . . . . . . . . . . . 80
5.2 Averagepercentageofcorrectpredictionswithsubjectmodelsforthebinarycomplexity(top)
andlikingratings(bottom)accordingtodifferentdemographicgroups. . . . . . . . . . . . . 86
5.3 Top-eightlabelswiththehighestnumberofassignedartists. . . . . . . . . . . . . . . . . . 88
5.4 α-levelsfor60techno(o)and60filmscoretracks(x),unordered. . . . . . . . . . . . . . . 89
5.5 Distributionsondecilesforthetwelvelabelswithmostsignificantdeviationfromequaldis-
tribution(solidhorizontallines). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
x