Microphone Arrays and Sound Field Decomposition for Dynamic Binaural Recording

284 Pages·2016·14.4 MB·English

by  Benjamin Bernschütz

Microphone Arrays and Sound Field Decomposition
for Dynamic Binaural Recording
vorgelegtvon
BenjaminBernschütz,M.Sc.
ausWesel
vonderFakultätI–Geistes-undBildungswissenschaften
derTechnischenUniversitätBerlin
zurErlangungdesakademischenGrades
DoktorderNaturwissenschaften
Dr.rer.nat.
genehmigteDissertation
Promotionsausschuss:
Vorsitzender: Prof. Dr. UlfSchrader
Berichter: Prof. Dr. StefanWeinzierl
Berichter: Prof. Dr. ChristophPörschmann
Berichter: Prof. Dr. SaschaSpors
TagderwissenschaftlichenAussprache: 15.02.2016
Berlin2016

Microphone Arrays and Sound Field Decomposition
for Dynamic Binaural Recording
BenjaminBernschütz
DoctoralDissertation
Researchandwritingfrom2009–2015,Publishedin2016
TechnicalUniversityofBerlin–AudioCommunicationGroup
TechnologyArtsSciencesTHKöln–InstituteofCommunicationsSystems
This dissertation was written at Technical University of Berlin, Faculty I – Audio
CommunicationGroupundersupervisionofProf.Dr.StefanWeinzierl.
Funded by the German Federal Ministry of Education and Research, support codes:
1707X08and17009X11,grantedtoTHKöln,Faculty07–InstituteofCommunications
Systems,Prof.Dr.Pörschmann.
LicensedunderaCreativeCommonsAttributionCC-BY4.0InternationalLicense.
WrittenwithaLATEXtypesettingsystemusingTeXLiveandTexmaker.
ReferencesmanagedwithPapersforMACandBibTeX.
VectordrawingsmadewithTikZandOmniGraffle.
Pixelgraphics/photoseditedwithPixelmator.
DiagramsplottedwithMATLAB⃝R.
CoverpicturetakenbyPhilippStadeatWDRKöln.

Tomybelovedparents
MonikaBernschütz-Hörnchen† andWalterBernschütz†.
f
Acknowledgements
I thank my doctoral adviser Professor Dr. Stefan Weinzierl for his fantastic support,
fruitfuldiscussionsandinvaluableadvises. Iamimpressedbythebreadthofhisknowl-
edge and his extraordinary sense of accuracy. It is a great honor and privilege to be
yourdoctoralstudent.
IthankProfessorDr.ChristophPörschmann,anincrediblevisionaryandrazor-sharp
mind, who made this work possible and had the initial ideas. I had the great chance
and honor to work with him for six years in an outstanding working environment.
Professor Pörschmann has supported this work and myself in so many different ways.
Ithankyouverymuch,onaprofessionalaswellasonapersonallevel.
IthankProfessorDr.SaschaSpors, oneoftheleadingexpertsinspatialaudiosignal
theory, for his steady support and guidance since the time I was writing my master
thesisunderhissupervision. Itisagreathonortohaveyouassupporterandreviewer
formydoctoralthesisnow.
ThisworkwasfundedbytheBundesministeriumfürBildungundForschung(BMBF)
in Germany in the scope of two different research projects. Thanks for the great
supportandtheopportunities.
Many thanks go to (in alphabetic order) Johannes Arend, Antje Goldenberg, Gary
Grutzek, Dominic Hemmer, Ulla Jantzen, Philipp Stade and Arnau Vázquez-Giner,
my dear colleagues and friends at the Institute of Communications Systems at TH
Köln for the incredibly great times we had and for the steady professional exchange,
supportandinspiration.
VeryspecialthanksfordirectcontributionstoArnauVázquez-Ginerwhocollaborated
with me in the research projects. He was responsible for implementing the listening
experimentsoftwareandsettingupthelisteningexperimentsthatappearinthiswork.
VeryspecialthankstoPhilippStadeandMaxRühlwhoperformedtheacousticmea-
surements at the WDR broadcast studios with me at any time of the day. This was
rock’n’roll. Another very special thanks to Johannes Arend for intense support with
thelisteningexperiments.
Whatwouldsciencebewithoutthatterrificscientific-lookinghardware? Specialthanks
fortransformingmycrazyideas,professionallysketchedonawrinkledtissue,intosolid
andfunctionalhardwaretothegreatguysfromthemechanicworkshopoftheInstitute
ofCommunicationsSystemsatTHKöln,AaronFinkenthei,MichaelSöntgenrathand
GerdMies. Yourworkisfantastic.
Thanks to the large amount of people who participated in the listening experiments.
Particularly many thanks to the group of expert listeners who spent a lot of time
vi
and effort. You are all really great. Sorry, I cannot mention all of your names here.
Also many thanks to several bachelor and master students who directly or indirectly
supportedthiswork.
ThankstotheWDRradiobroadcaststudiosinCologneandespeciallyMarkusHaßler
andBenediktBitzenhoferfortheirsteadysupport,andforenablingtheacousticmea-
surements.
Thanks to IOSONO/BARCO Audio Technologies for providing the fantasic binaural
renderer that was used for the listening experiments. Very special thanks to Lutz
Altmann, who modified the rendering software and remote protocol according to our
requirements.
Thanks for many fruitful discussions, scientific exchange, and a lot of joyful evenings
at various conferences and meetings to my dear colleagues from Technical University
of Berlin and University of Rostock (in alphabetic order) Dr. Jens Ahrens, Fabian
Brinkmann,VeraErbes,MatthiasGeier,MichaelHorn,Dr.AlexanderLindau,Frank
Schultz,andDr.HagenWierstorf.
Youcannotwriteadoctoralthesiswithoutmassivesupportfromyourfamily. Ideeply
thank my parents Monika Bernschütz-Hörnchen and Walter Bernschütz, who both
passedawayjustbeforeIdefendedandpublishedmywork. Youdidnotknowit,but
thisworkwasdedicatedtobothofyoufromthebeginning. Attheendyoucouldnot
be here with me to celebrate the graduation. I miss you and thank you so much for
everything.
ManyspecialthanksgotomydearrelativesintheUSA,toDr.HansGeorgRitterat
BerkeleyLab,forproofreadingmywork.
ThankstomysisterDr.RebeccaSchorm-BernschützwithherhusbandDr.Alexander
SchormandmybrotherBastianBernschützfortheirsupportinmanydifferentways. I
alsothankmysecondfamilyUrsulaandPeterSpang,aswellasDr.UlrikeandCarsten
Weinholdfortheirkindsupport.
I deeply thank my partner Dr. Astrid Spang for standing by my side during all this
time,forherlove,patience,andfull-timesupport. ManythankstooursonArminwho
enlightenedmydaysduringthelastfewweeksofthiswork.
BenjaminBernschütz,
December2015
vii
Abstract
Thisthesisdiscussesafield-relatedrecordingtechniquebasedonmicrophonearraysand
orthogonalsoundfielddecompositionthatdeliversasuitabledescriptionfordynamic
binauralreproduction.
Dynamic binaural reproduction refers to a mostly headphone-based reproduction
methodthatallowsforpresentinglocalizablevirtualsourcesandaccountsforthehead
movementsoftherecipientinordertodecouplethemfromthespatialorientationofthe
virtual auditory scene. Increased source localization and externalization stability can
be regarded as primary advantages compared to classic static binaural reproduction.
Spatiallystationaryordynamicvirtualsoundsourcescanbepresentedthatmaintain
theirspatialpositionsormoveinrelationtoafixedexternalworldcoordinatesystem,
whichisindependentfromtherecipient’sheadmovements.
Dynamic binaural reproduction requires either object-based audio production or spe-
cific field-related recording techniques. The focus of this thesis lies on the latter. Us-
ing microphone arrays paired with orthogonal sound field decomposition appears to
be a particularly promising approach for field-related dynamic binaural recording. It
is based on an elegant, closed-form mathematical solution and allows accounting for
head-tracking in all rotational degrees-of-freedom during the playback of a recorded
auditoryscene. Theoretically,eventranslationoftherecipientcanbeconsidered. The
method inherently comprises individualization capabilities by employing individual
head-related transfer functions (HRTFs) and allows for point-to-multipoint distribu-
tion. Duetotheclosemathematicalrelationshipwithhigher-orderAmbisonics(HOA),
therespectiveformatsandcodecscanbeusedforstorageanddistributionoftheaudio
data.
The theoretical mathematical approach under ideal physical conditions is discussed
andaclosed-formsolutionisderived. Duetoconstraintsintechnicalsystems,suchas
discretespatialsamplingornoiseinthesignalpathsforinstance,idealconditionscan-
notbemaintainedinpractice. Themajorconstraintsarepointedoutandtheirspecific
impact is analyzed and assessed. Various approaches for improving the transmission
characteristics of the system are proposed and evaluated. The perceptual properties
underdedicatedtechnicalconstraintsandrealisticconditionsareassessedinlistening
experiments. Optimal technical parameters for the system are also determined. It
turns out that an array-based system for dynamic binaural recording with satisfying
perceptual properties can be realized within reasonable technological and economical
limits.
viii
AbstractinGerman
Die vorliegende Dissertationsschrift behandelt ein feldbezogenes räumliches Audioauf-
nahmeverfahren,dasaufMikrofonarraysundorthogonalerSchallfeldzerlegungberuht
undeinegeeigneteBeschreibungfürdynamischebinauraleWiedergabeliefert.
DynamischebinauraleWiedergabebezeichneteinmeistkopfhörerbasiertesräumliches
Audiowiedergabeverfahren zur Darbietung lokalisierbarer virtueller Schallquellen, das
dieKopfbewegungdesRezipientenberücksichtigt,umsievonderräumlichenOrientie-
rungderwiedergegebenvirtuellenauditorischenSzenezuentkoppeln.Diewesentlichen
VorteilegegenüberstatischerbinauralerWiedergabebestehenineinerverbessertenLo-
kalisationundExternalisierungdervirtuellenQuellen,sowiederMöglichkeit,statische
oder dynamische virtuelle Quellen wiederzugeben, die unabhängig von der Kopfbe-
wegung des Rezipienten ortsfest verbleiben oder sich in Bezug zu einem statischen
weltbezogenenKoordinatensystembewegen.
Dynamische binaurale Wiedergabe setzt entweder objektbasierte Audioproduktion
oderspezifischefeldbezogeneAufnahmeverfahrenvoraus.LetztereliegenimFokusdie-
ser Arbeit. Der Einsatz von Mikrofonarrays in Kombination mit orthogonaler Schall-
feldzerlegungstellthierzueinenvielversprechendenLösungsansatzdar.DasVerfahren
beruht auf einer eleganten mathematisch geschlossenen Lösung. Kopfbewegungen des
Rezipienten können in allen rotatorischen Freiheitsgraden berücksichtigt werden. In
derTheorielassensichauchtranslatorischeFreiheitsgradeeinbeziehen.DurchEinsatz
individuellerkopfbezogenerÜbertragungsfunktionen(HRTFs)kanndieWiedergabein-
dividualisiertwerden.DasVerfahreneignetsichfürPunkt-zu-MehrpunktÜbertragung.
AufgrundengermathematischerVerwandtschaftzumhigher-orderAmbisonics(HOA)
Verfahren,lassensichdiedorteingesetztenFormateundCodecszurSpeicherungund
ÜbertragungderAudiodatennutzen.
UnterAnnahmeidealerphysikalischerBedingungenwirdzunächstdertheoretischeAn-
satz diskutiert und eine mathematisch geschlossene Lösung abgeleitet. Aufgrund ver-
schiedenerEinschränkungenintechnischenSystemen,wiebeispielsweiseraumdiskrete
AbtastungoderRauschenindenSignalwegen,sindinderPraxisallerdingskeineidea-
len Bedingungen erzielbar. Die wichtigsten Einschränkungen werden aufgezeigt und
ihrjeweiligerEinflussaufdasSystemverhaltenuntersucht.VerschiedeneMethodenzur
Verbesserung der Übertragungseigenschaften werden diskutiert. In Hörversuchen wer-
den perzeptive Eigenschaften des Systems im Hinblick auf spezifische technische Ein-
schränkungensowierealistischeBedingungenevaluiert.FernerwerdenoptimaleSystem-
parameterermittelt.Eszeigtsich,dassarraybasierteSystemefürdiefeldbezogenedy-
namische Binauralaufnahme mit guten perzeptiven Eigenschaften unter vertretbarem
technischenundwirtschaftlichenAufwandrealisierbarsind.
ix
x