assessing the mechanisms behind sound-taste correspondences and their impact on multisensory

384 Pages·2017·28.07 MB·English

ASSESSING THE MECHANISMS BEHIND SOUND-TASTE
CORRESPONDENCES AND THEIR IMPACT ON
MULTISENSORY FLAVOUR PERCEPTION AND EVALUATION

QIAN (JANICE) WANG

Jesus College

University of Oxford, Oxford, UK

Supervisor: Professor CHARLES SPENCE

Dissertation submitted for the degree of Doctor of Philosophy in Experimental
Psychology at the University of Oxford, Hilary Term, 2017.
2

“The integration of inputs from different sensory modalities not only transforms some
of their individual characteristics, but does so in ways that can enhance the quality of
life.”
- B. E. Stein & M. A. Meredith, 1993, “The Merging of the Senses”, p. 11.

“No animal can live without food. Let us then pursue the corollary of this: Namely,
food is about the most important influence in determining the organization of the
brain and the behavior that the brain organization dictates.”
- J. Z. Young, 1968, “Influence of the mouth on the evolution of the brain”, p. 21.
3

Declaration

I hereby declare that this work has not been submitted previously as an exercise for a
degree at this or any other university, and that it is entirely my own work.

Qian (Janice) Wang
4
Acknowledgments
I first embarked on this journey five years ago, for the simple reason that I ate at a
very good restaurant named Alinea. It’s been a difficult and often lonely road, but I
wouldn’t have wanted it any other way.
First of all, I want to thank my family for their unquestioning support, love, and
funding, despite my often incoherent explanations of my research.
My deepest gratitude goes to my supervisor, Professor Charles Spence. Thank you for
your infinite patience in tirelessly correcting all my manuscripts (in record time, no
less). Your incredible work ethic, encyclopaedic knowledge, and attention to detail
are truly inspirational, and I hope to be a researcher like you some day.
To my research collaborators from near and far, Klemens Knoeferle, Felipe Reinoso
Carvalho, Bruno Mesz, Maria Zegna, and Steve Keller, thank you for your guidance,
input, and collaboration.
To my colleagues and friends at the Crossmodal Research Laboratory and the
Department of Experimental Psychology, thank you for putting up with me.
To Dem, thank you for your continued support throughout my DPhil.
To my Crossmodal family, thank you for teaching me to play with others.
Finally, I dedicate this thesis to the Oxford University Blind Tasting Society. Two
years of presidency have taught me to appreciate the beauty of wine, and reaffirmed
my passion for this field of research. It has also provided me with an opportunity to
meet incredible, passionate people and make lifelong friends, not to mention giving
me impeccable dishwasher loading skills and a husband in the process.

The road continues.
5
Publications
Wang, Q. (J.), & Spence, C. (2015). Assessing the effect of musical congruency on
wine tasting in a live performance setting. i-Perception, 6:3.
Wang, Q. (J.), & Spence, C. (2015). Assessing the influence of the multisensory
atmosphere on the taste of vodka. Beverages, 1, 204-217.
Wang, Q. (J.), Woods, A., & Spence, C. (2015). “What’s your taste in music?” A
comparison of the effectiveness of various soundscapes in evoking specific tastes.
i-Perception, 6:6.
Spence, C., & Wang, Q. (J.) (2015). Sonic expectations: On the sounds of opening
and pouring. Flavour, 4:35.
Spence, C., & Wang, Q. (J.) (2015). Wine & music (I): On the crossmodal matching
of wine & music. Flavour, 4:34.
Spence, C., & Wang, Q. (J.) (2015). Wine & music (II): Can you taste the music?
Modulating the experience of wine through music and sound. Flavour, 4:33.
Spence, C. & Wang, Q. (J.) (2015). Wine & music (III): So what if music influences
taste? Flavour, 4:36.
Wang, Q. (J.), & Spence, C. (2016). ‘Striking a sour note’: Assessing the influence of
consonant and dissonant music on taste perception. Multisensory Research, 29,
195-208.
Reinoso Carvalho, F., Wang, Q. J., Van Ee, R., & Spence, C. (2016). The influence
of soundscapes on the perception and evaluation of beers. Food Quality and
Preference, 52, 32-41.
Wang, Q. (J.), Wang, S., & Spence, C. (2016). “Turn up the taste”: Assessing the
role of taste intensity and emotion in mediating crossmodal correspondences
between basic tastes and pitch. Chemical Senses, 41, 345-356.
6
Reinoso Carvalho, F., Wang, Q. J., De Causmaecker, B., Steenhaut, K., Van Ee, R.,
& Spence, C. (2016) Tune that beer! Listening for the pitch of beer. Beverages,
2:31.
Reinoso Carvalho, F., Wang, Q. J., Van Ee, R., Persoone, D., & Spence, C. (2017).
“Smooth operator”: Music modulates the perceived creaminess, sweetness, and
bitterness of chocolate. Appetite, 108, 383-390.
Wang, Q. (J.), & Spence, C. (2017). Assessing the role of emotional associations in
mediating crossmodal correspondences between classical music and wine.
Beverages, 3:1.
Wang, Q. (J), Keller, S., & Spence, C. (2017). Sounds spicy: Enhancing the
evaluation of piquancy by means of a customized Crossmodally congruent
soundtrack. Food Quality and Preference, 58, 1-9.
Spence, C., Wang, Q. J., & Youssef, J. (2017). Pairing flavours and the temporal
order of tasting. Flavour, 6:4.
Wang, Q. (J.), Reinoso Carvalho, F., Persoone, D., & Spence, C. (2017). Assessing
the effect of shape on expected and actual chocolate flavor. Flavour, 6:2.
Wang, Q. (J.), & Spence, C. (in press). The role of pitch and tempo in sound-
temperature crossmodal correspondences. Multisensory Research.
Wang, Q. (J.), Knoeferle, K., & Spence, C. (in press). Music to make your mouth
water? Measuring the influence of sour music on salivation. Frontiers in
Psychology: Eating Behavior.

Articles submitted
Wang, Q. (J.), & Spence, C. (submitted). Assessing the influence of music on wine
perception amongst wine professionals. Food Science and Nutrition.
7
Presentations
Wang, Q. (J.) (2015). Multisensory psychology of wine. Masterclass presentation.
Oxford Wine Festival, 22nd August, Oxford, UK.
Wang, Q. (J.) (2016). Exploring music-wine correspondences. Talk given at
Rethinking the Senses Flavour Workshop, 4th April, University of Oxford, UK.
Wang, Q. (J.) & Spence, C. (2016). The role of pitch and tempo in sound-
temperature crossmodal correspondences. Poster presentation. Synaesthesia and
Cross-Modal Perception Conference, 22nd April, Dublin, Ireland.
Wang, Q. (J.) (2016). Multisensory influences on food flavour and liking. Keynote
talk. Sensory Evaluation of Food workshop, 26th July, Fraunhofer IVV, Freising,
Germany.
Wang, Q. (J.), Keller, S., & Spence, C. (2016). “The sound of spice”: Enhancing the
evaluation of piquancy by means of a customised crossmodally congruent
soundtrack. Oral session at EuroSense 2016, 13th September, Dijon, France.
Wang, Q. (J.) (2016). That sounds delicious: How hearing influences flavour
perception in food and drinks. Graduate seminar presentation, 3rd October, School
of Agriculture, University of Buenos Aires, Argentina.
Wang, Q. (J.) (2016). That sounds delicious: How hearing influences flavour
perception in food and drinks. Seminar presentation, 14th December, Institute of
Acoustics, London, UK.
Wang, Q. (J.), Keller, S., & Spence, C. (2017). “The sound of spice”: Enhancing the
evaluation of piquancy by means of a customised crossmodally congruent
soundtrack. Oral session at ESCOM 2017, 31st July - 4th August, Ghent, Belgium.
8
Selected Press coverage
Cahill, H. (2015). I use music to change how food tastes. Financial Times. Accessed
from https://www.ft.com/content/5f109c1a-6d44-11e5-8171-ba1968cf791a.
Escobedo, E. (2014). Sonic Seasoning con Deezer y Elyx. GQ Mexico. Accessed from
http://www.gq.com.mx/lujo-gq-elyx/articulos/maridaje-musical-sonic-seasoning-
con-deezer-elyx-escondite/4286.
Gray, R. (2016). Can music change the taste of chocolate? Take the test to find out
how different sounds trick your tastebuds. The Daily Mail Online. Accessed from
http://www.dailymail.co.uk/sciencetech/article-3956566/Can-music-change-taste-
chocolate-test-different-sounds-trick-tastebuds.html.
Laube, N. (2015). Participar de una “cena emocional” creada por artistas. La Nacion.
Accessed from http://www.lanacion.com.ar/1833260-participar-de-una-cena-
emocional-creada-por-artistas.
Morley, K. (2017). Music makes curries taste 10pc spicier, scientists fine. The
Telegraph. Accessed from http://www.telegraph.co.uk/news/2017/03/12/music-
makes-curries-taste-10pc-spicier-scientists-find/.
Reynaud, S. (2015). Le vin, une question d’oreille. Le Figaro. Accessed from
http://avis-vin.lefigaro.fr/magazine-vin/o118131-le-vin-une-question-d-oreille.
Knapton, S. (2016). Why the sound of New Year bells could make your beer taste
sour. The Telegraph. Accessed from
http://www.telegraph.co.uk/science/2016/12/30/sound-new-year-bells-could-
make-beer-taste-sour/.
9
Note for the reader
15 experiments are presented in this thesis. 14 of which are presented in the main
body of the text (21 studies total), and one further experiment is presented in the
Appendix. Experiment 5 was co-led by Felipe Reinoso Carvalho (Vrije Universiteit
Brussel/KU Leuven, Belgium), where we jointly worked on the experiment design,
data collection, analysis, and writing. The contents of this thesis are, in part, based on
preprints and accepted versions of the articles included in the Publications section, for
which I gives full acknowledgment.
10
Table of Contents

Table of Contents .............................................................................................................. 10
Short Abstract ................................................................................................................... 14
Extended Abstract ............................................................................................................ 15
1 Background and conceptual framework ............................................................... 20
1.1 Crossmodal correspondences: What are they? ......................................................... 21
1.1.1 The role of crossmodal correspondences in the crossmodal binding problem .... 22
1.1.2 Examples of multisensory interactions ............................................................................. 23
1.1.3 Influences of crossmodal correspondences on human information processing and
perception .................................................................................................................................................. 24
1.1.4 Some properties of crossmodal correspondences .......................................................... 27
1.1.5 Theoretical accounts of crossmodal correspondences ................................................. 31
1.1.6 Unimodal correspondences? ................................................................................................ 37
1.1.7 Interim Summary ..................................................................................................................... 38
1.2 Associations between taste/flavour and sound attributes ....................................... 39
1.2.1 Establishing the correspondences between taste/flavour and audition .................. 39
1.2.2 Modifying the tasting experience with sound and music ........................................... 43
1.3 Multisensory influences on flavour perception ......................................................... 50
1.3.1 A unified flavour perception ................................................................................................ 51
1.3.2 Neural substrates underlying flavour ................................................................................. 52
1.3.3 Flavour as a multisensory perceptual system ................................................................. 53
1.4 Possible mechanisms underlying auditory modulation of taste/flavour .............. 55
1.4.1 Sensory expectations ............................................................................................................... 56
1.4.2 Attention ..................................................................................................................................... 57
1.4.3 Neurophysiological response ............................................................................................... 57
1.4.4 Emotion Mediation / Sensation Transfer ......................................................................... 59
1.4.5 Response bias ............................................................................................................................ 59
1.5 Aims and outline of the following chapters ................................................................ 60
2 How are crossmodal correspondences between sound and taste formed? The
emotion mediation hypothesis ........................................................................................ 61
2.1 Introduction ...................................................................................................................... 61
2.1.1 Emotional mediation of crossmodal correspondences ................................................ 61
2.1.2 Measuring emotions ................................................................................................................ 62
2.1.3 Taste and emotion .................................................................................................................... 63
2.1.4 Pitch and intensity .................................................................................................................... 64
2.1.5 Beyond pitch: Comparing taste soundtracks ................................................................... 65
2.2 Experiment 1A: Assessing the role of taste intensity and emotion in mediating
crossmodal correspondences between basic tastes and auditory pitch .......................... 67
2.2.1 Methods ....................................................................................................................................... 67
2.2.2 Results and discussion ............................................................................................................ 70
2.3 Experiment 1B: Assessing the role of taste intensity and emotion in mediating
crossmodal correspondences between basic tastes and auditory pitch .......................... 82
2.3.1 Methods ....................................................................................................................................... 82
2.3.2 Results and discussion ............................................................................................................ 84
2.4 Experiment 2A: A comparison of the effectiveness of various soundscapes in
evoking specific tastes and the role of emotion mediation in soundscape-taste
associations .................................................................................................................................. 94
2.4.1 Methods ....................................................................................................................................... 94
2.4.2 Results and discussion ......................................................................................................... 101