Table Of ContentDECISION
NEUROSCIENCE
AN INTEGRATIVE PERSPECTIVE
Edited by
J -C D
EAN LAUDE REHER
L´ T
EON REMBLAY
Institute ofCognitiveScience(CNRS), Lyon, France
AMSTERDAM(cid:129)BOSTON(cid:129)HEIDELBERG(cid:129)LONDON
NEWYORK(cid:129)OXFORD(cid:129)PARIS(cid:129)SANDIEGO
SANFRANCISCO(cid:129)SINGAPORE(cid:129)SYDNEY(cid:129)TOKYO
AcademicPressisanimprintofElsevier
AcademicPressisanimprintofElsevier
125LondonWall,LondonEC2Y5AS,UnitedKingdom
525BStreet,Suite1800,SanDiego,CA92101-4495,UnitedStates
50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom
Copyright©2017ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,
includingphotocopying,recording,oranyinformationstorageandretrievalsystem,withoutpermissioninwritingfrom
thepublisher.Detailsonhowtoseekpermission,furtherinformationaboutthePublisher’spermissionspoliciesandour
arrangementswithorganizationssuchastheCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbe
foundatourwebsite:www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(otherthanas
maybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenour
understanding,changesinresearchmethods,professionalpractices,ormedicaltreatmentmaybecomenecessary.
Practitionersandresearchersmayalwaysrelyontheirownexperienceandknowledgeinevaluatingandusingany
information,methods,compounds,orexperimentsdescribedherein.Inusingsuchinformationormethodstheyshouldbe
mindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliabilityforany
injuryand/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,orfromanyuseor
operationofanymethods,products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-805308-9
ForinformationonallAcademicPresspublications
visitourwebsiteathttps://www.elsevier.com/
Publisher:MaraConner
AcquisitionEditor:AprilFarr
EditorialProjectManager:TimothyBennett
ProductionProjectManager:EdwardTaylor
Designer:MatthewLimbert
TypesetbyTNQBooksandJournals
List of Contributors
B.Ahmed University ofOxford,Oxford,United Kingdom J.-E.Han McGill University,Montreal,QC,Canada
B.W.Balleine UniversityofSydney,Camperdown,NSW, M.J.A.G.Henckens Radboud UniversityMedicalCentre,
Australia Nijmegen,TheNetherlands
S.Ballesta CentreNationaldelaRechercheScientifique, E.J.Hermans RadboudUniversityMedical Centre,
Bron,France;Universite´ Lyon1,Villeurbanne,France;The Nijmegen,TheNetherlands
UniversityofArizona,Tucson,AZ,UnitedStates K.Izuma UniversityofYork,York,UnitedKingdom
S.Bernardi ColumbiaUniversity,NewYork,NewYork, M.Jazayari CentreNational delaRechercheScientifique,
UnitedStates Bron,France;Universite´ Lyon1,Villeurbanne,France
A.Blangero UniversityofOxford,Oxford,UnitedKingdom M.Joe¨ls UniversityMedical CenterUtrecht,Utrecht,
L.A.Bradfield UniversityofSydney,Camperdown,NSW, TheNetherlands
Australia T.Kahnt NorthwesternUniversity FeinbergSchoolof
W.Chaisangmongkon KingMongkut’sUniversityof Medicine,Chicago,IL,UnitedStates
TechnologyThonburi,Bangkok,Thailand;NewYork K.Krug UniversityofOxford,Oxford,UnitedKingdom
University,NewYork,NY,UnitedStates
D.Lee Yale University,NewHaven,CT,UnitedStates
G.Chierchia MaxPlanckInstituteforHumanCognitiveand
A.Lefevre InstitutdesSciencesCognitivesMarcJeannerod,
BrainSciences,Leipzig,Germany
UMR5229,CNRS,Bron,France;Universite´ClaudeBernard
L.Clark UniversityofBritishColumbia,Vancouver,BC, Lyon1,Lyon,France
Canada
R.Ligneul InstituteofCognitiveScience(CNRS),Lyon,
A.Dagher McGillUniversity,Montreal,QC,Canada
France
J.W.Dalley University ofCambridge,Cambridge,United K.Louie NewYorkUniversity,NewYork,NY,UnitedStates
Kingdom
R.B.Mars UniversityofOxford,Oxford,UnitedKingdom
J.A.Diaz UniversityofGlasgow,Glasgow,UnitedKingdom
G.K.Murray University ofCambridge,Cambridge, United
J.-C.Dreher InstituteofCognitiveScience(CNRS),Lyon,
Kingdom
France
S.Neseliler McGill University,Montreal,QC,Canada
J.-R.Duhamel CentreNationaldelaRechercheScientifique,
F.X.Neubert UniversityofOxford,Oxford,UnitedKingdom
Bron,France;Universite´ Lyon1,Villeurbanne,France
M.P.Noonan UniversityofOxford,Oxford,United
N.Eshel HarvardUniversity,Cambridge,MA,UnitedStates
Kingdom
L.K.Fellows McGillUniversity,Montreal,QC, Canada
N.Ortner Medical UniversityofVienna,Vienna,Austria
R.D.Fernald StanfordUniversity,Stanford,CA,United
S.Palminteri UniversityCollegeLondon,London,United
States
Kingdom;EcoleNormaleSupe´rieure,Paris,France
G.Ferna´ndez RadboudUniversity MedicalCentre,
M.Pessiglione Institut duCerveauetdelaMoelle(ICM),
Nijmegen,The Netherlands
InsermU1127,Paris,France;Universite´ PierreetMarie
P.C.Fletcher University ofCambridge,Cambridge,United
Curie(UPMC-Paris6),Paris,France
Kingdom
L.Pezawas MedicalUniversity ofVienna,Vienna, Austria
J.M.Fuster UniversityofCaliforniaLosAngeles,Los
M.G.Philiastides University ofGlasgow,Glasgow,United
Angeles,CA,UnitedStates
Kingdom
S.Gherman University ofGlasgow,Glasgow,United
U.Rabl MedicalUniversity ofVienna,Vienna,Austria
Kingdom
T.W.Robbins UniversityofCambridge,Cambridge,United
P.W.Glimcher NewYorkUniversity,NewYork,NY,United
Kingdom
States
D.W.Grupe UniversityofWisconsineMadison, Madison, C.C.Ruff UniversityofZurich,Zurich, Switzerland
WI,UnitedStates Y.Saga InstituteofCognitiveSciences(CNRS),Lyon,France
S.N.Haber University ofRochesterSchoolofMedicine, J.Sallet University ofOxford,Oxford,UnitedKingdom
Rochester,NY,UnitedStates
xi
xii
LISTOFCONTRIBUTORS
D.Salzman ColumbiaUniversity,NewYork, NewYork, P.N.Tobler University ofZurich, Zurich,Switzerland
UnitedStates;NewYorkStatePsychiatricInstitute,New L.Tremblay InstituteofCognitiveScience(CNRS),Lyon,
York,NewYork,UnitedStates
France
W.Schultz University ofCambridge,Cambridge,United C.Tudor-Sfetea University ofCambridge,Cambridge,
Kingdom UnitedKingdom
H.Seo YaleUniversity,NewHaven,CT,UnitedStates N.Uchida HarvardUniversity,Cambridge,MA,United
T.Singer MaxPlanck InstituteforHumanCognitiveand States
BrainSciences,Leipzig,Germany G.Ugazio UniversityofZurich,Zurich, Switzerland
A.Sirigu InstitutdesSciences CognitivesMarcJeannerod, A.R.Vaidya McGill University,Montreal,QC,Canada
UMR5229,CNRS,Bron,France;Universite´ClaudeBernard
V.Voon UniversityofCambridge,Cambridge,United
Lyon1,Lyon,France
Kingdom;CambridgeshireandPeterboroughNHS
J.Smith UniversityofOxford,Oxford,UnitedKingdom FoundationTrust,Cambridge,UnitedKingdom
A.Soltani DartmouthCollege,Hanover,NH,UnitedStates X.-J.Wang NewYorkUniversity,NewYork,NY,United
C.Summerfield University ofOxford,Oxford,United States;NYUShanghai,Shanghai,China
Kingdom K.Witt ChristianAlbrechtUniversity,Kiel,Germany
J.Tian HarvardUniversity,Cambridge, MA,UnitedStates
Preface
Decision Neuroscience: an Integrative Perspective ad- are forward-looking assessments of the current and
dresses fundamental questions about how the brain futureissuesfacedbyresearchers.Wewerefortunateto
makes perceptual, value-based, and more complex de- assemble an outstanding collection of experts who
cisionsinnonsocialandsocialcontexts.Thisbookpresents addressedvariousaspectsofdecision-makingprocesses.
recent and compelling neuroimaging, electrophysiolog- The book is divided into five parts that address distinct
ical,lesional,andneurocomputationalstudies,incombi- butinterrelatedtopics.
nationwithhormonalandgeneticstudies,thathaveledto
aclearerunderstandingoftheneuralmechanismsbehind
STRUCTURE OF THE BOOK
how the brain makes decisions. The neural mechanisms
underlying decision-making processes are of critical
interest to scientists because of the fundamental role Adecisionneuroscienceperspectiverequiresmultiple
that reward plays in a number of cognitive processes levelsofanalysesspanningneuroimaging,electrophysi-
(such as motivation, action selection, and learning) and ological,behavioral,andpharmacologicaltechniques,in
becausetheyhavetheoreticalandclinicalimplicationsfor combination with molecular and genetic tools. These
understanding dysfunctions of major neurological and approaches have begun to build a mechanistic under-
psychiatricdisorders. standing ofindividual andsocial decision-making.This
Theideaforthisbookgrewupfromoureditionofthe book highlights some of these advancements that have
HandbookofRewardandDecisionMaking(AcademicPress, led to the current understanding of the neuronal mech-
2009). We originally thought to revise and reedit this anisms underlying motivational and decision-making
book, addressing one fundamental question about the processes.
nature of behavior: how does the brain process reward
and makes decisions when facing multiple options? Part I is devoted to animal studies (anatomical,
However, given the developments in this active area of neurophysiological, pharmacological, and optogenetics)
research,wedecidedtofeatureanentirelydifferentbook on rewards/punishments and decision-making. In their
with new contents, covering results on the neural sub- naturalenvironment,animalsfaceamultitudeofstimuli,
strates of rewards and punishments; perceptual, value- veryfewofwhicharelikelytobeusefulaspredictorsof
based, and social decision-making; clinical aspects such reward or punishment. It is thus crucial that the brain
as behavioral addictions; and the roles of genes and learnstopredictrewards,providingacriticalevolutionary
hormones in these various aspects. For example, an advantageforsurvival.Thisfirstpartofthebookoffersa
exciting topic from the field of social neuroscience is to comprehensive view of the specific contributions of
know whether the neural structures engaged with various brain structures as the dopaminergic midbrain
various forms of social interactions are cause or conse- neurons, the amygdala, the ventral striatum, and the
quenceoftheseinteractions(Fernald,Chapter28). prefrontal cortex, including the lateral prefrontal cortex
Amechanisticunderstandingoftheneuralencoding andtheorbitofrontalcortex,tothecomponentprocesses
underlyingdecision-makingprocessesisofgreatinterest underlyingreinforcement-guideddecision-making,such
to a broad readership because of their theoretical and as the representation of instructions, expectations, and
clinical implications. Findings in this research field are outcomes;theupdatingofactionvalues;andtheevalua-
alsoimportanttobasicneuroscientistsinterestedinhow tion process guiding choices between prospective re-
the brain reaches decisions, cognitive psychologists wards.Specialemphasisismadeontheneuroanatomyof
working on decision-making, as well as computational the reward system and the fundamental roles of dopa-
neuroscientists studying probabilistic models of brain minergic neurons and the basal ganglia in learning
functions.Decision-makingcoversawiderangeoftopics stimuluserewardassociations.
and levels of analysis, from molecular mechanisms to Chapter 1 (Haber SN) describes the anatomy and
neural systems dynamics, neurocomputational models, connectivityoftherewardcircuitinnonhumanprimates.
and social system levels. The contributions to this book It describes how corticalebasal ganglia loops are
xiii
xiv
PREFACE
topographically organized and the key areas of conver- encoding and retrieval or incentive value, the matching
gencebetweenfunctionalregions. of that value to specific outcome representations, and
Chapter 2 describes three novel electrophysiological finally the integration of this information for action se-
properties of the classical dopamine reward-prediction lection. It also shows how each of these individual pro-
error (RPE) signal (Schultz W). Studies have identified cesses are integrated within the striatum for successful
three novel properties of the dopamine RPE signal. In goal-directedactionselection.
particular, concerning its roles in making choices, the Chapter7(RobbinsTWandDalleyJW)describesan-
dopamine RPE signal may not only reflect subjective imalmodels(mostlyinrodents)ofimpulsivityandrisky
rewardvalueandformaleconomicutilitybutcouldalso choices.Itreviewstheneuralandneurochemicalbasisof
fit into formal competitive decision models. The RPE various forms of impulsive behavior by distinguishing
signalmaycodethechosenvaluesuitableforupdatingor three main forms of impulsivity: waiting impulsivity,
immediately influencingobjectandactionvalues.Thus, risky choice impulsivity, and stopping impulsivity. It
the dopamine utility prediction error signal bridges the shows that dopamine- and serotonin-dependent func-
gap between animal learning theory and economic de- tionsofthenucleusaccumbensareimplicatedinwaiting
cisiontheory. impulsivity and risky choice impulsivity, as well as
Chapter 3 focuses on the electrophysiological prop- cortical structures projecting to the nucleus accumbens.
erties of another important component of the reward Forstoppingimpulsivity,dopamine-dependentfunctions
system in primates, namely the amygdala (Bernardi S ofthedorsalstriatumareimplicated,aswellascircuitry
and Salzman D). The amygdala contains distinct appe- including the orbitofrontal cortex and dorsal prelimbic
titive and aversive networks of neurons. Processing in cortex.Differencesandcommonalitiesbetweentheforms
thesetwoamygdalarnetworkscanbothregulateandbe of impulsive responding are highlighted. Importantly,
regulatedbydiversecognitiveoperations. various applications to human neuropsychiatric disor-
Chapter 4 extends the concept of appetitive and ders such as drug addiction and attention deficit hyper-
aversive motivational processes to the striatum (Saga Y activitydisorderarealsodiscussed.
andTremblayL).Thischapterdescribeshowtheventral Chapter 8 (Fuster JM) proposes that the neural
striatumandtheventralpallidum,twopartsofthelimbic mechanismsofdecision-makingareunderstandableonly
circuit in the basal ganglia, are involved not only in in the structural and dynamic context of the
appetitive rewarding behavior, as classically believed, perceptioneaction cycle, defined as the biocybernetic
butalsoinnegativemotivationalbehavior.Theseresults processingofinformationthatadaptstheorganismtoits
can be linked with the control of approach/avoidance environment.Itpresentsageneralviewoftheroleofthe
behaviorinanormalcontextandwiththeexpressionof prefrontal cortex in decision-making, in the general
anxiety-related disorders. The disturbance of this framework of the perceptioneaction cycle, including
pathwaymayinducenotonlypsychiatricsymptoms,but prediction, preparation toward decision, execution, and
alsoabnormalvalue-baseddecision-making. feedbackfromdecision.
Chapter5 (TianJ,UchidaN,and EshelN)highlights
new advances in the physiology, function, and circuit PartIIcoversthetopicoftheneuralrepresentationof
mechanism of decision-making, focusing especially on motivation, perceptual decision-making, and value-
theinvolvementofdopamineandstriatalneurons.Using based decision-making in humans, mostly combining
optogenetics in rodents, molecular techniques, and neurocomputationalmodelsandbrainimagingstudies.
genetic techniques, this chapter shows how these tools Chapter 9 (Tobler P and Kahnt T) reviews several
havebeenusedtodissectthecircuitsunderlyingdecision- definitions of value and salience, and describes human
making.Itdescribesexcitingnewavenuestounderstand neuroimaging studies that dissociate these variables.
a circuit, by recording from neurons with knowledge of Value increases with the magnitude and probability of
theircelltypeandpatternsofconnectivity.Furthermore, rewardbutdecreaseswiththemagnitudeandprobabil-
the ability to manipulate the activity of specific neural ity of punishment, whereas salience increases with the
typesprovidesanimportantmeanstotesthypothesesof magnitude and probability of both reward and punish-
circuitfunction. ment.Attheneurallevel,valuesignalsariseinstriatum,
Chapter 6 (Bradfield L and Balleine B) describes the orbitofrontal and ventromedial prefrontal cortex, and
neural bases of the learning and motivational processes superior parietal areas, whereas magnitude-based
controlling goal-directed action. By definition, the per- salience signals arise in the anterior cingulate cortex
formanceofsuchactionrespectsboththecurrentvalueof andtheinferiorparietalcortex.Bycontrast,probability-
its outcome and the extant contingency between that based salience signals have been found in the ventro-
actionanditsoutcome.Thischapteridentifiestheneural medialprefrontalcortex.
circuits mediating distinct processes, including the Chapter 10 (Louie K and Glimcher PW) reviews an
acquisition of action-outcome contingencies, the approach centered on basic computations underlying
xv
PREFACE
neuralvaluecoding.Itproposesthatneuralinformation structures relative to the size of our social network, but
processing in valuation and choice relies on computa- also species differences in prefrontaletemporal brain
tional principles such as contextual modulation and connectivity.Furthermore,thispartofthebookpresents
divisive normalization. Divisive normalization is a neurocomputational approaches starting to provide a
nonlinear gain control algorithm widely observed in mechanistic understanding of social decisions. For
multiple sensory modalities and brain regions. Identifi- example, reinforcement learning models and strategic
cation of these computations sheds light on how the reasoning models can be used when learning social hi-
underlyingneuralcircuitsareorganized,andneuralac- erarchiesorduringsocialinteractions.
tivity dynamics provides a link between biological A social neuroscience understanding requires multi-
mechanismandcomputations. ple approaches, such as electrophysiology and neuro-
Chapter 11 (Philiastides M, Diaz J, and Gherman S) imaging in both monkeys (Chapters 14, 15, 19) and
introduces the general principles guiding perceptual humans(Chapters16,18,20),aswellascausal(Chapter
decision-making. Perceptual decisions occur when 21), neurocomputational (Chapters 17e19), endocrino-
perceptualinputsareintegratedandconvertedtoforma logical,genetics,andclinicalapproaches(PartV).
categoricalchoice.Itreviewstheinfluenceofanumberof Chapter 14 (Duhamel JR and colleagues) presents
factors that interact and contribute to the decision pro- monkey electrophysiological data revealing that the
cess,suchasprestimulusstate,rewardandpunishment, orbitofrontal cortex is tuned to social information. For
speedeaccuracy trade-off, learning and training, confi- example,inoneexperiment,macaquemonkeysworked
dence, and neuromodulation. It shows how these deci- tocollectrewardsforthemselvesandtwomonkeypart-
sion modulators can exert their influence at various ners.Singleneuronsencodedthemeaningofvisualcues
stages of processing, in line with predictions derived that predicted the magnitude of future rewards, the
fromsequential-samplingmodelsofdecision-making. motivational value of rewards obtained in a social
Chapter 12 (Summerfield C) reviews the neural and context, andthetracking ofsocial preferences andpart-
computational mechanisms of perceptual decisions. It ner’s identity and social rank. The orbitofrontal cortex
addresses current controversial questions, such as how thus contains key neuronal mechanisms for the evalua-
we decide when to draw our decisions to a conclusion, tion ofsocial information.Moreover, macaquemonkeys
and how perceptual decisions are biased by prior takeintoaccountthewelfareoftheirpeerswhenmaking
information. behavioralchoicesbringingaboutpleasantorunpleasant
Chapter 13 (Soltani A, Chaisangmongkon W, and outcomestoamonkeypartner.Thus,thischapterreveals
Wang XJ) presents possible biophysical and circuit that prosocial decision-making is sustained by an
mechanisms of valuation and reward-dependent plas- intrinsic motivation for social affiliation and controlled
ticity underlying adaptive choice behavior. It reviews throughpositiveandnegativevicariousreinforcements.
mathematical models of reward-dependent adaptive Chapter15 (Sallet J andcolleagues) reviews the sim-
choice behavior, and proposes a biologically plausible, ilarities between monkeys and humans in the organiza-
reward-modulated Hebbian synaptic plasticity rule. It tion of the social brain. Using MRI-based connectivity
shows that a decision-making neural circuit endowed methods, they compare human and macaque social
with this learning rule is capable of accounting for areas, such as the organization of the medial prefrontal
behavioralandneurophysiologicalobservationsinava- cortex.Theyrevealedthattheconnectivityfingerprintof
rietyofdecision-makingtasks. macaquearea10bestmatchedthatofthehumanfrontal
pole,suggestingthatevenhigh-levelareassharefeatures
PartIIIofthebookfocusesontherapidlydeveloping betweenspecies.Theyalsoshowedthatanimalshoused
field of social neuroscience, integrating neuroscience in large social groups had more gray matter volume in
data from both nonhuman primates and humans. Pri- bilateral mid-superior temporal sulcus and rostral pre-
mates are fundamentally social animals, and they may frontalcortex.Beyondspeciessimilarities,therearealso
share common neural mechanisms in diverse forms of distinct differences between human and macaque
social behavior. Examples of such behavior include prefrontaletemporal brain connectivity. For example,
tracking intentions and beliefs from others, being functionalconnectionsbetweenthetemporalcortexand
observed by others during prosocial decisions, or the lateral prefrontal cortex are stronger in humans
learningthesocialhierarchyinagroupofindividuals.It compared to connections with the medial prefrontal
is also likely that at the macroscopic level, important cortexinhumans,buttheoppositepatternisobservedin
differences exist concerning social brain structures and macaques.
connectivity, and there is a need to directly compare Chapter16(IzumaK)focusesontwoformsofsocial
between species to answer this fundamental question. influence, the audience effect, which is an increased
Indeed,studiesinbothhumansandmonkeysreportnot prosocial tendency in front of other people, and social
onlyanincrease in gray matterdensity ofspecificbrain conformity,whichconsistsinadjustingone’sattitudeor
xvi
PREFACE
behaviortothoseofagroup.ThischapterdiscussesfMRI reward system that link together basic research areas,
findings in healthy humans in these two types of social including systems, cognitive, and clinical neuroscience.
influence and also shows how reputation processing is Dysfunction of the reward system and decision-making
impaired in individuals with autism. It also links social is present in a number of neurological and psychiatric
conformity and reward-based learning (reinforcement disorders, such as Parkinson’s disease, schizophrenia,
learning). drug addiction, and focal brain lesions. The study of
Chapter17(LigneulRandDreherJC)examineshow pathologicalgambling,forexample,andothermotivated
the brain learns social dominance hierarchies. Social states associated with, and leading to, compulsive
dominancereferstorelationshipswhereinthegoalsofone behavior provides an opportunity to learn about the
individualprevailoverthegoalsofanotherindividualina dysfunctions of reward system activity, independent of
systematicmanner.Dominancehierarchieshaveemerged direct pharmacological activation of brain reward cir-
asamajorevolutionaryforcetodrivedyadicasymmetries cuits.Ontheotherhand,becausedrugsofabusedirectly
in a social group. This chapter proposes that the emer- activate brain systems, they provide a unique challenge
gence of dominance relationships are learned incremen- in understanding how pharmacological activation in-
tally,byaccumulatingpositiveandnegativecompetitive fluences reward mechanisms leading to persistent
feedbacksassociatedwithspecificindividualsandother compulsivebehavior.
membersofthesocialgroup.Itconsiderssuchemergence Chapter21(MurrayGK,Tudor-SfeteaC,andFletcher
ofsocialdominanceasareinforcementlearningproblem PC) shows that principles of reinforcement learning are
inspiredbyneurocomputationalapproachestraditionally usefultounderstandtheneuralmechanismsunderlying
applied tononsocialcognition.Thischapter alsoreports impairedlearning,reward,andmotivationalprocessesin
how dominance hierarchies induce changes in specific schizophrenia. Two symptoms characteristic of this dis-
brain systems, and it reviews the literature on interindi- ease is considered in this framework, namely delusions
vidualdifferencesintheappraisalofsocialhierarchies,as andanhedonia.
well as the underlying modulations of cortisol, testos- Chapter22(VaidyaARandFellowsLK)takesaneu-
terone,andserotonin/dopaminesystems,whichmediate ropsychological approach to review focal frontal lobe
thesephenomena. damage effects on value-based decisions. It reveals the
Chapter18(SeoHandLeeD)describesreinforcement necessary contributions of specific subregions (ventro-
learning models and strategic reasoning during social medial, lateral, and dorsomedial prefrontal cortex) to
decision-making. It shows that dynamic changes in decision-making,andprovidesevidenceastothedisso-
choicesanddecision-makingstrategiescanbeaccounted ciability of component processes. It argues that the
forbyreinforcementlearninginavarietyofcontexts.This ventromedialfrontallobeisrequiredforoptimallearning
frameworkhasalsobeensuccessfullyadoptedinalarge fromrewardunderdynamicconditionsandcontributes
number of neurobiological studies to characterize the tospecificaspectsofvalue-baseddecision-making.Italso
functionsofmultiplecorticalareasandbasalganglia.For shows a necessary contribution of the dorsomedial
complex decision-making, including social interactions, frontallobeinrepresentingaction-valueexpectations.
thischaptershowsthatmultiplelearningalgorithmsmay Chapter23(PalminteriSandPessiglioneM)reviews
operateinparallel. reinforcement learning models applied to reward and
Chapter 19 (Ugazio G and Ruff C) reports brain punishment learning. These studies include fMRI and
stimulationstudiesonsocialdecision-making,whichtest neuralperturbationfollowingdrugadministrationand/
the causal relationship between neural activity and or pathological conditions. They propose that distinct
different types of processes underlying these decisions, brain systems are engaged, one in reward learning
including social emotions, social cognition, and social (midbrain dopaminergic nuclei and ventral prefrontos-
behavioralcontrol. triatal circuits) and another in punishment learning,
Chapter20(ChierchiaGandSingerT)showsthattwo revolvingaroundtheanteriorinsula.
important social emotions, empathy and compassion, Chapter 24 (Voon V) discusses decision-making im-
engage distinct neurobiological mechanisms, as well as pairments andimpulsecontrol disordersin Parkinson’s
different affective and motivational states. Empathy for disease. The author reports enhancement of the gain
painengagesanetworkincludingtheanteriorinsulaand associated with levodopa, reinforcing properties of
anterior midcingulate cortex, areas associated with dopaminergic medications, and enhancement of delay
negative affect; compassionate states engage the medial discounting in these patients. Lower striatal dopamine
orbitofrontalcortex and ventral striatum and are associ- transporter levels preceding medication exposure, and
atedwithfeelingsofwarmth,concern,andpositiveaffect. decreased midbrain D2 autoreceptor sensitivity, may
underlieenhancedventralstriataldopaminereleaseand
Part IV of the book focuses on clinical aspects activity in response to salient reward cues, anticipated
involving disorders of decision-making and of the and unexpected rewards, and gambling tasks.
xvii
PREFACE
Impairments in decisional impulsivity (delay discount- these genomic responses prepare the individual to
ing,reflectionimpulsivity,andrisktaking)implicatethe modifyitsbehaviortomoveintoadifferentsocialniche.
ventralstriatum,orbitofrontalcortex,anteriorinsula,and Both social success and failure produce changes in
dorsalcingulate.Thesefindingsprovideinsightintothe neuronal cell size and connectivity in key brain nuclei.
role of dopamine in decision-making processes in This approach bridges the gap between social informa-
addictionandsuggestpotentialtherapeutictargets. tion gathering from the environment and the levels of
Chapter 25 (Witt K) reports that motor control is the cellularandmolecularresponses.
result of a balance between activation and inhibition of Chapter 29 (Rabl U, Ortner N, and Pezawas L) ex-
movement patterns. It points to a central role of the amines the use of imaging genetics to explore the re-
subthalamic nucleus within the indirect basal ganglia lationships between major depressive disorder and
pathway, acting as a brake on the motor system. This decision-making.
subthalamicnucleusfunctionoccurswhenanautomatic Chapters 30e32 report neuroendocrinological find-
response must be suppressed to have more time to ingsinsocialdecision-making,likeningvariationsinthe
choosebetweenalternativeresponses. levelsof differenttypes of hormones (cortisol, oxytocin,
Chapter 26 (Grupe DW) discusses value-based ghrelin/leptin) to brain systems engaged in social de-
decision-making as one of a key behavioral symptoms cisions and food choices. Chapter 30 (Hermans EJ and
present in anxiety disorders. This chapter highlights al- colleagues) integrates knowledge of the effects of stress
terations to specific processes: decision representation, at the neuroendocrine, cellular, brain systems, and
valuation, action selection, outcome evaluation, and behavioral levels to quantify how stress-related neuro-
learning. Distinct anxious phenotypes may be charac- modulatorstriggertime-dependentshiftsinthebalance
terized by differential alterations to these processes and betweentwobrainsystems:a“salience”network,which
theirassociatedneurobiologicalmechanisms. supports rapid but rigid decisions, and an “executive
Chapter 27 (Clark L) presents a conceptualization of control”network,whichsupportsflexible,elaboratede-
disorderedgamblingasabehavioraladdictiondrivenby cisions. This simple model elucidates paradoxical find-
an exaggeration of multiple psychological distortions ingsreportedinhumanstudiesonstressandcognition.
that are characteristic of human decision-making, and Chapter31(LefevreAandSiriguA)reviewsevidence
underpinned by neural circuitry subserving appetitive fora roleforoxytocininindividualandsocialdecision-
behavior, reinforcement learning, and choice selection. making. It discusses animal and human studies to link
Thechapterdiscussestheneurobiological basis of path- the behavioral effects of oxytocin to its underlying
ological gambling behavior in loss aversion, probability neurophysiologicalmechanisms.
weighting, perceptions of randomness, and the illusion Chapter 32 (Dagher A, Neseliler S, and Han JE) ex-
ofcontrol. amines the neurobehavioral factors that determine food
choices and food intake. It reviews findings on the in-
Part V focuses on the roles of hormones and genes teractions between brain systems that mediate feeding
involved in motivation and social decision-making pro- behaviorandthegutandadiposepeptidesthatsignalthe
cesses. The combination of molecular genetic, endocri- currentstateofenergybalance.
nology, and neuroimaging has provided a considerable Chapter33(Dreher,Tremblay,andSchultz)concludes
amount of data that help in the understanding of the this decision neuroscience book by integrating perspec-
biological mechanisms influencing decision processes. tivesfromallcontributors.
These studies have demonstrated that genetic and hor- We anticipate that while some readers may read the
monal variations have an impact on the physiological volume from the first to the last chapter, other readers
response of the decision-making system. These varia- may read only one or more chapters at a time, and not
tions may account for some of the inter- and intra- necessarily in the order presented in the book. This is
individual behavioral differences observed in social why we encouraged an organization of this volume
cognition. whereby each chapter can stand alone, while making
Chapter 28 (Fernald RD) presents an original referencestoothersandminimizingredundanciesacross
approachforcognitiveneuroscientistsbyfocusingonthe the volume. Given the consistent acceleration of ad-
difficult question of how an animal’s behavior or vancesinthevariousapproachesdescribedinthisbook
perceptionofitssocialandphysicalsurroundingsshapes ondecisionneuroscience,youareabouttobedazzledby
its brain. Using a fish model system that depends on a first look at the new stages of an exciting era in brain
complexsocialinteractions,thischapterreportshowthe research.Enjoy!
socialcontextinfluencesthebrainand,inturn,altersthe
Jean-ClaudeDreher
behaviorandneuralcircuitryofanimalsastheyinteract.
Le´onTremblay
Gathering of social information vicariously produces
rapidchangesingeneexpressioninkeybrainnucleiand
C H A P T E R
1
Anatomy and Connectivity of the Reward
Circuit
S.N. Haber
University of RochesterSchoolofMedicine, Rochester, NY, United States
basal ganglia (BG) are traditionally considered to pro-
Abstract
cess information in parallel and segregated functional
Whilecellsinmanybrainregionsareresponsivetoreward,
streams consisting of reward processing, cognition,
thecorticalebasalgangliacircuitisattheheartofthereward
andmotorcontrolareas[1].Moreover,withintheventral
system. The key structures in this network are the anterior
cingulate cortex, the orbital prefrontal cortex, the ventral BG, there are microcircuits thought to be associated
striatum,theventralpallidum,andthemidbraindopamine with various aspects of reward processing. However, a
neurons. In addition, other structures, including the dorsal key component for learning and adaptation of goal-
prefrontalcortex,amygdala,thalamus,andlateralhabenular
directed behaviors is the ability not only to evaluate
nucleus, are key components in regulating the reward cir-
various aspects of reward but also developappropriate
cuit. Connectivity between these areas forms a complex
neural network that is topographically organized, thus action plansand inhibit maladaptivechoiceson theba-
maintaining functional continuity through the corticobasal sisofpreviousexperience.Thisrequiresintegration be-
gangliapathway.However,therewardcircuitdoesnotwork tween various aspects of reward processing as well as
in isolation. The network also contains specific regions in
interaction between reward circuits and brain regions
whichconvergentpathwaysprovideananatomicalsubstrate
involved in cognition. Thus, while parallel processing
forintegrationacrossfunctionaldomains.
providesthroughputchannelsbywhichspecificactions
canbeexpressedwhileothersareinhibited,theBGalso
plays a central role in learning new procedures and as-
INTRODUCTION sociations, implying the necessity for integrative pro-
cessing across circuits. Indeed, we now know that the
The reward circuit is a complex neural network that network contains multiple regions in which integration
underlies the ability to effectively assess the likely out- acrosscircuitsoccurs[2e8].Therefore,whiletheventral
comes of different choices. A key component to good BGnetworkisattheheartofrewardprocessing,itdoes
decision-making and appropriate goal-directed behav- not work in isolation. This chapter addresses not only
iors is the ability to accurately evaluate reward value, the connectivities within this circuit, but also how this
predictability,andrisk.Whilethehypothalamus iscen- circuit anatomically interfaces with other BG circuits.
tral for processing information about basic, or primary, Reward and aversive processes work together in
rewards higher cortical and subcortical forebrain struc- learning and decision-making. Aversive processing
tures are engaged when complex choices about these associated with punishment and negative outcomes is
fundamental needs are required. Moreover, choices addressedby other authors in this book.
ofteninvolvesecondaryrewards,suchasmoney,power, The frontaleBG network, in general, mediates all
challenge, etc., that aremoreabstract (comparedto pri- aspectsofactionplanning,includingrewardandmoti-
mary needs), and not as dependent on direct sensory vation,cognition,andmotorcontrol.However,specific
stimulation. Although cells that respond to various as- regions within this network play a unique role in
pects of reward such as anticipation, value, etc., are various aspects of reward processing and evaluation
found throughout the brain, at the center of this neural of outcomes, including reward value, anticipation,
network is the ventral corticobasal ganglia circuit. The predictability, and risk. The key structures are
DecisionNeuroscience
http://dx.doi.org/10.1016/B978-0-12-805308-9.00001-4 3 Copyright©2017ElsevierInc.Allrightsreserved.
