Multiphasic Temporal Dynamics in Responses of Midbrain Dopamine Neurons to Appetitive and ...

16 Pages·2013·1.24 MB·English

4710•TheJournalofNeuroscience,March13,2013•33(11):4710–4725
Systems/Circuits
Multiphasic Temporal Dynamics in Responses of Midbrain
Dopamine Neurons to Appetitive and Aversive Stimuli
ChristopherD.Fiorillo,1,2MinryungR.Song,1andSoraR.Yun1
1DepartmentofBioandBrainEngineering,KAIST(KoreaAdvancedInstituteofScienceandTechnology),Daejeon305-701,RepublicofKorea,and
2DepartmentofNeurobiology,StanfordUniversity,Stanford,California94305
Thetransientresponseofdopamineneuronshasbeendescribedasrewardpredictionerror(RPE),withactivationorsuppressionby
eventsthatarebetterorworsethanexpected,respectively.However,atleastaminorityofneuronsareactivatedbyaversiveorhigh-
intensitystimuli,castingdoubtonthegeneralityofRPEindescribingthedopaminesignal.Toovercomelimitationsofpreviousstudies,
westudiedneuronalresponsestoawidervarietyofhigh-intensityandaversivestimuli,andwequantifiedandcontrolledaversiveness
throughachoicetaskinwhichmacaquessacrificedjuicetoavoidaversivestimuli.WhereasmostpreviousworkhasportrayedtheRPEas
a single impulse or “phase,” here we demonstrate its multiphasic temporal dynamics. Aversive or high-intensity stimuli evoked a
triphasicsequenceofactivation-suppression-activationextendingoveraperiodof40–700ms.Theinitialactivationatshortlatencies
(40–120ms)reflectedsensoryintensity.Theinfluenceofmotivationalvaluebecamedominantbetween150and250ms,withactivation
inthecaseofappetitivestimuli,andsuppressioninthecaseofaversiveandneutralstimuli.Thepreviouslyunreportedlateactiva-
tionappearedtobeamodest“rebound”afterstrongsuppression.Similarly,strongactivationbyrewardwasoftenfollowedbysuppres-
sion.Wesuggestthatthese“rebounds”mayresultfromovercompensationbyhomeostaticmechanismsinsomecells.Ourresultsare
consistentwitharealisticRPE,whichevolvesovertimethroughadynamicbalanceofexcitationandinhibition.
Introduction dopamine neurons at all times (reviewed by Redgrave, 1999;
Thefiringofdopamineneuronsistypicallymodulatedfor(cid:1)0.5s Horvitz,2000;Bromberg-Martinetal.,2010).
after sudden stimulus onset. This entire period has often been Most previous studies tested only a single type of aversive
viewedasasingle“phasic”responsesignalingrewardprediction stimulus, and little was done to demonstrate that neuronal re-
error(RPE).AccordingtoasimpleinterpretationofRPE,firing sponseswererelatedtoaversivenessasopposedtootheraspects
rateshouldbesuppressedbyaversivestimuliandunchangedby ofthestimulus,suchassensoryintensity(e.g.,soundintensity,
neutralstimuli.Contrarytothisnotion,ithasbeenfoundthat visual contrast). Of those studies that provided evidence that
dopamineneuronscanbeactivatedbysensorystimulithatlack stimuliwereaversive,mostreliedonconditionedavoidancere-
obviousmotivationalvalue(StreckerandJacobs,1985;Horvitzet sponses,suchaseyeblink(GuarraciandKapp,1999;Joshuaetal.,
al.,1997;Schultz,1998;Horvitz,2000;Dommettetal.,2005)and 2008;MatsumotoandHikosaka,2009).Avoidancedoesnoten-
thataminorityofdopamineneuronscanbeactivatedbyaversive surenetaversiveness,especiallyifstimulicanbeeasilyavoidedat
stimuli(SchultzandRomo,1987;MirenowiczandSchultz,1996; lowcost(forexample,blinkinginresponsetoacoolbreezeona
GuarraciandKapp,1999;Coizetetal.,2006;Joshuaetal.,2008; hotday).Ifstimuliwereaversive,itisnotknownhowaversive
Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; theywererelativetotherewardvaluesthathavebeencarefully
MileykovskiyandMorales,2011;WangandTsien,2011;Budygin studiedwithrespecttodopamineneurons.Thelatterissueisof
etal.,2012;Cohenetal.,2012).Theseobservationshavecaused particular importance if the aversive stimuli are presented in a
doubt that RPE is an adequate description of the output of all context that includes reward stimuli because the responses of
dopamine neurons are strongly dependent on reward context
(Tobler et al., 2005). A context that causes the expectation of
ReceivedAug.14,2012;revisedJan.9,2013;acceptedJan.15,2013.
relativelylargerewardsmayrenderdopamineneuronsvirtually
Authorcontributions:C.D.F.designedresearch;C.D.F.performedresearch;C.D.F.,M.R.S.,andS.R.Y.analyzed
unabletodistinguishneutralfromweaklyaversiveorappetitive
data;C.D.F.wrotethepaper.
ThisworkwassupportedbytheWorldClassUniversityprogramthroughtheNationalResearchFoundationof stimuli.
Korea,fundedbytheMinistryofEducation,ScienceandTechnologyGrantR32-2008-000-10218-0toC.D.F.anda Toaddresstheseissues,wehavequantifiedaversivenessina
granttoWilliamT.NewsomefromtheHowardHughesMedicalInstitute.WethankBillNewsomeforadviceand
choicetaskbymeasuringhowmuchjuicemonkeyswouldsacri-
helpfuldiscussions,JulianBrownforexcellenttechnicalassistance,andMehwishBhattiforassistanceinconstruct-
ficetoavoidaversivestimuli.Wethenstudiedneuralresponsesto
ingsomefigures.
Theauthorsdeclarenocompetingfinancialinterests. stimulihavingaversivenessapproximatelyequalandoppositeto
CorrespondenceshouldbeaddressedtoDr.ChristopherD.Fiorillo,DepartmentofBioandBrainEngineering, juicereward.Stimuliincludedaversiveair,saline,andbitterso-
KoreaAdvancedInstituteofScienceandTechnology,291Daehak-ro,Building611,Yuseong-gu,Daejeon305-701,
lution,neutralloudsound,appetitivejuice,juiceomission,and
RepublicofKorea.E-mail:[email protected].
conditionedstimulithatpredictedthese.Herewefocusonthe
DOI:10.1523/JNEUROSCI.3883-12.2013
Copyright©2013theauthors 0270-6474/13/334710-16$15.00/0 temporaldynamicsofneuronalresponses,anissuethathasre-
Fiorilloetal.•MultiphasicResponsesofDopamineNeurons J.Neurosci.,March13,2013•33(11):4710–4725•4711
ceivedrelativelylittleattention(butseeNomotoetal.,2010).In tankandsolenoidvalvewerelocatedinanadjacentroom.Airwasdeliv-
anaccompanyingarticlebasedonthesamesetofdata,weaddress eredfromatubehavinganinnerdiameterandlengthof(cid:1)3mmand3m,
theissueofresponsediversityacrossdopamineneurons(Fiorillo respectively.Airwasdirected,paralleltotheground,attheleftnostril
et al., 2013). Our results are consistent with a realistic RPE in fromtheleftside,exitingthetubeatadistanceof(cid:1)1cmfromthenostril.
DuringallneuronalrecordingsandinFigure1,theairpressure(mea-
which stimulus identity is not known with certainty at short
suredatthegastank)was35and20psiinMonkeysOandF,andthe
latencies.
solenoidvalvewasopenfor200ms.Thesoundoftheairexitingthetube
hadanintensityof82dBatthelocationofthemonkey’shead.
MaterialsandMethods Choicetasks.Choicetaskswereusedtoquantifytheaversivenessof
Animals.Tworhesusmacaques(Macacamulatta)werestudied,onefe- stimuli,althoughotherbehaviorswereobservedaswell(seebelow).On
male(MonkeyO;10.5kg)andonemale(MonkeyF;5.5kg).Thesesame eachtrial,monkeysmadeeyemovementstochoosebetweentwovisual
twomonkeyscontributeddataondopamineneuronstotwootherstud- images presented on a computer monitor (see Fig. 1A). Each image
ies(Fiorillo,2011;Fiorilloetal.,2013),andMonkeyOcontributeddata spanned4degreesofvisualangleandwascentered6degreestotheleftor
toathirdstudy(Fiorilloetal.,2008).Procedurescompliedwithguide- rightofthecenterofthemonitor.Thepositionsofthetwoimagesvaried
linesestablishedbytheNationalInstitutesofHealthandwereoverseen randomlybetweenleftandrightacrosstrials.Continuousfixationofgaze
locally by the Stanford University Animal Care and Use Committee. oneitherimagefor500mswasimmediatelyfollowedbyasound(72dB)
Liquid intake was restricted to ensure the motivation of monkeys to that signaled correct performance. The sound was identical for each
participateinexperiments. stimulusandrewardoutcome.Onesecondafterthesound,thereward
Eyetrackinganddataacquisition.Amonkey’sheadwasfixedinplacein outcomewasdelivered,consistingofjuice,withorwithoutthesimulta-
frontofacomputermonitorinasound-insulatedroom.Eyeposition neousdeliveryofanaversivestimulus.Regardlessoftheanimal’schoice,
wasmonitoredwithaninfraredeyetrackingsystem(EyelinkII;SRRe- bothimageswereextinguishedatthesametime(synchronouswithoffset
search),exceptforinitialexperimentsperformedinMonkeyO,inwhich ofjuicedeliveryontrialsinwhichjuicewasdelivered).Theintertrial
ascleralsearchcoilwassurgicallyimplantedandeyepositionwasmon- intervalfromrewardoutcometopresentationofimagesonthenexttrial
itoredwithinamagneticfield.Exposoftware(writtenbyPeterLennie wasrandombetween0.5and1.5s.Iffixationononeoftheimageswas
andmodifiedbyJulianBrown)wasusedtodeliverstimuliandtocollect notacquiredwithin1.0s,bothimagesdisappearedandanotherintertrial
alldata. intervalbegan.
Stimuli.Juice,saline,andbittersolutionsweredelivereddirectlyinto Thegoalwastodecideuponasingleintensityforeachtypeofaversive
themouthbyacomputer-controlledsolenoidvalve(part002-0010-900; stimulus,havingasinglelevelofaversiveness,andthentolaterdeliver
ParkerHanafin)undertheforceofgravity.Thesolenoidswerelocated stimuliofthatsameintensityduringrecordingsofdopamineneurons.
(cid:1)0.7mfromthemonkey’shead,andtheiropeningandclosingemitted Intensitycorrespondedtoconcentration,airpressure,orduration(vol-
aclickof72dBasmeasuredatthelocationofthemonkey’shead.One ume),asdescribedindetailabove(seeStimuli).Weidentifiedaninten-
deliverysystem(reservoir,solenoidvalve,polyethylenetube,andmetal sityforwhichthemotivationalvaluewasequalandoppositetoasmall
spout)wasusedtodeliverjuice,andasecondsystemwasusedtodeliver dropofjuice(50–100(cid:1)l)byaskinghowmuchjuicethemonkeywould
salineorbittersolutions.Thus,appetitiveandaversivesolutionscouldbe sacrifice to avoid a given intensity of an aversive stimulus. In initial
deliveredtogetherorseparatelyoneachtrial. experiments,itwasnecessarytoestimatethemotivationalvaluesofmany
Juicewastwo-thirdsapplejuiceandone-thirdwater.Inchoiceexper- stimulusintensities.Thiswasdoneusingachoicetaskthatconsistedof
imentsofthesortillustratedinFigure1,choosingthe“standard”option justtwovisualconditionedstimuli.Inatypicalcase,oneconditioned
resultedindeliveryof180(cid:1)lofjuicedeliveredoveraperiodof200ms, stimulus(CS)wasassociatedwith180(cid:1)lofjuiceplustheaversivestim-
exceptinchoiceexperimentswithloudsound(seeFig.1E),inwhich130 ulus(theintensityofwhichwasfixedacrossalltrials),andtheotherwas
(cid:1)lwasdelivered(cid:2)150msforthestandardoption.Inallexperiments associatedwith120(cid:1)lofjuiceintheabsenceofanyaversivestimulus.
performedduringneuronalrecordings,130(cid:1)lwasdeliveredoverape- Approximately100trialswereperformed.Thegoalwastofindaninten-
riodof150ms. sityforwhichchoiceswouldbesplitapproximatelyevenlybetweenthe
Theintensitiesofaversivestimulithatweredeliveredduringneuronal twooptions,whichintheaboveexamplewouldcorrespondtoanaver-
recordingswereselectedbasedonthechoicetask(seeFig.1),asdescribed sivevalueof(cid:3)60(cid:1)l(120–180(cid:1)l)ofjuice.Iftheanimaldisplayeda
indetailbelow.“Saline”wasanaqueoussolutionof8%NaCldelivered strongpreferenceforoneoptionortheother,anewintensityforthe
(cid:2)60ms(30(cid:1)l)inMonkeyO,and4%NaClfor30ms(10(cid:1)l)inMonkey aversivestimuluswaschosen,novelvisualstimuliwereintroduced,and
F(seawatertypicallyhasasalinityof3.5%).Ininitialchoiceexperiments, anothersessionofchoicetrialswasperformed.Atthestartofexperi-
concentrationsof0.5,1,4,and8%weretestedinMonkeyO,butonly4% ments,intensitieswerechosentobeverylowandwerethengradually
wastestedinMonkeyF.Bittersolutionwas1or10mMdenatoniumfor increased(typicallybyafactorof10–1000over1–3weeks).
80ms(40(cid:1)l),andwasonlytestedinMonkeyO.Beforechoosingthese Onceanintensitywasfoundthathadarelativelystablevalue(across
concentrations,concentrationsof0.001,0.003,0.1,and0.3mMdenato- multiplesessionsandd)between(cid:3)50and(cid:3)100(cid:1)l,thechoicetaskde-
niumweretestedinchoiceexperiments.Whereas1mMwasdelivered pictedinFigure1wasperformedtomorepreciselyquantifytheaversive-
duringinitialrecordingsofneurons,10mMwasdeliveredduringthe ness. As in the choice task described above, there were two options
latterrecordings(seeFigs.1and2). (correspondingtotwoimages)oneachtrial.The“standard”imagewas
Some experiments in Monkey O used only sound as a stimulus. presentedoneverytrial,andchoosingitresultedinalargevolumeof
“Morse,”asoundavailableintheMacOSX10.4operatingsystem,was juice(180(cid:1)l)plusanaversivestimulus(whichhadanintensitythatwas
deliveredfor150ms.Itwasamplifiedto90dBinexperimentswith“loud fixedwithinandacrosssessions).Fixationonthe“alternative”image
sound,”andto72dBinexperimentswithmoderatesound(asmeasured resultedindeliveryofjuiceonly.Fourorfivealternativeimageswere
at the location of the monkey’s head). The walls of the experimental presentedineachsession(butonlyonepertrial),eachassociatedwitha
roomwereinsulatedtominimizesoundoriginatingoutsidetheroom. particularvolumeofjuice(seeFig.1A).Eachsessionconsistedof80or
The Pavlovian CS used in some experiments (see Fig. 6) had both 100totaltrials,with20trialsofeachtrialtype(aparticularalternativevs
visualandauditorycomponents.Thevisualstimuliconsistedofcirclesor thestandard).Eachtrialtypewaspresentedoncewithineverysequence
squaresof4degreesofvisualangleindiameterorwidth,whichdiffered of4or5consecutivetrials,withthepreciseorderbeingrandomized.
incolorandgratingfrequencydependingontheexperiment.Theywere Onceaparticularintensitywaschosen,itwasimportanttoensurethat
alwayspresentedinthecenterofthemonitor.Inallexperimentswith its aversiveness was sufficiently stable over time. Therefore, the same
audiovisualPavlovianstimuli,thesamesoundof72dBwaspresented choicetask,withpreciselythesamevisualandaversivestimuli,wasper-
(“ping,”availableintheMacOSX10.4operatingsystem). formedoneormoretimesperdayoveraperiodof1–2weeksbeforethe
Airwasdeliveredbyacomputer-controlledsolenoidvalve(part009- startofneuronalrecording(andforoneormoredaysafterthecomple-
0339-900;ParkerHanafin)connectedtoapressurizedgastank.Thegas tionofallneuronalrecordings).
4712•J.Neurosci.,March13,2013•33(11):4710–4725 Fiorilloetal.•MultiphasicResponsesofDopamineNeurons
It is worth noting that, whereas the aversiveness measured in the stimulusidentityatshortlatencies.However,thelongintertrialintervals
choicetaskwasforstimulithatwerewellpredictedbythemonkeys,the makethestimulilesspredictable,whichislikelytoreducediscrimination
neuronalresponsestothesestimulithatarereportedherearefortheless atshortlatencies.
predictabledeliveryofstimuli.Unpredictedairpuffcouldbemoreaver- OtherexperimentswereperformedwithPavlovianconditionedstim-
sivethanpredictedairpuff,despitethelikelihoodthattherewaslittlethe uli.Conditioningwithvisualoraudiovisualconditionedstimuliforsev-
monkeyscoulddotomitigatetheaversiveeffectsofairdirectedtoward eralhundredtrialswasperformedover2–4dbeforethestartofneuronal
thenostril.Similarly,althoughtheloudsoundwasfoundtobeneutralin recordings.Theunconditionedstimulus(US)onsetoccurred1.0safter
thechoicetask,itcouldhavebeenaversivewhenitwasmoresurprising. CS onset, and CS offset either coincided with the offset of the US or
Inadditiontothechoicetask,otherbehaviorsalsosuggestedtheaver- occurred1.0safterCSonsetinthecasethattheUSwasomitted.Offsetof
sivenessofthestimuli.Anticipatoryblinkingwasoftenobservedafter theCSthereforesignaledomissionoftheUS.Intertrialintervalswere
stimulithatpredictedair.Whereasgazeremainedfixedonvisualcondi- 2–6s.Theconditionedstimuliwerefollowedbyunconditionedstimuli
tionedstimulithatpredictedjuice,itwasoftenavertedfromaversive onpseudo-randomlyselectedtrialswithaprobabilityof0.5,althoughwe
conditionedstimuliafteranorientingresponse.Duringablockoftrials expectthatthiswasinconsequentialwithrespecttothetimecourseofthe
inwhichsalineorbittersolutionsweredelivered(intheabsenceofjuice), transientneuronalresponsestoCSonset.
the solutions were seen to run out of the mouth, suggesting that the Because most of the present stimuli had motivational value to the
monkeystriednottoswallowthem.Agapingresponsewasoftenob- monkeys and because they were not explicitly conditioned to predict
served,andthebittersolutionsometimescausedagagresponse,espe- future stimuli, they meet the common definition of “unconditioned
ciallyathighconcentration. stimuli.”However,theyarebetterunderstoodas“conditioned”stimuli.
Judgingaversivenessbasedonchoiceversusavoidancebehaviors.Avoid- The concept of an “unconditioned” stimulus, including its timing, is
ancebehaviors,suchaseyeblink,havebeenusedtomeasureaversiveness poorlydefined(Wise,2002).Itisnotclearwhichstimuliaretrulyun-
of air puff in previous studies (Joshua et al., 2008; Matsumoto and conditioned(andthushavemotivationalvaluesthatareinsensitiveto
Hikosaka,2008,2009).Onelimitationofrelyingonavoidancebehavior conditioning).Theconsequenceofthisissuetothepresentworkarises
isthatitdoesnotdemonstratenetaversiveness,asintheexampleofeye fromthefactthat,ifatrulyUSexists(specificactivationofnociceptors
blinkinresponsetoacoolbreezeonahotday.Italsodoesnotindicate beingaplausibleexample),itcouldfunctionthrough“hard-wired”cir-
themotivationalvalueofairpuffrelativetojuiceorotherrewards,a cuitry.Althoughthiswouldobviouslylimittheabilityofthenervous
factorofparticularimportanceifrewardsarepresentinthesamecontext systemtolearn,itwouldmakeitpossibletoalwaysandunambiguously
asairpuff(resultingfromadaptationofdopamineneuronstoreward recognizeastimulusasappetitiveoraversive,evenatshortlatencies.In
context) (Tobler et al., 2005). We tracked eye position and observed contrast,thediscriminationofCSofageneralnatureisinherentlymore
conditionedeyeblinktoconditionedstimulipredictingairpuff(datanot ambiguous and must necessarily require more time (because almost
shown).Asdescribedbelow,wefoundthatourairpuffwasmuchmore identicalstimulicouldhaveoppositevalues,orthesamestimuluscould
aversivethannecessarytoeliciteyeblink,suggestingthatourairpuffmay haveoppositevaluesdependingoncontext).Itisknownthatdopamine
havebeenmuchmoreaversivethanthatusedinpreviousstudies(Joshua neuronsrespondtothefirstpredictivestimulusinaseries,andnotto
etal.,2008;MatsumotoandHikosaka,2008,2009). laterstimulithatarewellpredicted.Thefirststimulusinallcasesstudied
Ininitialbehavioralexperiments,weadjustedairpressure,duration, herewaspresumablyauditory,followedwithinmillisecondsbytactile
andbodylocation(Fiorilloetal.,2013).Ourintentionwastostartwith stimulation(exceptinthecaseofpuresoundstimuli).Theauditoryand
minimallyaversiveairpuffandtoincreaseitsintensityuntilthemonkey tactilestimuliassociatedwithappetitivejuicewerenearlyidenticaltothose
would sacrifice at least 50 (cid:1)l of juice to avoid it. In the first choice associatedwithaversivesalineandbittersolutions.Themotivationalvalueof
experiments,air(10psifor10ms)wasdirectedattheleftcheek((cid:1)2cm theseauditoryandtactilestimuliwaspresumablyaresultofconditioning,in
belowtheeye)fromadistanceof(cid:1)2cm(incomparison,Matsumoto whichanimalslearnedtoassociatethemwiththegustatoryandnutritive
andHikosaka,2009deliveredair6–7cmfromtheface;seetheirOnline consequencesthatfollowed(gustationisknowntobearelativelyslowsense).
Methods).Thiswassufficienttocauseeyeblink,aswellasconditioned Thus,itwouldbemisleadingtocharacterizethesestimulias“uncondi-
anticipatoryeyeblinkinachoicetask,butthemonkeywasunwillingto tioned.”Weusetheterm“US”tofollowconventionandtodistinguishthese
sacrificeanyjuicetoavoidtheair.Airpressureanddurationweregrad- stimulifromourconventional“conditionedstimuli,”whichcouldinstead
uallyincreased,andtheairwasdirectedclosertotheeye:(cid:2)50psi(mea- belabeledas“higher-orderconditionedstimuli.”
suredatthevalveadjacenttothegastank,(cid:1)2.5mfromthemonkey’s Recordingandlocalizationofdopamineneurons.Glass-insulatedtung-
face), the solenoid valve leaked air. A total of 50 psi for 200 ms was stenelectrodes(Alpha-Omega)wereplatedwithgoldandplatinum.The
sufficienttopushlooseskinfromthecheekovertheeye,butstillthe regionofdopaminecellbodiesintheventralmidbrainwaslocalizedwith
monkeywasunwillingtosacrificeanyjuicetoavoidtheair.Airwasthen theaidofphysiologicallyidentifiedlandmarks.Dopamineneuronswere
directedattheear,againwithoutaversivevalueinthechoicetask.Finally, distinguishedfromotherneuronsintheregionbythecharacteristicsof
itwasfoundthataversivenesscouldbemeasuredinthechoicetaskwhen their extracellularly recorded impulses, including long, multiphasic
air(35psifor200ms)wasdirectedatthenostrilfromadistanceof(cid:4)1cm waveformsandslow,fairlyregularbasalfiringrates(SchultzandRomo,
(paralleltothegroundratherthandirectlyupthenostril).Thus,inall 1987).Forthesametwomonkeysusedinthisstudy,detailsoftheelec-
subsequentexperiments,200msofairwasdirectedatthenostrilfroma trodes,physiologicallandmarks,andmethodoflocalizingneuronswere
distanceof1cminbothMonkeysOandF.Theseobservationssuggest given by Fiorillo (2011), whereas details of waveforms and estimated
thattheminimalintensityofairthatevokesconditionedeyeblinkmay locationsofneuronsareprovidedbyFiorilloetal.(2013).
haveamotivationalvaluethatisfarbelowthatofaverysmalldropof Chronologyofexperiments.Neuronaldatawerecollectedinfoursetsof
juice((cid:1)50(cid:1)l),probablybyafactorofatleast10.Thismaybeinpart experimentsinMonkeyO,followedbyonesetinMonkeyF.Foreachset,
becausetheaversivenessofairpuffneartheeyecouldbeavoidedsub- choicebehaviorwasstudiedoveraperiodof3–5weeksbeforethestartof
stantially by eye blink, a very small movement that requires minimal neuronalrecordings,whichextendedoveranother3–6weeks.Atotalof
effort.Bycontrast,ourairwasdirectedatthenoseandcouldnotbe 35–66neuronswererecordedineachset,with1–6neuronsrecordedper
avoided.Ithadavalueof(cid:1)(cid:3)75(cid:1)lofjuice,anditseemslikelythatthis dayandanaverageof3or4.Multipleexperimentswereperformedwhile
wasmuchmoreaversivethantheairpuffadministeredinpreviousstudies. recording each neuron, with each experiment consisting of a discrete
Experimentaldesignforneuronalrecordings.Mostofthedataanalyzed blockoftrialslasting(cid:1)5–10minandconsistingof40–80trialspertrial
herewerefromrecordingsmadewhileunconditionedstimuliwerede- type(mostexperimentsreportedhereconsistedofjustonetrialtype).
liveredintheabsenceofanyCS.Ablockof30–80trialsconsistedofjust Mostneuronswererecordedfor(cid:1)1.0–1.5h.Set1inMonkeyOincluded
onestimulus(juice,air,saline,bittersolution,sound)deliveredwitha experimentswithhigh-intensitysound,aswellasexperimentswithair
randomlydistributedintertrialintervalof2–16s.Thedeliveryofjustone puffthatarenotpresentedhere.Set2includedexperimentswithboth
stimulusinablockoftrialswouldbeexpectedtominimize“generaliza- high-andmoderate-intensitysound,aswellaspreviouslyreportedex-
tion”andthereforeimprovetheabilityoftheneuronstodiscriminate perimentsontiming(Fiorilloetal.,2008).Set3includedexperiments
Fiorilloetal.•MultiphasicResponsesofDopamineNeurons J.Neurosci.,March13,2013•33(11):4710–4725•4713
Table1.Thetimeperiodsinwhichfiringratesweremeasuredineachcella whereFR(t)isfiringrateinbint,Mismotivationalintensity,Sissensory
Stimulustype Early Middle Late intensity,and(cid:3)M(t)and(cid:3)S(t)arefitcoefficientsrepresentingtheiresti-
matedcontributionstofiringrate.(cid:3) effectivelyrepresentstheresidual
0
Juice,saline,andbittersolutions 100–200 220–320 400–600 firingratenotaccountedforbysensoryintensityormotivationalvalue.
Airpuffandsound 40–100 160–260 380–480 Becausetheanalysiswasperformedaftersubtractingbaselinefiringrates,
VisualCSoffset(juiceomission) 80–160 160–280 360–560 allcoefficientsarenear0beforestimulusonset(seeFig.7).
VisualCSonset 50–150 220–320 None To perform this analysis, it was necessary to estimate sensory and
aDataaregivenasmillisecondsafterstimulusonset.Theseperiodswerechosentocapturethemajorityofeach motivationalintensitiesonacommonscale.Intheabsenceofaclear
phaseofresponsebasedonvisualinspectionofpopulationPSTHs(illustratedinFig.4).Themainreasonforusing rationalefortreatingthetwodimensionsdifferently,wenormalizedval-
differenttimeperiodswastheearlierresponsestoairandloudsoundrelativetoothertypesofstimuli.
uesineachdimensiontocoverarangeof0–1.Alloftheintensitiestested
herearelikelytohavebeenwithinarangethatcanbewelldiscriminated
atbehavioralandneuronallevels,andwithinsucharange,firingrates
with saline as well as conditioned stimuli that predicted saline. Set 4
havegenerallybeenfoundtoincreaseasanapproximatelylinearfunc-
includedexperimentswithairpuffandbittersolution,andconditioned
tionofthelogarithmofintensityinavarietyofsensorysystems.
stimulithatpredictedairpufforbittersolution.Onlyonesetofexperi-
Intryingtoquantifysensoryintensity,wereliedonmeasuredsound
mentswasperformedinMonkeyF,anditincludedthesametypesof
intensities,whichsignaledtheonsetofallstimuli,exceptjuiceomission.
experimentsasset4inMonkeyO,exceptthatsalinesolutionwasstudied
Thesewere72dBformoderatesoundandfordeliveryofjuice,saline,and
insteadofbittersolution.Inaddition,inset4inMonkeyO,andtheone
bittersolutions,82dBforairpuff,and90dBforloudsound(seeStimuli
setinMonkeyF,anexperimentwasperformedinwhichaCSpredicted
abovefordetails).Whereasonsetsofthesestimuliwererelativelyunpre-
juicewithaprobabilityof50%.Analysisoftheneuronalresponsestothe
dictable,theomissionofjuicewassignaledbythewell-predictedoffsetof
CS provided the basis of a previous report (Fiorillo, 2011). Here we
a1.0sdurationvisualstimulus,intheabsenceofanysound.Wethere-
reportontheneuronalresponsesafteromissionofexpectedjuiceinthis
forejudgedjuiceomissiontohaveasensoryintensitythatwassubstan-
sameexperiment.
tiallylessthantherelativelyunpredictedonsetofa72dBsound(cid:8)our
Data analyses. Data were analyzed using Matlab. Firing rates were
unpublisheddata,andobservationsofothers(Bromberg-Martinetal.,
measuredduringthetimeperiodsindicatedinTable1.Foralltypesof
2010),suggestthattheeffectsofsensoryintensityonfiringratearesup-
stimuli,thebaselinefiringratewasmeasuredinthe1.0speriodbefore
pressedbypredictability(cid:9).Intheabsenceofaprecisemethodtoquantify
stimulusonset.Toquantifythestatisticalsignificanceofthesefiringrates
the sensory intensity of such diverse stimuli on a common scale, we
relative to baseline firing rate in a single neuron, firing rates (spike
assignedequidistantvaluesof0,0.33,0.66,and1forjuiceomission,all
counts)werecalculatedforallindividualtrialsandcomparedwithbase-
linerateswithanunpairedttest;p(cid:4)0.05wastakentobesignificant, liquidsolutionsandmoderatesound,airpuff,andloudsound,respec-
tively.ThesevaluesarealmostthesameasifwehadconsideredvisualCS
without any correction for the fact that the same test was performed
offset(juiceomission)tohaveanintensityequivalentto64dB,which
separatelyonallneurons.Forcomparisonsacrossthewholepopulation
wouldhavenormalizedtogivevaluesof0,0.31,0.69,and1.0.
ofrecordedneurons,themeanfiringrateacrosstrialswascalculatedfor
WhereasthevaluesdescribedabovewereusedtogenerateFigure7,we
eachconditionineachneuron,andthesesingleneuronmeanfiringrates
obtainedqualitativelyidenticalresultsfortwoothersetsofestimated
werethencomparedbetweenconditionsacrossthepopulationofneu-
sensoryintensities.Specifically,weobtainedthesametemporalorderof
ronsusingpairedorunpairedttests.
dependenceonsensoryandmotivationalintensitiesasinFigure7A,and
Onset latencies of the early activation were identified according to
thesamerankorderofpredictedperistimulustimehistograms(PSTHs)
criteriathatmadenouseofeitherthepredefinedwindowsusedtomea-
acrossstimuliasinFigure7B.Inoneofthesecases,weentirelyomitted
surefiringratesorsignificancetests.Neuronswithcontrolfiringratesof
(cid:4)1.0Hzwereexcludedfromthisanalysis.Anactivationwasdefinedas thedatawithvisualCSoffset(juiceomission;becauseithadnosound
intensityandwasthusdifficulttocomparewithotherstimuli)andused
theearliesteventinwhichfiringrateinatleast3consecutivebinsof10ms
equidistantvaluesof0,0.5,and1.0forthestimulithatincludedsound
wasgreaterthantwicebaselinefiringrate.Theonsetwasthendefinedas
(datanotshown).ThisproducedresultsextremelysimilartoFigure7.In
theonsetofthefirstbinintheseries.Torestrictouranalysistothe“early”
anothercase,weconsideredthatthesensoryintensityofvisualCSoffset
period, activations were required to have onsets within 200 ms after
wasequivalenttojusta45dBsound,muchlessthanour72dBsound;
stimulusonset.
therefore, we used normalized values of 0, 0.6, 0.8, and 1.0. Still, the
Regressionanalyses.Weperformedtwomultiplelinearregressionanal-
resultsweresimilartoFigure7,buttheeffectofsensoryintensitywasless,
yses, in relation to Figures 2B and 7, using the function “regstats” in
andtheearlyactivationinthepredictedPSTHswaslessthanforthe
Matlab.WithrespecttoFigure2,wewishedtodeterminewhetherchro-
actualPSTHs.Whenwewentevenfurtherinthisdirection,considering
nologicalorderordorsal-ventral(DV)positionhadthegreaterinfluence
thatvisualCSoffsethadasensoryintensityof0onadBscale,andthen
intheshiftfromactivationtosuppressionintheresponsestosalinein
normalizedallsoundintensitiesbyarangeof90dB(toarriveat0,0.8,
Monkey O. Chronological order was quantified by the rank order of
0.9, and 1.0), the effect of sensory intensity nearly vanished, and the
neuronalrecordingsfrom1to7.DVpositionofeachneuronwasdeter-
predictedPSTHsforallstimuli,otherthanjuiceandjuiceomission,were
mined relative to the midpoint between the most dorsal and ventral
nearly identical. Thus, we obtained results similar to Figure 7 for all
dopamine neurons recorded during a single electrode penetration
reasonableestimatesofsubjectivesensoryintensitythatwerebasedon
(Fiorilloetal.,2013).Eachfactorwasnormalizedtocoveranequivalent
range.Allneurons(n(cid:5)46)wereanalyzedinasingleregressionanalysis. rawsoundintensities,andlessreasonableestimatesperformedlesswell
inpredictingactualPSTHs.
A second linear regression analysis was used to examine the time
Juicehadavolumeof130(cid:1)lduringneuronalrecordings.Weused
courseoftheinfluenceofsensoryintensityandmotivationalvalueon
datafromthechoicetasktoquantifymotivationalvaluerelativetojuice.
firingrate.Thisincludedresponsestojuice,saline,bitter,air,loudsound,
Althoughtherewassomevariationacrossstimuliinaverageaversiveness,
moderatesound,andjuiceomission.Weusedthemeanfiringratefor
neuronalresponsesdisplayedlittlevariationwithaversiveness(seeFig.
eachstimulusineachneurontoperformoneregressionanalysisacross
10). For regression analysis, we treated all aversive stimuli as having
theentirepopulation,withthenumberofdatapointsforeachstimulus
valuesof(cid:3)100(cid:1)lofjuice(seeFig.1).Wefurthermodifiedthesevalues
conditiondependingonthenumberofcellsrecorded(whichvariedfrom
in an attempt to account for the “rectification” of firing rates, which
14inthecaseof72dBsoundto194inthecaseofjuice).Becausedifferent
limitsthedecreaseinfiringratethatcanbecausedbyaversiveness.For
combinationsofstimuliweretestedindifferentcells,wedidnotperform
this,weestimatedtheaveragebaselinefiringratetobe3Hzandthepeak
individualregressionanalysesonresponsesofindividualneurons.
firingrateafterjuicedeliverytobe12Hz(seeFig.4A),andwetherefore
Forcellfiringratesineach20msbin,wefitalineaccordingtothe
multipliedtheappetitivenessofjuice(130(cid:1)l)byafactorof4(12/3).We
equation:
thennormalizedtocoverarangeof1,arrivingatvaluesof(cid:3)0.16,0,and
FR(cid:6)t(cid:7) (cid:2) (cid:3) (cid:4) (cid:3) (cid:6)t(cid:7)M (cid:4) (cid:3)(cid:6)t(cid:7)S 0.84foraversive,neutral,andappetitivestimuli,respectively.
0 M s
4714•J.Neurosci.,March13,2013•33(11):4710–4725 Fiorilloetal.•MultiphasicResponsesofDopamineNeurons
Results A Choice Task: 5 trial types B100 Air Puff
Choicetasktomeasureaversiveness Standard Alternative
Thefirststepinthepresentworkwasto Juice + Aversive Juice Only %) Monkey O
selectanappropriateintensityforaversive e (75 (n=7)
stimuli. One criterion was that the aver- 180 µL 0 µL ativ Monkey F
sivestimulimusthavelowenoughinten- n (n=14)
r
180 µL 70 µL e50
sity so that the overall experience of the Alt
monkeys in the laboratory was positive, r
180 µL 125 µL o
whichwasnecessarytoensuretheircontin- s f25
uedcooperation.Thus,thepositivevalueof e
juicereward,summedovertime,mustex- 180 µL 180 µL oic
ceedthevalueofaversivestimuli.Asecond Ch 0
criterion was that we wished to have the 180 µL 200 µL 0 50 100 150 200
Alternative Juice Volume (µL)
aversivenessofstimulibecomparabletothe
appetitiveness of small juice rewards that C 100 Saline D100Bitter
havebeencarefullystudiedwithrespectto
%) Monkey O
primate dopamine neurons, to minimize
thepossibilitythataversivenessmightbetoo e (75 (n=50) 75
weak to modulate dopamine neurons. In ativ Monkey F
studiesofprimateneurophysiology,typical n (n=22)
r
e50 50
juice volumes on rewarded trials are be- Alt
tween100and200(cid:1)l.Tobleretal.(2005) r
o
foundthatthefiringrateofdopamineneu- s f25 25 Monkey O
rons is highly effective in discriminating e 1 mM (n=33)
juice volumes as small 50 (cid:1)l, especially oic 10 mM (n=5)
h
whenthejuicevolumesarewellpredicted. C 0 0
We therefore wished to deliver aversive 0 50 100 150 200 0 50 100 150 200
Alternative Juice Volume (µL) Alternative Juice Volume (µL)
stimuliwithvaluesthatwereequalbutop-
positeto(cid:1)50–100(cid:1)lofjuice.Deliveryof E 100 Sound F200 Summary
largerjuicevolumes(130–180(cid:1)l),inother
%) Monkey O ) Monkey O
blocksoftrials,ensuredcooperationofthe L
monkeys. e (75 (n=17) µ150 Monkey F
Toquantifythevalueofaversivestim- ativ s (
uli,achoicetaskwasusedtomeasurehow rn es
much juice a monkey would sacrifice to Alte50 en100
aamvnaoddideMetehyteehmoavdoesvr)se.imvOeennstteismatcouhlciht(rosiaeoles,eMmbaeottnewkreieaeylnss es for 25 ersiv 50
c v
two visual images (Fig. 1A). The “stan- oi A
h
dard”imagewaspresentedoneverytrial, C 0 0
andchoosingitresultedinalargevolume 0 50 100 150 200 Air SalineBitterSound Air Saline
ofjuice(180(cid:1)l)plusanaversivestimulus Alternative Juice Volume (µL) Aversive Stimulus
(which had an intensity that was fixed
Figure1. Measuringaversivenesswithachoicetask.A,Thechoicetaskconsistedof4or5trialtypes,with20trialsofeachtype
within and across sessions). Fixation on
persession.Oneachtrial,twovisualimagesappeared,andfixationofgazeonanimageresultedinitsassociatedrewardoutcome.
thealternativeimageresultedindelivery
Choosingthe“standard”option,whichwasavailableoneachtrial,alwaysresultedindeliveryofalargevolumeofjuiceplusan
of juice only. The greater the volume of
aversivestimulus.The“alternative”optionresultedinjuicealone,withthevolumeofjuicedependingontheimage.B–E,Forall
juice associated with the alternative, the stimulustypes,theaveragepercentageofchoicesforthealternativeoptionincreasedwithitsassociatedjuicevolume.Thesolid
greater the fraction of trials on which it verticallineindicatestheamountofjuiceassociatedwiththestandardoption(180(cid:1)l,exceptinthecaseofloudsound).The
waschosen.Thepercentageofchoicesfor hyphenatedverticallinesshowtheestimatedpointsofindifferenceatwhich,onaverage,choiceswereevenlydividedbetweenthe
the alternative options was plotted as a twooptions.Curvesrepresentlogisticfunctionsoftheformasfollows:y (cid:2) a/(b (cid:4) exp((cid:5)cx)).Twoconcentrationsof
function of the alternative juice volume, bittersolutionweretested:1and10mMdenatonium.Numbersofsessionsareindicatedinparentheses.F,Aversivenessforeach
andcurveswerefit(Fig.1B–F).Thedif- typeofstimulus(mean(cid:10)SDacrosssessions,inmicrolitersofjuice).Fromlefttoright,stimuliareair,saline,bitter,andloudsound
ference between the standard juice vol- inMonkeyOandairandsalineinMonkeyF.Bittercorrespondstothelowerconcentrationofdenatonium(1mM).Aversivenesswas
definedasthedifferencebetweenthestandardjuicevolumeandthejuicevolumecorrespondingtothepointofindifference(the
ume(180(cid:1)l)andthepointofindifference
distancebetweenthehyphenatedandsolidlinesinB–E).Aversivenesswascalculatedforeachsessionbeforeaveragingitacross
(atwhich50%ofchoiceswerefortheal-
sessions(asopposedtoaveragingthechoicedataacrosssessionsandthenfittingcurves,aswasdoneinB–E).
ternative) provided our measure of the
valueoftheaversivestimulus.
Afteraperiodofseveralweeksofadjustingtheintensityof
the case of bitter solution, for which the concentration was
an aversive stimulus and performing choice experiments, a increased 10-fold approximately midway through several
singleintensitywaschosenthatwasmeasuredtohaveanav- weeks of neuronal recordings. The increase in concentration
eragevaluebetween(cid:3)70and(cid:3)110(cid:1)lofjuice(Fig.1F).This ledtoanincreaseinaverageaversivenessfrom(cid:1)(cid:3)110(cid:1)lto
intensity was then used in all neuronal recordings, except in (cid:3)200(cid:1)l(Fig.1D).
Fiorilloetal.•MultiphasicResponsesofDopamineNeurons J.Neurosci.,March13,2013•33(11):4710–4725•4715
A 160 B DV Position (mm) dB. By contrast, the air puff emitted a
Monkey O -2 -1 0 1 soundof82dBandcouldbeseentocause
4
) Bitter (n=8) z) Dorsal11 Ventral a strong tactile stimulation of the nose
(µL120 Saline (n=8) e (H 3 10 (hsaeveeMshaotwerniatlhsaatnddopMamethinoedns)e.uProanstssctaundibees
s at activatedbyhigh-intensitystimuli(seeIn-
s R
ne 80 ng 2 11 tvriooduuscstitound)ie,asltwhohuetghheirtwsuacshnosttcimleaurliinwperree-
e ri
siv n Fi 1 8 aaplspoesttiutidveie,davaelrosiuvde,soorunneduotrfa9l.0WdeBtihnerMefoonre-
er 40 e i 3 913 4 2 keyO.Althoughtheloudsoundwasexperi-
v g 0
A n 10 encedasaversivebytheexperimenter,itwas
a
h 10 1 foundtobeneutralinthechoicetask(de-
0 C−1
1 2 3 1 2 3 4 5 6 spitethefactthatweusedanarrowrangeof
Daily Session Order Daily Neuron Order juicealternativescenteredonthestandard
volumetobesensitivetoweakaversiveness
C 8 D 10 Bitter Denatonium orappetitiveness)(Fig.1E,F).
Saline 10 Days Recording
Weweresurprisedtofindthatthecon-
Concentration
First 2 neurons of day
centratedsalinesolution(8%NaCl)could
z)6 (n=20) 8 1 mM (n=19) sometimeshaveanetappetitivevaluein
H Last 1-2 neurons of day 10 mM (n=14)
MonkeyOandthatitbecamemoreaver-
( (n=19)
e 6 siveoverthecourseoftheday.Thedefin-
at4 itiveevidenceofappetitivenesscamefrom
R 4 choice experiments performed in the

g weeksaftertheendofthe3weekperiod
n
Firi2 2 dwuerrienrgewcohridcehdn.eTuhreonchaloriecseptoanskseisllutostsraaltinede
inFigure1demonstratedthatsalinehad
0 0 thesameaverageaversivenessafterneural
0 200 400 600 800 0 200 400 600 800 recordings as it had before recordings
Time (ms) Time (ms)
(datanotshown).Inadistinctchoicetask
performedafterneuralrecordings,Mon-
Figure2. Increasedaversivenessofsalineandbittersolutionsdiminishesneuronalactivation.A,Salineandbittersolutions
key O chose between two visual stimuli
becamemoreaversiveoverthecourseofthedayinMonkeyO.Aversivenesswasmeasuredin1–10sessionseachdayinthechoice
task(Fig.1).Aversiveness(mean(cid:10)SEMinmicrolitersofjuice)asafunctionofdailysessionnumberinchronologicalorder,forall (each of which was novel at the start of
eachsession):onedeliveringsalinealone
daysonwhichatleastthreesessionswereperformed(8dwithsalineandanother8dwithbittersolution).Atthestartofsome
days,MonkeyOworkedforsalineinadistinctchoicetask,demonstratingthatsalinewassometimesappetitive(seeResults).B, and the other delivering nothing. In 4
Neuronalactivation(mean(cid:10)SEM)tosalineinMonkeyOduringtheearlyperiod(100–200ms)declinedasafunctionofthe suchsessionsover3d,MonkeyOchose
numberofneuronsrecordedeachdayinchronologicalorder(black)butdidnotshowaclearrelationtorelativeDVposition(green), saline between 31 and 65 times in each
aconclusionfurthersupportedbyregressionanalysis(seeResults).DVpositionwasmeasuredrelativetothemidpointbetweenthe session while choosing “no outcome”
mostdorsalandmostventralneuronrecordedduringasingleelectrodepenetration.Thenumbersofcellscontributingtoeachdata only 7–13 times across these 4 sessions.
pointareshown:intwocases,twodistinctwaveformswererecordedsimultaneously,causingtheretobemoreneuronsrecorded
These sessions were performed near the
secondandthirdthanfirst.C,Responses(mean(cid:10)SEM)tosalineinMonkeyOnearthebeginningandendoftheday.Among46
startofeachofthe3dinthelaboratory.In
neuronsrecordedover10d(3–7neuronsperday),averageresponsesareshownforthefirst2(n(cid:5)20)andlast1or2(n(cid:5)19)
latersessionsonthesame3d,themonkey
neuronsrecordedeachday.Binsize,20ms.D,Responsesto1and10mMdenatonium,abittersubstance.Afterrecordingresponses
to1mMdenatoniumin19neuronsoveratotalperiodof(cid:1)2weeks,theconcentrationwasincreasedto10mM,andresponsesof wouldnotworkforsalinewhilestillwork-
14moreneuronswererecordedoverseveraladditionalweeks.Thehigherconcentrationwasmoreaversive(Fig.1D). ingforjuice.On2otherdays,themonkey
wouldnotworkforsalineevenatthestart
oftheday.Thus,salinewassometimesap-
Itisnoteworthythattheintensityofairthatwasrequiredfor
petitiveinMonkeyO,andthiswasprobablyforitswatercontent
the monkeys to sacrifice juice was much greater than that re-
becausewaterbutnotsaltintakewasrestricted,andbecausealow
quiredtoelicitconditionedblinking.Airpuffneartheeye,de-
concentrationofbittersolutionalsobecamemoreaversiveover
spitehavingatleast10-foldgreaterpressurethanthatrequiredto
thecourseoftheday(Fig.2A).
eliciteyeblink,wasmotivationallyneutralinourchoicetask.We
ThechoiceexperimentsofFigure1weredesignedtocarefully
werenotabletogetMonkeyOtosacrificeanyjuicetoavoidair
measureaversivenessbutnotappetitiveness(becausetherewaslittle
directedneartheeyeorear,andwethereforedirectedairatthe
ornooptiontochoosealargervolumeofjuiceratherthanasmaller
nose(seeMaterialsandMethods).Thus,itseemslikelythatour
volumeplustheteststimulus).Weobservedthat,insomesessionsat
airpuffwasmuchmoreaversivethanthatdeliveredinprevious
studiesthatjudgedaversivenessbasedonconditionedeyeblink thestartoftheday,MonkeyOappearednearlyindifferentinchoos-
(Joshuaetal.,2008;MatsumotoandHikosaka,2008;Matsumoto ingbetweenjuiceversusjuiceofequalvolumeplussaline(orbitter).
andHikosaka,2009). ForsalineandthelowconcentrationofbitterinMonkeyO,curvesof
Juice,saline,andbittersolutionsweredelivereddirectlyinto thetypeshowninFigure1shiftedupandtotheleftoverthecourse
the mouth and appeared to be very similar to one another in ofmostdays(datanotshown),andtheaversivenessthereforein-
termsofsensorymodalitiesandintensities.Inallthreecases,the creasedoverthecourseofthedayforbothsalineandbitterinMon-
openingandclosingofthesolenoidvalvescausedasoundof(cid:1)72 keyO(Fig.2A).Thisexplainstheflatnessoftheaveragecurvesin
4716•J.Neurosci.,March13,2013•33(11):4710–4725 Fiorilloetal.•MultiphasicResponsesofDopamineNeurons
thesecases(Fig.1C,D),whichwasnotobservedforairpuffineither inMonkeyF,commensuratewiththelackofashiftinaversive-
animal,orsalineinMonkeyF(Fig.1B,C). nessovertheday(datanotshown).
Theapparenteffectofdailychronologicalorderofrecording
on early activation to saline (Fig. 2B,C) could conceivably be
Neuronalrecordings
relatedinsteadtothepositionofneuronsintheDVdimension.
Werecordedextracellularlyfrom195neurons(134neuronsin
Electrodeswereadvancedverticallyfromdorsaltoventral,and
MonkeyO,61inMonkeyF)intheventralmidbrainthatwere
usually(butnotalways)thefirstneuronsrecordedeachdaywere
judgedtobedopamineneuronsbasedonlongdurationwave-
moredorsal.Wethereforeperformedamultiplelinearregression
formsaswellaslowandmoderatelyregularrates(seeMaterials
analysiswithtwofactors:chronologicalorderofrecordingand
andMethods).Otherthanlocation,theseweretheonlycriteria
relative DV position. Each of these was determined indepen-
usedinthepresentstudytoidentifyputativedopamineneurons.
dentlyforeachelectrodepenetration(onepenetrationperday,
Consistentwithpaststudies,thefiringrateof78%ofneuronswas
recording3–6neurons)andthenpooledacrossall46neurons.
significantly increased after delivery of relatively unpredicted
Orderofrecordinghadastrongandsignificanteffect((cid:3)(cid:5)(cid:3)3.8,
juice(p(cid:4)0.05).However,neitherresponsivenesstojuicenor
p(cid:5)0.009),whereasDVpositionhadlittleornoeffect((cid:3)(cid:5)1.6,
anyotherstimuluswasusedasacriterionforselectingwhichcells
p (cid:5) 0.51). This result is supported by Figure 2B, in which re-
to record. We estimated that 81% of all 195 neurons were in
sponsesareplottedbothasafunctionofneuronorder(black)as
substantianigra,andtherestwereinventraltegmentalareaand
wellasrelativeDVposition(green).
the retrorubral field. The companion paper provides details of
Satiationofthirstoverthecourseofthedayseemstobethe
waveforms,locationsofrecordedneurons,andcorrelationsbe-
most likely explanation for the increased aversiveness of saline
tween neuronal locations and responses (Fiorillo et al., 2013).
andbitter.Atthestartoftheday,theappetitivewatercontentof
Neuronal locations had only a modest influence on responses;
salineandbittersolutionsmayhaveoffsetandsometimesover-
therefore, the analyses presented here do not distinguish re-
cometheaversivepropertiesofthesaltorbittersubstance.Ifso,
sponsesbyneuronallocation.
thentherecouldalsobeadecreaseinactivationtojuiceoverthe
Duringneuronalrecordings,aversivestimuliweredelivered
courseoftheday.ThiswasindeedobservedinMonkeyO(data
havingthesameintensity,andpresumablythesameaversiveness
not shown). However, we could not be confident this was ex-
(70–110(cid:1)l)asinthechoicetask(Fig.1).Thevolumeofjuice
plained by chronological order of recording as opposed to DV
deliveredduringneuronalrecordings(130(cid:1)l)waslessthanin
position.Thelatterappearedtobeanimportantfactorwithre-
choice experiments (180 (cid:1)l). Experiments with juice omission
specttosuppressionandlateactivation(Fiorilloetal.,2013).
wereperformedwithavolumeof240(cid:1)lofjuicedeliveredwitha
Furtherevidencethatanincreaseinaversivenessabolishedthe
probability of 0.5; ignoring the critical issue of subjective scale
initial activation came from experiments with two concentra-
(Tobleretal.,2005),thiswouldcorrespondtoanegativepredic-
tion error of 120 (cid:1)l. Thus, our stimuli were all of comparable tionsofbittersolution(1and10mMdenatonium)inMonkeyO.
Thelowerconcentrationwasaversiveonaverage(Fig.1D,F),but
absolutevalues,exceptforneutralsound.Eachstimuluswaspre-
likesalineinMonkeyO,itwasnearlyneutralorperhapsappeti-
sentedinarelativelyunpredictablemannerinablockoftrialsin
tivenearthestartofsomedays,anditbecomemoreaversiveover
which no other stimuli were presented (with the exception of
thecourseofeachday(Fig.2A).Thehigherconcentrationhadan
“juiceomission”;seeMaterialsandMethods).
averageaversivenessapproximatelytwicethatofthelowercon-
centration, and it appeared to be aversive under all conditions
Increasingaversivenessofsalineandbitterdiminishes (Fig.1D;errorbarsaretoosmalltobevisibleforchoicesbetween
neuronalactivation juiceofequalvolumewithorwithoutbitter,indicatingthatthe
Anobviousandimportantexperimentistosystematicallyvary monkeyvirtuallyneverchosethebittersolution).Afterrecording
theaversivenessofastimulus,bychangingitsintensity,andex- responsestothelowconcentrationofbittersolutionin19neu-
aminetheeffectthishasonneuronalresponses.Despiteitsfun- rons, the high concentration was delivered while recording an
damentalimportance,thisexperimenthasnotbeendone,atleast additional14neurons.Theneuronalactivationobservedwith
inpartbecauseofthedifficultyofrecordingresponsestomultiple the lower concentration was virtually absent with the higher
stimuliindopamineneurons.Itwouldalsobeofparticularin- concentration(p(cid:5)0.04,unpairedttestofratesintheearly
teresttoexamineresponsestothesamestimulusunderdiffering period, 100–200 ms), whereas the suppression of firing rate
motivationalstates,asinthecasethatsaltcanbeappetitiveor appearedmorepronouncedinresponsetothehigherconcen-
aversivedependingonthenutritionalneedsoftheanimal.We tration(Fig.2D).
didnotsetouttoperformeitheroftheseexperiments,butwedid Thesedatasupporttheviewthatappetitivestimuliinducea
ultimatelycollectdatathatrelatestoeachofthem. netexcitationofdopamineneurons,andaversivestimuliinduce
Asdescribedabove,wefoundthatsaline(andperhapsbitter a net inhibition. Here we focus on the population average re-
solutionaswell)wasnearlyneutralorevenappetitiveatthestart sponse,whichillustratesthetypicalresponseofmostdopamine
of the day in Monkey O (probably for its water content), and neurons.Inadditionalanalyses,wesearch,butdonotfind,evi-
becamestronglyaversivebytheendoftheday(Fig.2A).Inneu- dence to support the hypothesis of Matsumoto and Hikosaka
ronsrecordednearthestartoftheday,theaverageresponsewas (2009) that a minority of dopamine neurons are activated by
an activation followed by a suppression slightly below baseline aversiveness(Fiorilloetal.,2013).
firingrate(Fig.2C,black).Asthedayprogressedandthemonkey
receivedmoresalineaswellasjuice,theactivationgraduallydis- Multiphasicresponses
appeared(Fig.2B)(linearregression,r(cid:5)(cid:3)0.39,p(cid:5)0.008,n(cid:5) Weobservedtriphasicresponsestoaversivestimuliandneutral
46).Inneuronsrecordedlaterintheday,therewasvirtuallyno loudsound,consistingofasequenceofearlyactivation,suppres-
activationbutaclearsuppression(Fig.2C,red).Asimilarobser- sion,andlateactivation,asillustratedbyaneuronrespondingto
vationwasmadeforresponsestoaCSpredictingsalineinMon- air(Fig.3A).Manyneuronsexhibitedjustoneortwoofthethree
keyO(datanotshown).Nocleareffectwasobservedwithsaline phases.Figure3B,Cshowsaneuronrespondingtosalinewith
Fiorilloetal.•MultiphasicResponsesofDopamineNeurons J.Neurosci.,March13,2013•33(11):4710–4725•4717
delivered at moderate intensity (72 dB)
A
Air (Fig. 4D). All modulations in firing rate
20 Hz weresmallcomparedwiththeactivation
observedinresponsetojuice(Fig.4E),a
fact that is unlikely to result from the
smalldifferencesinabsolutemotivational
valueacrossstimuli(Tobleretal.,2005).
NormalizationofaveragePSTHstotheir
peaksillustratestherelativetimecourses
(Fig.4E).Theearlyactivationtoairand
loudsoundwasvirtuallybacktobaseline
bythetimeofpeakactivationtojuiceand
saline-bitter. All three phases of the re-
B sponse appeared to occur earlier for air
Saline
andloudsoundthanforsaline-bitter.
Tocharacterizethethreephasesofre-
10 Hz sponse across all neurons, we measured
firingratesinperiods(“early,”“middle,”
and“late”),whichwerechosenbyvisual
inspectionofthepopulationPSTHstobe
centered on the average peak of each
phase(Fig.4).Firingratesweremeasured
in the same periods in each neuron, but
the periods differed across stimuli, all
threebeingearlierforairandsoundthan
forjuiceandsaline-bitter(Table1;Fig.4).
C
The scatter plots of Figure 5 display all
Bitter
5 Hz firingratesfromallneurons.
Thefirstpointtobemadeaboutthe
scatterplotsisthatthereisnoclearevi-
denceofmorethanonecluster,suggest-
ing an absence of statistically discrete
populationsofdopamineneurons(Fio-
rillo et al., 2013). Second, the response
distribution is skewed, with most neu-
ronsshowingsmallchangesfrombase-
line rates and a few neurons showing
large changes, especially increases. Be-
-1.0 0 1.0
Time (s) causetheskewofthedistributionisnot
extreme, population average PSTHs
providereasonablygooddescriptionsof
Figure3. Threeindividuallyrecordeddopamineneuronsrespondingtoaversivestimuliwiththreephasesofresponse.Rasters the population. Nonetheless, because
andPSTHsarealignedtotheonsetofeachstimulusattime0.Trialsarearrangedinchronologicalorderfrombottomtotop.Each dopamineneuronshavealowbasalfir-
PSTHhasadifferentverticalscale.A,Theresponsetoairinthiscelldisplayedallthreeresponsephasesthatweobservedacrossthe ing rate, activations tend to be larger
populationofneurons,consistingofanearlyandlateactivationseparatedbysuppression.B,Acellthatrespondedtosaline thansuppressions,andintakinganav-
solutionwithsuppressionfollowedbylateactivation.C,Acellthatrespondedtobittersolutionwithalong-lastingsuppressionin erage, a few cells (or trials) with large
theabsenceofanyactivation.
activations can overwhelm a majority
withweaksuppression.
suppressionfollowedbylateactivationandanotherneuronre- Wealsorecordedresponsestoaudiovisualconditionedstimuli
spondingtobitterwithalong-lastingsuppressionintheabsence that predicted the unconditioned stimuli. According to theory
ofanyactivation. (Montague et al., 1996), and consistent with empirical evidence,
InpopulationaveragePSTHs,theresponsetojuiceconsisted conditionedstimuliandunconditionedstimulibothmodulatedo-
ofwhatappearedtobejustone“phase”ofactivation(Fig.4A),as pamineneuronsbyinducingpredictionerrors.Wecansafelyas-
previously reported. Saline and bitter solutions caused a very sumethattheresponsesweobservedtoourunconditionedstimuli
small activation, followed by suppression below baseline, fol- wereactuallytoauditoryandperhapstactileaspectsofthestimuli
lowedbylateactivation(Fig.4B);becauseresponsestosalineand that had been conditioned to predict gustatory, nutritional, and
bitterweresosimilar,wepoolthemtogetherinmostsubsequent other consequences. Thus, we would not expect for responses to
analyses.Aversiveairandneutralloudsoundcausedasuppres- audiovisualconditionedstimulitodifferqualitatively,andthatis
sion and late excitation that were quite similar to saline-bitter, what we found. The average population response to conditioned
but they also caused an initial activation that was substantially stimulipredictingair,saline,orbitterwasasmallactivationfollowed
earlier and of shorter duration than activations observed with bysuppression(Fig.6A).Likewise,themosttypicalresponsewas
other stimuli (Fig. 4C,D). The activation by the neutral loud activationfollowedbysuppression(Fig.6B,C;thelargenumberof
sound (90 dB) was virtually absent when the same sound was responsesshownintheupperleftquadrants).Lateactivationswere
4718•J.Neurosci.,March13,2013•33(11):4710–4725 Fiorilloetal.•MultiphasicResponsesofDopamineNeurons
observedinsomecellsbutwerenotreadily A 20 Juice B 10 Saline and Bitter
apparentinaveragesacrosstheentirepopu-
monkey O (n=133) monkey O saline (n=25)
lation(Fig.6A).
z) monkey F (n=61) 8 monkey O bitter (n=25)
Initialactivationisrelatedtosensory H15 monkey F saline (n=56)
(

intensity,notmotivationalvalue e 6
t
The initial activation in response to air a10
R
was unlike responses to juice, saline, or 4
g
bitter.Ithadamedianonsetlatencyof55 n
ms(n(cid:5)28),comparedwith95ms(n(cid:5) ri 5
138)forjuice(p(cid:5)0.006),andtheactiva- Fi 2
tion was of shorter duration (Fig. 4F).
Very similar activations to aversive air 0 0
0 400 800 0 400 800
puffhavebeenseenpreviouslyinprimate
dopamine neurons (Joshua et al., 2008; C 15 Air D 15 Sound
MatsumotoandHikosaka,2009),aswell monkey O (n=31) monkey O 90 dB (n=31)
as neurons of lateral habenula (Matsu- )
z monkey F (n=55) monkey O 72 dB (n=14)
moto and Hikosaka, 2008). Our air puff H
(10 10
caused very strong tactile stimulation of
e
thenose,aswellasasoundof82dB(see at
Materials and Methods). We suspected R

thattheearlyactivationofdopamineneu- g
n 5 5
rons,startingbefore100ms,wasrelated ri
tosensoryintensityratherthanmotiva- Fi
tional value (Bromberg-Martin et al.,
2010).Itiswellknownthattheinitialac- 0 0
0 400 800 0 400 800
tivation of “high-level” sensory neurons
totheirpreferredstimulus(suchasaface) E 10 Juice F 1
isstronglydependentonsensoryintensity
(suchascontrast),andthattheinfluence Saline-Bitter 0.8
)
z
of specific, high-level features emerges H Air
0.6
later. Likewise, it has been widely ob- ( Sound
servedthatconditionedstimulicanevoke ate 5 Juice Omission 0.4
aninitialactivationindopamineneurons R
that is similar regardless of the reward 0.2
g
valueassociatedwiththeCS,andthatfir- n
ingratesdiscriminaterewardvalueatlon- Firi 0 0
ger latencies (e.g., Fiorillo et al., 2008; -0.2
Nomotoetal.,2010).
To test the hypothesis that short- 0 200 400 600 -100 0 100 200 300
latencyactivationisrelatedtosensoryin- Time (ms) Time (ms)
tensity,weadministeredaloudsound(90
dB)inMonkeyO.High-intensitystimuli Figure4. Populationaverageresponses(PSTHs)ofdopamineneuronstorelativelyunpredictedstimuli.Allrecordedneurons
have been found to activate dopamine wereincludedintheaverage,withoutselectionforanyresponse.Theshadedregionsillustratethe“early,”“middle,”and“late”
neurons(seeIntroduction),butprevious periods(Table1)inwhichfiringratesweremeasuredforfurtheranalyses(Fig.5).Black,MonkeyO;red,MonkeyF.A,Juice
studies have not verified their motiva- (mean(cid:10)SEM).B,Salineandbittersolutions.Responsesof29neuronsfromMonkeyOwereexcludedhere;thesewereamongthe
tionalvalue.Ourloudsoundwasfoundto first2neuronsrecordedatthestartofeachdaywithsalineoralowconcentrationofdenatonium(1mM,bitter)thatmayhavebeen
beneutralinthechoicetask(Fig.1E).It appetitive(Fig.2).C,Air.D,Sound.Thesamesoundwasdeliveredat90or72dB(withinpartiallyoverlappingpopulationsof
neurons).E,PopulationPSTHsforeachtypeofstimulusareshowntogetherforcomparison,averagingacrossallcellsfromeach
evokedaneuronalresponsethatappeared
monkey.Baselinefiringratesweresubtracted,andresponsesweresmoothedbycalculatingamovingaveragewith50msbinsand
virtuallyindistinguishableinshapeand
incrementsof1ms.F,Tocompareactivationkinetics,thesamePSTHsshowninEwerenormalizedtohaveapeakamplitudeof1.0.
time course from the response to air,
Airandloudsoundhadearlyonsets,whereasjuiceandsaline-bitterhadlateronsets.
consisting of a triphasic sequence of
activation-suppression-activation (Fig.
4E,F).Liketheresponsetoair,theinitialactivationwasbriefand 4E,F).Juiceomissioncausedvirtuallynoearlyactivation(Figs.
hadashortlatency(medianof55ms;n(cid:5)15).Thesamesoundat 4Eand5B).ItwassignaledbythepredictableoffsetofavisualCS,
moderateintensity(72dB)causedvirtuallynoearlyactivation aneventthatwouldbeexpectedtohavelowersensoryintensity
(Fig.4D).Salineandbittersolutions,whichhadmoderatesen- thananyothersensoryeventstudiedhere.
sory intensity (72 dB) but were as aversive as air puff, did not Tobetterquantifytheinfluenceofsensoryandmotivational
cause large or short-latency activation (Fig. 4E,F). Juice had intensitiesovertime,weperformedalinearregressionanalysis
moderate sensory intensity (72 dB) and was very similar to (seeMaterialsandMethods).Wedistinguishedfoursensoryin-
saline-bitterinitsauditoryandtactileaspects.Likewise,juice tensitiesbasedonsoundintensity,rangingfromvisualCSoffset
andsaline-bittercausedearlyactivationsthatdifferedinampli- onthelowend(whichsignaledjuiceomissionandhadnosound)
tude and duration but had very similar onset latencies (Fig. to loud sound on the high end. Based on the choice task, we
Fiorilloetal.•MultiphasicResponsesofDopamineNeurons J.Neurosci.,March13,2013•33(11):4710–4725•4719
A 60 20 18 (cid:1)40 and 150 ms, whereas motivational
Early Middle Late
valuehaditseffectbetween90and300ms
z)50 16 (Fig.7A).Theregressionmodelpredicted
H
(40 12 firingratesthatweresimilarinamplitude
te 30 12 and time course to the recorded rates,
a
R 8 suggesting that our linear model with
20 6
two factors accounted reasonably well
g
n10 4 for average neuronal responses (Fig. 7B;
Firi 0 Juice 0 0 comparewithFig.4E).Becauseourquan-
titativeassignmentsforsensoryintensity
B 14 14 20
weresomewhatarbitrary,werepeatedthe
12 12
16 regression analysis with other parameter
10 10
values, but the results remained similar
8 8 12 (see Materials and Methods). This sug-
6 6 8 geststhat,withourstimuliandtestcondi-
4 4 tions,thefiringrateofdopamineneurons
2 OmisJsuiiocne 2 4 didnotreliablyreflectmotivationalvalue
until(cid:1)150msafterstimulusonset.How-
0 0 0
ever,thetimeperiodduringwhichfiring
C 30 18 18
rates best reflect motivational value will
depend on the type of stimuli and their
20 12 12 predictability.Theperiodofsensitivityto
reward value is substantially later and
longer-lastingforperceptuallydifficultdis-
10 6 6 criminations(Nomotoetal.,2010).How-
ever, it could also be earlier than that
Air
observedhereifdiscriminationismadeeas-
0 0 0
ier by having stimuli that differ in spatial
D 20 18 18
location or sensory modality, or if condi-
16 tioningwereperformedforalongerperiod
12 12 oftime.
12
8 Saline Suppressionbyaversiveand
6 6 neutralstimuli
Bitter
4 Between one-third and one-half of the
populationofneuronsdisplayedstatisti-
0 0 0
0 5 10 15 0 5 10 15 0 5 10 15 callysignificantsuppressiontoeachofthe
E 25 6 8 aversiveorneutralstimuli,aswellasjuice
omission,andasubstantialmajoritydis-
20
6 playedsomesuppression,evenifitdidnot
4
15 reach significance (Fig. 5): saline-bitter,
4 43%ofneurons(57of134),medianfiring
10 rateof65%ofbaselinefiringrateacross
2
5 2 theentirepopulationofcells;air,34%of
neurons(30of87),medianrateof73%;
Sound
0 0 0 90dBsound,48%ofneurons(15of31),
0 2 4 6 0 2 4 6 0 2 4 6 medianrateof60%;72dBsound,36%of
Baseline Firing Rate (Hz)
neurons (5 of 14), median of 41%; juice
omission,51%ofneurons(45of88),me-
Figure5. Scatterplotsoffiringratesduringprestimulusbaselineandinearly,middle,andlateperiodsinallindividualneurons. dianrateof61%.Inthosecellswithstatis-
“Baseline”wasthe1.0speriodbeforestimulusonset.Early,middle,andlateperiodsarespecifiedpreviously(Table1;Fig.4).Each ticallysignificantsuppression,theaverage
pointrepresentsasingleneuron.Redcorrespondstoneuronsinwhichfiringratewassignificantlygreaterthanorlessthan timingandpeakamplitudeswereapprox-
baselinefiringrate(p(cid:4)0.5;unpairedttestsacrosstrials).Diagonallinesrepresentequality.“Zero”hasbeenshiftedslightlyoff imately similar for the various stimuli,
theaxestoaidvisualizationofsomedatapoints.A,Juice.B,Juiceomission.C,Air.D,Salineorbittersolutions(29neuronswere with peak suppressions occurring over a
excluded,asdescribedforFig.4B).E,Sound.Circlesandtrianglesrepresentneuronstestedwith90and72dBofthesamesound, rangeof(cid:1)200–280ms(Fig.8).
respectively.Redandgreenindicatesignificantdeviationsfrombaselinefiringrates.
The most interesting aspect of these
dataisthesuppressionbyneutralstimuli,
distinguishedthreemotivationalintensities,whichwemodified whichappearedtobeidenticaltoaversivestimuli.Thesuppres-
inanattempttoaccountfortheexpectednonlinearrelationship sion by moderate intensity sound was statistically significant
with firing rate (which is naturally rectified; see Materials and acrossthepopulationdespitethesmallernumberofcellstested
Methods).Regressionanalysissuggestedthatbothsensoryand (p(cid:5)0.01,pairedttest,n(cid:5)14).Thissuggeststhatsuppressionis
motivationalintensitieseachhadastronginfluenceonfiringrate notspecifictoaversivestimuli.Itshouldbenotedthat,although
butthatsensoryintensityexerteditsinfluenceearlier,between many previous studies of dopamine neurons in primates have