The roles of the reward system in sleep and dreaming

18 Pages·2012·1.34 MB·English

by  Lampros Perogamvros

NeuroscienceandBiobehavioralReviews36(2012)1934–1951

ContentslistsavailableatSciVerseScienceDirect

Neuroscience and Biobehavioral Reviews

journal homepage: www.elsevier.com/locate/neubiorev

Review
The roles of the reward system in sleep and dreaming

LamprosPerogamvrosa,b,SophieSchwartzb,c,d,∗

aDivisionofNeuropsychiatry,DepartmentofPsychiatry,UniversityHospitalsofGeneva,Geneva,Switzerland
bDepartm en tofNeuroscience ,Universityo fG eneva,Gen eva,Switze rland
cSwissCenter fo rAffectiveScie nces,Unive rs ityofGe neva,Ge neva,Switzerland
dGenev aNeur osc ienceCen ter,Unive rsityofGe ne va,Gene va,Switz erland

a r t i c l e i n f o a b s t r a c t

Articlehistory: Themesolimbicdopaminergicsystem(ML-DA)allowsadaptedinteractionswiththeenvironmentandis
Receiv ed11October2011 ther eforeofcrit icalsignificanc eforth eindividu al’ssu rvival.Th eML-DAsys tem isim plicatedinre war d
AReccceepivteedd i2n5 r Mevaisye2d0 f1 o2rm 23 May 2012 and emot ion al func tions, and it is pert urbed in sch izophren ia, addictio n, and de pression. T he ML-DA
rewardsystemisnotonlyrecruitedduringwakefulbehaviors,itisalsoactiveduringsleep.Here,we

introducetheRewardActivationModel(RAM)forsleepanddreaming,accordingtowhichactivationof
Keywords: theML-DA rew ardsys temduring sleep contrib ute stom emo ryprocess es,tother eg ulation ofrapid-e ye
Sleep
movement(REM)sleep,andtothegenerationandmotivationalcontentofdreams.Inparticular,the
Dreaming
engagementofML-DAandassociatedlimbicstructuresprioritizesinformationwithhighemotionalor
Emotion
Memory motivational relevance for (re)processing during sleep and dreaming. The RAM provides testable predic-
Learning tionsandhasclinicalimplicationsforourunderstandingofthepathogenesisofmajordepressionand

Dopamine addiction.
Mesolimbicdopaminergicsystem ©2012ElsevierLtd.Allrightsreserved.
Rewardsyst em

Ventraltegmentalarea

Hippocampus
Amygdala
Contents
1. Introduction........................................................................................................................................ 1935

1.1. Themesolimbicdopaminergic(ML-DA)system........................................................................................... 1935

1.2. Rewardprocessing......................................................................................................................... 1936

1.2.1. Generalinformationaboutrewardprocessing.................................................................................. 1936

1.2.2. Braincircuitsofrewardprocessing.............................................................................................. 1936

2. IntroducingtheRewardActivationModel(RAM)forsleepanddreaming....................................................................... 1937

2.1. ActivationoftheML-DArewardsystemduringsleep..................................................................................... 1937

2.1.1. VTAburstingactivityisincreasedduringREMsleep............................................................................ 1937

2.1.2. IncreasedactivityoftheNAcc,PFC,amygdalaandACCduringREMsleep...................................................... 1938

2.1.3. ActivationofHCandVSreward-relatedneuronsduringSWS.................................................................. 1939

2.1.4. Activationofotherreward-relatedstructuresandmechanismsduringsleep.................................................. 1939

2.2. Sleepdeprivationandrewardsystem..................................................................................................... 1939

Abbreviations:ACC,anteriorcingulatecortex;AIMmodel,activation,input/output,modulationmodel;dlPFC,dorsolateralprefrontalcortex;GABA,gamma-aminobutyric
acid;HC,hippocam pus; l-DOPA, precursor tothen euro transm ittersknow nascatechol amines,incl udingd opami ne.Asadrug itisusedt oincrea sedop amineconcentrations

inthecentralnervoussystemfortheclinicaltreatmentofParkinson’sdisease,forexample.;LH,lateralhypothalamus;LTP,long-termpotentiation;ML-DA,mesolimbic

dopaminergicsystem;mPFC,medialprefrontalcortex;MRI,magneticresonanceimaging;NAcc,nucleusaccumbens;N1,N2,N3,stagesN1,N2,andN3ofNREMsleep;NREM

sleep,non-rapid-eyemovementsleep;OFC,orbitofrontalcortex;PET,positronemissiontomography;PFC,prefrontalcortex;PPT,pedunculopontinetegmentalnucleus;PTO

junction,parieto-temporo-occipitaljunction;RAM,RewardActivationModel;RBD,REMsleepbehaviordisorder;REMsleep,rapid-eyemovementsleep;REMSD,REMsleep

deprivation;SD,sleepdeprivation;SLD,sublaterodorsalnucleus;SN,substantianigra;SNc,substantianigraparscompacta;SWS,slow-wavesleep;TST,threatsimulation

theory;vmPFC,ventromedialprefrontalcortex;VP,ventralpallidum;VS,ventralstriatum;VTA,ventraltegmentalarea.
∗
Correspondingauthorat:DepartmentofNeuroscience,FacultyofMedicine,UniversityofGeneva,rueMichel-Servet1,1211Geneva4,Switzerland.

Tel.:[phone];fax:[phone].

E-mailaddress:[email protected](S.Schwartz).

0149-7634/$–seefrontmatter©2012ElsevierLtd.Allrightsreserved.

http://dx.doi.org/10.1016/j.neubiorev.2012.05.010
L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951 1935

3. Rewardactivationandmemoryprocessingduringsleep......................................................................................... 1939

3.1. Memoryprocessingleadingtorewardactivationduringsleep........................................................................... 1940

3.2. Rewardactivationleadingtomemoryformationduringsleep........................................................................... 1940

4. RewardactivationandREMsleepgeneration..................................................................................................... 1940

5. Rewardactivationanddreaming.................................................................................................................. 1941

5.1. Theneuralcircuitsofdreaming............................................................................................................ 1941

5.2. ML-DArewardsystemactivationanddreamgeneration................................................................................. 1942

5.3. ML-DArewardsystemactivationanddreamcontent..................................................................................... 1942

5.4. LinksoftheRAMtootherdreamtheories................................................................................................. 1943

5.4.1. Freud’spsychoanalytictheoryofdreamsandRAM.............................................................................. 1943

5.4.2. Activation-input-modulation(AIM)modelandRAM............................................................................ 1943

5.4.3. ThreatSimulationTheory(TST)andRAM........................................................................................ 1943

6. Clinicalcontext..................................................................................................................................... 1944

6.1. Psychosisanddreaming.................................................................................................................... 1944

6.2. Sleepandemotionregulation.............................................................................................................. 1944

6.2.1. Insomniaandsleeplossasprecursorsofdepressionviaalterationofbrainrewardnetworks................................. 1944

6.2.2. Evidencefromsleepdeprivationstudies........................................................................................ 1945

6.3. Sleepandcompulsivebehaviors........................................................................................................... 1945

7. Limitationsandtestabilityofthemodel........................................................................................................... 1945

8. Conclusions........................................................................................................................................ 1946

Acknowledgements................................................................................................................................ 1946

References......................................................................................................................................... 1946

1. Introduction sleep; (b) influences the regulation of REM sleep; and (c) con-

tributestothegenerationandthemotivationalcontentofdreams.

‘Wehavenodreamsatallorinterestingones.Weshouldlearnto Importantly, these distinct functional roles of ML-DA activation

beawakethesameway—notatallorinaninterestingmanner.’ duringsleepinvolvepartiallydissociatedreward-relatedmecha-

(Friedrich Nietzsche, ‘The Gay Science’, Third Book, aphorism nisms.TheRAMalsosuggeststhattherewardfunctionofdreaming

232,1882) offers humans an evolutionary advantage by optimizing waking

behavior, in particular adapted emotional responses. Our main
Themesolimbicdopaminergic(ML-DA)systemoriginatesfrom
aimindevelopingtheRAMistoproposeatheoreticalframework
theventraltegmentalarea(VTA)ofthemidbrainandprojectsto
concerningtherolesoftheML-DAsysteminsleepandrewardpro-
variousdiencephalicstructureslikethenucleusaccumbens(NAcc),
cesses,thatintegratesrecentdatafromneurophysiologystudiesin
aswellastotheprefrontalcortex(PFC)(Alcaroetal.,2007).TheML-
animalswithbehavioralandbrainimagingstudiesinhumans.By
DAcircuitpromotesadapted,goal-directedbehaviorsbymediating
addressing and formalizing the interaction of sleep with reward
differentaspectsofrewardprocessing(HaberandKnutson,2010;
and emotional processes, an additional objective of the RAM is
Schultz,1998).TheML-DAsystemisalsocriticalforthedevelop-
toaccountforsomeneuropsychiatricsymptomsinpatientswith
mentofinstinctualandappetitivedrives,afunctionofthebrain
dopaminergicabnormalities(e.g.schizophrenia,depression,addic-
thatpermitsmammals,includinghumans,toseekforrewardsand
tion).Thismodelthusofferstosleepscienceamissinglinkbetween
developsearchstrategiesinordertoobtainthem(Panksepp,1998).
sleepneurophysiology,cognition,anddreamcontent.
SeverallinesofevidencesuggesttheML-DArewardsystemis

activatedduringsleep.Neurophysiologicalstudiesinanimalshave

revealedthatregionsoftheML-DAcircuitsuchastheNAccand

theVTAshowincreasedburstingneuralactivityduringrapid-eye 1.1. Themesolimbicdopaminergic(ML-DA)system

movement(REM)sleep(Dahanetal.,2007;Lenaetal.,2005),and

aroleofdopamineinthegenerationofREMsleephasbeensug- Therearetwomainascendingdopaminergicpathwaysinthe

gested (Dzirasa et al., 2006). It is also striking that brain circuits humanbrain:thenigrostriatalpathwayandtheML-DA(mesolim-

modulatingsleep–wakestates(e.g.hypocretin/orexinsystem)may bic or mesocorticolimbic) pathway (Fig. 1; Stahl and Muntner,

influencereward-relatedresponsesandneuralplasticityinML-DA 2008).Thenigrostriatalpathwayprojectsfromthesubstantianigra

regionsandamygdalainbothanimalsandhumans(Borglandetal., (SN) of the midbrain to the dorsal striatum (caudate and puta-

2008;Harrisetal.,2005;Ponzetal.,2010a,b).Moreover,neuronal men).Itismainlyimplicatedinthemodulationofcognitiveand

activity recorded during the encoding of reward-related mem- behavioralhabits,andinproceduralaspectsofmovements;itsdys-

ory may be spontaneously reactivated during slow-wave sleep function is associated with Parkinson’s disease (Jankovic, 2008).

(SWS) in the ventral striatum and the hippocampus in rodents TheML-DApathwayoriginatesintheVTAandinnervatesthelateral

(Lansinketal.,2009).Inhumans,thereplayofelementsfromwak- hypothalamus(LH),theNAccandolfactorytubercleoftheventral

ingexperienceinbothsleep(Maquetetal.,2000;Peigneuxetal., striatum(VS),thebednucleusofstriaterminalis,lateralseptum,

2004;Rudoyetal.,2009)anddreams(Wamsleyetal.,2010)was hippocampal complex (HC), amygdala, PFC, and anterior cingu-

foundtoenhanceovernightmemoryconsolidation,andrewarded latecortex(ACC).Thissystemisimplicatedinmotivatedbehaviors

informationmayparticularlybenefitfromsleep-relatedmemory (Alcaroetal.,2007),rewardprocessing(Ikemoto,2007),emotional

consolidation(FischerandBorn,2009).Finally,elevateddopamine processing(Alcaroetal.,2007),andlearning(Adcocketal.,2006).

levelintheML-DAsystemduringsleephasbeensuggestedtoplay DysfunctionsoftheML-DAsystemareobservedinschizophrenia

animportantroleinthegenerationofdreams(Solms,2000,2002). (Epsteinetal.,1999;Kapur,2003;Laruelle,2000;MeltzerandStahl,

Inthisarticle,wepresenttheRewardActivationModel(RAM), 1976;Sarteretal.,2005;WintererandWeinberger,2004),addic-

whichproposesthatactivationoftheML-DArewardsystemdur- tion (Kauer and Malenka, 2007; Koob, 1992; Koob and Volkow,

ingsleep(a)serveslearningandmemoryconsolidationfunctions 2010;Thomasetal.,2008),aswellasindepression(Daillyetal.,

byreactivatingrewardedoremotionally-relevantmemoriesduring 2004;DunlopandNemeroff,2007;NestlerandCarlezon,2006).
1936 L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951

Box1:Rewardsandpunishers Box2:TheSEEKINGsystem

Rewards are considered as stimuli that positively reinforce The SEEKING system has been proposed by Panksepp as

behavior (Schultz, 2006) and that usually (but not always) a “curiosity-interest-expectancy” command system which is

induce a conscious experience of pleasure. Examples of pri- associated with instinctual appetitive craving states (Alcaro

maryrewardsarestimulilikefood,water,andsexualstimuli. andPanksepp,2011;Panksepp,1982,1998).Asapsychobehav-

Theyreinforcebehaviorswithouthavingtobelearned.They ioral emotional and motivational system of the mammalian

derive their value from innate mechanisms, like the hunger brain, the SEEKING system is related to approach behav-

andthirststates.Secondary(conditioned)rewardsgaintheir iorsandtothefeelingofanticipationwhileseekingareward

rewardvaluefromalearnedassociationwithprimaryrewards. (actualexternalstimuliorinternalrepresentationofareward)

Money, positive feedback, social interactions, and pleasant (Alcaro and Panksepp, 2011). Its core circuits include dense

touch are typical secondary rewards. The organism’s inter- ML-DAprojectionsfromtheVTAtotheLH,mediodorsaltha-

action with a reward may implicate two distinct types of lamus,medioventralpallidum,NAccshell,olfactorytubercle,

processes,i.e.motivationalandhedonicprocesses(Berridge amygdala,anteriorlimbiccortex,andprefrontalcortex(Fig.2).

et al., 2009). The motivational component refers to switch- AlthoughitresemblestheML-DAsystem,theSEEKINGsystem

ing attention and behavior toward reward-related stimuli or ismoreextensive,becauseitincludesascendingnoradrener-

novel cues. Motivated behavior is mainly associated with gicandcholinergicexcitatoryinfluences,aswellasdescending

dopaminergicsignalinginmesolimbicstructuresliketheVTA, glutamatergic and GABA influences (Panksepp, 1998). Acti-

hypothalamus,amygdala,NAcc,andPFC.Ontheotherhand, vation of the SEEKING system does not only represent the

the hedonic impact of a reward concerns the pleasurable internalexperienceofareward,butalsotheemotionalantici-

reaction of the organism to a sensory cue (e.g. the sensory patorycomponentsofanappetitivesearchstrategy(Panksepp,

pleasureofsweettastes),andimplicatesopioid,cannabinoid, 1998).Interest,curiosityandanticipation,butnotpleasureor

andGABAtransmissionfromthebrainstemtotheNAcc,ven- consumption,arethefeelingsrelatedtotheactivationofthe

tralpallidum,insularcortexandorbitofrontalcortex. SEEKING system. This system can be sometimes activated

Punishers are stimuli that induce withdrawal behavior. They independentlyofvoluntarygoal-seekingbehavior.Forexam-

functionasnegativereinforcers,byincreasingabehaviorthat ple,addictivedrugslikecocaineandnicotinedirectlyuseand

decreases an aversive outcome. Punishers can induce emo- ‘abuse’thissystem,leadingtotheperpetuationofassociated

tionalstatessuchasangerorfear. activities,suchasthecontinuousquestforthesubstanceand

Rewardsorpunishersarenotalwaysofanexternalorpercep- thepursuitofthesubjectivefeelingofpleasureinducedbyits

tual origin; they may also correspond to a cognitive, mental consumption.Amaladaptiveviciouscycle,knownasdepen-

representationoramemoryreactivationofareward(Alcaro dence,addictionandtoleranceissooninstalled.

andPanksepp,2011).

ornovel(Wittmannetal.,2007),and(b)thecognitive,affective,

orphysiologicalresponsetotheseinternally-generatedrewarding
1.2. Rewardprocessing
or punishing signals while the organism is asleep. This defini-

tioncomesclosetoPanksepp’sconceptoftheSEEKINGdisposition
1.2.1. Generalinformationaboutrewardprocessing
(AlcaroandPanksepp,2011),whichcorrespondstoaninstinctual
Reward processing is one of the major functions of the ML-
affectiveandexploratorydrivetoseekbiologically-importantstim-
DA reward system. This system is responsible for appropriate
uliintheexternalorinternal(‘intrapsychic’)environment(seeBox
responses to external and internal rewards or punishers, for
2andFig.2).
incentive-basedlearning,andforthedevelopmentofgoal-directed

andadaptivebehaviors(HaberandKnutson,2010).Itisregardedas

acentralcomponentinthedevelopmentandmonitoringofmoti- 1.2.2. Braincircuitsofrewardprocessing

vated behaviors in all mammals (see Box 1). Two main types of The brain areas activated during reward processing vary as a

dopaminesignalsmaybedistinguishedwithinthissystem:pha- functionofthetypeofreward,thebehavioraltask,andwhethera

sicversustonicneuralactivity.Aphasicdopaminesignal(bursting reward(orpunisher)isbeinganticipatedordelivered(seeBox1).

activity)isdefinedasabriefincrease(upto2s)indopaminecon- Most of the areas involved during reward processing are inner-

centration in terminal mesolimbic regions (subcortical, like the vated by ML-DA projections, originating mainly from the VTA

NAcc,orcortical),resultingfromaburstfiringofoneormoreVTA (ML-DA reward system; Fig. 1). The VTA, as well as the NAcc,

neurons(Ikemoto,2007).Phasicactivityisinvolvedinrewardpro- constitutes the core of the reward circuit (Hikosaka et al., 2008;

cessing(HeienandWightman,2006;Redgraveetal.,1999;Schultz, Wise,2002).However,somenon-mesolimbicstructuresalsocon-

2010b),andalsorelatestotheprocessofswitchingone’sattention tribute to the processing of rewards, including the orbitofrontal

andbehaviortowardsalientcuesintheenvironment.Ontheother cortex(OFC),thesubstantianigraparscompacta(SNc),thesupra-

hand,atonicsignalisdefinedasaslowchangeindopamineconcen- mammillary nucleus, the midbrain raphe nuclei, and the dorsal

trationintheaforementionedregions,lastingfromtensofseconds striatum (Ikemoto, 2010; Schultz, 2010a). In addition, dopamine

tohoursordays,andsupportingrelativelystableortonicaffective isnottheonlyneurotransmitterrelatedtorewardresponses.Glu-

statesandmotivation(Ikemoto,2007). tamate (Kenny et al., 2009), GABA (Vlachou and Markou, 2010),

The term ‘reward’ can have slightly different uses in neu- serotonin (Higgins and Fletcher, 2003) and acetylcholine (Rada

roscience. On the one hand, reward may primarily refer to etal.,2000;Sarteretal.,2005)caninfluencerewardprocessing,by

reinforcement-learningfunctionsofdopaminesignals,suchasfor actingeitherasmodulatorsofdopaminefunctionorindependently

examplerewardpredictionerror(Redgraveetal.,1999;Schultz, ofdopamine.

1997, 2002, 2010a; Schultz and Dickinson, 2000). On the other Recentneurobiologicalmodelsofrewardprocessingcommonly

hand,rewardmayrelatetomotivationalandappetitivefunctions describe a transfer of information from ventral to dorsolateral

ofthedopaminergicrewardsystem,suchasforexample‘incentive cortico-basal ganglia circuits, which transforms basic reward

salience’ (Berridge, 2007). In this article, reward processing dur- responses and unconditioned responding (mesolimbic pathway)

ingsleepwillspecificallyconcern(a)thespontaneousactivation intoactionplanningandassociativelearning(nigrostriatalpath-

ofinternally-generatedrewardsorpunishersduringsleep,suchas way) (Haber and Knutson, 2010; Ikemoto, 2007). According to

elementsfrommemorythatarerelevant(Montagueetal.,2006) Ikemoto(2007),mesolimbicprojectionsfromtheVTAtotheNAcc,
L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951 1937

Fig.1. Mainascendingmesolimbicandnigrostriataldopaminergicpathways.Abbreviations:ACC:anteriorcingulatecortex;HC:hippocampalcomplex;NAcc:nucleus

accumbens;PFC:prefrontalcortex;VTA:ventraltegmentalarea.

LH,andPFCareactivatedbyincentivestimuliandcanleadtoposi-

tiveaffectandapproachlearning(‘flexible’responses).Ontheother

hand,thenigrostriatalprojectionsfromtheSNctothedorsalstria-

tumandfromtheretoglobuspallidusandfinallytomotorregions

inthefrontallobewouldberesponsiblefortheinitiationofmotor

activityandthetransientactivationoftherewardsysteminwell-

learnedsituations,withpredictableoutcomes(‘habit’responses).

2. IntroducingtheRewardActivationModel(RAM)for

sleepanddreaming

BasedontheevidencethattheML-DAsystemisactivateddur-

ingsleep,theRAMproposesthatthisactivation(a)contributesto

memoryconsolidationmechanismsbyprioritizingtheprocessing

ofinformationwithhighemotionaland/ormotivationalrelevance,

(b)participatesinthemodulationofREMsleepthroughprojections
Fig.2. TheML-DAcomponentoftheSEEKINGsystem.Schematicrepresentation
to REM generating structures (in particular the sublaterodorsal
oftheML-DAprojectionsoftheSEEKINGsystemoriginatingintheVTAofthe
nucleusofthepons),and(c)contributestothegenerationofdreams m idbra in.Elec tricalorche m ical stimulatio nofth eregionsin re dco lori ndu ces
bymean s ofm otivat iona la ndaffective dr ives oftheSEE KI NGsys- exploratio nandapp roa chbehav iors,wherea sr egio nsinblu e colo rare involved

tem (Box 2). These characteristics of the ML-DA system suggest inthelarge-scaleorganizationanddiffusionoftheSEEKINGsignal(Alcaroand
that dream in gmay potentiallypla ya role inlear ningan dmem- Pa nkse pp, 2011). H ere, anterior limb ic regions inc lude the vent romedi al prefro ntal
cortex,orbitofrontalcortex,anteriorcingulatecortex,andtheinsula.Ascending
ory,includingemotionregulationprocesses.Notehoweverthatup
ton owthere islittlee mpiricale videncein supp ortofthi slat ter nSEoEraKdINreGnesrygsitce/mchoalrienenrogitc raenpdre dseenscteedndiinngt hgilsutfiagmuraete.rAgbicb/rGevAiBaAtio pnrso:jeAcCtCio:nasn otef rtihoer

hypothesis. cingulatecortex;NAcc:nucleusaccumbens;VTA:ventraltegmentalarea.(Forinter-

pretationofthereferencestocolorinthisfigurelegend,thereaderisreferredtothe

webversionofthearticle.)
2.1. ActivationoftheML-DArewardsystemduringsleep

Anincreasingnumberofstudiesindicatethatsomekeystruc- Krebs et al., 2011; Wittmann et al., 2007). VTA bursting activity
tures of the rewa rd proce ss ing circ uit, and m ain ly of t he M L-DA inthe ra tw asfirst recordedd ur ing sleepb yMi lleretal. (1983),
rewar d syst em, like the VTA a nd the NAcc are ac tiva ted during wh o s how ed t hat V TA neuro ns fire at the sa me slo w fre quency
sleep.In Fig.3,w epr opo sean inte grat edsum ma ryofthefi ndings durin gSWS, REM sleep andwak ing, bu tth efirin gism oreirreg-
that in di cate th e a ctivation of the ML-D A reward sy stem during ulardu ringR EMs leep.I n20 02,Malo ney eta l.dem on strate dthat
sleep . in ra ts dop amin ergic n eu rons in the VT A are more active dur-

ingREMandwakefulnessthanduringNREM,andaremaximally

2.1.1. VTAburstingactivityisincreasedduringREMsleep activeduringREM(Maloneyetal.,2002).In2007,Dahanetal.con-

VTAburstingactivityisstronglyrelatedtorewardprocessing firmedthattheVTAhasanincreasedburstingactivityduringREM

(Kiyatkin, 1995; Yun et al., 2004). The VTA fires bursts of spikes sleep in rats (phasic dopamine signals), inducing a large synap-

during reward and punisher anticipation (Carter et al., 2009), as ticdopaminereleaseintheNAccshell(Dahanetal.,2007).Inthis

wellasinresponsetostimulusnovelty(BunzeckandDuzel,2006; study, the percentage of spikes firing in bursts was significantly
1938 L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951

Fig.3. SchematicillustrationoftheactivationoftheML-DArewardsystemduringsleep.(A)DuringNREMsleep,theHC–VSactivationunderliesaspontaneousreactivation

(replay)ofreward-relatedneuronalfiringpatternsintheVS,whichinvolvesatransferofnovelty/relevancesignalfromtheHCtotheVTA(seemaintext,Lansinketal.,2008;

LismanandGrace,2005;Pennartzetal.,2011).VTAisactivatedduringthetransitionfromaNREMepisodetoREMsleep,withinductionofbothtonic(HC–VTAprojection)and

phasic(PPT–VTA)increaseofdopamine.Otherreward-relatedstructuresactivatedduringNREMsleepincludetheamygdala,theACCandtheinsula.(B)DuringREMsleep,

increasedburstingactivity(phasicresponse)intheVTA(Dahanetal.,2007)mayrepresentstimulussaliencyandcouldfulfillreward-relatedfunctions,likeprocessingof

stimulus-rewardassociations,novelty-seekingandenhancementoflearningprocedures.DuringREMsleep,severalVTAprojectionsareactivated,includingtheupwardarc

ofhippocampal–VTAloop(dopaminergicinputfromtheVTAtotheHC),theNAcc,theamygdala,theorexin/hypocretinneurons,theACC,andthePFC.Alltheseregionshave

stronganatomicalandfunctionallinkswiththehippocampusandVTA(amongothers).Activationoftheupwardarcofthehippocampal–VTAloopcontributestosynaptic

plasticityandlearningbyenhancinglong-termpotentiation(LismanandGrace,2005).Abbreviations:HC:hippocampus;PFC:prefrontalcortex;PPT:penduculopontine

tegmentalnuclei;VS/NAcc:ventralstriatum/nucleusaccumbens;VTA:ventraltegmentalarea.

higherinREMcomparedtowakingandtoNREM.Thesustained intra-hippocampal infusion of tetrodotoxin (Legault and Wise,

bursting activity was comparable in duration with VTA bursting 2001).Inhumans,brainimagingstudiesalsodemonstratethatnov-

duringmotivatedbehaviorssuchasfeeding,punishment,orsex. eltyactivatestheamygdala,theVTAandtheHC(Bunzecketal.,

Moreover, this VTA activation, which started 10–20s before the 2012; Guitart-Masip et al., 2010; Krebs et al., 2011). The HC can

onsetofREMsleep,wasasimportantinintensityasanactivation activateVTAdopaminecellsviaapathwayinvolvingtheactivation

inducedbytheconsumptionoffood.Becausehighphasicdopamine ofNAcc,whichinturninhibitstheventralpallidum(VP)leading

signalinginthemidbrainhasprimarilybeenlinkedtorewardpro- tothedisinhibitionofdopaminecells(Florescoetal.,2003).This
cessing(S ch ultz ,2010b), stro ngburstin gacti vityof the VTAbe fore pa thw ay(HC→VTA ), whichcorr espon dstothe do wn warda rcof

andduringREMsleepmayputativelyberelatedtoreward-related the‘hippocampal–VTAloop’(LismanandGrace,2005),increases

processes,aswediscussbelow. thepopulationactivityofdopaminergicneurons,modulatestonic

TheincreaseofVTAburstingactivityattheendofanepisode extrasynapticdopaminelevelsin the VTA(Florescoet al., 2003),

of SWS (or NREM sleep) is explained by the activation, during andisrelatedtomotivationalsalience(LismanandGrace,2005)

SWSsleep,ofseveralstructuresthatprojecteitherdirectlyorindi- (Fig. 3A). The upward arc (dopaminergic input from the VTA to

rectlytotheVTA(pedunculopontinetegmentalnucleus,amygdala, theHC)isrelatedtosynapticplasticityandlearningbyenhancing

HC,striatum,hypothalamus,basalforebrain).Studieshavefocused long-term potentiation (LTP) (Adcock et al., 2006) (Fig. 3B). Fur-

mainlyontwooftheseprojectionsites,i.e.theHCandthepeduncu- thermore,thePPT,whichsendsdirectexcitatoryinputstotheVTA

lopontinetegmentalnucleus(PPT)(HolmstrandandSesack,2011; (OmelchenkoandSesack,2006)andplaysakeyroleinthegener-

LismanandGrace,2005).However,itisimportanttonotethatthe ationofREMsleep(Bandyopadhyaetal.,2006;Saperetal.,2005),

amygdala(FudgeandHaber,2001),aswellassleep-relatedstruc- hasbeenshowntobeactivatedspecificallybysalientstimuli(Pan

tures,suchasthesuprachiasmaticnucleus(LuoandAston-Jones, andHyland,2005)andtoparticipateinincreasedburstingactiv-

2009)andorexin/hypocretinneuronsinthehypothalamus(Fadel ityandphasicsynapticdopaminelevelsintheVTA(Florescoetal.,

andDeutch,2002)alsosenddenseprojectionstotheVTA.Asfor 2003).

orexin neurons specifically, their receptors are expressed at the To summarize, several projection circuits may contribute to

surfaceofVTAdopaminergicneurons(Marcusetal.,2001;Narita increase VTA activation during the transition from a NREM

etal.,2006).Inaddition,theorexinsystemisinvolvedbothinthe episode to REM sleep, among which are the downward arc of

regulationofsleepandindrug-seekingbehaviorsandassociated thehippocampal–VTAloopandthePPT(Fig.3A).Thesimultane-

ML-DAactivity,asobservedinanimalstudies(Borglandetal.,2006; oushippocampal–VTAandPPT–VTAactivationcaninducea4-fold

Boutreletal.,2005;Harrisetal.,2005)andinneuroimagingstud- increaseinthenumberofdopaminergicneuronsfiringinbursts

iesperformedonorexin-deficientnarcolepticpatients(Ponzetal., (LodgeandGrace,2006).Notethatcircadianfactors(CLOCKgene)

2010a;Schwartzetal.,2008). seemalsotobeimplicatedinthisactivation(Roybaletal.,2007).

LismanandGrace(2005)proposedthattheVTAandhippocam-

pusformafunctionalloopdesignedtocontroltheentryofrelevant 2.1.2. IncreasedactivityoftheNAcc,PFC,amygdalaandACC

informationintolong-termmemory.Belowweintroducethemain duringREMsleep

componentsofthisloopbecauseitconstitutesaveryusefulmodel ActivityintheNAccisproportionaltothemagnitudeofantic-

onwhichwewillrelyfortheintegrationofexistingdataaboutthe ipated reward (Knutson et al., 2001) and is the greatest when

activation of the reward system during sleep. In this model, the salience(e.g.uncertaintyaboutoutcomes)ismaximal(Cooperand

activationoftheVTAinducedbynovelsalientstimuliandrequired Knutson,2008).Exposuretobothprimaryandsecondaryrewards

formemoryconsolidationistriggeredmainlybytheHCandthe also increases activity in the PFC, particularly in the ventrome-

PPT. This hypothesis is based on the observation that novelty- dialPFC(vmPFC)(HaberandKnutson,2010).TheACCisanother

inducedincreaseinextracellulardopaminelevelscanbeblockedby important component of reward processing as it is related to
L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951 1939

predictingreward,toassigningapositiveornegativevaluetofuture in these patients provide further support for the activation of

outcomes(Takenouchietal.,1999),andtoreward-baseddecision- reward-seeking mechanisms during sleep. Moreover, the orexin

making(Bushetal.,2002).In2005,Gottesmann’sgroupshowed neurons,whichplayanimportantrolebothinsleep–wakeregu-

thattherewasanincreaseinextracellularlevelsofdopaminein lation(Saperetal.,2005)andinmotivatedbehaviors(Harrisetal.,

theNAccduringREMsleep,notsignificantlydifferentfromwaking, 2005; Thompson and Borgland, 2011) have occasional burst dis-

butsignificantlyhigherthanNREMsleep(Lenaetal.,2005).Dur- charges during REM sleep (Mileykovskiy et al., 2005; Takahashi

ingREMsleep,increasedactivityisalsoobservedinthePFC(Lena et al., 2008), with levels comparable to those of quiet waking

et al., 2005), in the human vmPFC, amygdala, and ACC (Maquet (Kiyashchenko et al., 2002). This phasic activation, which might

et al., 1996, 2000). Dense ML-DA projections from the activated berelatedtoorexinprojectionstoandfromtheVTA(Nakamura

VTA(Dahanetal.,2007)tothesesitescould,atleastinpart,explain etal.,2000),couldexpressmotivationaland/oremotionalprocess-

theseactivations. ingduringREMsleep.

2.1.3. ActivationofHCandVSreward-relatedneuronsduring
2.2. Sleepdeprivationandrewardsystem
SWS
In2004,Pennartz’steamdemonstratedthataspontaneousreac-
Theroleofsleepinrewardprocessingcanbeprovedindirectly
tivation (replay) of neuronal firing patterns may occur in the VS
by studies of sleep deprivation (SD). Steiner and Ellman (1972)
duringSWSafterarewardsearchingbehavioroftherat(Pennartz
demonstratedthatREMsleepdeprivation(REMSD)causedratsto
etal.,2004).In2008,thesamegroupconfirmedtheactivationofVS
seekmoreofapreviouslytrainedrewardingstimulation.Moreover,
reward-relatedneurons(sensitivetoeitherthepresenceorabsence
afterallowingratstoself-stimulate,thetypical‘rebound’increase
ofreward)duringSWSafterarewardsearchingbehavior(Lansink
in REM sleep following REMSD was decreased. Similarly, acute
et al., 2008). This spontaneous reactivation during SWS was as
sleepdeprivationinratsincreasedgoal-directedbehaviorstoward
strongasduringquietwakefulnessandcouldnotbedetecteddur-
cocaine(Puhletal.,2009).Inhumans,SDincreasestheriskofsub-
ingREMsleep.Itappearstobeinducedbyandtemporallyaligned
stanceabuse(Shibleyetal.,2008;Wongetal.,2009)andappetitive
tohippocampalripples(100–300Hzoscillations)andcouldcon-
behavior(Benedictetal.,2012).Thisincreaseinimpulsivityand
tributetolinkingamemorytracetoamotivationalvalueduring
rewardseekingpost-SDmayreflectacompensatorymechanismto
sleep(Lansinketal.,2009;Pennartzetal.,2011;SingerandFrank,
adjustforthedownregulationofD2andD3receptorsintheven-
2009),aswellastothetransferofanoveltysignalfromtheHCtothe
tralstriatumimmediatelyafterSD(Volkowetal.,2012).Decreased
VTA(LismanandGrace,2005).Thismechanismmaythususethe
D2/D3 receptor availability (Volkow et al., 2012) and attenuated
downwardarcofthehippocampal–VTAloop(Fig.3A).Inhumans,
representationsofparametricvalueintheSN/VTA,bilateralinsula,
HCandVSactivationduringNREMsleephasbeendemonstrated
vmPFCandparietalcortex(Menzetal.,2012),mayalsoaccount
byneuroimagingstudies(e.g.Peigneuxetal.,2004;Nofzingeretal.,
fortheimpairmentinperformance,rewardlearninganddecision
2002),althoughitremainsuncertainifactivationinbothHCand
makingafterSD.Inlinewiththisinterpretation,Hanlonetal.(2005)
VSistemporallycoordinated.
demonstratedthatREMSDreducestherateofrespondingtothe
Itshouldbenotedthat,inrats,SWSissynonymoustohuman
acquisitionandmaintenanceofanoperanttaskforfoodrewardin
NREMsleep(Frankenetal.,1998),andisnotrestrictedtohuman
rats,whichmightbeduetoasuppressionofdopamineactivityin
SWS(stageN3).Inhumans,thehippocampalripplesaretempo-
theNAccduringREMSD(Hanlonetal.,2010).Inaddition,totalSD
rallycoupledwiththesleepspindles(Clemensetal.,2007,2011),
candisruptthereconsolidationofmorphinerewardmemory(Shi
whicharemostpronouncedinstageN2butalsooccurinSWS(De
etal.,2011).Inhumans,studieshavereportedthatinsufficientsleep
GennaroandFerrara,2003).Thus,theaforementionedactivations
isassociatedwithchangesinreward-relateddecisionmaking:peo-
ofHCandVSinratsmayprimarilybutnotuniquelyconcernstage
pletakegreaterrisks(HarrisonandHorne,2000;McKennaetal.,
N2inhumans.Forexample,arecentstudyshowedthatsleepwalk-
2007),arelessconcernedwithnegativeconsequencesofriskydeci-
ingpatientsmayre-enactduringNREMsleepamotortaskinwhich
sions(CheeandChuah,2008;Venkatramanetal.,2007,2011),and
theyweretrainedbeforesleep,thusprovidingdirectevidencefor
overestimatepositiveemotionalexperiences(Gujaretal.,2011).
theoff-linereplayof newlyacquiredinformationduringSWS in
In2009,Holmetal.showedthatinbothrewardanticipationand
humans(Oudietteetal.,2011).
outcomephasesofacard-game,adolescentswithfewerminutes

asleep and later sleep onset time exhibited less caudate activa-
2.1.4. Activationofotherreward-relatedstructuresand
tion(Holmetal.,2009),astructureimplicatedinlinkingreward
mechanismsduringsleep
tobehaviorandlearning(HaberandKnutson,2010).Collectively,
Apartfromtheregionsalreadymentionedabove,someother
these recent data suggest that less sleep may impact on neural
reward-relatedregionsarealsofoundtobeactivatedduringNREM
systems of reward in ways that exacerbate behavioral problems
andREMsleep.Inhumans,theposteriorinsula,ACC(Schabusetal.,
(e.g.increasedrisk-taking).Thus,sleepdeprivationmayhavemajor
2007),andtheamygdala(Nofzingeretal.,2002)werefoundtobe
healthimplicationsinadolescentsandadults,byalteringreward
activatedduringNREMsleep.DuringREMsleep,theamygdalaand
andemotionalprocessing.
theHCarealsoactivated(Braunetal.,1997;Maquetetal.,1996,

2000; Nofzinger et al., 1997). Activation in the HC is consistent

withanimal(Popaetal.,2010;Winson,1972)andhuman(Cantero 3. Rewardactivationandmemoryprocessingduringsleep

etal.,2003)datashowingthattheHCexhibitsathetarhythmdur-

ing REM sleep. The theta rhythm is common when animals are The data reviewed above suggest that many components of

exploringtheirenvironmentbysniffing,anexploratorybehavior theML-DArewardsystemareactivatedduringsleep.Whatcould

which is present during REM sleep (Panksepp, 1998; Seelke and be the functional meaning of activating a system devoted to the

Blumberg,2004).Similarexploratoryandinstinctualbehaviorsin processing of rewards while the organism sleeps and is thus

humans are observed in parasomnias: locomotion in sleepwalk- typically deprived of interactions with its environment? Based

ing,aggressioninREMsleepbehaviordisorder,sexualbehaviors on recent accumulating evidence showing that memory replay

inconfusionalarousals,andfeeding,chewingorswallowinginthe and consolidation processes are present during all sleep stages

sleep-relatedeatingdisorder(Schencketal.,2007;Vetrugnoetal., (DiekelmannandBorn,2010;Maquet,2001;Stickgold,2005),we

2006;Winkelman,2006).Hence,thespecificbehaviorsobserved suggest that activation of the ML-DA system during sleep may
1940 L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951

relate to the reprocessing of memories with a high emotional of a reward can boost offline learning mechanisms during sleep

or motivational relevance (e.g. related to current or future chal- (FischerandBorn,2009).

lenges).Afunctionallinkbetweenmemoryandrewardprocesses

is substantiated by strong mutual interactions between the hip-

pocampus and the VTA: while memory processing may lead to 4. RewardactivationandREMsleepgeneration
reward ac tivat ion (‘mem ory→ reward’ d irection), f ollow ing t he

downwardarcoftheHC–VTAloop(Fig.3A),VTAactivitycaninturn The role of dopamine in the sleep/wake cycle has been stud-

lead to the reactivation of memories and learning enhancement iedinrodentssincemorethan30years(e.g.Montietal.,1988).
(‘rew ar d→ memory’dir ect ion)throug hac tivationo ftheupward Sm all doses of a D2 prefe rring dop amin e ago nist (a po mor phine,

arc of the HC–VTA loop (Fig. 3B) (Adcock et al., 2006; Fischer bromocriptine,quinpiroleorpergolide)decreasewakefulnessand

and Born, 2009; Lisman and Grace, 2005; Shohamy and Adcock, increasethedurationofSWSandREMsleep(Dzirasaetal.,2006;

2010). Montietal.,1988,1989).Ithasbeenproposedthatthesesoporific

effects are related to an activation of the D2 autoreceptors in

3.1. Memoryprocessingleadingtorewardactivationduringsleep theVTA(Bagettaetal.,1988),whichinducesaninhibitioninthe

mesolimbicpathway(Szabadi,2006).Ontheotherhand,moder-

OneofthemainhypothesesoftheRAMisthatrewardprocessing ate doses of the D3 agonist pramipexole were found to increase

andmemoryconsolidationinteractduringsleep.DuringSWS,this SWS and REM sleep via D3 receptors (Lagos et al., 1998), which

interactioninvolvesthedownwardarcoftheHC–VTAloop(Fig.3A). arenotautoreceptors(inhibitory).Similarresultshavebeenfound

During this sleep stage, activation of the HC and VS enables the in humans. A small dose of the D2 preferring agonist pergolide

formationofamemorytracecomprisingbothcontextualandmoti- (Staedt et al., 1997) or the dopamine reuptake inhibitor, bupro-

vationalcomponents(Lansinketal.,2009)andcouldbeassociated pion(Nofzingeretal.,1995;Ottetal.,2004)inducedanincrease

withtonicdopaminereleaseintheVTA(Florescoetal.,2003).The ofREMsleepdurationanddensity.Amoderatedoseofpramipex-

coordinatedreactivationofbothHCandVSduringSWSprovidesa oleproducedareductionofsleeplatency,anincreaseoftotalsleep

possiblemechanismfortheconsolidationofmemory-rewardasso- time(Micallefetal.,2009),andoftimeinREMsleep(Jamaetal.,

ciations (Lansink et al., 2009). In particular, the activation of VS 2009).Together,thesedatashowthatadministrationofD2orD3

reward-related neurons during SWS in rats seems important for agonistscanfacilitatesleep(includingREMsleep),buttheexact

selecting memories with a high storage priority (Lansink et al., underlyingmechanismsarenotyetwellunderstood.Actually,in

2008)andmayleadtotheoptimizationofadaptedbehaviorduring 2006,Dzirasaetal.demonstratedthatahyperdopaminergicstate

wakefulness.Consistentwiththishypothesis,declarativememory isnecessaryforthegenerationofREMsleep.ItseemsthatbothD2

andskillconsolidationobservedduringsleepseemtobeguidedby autoreceptorsandD3postsynapticreceptorsareimplicatedinthis

emotionalrelevanceand/ormotivationalbiases(Cohenetal.,2005; mechanism.

Fischer and Born, 2009; Pennartz et al., 2004; Sterpenich et al., Can reward processing also influence the regulation of REM

2007,2009;Wilhelmetal.,2011).Reciprocally,activememorypro- sleep? Low doses of cocaine, a drug that stimulates the ML-DA

cessingcouldalsoexplainthereward-relatedsustainedbursting reward system, can increase REM sleep in rats (Knapp et al.,

activityoftheVTA(Dahanetal.,2007)andNAcc(Lenaetal.,2005) 2007). Moreover, in decerebrate cats the introduction of a tube

duringREMsleep.TheRAMsupportsthatphasicVTAdopaminesig- forgastricfeeding(Villablanca,1966),orothercutaneous,propri-

nalingduringREMsleepfavorsanoff-linereplayofrecentmemory oceptive and sensory stimuli like repetitive sounds (Jouvet and

tracesduringthissleepstage(LouieandWilson,2001;Peigneux Delorme, 1965; Villablanca, 2004), can induce REM sleep. This

et al., 2003; Walker and van der Helm, 2009). These memories phenomenonmaybearsomeanalogytocataplexyinnarcoleptic

couldservebothassalientandnovelstimuliforthePPTandVTA, patients. Indeed, episodes of REM-like atonia (cataplexy) during

becauserecentrelevantmemories(e.g.emotionalevents,current wakefulness can be triggered by emotional experiences, includ-

concerns) are activated in the absence of associated contextual ingtheanticipationofreward(Anic-Labatetal.,1999;Ponzetal.,

cuesfromwakefulness,andofcognitivecontrolfromdorsolateral 2010a;Schwartzetal.,2008;SturzeneggerandBassetti,2004),and

PFC (dlPFC) during REM sleep (Fosse et al., 2003; Maquet et al., cataplexyisalsotriggeredbyfeeding,innarcolepticdogsandorexin

1996; Schwartz, 2003; Schwartz and Maquet, 2002). Moreover, knockoutmice(Clarketal.,2009;Mitleretal.,1974;Reidetal.,

duringREMdreaming,dreamelements(includingbizarrefeatures 1994).

indreams;RevonsuoandSalmivalli,1995)couldalsoactasstimuli Based on these findings, we suggest that the activation of

thatpotentiallydriveVTAandNAccburstingactivity. emotionally-relevantinformationsuchasrewardcanpotentially

influencethegenerationofREMsleep.Morespecifically,increased

3.2. Rewardactivationleadingtomemoryformationduringsleep dopaminergicactivityintheVTAcouldmodulateactivityinbrain

regionsthatarecriticalforthegenerationofREMsleep,inpartic-

The activation of the VTA (predominantly during REM sleep) ularthesublaterodorsalnucleus(SLD)ofthepons,whichisakey

maycontributetolong-termpotentiation(LTP)andmemoryfor- structureforgeneratingREMsleep(Boissardetal.,2002;Clement

mationintheHC(Adcocketal.,2006;LismanandGrace,2005).This etal.,2011;Fortetal.,2009).Indeed,VTAprovidesefferentpro-

mechanism,whichisconsiderednecessaryforassociativelearning jectionstotheSLD,thuspotentiallygatingtheonsetofREMsleep

(Adcock et al., 2006), is served mainly by the upward arc of the (Boissardetal.,2003).TheRAMsupportsthat,attheendofanNREM

hippocampal–VTAloop,namelydopaminergicprojectionsfromthe episode,thedownwardarcofthehippocampal–VTAloop,which
VTAtotheHC(Fig .3B). LTPint heHCisdepen dentondo pami ne: involves acti vememory proc es sing (‘memory→rewar d’dire ction;

it is blockedby D1 antagonists(Bachet al., 1999) andenhanced Fig.3A),leadstoanactivationoftheVTA,whichinturnsprojects

byD1agonists(Lietal.,2003;Swanson-Parketal.,1999).Besides, toSLDandgatestheonsetofREMsleep.Inaddition,theLH-to-VTA

dopaminetransmissioncanregulategeneexpressionandleadto orexinpathwaycouldalsobeimplicated.Indeed,occasionalburst

permanentsynapticchanges(Nestler,2004;Wolfetal.,2003).A dischargesoforexinneuronsduringREMsleep(Takahashietal.,

recentbehavioralstudyinhumansshowedthatsleepcaninduce 2008),withtheirprojectionstoVTAdopaminergicneurons(Fadel

asignificantlystrongerimprovementofamotorfingersequence and Deutch, 2002), may contribute to reward-driven dopamin-

associatedwithananticipatedmonetaryreward(‘sequenceto-be- ergic induction or maintenance of REM sleep (a hypothesis that

rewardedaftersleep’),thussupportingthatthemereexpectation wouldrequireandmayinspirefutureexperimentalinvestigations).
L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951 1941

inbrainconnectivityacrossallsleepstages(e.g.Massiminietal.,

2005,2010).

The specific pattern of regions activated during REM sleep in

humansisconsistentwithsomemainfeaturesofdreamingexpe-

rience,aswebrieflyreviewbelow(Hobsonetal.,1998;Schwartz,

2004;SchwartzandMaquet,2002)(Fig.5).

AfirstsetofactivatedregionsconsistsofthePPT,thalamus,basal

forebrain,aswellasoflimbicandparalimbicstructures,including

bilateralamygdala,hippocampalformation,mPFCandACC(Braun

etal.,1997;Maquetetal.,1996;Nofzingeretal.,1997).Activationof

theamygdalamayreflectintenseemotions,inparticularfearand

anxiety,oftenexperiencedindreams(Smithetal.,2004).Amyg-

dala,mPFCandACCactivitycouldalsosubserveemotionregulation
Fig. 4. Activation of ML-DA reward system during NREM sleep and the generation proce sses( e.g. extin ctionof fear; Fue tal.,2007 ;Pace-Sc hottetal.,
ofREMsleep.AccordingtotheRAM,tonicdopamineresponseinducedbythedown-
2009).ConsistentwiththeRAM,activationoflimbicandparalim-
wardarcofthehippocampal–VTAloopmaycontributetothegenerationofanREM
sleep epi so de. Thedownwardarc oft hehi ppocampa l–V TA loop,which i nvo lves bic regions suggests that REM sleep could favor the reactivation
active memory pro cessing,lead sto an act ivationoftheVTAa tthe endofa nNREM and process ingofre levan t,nov eland emot ionall ysa lientmemo-

episode.VTAwouldthenprovideanexcitatoryprojectiontotheSLD,whichiscon- ries(Sterpenichetal.,2007,2009;Wagneretal.,2001;Walkerand
sidered a s the REM s leep generato r ( red dashed arrow). A h yp erd opam inerg ic state van derHelm,2 00 9). Thisp attern ofactiv at ion would alsoal low
relatedtorewardprocessingcouldthuspotentiallycontributetothegenerationof
the incorporation of recent memories in dreams, which is com-
RbeEMim spleliecpa.t Perdo.jAebctbiroenvsia ftrioomns :orHeCx:inh nipepuorcoanms opru fsr;oRmE Mth:e raampiydgdeyaelam too VveTmA ceonutlsdl eaelspo; mon ly observed i n d ream s tudies (Sch re dl and H ofman n, 2003;

SLD:sublaterodorsalnucleusVS:ventralstriatum;VTA:ventraltegmentalarea.(For Schwartz,2010).Whencomparedtowakefulness,severalregions
inter pretationofthe referenc est ocolor inthisfig urele gend,th ereaderis refe rred implicated in ex ecutive and atten tio nal functions during wake-

tothewebversionofthearticle.)
fulnessaresignificantlydeactivatedduringREMsleep,including

thedlPFC,OFC,posteriorcingulategyrus,precuneus,andtheinfe-

riorparietalcortex(Braunetal.,1997;Maquetetal.,1996,2000,
Finally,itwasalsosuggestedthatprojectionsfromtheamygdala
2005;Nofzingeretal.,1997).Deactivationsintheseregionsmay
toSLDmaybeimplicatedinemotion-relatedinductionofREM-like
causedisorientation,illogicalthinkingandimpairedworkingmem-
features, such as muscle atonia (cataplexy) in narcolepsy (Luppi
ory in dreaming (Hobson et al., 1998; Schwartz, 2004; Schwartz
etal.,2011).
andMaquet,2002).Finally,otheractivatedregionscompriseasso-
TheVTAwasuntilrecentlythoughttobeadopaminergicstruc-
ciative sensory regions and the parieto-temporo-occipital (PTO)
ture predominantly involved in promoting wakefulness (Bagetta
junctionwhichcouldcontributetothegenerationofsensoryand
et al., 1988; Rye, 2004; Szabadi, 2006). However, there is some
re cent indica tiont hatN- methyl-d -aspart ate(NMDA )-indu ce dVTA spatial imagery in dreams (Braun et al., 1997, 1998), and activa-
tioninmotorcircuits(includingthecerebellumandbasalganglia,
lesionsincatsproduceanincreaseinwakefulness(ratherthanan
Braunetal.,1997;Maquetetal.,2000)isconsistentwithdreamed
increaseinsleep)(Vetrivelanetal.,2010).Furtherevidenceforthe
motoractions.
roleoftheVTAinREMsleepcomesfromthecaninemodelofcat-
Changes in dreaming can also be observed after focal brain
aplexy:administrationofD2/D3agonistsintheVTAofnarcoleptic
lesions.In1997,Solmsestablishedthefirstnosologyofdreamdis-
dogsinducedsleepattacksandaggravatedcataplexy,whoseunder-
ordersbasedonthestudyof361neurologicalandneurosurgical
lyingmechanismimplicatestheactivationofREM-onSLDneurons
patients(Solms,1997).Approximatelyonethirdofthesepatients
(Luppietal.,2011;Reidetal.,1996).
reported a global cessation of dreaming subsequent to the brain
Fig.4showshowactivationoftheVTAattheendofaNREM
damage,whichpredominantlyinvolvedalesioninthePTOjunction
episode (stage N2 in humans) may result in an enhancement of
(seealsoBischofandBassetti,2004;Cathalaetal.,1983;Murrietal.,
REMsleep.
1984).GlobalcessationofdreamingafteralesioninthePTOregion

isconsistentwiththeideathatthisregionmaysubservecognitive

5. Rewardactivationanddreaming functionsthatarenecessaryfordreamingsuchasvisualimagery

andspatialcognition.Cessationofdreamingwasalsoobservedin

5.1. Theneuralcircuitsofdreaming a few patients with lesions in the white matter surrounding the

frontal horns of the lateral ventricles, in about the same region

Dreaming is a state of consciousness in which internally- that was targeted in prefrontal leucotomy (Bradley et al., 1958).

generated sensory, motor, emotional and cognitive experiences Lesionsinthisregion,whichprovokedreamcessationbutmaintain

developintoactionsandimaginaryplots(Desseillesetal.,2011). REMsleep(Solms,2000),disrupttheML-DAconnectionsfromthe

Our knowledge about the neural bases of dreaming derives pri- VTAtotheshelloftheNAcc,amygdala,HC,ACCandfrontalcortex

marilyfromthestudyofREMsleep,becausethissleepstagehas (insularandmedialOFC,medialfrontalcortex,vmPFC).Thiscir-

initiallybeenlinkedtodreamingactivity(DementandKleitman, cuitcorrespondstothemesolimbiccircuitsoftheSEEKINGsystem

1957). Moreover, dream reports are on average more vivid and asdescribedbyPanksepp(AlcaroandPanksepp,2011;Panksepp,

bizarre,withmorecomplexnarratives,i.e.aremore‘dream-like’, 1982; Panksepp, 1998) (see Box 2). Solms proposed that a REM

after awakenings from REM than from NREM sleep (Fosse et al., state is not in itself mandatory for dreaming; instead dreaming

2001;StrauchandMeier,1996).Earlyneuroimagingstudieshave wouldrequiretheactivationoftheSEEKINGsystem(Solms,2000).

revealed that the distribution of brain activity during REM sleep Severalotherobservationsconvergetofurthersupporttheroleof

isnothomogeneousbutischaracterizedbyspecificactivationand thissystemindreaming.PrefrontalleukotomytargetingthevmPFC

deactivationpatterns(e.g.Braunetal.,1997;Maquetetal.,1996; inpatientswithschizophreniacausedanhedoniaandcessationof

Nofzingeretal.,1997).Morerecentstudiesusingimagingmeth- dreams(Jusetal.,1973).HyperactivationoftheSEEKINGsystemby
odswithi ncr eas inglyh igher tempor alands patial resolutio n(e.g. theadm inis tra tio nofdo paminergicagen ts like l-DOPA (Nausie da

functionalMRI,high-densityEEG)havestartedtodescribemore et al., 1982; Sharf et al., 1978), amphetamines (Thompson and

transientchangesinbrainactivity(e.g.Dang-Vuetal.,2008)and Pierce, 1999), bupropion (Balon, 1996), and dopamine agonists
1942 L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951

Fig.5. FunctionalneuroanatomyofhumanREMsleep.BrainregionsmoreactivatedduringREMsleepareshowninredandlessactivatedinblue.Thisfigurecombines

theresultsfromstudiesusingPETimaging(Braunetal.,1997,1998;Maquetetal.,1996,2000;Nofzingeretal.,1997).Abbreviations:ACC:anteriorcingulatecortex;dlPFC:

dorsolateralprefrontalcortex;HC:hippocampus;mPFC:medialprefrontalcortex;OFC:orbitofrontalcortex;PPT:penduculopontinetegmentalnuclei;REM:rapideye

movementsleep.(Forinterpretationofthereferencestocolorinthisfigurelegend,thereaderisreferredtothewebversionofthearticle.)

(Pinteretal.,1999;ThompsonandPierce,1999)caninducevivid and motivational value for the individual are privileged to be

dream-like experiences. By contrast, hypoactivity of this system reactivated during sleep, and may thus amplify the activation of

for example after administration of D2 antagonists is associated theML-DAsystemandoftheSEEKINGsystem.

withareductioninvividdreaming(GaillardandMoneme,1977),

althoughthisfindingwouldneedreplication.Finally,aswepre- 5.3. ML-DArewardsystemactivationanddreamcontent
viously m enti oned, r eward- relate d componen ts of th e S EEK ING

systemareactiveduringREMsleep,includingtheVTA,VS,amyg- LesionstudiesincatsinwhichREMsleepatoniawasprevented
dala,HC ,AC C,and vmPFC . revealedt hatthe se anim a lsexhi bita specifi crang eof behaviors
H owe ver,i tsho uldbenotedthateffectsofdopamineondream- during R EM s leep, such as explorat io n, fear, anger, an d groom-
ingmayinvol ve intera ctio nswi thot herneu ro modulator ys ystems, ing be havio rs (‘on eiric beh aviors’, Jouv et an d Delo rme , 1965).
like basa lforebr aincholinerg icce llstha tactivatecortical andlim- Inte restingly, t hese beh aviors could be dri ven b y emotion al and
bic struct ures (Perr y and Pigg ott, 2 000 ). Lesion s result ing f rom motivationalc ompo nentsofth eSEEK ING system (P anksepp,19 98).
leu cotomy ma y alter not only do pamine rgic, but also cho liner- Similarly, pat ients with R BD m ay act ou t their dreams, of ten in
gicandnor adren ergic effe renta ndafferentcon nec tions (Doricchi the form of aggres sive-v iolen t mo vem ent s mim icking a ttacks or
and Vio lani, 2000). Fi nally, dr eam ing enha ncement, as induced defe nse b eh aviors, with non-v iolent beha viors being compa ra-
byd opamine rgicag ents,can alsobeo bservedinpati ent sreceiv- tivelyle ssfrequent (e.g.O udietteetal .,2009).Dr eamre portsalso
ing anoradrenerg icbeta- rece ptor blo cker(Tho m psonand Pierce, appea rto bebiased tow ardconte nt wi themo tional andmot iva-
199 9 )orcholinester aseinhibitors( Zadra,1 996).Conseq uen tly,the tionalv alu e( e.g.soc ializing ,fighting ,ors exualactiv ity) andless
intera cti on of dopamin ergic func tions w ith oth er neuromodu la- orient ed tow ard life content with no su ch par ticular v alue. For
torysystem sm ustbetakeni ntoconsid eratio nwhe nstudyingthe example ,dailyro utin eactiviti essuc ha stypin g,washing dishe s,or
neur obiology ofdr eam ing. buyingfo odat thesupe rmarket, aren ot frequen tindrea mrepor ts

(Schredl,2010).

5.2. ML-DArewardsystemactivationanddreamgeneration SustainedactivationoftheML-DArewardsystem(inparticular

the VTA) during REM sleep may favor the activation of stimu-

What force would activate the ML-DA reward system dur- lus representations or behaviors of high motivational relevance,

ing sleep and thus induce dreaming? The RAM proposes that which would induce instinctual exploration as well as approach

dreamingmayresultfromtheactivationoftherewardandemo- andavoidancebehaviors.Forexample,pleasantandpositivecon-

tional components of the SEEKING system by ‘off-line’ memory tentofadream(e.g.winningagameorhavingsex)wouldconstitute
and co gnitive proc ess es d uring slee p (‘mem or y→rewar d’ direc- a re wa r ding (a ppro ach-pro n e) stim u lus, wh erea s thre at-related

tion). This is in agreement with Wamsley and Stickgold (2010) content (e.g. being chased or attacked) would be an aversive

whorecentlyproposedthatdreamingmayreflectsleep-dependent (avoidance-prone)stimulus.NAccandVTAmaybeactivatedinde-

memory processing during sleep, as well as with models stating pendentlyoftheemotionalvalenceofthedreamcontent,because

thatdreamsareinfluencedbywakingemotionalconcernsofthe bothstructuresarefoundtobeactivatedduringbothrewardand

sleeper(Cartwrightetal.,2006;Mancia,2004;NielsenandLevin, punisher anticipation (Carter et al., 2009; Delgado et al., 2008).

2007;Schredl,2010).Substantialmesolimbicreward-systemacti- Motivational and emotional content may be more prominent in

vationduringREMsleepwouldthusexplainwhythisstagemaybe REMthaninNREMdreaming(Smithetal.,2004).Thisisconsistent

consideredasa‘dreamfacilitator’.Ontheotherhand,activationof withthefindingthatseverallimbicandML-DAregionsareselec-

contextualandemotionalmemoriesbyreward-relatedstructures tivelyactivatedduringREM,withamygdalaactivityandburstfiring

during NREM sleep (amygdala, ACC, hippocampus, ventral stria- intheVTAbeingsignificantlyhigherinREMcomparedtoNREM.

tum)couldalsoexplainNREMdreaming.Wedonotknowyetwhat ForSmithandcollaborators‘theprimaryroleofthelimbicsystem

theconditionsareforamemorytracetofacilitatetheproduction duringsleepmaybetoactivatethemotivationalcharacteristicsof

ofadream.Wehypothesizethatmemorieswithhighemotional thesleepexperience’(Smithetal.,2004,p.502).
L.Perogamvros,S.Schwartz/NeuroscienceandBiobehavioralReviews36(2012)1934–1951 1943

5.4. LinksoftheRAMtootherdreamtheories behaviorsordrives(suchasfeeding,mating,fighting,fleeing,etc.)

ortomemoryelements(suchascurrentconcerns),aswellasto

Dreamsmayfulfillimportantpsychologicalorcognitivefunc- novel cues (e.g. novel combinations of elements from memory).

tions. Many theories have been proposed in relation to such Thus,whiletheRAMsupportsthattheoff-linereplayofamem-

functionsofdreaming.Below,weshowhowtheRAMmayrelate oryisresponsibleforthegenerationofadream(seeSection5.2),it

anddifferfromsomeofthesetheories,inparticularFreud’stheory alsoclaimsthatthecontentandfunctionofthisverydreamdoes

ofdreams,Hobson’sAIMtheory,andRevonsuo’sthreatsimulation notonlyrelatetoepisodicmemories,assuggestedbyclassicpsy-

theory.NotethatbyconfrontingtheRAMwiththesetheories,our choanalyticaltheoriesbutalso,andimportantly,toprobableand/or

aimistobettercharacterizesomemainfeaturesoftheRAMbut novelexperiences.Thismechanismwouldpotentiallycontributeto

nottoprovideanexhaustivereviewoftheoriesaboutdreaming. adaptivebehaviorbyimprovingfutureperformanceandemotion

WhileactivationofML-DAcircuitsduringdreamingmightbe regulationprocesses.

consistentwithsomeofthemostimportantpropositionsofthese

theories, the RAM places the dopaminergic-driven motivational 5.4.2. Activation-input-modulation(AIM)modelandRAM

andemotionaldrivesatthecenterofthedreamingstate.Wesug- The AIM was introduced by Hobson and offers a charac-

gest that activation of the ML-DA system allows the dreamer to terization of the brain states according to three main factors:

becomeacquaintedwithdiverseconfigurationsandrewardingval- activation(totalandregionalbrainactivitylevels),input–output

ues of a simulated internal environment and thereby establish gating(externalversusinternalsource),andmodulationorinfor-

realistic and adaptive expectations, which could be used in real mationprocessingmode(cholinergicversusaminergic)(Hobson,

lifesituations.Thus,dreamsnotonlyresultfromtheactivationof 1988, 1999). Within the AIM framework, REM sleep and dream-

memoryprocessesduringsleep,buttheymayalsocontributeto ing are both characterized by high levels of activation, internal

offlinememoryconsolidationandlearning.Thelatterhypothesis input,andaminergicmodulation.Morespecifically,dreamsresult

issupportedbyrecentstudiesondreaming(Wamsleyetal.,2010), frominternally-generatedsensoryactivationoriginatingfromthe

ontheSEEKINGsystem(AlcaroandPanksepp,2011),andonVTA’s brainstem(ponto-geniculo-occipitalwaves),latercombinedwith

roleinmemoryandlearning(Adcocketal.,2006). memoryandcognitivefunctions.Hence,accordingtothelastrevi-

sionofthismodel,“dreamingisanindispensable—ifsometimes

5.4.1. Freud’spsychoanalytictheoryofdreamsandRAM misleading—subjectiveinformantaboutwhatthebraindoesdur-

Inbrief,thetheoryproposedbyFreudin1900,inhisbook“The ingREMsleep”(Hobson,2009,p.805).Moreover,REMsleepand

InterpretationofDreams”(Freud,1995),isbasedonthefundamen- dreamingwouldprovideuswithavirtualrealitymodel,whichmay

talassumptionthatthepurposeofeachdreamisawish-fulfillment: supportlearningandmemory,andmayinfluencethedevelopment

dreamsareattemptsoftheunconscioustoresolvea,usuallydis- andmaintenanceofwakingconsciousness.

turbing, conflict from the recent or distant past. A censorship Concerningtheroleofthedopaminesystemindreaming,Hob-

mechanismrendersdreamsalteredandincomprehensible,because sonhassuggestedthattherelativepersistenceofdopaminerelease

theimmediategratificationofawishisusuallyimpossibleorcould duringREMsleepcouldexplainsomeofthecognitivecharacter-

beunwiseorunsafe(‘pleasureprinciple’opposingthe‘realityprin- isticsofdreaming(suchasvisualhallucinations,bizarreness,and

ciple’).Thismechanismmakesanxiety-arousingwishestolerable lackofself-reflectiveawareness),whichmayresemblepsychosisin

for the conscious. Furthermore, because dreams would preserve mentalillnesses(Scaroneetal.,2008).Whilebothmodelsarenot

sleepbygivinganacceptableformtothesatisfactionofadesire incompatible,amaindifferencebetweenHobson’sviewandour

(mainlysexual)theyareconsideredbyFreudtobethe‘guardians’ proposalinthecontextoftheRAMconcernsthefunctionalroleof

ofsleep. ML-DAsystemactivationduringsleep.Inparticular,theRAMpro-

On the one hand, the RAM supports some of Freud’s claims videsthefirstintegratedanddetaileddescriptionofhowactivation

aboutdreamingbecauseitproposesthatactivationoftheSEEKING ofthedopaminesystemandlimbiccircuitsduringsleepanddream-

reward-relatedsystem(urgeforFreud)couldgeneratedreaming ingmaycontributetoimportantcognitiveandaffectivefunctions

(production) and could preserve dream continuation (mainte- pertainingtomemory,emotion,andmotivation.

nance). In particular, a wish in the Freudian taxonomy could

correspondtotheanticipationofareward,andathreatcouldcorre- 5.4.3. ThreatSimulationTheory(TST)andRAM

spondtotheanticipationofapunisherthatwouldtriggeravoidance TheThreatSimulationTheory(TST)wasinitiallyproposedby

behavior or seeking of safety (which may typically fail in night- Revonsuo (2000) and then further developed by Revonsuo and

mares).Ontheotherhand,whilemanybizarreaspectsindreams Valli(2009).AccordingtotheTST,thefunctionofdreamingisto

mayrelatetochangesinregionalbrainactivityandinfunctional allow an organized and selective offline simulation of threaten-

connectivity between brain regions (see Section 5.1 above), the ing events that would ultimately promote the development and

Freudiannotionofanactivecensorshipthatwouldtransformthe maintenanceofthreat-avoidanceskills.Supportingtheuniversal-

originalor‘latent’contentofadreamintosome‘apparent’(usually ity and prevalence of threatening experiences in dreams, being

bizarre) content lacks neurobiological support. Moreover, unlike chased or attacked is the most typical dream theme around the

theFreudianmodelofdreaming,theRAMalsoimpliesthatdreams world(Zadra,1996).Behaviorallyandperceptuallyrealisticthreat-

canbepotentiallyrelatedtonoveleventsortosomeprobabilistic ening experiences in a dream would trigger the activation of a

future,andnotonlytothepast.Although,thereisevidenceforthe threat simulation system, and also underlying brain circuits, in

presenceofpastandcurrentwakingconcernsinthedreamcon- particular the amygdala. Such a virtual rehearsal during dream-

tent(Cartwrightetal.,2006;SchredlandHofmann,2003),dreams ingwouldleadtoimprovedperformanceinreallifesituationsand

are usually novel constructions and rarely reproductions of past this training could be used as learning or maintenance of threat

events(Meier,1993).Actually,whilealargeproportionofdream avoidanceskills(ValliandRevonsuo,2009).Supportforthethe-

elementscomesfromrecentmemory(Schwartz,2003),integrated orycomesfromthestudyofdreamsintraumatizedpopulations

life episodes are incorporated in no more than 1–2% of dreams whosimulatethreateningeventsintheirdreamsmoreoftenthan

reports(Fosseetal.,2003). non-traumatizedpopulations(Vallietal.,2005),aswellasfromthe

TheRAMproposesthatdreaming,byactivationofthereward observationthatREMSDimpairsthreatavoidanceresponsesinreal

system, exposes and trains the dreamer to motivationally- or life(e.g.Martinez-Gonzalezetal.,2004).TheTSTthereforesuggests

emotionally-relevant stimuli, which may relate to instinctual plausiblepsychologicalandbiologicalfunctionsfordreamingandis