Don't Shoot the Dog!: The New Art of Teaching and Training

82 Pages·1999·3.07 MB·English

by  Karen Pryor

eVersion 1.0-clickfor scannotes
DON'T SHOOT THE DOG
Karen Pryor
To my mother,Sally Ondeck;my stepmother,Ricky Wylie;andWinifred Sturley,my
teacher andfriend.
Contents
Foreword
1—Reinforcement:Betterthan Rewards
Inwhichwe learn ofthe ferocity ofWallStreetlawyers;ofhow to—andhow notto—buypresents and
give compliments;ofagrumpy gorilla,agrudging panda,andatruculent teenager(the author);ofgambling,
pencilchewing,falling inlove withheels,andother badhabits;ofhow toreformascoldingteacherora
crabby bosswithout theirknowingwhatyou'vedone;andmore.
2—Shaping:Developing SuperPerformance WithoutStrainorPain
Howtoconduct anopera;how toputt;how tohandle abadreportcard.Parlorgames for trainers.Notes
onkiller whales,NimChimpsky Zen,GregoryBateson,the BrearleySchool,why cats get stuck intrees,and
how totrainachicken.
3—Stimulus Control:CooperationWithoutCoercion
Orders,commands,requests,signals,cues,andwordstothe wise;whatworksandwhatdoesn't.What
disciplineisn't.Whogets obeyedandwhy.Howtostop yelling at yourkids.Dancing,drill teams,music,
martialarts,andother recreational uses ofstimuluscontrol.
4—Untraining:UsingReinforcementtoGetRidofBehavior You Don'tWant
Eight methodsofgetting rid ofbehavior youdon'twant,frommessyroommatestobarking dogstobad
tennistoharmful addictions,starting withMethod 1:Shootthe Animal,whichdefinitelyworks,andending
withMethod 8:Change the Motivation, whichis more humane anddefinitelyworkstoo.
5—Reinforcementinthe RealWorld
What it all means.Readingminds,coaching Olympic teams,how happinesscanaffectcorporate profits,
ways todeal withother governments,andother practicalapplications ofreinforcement theory.
6—ClickerTraining:A NewTechnology
From the dolphin tanks toeveryone'sbackyard:dogownersaroundthe worldputaway the chokechain
andpick upthe clicker.Long-termbenefits:accelerated learning,precision,reliability,better
communication,andfun.The Great Internet CanineHotDog Challenge;some trulyfetchinghorses;apilot
pilotprogram;andclicking andautism.Changingthe worldoneclickat atime.
Resources
Acknowledgments
Aboutthe Author
Foreword
Thisbookis about how totrainanyone—humanoranimal,youngorold,oneselforothers—todo
anything that canandshould bedone.Howtoget the cat off the kitchentableoryourgrandmother tostop
nagging you.Howtoaffectbehavior inyourpets,yourkids,yourboss,yourfriends.Howtoimproveyour
tennisstroke,yourgolf game,yourmath skills,yourmemory.Allbyusing the principlesoftraining with
reinforcement.
http://avaxhome.ws/blogs/ChrisRedfield
Theseprinciplesare laws,like the lawsofphysics.They underlie all learning-teachingsituations assurely
asthe lawofgravityunderlies the falling ofanapple.Whenever we attempt tochangebehavior,inourselves
orinothers,we are using these laws,whether we know it ornot.
Usually we are using them inappropriately.Wethreaten,we argue,we coerce,we deprive.Wepounceon
otherswhen thingsgowrong andpass upthe chancetopraisethem when thingsgoright.Weare harsh and
impatientwithourchildren,witheachother,withourselves even;andwe feel guiltyoverthat harshness.We
know that withbetter methodswe could accomplish ourendsfaster,andwithout causing distress,butwe
can'tconceive ofthose methods.Weare justnotattunedtothe ways inwhichmodern trainers take advantage
ofthe lawsofpositive reinforcement.
Whateverthe training task,whether keeping afour-year-oldquiet inpublic,housebreakingapuppy,
coaching ateam,ormemorizingapoem,it will gofaster,andbetter,andbemore fun,if youknow how to
usepositive reinforcement.
The lawsofreinforcement are simple;youcanputthe wholebusinessonablackboardinten minutes and
learn it inanhour.Applyingthese lawsis more ofachallenge;training byreinforcement is like agame,one
dependent uponquick thinking.
Anyone canbeatrainer;some people are goodat it fromthe verystart.You donotneedspecial qualities
ofpatience,oraforceful personality,oraway withanimals orchildren,orwhatcircustrainer FrankBuck
usedtocallthe power ofthe humaneye.You justneedtoknow whatyou'redoing.
Therehavealwaysbeenpeople withanintuitive understanding ofhow toapply the lawsoftraining.We
callthem giftedteachers,brilliant commandingofficers,winningcoaches,genius animal trainers.I've
observed some theater directors andmanysymphony orchestra conductorswho are wonderfullyskilled at
using reinforcement.Thesegiftedtrainers don'tneedabooktobeable totake advantage ofthe lawsthat
affecttraining.Forthe restofus,however,those ofusmuddlingalong withanuncontrolled pet orat cross-
purposes withachild orcoworker,aknowledgeofhow reinforcement really workscanbeagodsend.
Reinforcementtraining is notasystemofrewardandpunishment—byandlarge modern trainers don't
evenusethose words.The concept ofrewardandpunishment carries agreat freight ofemotional associations
andinterpretations,suchasdesireanddread andguiltandshoulds andoughtto's.Forexample,we give
rewards toothersfor thingswe didourselves—suchasice creamtoachild tomake upfor ascolding.We
also tend tothink we know whatarewardshould be:ice cream,for example,orpraise.Butsome people don't
like ice cream,andpraisefromthe wrong person orfor the wrong reason mayhurt.Insome cases praise
fromateachermayguaranteeridicule fromclassmates.
Weexpect people todothe rightthing without reward.Our teenage daughter should wash the dishes
because that'sherdutytous.Weare angryif children oremployeesbreak things,steal,arrivelate,speak
rudely,andsoon,because they should know better.Wepunish,often longafterthe behavior occurred—
sending people toprisonbeing aprimeexample—thus creating anevent that mayhavenoeffectonfuture
behavior,andwhichinfact is merelyretribution.Nevertheless we think ofsuchpunishment aseducation,and
people easilyrefertoit inthat way:"Itaught him alesson."
Modern reinforcement training is basednotonthese folk beliefs butonbehavioral science.Scientifically
speaking,reinforcement is anevent that (a)occurs during oruponcompletion ofabehavior;and(b)increases
the likelihood ofthat behavior occurringinthe future.The keyelements here are two:the twoevents are
connected inreal time—the behavior engenders the reinforcement—andthen the behavior occurs more
frequently.
Reinforcers maybepositive,something the learner mightlike andwantmore of,suchasasmile orapat,
orthey mightbenegative,something toavoid,suchasayankonaleash orafrown.What'scritical is that
there is atemporal relationship betweenthem—the behavior occurs,then the reinforceroccurs,and
subsequently the behavior that brought the goodresult oravertedthe badoccurs more often.Infact,the
definitionworksinbothdirections,like afeedback loop:If the behavior doesnotincrease,then either the
reinforcerwas presentedtooearly ortoolate,orthe payoff youselected was notreinforcingtothat
individual.
Inaddition,Ibelievethere'sanimportantdifference betweenreinforcement theory,the science,and
reinforcement training,aspecific applicationofthat science.Research showsthat followingabehavior with
apleasant consequence increases the behavior.That'strue;butinpractice,toget the sensationalresults we
trainers havenow come toexpect,the reinforcerhastooccur inthe veryinstant the behavior is taking place.
Bingo!Now!Inthe instant,inreal time,you,the learner,needtoknow that whatyou'redoingrightnow has
won youaprize.
Modern trainers havedeveloped some great shortcuts for reinforcinginstantaneously:primarily the useof
amarkersignaltoidentify the behavior.ThisrevisedversionofDon'tShootthe Dog!is about the lawsof
reinforcement,some practicalways tousethose lawsinthe real world,andthe grassroots movementcalled,
at leastat present,clicker training,whichis taking the technologyinto new andunexplored terrain.
Ifirst learnedabout training withpositive reinforcement inHawaii,wherein1963Isigned onashead
dolphin trainer at anoceanarium,SeaLife Park.Ihadtrained dogsandhorses bytraditional methods,but
dolphinswereadifferentproposition;youcannot usealeash orabridleorevenyourfistonananimal that
justswims away.Positive reinforcers—primarily abucket offish—werethe onlytoolswe had.
A psychologist outlined for methe principlesoftraining byreinforcement.The art ofapplying those
principlesIlearnedfromworkingwiththe dolphins.Schooled asabiologist,andwithalifelong interestin
animal behavior,Ifoundmyselffascinated,notsomuchwiththe dolphinsaswithwhatcould be
communicatedbetweenus—frommetothe animal andfromthe animal tome—during thiskindoftraining.I
appliedwhatI'dlearnedfromdolphin training tothe training ofother animals.And Ibegantonoticesome
applications ofthe systemcreepinginto mydaily life.Forexample,Istopped yelling at mykids,because I
was noticing that yelling didn'twork.Watchingfor behavior Iliked,andreinforcingit when it occurred,
worked alot better andkept the peacetoo.
Thereis asolidbody ofscientific theory underlyingthe lessonsIlearnedfromdolphin training.Weshall
goconsiderably beyondtheory inthisbook,since asfar asIknow,the rulesfor applyingthese theories are
largely undescribedbyscienceandinmyopinion often misappliedbyscientists. Butthe fundamental laws
are well establishedandmustbetaken into account when training.
The study ofthisbodyoftheory is variously known asbehavior modification,reinforcement theory,
operant conditioning,behaviorism,behavioral psychology,andbehavior analysis:the branch ofpsychology
largely credited toHarvardprofessor B.E Skinner.
Iknow ofnoother modern bodyofscientific information that hasbeensovilified,misunderstood,
misinterpreted,overinterpreted,andmisused.The veryname ofSkinner arouses ire inthose who champion
"free will" asacharacteristic that separates manfrombeast.To people schooled inthe humanistictradition,
the manipulationofhumanbehavior bysome sortofconscioustechniqueseemsincorrigibly wicked,inspite
ofthe obvious fact that we all goaroundtryingtomanipulateoneanother'sbehavior all the time,bywhatever
means come tohand.
Whilehumanists havebeenattacking behaviorismandSkinner himself withafervorthat usedtobe
reservedfor religious heresies,behaviorismhasswelled into ahugebranch ofpsychology,withuniversity
departments,clinicalpractitioners,professionaljournals,international congresses,graduatestudies programs,
doctrines,schisms,andmassesandmassesofliterature.
And there havebeenbenefits.Somedisorders—autism,for example—seem torespond toshaping and
reinforcement astonoother treatment.Manyindividual therapists havebeenextremely successful insolving
the emotional problems ofpatients byusing behavioral techniques.The effectiveness,at leastinsome
circumstances,ofsimply altering behavior ratherthan delving into itsorigins hascontributed tothe riseof
family therapy,inwhichevery family member'sbehavior is looked at,notjustthe behavior ofthe onewho
seemsmostobviouslyindistress.Thismakes eminent goodsense.
Teaching machinesandprogrammed textbooksderived fromSkinneriantheory wereearly attemptsto
shapelearning step bystep andtoreinforce the studentfor correct responses.Theseearly mechanismswere
clumsybutled directly toCAI,Computer-AssistedInstruction,whichis great funbecause ofthe amusing
nature ofthe reinforcers(fireworks,dancing robots)andhighly effective because ofthe computer'sperfect
timing.Reinforcementprogramsusing tokens orchitsthat canbeaccumulated andtraded for candy,
cigarettes,orprivilegeshavebeenestablishedinmental hospitals andother institutions.Self-training
programsfor weight controlandother habit changes abound.Effective educational systems basedon
principlesofshaping andreinforcement,suchasPrecision Teaching andDirect Instruction,are making
inroadsinourschools.And biofeedbackis aninteresting applicationofreinforcement totraining of
physiologicalresponses.
Academicians havestudiedthe mostminuteaspectsofconditioning.One findingshows,for example,that
if youmake achart tokeeptrack ofyourprogressinsome self-training program,youwill bemore likely to
maintain new habitsif yousolidly fill inalittle square every dayonthe chart,ratherthan justputtingacheck
mark inthe square.
Thisabsorption withdetail hasvalid psychological purposes,butonedoesnotoften find muchgood
training init.Trainingis aloop,atwo-way communicationinwhichanevent at oneendofthe loopchanges
events at the other,exactlylike acybernetic feedback system;yet manypsychologiststreat theirwork as
something they dotoasubject,notwith the subject.To areal trainer,the idiosyncratic andunexpected
responsesanysubjectcangive are the mostinteresting andpotentially the mostfruitfulevents inthe training
process;yet almost all experimental work is designed toignore orminimizeindividualistic responses.
Devising methodsfor whatSkinner named shaping,the progressive changing ofbehavior,andcarryingout
those methods,is acreative process.Yetthe psychological literature aboundswithshaping programsthat are
sounimaginative,nottosayham-handed,that they constituteinmyopinion cruel andunusual punishment.
Take,for example,inonerecentjournal,atreatmentfor bed-wetting that involved notonlyputting"wetness"
sensorsinthe child'sbedbuthaving the therapistspendthe night withthe child!The authorshadthe grace to
sayapologetically that it was ratherexpensive for the family.Howabout the expense tothe child'spsyche?
Thiskindof"behavioral" solution is like tryingtokillflies withashovel.
Schopenhauer oncesaid that every original idea is first ridiculed,then vigorously attacked,andfinally
taken for granted.As far asIcansee,reinforcement theory hasbeennoexception.Skinner was widely
ridiculed years agofor demonstrating shaping bydevelopingapair ofPing-Pong-playing pigeons.The warm,
comfortable,self-cleansing,entertainment-providingcrib hebuiltfor his infantdaughterswas derided asan
inhumane"babybox," immoralandheretical.Rumors stillgoaroundthat his daughterswentmad,when in
fact bothofthem are successful professionalwomenandquite delightfulpeople.Finally,nowadays many
educated people treat reinforcement theory asif it weresomething notterriblyimportantthat they have
known andunderstoodall along.Infact mostpeople don'tunderstandit,orthey wouldnotbehavesobadly to
the people aroundthem.
Inthe years since mydolphin-training experiences,Ihavelectured andwrittenabout the lawsof
reinforcement inacademicandprofessionalcircles aswell asfor the generalpublic.I'vetaught thiskindof
training tohighschool,college,andgraduatestudents,tohousewives andzookeepers,tofamily andfriends,
and,inweekend seminars,toseveralthousand dogownersandtrainers.Ihavewatchedandstudiedall kinds
ofother trainers,fromcowboys tocoaches,andI'venoticedthat the principlesofreinforcement training are
gradually seeping into ourgeneralawareness.Hollywoodanimal trainers callthe useofpositive
reinforcement "affection training" andare using these techniques toaccomplish behaviorsimpossibleto
obtain byforce suchasmanyofthe behaviorsofpigs andother animals inthe movieBabe.ManyOlympic
coaches nowadays usepositive reinforcement andshaping,insteadofrelyingonold-fashionedbrowbeating,
andhaveachieved notableimprovementsinperformance.
Nowhere,however,haveIfoundthe rulesofreinforcement theory writtendown sothat they could beof
useinimmediatepracticalsituations.So here they are,explainedinthisbookasIunderstandthem andasI
seethem usedandmisusedinreal life.
Reinforcementtraining doesnotsolve all problems—it will notfattenyourbankaccount,it cannot savea
badmarriage,andit will notoverhaul seriouspersonalitydisorders.Somesituations,suchasacrying baby,
are nottraining problems andrequireother kinds ofsolutions.Somebehaviors,inanimals andpeople,have
geneticcomponentsthat maybedifficult orimpossibletomodifybytraining.Someproblems are notworth
the training time.Butwithmanyoflife'schallenges,tasks,andannoyances,correct useofreinforcement can
help.
Usingpositive reinforcersinonesituationmayshow youhow tousethem inothers.As adolphin
researcher whomIworked withsourlyputit,"Nobody should beallowed tohaveababyuntilthey havefirst
beenrequiredtotrainachicken," meaningthat the experienceofgetting results withachicken,anorganism
that cannot betrained byforce,should make it clearthat youdon'tneedtousepunisherstoget results witha
baby.And the experienceshould give yousome ideas about reinforcingbabybehavior youwant.
Ihavenoticedthat mostdolphin trainers,who mustdevelop the skills ofusing positive reinforcersintheir
daily work,havestrikingly pleasant andagreeablechildren.Thisbookwill notguaranteeyouagreeable
children.Infact,it promises nospecific results orskills.What it will give youis the fundamental principles
underlyingall training,andsome guidelines onhow toapply these principlescreativelyinvarying situations.
It mayenable youtoclearupannoyances that havebeenbotheringyoufor years,ortomake advances in
areas whereyouhavebeenstymied.It will certainly,if youwish,enable youtotrainachicken.
Thereseemstobeanatural order toreinforcement training.Thesechapterscome inthe sequence inwhich
training events,fromsimple tocomplex,really take place,andthisis also the sequence inwhichpeople seem
tolearn mosteasilytobereal trainers.The organization ofthisbookis progressive inorder todevelop a
comprehensive understanding oftraining withpositive reinforcers.Itsapplications,however,are meanttobe
practical.Throughout the book'schaptersreal-lifesituations are offeredasillustrations.Specificmethods
should betreated assuggestions orinspirations,ratherthan asdefinitiveinstructions.
1—Reinforcement: Better than Rewards
WhatIsa PositiveReinforcer?
A reinforceris anything that,occurringinconjunction withanact,tends toincrease the probabilitythat
the act will occur again.
Memorize that statement.It is the secretofgoodtraining.
Thereare twokinds ofreinforcers:positive andnegative.A positive reinforceris something the subject
wants,suchasfood,petting,orpraise.A negativereinforceris something the subjectwantstoavoid—a
blow,afrown,anunpleasant sound.(The warningbuzzer inacar if youdon'tfastenyourseat belt is a
negativereinforcer.)
Behavior that is alreadyoccurring,nomatter how sporadically,canalwaysbeintensified withpositive
reinforcement.If youcallapuppyandit comes,andyoupet it,the pup'scoming when calledwill become
more andmore reliable evenwithout anyother training.Supposeyouwantsomeone totelephoneyou—your
offspring,yourparent,yourlover.If heorshedoesn'tcall,there isn'tmuchyoucandoabout it.A majorpoint
intraining withreinforcement is that youcan'treinforce behavior that is notoccurring.If,onthe other hand,
youare alwaysdelighted when yourloved onesdocall,sothat the behavior is positivelyreinforced,the
likelihood is that the incidenceoftheircalling will probably increase.(Of course,if youapply negative
reinforcement—"Whyhaven'tyoucalled,why doIhavetocallyou,younever callme," andsoon,remarks
likely toannoy—youare setting upasituationinwhichthe caller avoids suchannoyancebynotcalling you;
infact,youare training them nottocall.)
Simplyoffering positive reinforcement for abehavior is the mostrudimentarypart ofreinforcement
training.Inthe scientific literature,you canfind psychologistssaying,"Behavioralmethodswereused," or,
"The problemwas solved byabehavioral approach." Allthismeans,usually,is that they switched topositive
reinforcement fromwhatever other method they wereusing.It doesn'timplythat they usedthe wholebagof
tricks describedinthisbook;they maynotevenbeawareofthem.
Yetswitchingtopositive reinforcement is often all that is necessary.It is byfar the mosteffective way to
help the bed-wetter, for example:private praiseandahugfor drysheetsinthe morning,when they dooccur.
Positive reinforcement canevenwork on yourself.AtaShakespeare study groupIoncebelonged toImet
aWallStreetlawyer inhis lateforties who was anavid squash player.The manhadoverheard mechatting
about training,andonhis way outthe doorafterwardheremarkedthat hethought hewouldtry positive
reinforcement onhis squash game.Insteadofcursinghis errors,aswas his habit,hewouldtry praising his
goodshots.
Twoweeks later Iraninto him again."How'sthe squash game?" Iasked.A lookofwonder andjoy
crossed his face,anexpression notfrequently seenonWallStreetlawyers.
"At first Ifeltlike adamned fool," hetold me,"saying 'Waytogo,Pete,attaboy' for every goodshot.
Hell, when Iwas practicing alone,Ievenpattedmyselfonthe back.And then mygame started toget better.
I'mfour rungs higher onthe club ladder than I'veever been.I'mwhippingpeople Icould hardly take apoint
frombefore.And I'mhaving more fun.Since I'mnotyelling at myselfall the time,Idon'tfinishagame
feelingangryanddisappointed.If Imade abadshot,never mind,goodoneswill come along.And Ifind I
really enjoy it when the other guymakes amistake,gets mad,throws his racquet—Iknow it won'thelp his
game,andIjustsmile ..."
What afiendish opponent.And justfromswitchingtopositive reinforcement.
Reinforcers are relative,notabsolute.Rainis apositive reinforcertoducks,anegativereinforcertocats,
andamatter ofindifference,at leastinmildweather,tocows.Food is notapositive reinforcerif you'refull.
Smiles andpraisemaybeuselessasreinforcersif the subjectis tryingtoget youmad.Inorder tobe
reinforcing,the itemchosenmustbesomething the subjectwants.
It is usefultohaveavariety ofreinforcersfor anytraining situation.Atthe SeaWorld oceanariums,killer
whalesare given manyreinforcers,including fish(theirfood),stroking andscratching ondifferentpartsof
the body,socialattention,toys,andsoon.Whole showsare runinwhichthe animals never know which
behavior will bereinforced next orwhatthe reinforcerwill be;the "surprises" are sointeresting for the
animals that the showscanberunalmost entirely without the standardfishreinforcers;the animals get their
foodat the endofthe day.The necessity ofswitchingconstantly fromonereinforcertoanother is
challenging andinteresting for the trainers,too.
Positive reinforcement is goodfor humanrelationships.It is the basisofthe art ofgiving presents:
guessingat something that will bedefinitelyreinforcing(guessingrightis reinforcingfor the giver,too).In
ourculture,presentgiving is often lefttowomen.Ievenknow ofonefamily inwhichthe motherbuysall the
Christmas presents toandfromeveryone.It causes amusement onChristmas morning,brothers andsisters
saying,"Let'ssee,thisis fromAnne toBilly," when everyone knows Anne hadnothing todowithit.Butit
doesnotsharpen the children'sskills at selecting ways toreinforce other people.
Inourculture amanwho hasbecome observantabout positive reinforcement hasagreat advantage over
other men.As amother,Imade sure that mysonslearnedhow togive presents.Once,for example,when
they werequite young,sevenandfive,Itookthem toaratherfancy storeandhadthem select twodresses,
oneeach,for theirevenyoungersister.They enjoyed lolling about inthe plush chairs,approving or
disapproving ofeachdress asshemodeledit.Their little sister enjoyed it too;andshehadthe ultimateveto
power.And so,thanks tothisandsimilarexercises,they all learnedhow totake areal interestinwhatother
people want;how toenjoy findingeffective positive reinforcersfor the people youlove.
Negative Reinforcement
A reinforceris something thatincreases abehavior;butit doesn'thavetobesomething the learner wants.
Avoidingsomething youdislike canbereinforcing,too.Laboratoryresearchshowsthat behavior canbe
increased byaversivestimuliif achangeinbehavior will make the aversivestimulusgoaway.Suchstimuli
are callednegativereinforcers:thingsaperson oranimal will work toavoid.
Negative reinforcersmayconsist ofthe mildestofaversivestimuli—aderisive glance fromafriendwhen
youmake apoorjoke,oraslight draftfromanair conditioner that causes youtoget upandmovetoanother
chair.However,evenveryextreme aversives,frompublic humiliation toelectricshock,mayfunctionas
negativereinforcersaswell asbeing punishingexperiences.Wemayexperiencebeing yelledat ashighly
punitive,butwe also quickly learn tocome intowork the backway when the bosswho hasoften yelledat us
is standinginthe frontdoor.
Negative reinforcersare aversives that canbehaltedoravoided bychanging behavior.As soonasthe new
behavior starts,the aversivestimulusstops,andthus the new behavior is strengthened.Supposethat while
sittinginmyaunt'slivingroom,Ihappentoputmyfeet onthe coffee tableasIwouldat home.Myaunt
raises adisapproving eyebrow.Iputmyfeet onthe flooragain.Her face relaxes.Ifeel relieved.
Reinforcement:Betterthan Rewards
The raisedeyebrowwas anaversivestimulusactingasanegativereinforcer.BecauseIwas able tohalt
the aversivestimulus,the new behavior—keeping myfeet onthe floor—is more likely tooccur again,at
leastat myaunt'shouse,butpossiblyinother houses,too.
Trainingcanbedonealmost entirely withnegativereinforcers,andmuchtraditional animal training is
doneexactlythat way The horse learnstoturn leftwhen the leftrein is pulled,because the annoying pressure
initsmouth ceases when the turn is made.The lion backsontoapedestal andstays there,toavoid the
intrusive whiporchair held near itsface.
Negative reinforcement,however,is notthe same aspunishment.So whatis the difference?Inthe first
edition ofthisbookIwrotethat punishment is anaversivestimulusthat occurs afterthe behavior it was
meanttomodify,andtherefore it canhavenoeffectonthe behavior."A boybeing spankedfor abadreport
card mayormaynotget better reportcards inthe future,buthesurelycan'tchangethe onehehasjust
brought home." Indeed,when we punish withintent,we frequently doit far toolate,butthat is notthe actual
difference betweenpunishment andnegativereinforcement.
Modern behavior analysts identify punishment asanyevent thatstops behavior.A babystarts toputa
hairpininto the electricsocket.Hismothergrabs him and/orslapshis handaway fromthe socket:thislife-
threatening behavior hastobeinterruptednow.The behavior stops.Lotsofother thingsmaystart—the baby
cries,the motherfeelsbad,andsoon—butthe hairpin-in-electric-outlet behavior ceases,at leastfor that
moment.That'swhatpunishment does.
B.F.Skinner was more precise.He defined punishment aswhathappenswhen abehavior results inthe
loss ofsomething desirable—the pleasureofinvestigating if thisobjectcanfitinto that hole,apopular
pastime withbabies—orwhen the behavior results inthe delivery ofsomething undesirable.However,in
bothcases,while the ongoingbehavior stops,there is nopredictableoutcomeinthe future.Weknow that
reinforcersstrengthen behavior inthe future,butapunisherwill notresult inpredictable changes.
Forexample,will grabbing the babyorsmackinghis hand,evenif his mother'stiming is perfect,
guaranteethat the babywon'ttry sticking thingsinto outlets again?Idoubtit.Ask anyparent.What really
happensis that we pick upsmall objects,we putcovers overthe wall outlets,orwe movefurniture infrontof
them,andeventually the babyoutgrows thisparticular urge.
The behavior analysts lookat it thisway.Reinforcementandpunishment are eachaprocess, defined by
results.Negative reinforcerscanbeusedeffectivelytotrainbehavior,andeventhoughaversivestimuliare
involved,the process canberelatively benign.Here(withthanks tollama expert JimLogan)is anice useof
the negativereinforcerwithasemidomestic animal,the llama,kept inthe United Statesaspets and
elsewhere aspackanimals andfor theirwool.
Llamasare timid andshy,like horses.Unless handled alot when young,they canbehardtoapproach.So,
while operant conditioning withafoodreinforcerworkssplendidly withllamas,if allama is tooskittishto
come close enoughtoaperson totake the food,here'swhatmodern llama trainers do.They useaclicker asa
signaltotell the llama that whatit is doinghasearned areinforcer,butthe primary orreal reinforceris the
removalofanegativereinforcer,anaversive.
Ineffect,yousaytothe llama,"Willyoustand stillif Iapproach within thirty feet?Yes?Good.I'll click
myclicker andturn andgofarther away.
"Nowwill youstand stillif Iapproach within twenty-five feet?Yes?Good.I'll clickandgoaway."
Usingthe clicktomark the behavior ofstandingstill,withthe scary person turningandgoingaway again
asthe reinforcer,onecansometimesget within touching distance infive orten minutes.The llama,asit
were,is incontrol.As longasit standsstill,it canmake yougoaway!So it stands still,evenwhen the person
is rightnext toit.
Whenonehastouched the llama severaltimes andthen retreated,the ice is broken.Thisperson is no
longer asscary.Nowit'stimefor the feed bucket.The communicationloopbecomes "MayItouch youwhile
youstand still?Yes?Click andhere'ssome delicious food." And the llama is onitsway toearning positive
reinforcers,including foodandscratching andpetting,withitssplendidnew behavior ofstandingstillinstead
ofheading for the next county.
Thisuseofretreat,oreasing backwhen the desiredbehavior occurs,is animportantaspect ofmostofthe
so-called"horse whisperer" techniques.Inmostofthese methodsthe trainer workswithaloose horse ina
confined area andproceeds inarelatively shorttimetotransformahorse inflight toahorse calmly accepting
ahuman.The horse,onceperhaps completelywild,becomes socalm,evenacceptingasaddle andrider,that
the total effectis magical.
Trainerswho usethese techniques often havesuperstitious explanations for whatis happening;andwhile
manyhaveformedthe habit ofmaking some soundormotionthat functions asthe markersignalorthe
conditioned reinforcer,few are consciously awareofdoingso.Nevertheless,it is notmagicat work;it is the
lawsofoperant conditioning.
Whilenegativereinforcement is ausefulprocess,it'simportanttoremember that eachinstance ofnegative
reinforcement also containsapunisher.Whenyoupull onthe leftrein,untilthe momentthat the horse turns,
youare punishinggoingstraight ahead.Overuseofnegativereinforcersandother aversives canlead towhat
Murray Sidman,Ph.D.,calls"fallout," the undesirable side effects ofpunishment (seeChapter 4).
TimingofReinforcers
As alreadystated,areinforcermustoccur inconjunction withthe act it is meanttomodify.The timing of
the arrival ofthe reinforceris information.It tells the learner exactlywhatit is youlike.Whenoneis trying
tolearn,the informationalcontentofareinforcerbecomes evenmore importantthan the reinforceritself.In
coaching athletes ortraining dancers,it is the instructor'sshouted "Yes!" or"Good!," markingamovement
asit occurs,that trulygives the neededinformation—notthe debriefing later inthe dressingroom.
Laggardly reinforcement is the beginning trainers biggestproblem.The dogsits,butbythe timethe
owner says "Good dog," the dogis standingagain.What behavior did"Good dog" reinforce?Standing up.
Whenever youfind yourselfhaving difficultiesinatraining situation,the first questiontoaskyourselfis
whether youare reinforcingtoolate.If youare workingwithaperson orananimal andare caught upinthe
thick ofthe action,it sometimeshelps tohavesomeone else watchfor latereinforcers.
Weare alwaysreinforcingoneanother toolate."Gee,honey,youlooked great lastnight" is quite
differentfromthe same commentsaid at the moment.The delayed reinforcermayevenhavedeleterious
effect("What'sthe matter,don'tIlookgreat now?").Wehaveatouching trust inthe powersofwordsto
cover ourlapsesintiming.
Reinforcingtooearly is also ineffective.Atthe BronxZoo the keepers werehaving troublewithagorilla.
They neededtoget it into itsoutdoor peninorder toclean the indoor cage,butit hadtaken tosittinginthe
doorway,wherewithitsenormousstrength it could prevent the sliding doorfrombeing closed.Whenthe
keepers putfoodoutside,orwaved bananas enticingly,the gorilla either ignored them orsnatched the food
andranbacktoitsdoorbefore it could beshut.A trainer onthe zoostaffwas askedtolookat the problem.
He pointed outthat bananawavingandthe tossinginoffoodwereattemptstoreinforce behavior that hadn't
occurred yet.The name for thisis bribery.The solution was toignore the gorilla when it sat inthe door,but
toreinforce it withfoodwheneverit didhappentogooutbyitself.Problemsolved.
Sometimes,Ithink,we reinforce children toosoonunderthe misimpressionthat we are encouragingthem
("Atta girl,that'sthe way,youalmost gotit right").What we maybedoingis reinforcingtrying.Thereis a
difference betweentryingtodosomething anddoingit.Wailsof"Ican't" maysometimesbeafact,butthey
mayalso besymptoms ofbeing reinforced toooften merelyfor trying.Ingeneral,giving gifts,promises,
compliments,orwhatever for behavior that hasn'toccurred yet doesnotreinforce that behavior inthe
slightest.What it doesreinforce is whatever was occurringat the time:soliciting reinforcement,mostlikely.
Timingis equallyimportantwhen training withnegativereinforcers.The horse learnstoturn leftwhen
the leftrein is pulled,butonlyif the pullingstops when it doesturn.The cessation is the reinforcer.You get
onahorse,kick it inthe sides,andit moves forward;youshould then stop kicking (unless youwantit to
movefaster).Beginning riders often thump away constantly,asif the kicking weresome kindofgasoline
necessarytokeepthe horse moving.The kicking doesnotstop,soit containsnoinformation for the horse.
Thus are developed the iron-sided horses inridingacademies that moveat asnail'spacenomatter how often
they are kicked.
The same appliestopeople getting naggedandscolded byparents,bosses,orteachers.If the negative
reinforcerdoesn'tcease the instant the desiredresult is achieved,it is neither reinforcingnorinformation.It
becomes,bothliterally andinterms ofinformation theory,"noise."
Watchingfootball andbaseball onTV,Iamoften struckbythe beautifullytimedreinforcersthat the
playersreceiveagain andagain.As atouchdownis made,asthe runner crosseshomeplate,the roar ofthe
crowd signals unalloyedapproval;andthe instant ascore is made oragame is won,justwatchthe frenzied
exchangeofmutualreinforcersamongthe players.It is quite differentfor actors,especially movie actors.
Even onstage the applause comes afterthe jobis done.Formovie actors,except for occasional response
fromadirector orcameraoperatororgrip,there is notimely reinforcement;fanlettersandgoodreviews,
arriving weeks ormonthslater,are pallid comparedwithall ofYankee Stadiumgoingberserkat the moment
ofsuccess.No wonder some starsoften exhibit aseemingly neurotic craving for adulation andthrills;the
work canbepeculiarly unsatisfyingbecause the reinforcers,howeversplendid,are always"late."
SizeofReinforcer
Beginning trainers who usefoodreinforcement withanimals are often confused astohow bigeach
reinforcershould be.The answeris:assmall asyoucanget away with.The smallerthe reinforcer,the more
quickly the animal eats it.Notonlydoesthiscut down onwaiting time,it also allows for more reinforcers
persession,before the animal becomes satiated.In1979Iwas hired asaconsultant bythe National
ZoologicalParkinWashington,D.C.,toteach positive reinforcement techniques toagroupofzoo
employees.One ofthe keepers inmytraining classcomplained that hertraining ofthe pandahadbeen
proceedingtooslowly.Ithought thisodd because intuitively Ifeltthat pandas—big,greedy,activeanimals—
should beeasytotrainwithareinforceroffood.Iwatchedasessionandfoundthat while the keeper was
gradually succeedinginshaping abodymovement,shewas giving the pandaawholecarrotfor each
reinforcement.The pandatookitsown sweettimeeatingeachcarrot,sothat infifteen minutes ofvaluable
keeper timeit hadearned onlythree reinforcers(andwas incidentally getting tiredofcarrots).A singleslice
ofcarrotperreinforcement wouldhavebeenbetter.
Ingeneral,areinforcerthat constitutes onesmall mouthful for that animal is enoughtokeepit interested
—agrain ortwoofcornfor achicken,aquarter-inch cubeofmeatfor acat,half anapple for anelephant.
With anespecially preferred foodyoucangoevensmaller—ateaspoon ofgrain for ahorse,for example.
Keepersat the NationalZoo havetrained theirpolar bears todomanyusefulthings,suchasmoving to
another cageoncommand,withraisins.
A trainer'srule ofthumb is that if youare goingtohaveonlyonetraining sessionaday,youcancount on
the animal workingwell for about aquarterofitsrations;youthen give it the restfor free.If youcanget in
three orfour sessions aday,youcandivide the normalamount offoodinto about eighty reinforcersandgive
twenty orthirty ineachsession.Eighty reinforcersseemstobeabout the maximum for anysubject'sinterest
during anyoneday.(Perhaps that'swhy slidetraysusuallyholdeighty slides;Iknow Ialwaysgroanif a
lecturer asks the projectionist for the secondtray ofslides.)
The difficulty ofthe task also hassome effectonthe size ofthe reinforcer.AtSeaLife Parkwe foundit
necessarytogive eachofourwhalesalarge mackerel for theirOlympic-effort,twenty-two-foot straight-up
jump.They simply refusedtodoit for the usual reinforceroftwosmall smelt.Forpeople,sometimesif not
always,harder jobs get bigger rewards.And how we hate it when they don't,if we are the onesdoingthe
hardjob.
Jackpots
One extremely usefultechniquewithfoodoranyother reinforcement,for animals orpeople,is the
jackpot.The jackpotis arewardthat is muchbigger,maybe ten times bigger,than the normalreinforcer,and
onethat comes asasurprise tothe subject.AtanadagencywhereIonceworked we hadthe usual office
party at Christmas,aswell asinformalcelebrations tosignalize the completion ofabigjoborthe signingofa
new client.Butthe president was also inthe habit ofthrowing oneortwototally unexpected parties ayear.
Suddenlyinmidafternoonhewouldstride through all the offices,yelling for everyone tostop working.The
switchboardwas closed down,andincame aprocession ofcaterers,musicians,bartenders,champagne,
smoked salmon,the works:justfor usandfor nospecial reason.It was anunexpected jackpotfor fifty
people.It contributed vastly,Ithought,tothe company'shighmorale.
A jackpotmaybeusedtomark asuddenbreakthrough.Inthe case ofonehorse trainer Iknow,when a
younghorse executesadifficult maneuverfor the first time,the manleaps fromitsback,snatchesoff saddle
andbridle,andturnsthe horse loose inthe ring—ajackpotofcomplete freedom,whichoften seemstomake
the new behavior stick.
Paradoxically,asinglejackpotmayalso beeffective inimproving the response ofarecalcitrant,fearful,
orresistant subjectthat is offering nodesirable behavior at all.AtSeaLife Parkwe weredoingsome U.S.
Navy-fundedresearchthat involved reinforcingadolphin for new responses,insteadofold,previously
trained behaviors.Our subjectwas adocileanimal named Hou that rarelyofferednew responses.Whenshe
failedtoget reinforced for whatshedidoffer,shebecame inactive,andfinally inonesessionshewenttwenty
minutes offering noresponsesat all.The trainer finally tossedhertwofish"for nothing." Visiblystartledby
thislargesse,Hou became activeagain andsoonmade amovementthat could bereinforced,leadingtoreal
progressinthe next few sessions.