Occasion Setting: Associative Learning and Cognition in Animals (Apa Science Volumes)

407 Pages·1998·28.22 MB·English

by  American Psychological Association

Analogies Between Occasion
Setting and Pavlovian
Conditioning
Ralph R. Miller and Philippe Oberling
0
ccasion setting refers to the potential of a stimulus to clarify the
predictive value of an ambiguous cue. More technically speaking,
occasion setting refers to the ability of a punctate stimulus (or diffuse
context), sometimes called an occasion setter or feature, to modulate re-
sponding to a partially reinforced conditioned stimulus (CS) when the
occasion setter and the target CS are presented in close temporal proxim-
ity. In other words, the occasion setter (X) “sets the occasion” for respond-
ing (or not responding) to the CS (A). Specifically, positive occasion setters
signal that a CS will be reinforced, and negative occasion setters signal
that a CS not be reinforced (see Exhibit 1). The term occasion setter
will
originated with Skinner (1938), who applied it to operant situations (see
Bonardi, 1988, 1989; Colwill & Rescorla, 1990; Davidson, Aparicio, &
Rescorla, 1988; for contemporary examples of occasion setting in instru-
Support for this research was provided by National Institute of Mental Health Grant 33881, Universite
Louis Pasteur-Strasbourg, Fondation Bettencourt-Schueller,S andoz Laboratories (France), and Fondation
Guy Weisweiller. The authors would like to thank Moore Arnold, Robert Barnet, Aaron Blaisdell, Daniel
Burger, Robert Cole, James Denniston, Lisa Gunther, James Esposito, Haney Mallemat, Hernan Savastano,
and Russell Wishtart for their assistance in gathering the data from our laboratory that we present here
and for their comments on an early version of this chapter.
3
MILLER AND OBERLING
Defining Occasion Setting
Train Test Result
No occasion setting A-USIA A CR
Positive occasion setting X+A+US/A X+A CR
A -
X -
Negative occasion setting A+US/X+A A CR
X+A -
X -
NOTE: Partial reinforcement without an occasion setter is ambiguous. Occasion setters can
remove this ambiguity. X and A represent initially neutral punctate stimuli, which as a conse-
quence of training has resulted in A becoming an excitatory conditioned stimulus and X becoming
an occasion setter. + denotes “followed by”; different trial types that were given interspersed are
separated by I; the approximate magnitude of the observed conditioned responding is indicated
by the presence or absence of a conditioned response (CR) in the result column.
mental situations with a discriminative stimulus playing the role of an
occasion setter).
Although interest in occasion setting was dormant for several decades,
there has been a recent resurgence of interest in the topic, particularly in
Pavlovian situations. This resurgence is a result in large part to the fact
that occasion setters have been found to modulate simple associations, an
unexpected phenomenon in the framework of reflexive Pavlovian re-
sponding in which the acquired potential of a stimulus is limited to being
able to directly elicit a conditioned response (see Pavlov, 1927). This find-
ing encourages the use of occasion-setting principles to augment associa-
tive theory in providing explanations of complex acquired behavior, par-
ticularly choice behavior. It should be emphasized that this renewed
interest in occasion setting relies mainly on the monumental work of Peter
Holland (e.g., 1992) and colleagues in illuminating the nature of occasion
setting. Central to our understanding of occasion setting are two early
findings: First, a single stimulus can simultaneously possess both excitatory
(i.e., Pavlovian) and modulatory (i.e., occasion-setting) properties, and
these two properties are (almost) independent of each other (e.g., Holland,
4
ANALOGIES: OCCASION SETTING AND CONDITIONING
1983, 1985; Rescorla, 1985, 1986, 1987; Ross, 1983). In actuality, positive
occasion-setting training often gives the occasion-setting stimulus both
occasion-setting properties and Pavlovian excitatory properties. Nonrein-
forced presentations of the occasion setter alone during or after training,
however, commonly eliminate (i.e., extinguish) the Pavlovian excitatory
properties of the stimulus without influencing its occasion-setting proper-
ties (see “Extinction,” later in this chapter). Second, Holland and other
researchers have determined that occasion setters affect behavior through
some means other than their associations to the unconditioned stimulus
(US) summating with CS-US associations (e.g., Holland, 1983, 1985,
1989). These two important findings themselves do not illuminate the way
in which occasion setters act to modulate Pavlovian responding or the
conditions under which a stimulus will come or not come to serve as an
occasion setter. This chapter primarily addresses the latter question.
Like many researchers, our interest in occasion setting was kindled
primarily by its potential use as a means of accounting for complex learn-
ing phenomena within the framework of the simple and relatively well
understood principles of Pavlovian conditioning. Our presentation in this
chapter will be relatively nontheoretical. In our view, models at the associa-
tive or cognitive levels rarely reflect actual psychological mechanisms
(which potentially provide explanations of behavior at a more molecular
level of explanation), but instead serve primarily as heuristic devices to
guide us toward discovering important new behavioral phenomena. Our
heuristic with respect to this chapter is the potential analogy between serial
occasion-setting phenomena and the phenomena discovered during 80
years of research in Pavlovian conditioning.
EXTINCTION
The earliest phenomenon that led us to think about a set of analogies
between serial occasion setting and excitatory Pavlovian conditioning was
extinction. In Pavlovian conditioning, extinction is often described opera-
tionally as repeated nonreinforced presentations of the CS, which results
in a decrement in conditioned responding (see top part of Exhibit 2).
5
MILLER A N D OBERLING
~ ~~ ~
Group Train Extinguish Test Result
Extinction in Pavlovian conditioning (Pavlov, 1927)
Extinction A+US A A cr
Control A+US B A CR
Extinction in occasion setting (Rescorla, 1986; see also 1983)
Ross,
OSing X+A+US/AN+B+US/B Context only X+B CR
B cr
Exp. 1 X+A+US/AN+B-+US/B X X+B CR
B cr
Exp. 2 X+A+US/A/Y+B+USIB X+A/A+US X+B cr
B cr
Exp. 3 X+A+US/AN+B+US/B X+A X+B cr
B Cr
NOTE X, Y, A, and B were initially neutral stimuli. --f denotes "followed by"; different trial
types that were given interspersed are separated by /; the approximate magnitude of the observed
conditioned responding is indicated by the case, size, and boldness of the font used in the result
column. Only select groups are represented here.
Seeking an analogy to Pavlovian extinction within occasion setting, Res-
corla (1986; see also Ross, 1983) initially examined the effects of presenting
the occasion setter alone (i.e., group Exp. 1 in the bottom part of Exhibit
2). (Note that throughout this chapter, we ignore procedural details and
the fine grain of the data of individual studies. For this information, readers
should consult the primary sources.) As indicated in Exhibit 2, Rescorla
found that simple, nonreinforced presentation of occasion setter X did
not attenuate occasion setting by X (group Exp. 1). However, duplication
of the training treatment with the reinforcement contingencies reversed
from those of training did attenuate occasion setting. That is, reinforce-
ment on trial types that lacked reinforcement during initial occasion-
setting training and nonreinforcement on trial types that were reinforced
during initial training sufficed to degrade the initial occasion-setting value
of a stimulus. Moreover, this outcome was observed with or without
6
ANALOGIES: OCCASION SETTING AND CONDITIONING
A+US trials (groups Exp. 2 and Exp. 3, respectively). Thus, with regard
to behavioral consequences, presentation of X alone is actually a poor
analogy to Pavlovian extinction. If extinction treatment is operationalized
as exposure to the training conditions (either for all trials or only for
those trials involving the feature) with the trial-by-trial reinforcement
contingencies reversed, however, then a good analogy of the extinction of
Pavlovian excitation exists within occasion setting. This analogy follows
the same operational rule for decrementing stimulus control by an occa-
sion setter as by a Pavlovian excitor (i.e., training conditions with reversal
.'
of the reinforcement contingencies)
TEMPORAL CODING
In recent years, considerable attention in Pavlovian conditioning has been
directed toward the role of timing, beyond the long-recognized benefits
for behavioral control of close temporal proximity. Those espousing the
traditional view assume that temporal proximity is a catalyst for Pavlovian
conditioning. In contrast, recent research has suggested that (a) Pavlovian
associations include information about the temporal relationship of the
paired events, and (b) this temporal information is used by subjects in
determining the form and timing of conditioned responding (see Miller
& Barnet, 1993, for a brief review).
One example of such a demonstration is provided by Barnet, Cole,
and Miller (1997). In this study (see Exhibit 3), group 1st-ORDER dis-
~ ~~~
'Although it is a digression from occasion setting, consider for a moment the implications of this finding
for our conceptualization of extinction in general-that is, rather than defining extinction in the abstract
as nonreinforced presentation ofa stimulus that previously had acquired control over behavior, extinction
might be viewed more generally duplication of the conditions of training with the trial-by-trial rein-
as
forcement contingencies reversed. Thus, our seeking analogies to Pavlovian conditioning in order to
better understand occasion setting not only might tell us something about occasion setting but also about
the real essence of the phenomenon in question (e.g., extinction) in a variety of situations, thereby freeing
us from conceptualizations that are specific to a single situation, such simple Pavlovian excitatory
as
conditioning. This approach immediately gives greater generality to the phenomenon of extinction. For
example, extinction, operationally defined simply nonreinforced exposure to a stimulus, is not applica-
as
ble to a stimulus following Pavlovian inhibition training (i.e., A+US/AB), because this treatment does
not yield a loss of inhibitory control by stimulus B. However, extinction, operationally defined as a
reversal of the reinforcement conditions of Pavlovian inhibition training (i.e., A/AB+US treatment
following A+US/AB training), does degrade B si nhibitory value. Thus, in this framework, the conclusion
that inhibition is not subject to extinction is incorrect.
7
MILLER A N D OBERLING
Temporal Coding in Pavlovian Conditioning
(As Demonstrated in Sensory Preconditioning, Barnet, Cole, & Miller, 1997)
Group Phase L Phase 2 Test Result
EXP s3+s1/s4-+s2 s1+us/us+s2 s3 CR
s4 CR
CONTROL s1/s2/s3/s4 s1-+us/us+s2 s3 -
s4 -
1s t-ORDER S3+SlIS4-+S2 s1+us/us+s2 s1 CR
s2 cr
NOTE: S1, S2, S3, and S4 were initially neutral stimuli. + denotes “followed by”; different trial
types that were given interspersed are separated by I;t he approximate magnitude of the observed
conditioned responding (CR) is indicated by the case, size, and boldness of the font used in the
result column.
played the well-known superiority of forward (Sl) over backward (S2)
CSs with regard to subsequent behavioral control. But when second-order
cues were paired with the first-order forward and backward CSs (i.e.,
group EXP), stronger excitatory responding to the second-order CS (S4),
which had been paired with the backward first-order CS (S2), was seen
than to the second-order CS (S3), which had been paired with the forward
first-order CS (Sl). This observation is surprising in light of the prevailing
view that responding to the second-order CS is mediated by the first-order
CS. In that framework, more vigorous responding to a first-order CS (in
this case the forward paired CS) should produce more vigorous responding
to the second-order CS that had been paired with it. Barnet et al. explained
the observed data in terms of an integration of temporal maps from each
of the two phases of training, so that, in the integrated maps, onset of S4
had better temporal contiguity with onset of the US than did onset of S3.
This and many other studies (e.g., Matzel, Held, h Miller, 1988) strongly
indicate that CS-US temporal relationships are an inexorable part of the
content of Pavlovian associations.
Now consider a recent demonstration of temporal encoding in occa-
sion setting reported in a study by Holland, Hamlin, and Parsons (1997).
8
ANALOGIES: OCCASION SETTING AND CONDITIONING
~ ~~~ ~~
Temporal Coding in Occasion Setting
(Holland, Hamlin, & Parsons, 1997)
Train Test Result
os.L~cs.L+us/cs.L/os.s+cs.s+us/cs.s OS.L+CS.L CR
CS.L cr
os.s+cs.s CR
cs.s cr
0S.L-cS.L CR
CS.L cr
os.s+cs.s CR
cs.s cr
OS.L--+CS,S CR
cs.s cr
OS.S+CS.L CR
CS.L cr
OS.L+CS.S CR
cs.s cr
OS.S----+CS.L CR
CS.L cr
NOTE: 0S.S was the occasion setter (0s) trained with a short (S) occasion setter onset to CS
onset interval; 0S.L was the occasion setter (0s)t rained with a long (L) occasion setter onset to
CS onset interval, + denotes “followed by” with a short interval; +d enotes “followed by”
with a long interval; CS.S was the CS trained with a short (S) occasion setter to CS interval; CS.L
was the CS trained with a long (L) occasion setter to CS interval; different trial types that were
given interspersed are separated by /; the approximate magnitude of the observed conditioned
responding (CR) is indicated by the case, size, and boldness of the font used in the result column.
In this study, occasion setters were trained with short or long intervals
between their onset and that of their accompanying target CSs. Each fea-
ture was then tested for transfer to both its own training target CS and
the target CS used in training the other occasion setter. Moreover, at test,
intervals between termination of the feature and onset of the test CS
included both the short and long intervals of training for all combinations
of features and target CSs (see Exhibit Holland and his colleagues
4).
9
MILLER AND OBERLING
Overshadowing Pavlovian Excitation
in
(Kaufman & Bolles, 1981; Matzel, Schachtman, & Miller, 1985)
Group Train Extinguish A Test Result
ov BA-tUS Context only A cr
CONTROL A-tUS Context only A CR
RECOV BA-tlJS B A CR
~~ ~
NOTE: B and A were initially neutral stimuli, with B being more salient than A. The approximate
magnitude of the observed conditioned responding (CR) is indicated by the case, size, and bold-
ness of the font used in the result column. Only select groups are represented here.
found that occasion setters retained their modulatory potential best on
these transfer tests when they were conducted with an interval between
their onset and the onset of the test target CS that was the same as the
interval that prevailed during training of the feature. However, the modu-
latory potential of the occasion setter was unaffected by the occasion
setter-target CS interval that prevailed during training of the target CS.
Thus, the information encoded about a stimulus that allows it to serve as
an occasion setter appears to include the interval from the onset of the
occasion setter to the onset of a target CS-that is, feature-target inter-
vals-are encoded as part of the content of occasion setting. This property
of occasion setters is analogous to the previously discussed propensity of
conventional CS representations to be encoded to include information
concerning the interval between CS and US onset.
OVERSHADOWING
Let us consider overshadowing. When two stimuli are presented in com-
pound and reinforced (e.g., BA-US), each cue (particularly the less salient
cue) accrues less potential to elicit conditioned responding than does the
same cue when reinforced alone (i-e., A+US). Exhibit 5 illustrates over-
shadowing of CS A by CS B in group OV relative to group CONTROL.
Notably, Kaufman and Bolles (1981) and Matzel, Schachtman, and Miller
(1985) found that recovery of responding to overshadowed cue (A) can
10
ANALOGIES: OCCASION SETTING AND CONDITIONING
be obtained through massive posttraining extinction of the overshadowing
cue (B, see group RECOV in Exhibit 5). This recovery from overshadowing
without further training strongly suggests that overshadowing is a result
at least in part of a failure to express an A-US association that was acquired
during training, rather than a failure to acquire the A-US association as
assumed by most models of learning (e.g., Rescorla & Wagner, 1972).
We recently conducted a series of studies that examined overshadow-
ing of potential occasion setters (Gunther, Cole, & Miller, 1997). In this
and all subsequent studies from our laboratory that we will describe, the
basic procedure was conditioned suppression of licking by water-deprived
rats. Lights, tones, and clicks served as the controlling stimuli, and a brief,
mild foot shock served as the unconditioned stimulus. Duration of condi-
tioned suppression of ongoing drinking behavior was our basic dependent
variable. In this first series of studies, the occasion setters were local context
cues that cycled 5 min on, then 5 min off. More specifically, the overshad-
owing occasion setter was a bright flashing light, and the overshadowed
occasion setter was a soft click train. The three positive occasion-setting
studies of this series are represented in Exhibit 6. Experiment 1 merely
documented in our preparation occasion setting by stimulus X (as opposed
to the possibility that X came to simply serve as another excitatory Pavlov-
ian CS that associatively summated with A). This was done by taking
advantage of the knowledge that occasion setters do not transfer their
modulation to unambiguous CSs (e.g., Holland, 1985; Rescorla, 1985)
such as CS B, which was reinforced during training both in the presence
and absence of X. As can be seen in Exhibit 6, X modulated responding
to A but not to B. Notably, responding to B was not at ceiling, so if X had
had Pavlovian excitatory potential, it would have been expected to aug-
ment responding to CS B as well as CS A. The modulation of conditioned
responding of CS A, but not CS B, indicates that our preparation estab-
lished X as an occasion setter when no competing occasion setter was
present.
In Experiment 2 (see Exhibit 6), group CONTROL was trained with
stimulus X as an occasion setter for target CS A using exactly the same
parameters as in Experiment 1. For group OV, however, X was com-
11
MILLER AND OBERLING
Overshadowing in Occasion Setting
(Gunther, Cole, & Miller, 1997)
~~ ~~
Group Train Extinction Test Result
Exp. 1 X -A~jU S/A/XjB+US/B+ US X+A CR
A cr
X+B CR
B CR
Exp. 2 OV YX+ AAJSIA X+A CR
A cr
CONTROL X-+A-+US/A Y+A CR
X+A CR
A cr
Exp. 3 OV XY+A+US/A Context X+A CR
A CT
EXTINCTION XY+A+US/A Y-tA X+A CR
A cr
NOTE. X, Y, A, and B were initially neutral stimuli. X+A+US denotes X onsetting before the
A+US trial with X staying on for some time after the completion of the trial (i.e., X was a local
context that served as an occasion setter); different trial types that were given interspersed are
separated by /; the approximate magnitude of the observed conditioned responding (CR) is
indicated by the case, size, and boldness of the font used in the result columns.
pounded (simultaneous onset, simultaneous termination) with the more
salient stimulus Y. As indicated in Exhibit 6, responding to A in the pres-
ence of X was greater for group CONTROL than for group OV. This
demonstrates overshadowing of X as an occasion setter by Y. In Exp. 3,
group OV received the same treatment as group OV in Exp. 2 and yielded
similar weak occasion setting. However, group EXTINCTION had the
occasion-setting value of Y degraded following overshadowing treatment.
Enhanced modulatory control of responding to A by X was observed as
a result of this degradation of Y. In fact, X exhibited modulatory control
in group EXTINCTION comparable to that observed in group CONTROL
of Exp. 2. Thus, we see that, analogous to Pavlovian conditioned excitation,
both overshadowing and recovery from overshadowing as a result of de-
12