|Home | About | Journals | Submit | Contact Us | Français|
Operant generalization has been demonstrated in neonates only recently. To investigate the development of intradimensional stimulus control immediately after hatching, northern bobwhite chicks (Colinus virginianus) pecked for brief heat presentations while hearing a high-pitched sound repeated at two constant rates: an S+ tempo signaling a rich reinforcement schedule, alternating with an S− tempo signaling a leaner schedule. Tempo generalization was then assessed in extinction. The expected excitatory gradients were produced after a threshold number of training sessions; unexpectedly, below that threshold, gradients were inhibitory. The chicks' rapidly developing thermoregulatory capability may have resulted in a change from perceived negative reinforcement initially to positive reinforcement later. Given past research showing excitatory gradients after negative reinforcement, we suggest that these results demonstrate that all negative reinforcement is not equivalent, and, further, that classical conditioning effects require consideration.
Generalization, a ubiquitous and important phenomenon, has been widely studied (e.g., Ghirlanda and Enquist, 2003; McLaren and Mackintosh, 2002). However, only recently has operant generalization been demonstrated in neonates (Hauf, Prior, and Sarris, 2008; Schneider and Lickliter, 2009), and little is known about its properties in early development. Further, temporal generalization has long been known to present special challenges (for current research with adults, see, e.g., Roberts, 2006; Wearden, 2008). Detection of the temporal characteristics of the species-typical vocalizations in many bird species is particularly important for communication and survival, and that includes non-passerines like quail (e.g., Stokes, 1967). We studied tempo generalization in a precocial quail species, the northern bobwhite (Colinus virginianus).
The most basic theoretical questions addressed by developmental research on generalization concern its provenance and range (e.g., Mackintosh, 1977). Such work with classical conditioning and habituation generalization in neonates has revealed some surprises (e.g., Hyson and Rudy, 1987; Kerr, Ostapoff, and Rubel, 1979). Of particular note, in a shock-based free-operant conditioned suppression procedure, Frieman, Warner, and Riccio (1970) found a steeper suppression ratio gradient for rat pups than for adults, with overall suppression greater for adults. These results contrasted to those of Campbell and Haroutunian (1983), who found flat gradients for younger rats on a habituation measure, while rats only slightly older showed steeply peaked gradients (also see Rubel and Rosenthal, 1975). Campbell and Haroutunian pointed out that the development of Pavlovian generalization gradients appears to be nonmonotonic in some cases.
In a previous study (Schneider and Lickliter, 2009), northern bobwhite neonates received nondifferential operant training to a constant auditory tempo, with tempo as the tested dimension. Despite minimal exposure, typical generalization gradients resulted. The present study investigated the effect of prior stimulus discrimination training within the generalization dimension.
In such intradimensional training, an S+ is normally associated with a reinforcement schedule, and an S− on the same stimulus dimension is associated with extinction. After sufficient training, the peak of the generalization gradient is often displaced away from the S+, on the side distant from the S−. The basis for this “peak shift” is still the subject of empirical and theoretical debate (e.g., Bizo and McMahon, 2007; Ghirlanda and Enquist, 2003, 2007). Regardless of whether peak shift occurs, intradimensional training usually produces the steepest gradients and thus the greatest degree of stimulus control, compared to other forms of experience with the stimuli (Rilling, 1977).
As a precocial species, the northern bobwhite offered an opportunity to follow the developmental trajectory of generalization from shortly after hatching. We focused particularly on the rapidity of acquisition of the intradimensional discrimination. Because the chicks weigh only 6 to 7 g upon hatching and have limited thermoregulatory ability (Borchelt and Ringer, 1973), rich schedules of heat reinforcement were essential. Pilot testing showed that any extended imposition of extinction in our paradigm risked the complete cessation of pecking, so a lean reinforcement schedule rather than extinction was associated with the S−. This usage of two reinforcement schedules also enabled investigation of the degree to which neonates can discriminate relatively small differences in reinforcement rate over time, an important skill that is well developed in adults of most vertebrate species (e.g., Davison and McCarthy, 1988; Loewenstein and Seung, 2006).
In our between-groups procedure, chicks were autoshaped at 24 hours of age. Rich and lean variable ratio (VR) heat reinforcement schedules were signaled by an ecologically valid sound repeated at two different tempos. After 3 to 5 days, a generalization test was administered in extinction. We expected to show differential control by the S+ and the S−, manifested as an excitatory gradient for the S+; results for the nominal S− were harder to predict because of its mild nature. We did not expect peak shift with the minimal amount of exposure that would be provided.
Commercially supplied northern bobwhite eggs were incubated and hatched in our lab under sound-attenuated conditions, and the chicks were reared socially in a sound-attenuated rearing room maintained at 33 C. Four to seven chicks of the same age were kept in bins (430 mm × 250 mm × 150 mm) that provided free access to food and water. New groups of chicks were hatched weekly (see Schneider and Lickliter, 2009).
Isolation in the operant chamber was expected to be stressful for the chicks--members of a highly social species--so a fuzzy surrogate of appropriate size, texture, and color pattern was placed in each rearing bin. A surrogate was always present in the back of the operant chamber.
The operant chamber was a box that fitted against a flatscreen monitor with an infrared-based “touchscreen” detection frame attached (Elotouch Extended Resolution; for details, see Schneider and Lickliter, 2009). One computer speaker was placed on each side, and a 250-W heat lamp hung over the chick's position. The apparatus, a larger sound-attenuating chamber, and the controlling computer were all housed inside a completely partitioned enclosure.
The auditory stimulus was a complex 90-ms, 75-dB tonal “beep” designed to be salient for the chicks (for its ecologically valid acoustic characteristics, see Schneider and Lickliter, 2009). An inter-beep interval of 259 ms provided the constant S+ tempo (in the generalization test, this became inter-beep interval 3). The S− had an inter-beep interval of 199 ms (inter-beep interval 5).
Autoshaping was used to establish pecking at the target, which was a white rectangle (105 mm × 50 mm) on a dark red background. The target was centered at chick height. A 10 s target on/10 s target off cycle was accompanied by 0.3 s of heat from the heat lamp at the moment the target disappeared. A peck to the target turned it off and initiated heat delivery, accompanied by a relay click.
After the first four such pecks, autoshaping ended, the target remained on, and a continuous reinforcement schedule was instituted. When fewer than 10 pecks had accumulated, and when a responding pause of more than 30 s occurred, the 10 s/10 s autoshaping sequence restarted. To enhance response acquisition and maintenance, post-autoshaping target pecks produced a 0.2-s blink-off of the target as feedback. After 25 pecks, a constant-probability-based variable ratio 1.25 took effect for the remainder of the session. (Because .8 is its inverse value, on average 8 of every 10 pecks were reinforced.) The VR schedule included two additional features to enhance responding: First, when an interresponse time greater than 10 s occurred, the peck that ended it was reinforced. Second, a differential-reinforcement-of-high-rate feature provided reinforcement for all pecks with interresponse times shorter than 1.5 s. Whenever the target was present, so was the sound stimulus at the S+ tempo.
During the 15-min autoshaping session, 87% of the chicks pecked at least once; most pecked between 30 and 200 times. All chicks that pecked at least 30 times were included, along with some that pecked in the range of 1-30. (Chicks with low autoshaping rates had a somewhat decreased likelihood of completing the study, but their data were otherwise indistinguishable.) The autoshaping session was run when the chicks were 24 hrs of age.
Each chick was tested twice each day for 3 to 5 days, including the autoshaping session on the first day. After autoshaping, differential training took place for 4, 5, 6, 7, or 8 sessions, followed by a generalization test session. We intended to obtain 5 gradients at each level. However, a transition occurred at 6 training sessions, so we gathered 4 extra gradients at that level. At 8 training sessions, after 2 of the first 4 chicks produced what appeared to be excitatory double peak shift gradients, we stopped running at that training level (see Figure 1 for these exceptional gradients). Twenty-eight gradients were thus obtained, but these two were excluded from further analysis.
The rich schedule associated with inter-beep interval 3 throughout these training sessions was a VR 1.18 in which, on average, 85 of every 100 responses were reinforced. As in the autoshaping session, this schedule was accompanied by two added features to help maintain responding: Pecks that occurred after pauses of at least 10 s were always reinforced, and so were pecks that occurred after pauses of no more than 1.5 s since the previous peck. The lean schedule associated with inter-beep interval 5 was a simple VR 2 with no additional features (i.e., on average, 50 of every 100 responses were reinforced). The schedule at the start of training was thus a multiple VR 1.18 (inter-beep interval 3) VR 2 (inter-beep interval 5). A 0.5-s period of silence occurred between components, and responses during this brief pause were consequated according to the schedule that was just ending. The same visual stimulation was present on the screen throughout training (i.e., the white target on the dark background).
Pilot testing had shown that some chicks did not maintain responding adequately on a VR 2 schedule, so it was introduced gradually; this approach also meant less predictability in the schedule changes, and thus more likelihood that discrimination would be made based on the auditory stimuli and not just on characteristics of the schedules themselves. Chicks progressed from a cycle of 50 s on the rich schedule and 10 s on the lean (Mult 1) to a 45 s/15 s alternation (Mult 2). The normal progression was two sessions on each, ending with sessions on Mult 3 (a 40 s/20 s alternation). In Mult 3, the differential-reinforcement-of-high-rate parameter associated with the rich schedule was changed from 1.5 s to 1.0 s. To provide a minimum level of exposure to the S−, all birds had at least 2 sessions of experience on Mult 2 prior to their generalization test. Thus, the normal progression was autoshaping - Mult 1 (2 sessions) - Mult 2 (2 sessions) - Mult 3. Minor variability in the progression occurred, depending on individual performance; similarly, most sessions were 15 min, but a few at 6 or more training sessions were 20 min for the heavier chicks, because of their longer latency to respond.
Throughout the study, starting chamber temperatures were adjusted based on age and weight in an attempt to achieve constant motivation across and within chicks. Toward this end, relatively warm temperatures were used (20-24 C), just sufficient to maintain adequate responding. As a result, attrition was relatively high (as is typical in neonate research). To equalize chick weight distribution across training session category, matching based on weight was utilized to the degree possible.
The generalization test session was either 15.5 or 20.5 min long, depending on the lighter or heavier weight of the chick. Training on Mult 3 for either 5 or 10 min began the generalization test session, depending on the session duration. In both cases, the last 10.5 min consisted of 10 presentations each in extinction of 7 constant beep rates, each 8.5 s in duration and succeeded by a 0.5-s pause. Each inter-beep interval differed from the next by a constant difference in spacing: 30 ms. The 90-ms beep was followed by 319 ms of silence as inter-beep interval 1, the slowest rate, and 139 ms as inter-beep interval 7, the fastest. The 7 beep rates occurred in 10 cycles, each in random order without replacement. To ensure a sufficient sample size, data were used only when a chick responded at least 50 times during the 10.5-min generalization test (i.e., about 5.0 resp/min). Although the subjects were neonates run at relatively warm temperatures, 65% of those that started training met the generalization test response requirement (i.e., 28 out of 43, a proportion nearly identical to that of Schneider and Lickliter, 2009; because of the exclusion of the two gradients in Figure 1, 26 gradients were analyzed in the Results).
The 26 gradients showed control by the S+ and the S−, as predicted. Unexpectedly, however, inhibitory gradients occurred after less training, and excitatory gradients occurred only after more. The Appendix provides complete data, grouped on the basis of number of training sessions: “fewer” (4 and 5 training sessions), “transition” (6 training sessions), and “more” (7 and 8 training sessions). Despite the differences in gradient form, the Appendix shows that average training response rates remained similar across the groups, suggesting success in our effort to maintain similar motivation levels as the chicks grew.1 This was the case even though, as expected, average weight increased over time.
Gradient classification was as follows: Excitatory gradients had a peak at or adjacent to the S+, with endpoints generally lower than the rest of the gradient. Inhibitory gradients had a low point at or adjacent to the S−, with the endpoints generally higher than the rest of the gradient. As the Appendix shows, after 4 or 5 training sessions, gradients were consistently inhibitory; after 7 or 8, excitatory. Four “exceptional” gradients could not easily be categorized, two of which fell in the transitional category of 6 training sessions. Within this group, high-weight chicks tended to produce excitatory gradients, and low-weight chicks, inhibitory gradients.
Figure 2 shows the average absolute and relative gradients for the ten chicks with fewer (4 and 5) training sessions. The relative results are based on the proportion of responding at each test stimulus, rather than the absolute response rates: Any individual gradient has the same shape when presented in either form, but averaged absolute and relative gradients are usually somewhat different. The relative method of presentation eliminates the bias otherwise introduced by chicks with unusually high response rates (in this case, three chicks with rates over 30 resp/min; for the same reason, as is typical for this type of research, no variability measure is provided for the average absolute results). The only excitatory gradient in this group came from the chick (F10) with the highest generalization test response rate in the study.
A sign test compared relative responding at the S− to the average relative level (i.e., simply 1/7, or .14). For 9 of the 10 gradients, S− responding was below the average (χ2 = 6.4, df=1, p<.05), confirming the inhibitory tendency.2
At the transitional level of 6 training sessions, the Appendix shows that the 9 chicks produced 4 inhibitory, 3 excitatory, and 2 exceptional gradients. The three lowest-weight chicks had inhibitory gradients, as did the chick with the lowest generalization test response rate. Figure 3 shows the average result, a flat gradient.
Consistently excitatory gradients resulted when chicks were older and had received more (7 or 8) training sessions, as is evident from Figure 4. Both the absolute and the relative average gradients show a pronounced peak at the S+. All seven chicks produced relative S+ results greater than the average (χ2 = 7.0, p<.01).3
After only a few hours of intradimensional training, northern bobwhite neonates were able to discriminate between both different auditory tempos and different reinforcement rates. Unexpectedly, however, their generalization gradients were inhibitory after fewer training sessions and excitatory only after more. We will briefly discuss the discrimination results first.
Reinforcement schedule discrimination was evident despite the relatively minor difference between the schedules and the short period of exposure. The chicks appear to require only minimal experience to develop some sensitivity to behavior-consequence relations over time. Intradimensional training typically produces more steeply peaked gradients than nondifferential training, and this expectation appeared to be confirmed: The nondifferential-training peak from Schneider and Lickliter's data (2009) was short and shallow, contrasting with the steeper, higher relative peak for Figure 4. (However, the relative nearness of the S− to the S+ is a confound, as is the unusual nature of the S−.)
With respect to temporal generalization, Roberts (2006) concluded that insufficient data existed to determine whether the discrimination follows a logarithmic or linear basis. While our study was not designed to address this question, for our “absolute interval” linear basis, the average generalization gradients did suggest a “mirror image” for the excitatory and inhibitory outcomes. Honig, Boneau, Burstein, and Pennypacker (1963) developed an interdimensional generalization paradigm that provided exact corollaries for the S+ and the S− (extinction), and found mirror-image excitatory and inhibitory relative gradients (also see Rilling, 1977). These findings suggest that generalization operates similarly in this respect across reinforcers and aversives. Our data suggest that there are some differences as well, however.
The presence of inhibitory gradients was unexpected: Under either positive or negative reinforcement, both intradimensional and nondifferential generalization gradients are normally excitatory in form, with a pronounced peak either at the S+ or at its adjacent peak-shifted equivalent (e.g., Hearst, 1962; Hoffman and Fleshler, 1963; and Sidman, 1961, who studied intradimensional avoidance training using tempo as the stimulus dimension; see Rilling, 1977). Indeed, Weiss and Schindler (1981) found very similar excitatory relative generalization gradients to the target stimulus after intradimensional training with either positive food reinforcement or shock avoidance, contrasted to extinction. Similarly, after interdimensional training with an S+ shock-avoidance schedule, excitatory gradients occur (e.g., Rilling and Budnik, 1975).
Inhibitory gradients are usually produced only after low response rates in the presence of an associated aversive, such as shock or extinction, in contrast to a positive reinforcement baseline. Standard S+/S− interdimensional training provides the classic examples (e.g., Bowers and Richards, 1986; Jenkins and Harrison, 1962). In his review, Rilling (1977) suggested that inhibitory gradients might occur only at the S− after appropriate interdimensional training; he did not anticipate the existence of intradimensional inhibitory gradients. The occurrence in our data of inhibitory gradients appears to be unprecedented; so is the occurrence of both inhibitory and excitatory gradients in the same context (i.e., no change in experimental variables).4
As a possible explanation, we suggest the classical conditioning effects of the aversives present in negative reinforcement. Separating positive and negative reinforcement can be surprisingly difficult: Were our chicks responding primarily to produce the positive reinforcer of heat, to reduce the aversive condition of being cool (negative reinforcement), or to do both simultaneously to varying individual degrees (see Baron and Galizio, 2005; Magoon and Critchfield, 2008; Michael, 1975)? The behavior can appear exactly the same in each case. However, differences in the classical conditioning associated with aversives and appetitives could provide a means to distinguish between positive and negative reinforcement (see, e.g., Staats, 2006; again, both aversive and appetitive features may be present in different proportions, presumably along a continuum in the case of subjective warmth). The terms positive and negative would be useful as a function of the presence or absence of degrees of relative aversiveness.
Among the variables that can affect the perceived aversiveness are age and weight, both of which are correlated with the number of training sessions. For example, 75% of the chicks producing inhibitory gradients weighed 8 g or less, but only 27% of those producing excitatory gradients. Chicks that weigh less tend to be smaller and thus have a higher surface to volume ratio, making thermoregulation harder. In addition, at younger ages, the physiological ability for thermoregulation is simply less developed: Borchelt and Ringer (1973) showed for this species a steady and significant increase in thermoregulation over these first five days after hatching. Response rate appears to be a secondary variable, but, other factors being equal, birds that pecked more received more heat and experienced a less aversive situation. Thus, chicks that had aversive associations with training (i.e., primarily negative reinforcement), may have produced inhibitory gradients. In effect, the chicks may have been reporting their hedonic states--and revealing to us when negative reinforcement changed to positive.
As anecdotal support for our hypothesis, contaminated feed caused the chicks to become sick one week. Three of them were run on the generalization test after 7 training sessions, when they had started to lose weight (two were below 8 g) but still appeared to be behaving normally. Excitatory gradients would be expected at this level of training, and response rates were high, but two of the gradients were strongly inhibitory and the third was exceptional. Sickness is an aversive, of course. (These data were excluded from our analyses.)
Shock avoidance, the basis for most existing negative reinforcement-based generalization gradients, seems clearly to be negative reinforcement: the avoidance of an aversive, rather than the presentation of an appetitive. Yet gradients are excitatory, not inhibitory. However, the rats and pigeons commonly used in shock avoidance studies become proficient and rarely experience the aversive (although the situation itself can remain aversive, e.g., Courtney and Perone, 1992). Our motivational modality was inherently different, especially for this very young, rapidly developing population. Despite our attempts to provide only mildly cool temperatures that would make the heat a positive reinforcer, temperatures cold enough to be aversive may have been essentially unavoidable at the youngest ages. In that case, our preparation might have had more in common with the associated-aversives paradigm than with shock avoidance.
Most of the classical conditioning generalization literature corresponds with the interdimensional operant procedure (e.g., the conditioned suppression procedure of Frieman et al., 1970, in which positively reinforced baseline responding provided a contrast for their inhibitory results). Results that may be more comparable to ours come, for example, from the shuttle box procedure of Frieman, Rohrbaugh, and Riccio (1969). Juvenile and adult rats were given forced exposure to shock during an auditory signal, after which the amount of time spent on that side of the shuttle box was measured during the presentation of different signals. A steep inhibitory gradient resulted for most of the rat pups and a few of the adults (other adult gradients were flat; excitatory gradients in this context would have been puzzling indeed). Further, Rilling and Budnik (1975) found that including aversives during a generalization test as well as training added to an inhibitory effect.
Under this hypothesis, inhibitory gradients normally occur whenever an S− associated with significant relative aversiveness is present, especially if it is also associated with a decrement in responding, as in the interdimensional associated-aversives paradigm, or in Frieman et al.'s (1969) unavoidable shock paradigm. Accordingly, Wilkie (1974) showed that stimuli associated with the slow-responding, relatively aversive start of an interval on a fixed interval positive reinforcement schedule produced inhibitory gradients; those associated with the end, excitatory. Inhibitory gradients do not occur for classic nondifferential shock-avoidance generalization because the level of experienced aversives is, we suggest, relatively low. The generally low maintained response rates in our heat-based paradigm may have been a secondary factor tending toward an inhibitory result, although they were comparable to the response rates found in some of the avoidance paradigms that still produced excitatory gradients (e.g., Sidman, 1961).
If this hypothesis is correct, a more aversive context could potentially produce inhibitory gradients despite standard positive reinforcement training. We can also speculate that, when shock avoidance is the basis of generalization, occasional inhibitory or at least flatter gradients might occur for less proficient subjects. Suggestively, one study that utilized a differential-reinforcement-of-low-rate baseline schedule showed flatter gradients for poorly-performing subjects (Gray, 1976; methodologically, it is no more possible to control proficiency directly in these paradigms than it was in our study.)
To illustrate the complexity of such analyses, in a variation of the shock avoidance generalization paradigm, Dinsmoor and Sears (1973) ran an unsignaled (Sidman) avoidance procedure with pigeons in which each treadle foot-press delayed shock. The birds became proficient, as expected. During training, each press was followed by a brief auditory safety signal. In the generalization test, response-contingent signals of different frequencies were presented in extinction. All three birds showed an excitatory gradient. Thus, although few responses were made during the safety signal during training, and the overall context was that of negative reinforcement, the status of the signal as an excitatory S+ was evident, as revealed by the generalization test.
All negative reinforcement is clearly not equal, and generalization testing is one way to discover differences. The unexpected inhibitory results of our exploratory study reinforce the lesson that generalization gradient form can be a complex function of behavioral history, contextual, and developmental factors, in contrast to the more straightforward characterizations that have sometimes appeared to apply. We suggest in particular that the relationship between operant and Pavlovian effects in generalization deserves closer examination (see also Schneider, 2003). In other respects, the precocial quail chicks showed the capability for quickly discriminating between different auditory tempos and reinforcement rates, and producing otherwise-standard gradients as a result.
This research was supported by NIMH grant RO1-62225 and NICHD grant RO1-048423 (R.L.). We thank Christian Krägeloh for his helpful comments on a previous version of the manuscript.
|Fewer Training Sessions|
|Transition: 6 Training Sessions|
|More Training Sessions|
Gen Test Rate: Response rate during the 10.5-min generalization test
Avg Train: Average response rate during the post-autoshaping training sessions
2-sess Avg: Average response rate during the last 2 sessions of training
IBI3/IBI5: Ratio of responding to these stimuli during training
# Train Sess: Number of post-autoshaping training sessions
Weight: Weight immediately after the generalization test
Grad Type: Gradient type, categorized as excitatory (Excit.), inhibitory (Inhib.), flat, or exceptional (Except.)
NA: Data not available
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
1The higher average generalization test response rates compared to training rates reflect expected extinction bursting, seen in a number of chicks. The Appendix also shows average rich-to-lean response ratios that declined along with the similar component duration ratios, indicating that responding remained fairly constant across components, not surprising given the minor difference between the schedules. Thus, absolute responding at the S− often remained roughly similar in training and testing even for the chicks with inhibitory gradients.
2Four of the 10 gradients showed S+ responding above the average, χ2 = 0.4, p>.05. These inhibitory gradients were not expected so, depending on their explanation, no prediction can be made for S+ responding.
3No prediction can be made for responding at the S− because of its association with a lean reinforcement schedule rather than an aversive like extinction or shock. Five of the 7 chicks produced S− rates less than the average (χ2 = 1.3, p>.05). A lean schedule can be inhibitory in comparison to a rich one (e.g., Guttman, 1959; Migler and Millenson, 1969), but need not be (Rilling, 1977).
4Hearst, Besley, and Farthing (1970; with further details in Hearst and Sutton, 1993) appears to be the only other such study. Procedurally, Hearst et al. utilized initial interdimensional extinction training, followed by maintained generalization testing with positive reinforcement at each test point. What was surprising was that within-subject responding at the S− inverted from low point to peak, rather than stabilizing at a flat gradient after sufficient positively reinforced testing (see Hinson and Malone, 1980; Tennison and Hinson, 1993 for possibly related contrast effects). Our method was quite different, and no apparent relation exists.
Susan M. Schneider, Department of Psychology University of the Pacific and Florida International University, Stockton, CA 95211.
Robert Lickliter, University of the Pacific and Florida International University.