|Home | About | Journals | Submit | Contact Us | Français|
Because the occurrence of primary reinforcers in natural environments is relatively rare, conditioned reinforcement plays an important role in many accounts of behavior, including pathological behaviors such as the abuse of alcohol or drugs. As a result of pairing with natural or drug reinforcers, initially neutral cues acquire the ability to serve as reinforcers for subsequent learning. Accepting a major role for conditioned reinforcement in everyday learning is complicated by the often-evanescent nature of this phenomenon in the laboratory, especially when primary reinforcers are entirely absent from the test situation. Here we found that under certain conditions, the impact of conditioned reinforcement could be extended by lesions of the basolateral amygdala (BLA). Rats received first-order Pavlovian conditioning pairings of one visual conditioned stimulus (CS) with food prior to receiving excitotoxic or sham lesions of the BLA, and first-order pairings of another visual CS with food after that surgery. Finally, each rat received second-order pairings of a different auditory cue with each visual first-order CS. As in prior studies, relative to sham-lesioned control rats, lesioned rats were impaired in their acquisition of second-order conditioning to the auditory cue paired with the first-order CS that was trained after surgery. However, lesioned rats showed enhanced and prolonged second-order conditioning to the auditory cue paired with the first-order CS that was trained before amygdala damage was made. Implications for an enhanced role for conditioned reinforcement by drug-related cues after drug-induced alterations in neural plasticity are discussed.
Many accounts of behavior, including pathological behavior such as abuse of alcohol or drugs, make use of principles of simple associative learning. After pairing with emotionally-significant events, such as naturally-occurring or artificial reinforcers, initially neutral cues can acquire incentive motivation, the ability to guide behavior, and reinforcer expectancies. Because the occurrence of primary reinforcers is a relatively rare event, a key to applying conditioning principles to real environments is the concept of higher-order conditioning or conditioned reinforcement, whereby conditioned stimuli (CSs) acquire the ability to reinforce learning in lieu of primary reinforcers. As a result of their prior pairing with primary reinforcers, Golden Arches®, drug paraphernalia, and other stimuli may also serve as adequate reinforcers for the acquisition and control of behavior.
Recent research has provided considerable insight into brain systems and mechanisms involved in conditioned reinforcement in the laboratory, encouraging hopes that many examples of pathological behavior might be understood similarly (Everitt & Robbins, 2005). For example, intact function of a network including orbitofrontal cortex (OFC), basolateral amygdala (BLA) and nucleus accumbens (nAC) seems essential for normal conditioned reinforcement (e.g., Burke, Franz, Miller, & Schoenbaum, 2008; Everitt & Robbins, 2005; Setlow, Holland, & Gallagher, 2002; Theberge, Milton, Belin, Lee, & Everitt, 2010). We have suggested an especially important role for neuroplasticity in the BLA (Lindgren, Gallagher, & Holland, 2003; Setlow, Gallagher, & Holland, 2002): both acquisition of and extinction of a CS’s ability to reinforce second-order Pavlovian conditioning depend on intact BLA function.
A problem for understanding the role of conditioned reinforcement in the control of real-world behavior is that in the laboratory this phenomenon is often fleeting, especially when primary reinforcers are entirely absent from the situation. Just as a CS’s ability to serve as a conditioned reinforcer depends originally on its relation with a primary reinforcer, that ability extinguishes when its relation with the reinforcer is degraded, for example, if the CS is presented in the absence of the reinforcer.
Lindgren et al. (2003) found that BLA lesions made after a CS had already acquired conditioned reinforcement power rendered that power resistant to extinction by nonreinforced presentation of the CS. Although food-cup approach conditioned responses (CRs) to the CS extinguished normally, the CS retained its ability to reinforce second-order conditioning. Here we exploited that observation to produce Pavlovian second-order conditioning of greatly enhanced magnitude and persistence.
Twenty-four male Long-Evans rats (Charles River Laboratories, Raleigh, NC, USA) weighed 300–325 g on arrival at the laboratory vivarium, and were individually housed with water freely available in a colony room with a 12:12-hr light-dark cycle. They were given one week of free access to food prior to restriction of their food access to maintain 85% of their free-feeding weights for the study. The experiment was conducted in two replications, one at Duke University and one at Johns Hopkins University. The care and experimental treatment of rats were conducted according to the National Institutes of Health’s Guide for the Care and Use of Laboratory Animals, and protocols were approved by the Johns Hopkins University and Duke University Animal Care and Use Committees.
The behavioral training apparatus consisted of four separate chambers (22.9 × 20.3 × 20.3 cm). Each chamber had aluminum front and back walls, clear acrylic sides and top, and a floor of stainless steel rods (0.48 cm in diameter spaced 1.9 cm apart). A recessed food cup was located in the center of the front wall, 2 cm above the floor, and was fitted with phototransistors to detect head entries. The panel light conditioned stimulus (CS) was generated by illumination of a 6-W lamp with a translucent covering, centered 15 cm above the food cup, and the house light CS was generated by intermittent (3 Hz) illumination of 6-W bulb mounted on the inside wall of a sound-attenuating box that surrounded each chamber. A 1500-Hz, 80-dB tone CS and an 80-dB white noise CS were presented via a speaker mounted next to the house light lamp. A video camera and a constantly-illuminated 6-W lamp in a red lens were mounted next to the speaker.
The rats were anesthetized with either 50 mg/kg pentobarbital (Duke) or 2–3% isoflurane mixed with oxygen (Johns Hopkins) and placed into a stereotaxic apparatus (Model 902, Kopf, Tujunga, CA, USA). Fourteen rats received bilateral lesions of BLA, using stereotaxic coordinates (Paxinos & Watson, 1998) 2.8 mm posterior of bregma and 5.0 mm from the midline, with infusions at 8.7 and 8.4 mm ventral from the skull surface at the drill site. The lesions were made using 12.5 mg/mL N-methyl-D-aspartate (NMDA, Sigma, St. Louis, MO, USA) dissolved in phosphate-buffered saline (PBS) solution, infused with a 2.0-μL microsyringe (Hamilton, Reno, NV, USA) at a rate of 0.1 μl/min; 0.2 μL at the deeper site and 0.1 μL at the shallower site. Ten sham-lesion control rats received infusions of PBS alone to these sites, in the same manner. Each rat received a subcutaneous injection (0.02 mg/kg) of buprenorphine HCl (Sigma, St Louis, MO, USA) after surgery to ameliorate pain. The rats were allowed to recover for 10–14 days before being returned to the training protocol.
Figure 1A provides an outline of the behavioral training procedures. Each 64-min session in each phase of the experiment included 16 trials, distributed across random intertrial intervals, which averaged 4 min (range 2–6 min). In the first session, the rats were trained to eat food pellets from the food cups in a single session, which included 16 unsignaled reinforcer deliveries. The reinforcer used throughout was the delivery of two 45-mg grain pellets (Formula 5TUM, Test Diets, St Louis, MO, USA). Next, all rats received pre-surgical first-order conditioning of a 10-s visual CS, VPre (house light or panel light, counterbalanced). In each of the first 4 sessions of this phase, the rats received 16 reinforced presentations of VPre. In each of the next 4 sessions, they received 8 reinforced presentations of VPre intermixed randomly with 8 nonreinforced 10-s presentations of the other visual CS, VPost, to reduce generalized responding to VPost. Then, after surgery and recovery, rats received 16 10-s reinforced presentations of VPost in each of 4 sessions. Thus, rats acquired first-order CRs to VPre prior to surgery, and first-order CRs to VPost after surgery.
Finally, all rats received second-order conditioning of each of the two 10-s auditory CSs: APre was paired with VPre, and APost was paired with VPost. The identities of APre and APost (tone or noise) and their combinations with VPre and VPost were completely counterbalanced. In the first session, the rats initially received 4 presentations each of APre and APost alone as a pretest of responding, followed by 4 APre→VPre and 4 APost→VPost pairings, randomly intermixed. In each of the remaining 7 sessions, the rats received 8 APre→VPre and 8 APost→VPost pairings. For these sessions, the trial sequence was pseudo-randomly determined in each session, with the constraint that 4 of each trial type were presented in each half-session block.
Our expectation was that, compared to sham-lesioned rats, BLA-lesioned rats would show deficits in the acquisition of second-order CRs to APost, but because the conditioned reinforcement power of VPre established prior to the lesions would resist extinction, BLA-lesioned rats would show greater and more persistent second-order CRs to APre than sham-lesioned rats.
The measure of first-order conditioning to the visual CSs was the percentage of time spent in the food cup during the last 5-s of each CS, as assessed by interruption of the infrared photobeam. Food-cup CRs occurring in this interval are more frequent and less contaminated by conditioned orienting behaviors (Holland, 1977).
Second-order conditioning was assessed by observations made from video tapes. Previous studies showed that second-order CRs established to auditory second-order CSs paired with first-order visual VSs include general locomotion and behaviors directed to the food cup, fairly evenly distributed in time (Holland, 1977; Setlow, Gallagher, & Holland, 2002). Observations were made at 1.25-s intervals during the 10-s period prior to CS presentations (pre-CS), and during the 10-s CS auditory cue presentations. One and only one behavior was recorded at each observation. Second-order CRs were scored if a rat’s nose or head was in the recessed food cup, if a rat displayed rapid horizontal or vertical head movements (head-jerk), or if a rat was changing position (e.g., walking). The index of these behaviors was their absolute frequency over each 10-s period, expressed as a percentage of the number of observations made, which was constant for each measurement interval. One observer scored the data from all second-order conditioning sessions, and a second observer scored portions of those data; they agreed on 96% of 2872 joint observations, r = 0.97. Neither observer was aware of the rats’ lesion or training conditions when the data were scored.
Responding during the pre-CS, visual CS and auditory CS (when applicable) epochs were each analyzed with separate analyses of variance (ANOVAs), with BLA lesion condition (sham or NMDA) and two counterbalancing variables (identity of VPre, panel light or house light, and identity of APre, noise or tone) as between-subject variables, and repeated measures on the within-subjects variables of stimulus (either VPre or VPost, or APre or APost) and sessions or half-session blocks. The Greenhouse-Geisser procedure was used to correct for any violations of sphericity. Post-hoc contrasts used the Tukey honestly-significant difference (HSD) procedure for unequal ns. Effect sizes are given as partial η2.
The rats were deeply anesthetized with either 100-mg/kg sodium pentobarbital (Duke) or isoflurane (Johns Hopkins) and perfused intracardially with 0.9% saline followed by 3.7% formalin solution. The brains were removed and stored at 4°C in a 3.7% formalin+12% sucrose solution. Coronal sections (40 μm) were taken on a freezing microtome. Every third section was mounted on glass slides, dehydrated in ascending concentrations of alcohol, defatted in xylene, and stained with thionin.
Lesions were drawn on atlas sections (Paxinos & Watson, 1998) at four rostral-caudal levels, in Photoshop. At each level, the pixel area of each lesion within the borders of the basolateral, lateral, basomedial, and central nuclei of the amygdala was obtained. The total lesion area for each nucleus over all four sections was divided by the total pixel areas of that nucleus over those sections, to obtain the percentage damage to each nucleus. Percentage damages were determined for each hemisphere, and the values for the two hemispheres averaged for final damage scores for each rat.
The data of four lesioned rats were discarded from the study; 3 brains showed only minor damage, and 1 showed substantial, irregular damage throughout the amygdala and surrounding regions in one hemisphere. The accepted lesions showed (mean ± s.e.m.) damage of 87.9 ± 2.2% to the basolateral nucleus, 76.8 ± 3.0% to the lateral nucleus, 56.3 ± 6.1% to the basomedial nucleus, and only 5.9 ± 2.0% to the central nucleus. One sham-lesioned rat died after surgery, leaving 10 BLA-lesioned rats and 9 sham-lesioned rats. Figure 1B shows the extent of each of the 10 lesions, in a stacked fashion, each with 10% opacity.
Figure 1C shows the percentages of time rats spent in the food cup in the first-order conditioning phases. Conditioning to VPre and discrimination between the reinforced VPre and the nonreinforced VPost both proceeded rapidly, and were unaffected by the identity of those cues or future lesion state of the rats. Similarly, after surgery, responding to VPost was acquired rapidly when that cue was paired with food, and was unaffected by the BLA lesion. Pre-CS responding declined over the course of VPost training, but otherwise did not differ systematically.
A VPre identity X subsequent lesion X session ANOVA of responding in the initial conditioning sessions showed only a significant effect of session, F(7,105) = 30.12, p < .001, η2 =.667, other ps > .135, η2s < .04. Similarly, a VPre identity X contingency (reinforced VPre or nonreinforced VPost) X session ANOVA over the discrimination sessions showed only a significant effect of contingency, F(1,15) = 98.84, p < .001, η2 = .868, and a Contingency X Session interaction, F(3,45) = 5.70, p = .002 η2 = .275, other ps < .208, η2s < .095. A lesion X VPost identity X session ANOVA of post-surgical acquisition of responding to VPost showed only a main effect of session, F(3,45) = 11.72, p < .001, η2 =.439, other ps > .086, η2s < .135. Finally, comparable ANOVAs of pre-CS responding showed no significant effects prior to surgery, Fs < 1, ps > .242, η2s < .082, and only a significant effect of sessions, F(3,45) = 5.72, p = .002, η2 = .276, after surgery; other ps > .118, η2s < .121.
Figure 1D shows the primary data of this experiment, the acquisition and loss of second-order CRs. Initially, sham-lesioned rats acquired comparable levels of CRs to APre and APost, which then declined similarly with continued second-order training, in the absence of any food reinforcement of the first-order visual CSs. By contrast, BLA-lesioned rats showed little evidence of acquisition of second-order CRs to APost, which had been paired with a first-order CS (VPost) trained after lesions were made. This observation replicates Hatfield et al.’s (1996) and Setlow, Gallagher, & Holland (2002) findings that pretraining BLA lesions impaired the ability of first-order cues to acquire conditioned reinforcement power. More dramatically, BLA-lesioned rats showed substantially greater acquisition and persistence of second-order CRs to APre, which had been paired with a first-order CS (VPre) trained prior to the lesion surgery. This observation is consistent with Lindgren et al.’s (2003) observation that in BLA-lesioned rats, post-lesion extinction of first-order CRs to a first-order CS trained prior to the lesion had little effect on the ability of that CS to reinforce subsequent second-order learning. Thus, because the BLA lesions impaired extinction of VPre’s conditioned reinforcement power during the second-order conditioning sessions, VPre could support second-order conditioning to APre over many more training trials than was observed in sham-lesioned rats.
An initial ANOVA showed no significant effects or interactions for the auditory and visual CS counterbalancing variables, ps > .097, η2s < .130, so those variables were omitted from subsequent analysis. A lesion by stimulus (APre or APost) X block ANOVA over the 15 half-session blocks of second-order conditioning showed significant main effects of lesion F(1,17) = 25.55, p < .001, η2 = .600, stimulus, F(1,17) = 76.15, p < .001, η2 = .817, and block, F(14,238) = 54.14, p < .001, η2 = .761, as well as interactions of Stimulus X Block, F(14,238) = 13.21, p < .001, η2 = .437, Lesion X Stimulus, F(1,17) = 51.09, p < .001, η2 = .752, Lesion X Block, F(14,238) = 22.70, p < .001, η2 = .572, and Lesion X Stimulus X Block, F(15,255) = 18.59, p < .001, η2 = .522. A post-hoc HSD test showed that responding to APre in the BLA-lesioned rats was significantly greater than responding to APost in those rats and responding to either cue in sham-lesioned rats, ps < .001. There were no other significant differences among these means, ps > .382.
A second ANOVA including only the second, third and fourth blocks of trials, reflecting early acquisition of second-order conditioning, showed significant main effects of stimulus F(1,17) = 29.37, p < .001, η2 = .633, and block, F(2,34) = 32.51, p < .001, η2 = .656, and a significant Lesion X Stimulus interaction, F(1,17) = 12.85, p = .002, η2 = .431. A post-hoc HSD test showed that over these early acquisition sessions, responding to APost in the BLA-lesioned rats was significantly lower than responding to APre in those rats and lower than responding to either cue in sham-lesioned rats, ps < .027. There were no other differences among these means, ps > .132.
Pre-CS responding declined over the course of the second-order conditioning phase, F(14,238) = 2.11, p = .012, η2 = .111, other ps > .260, η2s < .074.
These differences in responding to the second-order CSs were not reflected in overt responding to the first-order CSs. Figure 1E shows extinction of food-cup responding to VPre and VPost over the course of second-order training. Initial losses in responding were rapid for both VPre and VPost, although responding to VPre was somewhat more resistant to extinction. Importantly, the course of extinction of these CRs was unaffected by BLA lesions, consistent with previous results (Lindgren et al., 2003; Setlow, Gallagher, & Holland, 2002). Pre-CS responding also declined over the course of extinction.
An initial ANOVA showed no significant effects or interactions involving the visual and auditory cue counterbalancing variables, ps < .213, η2s < .138, so those variables were dropped. A lesion X stimulus (VPre or Post) X block ANOVA showed significant effects of stimulus, F(1,17) = 21.16, p < .001, η2 = .555, and block, F(14,238) = 39.53, p < .001, η2 = .699, and a Stimulus X Block interaction, F(14,238) = 3.73, p < .001, η2 = .180, other ps > .181, η2s < .103. A lesion X block ANOVA of pre-CS responding showed only a significant effect of block, F(14,238) = 2.23, p = .007, η2 = .116, other ps > .364, η2s < .049.
These results confirmed roles for BLA plasticity in both the acquisition and extinction of a food-paired CS’s conditioned reinforcement powers. As in many conditioning preparations (Burns, Roberts, & Everitt, 1991; White & McDonald, 1993), a CS paired with a reinforcer after BLA damage failed to acquire incentive properties, including conditioned reinforcement power, whereas extinction of that power already established to a CS was impaired by subsequent BLA damage (e.g., Fuchs et al., 2002; Lindgren, et al., 2003). The novel consequence of that impairment of extinction in the present experiment was the enablement of second-order conditioning of great magnitude and persistence. Nevertheless, it is important to acknowledge that after extended second-order conditioning pairings, second-order CRs to APre eventually declined. Our study does not permit determining whether that decline reflected the eventual loss of VPre’s conditioned reinforcing power, or some other unspecified process. It would have been of interest to determine if VPre retained its ability to establish second-order CRs to a new CS after it no longer supported responding to APre.
The observation of enhanced second-order conditioning to APre in BLA-lesioned rats has important implications for understanding neuropsychiatric disorders in which pathological behavior is maintained by conditioned reinforcement, because it shows that under some circumstances loss of function in circuit elements critical for conditioned reinforcement may result in enhanced roles for that process in establishing and maintaining those behaviors. Furthermore, extended exposure to some drugs (e.g., Kalivas & O’Brien, 2008; Uys & Reissner, 2011; Okvist et al., 2011) is known to produce impairments in neuroplasticity in brain regions involved in conditioned reinforcement. Thus, it is possible that drug-induced loss of function in these regions may paradoxically increase the importance of conditioned reinforcement in the control of pathological drug-seeking and drug-taking behavior. Not only may the link between drugs and cues previously associated with drug use be difficult to extinguish therapeutically, but also those cues may continue to serve as conditioned reinforcers for the establishment of new pathological behaviors long after the original cue-drug link is degraded.
This work was supported in part by NIH grant MH-53667.