BASOLATERAL AMYGDALA LESIONS AND SENSITIVITY TO REINFORCER MAGNITUDE IN CONCURRENT CHAINS SCHEDULES

doi:10.1016/j.bbr.2008.03.028

Behav Brain Res. Author manuscript; available in PMC 2009 August 22.

Published in final edited form as:

Behav Brain Res. 2008 August 22; 191(2): 210–218.

Published online 2008 March 27. doi: 10.1016/j.bbr.2008.03.028

PMCID: PMC2475334

NIHMSID: NIHMS54527

BASOLATERAL AMYGDALA LESIONS AND SENSITIVITY TO REINFORCER MAGNITUDE IN CONCURRENT CHAINS SCHEDULES

Christa M. Helms¹ and Suzanne H. Mitchell^2,³

¹Oregon National Primate Research Center - Division of Neuroscience, Oregon Health & Science University

²Department of Behavioral Neuroscience, Oregon Health & Science University

³Department of Psychiatry, Oregon Health & Science University

Correspondence to: Christa M. Helms, Ph.D., Oregon Health & Science University, Oregon National Primate Research Center, L-584, 505 N.W. 185^th Avenue, Beaverton, OR 97006, Tel 503-614-3747, Fax 503-690-5384, Email helmsc/at/ohsu.edu

The publisher's final edited version of this article is available at Behav Brain Res

See other articles in PMC that cite the published article.

Publisher's Disclaimer

Abstract

Previous studies show that the basolateral amygdala (BLA) is required for behavior to adjust when the value of a reinforcer decreases after satiation or pairing with gastric distress. This study evaluated the effect of pre- or post-training excitotoxic lesions of the BLA on changes in preference with another type of contingency change, reinforcer magnitude reversal. Rats were trained to press left and right levers during a variable-interval choice phase for 50 µl or 150 µl sucrose delivered to consistent locations after a 16-s delay. Tones were presented during the first and last 2 s of the delay to reinforcement. The tone frequency predicted the magnitude of sucrose reinforcement in baseline conditions. All groups acquired stable preference for the lever on the large (150-µl) reinforcer side. However, nose poking during the delay to large reinforcement was highly accurate (i.e., to the reinforced side) for all groups except the rats with BLA lesions induced before training, suggesting impaired control of behavior by the tone. After the acquisition of stable preference, the locations of the reinforcer magnitudes were unpredictably reversed for a single session. Pre-training lesions blunted changes in preference when the reinforcer magnitudes were reversed. Lesions induced after stable preference was acquired, but prior to reversal, did not disrupt changes in preference. The data suggest that the BLA contributes to the adaptation of choice behavior following changes in reinforcer magnitude. Impaired learning about the tone-reinforcer magnitude relationships may have disrupted discrimination of the reinforcer magnitude reversal.

Keywords: concurrent chains, basolateral amygdala, choice, choice in transition, lever-press, rats

Introduction

The basolateral amygdala (BLA) has been repeatedly implicated in the control of behavior by Pavlovian stimulus-reinforcer relationships (for a review, see [1]), and adaptation to decreases in reinforcer value or to reinforcer omission [2, 3, 4, 5, 6]. In one Pavlovian conditioning protocol, rats are trained to approach a food source in the presence of a stimulus, then the food is devalued outside of the Pavlovian or instrumental conditioning context by pairing its consumption with nausea induced by lithium chloride. After devaluation, normal rats decrease their rate of approach toward the food source in the presence of the stimulus. Behavioral adaptation to reinforcer devaluation occurs even though the animals are not re-exposed to the reinforcer. This indicates that behavior has adapted to reflect the updated value of the reinforcer as signaled by the stimulus [7]. However, rats with pre- [2] but not post- [5] training BLA lesions continue to approach to the food source, suggesting impaired re-valuation. Analogous results have been obtained using instrumental conditioning protocols for which food is devalued by satiation [6, 8, 9]. The types of reinforcer variables manipulated in previous studies, however, are limited and it is not known whether the BLA supports adjustment of choice behavior following changes in reinforcer properties such as magnitude.

Animals’ choices are sensitive to a variety of reinforcer properties. For example, when presented with two sources of reinforcement, animals allocate their behavior to match the ratio of reinforcement schedules [10, 11], reinforcer amounts [12, 13], reinforcer immediacies [14, 15], and the relative rate of conditioned reinforcers [16]. Although the factors determining steady-state preference are well described [17], the factors controlling how preference adapts when reinforcer properties change are poorly understood.

Research shows, however, that both current and past reinforcement contingencies influence preference [18, 19]. For example, Davison and Baum [20, 21] presented pigeons with concurrent reinforcement schedules differing in magnitude, and then switched the magnitudes for each alternative after 10 reinforcers, 7 times per session. Pigeons’ ratio of responding between the two alternatives reflected the current ratio of reinforcer magnitudes, and to a lesser extent, past reinforcer ratios. Similar results are obtained when reinforcer delays shift between two choice alternatives [22, 23].

The adaptation of choice behavior to changes in reinforcement contingencies has most often been studied using concurrent schedules of reinforcement, as described above. In contrast, in concurrent chains schedules [24], subjects choose between two concurrent schedules (choice phase, or initial link) leading to a stimulus (e.g., tone) and mutually exclusive schedules of primary reinforcement (reinforcer phase, or terminal link). Concurrent chains schedules were developed to study the control of behavior by conditioned reinforcers. The stimuli signaling terminal link onset are considered conditioned reinforcers because they are associated with reinforcer delivery.

In concurrent chains schedules, choice behavior in the initial links is influenced by the stimuli presented during the terminal links [25]. These stimuli affect choice as conditioned reinforcers because they predict reinforcer delivery and can acquire conditioned value [26]. For example, in one study [27], pigeons chose between concurrent variable-interval (VI) 120-s initial links leading to illumination of a center key light (terminal link stimulus) that co-occurred with a fixed-interval (FI) 20-s terminal link ending in food delivery. To assess the conditioned reinforcing value of the terminal link stimulus, 20-s key light presentations were also contingent on the pigeons’ pecking during the initial links according to a VI 30-s schedule. In conditions lasting 18–34 sessions each, the percentage of key light presentations were differentially allocated between the left and right initial links, e.g., 50-50, 80-20, 20-80, 80-20. The pigeons allocated their initial link responses in proportion to the relative frequency of additional terminal link stimulus presentations, indicating that these stimuli had conditioned reinforcing value.

The BLA has a well-known role in conditioned reinforcement. Disruption of BLA activity decreases the ability of Pavlovian conditioned stimuli to reinforce new behavior [28, 29] and to sustain responding in second-order schedules [3, 30]. Concurrent chains schedules may be useful for studying the role of the BLA in sensitivity to changes in reinforcer variables because the contribution of primary and conditioned reinforcers can be dissociated.

The current study assessed the role of the BLA in the adaptation of preference to changes in reinforcer magnitude using concurrent chains schedules. We expected normal acquisition of preference, since previous studies have shown that BLA lesions do not affect the acquisition of operant behavior [6]. Pre- [2, 6, 8, 9], but not post-training [5] BLA lesions decrease behavioral changes after reinforcer devaluation. We therefore hypothesized that pre-training but not post-training BLA lesions would disrupt changes in choice behavior during another type of contingency change, reinforcer magnitude reversal. The current protocol differs from typical reinforcer devaluation protocols, i.e., the rats in the current study were exposed to the altered reinforcer value in the training context. Thus, sensitivity to changes in reinforcer value, in addition to changes in reinforcer value signaled by a stimulus, could be important processes contributing to behavioral adaptation in the current study.

Materials and Methods

Subjects and apparatus

Male Long Evans rats (N = 43; Charles River, Raleigh, NC) obtained at 50–55 days of age (225–250 g) served as subjects. Rats in Experiment 1 underwent surgery before behavioral training (pre-training surgery; PRE, N = 23) and Experiment 2 rats underwent surgery after baseline behavioral training (post-training surgery; POST, N = 20), followed by reacquisition of stable preference. Table 1 outlines the protocol for both groups. PRE required 54–77 days to complete the protocol, and POST required 65–102 days.

Table 1

Experimental protocol for rats receiving excitotoxic lesions of the basolateral amygdala before (PRE) or after (POST) acquiring stable preference in a concurrent chains schedule. Stable preference was reacquired after surgery for POST rats. Also shown (more ...)

On arrival, the animals were habituated to a temperature (21 ± 1°C) and light-controlled vivarium (12:12-hr light-dark cycle with lights on at 6:00 am), then weighed and handled daily. During recovery from surgery, the rats were free-fed, then food restricted to 90% of their free-feeding body weights prior to training and maintained at 90% of free-feeding weight with supplemental lab chow following experimental sessions. Sessions occurred during the light cycle, 5–7 days per week. Water was freely available except during experimental sessions. Care followed the guidelines provided by the Oregon Health & Science University Department of Comparative Medicine. The Institutional Animal Care and Use Committee approved all procedures.

Behavior was measured in 4 identical Med Associates (St. Albans, VT) operant chambers (ENV-008) in sound-attenuating ventilated boxes (60 dB). Behavioral data were recorded using an IBM-compatible computer running Med-PC software. The panel to the left of the door contained 3 equally spaced nose poke response units (ENV-254): the left and right, but not the center (ENV-114BM), units contained a reinforcer cup (ENV-200R3BM). Horizontal infrared beams were broken when the animal entered 0.64 cm into a response unit. Plastic tubing (PHM-122) was attached to a stainless steel pipe protruding from the rear of each reinforcer cup and connected to 60-ml syringes filled with 25% (w/v) sucrose in deionized water delivered via Med-Associates pumps (PHM-100; 3.33 RPM, 60.2 µl/s). A response lever (ENV-110M) was set above each head-entry device. A stimulus light (ENV-221M) was set 5.7 cm above each lever, and a house light (ENV-215M) was set 14 cm above the center head-entry device. The panel to the left of the door contained a speaker (ENV-224BM) connected to a multiple tone generator (ENV-223).

Surgical procedures

At the time of surgery, PRE (lesion, N = 12; sham, N = 11) and POST (lesion, N = 11; sham, N = 9) rats weighed (mean ± SEM) 347.10 ± 3.30 g and 318.75 ± 2.75 g, respectively. POST rats completed baseline behavioral training immediately before surgery, so they were food-deprived at the time of surgery. Food was freely available during recovery from surgery. PRE rats were anesthetized with ketamine hydrochloride (130.21 ± 8.89 mg/kg, i.p.) and xylazine (8 mg/kg, intraperitoneally, i.p.). POST rats were anesthetized with isoflurane gas because the technology became available to us and because the rats exhibited variable sensitivities to ketamine and recover more quickly from gas anesthesia.

All rats were secured in a stereotax (Cartesian) with incisor and ear bars then the scalp was cut to expose the skull. Holes were drilled above the BLA [basal, lateral, accessory basal nuclei: coordinates relative to bregma, mm; antero-posterior (AP), −2.9; mediolateral (ML), ± 5.0; dorsoventral (DV), −8.1]. Rats received bilateral infusions of 0.50 µl (0.05 µl/min + 4 min diffusion) of 0.09 M quinolinic acid (Sigma) dissolved in phosphate-buffered saline (PBS, pH 7–7.4) using a 1 µl Hamilton syringe (Hamilton, Reno, NV). Sham-surgery rats underwent identical treatment except that PBS was infused instead of quinolinic acid. Buprenorphine, ketoprofen, and warm sterile saline were administered after surgery to prevent pain, inflammation, and dehydration, respectively.

Histological procedures

The rats were injected with an overdose of sodium pentobarbital or ketamine HCl and perfused transcardially with 0.9% saline followed by 10% formaldehyde solution in PBS (formalin). The brains were stored for 24 h each in solutions of 10%, 20%, and 30% sucrose in PBS. Frozen coronal sections (40 µm) were cut through the BLA using a cryostat. Every fourth section was mounted on a glass slide, stained with thionin, and inspected under a light microscope. The brains of animals for which sham and BLA lesions were intended were compared by an observer blind to the behavioral data at −1.88, − 2.30, −2.80, −3.30, and −3.80 mm posterior from bregma. A 2 × 2 mm grid was placed over drawings of coronal sections [31] at each coordinate. The squares containing damage as shown in the photomicrograph were marked and the percent in a region that appeared damaged relative to sham sections was calculated. Rats for which the average of the left and right damage was >30% were included in the behavior analyses based on the magnitude of the lesions. This criterion ensured a sufficient number of rats in the lesion groups.

Behavioral procedures

Each rat was assigned to a single experimental chamber for the duration of the experiment, with assignments balanced among the groups.

Instrumental shaping

The rats were trained in 6 phases (Table 2) to respond in the concurrent chains schedule. In Phases 1–4, rats advanced to the next phase by obtaining 64 reinforcers in 60 minutes or less for 2 consecutive sessions. In Phase 1, 150 µl 25% sucrose was delivered immediately after a left or right lever press but also according to a variable-time (VT) 120-s schedule. When sucrose was delivered, the house light and the stimulus light above the reinforced cup shut off for 2 s. In Phase 2, concurrent chains were introduced. The initial link variable-interval (VI) 8-s schedules were randomly selected from 12 intervals generated from an exponential progression [32]. After the VI elapsed, if the rat pressed the lever assigned for reinforcement, sucrose was delivered after a fixed-time (FT) 2-s terminal link. Unlike Phase 1, reinforcer delivery co-occurred with both stimulus lights shutting off, but the house light remained illuminated. After the reinforcer was delivered, the lights turned on and the initial link VI schedule restarted. The initial links were interdependently scheduled [33] so the rats experienced an equal number of reinforcers from each side per session. Every 8 reinforcers, 4 were delivered to the left, and 4 to the right, in a random order. In Phase 3, a change-over delay (COD) was introduced. Once the initial link VI interval elapsed, the next press to the lever assigned for reinforcement resulted in terminal link entry if a) it was the first press to that lever during the trial, or b) 2.5 s had elapsed after the rat switched from the other lever. This COD was used in preliminary experiments with this protocol, and is similar to those used for pigeons [34]. Greater switching is associated with lower preference; the COD was used to decrease switching and maximize preference [35], increasing our ability to detect changes in preference. In Phase 4, a 10-s time-out occurred after reinforcer delivery during which all the lights shut off; responses were recorded but inconsequential. The lights came on after the time-out to signal initial link onset. Phase 5 was a single session during which terminal links lengthened by 2 s every 8 trials from fixed-time (FT) 2 to 16 s. One of the reinforcer magnitudes was reduced to 50 µl, while the other remained at 150 µl. Both the left and right stimulus lights shut off at the onset of the terminal link, which was segmented by tones in the first and last 2 s of terminal links greater than FT 6 s; a tone occurred in the first 2 s of FT 2- and 4-s terminal links. The tone segmented the delay because stimuli of longer duration may be less efficacious conditioned reinforcers [36]. The frequency of the tone (1 or 15 kHz, 75 dB) signaled the reinforcer magnitude. The magnitude, location, and tone frequency assignments were counterbalanced across sham and lesion groups. Reinforcement was delayed in the terminal links to allow assessment of stimulus control of behavior before reinforcer delivery.

Table 2

Reinforcement contingencies during the six phases of training.

Concurrent chains training

The final concurrent chains schedule was concurrent VI 16-s initial links leading to mutually exclusive FT 16-s terminal links (Figure 1). At the end of one terminal link, 50 µl 25% sucrose was delivered. The other terminal link ended with the delivery of 150 µl 25% sucrose. A 10-s time-out preceded the next initial link after reinforcer delivery. To ensure that the rats received a similar number of large and small reinforcers every session, every 8 trials, the left and right initial links were scheduled for reinforcement 4 times each in a random order. Thus, every 8 trials, 4 terminal links ended in 150 µl sucrose and 4 ended in 50 µl sucrose. Sessions ended after 64 reinforcers or 90 min, whichever occurred first. A 2-s tone was presented in the first and last 2 s of the terminal link, with the frequency of the tone consistently preceding one of the reinforcer magnitudes, as described above. Phase 6 training continued until preference was stable, but for at least 14 sessions. Preference was measured as the total number of initial link presses to the large reinforcer lever divided by the total number of initial link presses per session. Stability was attained when preference in the most recent session deviated by <10% from the previous 4 sessions and the first and last values in the series differed from the next smallest or largest value by ≥ 1%, i.e., no increasing or decreasing trends in preference. For example, the sequence of percentages 70, 73, 71, 69, 73 is stable because the difference between 73 and the preceding 4 values is < 7.3. Stability was re-acquired after recovery from surgery in POST-lesioned rats, so these rats experienced approximately 9 more training sessions than PRE rats (Table 1).

Figure 1

Schematic illustration of the concurrent chains schedule. Rats responded on left and right levers to access mutually exclusive terminal links according to variable-interval (VI) 16-s schedules. The fixed-time (FT) 16-s terminal link ended with the delivery (more ...)

Reinforcer magnitude reversals

After preference was stable, the locations to which the large and small reinforcers were delivered were reversed at the start of the next session, and returned to baseline at the start of the following session. Reversals occurred unpredictably 3 times per rat, with a series of 3 or 4 baseline sessions following each reversal session; the order of the two series was randomly determined. Three baseline sessions followed the third reversal. We expected preference to approximate baseline values within the 3 or 4 sessions following a reversal because Davison & McCarthy [17] estimated that reinforcer ratios more than 3 sessions in the past exert no appreciable influence on choice. Caution should be exercised, however, because their estimate was based on pigeon data from concurrent VI schedules so this may not generalize to the current protocol.

Due to scheduling errors for PRE rats only, 1 lesioned rat experienced 6 instead of 5 sessions during the first series of baseline sessions, and 1 sham rat experienced 5 instead of 4 baseline sessions during the second series. For 1 sham rat, 4 baseline sessions intervened between each reversal session, and on one occasion, 2 rats were exposed to the reversed reinforcer magnitudes for 2 consecutive sessions. Omitting these subjects did not alter the outcomes of statistical analyses of the primary dependent measure (i.e., preference) and so they were included in the analyses after omitting the extraneous sessions.

Statistical Analysis

Most dependent measures were evaluated with 2 (surgery: PRE, POST) × 3 (lesion: sham, partial lesion, lesion) mixed factor ANOVAs. Huynh-Feldt corrections were used when repeated factors were included; adjusted degrees of freedom are cited throughout the manuscript. Main effects and interactions were evaluated with Bonferroni-corrected pairwise comparisons. For all analyses, α was .05.

Results

Lesions

Figures 2A–2D are representative photomicrographs of sham and BLA lesions showing the least and greatest extents of lesion throughout the rostrocaudal extent of the BLA for PRE and POST rats. The lesions matched our criterion for 7/12 PRE and 6/11 POST rats. However, one POST rat had 50% damage to the left central nucleus and a complete lesion of the right central nucleus and was omitted from the analysis. Thus, 7 lesioned PRE rats and 5 lesioned POST rats were classified as having complete lesions and were available for analysis. On average (± SEM), 50.0 ± 2.0% and 53.6 ± 6.9% of the basal and lateral amygdala was damaged bilaterally in PRE and POST rats, respectively. The lesions in POST rats were slightly more extensive than the lesions in PRE rats, possibly due to the neuroprotective effects of ketamine [37]. POST rats had minor unilateral central nucleus damage; rats with unilateral central nucleus damage are typically included in behavior analyses [e.g., 29]. PRE (N = 5) and POST (N = 5) rats with partial BLA lesions (11.14 ± 4.70 and 17.23 ± 1.63%, respectively) were analyzed as separate groups because they provide information about the quantity of neuronal damage necessary to induce behavioral deficits. One POST sham rat had >10% bilateral damage to the BLA and was excluded from the analyses because the quality of damage may have differed from rats with partial lesions induced by quinolinic acid. Otherwise, none of the PRE (N = 11) or POST (N = 8) sham rats had damage to any structure and all were included in the analyses.

Figure 2

Representative photomicrographs from sections −2.8 mm posterior from bregma (top) and diagrams of coronal sections (bottom) showing the least (lighter) and greatest (darker) extent of lesion in rats with bilateral quinolinic acid or PBS infusions (more ...)

Instrumental acquisition

BLA lesions did not affect the mean number of sessions to criterion performance during the 6 phases of concurrent chains training (Figure 3). Analyses of the sessions to criterion in each phase using 2 (surgery: PRE, POST) × 3 (lesion: sham, partial lesion, complete lesion) between-subjects ANOVAs revealed no main effects or interactions. Baseline preference was acquired in approximately 20 sessions, consistent with previous studies using concurrent chains schedules [38]. As expected, the percent of initial link lever presses to the large reinforcer side increased across Phase 6 training sessions before stabilizing (Figure 4) as indicated by a main effect of session from a 2 (surgery) × 3 (lesion) × 14 (session) mixed factor ANOVA, F(12.44, 448.00) = 2.88, p = 0.01).

Figure 3

Mean (± SEM) number of sessions to criterion across the 6 phases of concurrent chains training described in Table 2. Pre-training surgery (PRE) occurred before Phase 1 and post-training surgery (POST) occurred after Phase 6. The rats received (more ...)

Figure 4

Mean (± SEM) percent choice of the large reinforcer side during initial links across the first 14 sessions of concurrent chains training (Phase 6), the minimum number of sessions over which stability could have been attained. After a 16-s delay, (more ...)

Preference did not differ between groups across the first 5 stable sessions according to a 2 (surgery) × 3 (lesion) ANOVA; there were no main effects or interactions. For the POST groups, the mean number of sessions to attain stable preference after surgery, including the 5 stable baseline sessions, was 7.37 ± 0.63, 10.60 ± 2.84, and 8.20 ± 1.60 for the sham, partially lesioned, and lesioned rats, respectively. Stable preference after surgery did not differ from stable preference before surgery according to a 3 (lesion) × 2 (time: before surgery, after surgery) mixed factor ANOVA. This analysis revealed an interaction between the factors, F(2.00, 15.00) = 4.51, p = 0.03, but post hoc tests did not indicate differences between pre- and post-surgery baseline preference for any POST group.

Terminal link response accuracy

Nose poking was assessed during the terminal link when tones predicting reinforcer magnitude were presented. Terminal link nose pokes occurred almost exclusively in the first 4 s and final 2 s of the 16-s terminal link period, except for 3 lesioned and 4 sham PRE rats that did not nose poke during the terminal link. These non-responders were excluded so that their data would not bias the analysis. Responding in the final 2-s of the interval should reflect anticipatory responding prior to reinforcer delivery. Responding at the beginning of the terminal link is more difficult to interpret but may reflect carryover from the initial link, rats’ ability to discriminate onset of the terminal link, and the ability of terminal link stimuli to control behavior. Each session, some terminal links ended with delivery of a small reinforcer, and some ended with delivery of a large reinforcer. To compare the accuracy of nose poking to the reinforced side depending on whether the terminal link stimuli signaled a large or small reinforcer, the number of nose pokes to the reinforced side was divided by the mean total nose pokes for the 5 stable baseline sessions prior to reversal. As shown in Figure 5, accuracy decreased over time during large, F(2, 58) = 20.37, p < 0.001, and small, F(1.90, 66.35) = 61.04, p < 0.001, reinforcer terminal links according to 2 (surgery) × 3 (lesion) × 3 (time: first 2 s, second 2 s, last 2 s of the terminal link) × 2 (magnitude: small, large) mixed factor ANOVA. However, accuracy decreased more rapidly across small compared to large reinforcer terminal links (time × magnitude interaction: F(1.87, 52.39) = 6.33, p = 0.004). Further, PRE lesioned rats were significantly less accurate than POST lesioned rats, particularly during the later two terminal link intervals (surgery × lesion interaction, F(2, 29) = 6.31, p = 0.005, surgery × lesion × time interaction, F(2, 29) = 3.73, p = 0.04). In contrast, for small reinforcer terminal links, PRE rats (69.8 ± 3.1%) were more accurate than POST rats (59.8 ± 3.4%), (F(1, 35) = 4.93, p = 0.03), irrespective of lesion group.

Figure 5

Mean (± SEM) percent of nose pokes to the reinforced side during seconds 1–2, 3–4 and 15–16 of large or small reinforcer terminal links. Reinforcers were delivered at the end of each terminal link. PRE, pre-training surgery; (more ...)

Reinforcer magnitude reversals – preference change

We analyzed changes in preference during reversal sessions after an equal number of large and small reinforcers were delivered. To assess changes across time, we divided the session into quarters (16 trials each). One lesioned and 1 sham PRE rat completed fewer than 16 trials for some reversals (trials completed for reversals 1–3 for the lesioned rat and sham rat, respectively: 15, 10, 21 and 20, 13, 22) and were excluded from this analysis. The mean number of trials completed averaged over the 3 reversal sessions for the other subjects did not differ between the groups (data not shown). The data were collapsed across the three reversals because a 2 (surgery) × 3 (lesion) × 3 (repetition) × 2 (first and last 16-trial bin) mixed factor ANOVA for preference indicated no main effects or interactions involving repetition. Mean preference during each 16-trial bin was subtracted from preference during the preceding baseline session to produce Figure 6. Preference for the large reinforcer side declined when the reinforcer magnitudes were reversed. As a result, preference change is negative, with more negative values reflecting greater decreases in preference.

Figure 6

Mean (± SEM) decrease in percent of initial link lever presses to the originally large reinforcer lever after the 150- and 50-µl reinforcer magnitudes were unpredictably reversed for a single session, collapsed across the three repetitions (more ...)

The decrease in preference grew larger across the 16–trial bins of the reversal sessions (main effect of trial bin: F(2, 33) = 10.79, p = 0.002), although this progressive change in preference was not discernable in POST lesioned rats (Figure 6). A surgery × lesion interaction (F(2, 33) = 3.86, p = 0.03) and Bonferroni-corrected pairwise comparisons, indicated that for the PRE group, lesioned rats changed their preference less than partially lesioned rats, but partially lesioned and sham rats’ changes in preference did not differ. The decrease in preference was numerically greater for PRE partial lesion compared to PRE sham rats, so that the difference between PRE partial lesion and lesion groups reached significance. That the difference between PRE sham and lesioned groups did not reach significance is probably due to measurement error resulting from small sample size. In contrast, preference change did not differ between the POST groups. The data suggest that damaging the BLA before training impaired changes in preference when the reinforcer magnitudes were reversed.

Reinforcer magnitude reversals – carryover effects

Decreases in preference with successive trials were unique to reversal sessions. In the baseline sessions that immediately followed reversal sessions, when the magnitudes were returned to their baseline assignments, change in preference during the first 16 trials (−1.36 ± 1.18) was greater than the last 16 trials (0.48 ± 1.22). Compared to pre-reversal baseline preference, preference was reduced in the baseline session immediately following the reversal session for all groups (Figure 7) according to a 2 (surgery) × 3 (lesion) × 2 (session type: pre-reversal, post-reversal) × 3 (repetition) mixed factor ANOVA. Only the main effect of session type was significant, F(1.00, 34.00) = 59.04, p < 0.001. The absence of group differences indicates that neither pre- nor post-training BLA lesions affected the rate at which behavior recovered from temporary perturbation of the reinforcement contingencies.

Figure 7

Mean (± SEM) percent of initial link lever presses to the originally large reinforcer lever before and after the 150- and 50-µl reinforcer magnitudes were unpredictably reversed for a single session, collapsed across the three repetitions (more ...)

Discussion

Consistent with previous reports using simple chain or concurrent schedules of reinforcement [e.g., 6], BLA lesions did not impair learning or performance in concurrent chains schedules. The lesions in the current study did not affect rats’ sensitivity to primary reinforcement as indicated by their normal acquisition of preference (Figure 3). All groups attained stable baseline preference in the concurrent chains schedule with VI 16-s initial links and FT 16-s terminal links in approximately 20 sessions. During these sessions, the groups did not differ in their preference for the initial link leading to the large reinforcer, suggesting that the BLA lesions did not disrupt sensitivity to differential reinforcer magnitude.

In contrast, pre-training BLA lesions disrupted nose poking during the terminal links. When a large reinforcer was to be delivered, nose poking remained highly accurate throughout the terminal link in all groups except PRE lesioned rats. The nose poking of PRE lesioned rats was similar during both large and small reinforcer terminal links, with accuracy declining over seconds (Figure 5). Since tone frequency was the only stimulus distinguishing small and large reinforcer terminal links, these data suggest that pre-training BLA lesions impaired learning about the tone frequency-reinforcer magnitude relationship.

The BLA is well-known for its role in conditioned reinforcement, which relies on stimulus-reinforcer relationships [1]. Impaired learning about conditioned reinforcers may not, however, be detected in the acquisition of preference in concurrent chains schedules. For example, although conditioned reinforcers (i.e., terminal link stimuli) contribute to initial link preference [38], pigeons prefer an initial link leading to larger reinforcement even when the terminal links are not differentially signaled [39]. In the current study, the location of the lever associated with large reinforcement was consistent during baseline sessions. Location may, therefore, have been a salient stimulus directing initial link choice. This could explain why BLA lesions, which are critical for conditioned reinforcement [1], did not disrupt acquisition of preference in concurrent chains in this study.

Rats with pre-training BLA lesions were also insensitive to reversal of the reinforcer magnitudes during the first 16 trials of the session (Figure 6). One possible explanation is that pre-training BLA lesions impaired learning about tone-reinforcer relationships in PRE lesioned rats, as suggested by terminal link nose poking. Such an impairment may have prevented them from discriminating changes in the tone-reinforcer relationship that could be a basis for adaptation of initial link choices in normal rats. For example, behavioral adaptation after reinforcer omission occurs, in part, because omission violates the animal’s expectation [40]. A rat expecting, for example, to receive 150 µl sucrose after a 15 kHz tone would be more surprised to receive 50 µl sucrose after a 15 kHz tone during a reversal session compared to a rat that had not learned the tone-reinforcer relationships. With greater surprise, behavior would be expected to change more rapidly [41], a principle that also applies to instrumental conditioning [42]. The PRE lesioned rats changed their preference to some extent during the last 16 trials, indicating that they were not completely insensitive to the reinforcer magnitude reversal.

In a conditional discrimination protocol, BLA lesions disrupt rats’ reversal of responding when switching from familiar to new contingencies [43]. The current protocol differs from conditional discrimination protocols. Whereas the initial link response scheduled for reinforcement was not signaled, the terminal link stimuli signaled the magnitude and location of reinforcement, although not a response contingency. Sucrose was always delivered at the end of the terminal link, but a nose poke was necessary for consumption. Both reversal of responding in a conditional discrimination and adaptation of preference to reinforcer magnitude reversal may involve learning about stimulus-reinforcer relationships, a process supported by the BLA [1, 43]. Studies using other protocols suggest that the amygdala is important for behavioral control by differential reinforcer magnitudes, e.g., delayed responding to stimuli predicting relatively large reinforcers [44], memory for reinforcer magnitude changes [45], and for acquisition of a conditional discrimination in which the discriminative stimuli are different concentrations of sucrose solution [46]. Consistent with these results, the nose poke accuracy of PRE rats with BLA lesions decreased across both large and small reinforcer terminal links in the current study (Figure 5), whereas other groups maintained accuracy throughout the large reinforcer terminal links. PRE rats with BLA lesions were, however, sensitive to the differential reinforcer magnitudes as indicated by normal preference for the initial link associated with large, delayed reinforcement over the initial link associated with small, delayed reinforcement. Pre-training BLA lesions may have disrupted rats learning about the sensory, but not the reinforcing, properties of the differential reinforcer magnitudes.

The PRE and POST groups differed in the total number of training sessions and in the number of days between surgery and the reinforcer magnitude reversals (Table 1). POST rats experienced approximately 9 additional training sessions because they were re-trained to stable baseline performance after surgery. Overtraining can render behavior resistant to changes in reinforcement [47]. Indeed, when the reinforcer magnitudes were reversed, POST rats changed their preference less than PRE rats as the trials progressed (Figure 6), which could result from additional training. On the other hand, POST rats were slightly, but significantly, less accurate than PRE rats during small reinforcer terminal links, but one would expect greater accuracy with more training sessions. It is unlikely that 9 additional training sessions can account for the differences between the PRE and POST lesioned rats in response to the reinforcer magnitude reversal. In addition, the PRE groups experienced more days between surgery and the reinforcer magnitude reversals compared to POST groups. Typically, however, functional recovery increases with time after neural damage [e.g., 48, 49]. Despite an approximately three-fold greater lesion-to-test interval in PRE lesioned rats (~45 days) compared to POST lesioned rats (~15 days), PRE lesioned rats showed significant behavioral impairment. Furthermore, these impairments depended on the magnitude of neural damage. Partially lesioned rats performed similarly to intact rats. The data are unlikely, therefore, to be explained solely by differences in the lesion-to-test interval between PRE and POST groups.

The effect of reversing the reinforcer magnitudes on initial link preference carried over into the subsequent session when the original contingencies were reinstated. Baseline preference for the large reinforcer initial link was lower in the session after the reversal compared to the session before the reversal for all groups. This suggests that the BLA does not contribute to rats’ estimation of past reinforcement used to determine current preference.

Researchers have assumed that studies of choice in transition will provide information about both the processes supporting preference acquisition and animals’ matching of response allocation to reinforcers between concurrent schedules of reinforcement [e.g., 25]. With respect to the former, these data show that pre-training BLA lesions do not affect preference acquisition, but do disrupt rats’ immediate changes in choice behavior after reversal of the reinforcer magnitudes. The lesions did not disrupt the effect of past reinforcement contingencies on current choice. This suggests that distinct processes underlie preference acquisition, the influence of past reinforcement contingencies on current preference, and the adaptation of preference to local changes in reinforcement contingencies. BLA activity appears to support the latter, perhaps by allowing animals to learn about various attributes of reinforcers [50] that provide a basis for rapid discrimination of changes. Future studies could evaluate the role of the BLA in behavioral adaptation when other properties of the primary reinforcer or the conditioned reinforcer (terminal link stimulus) are manipulated.

Acknowledgement

The authors thank William Guethlein, Jamie Reeves, Charlie Meshul, Greg Mark, and Andrey Ryabinin for technical assistance, and Chris Cunningham, Allen Neuringer, Robert Hitzemann, and Matt Lattal for helpful comments. This research was supported by grants T32-AA07468, MH070219 (CMH), DA016727 (SHM), and an APA Dissertation Research Award (CMH).

Footnotes

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

1. Everitt BJ, Cardinal RN, Parkinson JA, Robbins TW. Impact of amygdala-dependent mechanisms of emotional learning. Ann NY Acad Sci. 2003;985:233–250. [PubMed]

2. Hatfield T, Han J-S, Conley M, Gallagher M, Holland P. Neurotoxic lesions of the basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J Neurosci. 1996;16:5256–5265. [PubMed]

3. Burns LH, Everitt BJ, Robbins TW. Effects of excitotoxic lesions of the basolateral amygdala on conditional discrimination learning with primary and conditioned reinforcement. Behav Brain Res. 1999;100:123–133. [PubMed]

4. Fuchs RA, See RE. Basolateral amygdala inactivation abolishes conditioned stimulus- and heroin-induced reinstatement of extinguished heroin-seeking behavior in rats. Psychopharmacol. 2002;160:425–433.

5. Pickens CL, Saddoris MP, Setlow B, Gallagher M, Holland PC, Schoenbaum G. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J Neurosci. 2003;23:11078–11084. [PubMed]

6. Balleine BW, Killcross AS, Dickinson A. The effect of lesions of the basolateral amygdala on instrumental conditioning. J Neurosci. 2003;23:666–675. [PubMed]

7. Colwill RM, Motzkin DK. Encoding of the unconditioned stimulus in Pavlovian conditioning. Anim Learning Behav. 1994;22:384–394.

8. Wang S-H, Ostlund SB, Nader K, Balleine BW. Consolidation and reconsolidation of incentive learning in the amygdala. J Neurosci. 2005;25:830–835. [PubMed]

9. Málková L, Gaffan D, Murray EA. Excitotoxic lesions of the amygdala fail to produce impairment in visual learning for auditory secondary reinforcement but interfere with reinforcer devaluation effects of rhesus monkeys. J Neurosci. 1997;17:6011–6020. [PubMed]

10. Herrnstein RJ. Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav. 1961;4:267–272. [PMC free article] [PubMed]

11. Baum WM. Chained concurrent schedules: reinforcement as situation transition. J Exp Anal Behav. 1974;22:91–101. [PMC free article] [PubMed]

12. Catania AC. Concurrent performances: a baseline for the study of reinforcement magnitude. J Exp Anal Behav. 1963;6:299–300. [PMC free article] [PubMed]

13. Ito M, Takatsuru S, Saeki D. Choice between constant and variable alternatives by rats: effects of different reinforcer amounts and energy budgets. J Exp Anal Behav. 2000;7:79–92. [PMC free article] [PubMed]

14. Chung S-H, Herrnstein RJ. Choice and delay of reinforcement. J Exp Anal Behav. 1967;10:67–74. [PMC free article] [PubMed]

15. Williams BA, Fantino E. Effects on choice of reinforcement delay and conditioned reinforcement. J Exp Anal Behav. 1978;29:77–86. [PMC free article] [PubMed]

16. Shahan TA, Podlesnik CA, Jimenez-Gomez C. Matching and conditioned reinforcement rate. J Exp Anal Behav. 2006;85:167–180. [PMC free article] [PubMed]

17. Davison M, McCarthy D. The Matching Law: A Research Review. Hillsdale, NJ: Erlbaum; 1988. p. 273.

18. Killeen PR. Averaging theory. In: Bradshaw CM, Szabadi E, Lowe CF, editors. Quantification of steady-state operant behavior. Amsterdam: Elsevier; 1981. pp. 21–34.

19. Shettleworth SJ, Plowright CMS. How pigeons estimate rates of prey encounter. J Exp Psychol: Anim Behav Process. 1992;18:219–235. [PubMed]

20. Davison MC, Baum WM. Choice in a variable environment: every reinforcer counts. J Exp Anal Behav. 2000;74:1–24. [PMC free article] [PubMed]

21. Davison M, Baum WM. Every reinforcer counts: reinforcer magnitude and local preference. J Exp Anal Behav. 2003;80:95–129. [PMC free article] [PubMed]

22. Schofield G, Davison M. Nonstable concurrent choice in pigeons. J Exp Anal Behav. 1997;68:219–232. [PMC free article] [PubMed]

23. Grace RC, Bragason O, McLean AP. Rapid acquisition of preference in concurrent chains. J Exp Anal Behav. 2003;80:235–252. [PMC free article] [PubMed]

24. Autor SM. The strength of conditioned reinforcers as a function of frequency and probability of reinforcement. In: Hendry DP, editor. Conditioned reinforcement. Homewood, Ill: Dorsey Press; 1969. pp. 127–162.

25. Grace RC. Acquisition of preference in concurrent chains: comparing linear-operator and memory-representational models. J Exp Psychol: Anim Behav Process. 2002b;28:257–276. [PubMed]

26. Dunn R, Williams BA, Royalty P. Devaluation of stimuli contingent on choice: evidence for conditioned reinforcement. J Exp Anal Behav. 1987;48:117–131. [PMC free article] [PubMed]

27. Williams BA, Dunn R. Preference for conditioned reinforcement. J Exp Anal Behav. 1991;55:37–46. [PMC free article] [PubMed]

28. Cador M, Robbins TW, Everitt BJ. Involvement of the amygdala in stimulus-reward associations: interaction with the ventral striatum. Neurosci. 1989;30:77–86.

29. Burns LH, Robbins TW, Everitt BJ. Differential effects of excitotoxic lesions of the basolateral amygdala, ventral subiculum and medial prefrontal cortex on responding with conditioned reinforcement and locomotor activity potentiated by intra-accumbens infusions of D-amphetamine. Behav Brain Res. 1993;55:167–183. [PubMed]

30. Di Ciano P, Everitt BJ. Direct interaction between the basolateral amygdala and nucleus accumbens core underlie cocaine-seeking behavior by rats. J Neurosci. 2004;24:7167–7173. [PubMed]

31. Paxinos G, Watson C. The rat brain in stereotaxic coordinates. San Diego: Academic; 1998.

32. Fleshler M, Hoffman HS. A progression for generating variable-interval schedules. J Exp Anal Behav. 1962;5:529–530. [PMC free article] [PubMed]

33. Stubbs DA, Pliskoff SS. Concurrent responding with fixed relative rate of reinforcement. J Exp Anal Behav. 1969;12:887–895. [PMC free article] [PubMed]

34. Stubbs DA, Pliskoff SS, Reid HM. Concurrent schedules: a quantitative relation between changeover behavior and its consequences. J Exp Anal Behav. 1977;25:85–96. [PMC free article] [PubMed]

35. Leung J-P, Winton ASW. Preference for unsegmented interreinforcement intervals in concurrent chains. J Exp Anal Behav. 1985;44:89–101. [PMC free article] [PubMed]

36. Mazur JE. Choice, delay, probability, and conditioned reinforcement. Anim Learning Behav. 1997a;25:131–147.

37. Lees GJ. Influence of ketamine on the neuronal death caused by NMDA in the rat hippocampus. Neuropharmacol. 1995;34:411–417.

38. Grace RC. The value hypothesis and acquisition of preference in concurrent chains. Anim Learning Behav. 2002a;30:21–33.

39. Ploog BO. Effects of primary reinforcement on pigeons’ initial-link responding under a concurrent-chains schedule with nondifferential terminal links. J Exp Anal Behav. 2001;76:75–94. [PMC free article] [PubMed]

40. Bouton ME. Context and behavioral processes in extinction. Learn Mem. 2004;11:485–494. [PubMed]

41. Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical conditioning II. New York: Appleton-Century-Crofts; 1972. pp. 64–99.

42. Williams BA. The blocking of reinforcement control. J Exp Anal Behav. 1975:215–225. [PMC free article] [PubMed]

43. Schoenbaum G, Setlow B, Nugent SL, Saddoris MP, Gallagher M. Lesions of the orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn Mem. 2003;10:129–140. [PubMed]

44. Kesner RP, Andrus RG. Amygdala stimulation disrupts the magnitude of reinforcement contribution to long-term memory. Physiol Psych. 1982;10:55–59.

45. Salinas JA, McGaugh JL. The amygdala modulates memory for changes in reward magnitude: involvement of the amygdaloid GABAergic system. Behav Brain Res. 1996;80:87–98. [PubMed]

46. Kesner RP, Williams JM. Memory for magnitude of reinforcement: dissociation between the amygdala and hippocampus. Neurobiol Learn Mem. 1995;64:237–244. [PubMed]

47. Adams CD. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q J Exp Psychol. 1982;34:77–98.

48. Brown CE, Boyd JD, Delaney KR, Murphy TH. Extensive turnover of dendritic spines and vascular remodeling in cortical tissues recovering from stroke. J Neurosci. 2007;27:4101–4109. [PubMed]

49. Jain S, Mathur R, Sharma R, Nayar U. Recovery from lesion-associated learning deficits by fetal amygdala transplants. Neural Plast. 2002;9:53–63. [PMC free article] [PubMed]

50. Balleine BW, Killcross S. Parallel incentive processing: an integrated view of amygdala function. Trends Neurosci. 2006;29:272–279. [PubMed]