Sixty-one adult male Long–Evans rats (Harlan, Indianapolis, IN, USA) were used in these experiments. The subjects were ~60 days of age at the initiation of training and ranged in weight from 300 to 400 g during the experimental period. All rats were initially food-restricted to 80–85% of their free-feeding weights and subsequently fed ~15 g rat chow per day in their home cage within 1–3 h after testing. Water was continuously available, except while in the operant testing chambers. Rats were maintained in 14/10-h light/dark schedule (lights on at 7 a.m.).
In addition, four male vervet monkeys (Chlorocebus aethiops sabaeus) from the UCLA Vervet Research Colony were included in the experiments; they were 9–11 years of age and weighed 6.0–8.0 kg at the time of testing. The monkeys were housed individually in a climate-controlled vivarium on a 12-h light/dark cycle (lights on at 6 a.m.); they had unlimited access to water and received a nutritionally balanced diet of monkey chow (Teklad, Madison WI, USA) supplemented with fresh fruit. The monkeys received their full daily allotment of food (which was not reduced in order to support behavioral performance) immediately after morning testing (one half portion) and again at 1600–1700 hours (one half portion).
The experimental protocols employed were consistent with the NIH “Guide for the Care and Use of Laboratory Animals” and were approved by the Chancellor’s Animal Research Committee at UCLA. All methods for the care and use of nonhuman primates conformed to US Department of Agriculture and Public Health Service standards.
For the rat studies, doses of atomoxetine hydrochloride (1.0 mg/kg, gift from Pfizer) and desipramine hydrochloride (5.0 mg/kg; Sigma-Aldrich; St Louis, MO, USA) were chosen on the basis of a pilot study with a two-choice variant of the task described here (Seu and Jentsch 2006
). Doses of methylphenidate hydrochloride (0.33–1.0 mg/kg; Sigma-Aldrich) and GBR-12909 dihydrochloride (2.5–5 mg/kg; Sigma-Aldrich) were chosen based upon previous studies showing relevant effects of this drugs in other behavioral procedures that measure inhibitory control in rats (Eagle et al. 2007
; van Gaalen et al. 2006a
All drugs were dissolved in sterile saline (0.9%) and were administered in a volume of 1.0 ml/kg, with the exception of GBR-12909 that was administered in a volume of 2.0 ml/kg. Methylphenidate, GBR-12909, and desipramine were injected 30 min before testing, while atomoxetine was administered 45 min prior to testing.
For the monkey studies, doses of atomoxetine (1 mg/kg; Tocris-Cookson; Ellisville, MO, USA) and methylphenidate (0.33 mg/kg; Sigma-Aldrich) were selected on the basis of pilot studies in which two different doses were tested on the same subjects but using a slightly different reversal learning task. We chose to administer all drugs by oral delivery rather than via intramuscular injections for the reason that we expected such route of administration to be better comparable with clinical studies. Drugs were mixed with fruit jam or peanut butter and put in a small cookie. Before every acquisition session, a cookie with jam or peanut butter (but without drug) was given to the subjects to avoid the fact that the cookie might be recognized as cue for the subsequent retention or reversal session. Drugs were administered 1 h before the retention or reversal sessions, and the experimenter assured whether the cookie and jam were eaten or not.
Rat behavioral testing apparatus
Standard extra-tall aluminum and Plexiglas operant conditioning chambers with a photocell-equipped pellet delivery magazine on one side and a curved panel with five photocell-equipped apertures on the opposite side (Med Associates, Mount Vernon, VT, USA) were used. The boxes were housed inside a sound-attenuating cubicle, background white noise was broadcasted, and the environment was illuminated with a houselight (a light diffuser that was located outside of the operant chamber but within the cubicle).
Most of the prior reversal learning studies described in rodents and monkeys have been conducted using two-choice discrimination tasks (Boulougouris et al. 2007
; Dias et al. 1996
; Iversen and Mishkin 1970
; McAlonan and Brown 2003
; Schoenbaum et al. 2002
). We chose to adopt a novel four-choice task in order to increase general task difficulty and to permit us to dissect perseverative from neutral errors.
The procedure for the initial training was similar to that used for a lateralized reaction time task (Jentsch 2003
). Rats were first trained in a single session in which the houselight was continuously illuminated and single pellets (45 mg Dustless Precision Pellets; Bio-serv, Frenchtown, NJ, USA) were delivered into an illuminated magazine on a fixed-time 20-s schedule over a 45-min period. Across three subsequent daily sessions, the rats were then trained to make a sustained, variable duration nose poke (200, 500, 700, or 1,000 ms) in an illuminated center nose poke aperture (hole 3) to receive a pellet. This response (called the observing response
) was used in the subsequent sessions to begin a new trial in order to demonstrate task engagement and to avoid random responding. All rats were trained until they earned at least 70–80 pellets in this initial shaping component.
The rats were then tested in daily discrimination sessions in which the initiation of individual trials was signaled by the illumination of the central aperture. A variable-duration observing response at that location resulted in the immediate switching off of the central light and illumination of the four remaining apertures, two to the left of the central aperture (holes 1 and 2; H1 and H2) and two to the right (holes 4 and 5; H4 and H5). On any given day, one of the holes was chosen by the experimenter to be the target hole, and rats were reinforced with a pellet for a response into that hole only (correct response). Importantly, no signal, other than reinforcement feedback, indicated which of the four holes was the target on any given day.
If the rat responded at a location that was not the established target, all lights in the box were extinguished, and the rat was given a 3-s time-out period in complete darkness (incorrect response). If no response was made within 15 s, the rat received a 3-s time-out in darkness (omission). The inter-trial interval that followed a completed trial or omission was 3 s. On occasion, rats responded into one of the lateral apertures before completing the sustained nose poke (and before the target presentation); in this case, a 3-s time-out was delivered (as above), and an anticipatory response was scored.
Sessions were terminated when rats reached criteria of 18 correct responses in 20 consecutive trials, after 1 h or when 200 trials were completed, whichever came first. If rats failed to achieve criterion performance in 1 h or 200 trials, the discrimination was repeated on subsequent days until criterion was met.
The subjects were exposed to approximately 1 month of initial training. During this time, rats were tested every 2 to 4 days for an average of 12 discrimination sessions (including one to two retention sessions), with each of the four holes assigned as target locations at least twice.
Rat testing—experimental design
After initial training, rats were tested for 2 days a week in pairs of experimental sessions. During the first session, referred to as the “new-hole session”, an aperture was selected pseudorandomly. In the second session, rats were tested for “retention” of the discrimination learned in the prior new-hole session (the reinforcement rule was kept the same) or they were subjected to a “reversal” of the discrimination learned in the prior new-hole session (a different aperture was rewarded). Pharmacological treatments were administered only during retention or reversal sessions, while the new-hole sessions were always performed drug-free.
Note that the new-hole sessions may be considered a reversal session because the rats are experiencing a change in the condition learned in the previous session. The new-hole sessions were used because it seemed to be important to always test retention or reversal of a discrimination learned in a drug-free state. This allowed us to ensure that there were no differences amongst the various drug and reversal vs. retention conditions in terms of performance in the immediately preceding session. In addition, a difference between new-hole and reversal sessions was that more restrictions were imposed in the choice of the target hole during reversal sessions in order to simplify the design and assure the all the conditions within and between drug studies were balanced. For example, only a switch from holes positioned on different sides of the central hole (H5–H1, H5–H2, H4–H2, or H4–H1 and vice versa) was allowed, while this was not a requirement for new-hole sessions.
If in a drug study a rat failed to reach performance criteria in one or more drug sessions, these sessions were repeated after completion of the formal Latin square design. However, only one to three rats in each of the drug studies ever failed to complete a session, and the frequency of failures never differed between treatment groups.
All rats required at least eight to 12 discrimination sessions (including retention, reversal, and new-hole sessions) to complete each pharmacological study; considering initial training and pharmacological testing, each subject was exposed to 20–40 sessions over the course of the procedures described in this manuscript.
A minimum of four and a maximum of 14 drug administrations (of which two to four were saline administrations) were given to each rat, with an interval between injections of at least 1 week.
The measures collected during daily sessions included total number of trials (number of trials required to reach criterion), the mean trial initiation latency (the average interval between illumination of the central aperture and the initiation of the observing response), the mean pellet retrieval time (the average interval between pellet delivery and head entry into the magazine), and the number of anticipatory responses (calculated as a fraction of completed trials). Omissions were very rare and all drugs tested failed to affect them, so omissions are not presented here.
Correct and incorrect responses were measured as percentage of all completed trials in five-trial bins (trials 1–5, 6–10, etc.). In each study, we analyzed these measures across the maximum number of trials bins where data points were present for all rats (because the number of trials available for analysis depended upon how quickly the rat reached criteria). For the retention sessions, only four bins were considered in all drug studies (all experiments included rats that completed the retention sessions in the minimum number of trials, i.e., 20), while for the reversal sessions, the number of bins was variable (between four and seven).
Furthermore, incorrect responses were defined as perseverative (i.e., responses to the hole rewarded in the previous new-hole session) or neutral (responses to the other two incorrect holes) and were analyzed as a percentage of all completed trials in five-trial bins, as described above.
Monkey behavioral testing apparatus
A modified Wisconsin general test apparatus (WGTA) consisting of an opaque screen (that could be raised and lowered) that separated the monkey from a tray fitted with three opaque square food boxes with hinged opaque lids was used. The tops of the food boxes were fitted with distinctive colored pictures (clip art from the Microsoft Office library), and the monkeys were trained to open the lids of food boxes to retrieve food rewards (bits of apple, grape, or banana) hidden within. The monkeys, which had previously been trained to move voluntarily from their cages to a transport cart, were moved, one at a time, to an adjacent testing room. In this room, the transport cart was aligned to the WGTA so that monkeys could easily access the food boxes on the tray when the opaque screen was raised. The monkeys were presented with trials on which the screen was raised to reveal three food boxes with the visual cues. They were allowed to open only one box lid per trial. Each trial lasted either until the animal opened a lid or 2 min passed, whichever came first. The inter-trial interval was approximately 15 s. For each set of three visual cues, the picture that was chosen to be the “positive” stimulus was varied across the four subjects. The position of the food boxes varied pseudorandomly across all trials. Each subject was tested between 8:30 and 11:00 a.m. on each day.
Monkey testing—reversal learning task
Before the four monkeys participated in this study, they were trained on a slightly different reversal learning paradigm and were involved in other pharmacological studies (Lee et al. 2007
). We modified our earlier task design in an attempt to increase the general difficulty of the task. Our procedure, like some human discrimination learning tasks (Frank et al. 2007
), involved training subjects on two separate discriminations, composed of three stimuli each, within single sessions (discrimination set 1: A,B,C; discrimination set 2: D,E,F). In every session, half of the trials included discrimination set 1 and the other half involved discrimination set 2. The two sets of stimuli were mixed pseudorandomly across trials; the only constraint on stimuli presentation order was that the same triad of pictures was not presented in more than three consecutive trials. Across daily sessions of 30 trials (15 trials with stimuli A,B,C and 15 with D,E,F), the monkeys had first to learn that one stimulus per discrimination was associated with reward while the others were not (i.e., A+, B−, C− and D+, E−, F−). Once a minimum criterion of 13 correct responses out of 15 trials was reached for both sets of stimuli in a single session, the subjects were tested on the subsequent day for retention or reversal of the two discriminations.
The maximum number of acquisition sessions allowed was four, and when a monkey did not meet this criterion in four sessions, the testing for that subject was interrupted and restarted the following week with new sets of stimuli. However, this circumstance was not very common, and often, monkeys learned both discriminations within one session.
The retention and reversal sessions consisted of two contiguous blocks of 20 trials each consisting of an equal mix of 10 A-B-C trials and ten D-E-F trials (Fig. ). In the retention sessions, the reward contingencies in the two blocks were the same as in the acquisition sessions (A+, B−, C− and D+, E−, F−). Differently, in the reversal sessions, the stimulus–reward associations were changed so that discrimination set 1 was reversed from the beginning of the test session (A−, B+, C−), while discrimination set 2 was retained in the first block (D+, E−, F−) and reversed in the second block (D−, E+, F−). In this way, monkeys were subjected to two different reversal conditions in the session, one preceded by a retention test to “prime” responding (involving discrimination set 2) and one that was not (involving discrimination set 1). The retention and reversal sessions were alternated in a pseudorandom way so that no more than two consecutive weeks included the same learning conditions.
Fig. 1 Schematic representation of the experimental design used in the monkey studies. The acquisition sessions consisted of 15 trials for each set of discriminanda (discrimination 1: A, B, C, disc1; discrimination 2: D, E, F, disc2) presented in a pseudo-random (more ...)
The measures collected were retention session errors (total errors committed in the retention sessions), reversal session retention errors (errors committed during retention of discrimination set 2 in the first block of the reversal session), perseverative errors (number of responses to the previously reinforced stimulus during reversal of discrimination set 1 or 2), neutral errors (number of non perseverative errors during reversal session), retention session duration (total time required to complete the retention session; in s), and reversal session duration (total time required to complete the reversal session; in s).
All studies were within subjects and the order of drug conditions counter-balanced (cyclic Latin square design) across subjects and testing conditions (retention and reversal).
For the rat experiments, the measures described above were subjected to repeated measures analysis of variance (ANOVA), with testing condition, bin, and dose as factors. Where significant main effects or interactions were detected, they were further analyzed using a paired two-tailed t tests. Our a priori hypotheses were tested using paired one-tailed t tests.
For the monkey studies, specific a priori hypotheses were examined using one-tailed paired t tests, while for all the other comparisons, two-tailed paired t tests were used.