|Home | About | Journals | Submit | Contact Us | Français|
Opioid-conditioned reinforcement is thought to exacerbate opioid abuse and dependence. Sex/gender can influence opioid abuse behaviors, but the effects of sex/gender on opioid-conditioned reinforcement, specifically, are unclear. In this study we compared new-response acquisition with opioid-conditioned reinforcement in male and female rats. First, separate groups received response-independent remifentanil injections (0.0-32.0 μg/kg, IV) and presentations of a light–noise stimulus. In experimental groups, injections and stimulus presentations always co-occurred (“paired PAV”); in control groups, the two occurred independently of each other (“random PAV”). Next, in instrumental acquisition (ACQ) sessions, two novel nose-poke manipulanda were introduced. All animals (regardless of sex, dose, PAV type) could respond in the active nose-poke, which produced the stimulus alone, or the inactive nose-poke. Both males and females dose-dependently acquired nose-poke responding (active > inactive) after paired PAV, but not after random PAV. Therefore, the stimulus was a conditioned reinforcer. We identified three sex differences. First, only females acquired responding after PAV with 32.0 μg/kg remifentanil. Second, using a progressive ratio schedule for ACQ, both sexes acquired responding, but females made significantly more active responses. Third, when a single session of PAV was conducted, only males acquired responding. Thus, rats’ sex interacts with pharmacological and environmental factors to determine opioid-conditioned reinforcement.
Opioid-associated environmental stimuli (i.e., cues) can significantly exacerbate opioid abuse behaviors. Cue presentation enhances ongoing opioid self-administration responding in laboratory animals (e.g., Alderson et al., 2000; Goldberg and Tang, 1977), and exposure to cues and cue-induced craving have been associated both retrospectively and prospectively with relapse in human opioid abusers (Fatseas et al., 2011; Garland and Howard, 2014; Heather et al., 1991; Lubman et al., 2009; Unnithan et al., 1992). Attenuating cue effects is an important goal in developing refined treatments for drug abuse and dependence (Milton and Everitt, 2010; Myers and Carlezon, 2010; Peck and Ranaldi, 2014; Taylor et al., 2009). There are currently FDA-approved opioid antagonist and agonist medications available to block or substitute, respectively, for the effects of abused opioids themselves (Bart, 2012). However, opioid cues can still produce significant motivational effects in patients receiving these medications (Fatseas et al., 2011; Hyman et al., 2007; Lubman et al., 2009). Additional work is needed specifically to target opioid cue effects, and so it is important to clarify the mechanisms by which opioid cues control behavior.
Among Pavlovian drug-conditioned effects (reviewed by Milton and Everitt, 2010), conditioned reinforcement may be particularly robust or “insidious” (Robinson et al., 2014 p. 452; see also Di Ciano and Everitt, 2004; Taylor et al., 2009). Drug-conditioned reinforcement can maintain operant responding for extended periods when drug is unavailable, and drug-conditioned reinforcers can be used to train novel responses that were not previously reinforced by the drug itself (e.g., Bertz and Woods, 2013; Bertz et al., 2015; Di Ciano and Everitt, 2004; Palmatier et al., 2007). More generally, conditioned reinforcers can be unaffected by manipulations targeting the primary reinforcer that initially established them (Burke et al., 2007; Parkinson et al., 2005). Rather, conditioned reinforcement may help maintain behavior in opposition to primary reinforcement contingencies (Pears et al., 2003). These observations indicate that drug-conditioned reinforcement can enhance both the persistence and diversity of drug-related behavior in a manner that resists efforts to block or devalue the primary drug reinforcer.
It is, however, also important to recognize the significant sex/gender differences in opioid abuse behaviors that have been reported both preclinically and clinically. Studies in rats show that females acquire opioid self-administration more readily than males (Lynch and Carroll, 1999; Carroll et al., 2002) and have enhanced responding once self-administration is established (Alexander et al., 1978; Carroll et al., 2001; Cicero et al., 2003; Klein et al., 1997). Patterns are more complicated among human drug users, but work with opioid abusers from several different countries suggests that, broadly, women experience more severe clinical courses than men: women transition more quickly from initial opioid use to regular use and/or to treatment intake (i.e., “telescoping”) while experiencing more adverse events/effects across a wider range of life domains (Anglin et al., 1987; Chiang et al., 2007; Hernandez-Avila et al., 2004; Hölscher et al., 2010; Hser et al., 1987; Kelly et al., 2009; Shand et al., 2011).
The specific contributions of opioid-associated stimuli to these sex/gender differences are presently unclear. For example, heroin abusing women have reported greater craving than heroin abusing men when exposed to personalized descriptions of heroin use situations (Yu et al., 2007). This effect may indicate sex/gender differences in the motivational effects of drug cues per se, but several other sex/gender-dependent factors, such as participants’ willingness to report emotional states in a laboratory setting, may also significantly influence results of this type (Robbins et al., 1999). In rats, females have shown greater resistance to extinction of oral opioid self-administration when responding continued to produce the stimuli previously associated with drug delivery without the drug itself (Klein et al., 1997). This difference is consistent with enhanced conditioned reinforcement for the females. However, the same response produced both the drug and the stimuli in this case, and so the rats’ behavior may have depended entirely on the primary reinforcing effects of the drug, with the stimuli having no (conditioned) reinforcing effects of their own (Mackintosh, 1974; Williams, 1994). Female rats also show enhanced opioid-conditioned place preferences (Cicero et al., 2000; Karami and Zarrindast, 2008), but these differences could, likewise, arise from several different cue-based and/or drug-based behavioral mechanisms (Bardo and Bevins, 2000; Huston et al., 2013).
To provide a valid measure of sex differences in the conditioned reinforcing effects, in particular, of an opioid-associated stimulus, the present study tested male and female rats for new-response acquisition. To establish a new response with drug-conditioned reinforcement (e.g., Bertz and Woods, 2013), animals are first given response-independent drug injections and presentations of an exteroceptive stimulus such that the two events either are or are not consistently paired. Then, the animals are given access to novel instrumental responses that either do or do not produce the stimulus alone (i.e., without drug). If the stimulus is a conditioned reinforcer, animals should significantly prefer the response that produces the stimulus (vs. the response that does not) only after the stimulus was paired with drug. These procedures provide a stringent test for conditioned reinforcement (Mackintosh, 1974; Williams, 1994): the response that produces the stimulus does not and did not produce the drug, controlling for primary reinforcing effects, and it can be clearly shown that responding depends on both (1) the Pavlovian contingency between the stimulus and the drug and (2) the instrumental contingency between a particular response and the delivery of the stimulus. Using these strict criteria, the conditioned reinforcing effects of stimuli paired with several classes of abused drugs have been established in male rats (Bertz and Woods, 2013; Di Ciano and Everitt, 2004; Palmatier et al., 2007). However, previous studies have not, to our knowledge, directly compared the acquisition of responding with drug-conditioned reinforcement in male and female rats.
Three experiments were conducted using the potent, short-acting opioid agonist, remifentanil. Experiment 1 characterized the effect of remifentanil dose on the conditioned reinforcing effects of a remifentanil-paired stimulus in male and female rats. Having found in Experiment 1 effective doses of remifentanil for both males and females, Experiment 2 assessed the relative reinforcing efficacy of the stimulus under a progressive ratio (PR) schedule of reinforcement. Finally, Experiment 3 used a reduced number of drug-stimulus pairings to compare the speed with which the stimulus can be established as a conditioned reinforcer in males and females.
Male and female Sprague-Dawley rats (Harlan Laboratories; Indianapolis, IN) aged at least 65 days served as subjects in all experiments. During experiments, males weighed (mean ± SEM) 353 ± 2 g, whereas females weighed 256 ± 1 g. Each experimental group contained 10 or 12 animals. Animals were housed in a climate-controlled facility under a 12 h light-dark cycle (lights on at 07:00 h) and were allowed to acclimate to the facility for at least 1 week before beginning experiments. Experimental sessions were conducted 5-7 days/week during the light phase of the cycle. Animals were housed in same-sex groups of 3/cage before catheterization surgery (described below) and individually thereafter to protect their implants. All animals had unrestricted access to standard pellet chow and laboratory tap water in the homecage throughout. All studies were performed in accordance with the Guide for the Care and Use of Laboratory Animals (Institute of Laboratory Animal Research, 2011), and all experimental procedures were approved by the University of Michigan Committee on the Use and Care of Animals.
After acclimating to the facility, animals were catheterized for IV drug administration. Catheters were custom made from polyurethane tubing (MRE-040 or MRE-033, Braintree Scientific; Braintree, MA) and Tygon tubing (S-54-HL, Norton Performance Plastics; Akron, OH). Surgery was performed under ketamine/xylazine (90:10 mg/kg, IP) anesthesia. The catheter was inserted into the left femoral vein and routed subcutaneously to the scapulae, where it was secured to a fabric mesh back-plate (DC95BS, Instech Laboratories; Plymouth Meeting, PA, USA or 313-000BM, Plastics One; Roanoke, VA, USA) and attached to 22 ga stainless steel tubing for externalization. Carprofen (5.0 mg/kg, SC) was given for analgesia on the day of and the day following surgery. Rats were allowed to recover for at least 5 days after surgery before the start of experimentation. Catheters were flushed daily with 0.25 ml of heparinized saline (50 U/ml) to ensure patency.
Experimental sessions were conducted in two standard operant conditioning chambers controlled by Med-PC IV software (Med Associates; St. Albans, VT) as described in detail previously (Bertz and Woods, 2013). Each chamber was contained inside a light- and sound-attenuating cubicle and was located in a separate room of the laboratory. The right wall of each chamber contained a white incandescent houselight (ENV-215M, Med Associates) and the speaker for a tone generator (ENV-224AM and ENV-230, Med Associates). Two nose-poke manipulanda containing LED stimulus lights (ENV-114BM, Med Associates) could also be inserted into the right wall. When the nose-pokes were removed from the chamber, they were replaced by blank aluminum panels.
Motorized syringe drivers (PHM-107, Med Associates) were located outside of the light- and sound-attenuating cubicles to deliver IV drug injections. Syringes were attached to Tygon tubing (S-54-HL, Norton Performance Plastics) leading to a counterweighted fluid swivel (375/22PS, Instech Laboratories) and spring tether.
The behavioral testing procedures used were based on a previous study of remifentanil-conditioned reinforcement in male rats (Bertz and Woods, 2013). Within each experiment, the control animals were tested after the experimental animals, whereas the other experimental conditions (remifentanil doses, male vs. female rats) were tested in a nonsystematic order.
After recovery from surgery, animals received either “paired” or “random” Pavlovian conditioning (PAV) sessions. During PAV sessions, the nose-poke manipulanda were removed from the experimental chambers, and all animals received response-independent i.v. injections of remifentanil and presentations of a light–noise compound stimulus. The light–noise stimulus consisted of houselight illumination and white noise (80±5 dB measured at the chamber’s center). Injections and stimulus presentations lasted approximately 2.0 s, varying depending on the weight of the individual animal (male average: 2.0 s, male range: 1.5-2.4 s; female average: 2.2 s, female range: 1.8-2.6 s). In paired PAV groups, a single variable time (VT) 3 min schedule (range: 0.0-6.0 min) controlled both remifentanil injection and stimulus presentation, and injections and stimulus presentations always co-occurred. In random PAV control groups, remifentanil injection and stimulus presentation were controlled by independent VT 3 min schedules and were not explicitly unpaired. All PAV sessions lasted until 20 injections and 20 stimulus presentations occurred, approximately 60 min.
In Experiment 1, the following unit doses (μg/kg) of remifentanil were administered to separate groups of male or female rats in paired PAV: 0.0, 1.0, 3.2, 10.0, or 32.0. These doses were chosen from previous studies of rats’ remifentanil self-administration (Panlilio and Schindler, 2000; Panlilio et al., 2003). Based on their effects in paired PAV, the following unit doses (μg/kg) of remifentanil were administered to separate groups of control male or female rats in random PAV: 3.2, 10.0, or 32.0. On the basis of the results of Experiment 1, 10.0 μg/kg was administered to all groups in Experiments 2 and 3. In all cases, remifentanil dose was set by changing the concentration of the remifentanil solution, not by altering the infusion duration.
In Experiments 1 and 2, PAV was conducted for 5 consecutive sessions (100 total drug/stimulus pairings, “extensive pairing”). In Experiment 3, PAV was conducted for 1 session (20 total drug/stimulus pairings, “limited pairing”).
After PAV, all animals (i.e., regardless of sex, remifentanil dose, and PAV type) were tested in a series of instrumental acquisition (ACQ) sessions. During ACQ, the two nose-pokes were present in the experimental chambers. The start of each ACQ session was indicated by the illumination of the stimulus lights inside both nose-pokes, and both nose-pokes remained illuminated for the duration of the session. The side of the active nose-poke (right vs. left) was counterbalanced between subjects within each group. Remifentanil was never given during ACQ. Responses in the inactive nose-poke were recorded but had no scheduled consequences. Responses in the active nose-poke produced the light-nose stimulus alone, as detailed below.
In all experiments, seven ACQ sessions (ACQ1-7) were conducted, but the schedule of reinforcement and session duration varied among experiments. In Experiments 1 and 3, active responses were reinforced under a modified random ratio (RR) 2 schedule: the first active response in each session produced the stimulus with a probability of 1.0, whereas each subsequent active response in the session produced the stimulus with a probability of 0.5. RR 2 ACQ sessions lasted 60 min. In Experiment 2, ACQ1 was conducted under the RR 2 schedule as described above. Then, in ACQ2-7, active responses were reinforced under a PR schedule: ratio requirements increased within the session (1, 2, 4, 6, 9, 12, 15, 20, 25, 32, etc.) according to the equation of Richardson and Roberts (1996): ratio value = [5e(reinforcer number * 0.2)]–5. PR sessions lasted for 240 min or until a ratio requirement was not completed for 45 min, whichever occurred first.
Remifentanil (Ultiva brand, GlaxoSmithKline; Uxbridge, Middlesex, UK or Mylan Institutional; Rockford, IL, USA) was obtained from the hospital pharmacy of the University of Michigan Health System and dissolved in sterile physiological saline. All injections were given in a volume of 0.1 ml/kg.
In Experiments 1 and 3, preference for the active response was calculated (active responses–inactive responses) for each animal in each ACQ session. As raw data, considering all conditions, responding ranged as follows: male active, 0-81; male inactive, 0-56; female active, 0-143; female inactive, 0-61. Animals successfully acquired responding for the stimulus if they made significantly more active responses than inactive responses, which in this case manifested as preference scores being significantly greater than zero. Preference scores were analyzed separately for paired PAV groups and random PAV groups. In Experiment 1, preference scores were analyzed using three-way ANOVA with the within-subjects factor of session (ACQ1-7) and the between-subjects factors of sex (male, female) and remifentanil dose (0.0-32.0 μg/kg). In Experiment 3, because only one dose of remifentanil was tested, and because only males acquired responding, preference scores were analyzed using two-way ANOVA with the within-subjects factor of session and the between-subjects factor of sex (after paired PAV) or using one-way ANOVA with the within-subjects factor of session (after random PAV). In both experiments, following non-significant effects involving session, mean preference scores were compared pairwise using post hoc Bonferroni-corrected t-tests. To assess significant effects involving dose, one-sample t-tests were used to compare preference scores with zero. To assess sex differences, independent-samples t-tests were used to compare males with females. Corrected p values are reported in all cases. Finally, in Experiment 3, the number of “chance pairings” during random PAV was calculated as the number of times remifentanil injection and stimulus delivery co-occurred given the independent operation of the two VT schedules. The Pearson correlation coefficient between mean preference scores during ACQ and number of chance pairings during PAV was then calculated.
In Experiment 2, active and inactive responses under the PR schedule were first analyzed using three-way ANOVA with factors for sex, manipulandum, and session. Because this analysis revealed significant differences in active, but not inactive responding, the following aspects of PR performance were then each analyzed using two-way ANOVA with factors for session and sex: reinforcers (i.e., the number of ratios completed), break point (i.e., the value of the final ratio completed), and total session duration (min). For all endpoints, following non-significant effects involving session, pairwise comparisons of the mean values were made post hoc using Bonferroni tests to compare active vs. inactive responding and/or males vs. females. To determine whether males and females differed in ACQ1, prior to the start of testing under the PR schedule, responding was analyzed using two-way ANOVA with factors for sex and manipulandum.
All analyses were performed with Prism 6.0 (GraphPad Software; La Jolla, CA) or SPSS Statistics 21 (IBM; Armonk, NY). Differences were considered significant when p < .05, two-tailed.
Figure 1 presents rats’ preference for the active nose-poke response after 5 sessions of paired PAV conducted with different doses of remifentanil. Figure 1a presents separately each of the 7 ACQ sessions. During ACQ, preference for the active response in both males and females was significantly affected by the dose of remifentanil with which the stimulus had been paired [main effect of dose: F(4,106) = 13.48, p < .001; dose X sex: F(4,106) = 1.32, NS]. Preference did not significantly change across sessions [main effect of session and all interactions: 0.46 < F < 1.97, p’s > .05]. Therefore, the data were collapsed across sessions for pairwise comparisons (Figure 1b): males acquired responding for the stimulus after it was paired with 3.2 μg/kg [t(11) = 5.37, p < .001] or 10.0 μg/kg [t(9) = 6.47, p < .001], whereas females acquired responding for the stimulus after it was paired with 3.2 μg/kg [t(11) = 3.51, p < .025], 10.0 μg/kg [t(11) =5.05, p < .002], or 32.0 μg/kg [t(11) = 4.19, p <= .01]. All other preference scores were not different from zero [0.21 < t < 2.70, p’s > .10]. The main effect of sex only approached significance [F(1,106) = 3.73, p = .056]; however, pairwise comparisons were performed given the difference between males and females in the range of doses that produced successful acquisition noted above. Numerically, females had greater preferences than males for the active response after receiving all non-zero doses of remifentanil; however, females had a significantly larger preference for the active response after paired PAV only with 32.0 μg/kg [t(20) = 2.92, p < .05].
Figure 2a presents animals’ preference for the active nose-poke after 5 sessions of random PAV conducted with the remifentanil doses that produced successful acquisition of responding after paired PAV. After random PAV, animals’ preference for the active response varied across sessions [main effect of session: F(6,324) = 4.27, p < .001]; however, no effects of dose or sex were significant [main effects and all interactions: 0.24 < F < 2.60, p’s > .05]. Averaged across sessions (Figure 2b), neither males [2.07 < t(9) < 2.27, p’s > .10] nor females [0.01 < t(9) < 2.48, p’s > .10] acquired responding after random PAV with any remifentanil dose.
Figure 3 presents male and female rats’ responding for the remifentanil-paired stimulus under the PR schedule of reinforcement when ACQ was assessed after 5 sessions of paired PAV with 10.0 μg/kg remifentanil. Figure 3a presents the animals’ responding in each ACQ session. In ACQ 1, under the RR 2 schedule, responding did not differ by sex [main effect and manipulandum 2.08 < F < 2.14, p’s > .05]. Under the PR schedule, active and inactive responding differed significantly [main effect of manipulandum: F(1,20) = 25.62, p < .001]. Responding varied across sessions [main effect of session: F(5,110) = 2.60, p < .05]; however, this difference did not depend on either the manipulandum or animals’ sex [all interactions: 0.63 < F < 1.03, p’s > .10]. Averaged across sessions (Figure 3b), both males and females acquired responding, making more active responses than inactive responses [males: t(22) = 2.53, p < .05; females: t(22) = 4.62, p < .001]. Responding under the PR schedule was also affected by sex [main effect of sex: F(1,22) = 5.71, p < .05]. Although the sex X manipulandum interaction was not significant [F(1,22) = 2.19, NS], pairwise comparisons showed that females made more active responses than males [t(44) = 2.79, p < .02], whereas inactive responding did not differ by sex [t(44) = 0.91, NS]. Among the other aspects of PR performance measured (Figures 3c-d), females had significantly longer sessions than males [main effect of sex: F(1,22) = 5.84, p < .025; t(22)=2.41, p < .025], and there was a trend for females to have higher break points than males [main effect of sex: F(1,22) = 3.78, p = .064; t(22) = 1.94, p = .064]. All other differences were not significant.
Figure 4 presents rats’ preference for the active nose-poke response after 1 single session of paired PAV with 10.0 μg/kg remifentanil. Preference did not differ significantly by sex or across sessions [both main effects and interaction: 0.13 < F < 0.30, p’s > .05]. Averaged across sessions (Figure 4a), males significantly preferred the active response [t(9) = 4.04, p < .005], whereas females did not [t(11) = 1.77, NS]. As shown by the plot of individual animals in Figure 4a, this difference in the presence vs. absence of acquisition depended on the greater variability of females’ preference, rather than females having a smaller mean preference [t(20) = 0.18, NS]. Numerically, females had both the largest preference and the largest negative preference score for the active response.
Because males acquired responding after paired PAV, a random PAV control group of males was tested (Figure 4b). After 1 session of random PAV with 10.0 μg/kg remifentanil, males’ responding did not differ across sessions [F(6,54) = 0.57, NS]. Averaged across sessions, males did not prefer the active response after random PAV [t(9) = 2.06, NS]. As shown in Figure 4b, there was considerable variability in animals’ preference scores after random PAV, with two animals (#2, #4) showing a larger numerical preference for the active response than the others. To determine whether the chance pairing of remifentanil and the stimulus in the random PAV session could account for these differences, the number of pairings animals experienced during the random PAV session (range: 2-9) was correlated with their mean preference during ACQ (data not shown). The association between chance pairings and preference for the active response was not significant [r = −0.35, NS].
Separate streams of research have made considerable progress in recent years clarifying the importance of (1) understanding the specific psychological processes that allow drug cues to control behavior (e.g., Berridge and Robinson, 2003; Milton and Everitt, 2010) and (2) sex/gender differences in drug abuse behaviors (e.g., Becker and Hu, 2008; Carroll and Anker, 2010). However, work is still needed to bring these ideas together: to characterize the particular effects of sex/gender on the different behavioral mechanisms acting in drug abuse and dependence. To this end, the present study used new-response acquisition to compare directly opioid-conditioned reinforcement in male and female rats.
The remifentanil-paired stimulus dose-dependently served as a conditioned reinforcer in both males and females. During ACQ, both males and females made significantly more active responses than inactive responses after paired PAV, but not after random PAV. The animals’ behavior was, thus, determined by both the particular consequences of the nose-pokes during ACQ and the contingency between remifentanil and the stimulus during PAV, as required for conditioned reinforcement (Mackintosh, 1974 p. 234). Having established that the stimulus was a conditioned reinforcer, we identified several differences between males and females.
In the dose-effect analysis, the stimulus was reinforcing for females after it was paired with a broader range of remifentanil doses. In particular, females, but not males, acquired responding for the stimulus after it was paired with the highest dose of remifentanil tested, 32.0 μg/kg. This dose also comprised the descending limb of the dose-response function in both sexes. Previous studies of morphine or oxycodone in rats have produced results consistent with monotonic increases in opioid-conditioned reinforcement with increasing drug dose (Crowder et al., 1972; Grella et al., 2011). However, these studies did not include the control conditions necessary to attribute the changes in responding to conditioned reinforcement, as opposed to other associative and non-associative processes that can change responding, and so dose-response effects are difficult to interpret. The present results are consistent with a dose-response study of rats’ morphine-conditioned place preference (Cicero et al., 2000): females, but not males, significantly preferred the morphine-paired chamber after it was paired with higher doses of morphine, and this behavioral difference was not associated with sex differences in brain levels of morphine.
The differences in ACQ observed presently are also unlikely to depend entirely on sex differences in remifentanil metabolism during PAV, i.e., males and females experiencing vastly different remifentanil blood/brain levels being paired with stimulus presentation given the same unit doses of remifentanil from the syringe drivers. First, a general sex difference in remifentanil metabolism would be expected to affect animals’ responses to both lower and higher doses, not the highest dose alone. Second, sex/gender differences in pharmacokinetics have been observed for other opioids that are metabolized hepatically, although these differences are not observed consistently and are not consistently related to behavioral differences (Fillingim and Gear, 2004). Remifentanil is, in contrast, metabolized by non-specific tissue esterases independently of hepatic and renal function (Servin and Billard, 2008), and sex/gender differences in remifentanil metabolism have not been reported in clinical studies (Minto et al., 1997; Westmoreland et al., 1993).
Considering opioid pharmacodynamics, although sex differences in rats’ opioid receptor density and function have been reported (reviewed by Craft, 2008), the associative complexity of conditioned reinforcement makes it difficult to interpret sex/gender differences strictly in terms of basic receptor properties (cf., Palmatier et al., 2008). By definition, drug-conditioned reinforcement depends not only on the initial formation of an interoceptive stimulus (i.e., as drug molecules commence signaling cascades), but also on Pavlovian and instrumental learning operations performed on that stimulus. For instance, male and female rats may differ in the molecular mechanisms of opioid transduction (Selley et al., 2003), but the behavioral differences observed may depend on sex differences in associative learning processes that involve other neurotransmitter systems (e.g., dopaminergic influences on Pavlovian incentive learning; Dickinson et al., 2000). Despite these complexities, the dose-effect functions obtained presently show that drug-conditioned reinforcement is amenable to foundational pharmacological analyses. More generally, new-response acquisition procedures should provide a valid behavioral basis for the work necessary to resolve further the neurobiological substrates of drug-conditioned reinforcement (see also Bertz et al., 2015).
On a behavioral level, at least three types of effect could account for decreased responding when the stimulus was paired with a high dose of remifentanil, producing a descending limb in the dose-effect function: (1) high doses of remifentanil have aversive effects, resulting in a negative or inhibitory Pavlovian association; (2) remifentanil has effects of memory/consciousness that prevent or disrupt association of the stimulus with remifentanil during PAV; (3) as a Pavlovian conditioned stimulus (CS), the stimulus elicits conditioned responses (CR) that are incompatible with nose-poke responding. Previous studies have shown that 32.0 μg/kg remifentanil maintains self-administration responding in male rats, and the inter-injection intervals experienced by the animals in these studies are similar to inter-injection intervals used presently for PAV (i.e., up to ~7 min; Panlilio and Schindler, 2000; Panlilio et al., 2003). There can be significant differences between self- and experimenter-administered remifentanil (Crespo et al., 2005), but it is unlikely that this high remifentanil dose is only aversive or unable to enter into learned associations. Response-independent drug injections were used presently to avoid sex differences in opioid self-administration behavior (reviewed above) that could affect subsequent responding for the stimulus alone, complicating analyses of conditioned reinforcement.
Regarding competing responses, it is important generally to recognize that any pairing of a stimulus with a primary reinforcer to create a conditioned reinforcer also renders that stimulus a Pavlovian CS that is capable of eliciting a variety of different CR. These CR may or may not be compatible with instrumental responding (Cunningham, 1993; Mackintosh, 1974 p. 243-244; Williams, 1994). Significant sex differences in rats have been observed in the unconditioned locomotor effects and rate-suppressant effects of opioids in operant conditioning experiments (reviewed by Craft, 2008). The difference in the descending limb of the remifentanil-conditioned reinforcement dose-effect function may reflect a sex difference in animals’ susceptibility to conditioned locomotor effects caused by CS presentation. It is, however, crucial to re-emphasize that neither conditioned nor unconditioned locomotor activation alone can account for the acquisition of responding during ACQ. These possibilities are excluded by the significant differences between the nose-pokes observed based on their consequences, as well as the different patterns of responding obtained after paired vs. random PAV. Rather, these multiple effects/functions of cues that interact to determine performance with drug-conditioned reinforcement are analogous to the multiple effects/functions of drugs themselves (e.g., primary reinforcing effects vs. “direct” effects on behavior) that interact to determine performance with drug self-administration.
Under the PR schedule, females showed enhanced active, but not inactive, responding and worked for longer periods of time for the stimulus. In contrast to these results under the PR schedule, males and females did not differ when ACQ was assessed under the RR 2 schedule after the stimulus was paired with 10.0 μg/kg remifentanil, considering either the preference scores presented in Figure 1b or the raw data (i.e., active and inactive responses separately; data not shown). These differences in PR responding may indicate that the stimulus had greater relative reinforcing effectiveness for females than males. This interpretation, including the lack of difference under a ratio schedule with a lower response requirement, is consistent with studies of drug self-administration under progressive vs. fixed ratio schedules (see reviews by Arnold and Roberts, 1997; O’Brien and Gardner, 2005; Heidbreder, 2013). However, we are hesitant to make this conclusion too forcefully, given the (small) size and (marginal to no) statistical reliability of the differences in breakpoint and number of reinforcers earned. Further work is needed to understand how the relative effectiveness of drug-conditioned reinforcers can best be established. Behavioral-economic demand curves have proven useful for characterizing the relative reinforcing effectiveness of primary drug reinforcers in drug self-administration studies (e.g., Bentzley et al. 2013; Koffarnus et al. 2012). Demand curves may, likewise, aid in understanding the effectiveness or “value” of conditioned reinforcers.
More generally, these differences obtained under the RR 2 vs. PR schedules highlight the importance of environmental factors in characterizing sex/gender differences in drug-related behaviors. Along with drug dose, schedule of reinforcement is one of the major determinants of the primary reinforcing effects of drugs as assessed in self-administration experiments (e.g., Lagorio and Winger, 2014; see also reviews by Arnold and Roberts, 1997; Moser et al., 2010; Spealman and Goldberg, 1978). Furthermore, schedule of reinforcement can change how pharmacological and neurological interventions affect drug self-administration behaviors (Bourland and French, 1995; Heidbreder, 2013; Hutcheson et al., 2001; Olmstead et al., 1998), and schedule of reinforcement can interact with other environmental factors in self-administration experiments (e.g., environmental enrichment; Green et al., 2002). Therefore, it is reasonable that performance with drug-conditioned reinforcement is affected by schedule of reinforcement during ACQ, even when the same PAV parameters are used, and findings of this type are important in determining the breadth of circumstances under which sex/gender differences in drug abuse behaviors manifest (cf., Caine et al., 2004).
Lastly, in Experiment 3, the remifentanil-paired stimulus served as a conditioned reinforcer after only one session of PAV in males, but not females. Examining the individual animals (Figure 4a), this difference depended on the relative consistency of males’ preference for the active response, rather than a difference between males and females in the magnitude of preference. Comparing the effects of limited pairing with 10.0 μg/kg remifentanil in Experiment 3 to the extensive pairing used in Experiment 1, increasing the drug-stimulus pairing approximately tripled the magnitude of the preference scores for both sexes (Figure 4a vs. Figure 1b), whereas increasing the number of random PAV sessions from 1 to 5 sessions produced a slight decrease in males’ mean preference score (8.1 vs. 7.9, Figure 4b vs. Figure 2b). Along with the non-significant correlation with chance pairings in Experiment 3, this difference in the effect of increasing the amount of drug/stimulus exposure indicates that the numerical preferences for the active response observed after random PAV are not due to drug-stimulus pairing, whereas the strength of the conditioned reinforcer does depend on the strength of the remifentanil–stimulus association (see also Bertz and Woods, 2013).
Experiment 3 was designed to characterize a boundary condition (i.e., how little Pavlovian conditioning could be used to create a conditioned reinforcer), and so numerically small preferences and/or high variability are not unexpected. It is still noteworthy that a single episode with remifentanil is sufficient to establish a conditioned reinforcer in males. Nonetheless, it may be particularly important in future studies to address the source(s) of females’ greater variability. As PAV was, in this case, confined to a single day, hormonal status may be a significant contributor. Little is currently known about the effects of gonadal hormones on the motivational effects of drug cues per se. In cocaine-trained animals, estrus cycle phase was found to have a small, but significant, effect on cue-primed reinstatement (Fuchs et al., 2005). Studies assessing specifically the effects of gonadal hormones on opioid-associated stimuli are presently lacking, but estradiol has been shown to enhance heroin self-administration in ovariectomized females (Roth et al., 2002; but see Stewart et al., 1996 for negative results). The potential for significant hormonal effects is also supported by the ability of gonadal hormones to regulate the activity of a number of brain regions associated with the behavioral effects of drug-associated stimuli (Hudson and Stamp, 2011).
Together, the present experiments provide, to our knowledge, the first demonstration of significant sex differences in a stringent test for drug-conditioned reinforcement. Rats’ sex interacted in a complex manner with both pharmacological (drug dose, number of injections) and environmental (schedules of Pavlovian and instrumental reinforcement) variables to determine responding for the remifentanil-associated stimulus. Sex/gender differences have implications for both pharmacotherapies and behavioral interventions for treating drug abuse and dependence (e.g., Wetherington, 2010), and so the present results may be relevant to reducing the control over behavior exerted by opioid-associated stimuli in opioid-abusing men and women.
This research was supported by the National Institute on Drug Abuse under grants T32DA07268 and R01DA024897 and by the University of Michigan Undergraduate Research Opportunity Program. A preliminary report of these data was given at the 2014 meeting of the College on Problems of Drug Dependence (San Juan, Puerto Rico).
Conflicts of interest
The authors declare they have no conflicts of interest.