|Home | About | Journals | Submit | Contact Us | Français|
The ability to make rapid, informed decisions about whether or not to engage in a sequence of actions to earn reward is essential for survival. Modeling in rodents has demonstrated a critical role for the basolateral amygdala (BLA) in such reward-seeking actions, but the precise neurochemical underpinnings are not well understood. Taking advantage of recent advancements in biosensor technologies, we made spatially discrete near-real time extracellular recordings of the major excitatory transmitter, glutamate, in the BLA of rats performing a self-paced lever-pressing sequence task for sucrose reward. This allowed us to detect rapid transient fluctuations in extracellular BLA glutamate time-locked to action performance. These glutamate transients tended to precede lever pressing actions and were markedly increased in frequency when rats were engaged in such reward seeking actions. Based on muscimol and tetrodotoxin microinfusions these glutamate transients appeared to originate from the terminals of neurons with cell bodies in the orbital frontal cortex. Importantly, glutamate transient amplitude and frequency fluctuated with the value of the earned reward and positively predicted lever pressing rate. Such novel rapid glutamate recordings during instrumental performance identify a role for glutamatergic signaling within the BLA in instrumental reward-seeking actions.
The ability to make rapid, informed decisions about whether or not to seek out a reward, such as a palatable food, is essential for survival. Indeed, aberrant reward-related decision-making is a hallmark of addiction, obesity and many other neuropsychiatric disorders. It is therefore imperative to elucidate the neural signals of decision-making to understand on a basic level the mechanisms of desire and on a disease level how these signals may be altered to produce maladaptive reward-seeking behavior.
Considerable research efforts, primarily utilizing pharmacological and genetic techniques, have revealed a complex circuitry underlying instrumental reward seeking that includes the amygdala, thalamus and various cortical regions implicated in decision-making (Balleine and Dickinson, 2000; Balleine et al., 2003; Corbit et al., 2003). Within this circuitry the basolateral amygdala (BLA) appears to be critical for encoding reward value and for use of this information to guide goal-directed actions (Balleine et al., 2003; Wassum et al., 2009b). This region receives excitatory glutamatergic inputs from several areas, but particularly strong direct and indirect reciprocal connections exist with the orbitofrontal cortex (OFC) (Ray and Price, 1992; Carmichael and Price, 1995; Ghashghaei and Barbas, 2002). Indeed, an interaction between the BLA and OFC has been suggested to be necessary for the performance of reward seeking actions (Pickens et al., 2003; Schoenbaum et al., 2003; Holland and Gallagher, 2004). More generally, activation of ionotropic glutamate receptors is necessary not only for instrumental learning (Kelley, 2004), but also for making informed decisions about whether to seek out reward (Johnson et al., 2005; Johnson et al., 2007). What remains to be elucidated is the profile of glutamatergic activity during instrumental behavior, and specifically during the decision-making processes that guides self-initiated action sequences.
This will require direct online examination of glutamatergic transmission during the decision-making process, which has been precluded until now by the lack of a method permitting selective measurement of extracellular glutamate with sufficiently high temporal and spatial resolution in freely-behaving animals. Here we take advantage of recent advances in glutamate biosensor technology, developed in our laboratory (Hamdi N, 2006; Wassum et al., 2008) and elsewhere (Hu et al., 1994; Ryan et al., 1997; Rutherford et al., 2007), to make spatially discrete near-real time recordings of extracellular glutamate in the BLA during performance of a multi-action, self-paced sequence task for sucrose reward. We identified rapid, transient glutamate events, potentially reflective of synaptic overspill, that were time-locked to lever-pressing actions and varied according to task-engagement. Such measurements of glutamate transients during free-operant performance provide evidence in support of a role for rapid BLA glutamatergic signaling in the seeking actions that are instrumental to obtaining reward.
Building on earlier work (Lowry JP, 1998; Kulagina et al., 1999; Pomerleau et al., 2003; Hamdi N, 2006) we developed a silicon microprobe-based platinum microelectrode array glutamate biosensor for sensitive, selective, and spatially precise second-to-second measurement of glutamate concentration changes in the brains of freely-behaving rats (Figure 1) (Wassum et al., 2008). These sensors use glutamate oxidase as the biological recognition element for glutamate and rely on electrooxidation (via constant potential amperometry) of enzymatically-generated hydrogen peroxide, the current output from which is recorded and converted into glutamate concentration via an in vitro calibration factor. Interference from both electroactive anions and cations is effectively excluded from the amperometric recordings, while still maintaining a <1 s response time, by application of Nafion and polypyrrole films to the electrode sites prior to enzyme immobilization (Wassum et al., 2008). Such selectivity of glutamate detection is a key factor in the sensor design given the presence of dopamine and other electroactive neurotransmitter-containing terminals in the BLA. Furthermore, incorporation of a non-enzyme-coated sentinel electrode on the microelectrode array, signals from which were subtracted from those of the enzyme-coated electrode, provided the noise reduction necessary to reveal rapid transient glutamate events.
The microelectrode array (MEA) probes were fabricated in the Nanoelectronics Research Facility at UCLA. These general procedures have been described previously (Wassum et al., 2008). A 1 μm thick layer of silicon dioxide was grown thermally on a thin (150 – 200 μm) silicon substrate. The thermal oxide is a high quality dielectric film that electrically isolates the substrate from the metal layer subsequently deposited. Electron-beam evaporation was used to deposit 1000 Å of platinum on a 200 Å chromium adhesion layer. The metal was patterned by photolithography and lift-off to define the bonding pads, connections, and electrode sites. Next, plasma enhanced chemical vapor deposition was used to deposit a 1 μm layer of silicon dioxide (5000 Å) and silicon nitride (5000 Å). This second dielectric layer chemically isolates the connections between bonding pads and electrodes. After patterning of the oxide layer with a conventional photolithographic technique, the bonding pads and electrode sites were exposed by reactive ion etching. A third photolithographic treatment was performed to pattern the outline of the probes. Reactive ion etching was then used to etch through the first and second dielectric layers, and deep reactive ion etching by the Bosch process was used to etch through the silicon substrate. The output of the micromachining process was a 4-inch diameter silicon wafer patterned with 150 MEA probes. The probes were 150 μm thick x 120 μm wide with 4 platinum electrode sites at the tip. The electrode sites measured approximately 40 × 100 μm, with a 4,800 μm2 surface area. Horizontally-paired electrodes were 40 μm apart, while vertically paired electrodes were 100 μm apart (see Figure 1A for probe tip image). 2 electrodes per MEA were utilized for the experiments here.
After the MEA probes were released individually from the wafer they were packaged and chemically cleaned to prepare the electrode surfaces for chemical modification with polymers and enzyme. Packaging involved soldering 30-gauge wire to the platinum bonding pads at the top of the MEA. Each MEA was chemically cleaned with isopropyl alcohol and then electrochemically cleaned in 0.5 M H2SO4 by cyclic voltammetry from -0.25 V to 1.65 V with a scan rate at 50 mV/s for 3 cycles. Following cleaning, the electrodes were dried with argon.
The coating schema of an electrode prepared for glutamate detection is represented in Figure 1B. Each electrode was coated with polypyrrole (PPy) to repel interference from cations, such as dopamine, and Nafion to electostatically repel anionic interference from molecules such as ascorbic acid and 3,4-dihydroxyphenylacetic acid (DOPAC). After the polymer treatments, the enzyme GluOx was immobilized onto select electrode sites to provide the biological recognition element for glutamate detection. The details of electrode modification have been reported previously (Wassum et al., 2008) and are briefly described below. Electrodes coated with PPy/Nafion and GluOx are referred to as glutamate biosensors. All electrodes were calibrated (see below) prior to implantation.
PPy was electrodeposited by holding the voltage constant at 0.85 V for 2.5-5 min until a total charge density of 20 mC/cm2 was reached in a 200 mM argon-purged solution of pyrrole in phosphate buffered saline (PBS) at pH 7.4. The polymer Nafion was deposited on the sites by rapid dip-coating of the probe tips in the Nafion solution and oven-casting at 180 °C for 3 min, followed by 1 min cooling in ambient air. This process was repeated twice. After the polymer treatments, enzyme immobilization was accomplished by chemical crosslinking using a solution consisting of GluOx (400 unit/mL), BSA (6 mg/mL), and glutaraldehyde (0.075%). A ~1 μL drop of the solution was formed on a syringe tip and fixed in place under a microscope. The probe was attached to a micromanipulator (Sutter Instrument Compant, Novato, CA) and positioned vertically relative to the enzyme solution droplet. With the aid of the microscope, the MEA was lowered into the enzyme droplet to coat only the bottom 2 electrodes. This was repeated 4-5 times with each application consisting of 2-3 dips. 3 days after enzyme immobilization the PPy was overoxidized by holding the potential applied to the electrode at -0.7 V for 30-45 min in phosphate buffered saline until a stable baseline was attained during the calibration. The MEAs were then sealed in a container with desiccant and stored at 4°C.
Nafion (5 wt.% solution in lower aliphatic alcohols/H2O mix), bovine serum albumin (BSA, min 96%), glutaraldehyde (25% in water), pyrrole (98%), L-glutamic acid, L-ascorbic acid, 3-hydroxytyramine (dopamine) were purchased from Aldrich Chemical Co. (Milwaukee, WI, USA). L-Glutamate oxidase (GluOx) from Streptomyces Sp. X119-6, with a rated activity of 24.9 units per mg protein (U mg−1, Lowry’s method), produced by Yamasa Corporation (Chiba, Japan), was purchased from Associates of Cape Cod, Inc. (Seikagaku America, MA, USA). Phosphate buffered saline (PBS) was composed of 50 mM Na2HPO4 with 100 mM NaCl (pH 7.4). Ultrapure water generated using a Millipore Milli-Q Water System was used for preparation of all solutions used in this work.
Electrochemical preparation of the sensors was performed using a Versatile Multichannel Potentiostat (model VMP3) equipped with the ‘p’ low current option and low current N’ stat box (Bio-Logic USA, LLC, Knoxville, TN, USA). In vitro and in vivo measurements were conducted using a multichannel Fast-16 potentiostat (Quanteon, LLC, Lexington, KY, USA), with reference electrodes consisting of a glass-enclosed Ag/AgCl wire in 3 M NaCl solution (Bioanalytical Systems, Inc., West Lafayette, IN, USA) or a 200 μm diameter Ag/AgCl wire, respectively. All potentials are reported versus the Ag/AgCl reference electrode.
Biosensors prepared for glutamate detection were calibrated in vitro to test for sensitivity and selectivity to glutamate. A constant potential of 0.7 V was applied to the working electrodes against a Ag/AgCl reference electrode in 40 mL of stirred PBS at pH 7.4 and 37 °C within a Faraday cage. Data were collected at 80 kHz and averaged over 1 s intervals. After the current detected at the electrodes equilibrated to baseline (approx. 30-45 min), aliquots of glutamate were added to the beaker to reach final glutamate concentrations in the range 5 - 60 μM. A calibration factor based on analysis of these data was calculated for each electrode on the MEAs to be used for in vivo experiments. The average calibration factor for the sensors used in these experiments was 149.6 μM/nA. Additionally, aliquots of ascorbic acid (250 μM final concentration) and dopamine (5-10 μM final concentration), were added to the beaker as representative examples of readily oxidizable potential anionic and cationic interferent neurochemicals, respectively, to confirm selectivity for glutamate. No current changes above the level of the noise were detected to the addition of these interferents, as we have reported previously (Wassum et al., 2008). In order to assess the sensitivity and response time to peroxide at sites uncoated with enzyme, aliquots of H2O2 were also added to the beaker. Importantly, electrodes coated with PPy and Nafion only, but not GluOx, showed no detectable response to glutamate, despite being sensitive to H2O2. Indeed, there was less than a 10% statistically insignificant (t15=1.68, p=0.11) difference in the H2O2 sensitivity on control electrode sites relative to enzyme-coated sites, indicating that any changes detected in vivo on the enzyme coated sites could not be attributed to endogenous H2O2.
For recording in the awake, freely-moving rat connection wires from the MEA and reference electrodes were soldered to gold-plated sockets (Ginder Scientific, Nepean, ON) and the silicon wafer-based MEA was attached with epoxy to a 9-pin miniature connector (Ginder Scientific) such that all the sockets were encased in the connector. The entire assembly was sealed with epoxy to ensure full insulation and allowed to cure for 1h prior to implantation. In some cases, see below, a guide cannula was also epoxied on top of the sensor to allow for local infusions around the sensor in vivo.
Male Sprague Dawely rats (N=23) weighing between 240-320 g were group housed and handled daily for 3 days prior to instrumental training. Training and testing took place during the light phase of the 12:12 h light:dark cycle. Rats were maintained on a food-deprived schedule whereby they received 10-12 g of their maintenance diet daily in order to maintain 85% free-feeding body weight. All rats were fed approximately 3 h after each day’s training session and all food was normally consumed within 1 h. Rats had free access to tap water in the home cage. All procedures were conducted in accordance with the Guide for the Care and use of Laboratory Animals and approved by the UCLA Institutional Animal Care and Use Committee.
Training and testing took place in a Med Associates (East Fairfield, VT) operant chamber housed within a sound- and light-resistant shell. The chamber contained two retractable levers that could be inserted to the left and right of the magazine. A 3 W, 24 V house light mounted on the top of the center wall opposite the magazine provided illumination. The chamber was equipped with a pellet dispenser that delivered a single 45 mg sucrose pellet (Bioserv, Frenchtown, NJ) into a small well mounted on the front wall of the chamber.
Single action-outcome reward seeking scenarios are rare in reality. Therefore we trained rats on a self-paced action sequence, a situation that better reflects decision-making problems that humans face in naturalistic settings. The behavioral procedure required rats to perform a fixed sequence of two different lever press actions to earn sucrose pellets, such that one action was temporally distal and the other temporally proximal to reward delivery. The distal lever was continuously available and, when pressed, resulted in the insertion of the proximal lever (a discriminative stimulus) into the chamber. Pressing the proximal lever resulted in the delivery of a sucrose pellet and caused that lever to be retracted. Importantly, this task was self-paced; the rat’s decision to engage in the sequence was not precipitated by an experimenter-delivered stimulus, rather the rat could control both the initiation of each ‘trial’ sequence as well as the speed with which the sequence of actions was performed. In this way the action sequence required both an initiation action, guided by some form of decision to engage in the sequence, as well as a termination action that directly provided reward delivery. Notably, the use of an action sequence task allowed for two primary behavioral measures to which we could correlate changes in glutamate detected with our glutamate biosensors: lever press rate, a measure of action performance, and action sequence time (the time from the initiating distal lever press through the sequence to the next initiating distal lever press).
Each session started with the illumination of the house light and insertion of the levers where appropriate and ended with the retraction of the levers and turning off of the house light. Rats received only one training session per day.
Rats received 2 days of magazine training in which they were exposed to non-contingent sucrose pellet deliveries (20 outcomes over 30 min) in the operant chamber with the levers retracted, in order to learn where to receive the sucrose reinforcement.
Rats were then given 2 days of single action continuous reinforcement training on the lever to the right of the magazine with the sucrose delivered on a continuous reinforcement schedule. Each session lasted until 20 outcomes had been earned, or 30 min elapsed.
On day 5 of instrumental training the distal lever (i.e. the lever to the left of the magazine) was introduced into the chamber. Initially, only the distal lever was present. One response on the left, distal lever resulted in the presentation of the right, temporally proximal lever, a response on which resulted in delivery of a single sucrose pellet and retraction of the proximal lever. This session continued until 20 outcomes were earned, or 30 min elapsed. This training was conducted for 5 sessions or until stable proximal response rates were attained. At this point rats underwent surgery for glutamate biosensor implantation.
The first experiment was intended to examine the profile of rapid glutamatergic changes in the BLA associated with action sequence performance. After training rats (N=7) were implanted with a pre-calibrated glutamate biosensor into the BLA (AP −3.0 mm, ML + 5.1, DV −8.0). After a brief recovery period the biosensor was tethered to the potentiostat (Fast-16, Quanteon) and a potential of 0.7 V was applied, versus a Ag/AgCl reference electrode in the contralateral cortex, for the oxidation of the GluOx-generated H2O2 (Wassum et al., 2008). This oxidative current was recorded at 80 kHz and averaged over 0.25 s intervals. Once the recorded amperometric signal equilibrated (approx. 30-45 min) the rat was transferred to the operant chamber and, after a 3 min baseline period, allowed to respond on the action sequence for a total of 30 sucrose pellet rewards. Importantly, rats were not retrained on the action sequence task after sensor implantation prior to test such that their initial responses (the first 5 cycles) were considered a ‘re-familiarization period’.
In order to examine whether changes in the value of the sucrose reward (information that is used to inform decisions) would affect glutamate signaling associated with subsequent action performance, a second set of rats were subjected to a specific satiety reward devaluation procedure. Following biosensor equilibration, rats were transferred to the operant chamber and allowed to respond on the action sequence for a total of 10 sucrose pellet rewards. After this initial re-familiarization period rats were then transferred back to the holding chamber and were either allowed to consume the sucrose pellets to satiety (for 1 h), or given no satiety treatment (1 h wait period), prior to being returned to the testing chamber. This sensory specific satiety procedure is well established to reduce the incentive value of food rewards, reflected by fewer seeking actions for the reward (Berridge, 1991; Dickinson et al., 1996; Balleine et al., 2005; Wassum et al., 2009a). Upon return to the testing chamber rats were allowed to respond on the sequence of actions again for up to 30 sucrose pellet rewards.
As glutamate can arise from release from both neurons and non-neuronal (i.e. glial) sources, it was imperative to assess the pool from which our glutamate concentration changes arose. Therefore, in a third groups of rats (N=4), we locally infused tetrodotoxin (TTX) into the region surrounding the glutamate biosensor to block action potential-dependent glutamate release. At surgery this group of rats was implanted with a glutamate biosensor into the BLA as above, however attached to these sensors was a 7 mm, 22 gauge guide cannula (Plastics One, Roanoke, VA) positioned such that when the 9 mm injector was inserted into the cannula the tip of the injector was ~50-100 μm away from the MEAs. At test, after equilibration rats were infused with sterile water vehicle (0.3 μl over 1 min). The injector was left in place for an additional minute before being removed. After a 10 min interval recording commenced while rats were in the recording chamber and lasted for 30 min. After this vehicle session rats were then infused with TTX (0.3 μl of 100 μM over 1 min, Tocris Bioscience, Ellisville, MI). Again after a 10 min injection-test interval the TTX recording period commenced. Although rats were freely-moving, no behavioral tasks were performed during this test based on our preliminary data suggesting that intra-BLA TTX treatment disrupts behavioral performance. The order of testing was not counterbalanced based on our preliminary data suggesting the effects of TTX on glutamate recordings to be long-lasting (i.e more than 24 h)
In a fourth group of rats (N=4), we further examined the origin of transient BLA glutamate concentration changes. Glutamatergic inputs to the BLA from the OFC are particularly strong and communication between these structures has been implicated in instrumental action performance. Therefore, we inactivated the ipsilateral OFC while monitoring transient glutamate concentration changes in the BLA of rats performing on the sequence of actions to earn sucrose pellet rewards. For this group, after training on the action sequence, rats were implanted with a pre-calibrated glutamate biosensor in the BLA along with an ipsilateral 22 gauge guide cannula projected towards the OFC (AP +3.5 mm, ML + 3.0, DV −3.4). After a 24 h recovery period the biosensor was tethered to the potentiostat, a potential of 0.7 V was applied and the signal was allowed to stabilize (approx. 45 min). Using a microinfusion pump, sterile water vehicle was infused into the OFC in a volume of 0.5 μl over 1 min via an injector inserted into the guide cannula fabricated to protrude 1 mm ventral to the tip. Injectors were left in place for at least 1 additional min to ensure full infusion. 15 min thereafter rats were placed in operant boxes and allowed to respond on the action sequence to earn 10 sucrose pellet rewards. After this session muscimol (1 μg/μl in sterile water vehicle; Tocris Bioscience) was infused into the OFC at a volume of 0.5 μl over 1 min, with an addition 1 min to ensure infusion. Unilateral infusions were intentionally used here to avoid impacting behavioral performance. Indeed, several reports have shown that lesions of the OFC lead to impulsive choice (Mar et al., 2011) and an inability to update behavior based on a change in reward value (Pickens et al., 2005). Again, 15 min elapsed post-muscimol infusion before rats were placed in the operant chamber and allowed to respond on the action sequence for a total of 10 sucrose pellet rewards. The following day rats were again tethered to the potentiostat and, after signal stabilization, received a second intra-OFC infusion of vehicle 15 min before being placed back in the operant box to respond on the action sequence for a total of 10 sucrose pellet rewards.
At the conclusion of each experiment rats were anesthetized with Nembutal and trans-cardially perfused with 0.9% saline followed by 10% formalin saline. The brains were removed and post-fixed in formalin, then cryosectioned into 60 μm slices, mounted onto slides and stained with cresyl violet. Light microscopy was used to examine sensor placement in the BLA and where applicable guide cannula placement. Histological data are presented Figure 2.
All data were processed with Microsoft Excel, then compiled and statistically analyzed, unless otherwise mentioned, with GraphPad Prism (San Diego, CA) and SPSS (Chicago, IL). In all cases the threshold for significance was set at p<0.05.
We developed a method of data analysis, derived from the techniques of background-subtracted fast-scan cyclic voltammetry (Michael et al., 1999) and glutamate biosensor self-referencing (Rutherford et al., 2007), wherein the current output from each electrode on the MEA was subtracted from the baseline current for that electrode, determined a 10 s average at the beginning of the 3 min baseline period prior to the beginning of the lever-pressing test. A representative example of this data output is shown in Figure 3 (the details of this figure are discussed later in the Results). We then exploited the utility of the control electrode and subtracted current change from baseline on the PPy/Nafion coated electrode from the current change from baseline on the PPy/Nafion/GluOx glutamate biosensor to obtain a low-noise change-from-baseline glutamate current signal (Figure 3C). This technique has the advantage of subtracting all current responses that result from anything other than enzymatically generated H2O2 oxidation from the glutamate response, thereby reducing the level of the noise and allowing the small glutamate concentration changes to be revealed. Without such noise reduction analysis glutamate transients are often barely detectable above the level of the noise.
Mini Analysis (Synaptosoft, Decatur, GA) was then used on this data output to determine the frequency, amplitude, half-width and decay time of the transient glutamate events. A fluctuation in the glutamate trace was deemed a glutamate transient if it was at least 2.5 times greater than the root-mean-square noise. To determine the transient amplitude a baseline was taken as the first minima located 0.5-2 s before the peak. Rise time to peak response was also calculated from this baseline point. Half-width was taken as the width of the peak at half the maximal peak amplitude, and decay time was calculated as the time it took for the current to drop from the peak to 37% of the peak amplitude (Figure 4A). Representative examples of individual transients taken from 4 different rats are shown in Figure 4. The majority of transients were approximately 2-5 s in duration; however as is seen in Figure 4D on rare occasions some transients lasted longer. A representative glutamate transient frequency and lever press histogram is presented in Figure 3D.
After the initial analysis for identification and characterization of BLA glutamate transients we split the data into 3 test phases for further analysis. The 3 min period prior the lever-pressing test session served as the baseline. Since there were long periods of time in which the rats were in the operant chamber and had the opportunity to, but were not engaged in the lever-pressing task we also divided the behavioral lever-pressing test into two separate phases: task-engaged and unengaged. Task-engaged was defined as the 10 s before or after any lever press event, while unengaged was any time during the lever-pressing test outside this window. These three test phases were used for all subsequent analyses.
Glutamate transient dynamics (amplitude, rise time, half-width and decay time) were averaged within each phase for each rat, and then averaged across rats. These data were analyzed with a one-way repeated measure ANOVA with ‘test phase’ as the variable; baseline, in the session but unengaged in the lever pressing task, or in the session and engaged in lever pressing (i.e. within ±10 s of a lever press event).
Averaged glutamate transient frequency data were analyzed with a one-way repeated measure ANOVA to examine the effects of test phase on the number of glutamate transients per minute. Post hoc analyses were then used to compare glutamate transient frequency during each lever-pressing test phase. After this a further level analysis was conducted to determine how lever-pressing activity was distributed around transients. We counted all the lever presses in the 10s prior to and after every transient and totaled this for each rat then averaged this across rats. These data were analyzed for normality (D’Agostino and Pearson Omnibus normality test), skewness and kurtosis.
As mentioned above, rats were not retrained on the lever-pressing task prior to test. Given the difference in behavioral output during the re-familiarization (first 5 cycles) and performance periods (last 25 cycles), glutamate transient frequency and amplitude were correlated with lever press rate for only the performance period of the lever-pressing test to avoid the behavioral confound of re-familiarization. Neither glutamate transient frequency (t6=1.33 p=0.23) nor amplitude (t6=0.25 p=0.81) was significantly different between the re-familiarization and performance phases. The average glutamate transient frequency and amplitude during the performance period of the lever pressing test (both task engaged and unengaged phases) for each rat were plotted as a function of press rate and these data were analyzed with a Pearson correlation.
The average glutamate transient frequency and amplitude for rats that underwent the specific satiety outcome devaluation procedure, together with their unsated controls, were analyzed with a 2-way ANOVA with ‘test phase’ as the within subjects variable and ‘satiety level’ as the between-subjects variable. Bonferroni post hoc analysis was used to compare the effects of satiety in each test phase. Similarly, the average glutamate transient frequency and amplitude for rats receiving infusions of vehicle and muscimol into the OFC were analyzed with a 2-way ANOVA with ‘test phase’ and ‘intra-OFC drug’ as the within subjects variables. Bonferroni post hoc tests were used to compare the effects of drug within each test phase. Glutamate transient frequency following local application of vehicle or TTX was analyzed with a paired t-test.
The behavioral data for rats receiving the specific satiety treatment were analyzed with a two-way ANOVA to compare the effects of the between-subjects variable ‘satiety level’ on lever press rate, with ‘lever’ as a within-subjects variable (either the distal or proximal response). Similarly, the data from the behavioral session in which rats were pre-treated with either intra-OFC vehicle or muscimol were also analyzed with a two-way ANOVA with within-subjects variables, ‘drug’ and ‘lever’.
In addition to this analysis of transient glutamate concentration changes across the entire test session we also cut the data into 20 s peri-press bins, reflective of the task-engaged phase of the lever-pressing session, to show the averaged transient glutamate concentration surrounding each lever press action. As above, the current output from each electrode on the MEA was subtracted from the baseline current for that electrode. For this analysis a local baseline was determined as the average of the 2 s prior to the first distal press in a sequence in order to specifically look at changes time-locked to lever pressing activity. The current change from baseline on the control electrode was then subtracted from the current change from baseline on the glutamate biosensor to get a low noise, interference-free glutamate trace in the 10 s prior to and after a lever press event. These traces were then averaged across trials for each rat and then across rats; these data are presented in Figure 3E and F.
The behavioral procedure required rats to perform a fixed sequence of two different lever press actions to earn sucrose pellets, such that one action was temporally distal and the other temporally proximal to reward delivery. The distal lever was continuously available and, when pressed, resulted in the insertion of the proximal lever into the chamber. Pressing the proximal lever resulted in the delivery of a sucrose pellet and caused that lever to be retracted. Importantly, this task was self-paced; the rat’s decision to engage in this action was not precipitated by an experimenter-delivered stimulus, rather the rat could control both the initiation of each ‘trial’ sequence as well as the speed with which the sequence of actions was performed. Rats were trained to asymptotic performance (5 days) on this sequence prior to biosensor implantation and test. All rats acquired and attained stable performance on the sequence of actions. At test, rats performed at an average response rate of 1.85 (SEM=0.39) presses/min on the distal lever and 1.21 (SEM=0.129) presses/minute on the proximal lever. Since rats were not retrained on the action sequence task after sensor implantation their initial responses may be considered a ‘re-familiarization phase’. In support of this, in the first 5 pressing cycles rats displayed a significantly longer latency (20.55 s, sem=4.94 s) to press the proximal lever after the distal lever had been pressed (releasing the proximal lever), relative to the remaining 25 ‘performance phase’ pressing cycles (6.05s, sem=0.72s; t6=2.97, p=0.02).
As can be seen in the representative example shown in Figure 3A and B, the background-subtracted (see Methods) baseline glutamate concentration was relatively stable over time during the lever pressing session (black line-glutamate biosensor output). However, while rats were performing on the sequence of actions to earn sucrose reward increases in current of relatively short duration were observed coincident with lever pressing activity (Figure 3B). Such transients were detected in the glutamate biosensor output, but not in the control electrode output (i.e. from the electrode that was not capable of detecting glutamate), and are therefore indicative of rapid glutamate concentration changes. These glutamate transients were further resolved when current changes from baseline on the control electrode were subtracted from current changes from baseline on the glutamate biosensing electrode to reduce the level of the noise (Figure 3C). The representative histogram in Figure 3D reveals that transient glutamate concentration change events (elevations in glutamate that were at least 2.5 times greater the root-mean-square noise) occurred both spontaneously and time-locked to lever pressing activity (see group analyses below).
Representative examples of individual transients taken from 4 different rats are shown in Figure 4. The vast majority of transients were approximately 2-5 s in duration (baseline-to-baseline) with rare examples of longer duration (Figure 4D). As can be seen in Table 1, the average rise time of glutamate transients was 0.9 s (standard error of the mean (SEM) =0.1), with a decay time of 0.6 s (0.008) and a half-width of 0.8 s (0.04). These transient characteristics did not differ with the test phase; there was no significant main effect of test phase on glutamate transient rise time (F2,20=0.15, p=0.86), decay time (F2,20=1.73, p=0.22) or half-width (F2,20=1.23, p=0.33). It is important to note that the temporal properties of these glutamate transients match closely the response time of the sensors in a flow cell (rise time: ~0.8 s) (Wassum et al., 2008). It is likely, therefore, that extracellular glutamate concentration is, in fact, changing more rapidly than these measurements suggest.
Examination of glutamate transient frequency across rats shows that glutamate transients occurred more when rats were engaged in the action sequence lever-pressing task. Analysis of transient frequency with respect to test session phase (pre-session baseline, in pressing session-unengaged, in pressing session-task engaged/within 10 s of a lever press) revealed a main effect of test phase (F2,20=16.12, p=0.0004). Post hoc analysis indicated that glutamate transient frequency was significantly elevated from baseline when rats were engaged in the task (p<0.001), but that during the lever-pressing session when rats were not engaged in lever pressing there was no significant change in glutamate transient frequency (p>0.05, Figure 5A). Indeed, within the lever pressing session the frequency of task-related glutamate transients was significantly higher than those falling outside the peri-press window (p<0.01, Figure 5A). Interestingly, the distribution of presses around glutamate transient events deviated significantly from normality (K2=7.59, p=0.03), with a positive skewness (1.29) and kurtosis (1.20), indicating that glutamate transients tended to closely precede lever-pressing activity (Figure 5B). However, it should be noted that 42% (SEM=11%) of proximal presses were followed immediately (within 5 s) by a glutamate transient, suggesting that these transients also occurred at the conclusion of the sequence when the sucrose pellet reward was consumed. (The tendency for transients to be diminished in frequency during the consumption phase in the example shown in Figure 3 was not typical).
These data are supported by analysis of the average glutamate concentration change in the task-engaged window (Figure 3E and F). As can see be seen from this figure, there is a clear transient increase in glutamate concentration from baseline preceding both the distal and proximal lever presses (Figure 3E and F). For these data a local baseline was established prior to each action sequence initiation (2 s average, 10 s prior to the distal press): Pre-event background-subtracted signals at the control electrode were subtracted from similarly background-subtracted signals at the glutamate oxidase-coated biosensor to reveal the rapid glutamate concentration changes time-locked to distal and proximal action performance. On average the largest glutamate transient peak occurred 2.4 s (SEM=0.2) prior to the distal action and 2.5 s (SEM=0.3) prior to the proximal action. The largest glutamate transient peak prior to the press was on average 6.8 μM (SEM=2.2) in amplitude prior to the distal press and 6.6 μM (SEM=1.7) prior to the proximal press. Despite time-averaging due to variable transient peak times, these data clearly show that transient glutamate concentration changes occurred when rats were engaged in lever-pressing. Statistical analysis of these changes revealed a main effect of time bin (baseline, pre-distal or pre-proximal; F1,3=7.91, p=0.006), with post hoc analyses confirming that the glutamate concentration peak prior to both the distal (p<0.01) and proximal (p<0.05) actions were significantly elevated above baseline levels. This finding, that glutamate concentration transients tended to precede lever presses, is notable for a self-paced task, in which performance, of the distal action especially, was not triggered by an experimenter-delivered cue or signal, and suggests that these transients may have occurred during the decision-making process prior to initiation of the action sequence. Importantly, however, that the transient glutamate elevations also preceded performance of the proximal action (the terminal link in the sequence) suggests that transient glutamate activity is not solely related to the initiating decision.
In support of this, not only did glutamate transient frequency increase with lever pressing activity, these factors were positively correlated across rats during the performance period of the test (i.e. after the initial 5 ‘re-familiarization’ response-reward cycles: see Table 2 for additional analysis), such that rats displaying high overall glutamate transient frequency (during the entire lever-pressing test both engaged and unengaged phases) performed at higher levels of lever pressing (Figure 5C, R2=0.57, p=0.05). Glutamate transient frequency was not correlated with other measurable reward-related behavioral outputs, such as the average time it took rats to complete the action sequence (Table 2).
Although the amplitude of the glutamate transients was not significantly higher when rats were engaged in the lever pressing task relative to unengaged or baseline phases (Figure 6A, F2,20=1.89, p=0.19), overall BLA glutamate transient amplitude during the performance phase of the test (both engaged and unengaged phases) was significantly positively correlated with lever-pressing activity during that phase (R2=0.85, p=0.003; Figure 6B). Glutamate transient amplitude was not correlated with other measurable behavioral outputs, such as the average time it took rats to complete the action sequence (Table 2). That the amplitude of glutamate transients, which often preceded lever press actions (Figure 5B), predicted the rate with which those actions were performed is supportive of the hypothesis that BLA glutamate transients are related to the decision-making and initiating processes that occur prior to lever press actions.
To further test the hypothesis that transient glutamate signaling in the BLA is a correlate of instrumental action performance and decision-making, in a second set of rats we experimentally manipulated the value of the sucrose reward, information that is used to inform decisions, with a specific satiety reward devaluation procedure. We assessed the effects of this manipulation on both lever-pressing activity and transient BLA glutamate concentration changes. After an initial 10 cycle re-familiarization session following sensor implantation and recovery, rats, while in the holding chamber, were either allowed to consume the sucrose pellets to satiety, or given no satiety treatment immediately prior to being returned to the testing chamber. Such satiety treatment is well established to produce both general motivational-induced and sensory specific reward-value-based reductions in lever-pressing (Berridge, 1991; Dickinson et al., 1996; Balleine et al., 2005; Wassum et al., 2009a), an effect which requires an intact BLA (Balleine et al., 2003). Accordingly, rats that received the specific satiety treatment showed significantly lower rates of responding on both levers (main effect of satiety: F1,6=6.68, p=0.04, no effect of lever: F1,6=3.81, p=0.09 or interaction: F1,6=1.17, p=0.32; Figure 7A).
Interestingly, the reduction in lever pressing activity brought about by specific satiety was accompanied by a decrease in glutamate transient frequency; an effect that was specific to task-related glutamate transients (Figure 7B). Statistical analysis of these data reveals marginally insignificant overall main effects of satiety (F1,12=1.25, p=0.09) and test phase (prior to the session, in the session - outside the 20 s peri-press window, or in the session - task-engaged, F2,12=2.70, p=0.11) on glutamate transient frequency. Importantly, there was a significant interaction between these two factors (F2,12=3.92, p=0.04). Post hoc analyses clarify this interaction to show that glutamate transient frequency was significantly lower in rats that received the specific satiety treatment relative to unsated controls only when the rats were engaged in the task (p<0.05). Indeed, after the specific satiety treatment rats showed a significantly lower (0.17 sem=0.05) transient/press ratio than unsated controls (0.60 sem=0.1; t6=2.45, p=0.05).
In addition, the amplitude of such transients was also attenuated. Analysis of the data shown in Figure 7C reveals a main effect of satiety on average glutamate transient amplitude (F1,12=6.56, p=0.04) with no effect of test phase (F2,12=1.98, p=0.18)), or interaction between these factors (F2,12=1.21, p=0.33), suggesting that, unlike the effect on glutamate transient frequency, specific satiety reduced the amplitude of all transient glutamate concentration changes. Thus, BLA glutamate transients related to instrumental activity varied in frequency and amplitude with changes in lever-pressing activity induced by a reduction in motivational state.
As glutamate can arise from release from both neurons and non-neuronal (i.e. glial) sources, it was imperative to assess the pool from which our glutamate concentration changes arose. Using glutamate biosensors similar to those employed in our work, others have shown glutamate concentration changes to be, at least in part synaptic in origin (Belay et al., 1999; Day et al., 2006; Rutherford et al., 2007; van der Zeyden et al., 2008; Hascup et al., 2010). We suspected that the transient glutamate concentration changes we detect in the BLA were similarly the result of neuronal activity. To test this hypothesis we locally infused TTX (0.3 μl of 100 μM) into the region surrounding the glutamate biosensor to block action potential-dependent glutamate release. In preliminary experiments this completely blocked lever-pressing performance; we therefore focused on baseline, spontaneous glutamate transient activity in freely-moving rats not engaged in lever pressing. The frequency of spontaneous glutamate transients was significantly reduced by TTX (Figure 8, t4=4.23, p=0.02), suggesting that they are likely the result of neuronal release.
To further examine the origin of the glutamate transients we next considered the possible glutamatergic neuronal inputs to the BLA that might contribute to these signals. There are known glutamatergic projections to the BLA from the hippocampus, thalamus and cortex. Of the cortical inputs, those from the orbital frontal region are particularly strong and reciprocal in nature. Indeed, an interaction between the OFC and BLA has been suggested necessary for goal-directed behavior (Pickens et al., 2003; Schoenbaum et al., 2003; Holland and Gallagher, 2004). Therefore, in a separate set of rats, we inactivated the OFC with ipsilateral infusions of muscimol (0.5 μl of 1 μg/μl) while monitoring transient glutamate concentration changes in the BLA of rats performing on the sequence of actions to earn sucrose pellet rewards. Infusion of muscimol to inactivate the OFC did not impact rats’ lever pressing activity (Figure 9A), most likely because unilateral infusions were intentionally employed; statistical analysis showed neither a main effect of drug (F1,3=1.14, p=0.36), nor an interaction between the effects of drug and lever (F1,3=0.29, p=0.62) on lever press rates. Importantly, between the two vehicle tests, there was no overall main effect of test on either lever press rates (F1,3=0.39, p=0.58) or glutamate transient frequency (F2,6=1.35, p=0.33), therefore these tests were collapsed for all further analyses.
As is clear from Figure 9B, OFC inactivation did, however, significantly attenuate glutamate transients, but only when rats were engaged in the lever press sequence. Statistical analysis of these data reveals a marginal effect of drug (F1,6=7.52, p=0.07) and test phase (F2,6=4.45, p=0.07), but most importantly a drug by test phase interaction (F2,6=6.05, p=0.03). Post hoc analyses clarify this interaction to suggest that OFC muscimol infusions decreased glutamate transient frequency only when rats were engaged in the lever press sequence (p<0.01). Indeed, muscimol infusion completely blocked (F2,6=2.60, p=0.15) the increase in transient frequency observed in the vehicle-treated condition when rats were engaged in the lever press sequence, relative to the baseline period, (F2,6=5.11, p=0.05). These data indicate that glutamate transients in the BLA that were related to the performance of instrumental actions likely arose from release from neurons with their cell bodies in the OFC.
While OFC inactivation altered BLA glutamate transient frequency, it did not affect the properties of the glutamate transients that did reach threshold, including glutamate transient amplitude; there was no overall main effect of drug on glutamate transient amplitude (F1,6=1.03, p=0.39), no effect of session phase (F2,6=1.98, p=0.22) and no interaction between these factors (F2,6=0.53, p=0.54, Figure 9C). As can be seen in Figure 10, the temporal dynamics of glutamate transients were also unaffected by OFC inactivation. Analysis of glutamate transient rise time revealed no main effect drug (F1,3=0.55, p=0.51), test phase (F2,2=0.13, p=0.89), or interaction between these variables (F2,2=0.79, p=0.56). Similarly, there was no main effect of drug (F1,3=2.02, p=0.25), test phase (F2,2=0.88, p=0.53), or interaction between these variables (F2,2=0.13, p=0.90) on BLA glutamate transient half-width. Lastly, no effect of drug (F1,3=0.70, p=0.47), test phase (F2,2=0.90, p=0.53), or interaction between these variables (F2,2=0.69, p=0.59) was identified for glutamate transient decay time.
Glutamate biosensors were used to identify rapid transient extracellular glutamate fluctuations in the BLA during performance of an instrumental task. The frequency of spontaneous glutamate transients in the BLA increased when rats were engaged in a self-paced lever-press sequence to earn sucrose rewards and decreased when the value of the earned reward was reduced by satiety. Glutamate transients tended to precede lever press actions and lever press rate was positively correlated with both transient frequency and amplitude. Lastly, these glutamate transients were action potential-dependent and appeared to originate from projections of neurons located in the OFC.
Escape of glutamate from the synapse is tightly regulated by reuptake transporters predominantly located on glia (Bergles et al., 1999). Indeed, the insensitivity of microdialysis extracellular glutamate measures to TTX and to calcium removal (Miele et al., 1996; Timmerman and Westerink, 1997) indicates that much of the tonic extracellular glutamate originates from non-vesicular release, such as via cysteine/glutamate exchange (Baker et al., 2002). However, reports describing electroenzymatic biosensor measures of tonic glutamate have demonstrated at least a degree of TTX and calcium sensitivity (Belay et al., 1999; Day et al., 2006; Rutherford et al., 2007; van der Zeyden et al., 2008) and have shown that tail pinch stressor-induced increases in glutamate concentration are completely abolished by TTX (Hascup et al., 2010). Importantly, the rapid BLA glutamate transients we observed here were significantly attenuated by TTX and by inhibition of a major cortical glutamatergic input to this region. Together, these data point to a vesicular origin, and corroborate electrophysiological evidence of synaptic overspill of glutamate (Bergles et al., 1999; Diamond, 2002).
In the context of extracellular neurochemical recordings, these are quite rapid events, with durations of the order of 2-5 s, comparable to transient dopamine events recorded in the striatum with fast-scan cyclic voltammetry (Wightman et al., 2007) and found to be associated with reward-related behavior (Phillips et al., 2003; Wanat et al., 2009). However, the kinetics of the BLA glutamate transients we observed must be considered in light of the response characteristics of the biosensors. Unlike detection of dopamine, which is essentially limited by diffusion through the extracellular milieu and active reuptake, glutamate detection is additionally impacted by enzyme kinetics and the multiple polymeric layers on the electrode surface. As previously reported (Wassum et al., 2008), the response time of our sensors is approximately 0.8s in vitro. The transients we observe in vivo occur over a comparable timeframe and are therefore potentially faster in reality, being time-averaged by the response characteristics of the sensor. In this context, it is notable that the transient events we describe here are considerably faster than the spontaneous baseline fluctuations (~13-18 s duration) in extracellular glutamate recently recorded in the prefrontal cortex using similar sensors, which were not correlated with any behavioral event (Hascup et al., 2011).
The amplitude of the events recorded here, estimated on the basis of in vitro calibration data to be in the 1-20 μM range, is sufficient to activate extrasynaptic group 2 metabotrobic glutamate receptors, which are considered to mediate presynaptic inhibition following synaptic overflow and thereby provide a negative feedback mechanism (Scanziani et al., 1997). These concentrations are also adequate to activate synaptic NMDA receptors and may therefore reflect volume glutamate transmission, recently proposed on the basis of electrophysiological and fluorescent reporter measurements (Szapiro and Barbour, 2007; Hires et al., 2008). Moreover, these concentration estimates may be conservative, given that in vitro calibration does not take into consideration the potential impact of the extracellular volume fraction and tortuosity in vivo (Nicholson and Syková, 1998).
Whether these BLA transients reflect time-averaged “spillout” of synaptic transmission, “spillover” volume transmission, or a combination of the two, their relationship to instrumental behavior, and potentially to the decision-making process itself, is supported by our observations. Not only did the frequency of transients increase during performance of the task, but both the frequency and amplitude of glutamate transients, which often preceded lever-presses, positively correlated with response rate. Neither BLA glutamate transient frequency nor amplitude correlated with other measurable behavioral outputs, such as the average time it took rats to complete the action sequence; thus glutamate transients did not simply track general reward-related behavioral output or motor activity. Moreover, that these correlations were only apparent after the ‘re-familiarization’ phase, i.e. after an opportunity to be ‘reminded’ of the action-outcome association and therefore be able to use goal value to determine their response rate, suggests that BLA glutamate transient frequency was specifically related to reward seeking action performance rather than to the general motor activation necessary for lever pressing. That glutamate transients occurred prior to both the distal and proximal actions in the sequence suggests that they may not be uniquely involved in the decision to initiate the action sequence. Future experiments will explore whether transients associated with the two levers are regulated by different behavioral processes (i.e. incentive value, Pavlovian cues, etc.).
The above considerations notwithstanding, the fact that glutamate transients fluctuated with motivational state, and therefore reward value, and tended to precede lever presses (including those that were self-initiated) supports the notion that these signals were related to the decision-making process that guides instrumental action performance. Indeed, the BLA has previously been implicated in goal-directed behavior and reward-related decision-making (Balleine et al., 2003; Wellman et al., 2005; Coutureau et al., 2009; Jenison et al., 2011). Our finding that those glutamate concentration transients in the BLA that were specifically related to instrumental actions were diminished by unilateral OFC inactivation may be considered to support a series of reports suggesting that an interaction between the OFC and the BLA is necessary for establishing the value of a reward, and for use of this value to update action performance (Schoenbaum et al., 1998; Schoenbaum et al., 2003; Holland and Gallagher, 2004). However, it has also been suggested that rather than being important for establishing and using incentive value to guide goal-directed actions, the OFC is necessary for the capacity of reward-paired cues, and presumably contexts, to provide predictive information about potential rewarding outcomes (Ostlund and Balleine, 2007a, b). Indeed, many of the aforementioned studies suggesting a role for OFC-BLA interactions in establishing reward value could in fact be reinterpreted as evidence for a role of OFC-BLA interactions in stimulus-outcome learning (Ostlund and Balleine, 2007a, b). Although the two-lever sequence task used in these experiments does not involve explicitly-presented reward-paired cues, its performance is undoubtedly guided in part by subtle contextual cues and discriminative stimuli, (i.e. insertion of the proximal lever) and likely also by internal signals within the animal. It is therefore alternatively possible, if not likely, that transient glutamate events in the BLA were related to these cues and their ability to facilitate the decision to engage the action sequence and that OFC inactivation reduced these predictive signals. These alternative possibilities are being specifically tested in ongoing experiments.
Those BLA glutamate transients that were not locked to lever-pressing activity may have arisen from a number of other known glutamatergic BLA afferents, including the thalamus, hippocampus and other cortical regions, or could be result of release from intrinsic neurons. Future examination of the interplay between these BLA inputs may reveal a more complex circuitry underlying BLA-dependent behaviors, including instrumental behavior.
In summary, an investigation coupling a novel biosensor technology to a self-paced instrumental task revealed rapid transient fluctuations in extracellular glutamate in the BLA that were time-locked to the performance of actions instrumental to gaining rewards, the frequency and amplitude of which were correlated with response rate. Future study of the neuropharmacological modulation of these signals may elucidate the complex BLA neurochemical interactions that underlie reward seeking. Moreover, given the recent suggestion that altered glutamate homeostasis may be critical for long-term drug seeking behavior (Reissner and Kalivas, 2010), changes in transient BLA glutamate signals arising from administration of abused substances may shed further light on the mechanisms of addiction. More broadly, the ability to monitor discrete behaviorally-induced glutamate concentration transients promises to facilitate elucidation of the role of this neurotransmitter in numerous other brain functions and disorders.
This research was supported by NIDA grants DA09359 and DA05010, and NINDS grant NS38367 to NTM, NS064547 from NINDS to HGM, an ARC Laureate Fellowship to BWB, and grant DA023774 from NIDA, Hatos scholarship and UC BREP GREAT Fellowship to KMW. The authors would like to thank Dr. Sean Ostlund for his helpful comments during manuscript preparation.