PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Neurosci. Author manuscript; available in PMC 2012 February 3.
Published in final edited form as:
PMCID: PMC3156031
NIHMSID: NIHMS315876

The Contribution of NMDAR Signaling in the Cortico-Basal Ganglia Reward Network to Appetitive Pavlovian Learning

Abstract

N-methyl-D-aspartate type glutamate receptors (NMDARs) contribute to phasic transmission and synaptic plasticity and are thought to be important for learning. To better understand where NMDAR signalling is necessary for learning, we combined viral-genetic strategies with genetic mouse models to investigate the contribution of NMDARs in the dopamine system to appetitive Pavlovian conditioning. NMDAR signaling in dopamine neurons was not required for Pavlovian conditioning; however, NMDARs in D1 dopamine receptor (D1R)-expressing medium spiny neurons (MSNs) which receive input from dopamine neurons were critical for this type of learning. NMDAR signaling was also required in brain regions that project to dopamine neurons, since removing NMDARs from afferent neurons to the ventral tegmental area (VTA) also prevented learning. This effect was likely due to loss of NMDAR signaling in the neurons of the prefrontal cortex (PFC), as learning could be restored in these animals by rescuing NMDAR expression in the PFC. Moreover, removing NMDARs exclusively from the PFC also prevented learning. Our findings suggest that NMDARs in neurons that project to and receive projections from the VTA are necessary for Pavlovian conditioning and specifically implicate the PFC and D1R-expressing MSNs in associative learning.

Keywords: Reinforcement Learning, NMDA, Prefrontal Cortex, Medium Spiny Neurons, D1-Receptor

Introduction

Repeated neuronal activation leads to alterations in synaptic strength that are widely accepted to represent a neuronal correlate of learning (Bliss and Lømo, 1973; Whitlock et al., 2006). Because NMDARs contribute to this process, these receptors are directly implicated in many forms of learning (Malenka and Bear, 2004). Appetitive Pavlovian conditioning is a simple form of learning in which a conditioned stimulus (CS) is repeatedly paired with an unconditioned stimulus (US) such as food. After multiple CS-US pairings, the CS becomes predictive of US delivery and elicits a conditioned response (CR). During appetitive Pavlovian conditioning in mice, a CS (e.g, a lever) is repeatedly paired with food delivery (US), and the CR is an increase in CS-elicited anticipatory behavior, commonly referred to as Pavlovian-conditioned approach (CA) (Kheirbek et al., 2008; Parker et al., 2010). CA reflects the predictive and motivational properties of the CS and serves as a behavioral measure of learning.

The most-implicated neurotransmitter in appetitive Pavlovian learning is dopamine. Rapid and phasic dopamine activity occurs in response to the US and emerges in response to the CS as learning occurs (Parker et al., 2010; Pan et al., 2005; Day et al., 2007). Intact dopamine signaling is necessary for CA in rodents (Parkinson et al., 2002), and evidence suggests that NMDARs in the dopamine system are also important (Di Ciano et al., 2001; Dalley et al., 2005; Stuber et al., 2008). We have previously demonstrated that NMDARs within dopamine neurons are not necessary for the emergence of either CS-elicited phasic dopamine release or CA in mice (Parker et al., 2010). We conclude from these findings that if NMDAR signaling within the dopamine system is critical for this type of learning, then it must occur within brain regions that project to dopamine neurons, receive dopaminergic innervation, or both.

Midbrain dopamine neurons receive input from many brain regions and send projections to many more (Geisler et al., 2007; Björklund and Dunnett, 2007). The VTA receives excitatory input from the PFC, the lateral hypothalamus, and the pedunculopontine and laterodorsal tegmental nuclei among others (Geisler et al., 2007). Which of these projections modulate dopamine neuron activity during appetitive Pavlovian learning and whether NMDARs within these afferent brain regions contribute to the emergence of CA is not known. The output of the VTA is divergent and includes projections to the ventral striatum, PFC, and amygdala (Lammel et al., 2008). While the ventral striatum is specifically thought to be important for appetitive Pavlovian conditioning (Parkinson et al., 2002; Di Ciano et al., 2001; Parkinson et al., 2000; Cardinal et al., 2002a), the other output structures of the VTA are also implicated in associative learning (Cardinal et al., 2002b). Moreover, within these output structures there are many different types of neurons (Kreitzer, 2009), and which populations require NMDARs for appetitive Pavlovian conditioning is not known.

Using viral genetic strategies and genetic mouse models, we characterized the requirement of NMDAR signaling in brain regions that project to and receive projections from the VTA during Pavlovian conditioning. By testing this requirement in specific neuronal subtypes, our findings elucidate the circuitry within the dopamine system that underlies associative learning.

Methods

Animals

All mouse lines used in these experiments were extensively backcrossed to C57BL/6 mice. Both male and female mice were used in all experiments, and all mice were between 8 and 10 weeks of age at the start of experiments. All animal protocols were approved by the University of Washington Institutional Animal Care and Use Committee.

DAT-Cre; NR1-KO mice: Mice with a conditional allele for Grin1 (Tsien et al., 1996) and Slc6a3Cre/+ mice (Zhuang et al., 2005) were crossed to generate Slc6a3Cre/+; Grin1Δ/lox (KO mice) and Slc6a3Cre/+; Grin1lox/+ (controls) mice as described (Zweifel et al., 2008).

GPR88-Cre; NR1-KO mice: Cre-GFP was targeted to the first coding exon of Gpr88 locus and the frt-flanked neomycin-resistance gene that was used for positive selection was removed by crossing these mice with Gt(ROSA)26Sor mice expressing Flp-recombinase. Gpr88Cre/+;Grin1Δ/+ males were crossed to Grin1lox/lox females to generate Gpr88Cre/+;Grin1Δ/lox KO and Gpr88Cre/+;Grin1Δ/l+ control mice as described (Beutler et al., 2011).

D1-Cre; NR1-KO mice: Drd1aCre/+;Grin1Δ/+ males were crossed to Drd1a+/+;Grin1lox/lox females to generate Drd1aCre/+;Grin1Δ/lox KO and Drd1aCre/+;Grin1lox/+ control mice as described (Beutler et al., 2011).

Virus-injected mice: Grin1lox/lox and Grin1lox/+ animals used in viral experiments were generated by crossing Grin1lox/+ and Grin1lox/lox animals. All animals injected with virus carried at least one copy of a Cre-activated YFP reporter mouse, Gt(Rosa)26Sor-flox-stop-YFP (Srinivas et al., 2001).

Viral Injections

For virus-mediated knockout of Grin1 from VTA afferents, a retrogradely-transported CAV-Cre vector was used (Soudais et al., 2004). The virus contains a myc-tagged Cre gene with a nuclear localization signal (NLS), expression of which is driven by the cytomegalovirus (CMV) promoter. For virus-mediated rescue of Grin1 in the PFC of animals injected with CAV-Cre in the VTA, an AAV-fsNR1 virus was used (Beutler et al., 2011). This virus contains a hemagglutinin (HA)-tagged NMDAR1-3a splice variant (Zukin and Bennett, 1995) preceded by a floxed SV40 late polyA-addition site and 7 in-frame termination codons. Expression of Grin1 is driven by the cytomegalovirus-chicken β-actin (CBA) promoter after Cre-mediated recombination. For virus-mediated removal of Grin1 from the PFC, an AAV-Cre vector was used (Beutler et al., 2011). This virus contains a myc-tagged Cre-GFP fusion gene with a 5′ NLS, whose expression is driven by CBA promoter.

To remove Grin1 from VTA afferents, Grin1lox/lox and Grin1lox/+ animals were injected bilaterally with 0.5 µl of CAV-Cre titered at 2.5 × 109 viral genomes/µl at the coordinates (ML= ±0.5 mm; DV= −4.5 mm) from bregma. The AP coordinate was corrected for the size of the animal according to the following formula: AP= (lambda to bregma distance)/4.21 × −3.5) mm. To restore Grin1 to VTA-projecting PFC neurons of animals injected with CAV-Cre in the VTA, Grin1lox/lox and Grin1lox/+ animals were injected bilaterally with 0.5 µl of AAV-fsNR1 titered at 6.0 × 109 viral genomes/µl at the coordinates (ML= ±0.27 mm; DV= −2.35 mm) from bregma. The AP coordinate was corrected for the size of the animal according to the following formula: AP= (lambda to bregma distance)/4.21 × 1.9) mm. To remove Grin1 from the PFC, Grin1lox/lox and Grin1lox/+ animals were injected bilaterally with 0.5 µl of AAV-Cre at the same coordinates. All animals were allowed two weeks to recover before behavioral testing began.

Appetitive Pavlovian Conditioning

All training was done in operant conditioning chambers (ENV-307W, Med Associates, Inc.). All mice were first trained to retrieve food pellets in a single magazine training session in which 10 pellets were delivered at a variable intertrial interval (ITI) of 90 s. After the mice had been trained to retrieve food pellets, they received five sessions of Pavlovian conditioning which consisted of 25 CS-US pairings in which a 10-s lever presentation was immediately followed by the delivery of a food reward. Importantly, the mice were not required to press the lever to obtain the food reward, and did not interact with the lever in this paradigm. This training procedure differed slightly from our previous study in that CS presentation terminated just prior to instead of just after US delivery (Parker et al., 2010). Head entries to the food receptacle were detected by the disruption of an infrared photobeam. Learning was measured as the increase in CS-elicited head entry rate (CS-HE rate) to the food receptacle relative to baseline (ITI-HE rate). Conditioned approach (CA) was calculated as the difference between CS-HE and ITI-HE rates for each mouse.

Immunohistochemistry

After behavioral experiments, all stereotactically injected animals were euthanized and perfused as described (Hnasko et al., 2006). Briefly, animals were given a lethal dose of Beuthenasia (Schering-Plough, Whitehouse Station, NJ) and transcardially perfused with phosphate-buffered saline (PBS) followed by 4% paraformaldehyde (PFA) in PBS. Brains were removed, postfixed in PFA overnight, cryoprotected in 30% sucrose, and frozen. Free-floating coronal sections (30 µm) were immunostained. For animals injected with CAV-Cre in the VTA, viral targeting and expression in the VTA and PFC was confirmed by immunostaining with a mouse anti-tyrosine hydroxylase (TH) antibody (1:1000, Millipore) and a rabbit anti-GFP antibody (1:1000, Invitrogen) to confirm expression of the Cre-dependent YFP reporter. Cy3- and Cy2-conjugated secondary antibodies were used (1:200, Jackson ImmunoResearch). For animals injected with AAV1-fsNR1 in the PFC, viral targeting and expression in the PFC was confirmed by immunostaining with a mouse anti-HA antibody (1:1000, Applied Biological Materials) and rabbit anti-GFP antibody as above. For animals injected with AAV-Cre, viral targeting and expression was confirmed in the PFC by immunostaining with rabbit anti-GFP. Images were acquired on a Nikon Eclipse E600 microscope.

Statistics

All statistical comparisons (two-way ANOVA) and all graphical representations were generated using GraphPad Prism. In all cases, statistical interactions (CS vs. ITI or Genotype × Day) are reported. Statistical values are not included in the figures when comparisons were not significant.

Results

NMDAR signaling in dopamine neurons is not required for appetitive Pavlovian conditioning

Although NMDAR signaling contributes to the burst-firing of dopamine neurons (Zweifel et al., 2009) and phasic dopamine release that occurs during learning, we have previously demonstrated that NMDARs in dopamine neurons are not required for the development of CS-elicited dopamine release or CA (Parker et al., 2010). To validate this experiment, we generated mice lacking NMDARs in dopamine neurons by crossing mice with a conditional allele encoding the essential NR1 subunit of the NMDAR with mice expressing Cre-recombinase (Cre) from the dopamine transporter gene (Slc6a3) locus (DAT-Cre; NR1-KO mice) (Fig. 1A). We used a conditioning paradigm that was slightly modified from our previous study (see methods) and more similar to those used by other labs (Day et al., 2007; Flagel et al., 2011). As before, learning was measured as an increase in CS-elicited head-entry (CS-HE) rate to the food receptacle relative to basal inter-trial interval head-entry (ITI-HE) rate. This pattern of responding has been termed “goal-tracking” (as opposed to “sign-tracking”) and thought to reflect the learned predictive value of the CS (Flagel et al., 2011). This distinction is noteworthy because these different response patterns are thought to reflect different learning strategies driven by divergent neural substrates (Flagel et al., 2011). Nonetheless, both sign-tracking and goal-tracking responses are valid behavioral metrics for monitoring the process of learning. Despite using a slightly modified training procedure, we again found that both DAT-Cre; NR1-KO and control mice increased their CS-HE rate relative to their basal ITI-HE rate (Fig. 1B,C; Data represent mean ± S.E.M; all statistical comparisons and subject numbers are reported in the figure legends). This similar pattern of responding to the CS resulted in equivalent increases in their CA scores during training (Fig. 1D). These observations confirm that NMDAR signaling within dopamine neurons is not required for learning in this paradigm.

Figure 1
Mice lacking NMDARs in dopamine neurons learn normally during Pavlovian conditioning. A, Gray dots in the midbrain indicate that only the dopamine neurons in this brain region lack NMDARs in DAT-Cre; NR1-KO mice. B, Control mice (n=11) increase their ...

NMDARs in striatal MSNs and D1R-expressing neurons are necessary for appetitive Pavlovian conditioning

Because NMDARs are not required in dopamine neurons, we predicted that NMDAR signaling might be important in brain regions that are modulated by dopamine release. During Pavlovian conditioning, neurons from the VTA release dopamine in the ventral striatum which modulates glutamate transmission in this brain region to generate appropriate action sequences in response to environmental cues (Redgrave et al., 1999). Evidence suggests that NMDAR signaling in the ventral striatum is important for the development of cue-dependent anticipatory behavior during Pavlovian conditioning (Di Ciano et al., 2001). However, there are many types of neurons within the striatum18 and which populations specifically require NMDARs for this behavior is not known. To assess the specific contribution of NMDAR signaling within MSNs to the development CA, we generated mice lacking NMDARs in all MSNs by crossing mice with a conditional Grin1 allele to mice expressing Cre from the Gpr88 gene locus (GPR88-Cre; NR1-KO mice) (Fig. 2A). GPR88 expression has been shown to be specific to D1- and D2-expressing striatal MSNs, with no detectable expression in other striatal cell types (Massart et al., 2009), and striatal NR1 expression is significantly reduced in GPR88-Cre; NR1-KO mice (Beutler et al., 2011).

Figure 2
Pavlovian conditioning is disrupted in mice lacking NMDARs in striatal MSNs and D1R-expressing neurons. A, Large gray dots indicate the brain region lacking NMDARs (striatal MSNs) in GPR88-Cre; NR1-KO mice. C, Control mice (n=9) increased their CS-HE ...

Unlike controls, GPR88-Cre; NR1-KO mice did not develop an increase in CS-HE rate relative to their ITI-HE rate (Fig. 2B,C). This observation indicates that GPR88-Cre; NR1-KO did not learn the predictive value of the CS as there was no increase in their CA score (Fig. 2D), and that NMDAR signaling, specifically in MSNs, is critical for Pavlovian conditioning.

NMDAR-dependent signaling and long-term potentiation in striatal neurons is modulated by D1Rs (Levine et al., 1996; Kerr and Wickens, 2001), and the phasic dopamine activity that occurs during Pavlovian conditioning is believed to specifically engage the lower-affinity D1Rs (Richfield et al., 1989; Hong and Hikosaka, 2011). Consistent with D1R involvement in Pavlovian conditioning, D1R-KO mice or wild-type mice given a D1R antagonist do not develop CA during training (Parker et al., 2010). Because D1R signaling is necessary for associative learning and because NMDAR signaling within striatal MSNs is critical for Pavlovian conditioning, we hypothesized that NMDAR signaling specifically within D1R-expressing cells would be important for Pavlovian conditioning. To test this prediction, we created mice selectively lacking NMDARs in D1R-expressing neurons by breeding our mice with a conditional Grin1 allele to mice expressing Cre from the Drd1a gene locus (D1-Cre; NR1-KO mice) (Fig. 2E). Similar to the GPR88-Cre; NR1-KO mice, D1-Cre; NR1-KO mice also did not develop CA when compared to controls (Fig. 2F–H). This observation suggests that NMDAR signaling specifically in D1R-expressing neurons is critical for Pavlovian conditioning, and that NMDARs in D2R-expressing MSNs alone is not sufficient for this type of learning. The combined results support the hypothesis that D1R-expressing MSNs integrate dopamine and glutamate via D1R and NMDAR signaling (Levine et al., 1996; Kerr and Wickens, 2001), respectively, to facilitate learning during Pavlovian conditioning.

Removing NMDARs from the VTA and its neuronal inputs prevents appetitive Pavlovian conditioning

CS-elicited dopamine release increases during Pavlovian conditioning in mice lacking NMDARs in dopamine neurons (Parker et al., 2010). Therefore, we predicted that NMDAR signaling within brain regions that modulate VTA dopamine neuron activity might also be important for this type of learning. To test this hypothesis we bilaterally injected canine adenovirus type 2 expressing Cre (CAV-Cre) into the VTA of mice with a conditional Grin1 allele (CAV-VTA; NR1-KO mice). CAV-Cre has previously been shown to preferentially transduce neurons and to be efficiently trafficked via retrograde axonal transport (Soudais et al., 2004; Hnasko et al., 2006). Therefore, we predicted that this strategy would lead to Cre expression and NMDAR inactivation in the VTA as well as in brain regions that project to the VTA (Fig. 3A). As indicated by the expression of a YFP reporter gene in both tyrosine hydoxylase (TH)-immunopositive and TH-immunonegative neurons, we observed Cre-mediated recombination in both dopamine and non-dopamine neurons of the VTA, respectively (Fig. 3B,C). To confirm efficient retrograde transport of CAV-Cre, we examined YFP expression in brain regions that are known to project to the VTA. We observed robust YFP expression within the pre- and infra-limbic regions of PFC (Fig. 3D), a finding that is consistent with the selective transduction of VTA-projecting PFC neurons (Geisler et al., 2007; Carr and Sesack, 2000).

Figure 3
Mice lacking NMDARs in the VTA as well as in VTA-projecting brain regions failed to learn during Pavlovian conditioning. A, Viral injection strategy for CAV-VTA; NR1-KO mice. Solid-green shading indicates that most cells within the VTA lack NMDARs and ...

When we tested CAV-VTA; NR1-KO mice in our Pavlovian conditioning paradigm, we found that the acquisition of CA was significantly attenuated in these animals (Fig. 3E–G). These results suggest that although NMDARs in dopamine neurons are not required for Pavlovian conditioning, intact NMDAR signaling in neurons that project to VTA dopamine neurons is critical for this type of learning.

NMDARs in the PFC are necessary for appetitive Pavlovian conditioning

If NMDAR signaling within neurons that project to the VTA is critical for learning, we predicted that removing these receptors from one of these input brain regions would disrupt Pavlovian conditioning. To test this prediction we designed a viral injection strategy to bilaterally remove NMDAR signaling from the pre-limbic region of the PFC. We selected this brain region because it sends a dense glutamatergic projection to the VTA (Geisler et al., 2007), and because it had robust CAV-Cre transduction in CAV-VTA; NR1-KO mice. Moreover, stimulation of the PFC activates VTA neurons and causes dopamine release in the ventral striatum, an effect that depends upon the VTA (Karreman and Moghaddam, 1996; Moorman and Aston-Jones, 2010).

To selectively remove NMDARs from the PFC, we injected adeno-associated virus (AAV1) expressing Cre (AAV-Cre) into the PFC of mice with a conditional Grin1 allele to generate AAV-PFC; NR1-KO mice (Fig. 4A). Cre-mediated recombination in the PFC of AAV-PFC; NR1-KO mice was evident by cytosolic YFP expression (Fig. 4B,C). In agreement with a contribution of NMDAR signaling in the PFC to Pavlovian conditioning, AAV-PFC; NR1-KO mice did not develop a CA response (Fig. 4D–F). These results indicate that NMDAR signaling in PFC neurons during Pavlovian conditioning is necessary for learning in this paradigm.

Figure 4
Mice lacking NMDARs in the PFC do not learn during Pavlovian conditioning. A, Viral injection strategy for generating AAV-PFC; NR1-KO mice. Solid-green shading indicates that most cells within PFC injection site lack NMDARs. B–C, AAV-Cre injection ...

Rescuing NMDARs in VTA-projecting PFC neurons restores learning in CAV-VTA; NR1-KO mice

The observation that NMDARs in the PFC are required for Pavlovian conditioning raises the possibility that the NMDAR signaling that is critical for this type of learning occurs within the subpopulation of PFC neurons that project to the VTA. To test this hypothesis we designed a combinatorial viral injection strategy to remove NMDARs from neurons that project to the VTA and then selectively restore NMDAR signaling only to the population of PFC neurons that project the VTA. We accomplished this by injecting AAV1 vector containing hemaglutinin (HA)-tagged, floxed-stop Grin1 cDNA (AAV-fsNR1) into the PFC of CAV-VTA; NR1-KO mice (Fig. 5A). We have previously demonstrated that virus-encoded NR1 expression occurs selectively in the presence of Cre when using AAV-fsNR1 (Beutler et al., 2011). We predicted that if NMDAR signaling in VTA-projecting PFC neurons is important for Pavlovian conditioning, then these rescued animals (CAV-VTA; PFC-NR1-Rescue mice) should have restored learning. To confirm that we had rescued NR1 expression in the PFC of these animals, we examined the PFC of CAV-VTA; PFC-NR1-Rescue mice for HA immunoreactive neurons that also expressed YFP (Cre-positive). Consistent with restored NR1 expression in the PFC of these animals, HA and YFP immunostaining overlapped in this brain region (Fig. 5B–D). During Pavlovian conditioning, CAV-VTA; PFC-NR1-Rescue mice learned at a similar rate to control mice subjected to the same surgical procedures. These results indicate that NMDAR signaling in the subpopulation of PFC neurons that project to the VTA is sufficient for appetitive Pavlovian conditioning.

Figure 5
Selectively restoring NMDARs to the VTA-projecting neurons of the PFC rescues learning in CAV-VTA; NR1-KO mice. A, (top) Diagram of AAV-fsNR1 construct; (bottom) viral injection strategy for generating CAV-VTA; NR1-PFC-Rescue mice. Solid-green shading ...

Discussion

Determining the precise contribution of dopamine to stimulus-reward learning is an area of ongoing research. While dopamine has been shown to be critical for the acquisition, consolidation, and expression of the appetitive association (Di Ciano et al., 2001; Dalley et al., 2005; Parker et al., 2010), precisely which brain areas and cell types contribute to each of these processes are incompletely understood. Moreover, whether dopamine encodes the predictive value of the CS, its incentive motivational properties, or both, remains unclear (Berridge and Robinson, 1998). Nonetheless, it is well established that dopamine is involved in appetitive Pavlovian conditioning, and given the general involvement of NMDARs in learning (Whitlock et al., 2006), it follows that NMDAR signaling within the dopamine system is likely to be important for appetitive Pavlovian learning. Because NMDAR signaling can be abolished in distinct neuronal populations by inactivating its obligatory NR1 subunit, we were able to utilize a combinatorial genetic approach to inactivate NMDAR signaling as a way to better understand the functional circuitry of the dopamine system during appetitive Pavlovian conditioning. It is important to note that because our behavioral readout represents the integration of all of the processes that underly learning (e.g., acquisition, consolidation, and expression), we can only conclude from our results that some aspect of appetitive Pavlovian conditioning is disrupted by our genetic manipulations.

Our initial observation that NMDARs in dopamine neurons are not necessary for appetitive Pavlovian conditioning led us to hypothesize that NMDAR signaling within brain regions that project to dopamine neurons, receive dopaminergic innervation, or both would be important for this type of learning. To address these possibilities we first focused on striatal neurons which receive dense dopaminergic innervation. Given that NMDAR signaling in the ventral striatum is important for appetitive Pavlovian conditioning and given the importance of dopamine (Di Ciano et al., 2001), it follows that NMDARs in the subpopulation of striatal neurons that densely express dopamine receptors (the MSNs) would be important for this type of learning. However, the specific requirement for NMDARs in MSNs for appetitive Pavlovian conditioning had not been directly demonstrated. For instance, there are many other types of neurons within the striatum that express NMDARs including aspiny GABAergic and cholinergic neurons (Kreitzer, 2009). The cholinergic neurons, in particular, have been implicated in processing information regarding CS-US relationships (Joshua et al., 2008). Our finding that NMDARs in MSNs are necessary for appetitive Pavlovian conditioning highlights the importance of these neurons for integrating information about predictive environmental cues to engage in anticipatory behavior.

MSNs can be subdivided into at least two classes: D1R- and D2R-expressing MSNs (Kreitzer, 2009). Because dopamine has been shown to modulate glutamatergic input to both classes of MSNs (Shen et al., 2008), NMDAR signaling in either or both could be important for appetitive Pavlovian conditioning. However, we already demonstrated that D1R signaling is required for this type of learning (Parker et al., 2010), so we predicted that NMDAR signaling within the D1R-expressing MSNs would be particularly important. Our finding that D1-Cre; NR1-KO mice had a deficit in appetitive Pavlovian conditioning further highlights the importance of D1R-expressing neurons for this type of learning by showing that NMDAR in addition to D1R signaling is required in these neurons. While our data suggest that NMDARs in D2R-expressing neurons were not sufficient for appetitive Pavlovian conditioning, future studies could determine the necessity of NMDAR signaling within the D2R subpopulation of neurons and better characterize the contribution of these MSNs to this type of learning.

We next sought to expand upon the observation that mice lacking NMDARs in dopamine neurons develop CS-elicited dopamine release and learn at the same rate as controls during appetitive Pavlovian conditioning (Parker et al., 2010). These results suggest that that the neurons that were driving VTA dopamine neuron activity during learning had formed the CS-US association. Therefore, we predicted that NMDAR signaling within brain regions that project to the VTA would be important for this type of learning. We explored this possibility using CAV-VTA; NR1-KO mice and found that this was indeed the case. When we subsequently asked which VTA-projecting brain regions required NMDAR signaling, we found that NMDARs in the PFC were critical for appetitive Pavlovian conditioning. Moreover, our experiments allowed us to specifically pinpoint the subset of PFC neurons within the prelimbic area that project to the VTA as being important for learning in this paradigm.

The observation that the PFC projects to the ventral striatum in addition to the VTA raises some additional points for consideration (French and Totterdell, 2002). For instance, it is not known whether the VTA-projecting and striatum-projecting PFC neurons are the same population of cells. Therefore, it is possible that PFC-striatum connectivity is also disrupted in CAV-VTA; NR1-KO mice, and this could account for the learning deficit in these animals. It has also been reported that most of the VTA dopamine neurons that are innervated by the PFC actually project back to the PFC rather than to the ventral striatum (Carr and Sesack, 2000). This same study also reported that the VTA neurons that receive input from the PFC and send projections to the ventral striatum are GABAergic rather than dopaminergic neurons (Carr and Sesack, 2000). In the context of these findings, our data suggest that VTA-PFC interconnectivity is important for forming appetitive Pavlovian associations, and that the PFC may, in turn, influence behavior by directly or indirectly modulating the activity of the ventral striatum. Future studies will be important to better understand the interconnectivity of the PFC with other brain structures in order to assess how these connections contribute to learned behavior.

Important afferents to the PFC include glutamatergic projections from the basolateral amygdala and the ventral subiculum of the hippocampal formation (Bacon et al., 1996; Carr and Sesack, 1996). Both of these brain regions are critical for memory formation and both also send projections to the ventral striatum (French and Totterdell, 2002). Moreover, the amygdalar projection to the PFC is implicated in cue-based learning (McGinty and Grace, 2008). It seems likely that the crosstalk between all of these structures is important for modulating dopamine and MSN transmission to generate learned behavior (Cardinal et al., 2002b). Because all of the aforementioned brain regions are implicated in Pavlovian conditioning, our current findings behaviorally corroborate previous anatomical and electrophysiological studies characterizing the involvement of the PFC in a cortico-basal ganglia reward network (McGinty and Grace, 2008; Sesack and Grace, 2010).

While we report here that attenuating NMDAR signaling within the PFC disrupts associative learning, NMDAR hypofunction (Mohn et al., 1999; Coyle, 2006), particularly within this brain region (Akbarian et al., 1996), has been hypothesized to contribute to many of the symptoms observed in schizophrenia. While it is not precisely known whether the NMDAR hypofunction in the PFC observed during schizophrenia occurs in glutamatergic pyramidal neurons, GABAergic interneurons, or both (Rolls et al., 2008), our results raise the possibility that some of the cognitive deficits in schizophrenic patients may be due to NMDAR hypofunction within the subpopulation of PFC neurons that project to the VTA (Sesack and Carr, 2002). Using behavior to better understand the roles of these different populations of PFC neurons based upon their anatomical connectivity as well as genetic identity will aid in our understanding of this brain region and provide more insights into therapeutic interventions for the complex psychiatric illnesses that result from its dysfunction.

We conclude from our findings that NMDARs and, presumably, NMDAR-dependent synaptic plasticity within neurons that project to and receive projections from VTA dopamine neurons is important for appetitive Pavlovian learning. Our data specifically implicate D1R-expressing MSNs and VTA-projecting PFC neurons in this process. These experiments pinpoint the specific brain regions and cell types involved in the acquisition and maintenance of learned associations and offer a more detailed representation of the functional connectivity of the cortico-basal ganglia reward network and how this network influences learning and behavior.

Acknowledgements

We thank Paul E. M. Phillips, Nigel S. Bamford, and Matt E. Carter for thoughtful discussion of this manuscript. We also thank Charlie Henschen and Patricia Salles for assistance with mouse breeding and histology. This work was supported in part by a National Institute on Drug Abuse grant T32 DA07278-14 (J.G.P.) and by the Achievement Rewards for College Scientists (ARCS) Foundation (L.R.B.).

Footnotes

These authors declare no conflicts of interest

References

  • Akbarian S, Sucher N, Bradley D, Tafazzoli A, Trinh D, Hetrick W, Potkin S, Sandman C, Bunney W, Jones E. Selective alterations in gene expression for NMDA receptor subunits in prefrontal cortex of schizophrenics. J Neurosci. 1996;16:19–30. [PubMed]
  • Bacon SJ, Headlam AJN, Gabbott PLA, Smith AD. Amygdala input to medial prefrontal cortex (mPFC) in the rat: A light and electron microscope study. Brain Res. 1996;720:211–219. [PubMed]
  • Berridge KC, Robinson TE. What is the role of dopamine in reward: Hedonic impact, reward learning, or incentive salience? Brain Res Rev. 1998;28:309–369. [PubMed]
  • Beutler LR, Wanat MJ, Quintana A, Sanz E, Bamford NS, Zweifel LS, Palmiter RD. Balanced NMDA receptor activity in dopamine D1 receptor (D1R)-and D2R-expressing medium spiny neurons is required for amphetamine sensitization. Proc Natl Acad Sci USA. 2011;108:4206–4211. [PubMed]
  • Björklund A, Dunnett SB. Dopamine neuron systems in the brain: An update. Trends Neurosci. 2007;30:194–202. [PubMed]
  • Bliss TVP, Lømo T. Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J Physiol. 1973;232:331–356. [PubMed]
  • Cardinal RN, Parkinson JA, Lachenal G, Halkerston KM, Rudarakanchana N, Hall J, Morrison CH, Howes SR, Robbins TW, Everitt BJ. Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behav Neurosci (United States) 2002a;116:553–567. [PubMed]
  • Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: The role of the amygdala, ventral striatum, and prefrontal cortex. Neuroscience & Biobehavioral Reviews. 2002b;26:321–352. [PubMed]
  • Carr DB, Sesack SR. Hippocampal afferents to the rat prefrontal cortex: Synaptic targets and relation to dopamine terminals. J Comp Neurol. 1996;369:1–15. [PubMed]
  • Carr DB, Sesack SR. Projections from the rat prefrontal cortex to the ventral tegmental area: Target specificity in the synaptic associations with mesoaccumbens and mesocortical neurons. J Neurosci. 2000;20:3864–3873. [PubMed]
  • Coyle JT. Glutamate and schizophrenia: Beyond the dopamine hypothesis. Cell Mol Neurobiol. 2006;26:365–384. [PubMed]
  • Dalley JW, Lääne K, Theobald DEH, Armstrong HC, Corlett PR, Chudasama Y, Robbins TW. Time-limited modulation of appetitive pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc Natl Acad Sci USA. 2005;102:6189–6194. [PubMed]
  • Day JJ, Roitman MF, Wightman RM, Carelli RM. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci. 2007;10:1020–1028. [PubMed]
  • Di Ciano P, Cardinal RN, Cowell RA, Little SJ, Everitt BJ. Differential involvement of NMDA, AMPA/Kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J Neurosci. 2001;21:9471–9477. [PubMed]
  • Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PE, Akil H. A selective role for dopamine in stimulus-reward learning. Nature. 2011;469:53–57. [PMC free article] [PubMed]
  • French SJ, Totterdell S. Hippocampal and prefrontal cortical inputs monosynaptically converge with individual projection neurons of the nucleus accumbens. J Comp Neurol. 2002;446:151–165. [PubMed]
  • Geisler S, Derst C, Veh RW, Zahm DS. Glutamatergic afferents of the ventral tegmental area in the rat. J Neurosci. 2007;27:5730–5743. [PMC free article] [PubMed]
  • Hnasko TS, Perez FA, Scouras AD, Stoll EA, Gale SD, Luquet S, Phillips PEM, Kremer EJ, Palmiter RD. Cre recombinase-mediated restoration of nigrostriatal dopamine in dopamine-deficient mice reverses hypophagia and bradykinesia. Proc Natl Acad Sci USA. 2006;103:8858–8863. [PubMed]
  • Hong S, Hikosaka O. Dopamine-mediated learning and switching in cortico-striatal circuit explain behavioral changes in reinforcement learning. Front Behav Neurosci. 2011;5:15. [PMC free article] [PubMed]
  • Joshua M, Adler A, Mitelman R, Vaadia E, Bergman H. Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J Neurosci. 2008;28:11673–11684. [PubMed]
  • Karreman M, Moghaddam B. The prefrontal cortex regulates the basal release of dopamine in the limbic striatum: An effect mediated by ventral tegmental area. J Neurochem. 1996;66:589–598. [PubMed]
  • Kerr JND, Wickens JR. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J Neurophys. 2001;85:117–124. [PubMed]
  • Kheirbek MA, Beeler JA, Ishikawa Y, Zhuang X. A cAMP pathway underlying reward prediction in associative learning. J Neurosci. 2008;28:11401–11408. [PMC free article] [PubMed]
  • Kreitzer AC. Physiology and pharmacology of striatal neurons. Annu Rev Neurosci. 2009;32:127–147. [PubMed]
  • Lammel S, Hetzel A, Häckel O, Jones I, Liss B, Roeper J. Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron. 2008;57:760–773. [PubMed]
  • Levine MS, Altemus KL, Cepeda C, Cromwell HC, Crawford C, Ariano MA, Drago J, Sibley DR, Westphal H. Modulatory actions of dopamine on NMDA receptor-mediated responses are reduced in D1A-deficient mutant mice. J Neurosci. 1996;16:5870–5882. [PubMed]
  • Malenka RC, Bear MF. LTP and LTD: An embarrassment of riches. Neuron. 2004;44:5–21. [PubMed]
  • Massart R, Guilloux J, Mignon V, Sokoloff P, Diaz J. Striatal GPR88 expression is confined to the whole projection neuron population and is regulated by dopaminergic and glutamatergic afferents. Eur J Neurosci. 2009;30:397–414. [PubMed]
  • McGinty VB, Grace AA. Selective activation of medial prefrontal-to-accumbens projection neurons by amygdala stimulation and pavlovian conditioned stimuli. Cereb Cortex. 2008;18:1961–1972. [PMC free article] [PubMed]
  • Mohn AR, Gainetdinov RR, Caron MG, Koller BH. Mice with reduced NMDA receptor expression display behaviors related to schizophrenia. Cell. 1999;98:427–436. [PubMed]
  • Moorman DE, Aston-Jones G. Orexin/Hypocretin modulates response of ventral tegmental dopamine neurons to prefrontal activation: Diurnal influences. J Neurosci. 2010;30:15585–15599. [PMC free article] [PubMed]
  • Pan W, Schmidt R, Wickens JR, Hyland BI. Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network. J Neurosci. 2005;25:6235–6242. [PubMed]
  • Parker JG, Zweifel LS, Clark JJ, Evans SB, Phillips PEM, Palmiter RD. Absence of NMDA receptors in dopamine neurons attenuates dopamine release but not conditioned approach during pavlovian conditioning. Proc Natl Acad Sci USA. 2010;107:13491–13496. [PubMed]
  • Parkinson JA, Willoughby PJ, Robbins TW, Everitt BJ. Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs pavlovian approach behavior: Further evidence for limbic cortical-ventral striatopallidal systems. Behav Neurosci. 2000;114:42–63. [PubMed]
  • Parkinson JA, Dalley JW, Cardinal RN, Bamford A, Fehnert B, Lachenal G, Rudarakanchana N, Halkerston KM, Robbins TW, Everitt BJ. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive pavlovian approach behaviour: Implications for mesoaccumbens dopamine function. Behav Brain Res. 2002;137:149–163. [PubMed]
  • Redgrave P, Prescott TJ, Gurney K. The basal ganglia: A vertebrate solution to the selection problem? Neuroscience. 1999;89:1009–1023. [PubMed]
  • Richfield EK, Penney JB, Young AB. Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience. 1989;30:767–777. [PubMed]
  • Rolls ET, Loh M, Deco G, Winterer G. Computational models of schizophrenia and dopamine modulation in the prefrontal cortex. Nat Rev Neurosci. 2008;9:696–709. [PubMed]
  • Sesack SR, Grace AA. Cortico-basal ganglia reward network: Microcircuitry. Neuropsychopharmacology. 2010;35:27–47. [PMC free article] [PubMed]
  • Sesack SR, Carr DB. Selective prefrontal cortex inputs to dopamine cells: Implications for schizophrenia. Physiol Behav. 2002;77:513–517. [PubMed]
  • Shen W, Flajolet M, Greengard P, Surmeier DJ. Dichotomous dopaminergic control of striatal synaptic plasticity. Science. 2008;321:848–851. [PMC free article] [PubMed]
  • Soudais C, Skander N, Kremer EJ. Long-term in vivo transduction of neurons throughout the rat central nervous system using novel helper-dependent CAV-2 vectors. The FASEB Journal. 2004;18:391–393. [PubMed]
  • Srinivas S, Watanabe T, Lin CS, William CM, Tanabe Y, Jessell TM, Costantini F. Cre reporter strains produced by targeted insertion of EYFP and ECFP into the ROSA26 locus. BMC Dev Biol. 2001;1:4. [PMC free article] [PubMed]
  • Stuber GD, Klanker M, de Ridder B, Bowers MS, Joosten RN, Feenstra MG, Bonci A. Reward-predictive cues enhance excitatory synaptic strength onto midbrain dopamine neurons. Science. 2008;321:1690–1692. [PMC free article] [PubMed]
  • Tsien JZ, Huerta PT, Tonegawa S. The essential role of hippocampal CA1 NMDA receptor-dependent synaptic plasticity in spatial memory. Cell. 1996;87:1327–1338. [PubMed]
  • Whitlock JR, Heynen AJ, Shuler MG, Bear MF. Learning induces long-term potentiation in the hippocampus. Science. 2006;313:1093–1097. [PubMed]
  • Zhuang X, Masson J, Gingrich JA, Rayport S, Hen R. Targeted gene expression in dopamine and serotonin neurons of the mouse brain. J Neurosci Methods. 2005;143:27–32. [PubMed]
  • Zukin RS, Bennett MVL. Alternatively spliced isoforms of the NMDARI receptor subunit. Trends Neurosci. 1995;18:306–313. [PubMed]
  • Zweifel LS, Argilli E, Bonci A, Palmiter RD. Role of NMDA receptors in dopamine neurons for plasticity and addictive behaviors. Neuron. 2008;59:486–496. [PMC free article] [PubMed]
  • Zweifel LS, Parker JG, Lobb CJ, Rainwater A, Wall VZ, Fadok JP, Darvas M, Kim MJ, Mizumori SJY, Paladini CA, Phillips PEM, Palmiter RD. Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior. Proc Natl Acad Sci USA. 2009;106:7281–7288. [PubMed]