PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of frontbehavneuroLink to Publisher's site
 
Front Behav Neurosci. 2010; 4: 172.
Published online 2010 November 17. Prepublished online 2010 September 1. doi:  10.3389/fnbeh.2010.00172
PMCID: PMC2991128

Roles of Aminergic Neurons in Formation and Recall of Associative Memory in Crickets

Abstract

We review recent progress in the study of roles of octopaminergic (OA-ergic) and dopaminergic (DA-ergic) signaling in insect classical conditioning, focusing on our studies on crickets. Studies on olfactory learning in honey bees and fruit-flies have suggested that OA-ergic and DA-ergic neurons convey reinforcing signals of appetitive unconditioned stimulus (US) and aversive US, respectively. Our work suggested that this is applicable to olfactory, visual pattern, and color learning in crickets, indicating that this feature is ubiquitous in learning of various sensory stimuli. We also showed that aversive memory decayed much faster than did appetitive memory, and we proposed that this feature is common in insects and humans. Our study also suggested that activation of OA- or DA-ergic neurons is needed for appetitive or aversive memory recall, respectively. To account for this finding, we proposed a model in which it is assumed that two types of synaptic connections are strengthened by conditioning and are activated during memory recall, one type being connections from neurons representing conditioned stimulus (CS) to neurons inducing conditioned response and the other being connections from neurons representing CS to OA- or DA-ergic neurons representing appetitive or aversive US, respectively. The former is called stimulus–response (S–R) connection and the latter is called stimulus–stimulus (S–S) connection by theorists studying classical conditioning in vertebrates. Results of our studies using a second-order conditioning procedure supported our model. We propose that insect classical conditioning involves the formation of S–S connection and its activation for memory recall, which are often called cognitive processes.

Keywords: octopamine, dopamine, classical conditioning, memory recall, olfactory learning, visual learning, crickets

Introduction

Biogenic amines regulate various functions of central nervous systems in vertebrates and invertebrates (Blenau and Baumann, 2001). In vertebrates, dopamine (DA) pathways are involved in the coordination of motor behavior, motivation, addiction, and reward-based learning of a wide range of sensory stimuli (Schultz, 1998, 2006). In insects, DA appears to play roles in regulating motor behavior (Blenau and Baumann, 2001) and arousal (Andretic et al., 2005), and octopamine (OA), the invertebrate counterpart of noradrenaline, plays roles in desensitizing sensory inputs and regulating various forms of behavior (Roeder, 1999), including aggression (Stevenson et al., 2005; Hoyer et al., 2008; Zhou et al., 2008) and sleep (Crocker et al., 2010). In this article, we review recent advances in studies on the roles of OA-ergic and DA-ergic neurons in classical conditioning in insects, focusing on findings from our behavioral and pharmacological studies on crickets Gryllus bimaculatus.

Roles of Aminergic Neurons in Formation of Olfactory Memory in Honey Bees and Fruit-Flies

Insects are useful animal models for the study of cellular and molecular mechanisms underlying learning and memory (Giurfa, 2003; Heisenberg, 2003; Davis, 2005; Menzel and Giurfa, 2006; Menzel et al., 2006; Keene and Waddell, 2007). This is mainly because insect brains consist of a relatively small number (<106) of neurons and the organization of “microbrains” (Mizunami et al., 1999, 2004) is relatively simple, and thus their functions are highly susceptive to genetic, pharmacological, and other means of experimental manipulation. One of the pioneering studies on the roles of aminergic neurons in insect learning was performed in honey bees by Hammer (1993). A hungry honey bee extends its proboscis in response to sucrose stimulation applied to its antennae, proboscis, or tarsi. The proboscis extension response can be conditioned by pairing an odor applied to the antennae (conditioned stimulus, CS) with sucrose stimulation (unconditioned stimulus, US) (Kuwabara, 1957; Erber et al., 1980). Hammer (1993) observed that pairing of an odor with intracellular stimulation of the VUMmx1 neuron, a putative octopamine immunoreactive neuron (Kreissl et al., 1994) that exhibited responses to sucrose stimulation, induced a conditioning effect. Hence, he concluded that this neuron mediates the reinforcing property of sucrose reward in olfactory conditioning. Later, Hammer and Menzel (1998) showed that local injection of octopamine into the antennal lobes and the calyces of the mushroom bodies, termination areas of the VUMmx1 neuron, substituted the sucrose US in olfactory conditioning; the antennal lobes are primary olfactory centers and the mushroom bodies are higher-order olfactory and multi-sensory association centers (Erber et al., 1980; Heisenberg et al., 1985; Mizunami et al., 1998a,b; Okada et al., 1999; Heisenberg, 2003; Davis, 2005; Menzel and Giurfa, 2006). In addition, Farooqui et al. (2003) showed that RNA interference of OA receptors or pharmacological blockade of OA receptors by mianserin in the antennal lobe impaired olfactory conditioning.

Another pioneering study was performed in the fruit-fly Drosophila by Schwaerzel et al. (2003). Fruit-flies can be conditioned to choose an odor associated with sucrose, or avoid an odor associated with electric shock (Dudai et al., 1976; Tully and Quinn, 1985). Schwaerzel et al. (2003) demonstrated that transgenic flies defective in OA or DA synthesis exhibited defects in appetitive olfactory learning with sucrose reward or aversive olfactory learning with electric shock, respectively. Subsequent studies in fruit-flies confirmed that OA- or DA-ergic neurons convey signals for sucrose reward or electric shock punishment, respectively, in olfactory conditioning in larval (Schroll et al., 2006; Honjo and Furukubo-Tokunaga, 2009; Selcho et al., 2009) and adult flies (Riemensperger et al., 2005; Tomchik and Davis, 2009; Gervasi et al., 2010), although a few exceptions have also been found (Kim et al., 2007, Sitaraman et al., 2008). Notable findings in these studies are that photoactivation of OA-ergic or DA-ergic neurons paired with an odor stimulation successfully induced an appetitive or aversive conditioning effect, respectively, in larval flies (Schroll et al., 2006) and that photoactivation of a class of DA-ergic brain neurons also induced an aversive conditioning effect in adult flies (Claridge-Chang et al., 2009). Roles of DA in conveying aversive US have also been demonstrated in honey bees in olfactory conditioning of the sting extension reflex, in which an odor was paired with electric shock punishment (Vergoz et al., 2007).

Procedures for Olfactory and Visual Pattern Conditioning in Crickets

We studied the roles of OA-ergic and DA-ergic signaling in classical conditioning in crickets Gryllus bimaculatus. We have demonstrated that crickets have excellent olfactory learning capabilities: for example, they can learn (1) to associate an odor with reward by a single operant or classical conditioning trial (Matsumoto and Mizunami, 2000, 2002a), (2) to form a lifetime olfactory memory (Matsumoto and Mizunami, 2002b), (3) to memorize seven pairs of odors at the same time (Matsumoto and Mizunami, 2006), and (4) to associate one odor with reward and another odor with punishment in one visual context and to associate the opposing in another visual context (Matsumoto and Mizunami, 2004). Their high learning capabilities may reflect their omnivorous foraging habit, i.e., they test many potential food items to assess whether they are edible or not. Capacity for forming visual place memory has also been demonstrated in crickets (Wessnitzer et al., 2008). Moreover, we have shown that crickets are suitable materials for the study of molecular mechanisms underlying learning and memory by using pharmacological manipulation (Matsumoto et al., 2006, 2009) and RNA interference (Takahashi et al., 2009).

We used a “classical conditioning and operant testing” procedure, which is based on a high capability of crickets to transfer memory formed in a classical conditioning situation to an operant testing situation (Matsumoto and Mizunami, 2002a; Unoki et al., 2005, 2006). For appetitive olfactory conditioning, crickets were individually placed in a beaker, and one of two odors (e.g., banana and apple odors) was presented to their antennae and then water reward was presented to their mouth (Figure (Figure1A).1A). For aversive olfactory conditioning, one of these two odors was presented to the antennae before presenting 2 M sodium chloride solution to the mouth. In the odor preference test, animals were allowed to freely choose between two odor sources. Each odor source consisted of a container containing a filter paper soaked with a solution of odor essence. The time that the animals touched the gauze net covering the top of each container with their mouths or palpi was measured for evaluating relative odor preference of the animals (Figure (Figure1B).1B). For visual pattern conditioning, either of a white-center and black-surround pattern or a black-center and white-surround pattern was paired with water reward or sodium chloride punishment (Figure (Figure1C).1C). In the preference test, the time that the animals touched each of the two patterns was measured for evaluating relative preference (Figure (Figure11D).

Figure 1
Procedures for olfactory and visual pattern conditioning in crickets. (A) Procedures for olfactory conditioning. One of two odors (e.g., banana and apple odors) was used as CS, and water or 20% sodium chloride solution was used as US. A syringe containing ...

These procedures were highly effective for achieving conditioning. In the case of appetitive olfactory conditioning, for example, one conditioning trial was sufficient to establish conditioning, with its memory lasting for several hours (mid-term memory; see Figure Figure4;4; Matsumoto and Mizunami, 2002a; Unoki et al., 2005). Two appetitive conditioning trials with a 5-min interval induce memory that lasts for at least 1 day (see Figure Figure4),4), which matches protein-synthesis-dependent long-term memory because it is blocked by injection of a protein-synthesis inhibitor, cycloheximide, into the hemolymph before conditioning (Matsumoto et al., 2003).

Figure 4
Time course of memory retention after appetitive (A) and aversive (B) conditioning. Seventeen groups of crickets were subjected to 1-trial (gray circles), 2-trial (black triangles), or 6-trial (open squares) appetitive or aversive conditioning trials ...

Roles of Aminergic Neurons in Formation of Olfactory Memory in Crickets

We studied the effect of OA and DA receptor antagonists on appetitive and aversive olfactory conditioning in crickets (Unoki et al., 2005). Crickets injected with epinastine or mianserin, antagonists of insect OA receptors (Roeder et al., 1998; Degen et al., 2000b), into the hemolymph before conditioning exhibited a complete impairment of appetitive conditioning to associate an odor with water reward (Figure (Figure2A);2A); the preference for the rewarded odor after conditioning did not significantly differ from that before conditioning. On the other hand, these animals exhibited no impairment of aversive learning with saline punishment (Figure (Figure2B).2B). The latter observation shows that OA receptor antagonists do not impair sensory function, motor function, or motivation necessary for learning, and we thus conclude that OA is specifically involved in conveying water reward. We also found that fluphenazine, chlorpromazine, or spiperone, antagonists of insect DA receptors (Degen et al., 2000a; Mustard et al., 2003), completely impaired aversive learning with sodium chloride punishment (Figure (Figure2C)2C) but did not affect appetitive learning with water reward (Figure (Figure2D).2D). The latter finding indicates that DA receptor antagonists do not impair the sensory function, motor function, or motivation necessary for learning, and thus we conclude that DA is specifically involved in conveying sodium chloride punishment. It should be cautioned that the specificity of antagonists used in our studies is not necessarily perfect (see Discussion in Unoki et al., 2005). However, two different kinds of OA receptor antagonists impaired appetitive learning but not aversive learning and three different kinds of DA receptor antagonists impaired aversive learning but not appetitive learning, suggesting that the impairments were due to the blockade of OA or DA receptors. Thus, we concluded that OA- or DA-ergic neurons convey information about appetitive or aversive US, respectively, for olfactory conditioning in crickets (Unoki et al., 2005).

Figure 2
Effects of OA or DA receptor antagonists on appetitive and aversive olfactory conditioning. (A) Dose-dependent effects of OA receptor antagonists on appetitive olfactory conditioning. Six groups of crickets were injected with 3 μl saline (white ...

Roles of Aminergic Neurons in Formation of Visual Pattern Memory and Color Memory

We next studied the effect of OA and DA receptor antagonists on appetitive and aversive conditioning of visual pattern (Unoki et al., 2006) and color (Nakatani et al., 2009). Crickets injected with epinastine or mianserin, OA receptor antagonists, into the hemolymph exhibited a complete impairment of appetitive learning to associate a visual pattern with water reward, but aversive learning to associate a visual pattern with sodium chloride punishment was unaffected (Figure (Figure3A).3A). In contrast, fluphenazine, chlorpromazine, or spiperone, DA receptor antagonists, completely impaired aversive learning but not appetitive learning (Figure (Figure3B).3B). We also found the same for color learning: OA receptor antagonists impaired appetitive color learning with water reward without affecting aversive color learning with sodium chloride punishment. In contrast, DA receptor antagonists impaired aversive color learning without affecting appetitive color learning (Nakatani et al., 2009). These findings indicate that the roles of OA-ergic and DA-ergic neurons in conveying information about appetitive and aversive US, respectively, are ubiquitous in learning of odor, visual pattern and color stimuli, suggesting that these neurons serve as the general reward or punishment system for insect learning.

Figure 3
Effects of OA or DA receptor antagonists on appetitive and aversive visual pattern conditioning. (A) Effects of OA receptor antagonists. Six groups of crickets were each injected with 3 μl saline or saline containing 1 μM epinastine or ...

In mammals, midbrain DA-ergic neurons play major roles in a wide range of visual, auditory, and somatosensory stimuli and thus are considered to serve as a general reward system (Schultz, 1998, 2006). It appears that the roles of aminergic neurons in conveying reinforcement signals are conserved across different phyla, but the kind of reinforcement signal that each biogenic amine mediates is different: DA mediates appetitive reinforcement in mammals but mediates aversive reinforcement in insects. Future studies on the roles of DA in learning in phylogenetically ancient species may clarify how different roles of DA in positive or negative reinforcement have emerged during the course of evolution. Neurotransmitters meditating negative reinforcement in mammals are less known, but roles of noradrenaline, serotonin, or DA in some forms of aversive learning have been suggested (Daw et al., 2002; Harley, 2004; Wise, 2004; Schultz, 2006).

Different Dynamics of Appetitive Memory and Aversive Memory

In the work described above, we noticed that the time courses of appetitive memory and aversive memory fundamentally differ (Unoki et al., 2005, 2006; Nakatani et al., 2009). In Figure Figure4,4, the time course of memory after 1-trial, 2-trial, and 6-trial appetitive olfactory conditioning and that after 2-trial and 6-trial aversive olfactory conditioning are shown. A group subjected to 1-trial appetitive conditioning exhibited a significant level of retention at 30-min after conditioning: the preference for rewarded odor was significantly greater than that before conditioning. The memory did not significantly decay from 30 min to 1 h after conditioning, but it was completely diminished at 1 day after conditioning. Two-trial and 6-trial groups exhibited a significant level of 30-min retention, with no significant decay of retention from 30 min to 1 day after conditioning. Two-trial aversive conditioning group exhibited a significant level of 30-min retention. However, the memory decayed to a non-significant level at 1 h after conditioning. Six-trial aversive conditioning group exhibited a significant decay of retention from 30 min to 1 h after training. Thereafter, a significant level of retention was maintained with no significant decay from 1 h to 1 day after conditioning. In short, aversive olfactory memory exhibited a prominent decay from 30 min to 1 h after conditioning, whereas appetitive olfactory memory exhibited little decay from 30 min to 1 day after conditioning.

We found similar distinction of dynamics between appetitive memory and aversive memory for visual pattern conditioning (Unoki et al., 2006) and color conditioning (Nakatani et al., 2009) in crickets. It was obvious that the number of conditioning trials, and hence the levels of initial acquisition, also influenced memory dynamics, but effects of these factors did not account for the difference in memory dynamics observed after appetitive and aversive learning: reward memory was sustained even when the level of 30-min retention was low, and punishment memory exhibited a characteristic decay even when the level of 30-min retention was high.

Comparisons with studies in other species of insects showed that our finding that aversive memory is less durable than appetitive memory is not specific to the type of US we used (water as appetitive US and sodium chloride as aversive US) or the species used (crickets). In fruit-flies, it has been reported that punishment memory after conditioning of an odor with electric shock punishment decays much faster than reward memory after conditioning of an odor with sucrose reward, regardless of intensity of electric shock and the concentration of the sucrose solution (Tempel et al., 1983). A recent study also suggested that aversive olfactory memory with quinine or saline punishment is less durable than appetitive olfactory memory with sucrose reward in fruit-fly larvae (Honjo and Furukubo-Tokunaga, 2009). Thus, dynamics of punishment memory and reward memory differ for different intensities or kinds of unconditioned stimulus (US; water or sugar as reward and saline, quinine, or electric shock as punishment), for different species of insects (crickets and fruit-flies), and for different conditioning paradigms (individual conditioning in crickets and group conditioning in fruit-flies).

By examining the literature in human psychology, we have proposed that these findings in insects are comparable to findings in humans (Nakatani et al., 2009). Many studies in educational psychology have proposed that punishment is highly effective for immediately suppressing behavior of children at school or home, but the effect tends to be short-lived compared to the effect of reward (Peine and Howarth, 1975; Gershoff, 2002; Driscoll, 2005). However, to our knowledge, no convincing evidence supporting the argument that the dynamics of reward and punishment memory fundamentally differ has been obtained in rigorously controlled animal experiments (Walters and Crusec, 1977), except for several studies on insects discussed above. Obviously, our proposal that different dynamics after punishment and reward learning is conserved across phyla is highly speculative, and more studies on vertebrates are clearly needed to evaluate the validity of this idea.

If different dynamics of punishment and reward memory is conserved across different phyla, what is the possible adaptive significance for it? We have proposed that it is related to a different significance of reward and punishment leaning for survival in a changing environment (Nakatani et al., 2009). Since the environment is constantly changing, stimuli that once served as predictors of punishment may change to predict reward or vice versa. Consider that an inedible food item in one season may become profitable in the next season. In this case, long-term retention of avoidance of stimuli that once predicted aversive stimuli is not necessarily beneficial, because it reduces the opportunity to obtain useful resources in the future. Stimuli that once predicted reward may also be changed to predict punishment, but long-term retention of preference for once-rewarded stimuli has no such cost, because animals can re-learn to avoid such stimuli when they encounter the stimuli again. Thus, we have proposed that different adaptive significance of durability of memory between reward and punishment learning is the basis of different dynamics of reward and punishment memories (Nakatani et al., 2009).

Participation of Octopaminergic and Dopaminergic Neurons in Appetitive and Aversive Memory Recall

We then studied the effect of an OA or DA receptor antagonist on appetitive or aversive memory recall (retrieval). The results suggested that intact OA-ergic or DA-ergic signaling is necessary for recall of appetitive or aversive memory, respectively, after olfactory learning and visual pattern learning (Mizunami et al., 2009). Crickets were subjected to appetitive or aversive olfactory conditioning and were injected with an OA or DA receptor antagonist before retention test. Injection of epinastine or mianserin, OA receptor antagonists, completely impaired appetitive olfactory memory recall but had no effect on aversive olfactory memory recall (Figure (Figure5A).5A). On the other hand, injection of fluphenazine, chlorpromazine, or spiperone, DA receptor antagonists, completely impaired aversive memory recall but had no effect on appetitive memory recall (Figure (Figure5B).5B). This is in accordance with observations in honey bees that disruption of OA-ergic transmission in the antennal lobe, the primary olfactory center, by an OA receptor antagonist (mianserin) or by RNA interference of the OA receptor gene disrupted appetitive olfactory memory recall (Farooqui et al., 2003), although the results of the study by Farooqui et al. were not conclusive as we have discussed before (Mizunami et al., 2009). We also found that OA and DA receptor antagonists impaired appetitive and aversive memory recall, respectively, in visual pattern conditioning (Mizunami et al., 2009). Therefore, we concluded that intact synaptic transmission from OA- and DA-ergic neurons is needed for the recall of appetitive memory and aversive memory, respectively, in both olfactory and visual pattern learning.

Figure 5
Octopamine and dopamine receptor antagonists impair appetitive and aversive olfactory memory recall, respectively. Effects of OA (A) or DA (B) receptor antagonists on olfactory memory recall. Twelve groups of crickets were each subjected to 2-trial appetitive ...

We noticed that our findings are not consistent with conventional neural models of insect classical conditioning. Figure Figure6A6A depicts perhaps the best model proposed to account for the roles of extrinsic and intrinsic neurons of mushroom bodies in olfactory conditioning in the fruit-fly Drosophila (Schwaerzel et al., 2003). This model assumes that (1) “CS” neurons (intrinsic neurons of the mushroom body, called Kenyon cells) that convey signals about a CS make synaptic connections with dendrites of “CR” neurons (efferent (output) neurons of the mushroom body lobe), activation of which leads to a CR (conditioned response) that mimics UR (unconditioned response), but these synaptic connections are silent or very weak before conditioning, (2) OA- or DA-ergic efferent neurons projecting to the lobes (“OA/DA” neurons), which convey signals for appetitive or aversive US, respectively, make synaptic connections with axon terminals of “CS” neurons, and (3) the efficacy of the synaptic transmission from “CS” neurons to “CR” neurons that induces a conditioned response (CS–CR or S–R connection) is strengthened by coincident activation of “CS” neurons and “OA/DA” neurons during conditioning (assuming Kandelian synaptic plasticity; see Abrams and Kandel, 1988). In short, this model assumes that presentation of a CS after conditioning activates the CS–CR or S–R connection to induce a CR. Thus, this model is characterized as an S–R model (Figure (Figure7A),7A), following terminology in studies on classical conditioning in higher vertebrates (Rescorla, 1988; Pickens and Holland, 2004; Holland, 2008). It can be pointed out that the S–R model accounts for most forms of classical conditioning in invertebrates, including classical conditioning of gill withdrawal reflex in the mollusk Aplysia, where pairing of a gentle tactile stimulus to the siphon (CS) and a strong tactile stimulus to the gill (US) results in an enhancement of the efficacy of synaptic transmission from siphon sensory neuron to gill motor neuron (Abrams and Kandel, 1988; Kandel, 2001; Roberts and Glanzman, 2003), which is characterized as an S–R connection. The model by Schwaerzel et al. (2003), however, is inconsistent with our findings because it predicts that activation of OA- or DA-ergic neurons is not required for appetitive or aversive memory recall, respectively.

Figure 6
Conventional and new models of classical conditioning in insects. (A) A model proposed to account for the roles of intrinsic and extrinsic neurons of the mushroom body in olfactory conditioning in fruit-flies (Schwaerzel et al., 2003). OA-ergic or DA-ergic ...
Figure 7
S–R and S–S theories to account of classical conditioning. Two theories, i.e., the stimulus–response (S–R) association theory and the stimulus–stimulus (S–S) association theory, have been proposed to account ...

We have proposed a new model (Figure (Figure6B),6B), with minimal modifications of the model proposed by Schwaerzel et al. (2003). We have assumed that (1) activation of “OA/DA” neurons and resulting release of OA or DA are needed to “gate” the sensori-motor pathway from the “CS” neurons to “CR” neurons after conditioning and (2) synaptic connection from “CS” neurons to “OA/DA” neurons representing US is strengthened by coincident activation of “CS” neurons and “OA/DA” neurons by pairing of a CS with a US (assuming Hebbian synaptic plasticity). The latter connection is termed a CS–US or S–S connection (Figure (Figure7B;7B; Rescorla, 1988; Pickens and Holland, 2004), In short, our model assumes that two kinds of memory traces are formed by conditioning and that activation of both memory traces is needed for memory recall. This model corresponds to a hybrid of the S–R and S–S models (for explanation of the S–S model, see legends of Figure Figure77).

An alternative possibility to explain our findings is that different sets of “OA/DA” neurons govern reinforcement and memory retrieval processes, respectively. This is achieved by modifying the model shown in Figure Figure6A6A by assuming other “OA/DA” neurons in neural pathways downstream of the “CR” neurons. This model, however, failed to account for our results with second-order conditioning described below.

Evaluation of Our Model by Using a Second-Order Conditioning Procedure

The critical assumption of our model is that the pathway from neurons representing CS to OA/DA neurons representing appetitive or aversive US (S–S connection) is strengthened by conditioning. We evaluated this assumption by using a second-order conditioning procedure (Mizunami et al., 2009). Second-order conditioning (Figure (Figure6C)6C) is a procedure for testing whether a CS can acquire the reinforcing property of a US, by pairing a CS (CS1) with a US and then pairing another CS (CS2) with CS1 (Rescorla, 1988). Our model predicts that blockade of OA or DA receptors (on axon terminals of “CS” neurons) during the initial CS1–US pairing stage does not impair the enhancement of synapses from “CS” neurons to “OA/DA” neurons. This is because blockade of OA or DA receptors should not affect normal activities of “CS” neurons and “OA/DA” neurons (Figure (Figure6C).6C). Hence, CS1 should act as a reinforcer in the second CS1–CS2 pairing stage, although CS1 per se does not induce a CR (notice that impairments of first-order conditioning by DA/DA receptor antagonists are shown in Figures Figures22 and and3).3). Therefore, our model predicts that blockade of OA or DA receptors during the first CS1-US pairing stage does not impair second-order conditioning, but the same treatment during the second CS2–CS1 pairing stage or during the final retention test impairs second-order conditioning.

We first studied whether second-order conditioning can be achieved in crickets (Figure (Figure8;8; Mizunami et al., 2009). We used an olfactory stimulus as CS1 and a visual pattern as CS2. For appetitive or aversive conditioning, an odor (CS1) was paired with water or sodium chloride solution, respectively, and then a visual pattern (CS2) was paired with an odor (CS1). A group of animals that was subjected to appetitive second-order conditioning trials exhibited significantly increased preference for the CS2 (Figure (Figure8A).8A). In contrast, control groups that were each subjected to unpaired presentations of stimuli at the first or second conditioning stage exhibited no significantly increased preference for the CS2 (Figure (Figure8A),8A), thus indicating that the increased preference for the CS2 in the experimental group is truly the result of second-order conditioning. Similarly, we showed that second-order aversive conditioning could be achieved (Figure (Figure88B).

Figure 8
Appetitive (A) and aversive (B) second-order conditioning. Two groups of animals were each subjected to appetitive (A) or aversive (B) second-order conditioning trials (P/P groups). Four control groups were each subjected to unpaired presentations in ...

We then studied the effect of OA or DA receptor antagonist on appetitive or aversive second-order conditioning (Figure (Figure9).9). A group of animals injected with epinastine before the first conditioning stage of appetitive second-order conditioning exhibited a significantly increased preference for the CS2, thus indicating that blockade of OA receptor during the first stage does not impair appetitive second-order conditioning (Figure (Figure9A).9A). In contrast, the group injected with epinastine before the second stage (Figure (Figure9B)9B) or before final test (Figure (Figure9C)9C) for appetitive second-order conditioning exhibited no significantly increased preference for the CS2, indicating a complete impairment of second-order conditioning. Similarly, blockade of DA receptor antagonist at the first stage did not impair aversive second-order conditioning, but that at the second stage or final test impaired aversive second-order conditioning (Figures (Figures99D–F).

Figure 9
Octopamine and dopamine receptor antagonists impair appetitive and aversive second-order conditioning. (A–C) Three groups of animals were each injected with 3 μl of saline containing 1 μM epinastine at 30 min before the ...

The results were in full accordance with our model. In addition, the results were inconsistent with an alternative model in which different sets of OA/DA neurons participate in reinforcement and memory recall, because the model predicts that blockade of OA/DA receptors at any stage of conditioning impairs second-order conditioning.

Our finding that OA-ergic signaling and DA-ergic signaling are needed for appetitive and aversive memory recall, respectively, is in accordance with some previous findings in honey bees and fruit-flies but not with other findings, as we have discussed elsewhere (Mizunami et al., 2009). Thus, we proposed that activations of OA- or DA-ergic neurons are needed for memory recall in some forms of classical conditioning in insects but not in other forms (Mizunami et al., 2009). The critical factors for determining the requirement of OA- or DA-ergic signaling in appetitive/aversive memory recall remain to be clarified.

Future Perspectives

Many questions concerning the roles of OA- or DA-ergic neurons in insect learning remain to be addressed. Firstly, although there has been a substantial progress in identifying DA- and OA-ergic neurons involved in appetitive and aversive learning in honey bees (Hammer, 1993) and fruit-flies (Schroll et al., 2006; Claridge-Chang et al., 2009), more studies need to be performed to clarify the morphology and physiology of DA or OA-ergic neurons participating in insect learning. In crickets, it has been suggested that the mushroom bodies are involved in olfactory learning (in Acheta domesticus: Scotto-Lomassese et al., 2003), and thus OA- and DA-ergic neurons projecting to the mushroom bodies need to be anatomically and physiologically characterized. Detailed immunohistochemical studies for OA and DA will provide the first step for this. Secondly, we have proposed that different dynamics of appetitive and aversive memory are the result of different dynamics of biochemical processes after activation of OA and DA receptors, respectively (Nakatani et al., 2009). To examine this hypothesis, physiology, and biochemistry of neurons participating in association of CS and US, most probably Kenyon cells of the mushroom bodies, need to be characterized. Thirdly, our successful demonstration of cross-modal second-order conditioning suggests that OA- or DA-ergic neurons participating in olfactory learning also participate in visual pattern learning (Mizunami et al., 2009), and this suggestion needs to be examined by electrophysiologically characterizing responses of OA- and DA-ergic neurons during olfactory learning and visual learning. Fourthly, mammalian midbrain DA-ergic neurons have been suggested to convey prediction error for rewarding outcomes, namely, they encode the error between expected and actual rewarding outcomes (Schultz, 1998, 2006), and it is important to clarify whether or not OA-ergic (resp. DA-ergic) neurons in insects work in the same way or not in appetitive (resp. aversive) learning, respectively. Further examination of these questions should contribute to clarification of phylogenetically conserved principles of brain systems underlying learning and memory among vertebrates and invertebrates, as well as each specific functional specialization in each group.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Our study was supported by grants from the Japan Society for Promotion of Science. We thank Ai Hatano for providing data on one-trial aversive olfactory conditioning shown in Figure Figure44B.

References

  • Abrams T. W., Kandel E. R. (1988). Is contiguity detection in classical conditioning a system or a cellular property? Learning in Aplysia suggests a possible molecular site. Trends Neurosci. 11, 128–13510.1016/0166-2236(88)90137-3 [PubMed] [Cross Ref]
  • Andretic R., van Swinderen B., Greenspan R. J. (2005). Dopaminergic modulation of arousal in Drosophila. Curr. Biol. 15, 1165–1175 [PubMed]
  • Blenau W., Baumann A. (2001). Molecular and pharmacological properties of insect biogenic amine receptors: lessons from Drosophila melanogaster and Apis mellifera. Arch. Insect Biochem. Physiol. 48, 13–38 [PubMed]
  • Claridge-Chang A., Roorda R. D., Vrontou E., Sjulson L., Li H., Hirsh J., Miesenböck G. (2009). Writing memories with light-addressable reinforcement circuitry. Cell 139, 405–41510.1016/j.cell.2009.08.034 [PMC free article] [PubMed] [Cross Ref]
  • Crocker A., Shahidullah M., Levitan I. B., Sehgal A. (2010). Identification of a neural circuit that underlies the effects of octopamine on sleep:wake behavior. Neuron 65, 670–68110.1016/j.neuron.2010.01.032 [PMC free article] [PubMed] [Cross Ref]
  • Davis R. L. (2005). Olfactory memory formation in Drosophila: from molecular to systems neuroscience. Annu. Rev. Neurosci. 28, 275–302 [PubMed]
  • Daw N. D., Kakade S., Dayan P. (2002). Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–61610.1016/S0893-6080(02)00052-7 [PubMed] [Cross Ref]
  • Degen J., Gewecke M., Roeder T. (2000a). Octopamine receptors in the honey bee and locust nervous system: pharmacological similarities between homologous receptors of distantly related species. Br. J. Pharmacol. 130, 587–59410.1038/sj.bjp.0703338 [PMC free article] [PubMed] [Cross Ref]
  • Degen J., Gewecke M., Roeder T. (2000b). The pharmacology of a dopamine receptor in the locust nervous tissue. Eur. J. Pharmacol. 396, 59–6510.1016/S0014-2999(00)00182-5 [PubMed] [Cross Ref]
  • Driscoll M. P. (2005). Psychology of Learning for Instruction. Boston: Pearson
  • Dudai Y., Jan Y. N., Byers D., Quinn W. G., Benzer S. (1976). Dunce, a mutant of Drosophila deficient in learning. Proc. Natl. Acad. Sci. U.S.A. 73, 1684–1688 [PubMed]
  • Erber J., Masuhr T. H., Menzel R. (1980). Localization of short-term memory in the brain of the bee, Apis mellifera. Physiol. Entomol. 5, 343–358
  • Farooqui T., Robinson K., Vaessin H., Smith B. H. (2003). Modulation of early olfactory processing by an octopaminergic reinforcement pathway in the honeybee. J. Neurosci. 23, 5370–5380 [PubMed]
  • Gershoff E. T. (2002). Corporal punishment by parents and associated child behaviors and experiences: a meta-analytic and theoretical review. Psychol. Bull. 128, 539–579 [PubMed]
  • Gervasi N., Tchénio P., Preat T. (2010). PKA dynamics in a Drosophila learning center: coincidence detection by rutabaga adenylyl cyclase and spatial regulation by dunce phosphodiesterase. Neuron 65, 516–52910.1016/j.neuron.2010.01.014 [PubMed] [Cross Ref]
  • Giurfa M. (2003). Cognitive neuroethology: dissecting non-elemental learning in a honeybee brain. Curr. Opin. Neurobiol. 13, 726–735 [PubMed]
  • Hammer M. (1993). An identified neuron mediates the unconditioned stimulus in associative olfactory learning in honeybees. Nature 366, 59–6310.1038/366059a0 [PubMed] [Cross Ref]
  • Hammer M. R., Menzel R. (1998). Multiple sites of associative odor learning as revealed by local brain microinjections of octopamine in honeybees. Learn. Mem. 5, 146–156 [PubMed]
  • Harley C. W. (2004). Norepinephrine and dopamine as learning signals. Neural Plast. 11, 1–1410.1155/NP.2004.191 [PMC free article] [PubMed] [Cross Ref]
  • Heisenberg M. (2003). Mushroom body memoir: from maps to models. Nat. Rev. Neurosci. 4, 266–275 [PubMed]
  • Heisenberg M., Borst A., Wagner S., Byers D. (1985). Drosophila mushroom body mutants are deficient in olfactory learning. J. Neurogenet. 2, 1–3010.3109/01677068509100140 [PubMed] [Cross Ref]
  • Holland P. (2008). Cognitive versus stimulus-response theories of learning. Learn. Behav. 36, 227–241 [PMC free article] [PubMed]
  • Honjo K., Furukubo-Tokunaga K. (2009). Distinctive neuronal networks and biochemical pathways for appetitive and aversive memory in Drosophila larvae. J. Neurosci. 29, 852–86210.1523/JNEUROSCI.1315-08.2009 [PubMed] [Cross Ref]
  • Hoyer S. C., Eckart A., Herrel A., Zars T., Fischer S. A., Hardie S. L., Heisenberg M. (2008). Octopamine in male aggression of Drosophila. Curr. Biol. 18, 159–167 [PubMed]
  • Kandel E. R. (2001). The molecular biology of memory storage: a dialogue between genes and synapses. Science 294, 1030–103810.1126/science.1067020 [PubMed] [Cross Ref]
  • Keene A. C., Waddell S. (2007). Drosophila olfactory memory: single genes to complex neural circuits. Nat. Rev. Neurosci. 8, 341–354 [PubMed]
  • Kim Y. C., Lee H. G., Han K. A. (2007). D1 dopamine receptor dDA1 is required in the mushroom body neurons for aversive and appetitive learning in Drosophila. J. Neurosci. 27, 7640–764710.1523/JNEUROSCI.1167-07.2007 [PubMed] [Cross Ref]
  • Kreissl S., Eichmüller S., Bicker G., Rapus J., Eckert M. (1994). Octopamine-like immunoreactivity in the brain and subesophageal ganglion of the honeybee. J. Comp. Neurol. 348, 583–595 [PubMed]
  • Kuwabara M. (1957). Bildung des bedingten Reflexes von Pavlovs Typus bei der Honigbiene, Apis mellifera. J. Fat. Sci. Hokkaido Univ. Ser. VI. 13, 458–464
  • Matsumoto Y., Hatano A., Unoki S., Mizunami M. (2009). Stimulation of the cAMP system by the nitric oxide-cGMP system underlying the formation of long-term memory in an insect. Neurosci. Lett. 467, 81–85 [PubMed]
  • Matsumoto Y., Mizunami M. (2000). Olfactory learning in the cricket Gryllus bimaculatus. J. Exp. Biol. 203, 2581–2588 [PubMed]
  • Matsumoto Y., Mizunami M. (2002a). Temporal determinants of olfactory long-term retention in the cricket Gryllus bimaculatus. J. Exp. Biol. 205, 1429–1437 [PubMed]
  • Matsumoto Y., Mizunami M. (2002b). Lifetime olfactory memory in the cricket Gryllus bimaculatus. J. Comp. Physiol. A. 188, 295–299 [PubMed]
  • Matsumoto Y., Mizunami M. (2004). Context-dependent olfactory learning in an insect. Learn. Mem. 11, 288–293 [PubMed]
  • Matsumoto Y., Mizunami M. (2006). Olfactory memory capacity of the cricket Gryllus bimaculatus. Biol. Lett. 2, 608–610 [PMC free article] [PubMed]
  • Matsumoto Y., Noji S., Mizunami M. (2003). Time course of protein synthesis-dependent phase of olfactory memory in the cricket Gryllus bimaculatus. Zool. Sci. 20, 409–416 [PubMed]
  • Matsumoto Y., Unoki S., Aonuma H., Mizunami M. (2006). Critical role of nitric oxide-cGMP cascade in the formation of cAMP-dependent long-term memory. Learn. Mem. 13, 35–44 [PubMed]
  • Menzel R., Giurfa M. (2006). Dimensions of cognition in an insect, the honeybee. Behav. Cogn. Neurosci. Rev. 5, 24–40 [PubMed]
  • Menzel R., Leboulle G., Eisenhardt D. (2006). Small brains, bright minds. Cell 124, 237–23910.1016/j.cell.2006.01.011 [PubMed] [Cross Ref]
  • Mizunami M., Okada R., Li Y., Strausfeld N. J. (1998b). Mushroom bodies of the cockroach: The activity and identities of neurons recorded in freely moving animals. J. Comp. Neurol. 402, 501–51910.1002/(SICI)1096-9861(19981228)402:4<501::AID-CNE5>3.3.CO;2-D [PubMed] [Cross Ref]
  • Mizunami M., Unoki S., Mori Y., Hirashima D., Hatano A., Matsumoto Y. (2009). Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in an insect. BMC Biol. 7, 46.10.1186/1741-7007-7-46 [PMC free article] [PubMed] [Cross Ref]
  • Mizunami M., Weibrecht J. M., Strausfeld N. J. (1998a). Mushroom bodies of the cockroach: Their participation in place memory. J. Comp. Neurol. 402, 520–53710.1002/(SICI)1096-9861(19981228)402:4<520::AID-CNE6>3.3.CO;2-B [PubMed] [Cross Ref]
  • Mizunami M., Yokohari F., Takahata M. (1999). Exploration into the adaptive design of the arthropod “microbrain.” Zool. Sci. 16, 703–709 [PubMed]
  • Mizunami M., Yokohari F., Takahata M. (2004). Further exploration into the adaptive design of the arthropod “microbrain”: I. Sensory and memory-processing systems. Zool. Sci. 21, 1141–1151 [PubMed]
  • Mustard J. A., Blenau W., Hamilton I. S., Ward V. K., Ebert P. R., Mercer A. R. (2003). Analysis of two D1-like dopamine receptors from the honeybee Apis mellifera reveals agonist-independent activity. Mol. Brain Res. 113, 67–7710.1016/S0169-328X(03)00091-3 [PubMed] [Cross Ref]
  • Nakatani Y., Matsumoto Y., Mori Y., Hirashima D., Nishino H., Arikawa K., Mizunami M. (2009). Why the carrot is more effective than the stick: Different dynamics of punishment memory and reward memory and its possible biological basis. Neurobiol. Learn. Mem. 92, 370–380 [PubMed]
  • Okada R., Ikeda J., Mizunami M. (1999). Sensory responses and movement-related activities in extrinsic neurons of the cockroach mushroom bodies. J. Comp. Physiol. A. 185, 115–129
  • Peine H. A., Howarth R. (1975). Children and Parents. Baltimore: Penguin Books; [PubMed]
  • Pickens C. L., Holland P. C. (2004). Conditioning and cognition. Neurosci. Behav. Rev. 28, 651–661 [PubMed]
  • Rescorla R. A. (1988). Pavlovian conditioning. Am. Psychol. 40, 151–16010.1037/0003-066X.43.3.151 [PubMed] [Cross Ref]
  • Riemensperger T., Völler T., Stock P., Buchner E., Fiala A. (2005). Punishment prediction by dopaminergic neurons in Drosophila. Curr. Biol. 15, 1953–1960 [PubMed]
  • Roberts A. C., Glanzman D. L. (2003). Learning in Aplysia: looking at synaptic plasticity from both sides. Trends. Neurosci. 26, 662–670 [PubMed]
  • Roeder T. (1999). Octopamine in invertebrates. Prog. Neurobiol. 59, 533–561 [PubMed]
  • Roeder T., Degen J., Gewecke M. (1998). Epinastine, a highly specific antagonist of insect neuronal octopamine receptors. Eur. J. Pharmacol. 349, 171–177 [PubMed]
  • Schroll C., Riemensperger T., Bucher D., Ehmer J., Völler T., Erbguth K., Gerber B., Hendel T., Nagel G., Buchner E., Fiala A. (2006). Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae. Curr. Biol. 16, 1741–1747 [PubMed]
  • Schultz W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 [PubMed]
  • Schultz W. (2006). Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–11510.1146/annurev.psych.56.091103.070229 [PubMed] [Cross Ref]
  • Schwaerzel M., Monastirioti M., Scholz H., Friggi-Grelin F., Birman S., Heisenberg M. (2003). Dopamine and octopamine differentiate between aversive and appetitive olfactory memories in Drosophila. J. Neurosci. 23, 10495–10502 [PubMed]
  • Scotto-Lomassese S., Strambi C., Strambi A., Aouane A., Augier R., Rougon G., Cayre M. (2003). Suppression of adult neurogenesis impairs olfactory learning and memory in an adult insect. J. Neurosci. 23, 9289–9296 [PubMed]
  • Selcho M., Pauls D., Han K. A., Stocker R. F., Thum A. S. (2009). The role of dopamine in Drosophila larval classical olfactory conditioning. PLoS ONE 4, e5897.10.1371/journal.pone.0005897 [PMC free article] [PubMed] [Cross Ref]
  • Sitaraman D., Zars M., Laferriere H., Chen Y. C., Sable-Smith A., Kitamoto T., Rottinghaus G. E., Zars T. (2008). Serotonin is necessary for place memory in Drosophila. Proc. Natl. Acad. Sci. U.S.A. 105, 5579–5584 [PubMed]
  • Stevenson P. A., Dyakonova V., Rillich J., Schildberge K. (2005). Octopamine and experience-dependent modulation of aggression in crickets. J. Neurosci. 25, 1431–134110.1523/JNEUROSCI.4258-04.2005 [PubMed] [Cross Ref]
  • Takahashi T., Hamada A., Miyawaki K., Matsumoto Y., Mito T., Noji S., Mizunami M. (2009). Systemic RNA interference for the study of learning and memory in an insect. J. Neurosci. Methods 179, 9–15 [PubMed]
  • Tempel B. L., Bonini N., Dawson D. R., Quinn W. G. (1983). Reward learning in normal and mutant Drosophila. Proc. Natl. Acad. Sci. U.S.A. 80, 1482–1486 [PubMed]
  • Tomchik S. M., Davis R. L. (2009). Dynamics of learning-related cAMP signaling and stimulus integration in the Drosophila olfactory pathway. Neuron 64, 510–52110.1016/j.neuron.2009.09.029 [PubMed] [Cross Ref]
  • Tully T., Quinn W. G. (1985). Classical conditioning and retention in normal and mutant Drosophila melanogaster. J. Comp. Physiol. A. 157, 263–277 [PubMed]
  • Unoki S., Matsumoto Y., Mizunami M. (2005). Participation of octopaminergic reward system and dopaminergic punishment system in insect olfactory learning revealed by pharmacological study. Eur. J. Neurosci. 22, 1409–1416 [PubMed]
  • Unoki S., Matsumoto Y., Mizunami M. (2006). Roles of octopaminergic and dopaminergic neurons in mediating reward and punishment signals in insect visual learning. Eur. J. Neurosci. 24, 2031–2038 [PubMed]
  • Vergoz V., Roussel E., Sandoz J-C., Giurfa M. (2007). Aversive learning in honeybees revealed by the olfactory conditioning of the sting extension reflex. PLoS ONE 2, e288.10.1371/journal.pone.0000288 [PMC free article] [PubMed] [Cross Ref]
  • Walters G. C., Crusec J. E. (1977). Punishment. San Francisco: Freeman
  • Wessnitzer J., Mangan M., Webb B. (2008). Place memory in crickets. Proc. R. Sci. B. 275, 915–921 [PMC free article] [PubMed]
  • Wise R. A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494 [PubMed]
  • Zhou C., Rao Y., Rao Y. (2008). A subset of octopaminergic neurons are important for Drosophila aggression. Nat. Neurosci. 11, 1059–1167 [PubMed]

Articles from Frontiers in Behavioral Neuroscience are provided here courtesy of Frontiers Media SA