We investigated whether dopamine signaling is required for overcoming absolute and relative response costs for rats performing an instrumental lever-pressing task for food reward. Rats exerting a substantial amount of effort for each food reward (group FR-20/FR-20) were more likely to reduce their performance when pretreated with the flupentixol than rats performing low-effort (group FR-1/FR-1) or intermediate-effort (group FR-10/FR-10) tasks. However, when all rats were tested on the intermediate-effort task, flupentixol pretreatment was most effective in suppressing reward seeking in those rats experiencing an upshift in effort (group FR-1/FR-10). Furthermore, compared with rats maintained on the intermediate-effort schedule (group FR-10/FR-10), rats experiencing a downshift in effort (group FR-20/FR-10) were protected from the suppressive effects of flupentixol. These findings demonstrate that dopamine signaling is critical for adapting to changes in task demands and suggest that these relative, or subjective, response costs have a fundamental role in decision-making, even when the absolute, or objective, costs are held constant.
Interestingly, both individual and group differences in task efficiency, measured as the average number of presses performed before a rat checked the food cup, correlated with the dopamine-dependence of task performance. Thus, it appears that the way in which rats organized their reward seeking behavior influenced that behavior's vulnerability to disruption by dopamine receptor blockade. This relationship is consistent with three substantively different accounts that deserve consideration. First, if it is reasonable to take the press-to-approach ratio as a measure of how readily a rat adapts to a change in response requirement, this pattern suggests that rats experiencing the strongest ratio strain (ie, the most difficulty adapting to an upshift in response requirement) also showed the greatest sensitivity to dopamine antagonism. This interpretation is bolstered by the finding that rats shifted from FR-20 down to FR-10 at test exhibited the highest press-to-approach ratio and were also least affected by flupentixol. However, an alternative interpretation along similar lines is that these differences in the press-to-approach ratio represent objective, rather than subjective, differences in response cost since it is likely that premature visits to the food cup increase the net energy expenditure for the task. While these accounts focus on the role of costs (either relative or absolute) in determining the dopamine-dependence of performance, the flexible approach hypothesis (Nicola, 2010
) provides a categorically different account. This framework assumes that dopamine transmission is required for initiating original (ie, non-habitual or undertrained) action sequences, particularly when subjects have disengaged from the instrumental task and must navigate back to the manipulandum from a new location in order to earn reward (Nicola, 2010
). To explain why high-effort tasks tend to be more sensitive to dopamine receptor antagonism than low-effort tasks, this account notes that high-effort tasks cause subjects to disengage from the task following reward delivery (ie, the post-reinforcement pause). It is the ability to reinitiate performance during such pauses that is supposedly compromised by dopamine receptor antagonism. By extension, the flexible approach hypothesis may explain the current finding that an upshift in effort increased the dopamine-dependence of instrumental performance, if it can be assumed that this manipulation caused rats' in group FR-1/FR-10 to disengage from the task. Although it is difficult to fully evaluate this account without detailed analysis of individual rats' locomotion in the operant chamber at test, the finding that upshifted rats tended to perform the task inefficiently, making unnecessary trips to the food cup between reinforced presses, indicates that they were more frequently required to transition back from the food cup to the lever than unshifted or downshifted rats. However, it is important to note that no statistically significant differences in session length were detected between groups during the control (saline) relative response cost test. Indeed, a more detailed analysis of the rats' mean latency to return to lever pressing after checking the food cup (regardless of whether or not a reward was delivered) during this test suggests that group FR-1/FR-10 (2.79
s; ±0.37) showed a tendency to be, if anything, more efficient in re-initiating task performance than group FR-20/FR-10 (4.23
s; ±0.51; Bonferroni post-hoc
=0.05) or group FR-10/FR-10 (3.39
s; ±0.32; p
>0.10). Therefore, although the current findings do not provide a critical test of the flexible approach hypothesis, they seem to be more compatible with an effort-based interpretation of dopamine function.
The finding that dopamine signaling has a role in processing response costs is consistent with a number of recent studies measuring dopamine signaling in the nucleus accumbens. For instance, one study using microdialysis to track session-to-session changes in task-related dopamine efflux in rats lever pressing for food reward on a random ratio schedule found that differences in dopamine efflux across sessions tracked fluctuations in experienced response cost (Ostlund et al, 2011
). Specifically, increases in the average number of presses required to earn reward were associated with decreases in dopamine efflux. These changes in dopamine may provide an incentive motivational function, in line with a recent computational model of tonic dopamine (Niv et al, 2007
). Thus, when adapting to a change in work requirement, increases in dopamine levels may invigorate behavior when rewards are cheap, whereas decreases in dopamine may discourage responding when rewards are rare or costly. These short-term fluctuations in dopamine signaling can be contrasted with patterns observed in situations involving well-trained subjects. In such cases dopamine efflux during instrumental performance tests tends to be greater for rats trained on high-effort tasks than for those trained on low-effort tasks (Salamone et al, 1994b
; Sokolowski et al, 1998
; Segovia et al, 2011
). Thus, it may be that short-term fluctuations in dopamine related to unexpected changes in response cost are countered by a more slowly acquired pattern of dopamine signaling (Ahn and Phillips, 2007
), which may be critical for overcoming more persistent response costs.
Dopamine's role in processing relative response costs may also explain its contributions to cost/benefit decision-making. It is well established that treatments that disrupt dopamine signaling bias rats towards less costly response options (Cousins and Salamone, 1994
; Denk et al, 2005
; Floresco et al, 2008a
). Furthermore, studies measuring phasic mesolimbic dopamine signaling during cost-based decision making tasks have found some, albeit mixed, evidence that anticipatory dopamine responses are lower for high-cost options, independent of changes related to reward magnitude (Day et al, 2010
; Gan et al, 2010
; Wanat et al, 2010
; Nasrallah et al, 2011
). Importantly, for two-option choice tasks, absolute and relative costs are necessarily conflated. For instance, when choosing between options that require either 16 presses or a single press to produce reward, the difference in absolute cost may be used to encode relative costs (eg, Option A is 16 times harder than Option B). Similarly, the progressive ratio task, where each new reward becomes harder to obtain, also conflates relative and absolute costs as subjects are likely tracking changes in effort over the session. To dissociate these two aspects of response cost, we applied a within-subjects procedure to manipulate relative response costs while holding absolute costs constant across groups, allowing us to attribute group differences in sensitivity to dopamine receptor antagonism to differences in relative response cost.
Although dopamine appears to incorporate information about past and present task conditions to regulate reward seeking, it is not required for learning about the incentive value of instrumental goals (Dickinson et al, 2000
; Wassum et al, 2011
) or using goal representations to guide action selection (Ostlund and Maidment, 2012
), suggesting that dopamine tracks nonspecific information about reward value and response costs. Although these variables are the direct products of instrumental action, the dopamine system may encode such information through a Pavlovian learning process in the background of instrumental learning. This would explain reports that dopamine receptor antagonism selectively disrupts the response-invigorating effects of Pavlovian, reward-paired cues on instrumental reward seeking (Dickinson et al, 2000
; Lex and Hauber, 2008
; Wassum et al, 2011
; Ostlund and Maidment, 2012
The current results demonstrate that the dopamine-dependence of instrumental performance is modulated by both its absolute and relative costs. Advancing our understanding of dopamine's role in reward seeking will likely require a deeper appreciation for how such costs are computed and implemented to guide decision-making. Such knowledge could be useful for developing treatments to encourage healthy but effortful behaviors and/or discourage unhealthy behaviors (eg, compulsive drug or food seeking) associated with dysregulated dopamine signaling.