PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Behav Processes. Author manuscript; available in PMC Jun 1, 2009.
Published in final edited form as:
PMCID: PMC2488404
NIHMSID: NIHMS50768
The Stay/Switch Model Describes Choice Among Magnitudes of Reinforcers
James S. MacDonall
James S. MacDonall, Fordham University;
James S. MacDonall, Department of Psychology, Fordham University, 441 E. Fordham Road, Bronx, NY 10458, Voice: 718 817 3880, Fax: 718 817 3785, Email: jmacdonall/at/fordham.edu
The stay/switch model is an alternative to the generalized matching law for describing choice in concurrent procedures. The purpose of the present experiment was to extend this model to choice among magnitudes of reinforcers. Rats were exposed to conditions in which the magnitude of reinforcers (number of food pellets) varied for staying at Alternative 1, switching from Alternative 1, staying at Alternative 2 and switching from Alternative 2. A changeover delay was not used. The results showed that the stay/switch model provided a good account of the data overall, and deviations from fits of the generalized matching law to response allocation data were in the direction predicted by the stay/switch model. In addition, comparisons among specific conditions suggested that varying the ratio of obtained reinforcers, as in the generalized matching law, was not necessary to change the response and time allocations. Other comparisons suggested that varying the ratio of obtained reinforcers was not sufficient to change response allocation. Taken together these results provide additional support for the stay/switch model of concurrent choice.
Keywords: choice, concurrent schedule, generalized matching law, magnitude of reinforcement, random-interval schedule, rats
Experiments examining choice continue more than 40 years after Herrnstein’s seminal paper “Relative and Absolute Strength of Response as a Function of Frequency of Reinforcement” (1961). Almost fifteen years later, Baum (1974) proposed the generalized matching law as a mathematical model that could describe choice among two different rates of reinforcers. This model can be expressed as,
equation M1
(1)
where Bn is the responses or time at alternative n and Rn is the reinforcers obtained at alternative n. The two fitted parameters are s, which is behavioral sensitivity to the reinforcer ratio, and log b, which is the bias towards an alternative not accounted for by the reinforcer ratio. Biases are interpreted as factors which affect choice but are unrelated to the reinforcer ratio, such as preexisting preferences for a response or stimulus signaling the alternative, differences in the manipulanda or topographies of the responses at the manipulanda (Baum, 1974). This approach to studying choice using concurrent schedules implicitly assumes the choice is between responding at one of the two alternatives (Sugrue, Corrado, & Newsome, 2004).
An alternative view of choice is the stay/switch model that differs from the generalized matching law in two important ways. First, this model was developed from the idea that the fundamental choice was between staying at and switching from each alternative (MacDonall et al, 2006) rather than between responding at one alternative or the other alternative. Second, according to the stay/switch model, the choice is between earning reinforcers (for staying and switching) rather than obtaining reinforcers (for responding at an alternative).
The distinction between earning and obtaining reinforcers was made explicit by Rachlin, Green and Tormey (1988). It is important, however, to recognize that all reinforcers are both earned and obtained. On simple ratio schedules responding earns reinforcers that are obtained by the same operant that earned them. On simple interval schedules responding during the session earns reinforcers that are obtained by some of those responses. A differential-reinforcement-of-low-rate schedule shows that reinforcers are earned on interval-type schedules. Engaging in behavior other than the operant for a specified interval earns a reinforcer that is obtained by the operant. Earning reinforcers is also clear in concurrent interval schedules. Time at an alternative earns reinforcers that are obtained by a response at that alternative: The next response at that alternative obtains that earned reinforcer. Time at an alternative also earns reinforcers that are obtained at the other alternative. When the subject switches to the other alternative an earned reinforcer is obtained, possibly after a changeover delay (COD), which requires a minimum time at an alternative before a reinforcer can be obtained. When a subject does not spend time at an alternative the reinforcers for staying at and switching from that alternative cannot be earned or obtained. The generalized matching law and the stay/switch model both allocate all reinforcers to describe choice. They differ in that the generalized matching law allocates all reinforcers based on where they were obtained and the stay/switch model allocates all reinforcers based on where they were earned.
The kernel of the stay/switch model is the choice between staying at and switching from an alternative and can be expressed as
equation M2
(2)
Bt and Bw represent the number of stay and switch responses, respectively, and Rt and Rw represent the number of stay and switch reinforcers earned at the alternative, respectively (MacDonall, 2006). The lower case ‘t’ and ‘w’ are used to refer to ‘staying’ and ‘switching,’ respectively. The three fitted parameters are l’, the sensitivity to the stay to switch reinforcer ratio, m’, the sensitivity to the sum of the stay and switch reinforcers earned per visit, and k’, the tendency to respond that is not otherwise explained by the other two expressions in the equation. This equation says that the average number of stay responses at an alternative (i.e., run lengths), and the average time at an alternative (i.e., visit durations), are power functions of the ratio of the reinforcers earned for staying at and switching from the alternative and the sum of the reinforcers earned for stay at and switching from the alternative.
A model of concurrent choice is obtained by dividing this equation with subscripts for one alternative by this equation with subscripts for the other alternative. Assuming the tendency to respond, k’, is the same at both alternatives and substituting k, sensitivities to the ratio of the reinforcers, l’, are the same at both alternatives and substituting l, and the sensitivities to the sum of the reinforcers, m’, are the same at both alternatives and substituting m, then simplifying results in the following equation in logarithmic form,
equation M3
(3)
This equation says the log of the ratio of stay responding is a linear function of the log of the ratio of the ratio of stay to switch reinforcers multiplied by a constant, plus the ratio of the sum of the stay and switch reinforcers multiplied by a constant, plus a constant.
Concurrent interval schedules are usually conceived as consisting of two simultaneously operating interval schedules, each associated with one alternative. According to the stay/switch model, however, concurrent interval schedules actually consist of two pairs of schedules. Each pair is associated with one alternative and only operates when the subject is at that alternative. Each pair is comprised of a schedule that arranges reinforcers for staying at the associated alternative, the stay schedule, and a schedule that arranges reinforcers for switching from that alternative, the switch schedule. In standard concurrent schedules the value of the stay schedule in one pair equals the value of the switch schedule in the other pair. This is called the symmetrical arrangement of schedules. The schedules do not have to be arranged symmetrically. In a different arrangement the value of the stay schedule in each pair equals the value of the switch schedule in the same pair, and was called the asymmetrical arrangement (MacDonall, 2005).
Recent research provides support for the stay/switch model. Two experiments exposed rats to several conditions using symmetrical arrangements and several conditions using asymmetrical arrangements. One experiment used concurrent interval schedules (MacDonall, 2005) and the other experiment used concurrent ratio schedules, programmed so they operated they same way that concurrent interval schedules operate (MacDonall et al, 2006). Responding at each alternative incremented stay and switch counters associated with that alternative, just as time spent at each alternative increments stay and switch timers associated with that alternative. In both experiments, the generalized matching law provided good descriptions of data from symmetrical conditions; however, it provided poor descriptions of each rat’s pooled data from both sets of conditions. The generalized matching law did describe the time and response data from the symmetrical conditions. The stay/switch model, in contrast, provided adequate descriptions of data pooled from both sets of conditions. In two additional experiments, again using either concurrent interval schedules (MacDonall) or concurrent ratio schedules (MacDonall et al.), the values of the stay and switch schedules associated with one alternative were a constant multiple of the schedules associated with the other alternative. This was called a weighted arrangement. Each experiment exposed rats to several conditions using symmetrical arrangements and several conditions using weighted arrangements. The generalized matching law provided poor descriptions of each rat’s data from all conditions, whereas, the stay/switch model provided adequate descriptions of these data.
Most research in concurrent choice has examined the relation between rates of reinforcers at the alternatives, however, additional research has shown that other features of reinforcement including, magnitude, delay and quality of reinforcement also influence choice. Several investigators found that the matching law (Herrnstein, 1961; Keller & Gollub, 1977; Neuringer, 1967; Todorov, 1973; Todorov, Hanna, & Bittencourt de Sa, 1984) and the generalized matching law (Davison & Baum, 2003; Landon, Davison, & Elliffe, 2003) described response or time allocations when magnitudes of reinforcers vary at the alternatives. The primary purpose of the present experiment was to extend the stay/switch model to choice when varying magnitudes of reinforcers. If the results showed that the stay/switch model described these data then a secondary purpose was to assess whether changing the ratio of magnitudes of reinforcers in the generalized matching law was necessary or sufficient for influencing behavior allocation. To prevent varying rates of reinforcers from also influencing choice non-independent scheduling was used (Stubbs & Pliskoff, 1969). In non-independent scheduling all interval schedules stop operating when one schedule arranges a reinforcer. In this way, the obtained reinforcers equal the scheduled reinforcers for staying at and for switching from each alternative. An additional benefit of non-independent scheduling, although seldom recognized, is that the allocation of earned reinforcers is a true independent variable. Using non-independent scheduling may drive preference towards indifferent and risks finding no differences between the models.
In all conditions 18 reinforcers were obtained for staying at the left lever, for switching from the left lever, for staying at the right lever and for switching from the right lever by using non-independent scheduling of the reinforcers. Rats were exposed to 9 or 10 conditions with varying magnitudes of reinforcers for staying at the left lever, switching from the left lever, staying at the right lever, and switching from the right lever. Four different methods were used to produce the arrangements of reinforcers, resulting in four types of conditions. Because the number of reinforcers was the same for each of the four operants in each condition the descriptions of the conditions will only focus on the arrangement of the magnitudes of reinforcers.
In the symmetrical arrangements, the magnitudes of the reinforcers earned from the stay schedule in one pair equaled the magnitudes of the reinforcers earned from the switch schedule in the other pair. This is the arrangement in standard concurrent schedules that vary magnitudes of reinforcers. If the generalized matching law describes these data then a failure of the generalized matching to describe data from other arrangements cannot be due to using two pairs of schedules.
If obtaining different magnitudes of reinforcers at the alternatives are sufficient to produce preference then in the sufficient arrangements there should be a preference towards the alternative where a larger magnitude of reinforcers is obtained. If the stay/switch model identifies the critical variables then there will be no preference and the deviations from the predictions by the generalized matching law will be in the direction predicted by the stay/switch model.
If obtaining different magnitudes of reinforcers at the alternatives is necessary to produce a preference then in the necessary arrangements there should not be a preference because the same magnitudes of reinforcers are obtained at both alternatives. If the stay/switch model identifies the critical variables then there will be a preference towards one alternative and the deviations from predictions by the generalized matching law will be in the direction predicted by the stay/switch model.
Because additional conditions were needed to quantitatively evaluate the stay/switch model two unsymmetrical arrangements were used. These arrangements the magnitudes of reinforcers were different than the previous three arrangements.
Subjects
The subjects were 6 naïve male Sprague-Dawley rats obtained from Hilltop Lab Animals (Scottdale, PA) and deprived to 85% of their just determined free-feeding weights. Following sessions they were fed 5 to 10 grams of food to maintain them at their 85% weights. They were approximately 100 days old when the experiment began and were housed individually in a temperature-controlled colony room on a 14-hr light/10 hr dark cycle beginning with lights on at 5 am. They were housed singly and had free access to water in their home cages.
Apparatus
Two operant conditioning chambers were used. Each chamber was approximately 225 mm wide, 195 mm high and 235 mm long. Each chamber was located in a light- and sound-controlled box. The 50-mm square opening for the food cup was centered horizontally on one 225- x 195-mm wall, 20 mm above the floor. Two response levers (Model G6312, R. Gerbrands Co.), 45 mm long by 13 mm thick, protruded 15 mm into the chamber. The centers of the levers were 60 mm to the left or right of the center of the food cup and 50 mm above the floor. Each lever required a force of approximately 0.3 N to operate. A feeder (Model ENV-203–20; Med Associates, St. Albans Vt.), located behind the wall containing the food cup, dispensed 20-mg food pellets (Formula A/1, P. J. Noyes Co.), which were 85% Purina rodent chow. A 24-V DC stimulus light was centered approximately 75 mm above each lever. Both chambers were illuminated during sessions by a pair of houselights mounted on the top center of the chamber. White noise was presented through a speaker centered between the houselights. An IBM-compatible computer and MED-PC software and hardware (MED Associates Inc.) controlled the experimental events and recorded responses.
Procedure
At the beginning of the experiment rats were placed in the operant conditioning chambers and a single random-interval (RI) 5 s schedule was operating. When a reinforcer was arranged the next response at either lever was reinforced (independent scheduling). Going to the food cup at the sound of the click was not trained and their behavior was not shaped to press the lever. The magnitudes of reinforcers were the same as listed for the first condition in Table 1. In the session after rats made 100 presses the values of the schedules were increased over three to five sessions to RI 80 s. After several sessions of RI 80 s, the schedule was changed to a single RI 60 s to be sure the same numbers of reinforcers were delivered for each of the four operants in each session. Now the single RI schedule was arranged non-independently, when a reinforcer was arranged the RI schedule stopped until the arranged reinforcer was delivered. In addition, a RI 60 s provided shorter session times.
Table 1
Table 1
The sequence of conditions and the value of the log reinforcer ratio in the generalized matching law, the log of the ratio of ratios, the log of the ratio of sums and the sum of these log ratios in the stay/switch model.
Rats were trained on a concurrent schedule programmed so that different magnitudes of reinforcers could be earned for staying at and for switching from each alternative while keeping the overall of number of reinforcer deliveries in sessions constant. The first response at either the left or right lever started the RI 60 s schedule. The reinforcers were scheduled non-independently, when the schedule arranged a reinforcer it stopped and a list was randomly sampled, without replacement, and the reinforcement of the corresponding response was arranged. The list contained 6 of each of the four types of responses (stay at left, switch from left, stay at right and switch from right). After reinforcement the RI 60-s schedule resumed. After all 24 reinforcers were selected the sampling continued with all 24 reinforcers in the list. Once a reinforcer for staying was arranged it was obtained by a response at the associated alternative, provided it was not the first response after responding at the other alternative (a switch response). If the rat was at the other alternative the rat had to switch alternatives, by pressing once at the switched to alternative and then press again to obtain the stay reinforcer. When a reinforcer for switching from either alternative was selected the first response at the designated alternative obtained that reinforcer. If the rat already was at that alternative the rat had to switch to the other alternative and then switch back to that alternative to obtain that switch reinforcer. There was no COD. The only difference among the conditions was the magnitudes of the stay and switch reinforcers at each alternative.
Rats were first exposed to three or four Symmetrical Conditions. Then they were exposed to two Sufficient Conditions, followed by two Necessary Conditions and finally two Unsymmetrical Conditions (Table 1). In the symmetrical arrangements the magnitude of the stay reinforcer in each pair equaled the magnitude of the switch reinforcer in the other pair, which is the arrangement of magnitudes in standard concurrent schedules providing different magnitudes of reinforcers.
In the Sufficient Conditions, the arrangement of the magnitudes of reinforcers were selected so that 1) a larger overall magnitude of reinforcers would be obtained at one alternative, 2) the ratio of magnitudes of earned reinforcers was larger at one alternative, and 3) the sums of the magnitudes of earned reinforcers were larger at the opposite alternative. Table 1 shows these ratios for each condition. If the latter two variables determine preference and if the preferences are for opposite alternatives there may be no preference for either alternative, even though a larger magnitude of reinforcers was obtained at one alternative. For example, in Condition 5 (see Table 1), the 12 pellets obtained at the left alternative (2 pellets plus 10 pellets) was greater than the 4 pellets obtained at the right alternative (2 pellets plus 2 pellets). If the ratio of obtained reinforcers is sufficient to produce a preference then this will produce preference for the left alternative. In this arrangement, however, the ratio of the magnitudes of reinforcers is 1 (2/2) at the left alternative and 0.2 (2/10) at the right alternative. If the ratio of magnitudes is a critical variable then it would drive preference towards the left alternative. The sum of the magnitudes is 4 (2+2) at the left alternative and 12 (2+10) at the right alternative. If the sum of magnitudes is a critical variable then it would drive preference towards the right alternative. These preferences for opposite alternatives could cancel yielding no preference for either alternative, even though a larger magnitude of reinforcers was obtained at the left alternative. This would indicate that obtaining different overall magnitudes of reinforcers is not sufficient for producing preference. A bias for the alternative where the smaller overall magnitudes of reinforcers were obtained (the right alternative in this example) could cancel the preference for the other (left) alternative produced by the obtaining the larger magnitude of reinforcers. To control for a possible bias the values of the magnitudes at the alternatives were reversed. Finding no preference in both conditions indicates that the lack of preference cannot result from a bias canceling out the effect of obtaining the larger overall magnitude of reinforcers.
In the Necessary Conditions, the arrangement of the magnitudes of reinforcers was selected so that 1) the same magnitude of reinforcers was obtained at both alternatives, 2) the ratio magnitudes was larger at one alternative, and 3) the sum of magnitudes was larger at the same alternative. If the latter two variables determine preference then there will be a preference for one alternative even though the same magnitudes of reinforcers were obtained at the alternatives. For example, in Condition 7 the 8 pellets obtained at the left alternative (1 + 7) equals the 8 pellets obtained at the right alternative (2 + 6). If the ratio of obtained magnitudes is necessary to produce preference then there will be no preference because the same magnitudes would be obtained at each alternative. In this arrangement, however, the ratio of the magnitudes is 0.5 (1/2) at the left alternative and 0.86 (6/7) at the right alternative. The ratio is larger at the right alternative which drives preference there. The sum of the magnitudes is 3 at the left alternative (1+2) and 13 (6+7) at the right alternative. The sum is larger at the right alternative which drives preference there. These preferences for the same alternative combine producing a preference for one alternative even though the same magnitudes were obtained at both alternatives. A bias can also produce a preference, even when obtaining equal overall magnitudes of reinforcers at the alternatives. To control for a possible bias in the subsequent condition the magnitudes of reinforcers at the alternatives were reversed. If the preference also reversed then a bias was not producing the preference. Finding a preference for one alternative and then the other when the overall magnitudes of reinforcers obtained at both alternatives were the same means varying the magnitude of reinforcers, as in the generalized matching law, was not necessary for influencing choice.
In the Unsymmetrical Conditions the magnitudes were selected so that 1) the value of the ratio of obtained magnitudes drives preference toward one alternative, 2) the value of the ratio of ratios drives preference towards the same alternative, and 3) the value of the ratio of sums also drives preference towards that same alternative. In this way a different combination of magnitudes allows an assessment of a larger area defined by the ratio of ratios and the ratio of sums in Equation 5 (see below).
The RI schedule was arranged by probing a probability generator every 0.5 s, thus the minimum time after a reinforcer delivery was completed and the next reinforcer could be arranged was 0.5 s. When an output occurred, the list was sampled and reinforcement for the associated response was arranged. When reinforcement was arranged the next occurrence of that response started delivering pellets every 0.5 s until all the pellets were delivered. Sessions ended after the list was completely sampled three times. Consequently each type of response was reinforced 18 times per session. Conditions remained in effect for at least 15 sessions and until session plots of the logs of the response, time and reinforcer ratios showed no upward or downward trend for 7 sessions.
The analysis of all results is based on the sums over the last seven sessions of the number of stay and switch responses at each alternative, the number of reinforcers earned for staying at and for switching from each alternative and the time at each alternative (Table 2). The stay responses at an alternative included all presses after the first press during each visit to that alternative. The responses for switching from an alternative were the first press at the other alternative. The time at an alternative was the cumulative intervals from the first press at the alternative to the first press at the other alternative. The reinforcers earned for staying at an alternative were the reinforcers arranged by the schedule for staying at that alternative. The reinforcers earned for switching from an alternative were the reinforcers arranged by the schedule for switching from that alternative. All response, time and reinforcer ratios were calculated as left data divided by right data.
Table 2
Table 2
This table shows time at each alternative (min), stay responses at each alternative and changeovers from each alternative.
Before using the results of this experiment to evaluate the stay/switch model, the equation for describing choice among different magnitudes of reinforcers for staying at and switching from each alternative needs to be developed. Including the magnitudes of reinforcers in Equation 3 produces,
equation M4
(4)
Mtn is the magnitude of the reinforcer for staying at alternative n and Mwn is the magnitude the reinforcer for switching from alternative n. Because the number of reinforcer earned for staying at and switching from each alternative was the same, they can be factored and cancel yielding,
equation M5
(5)
Except for Rat 819’s stay-response ratios, the stay/switch model described the stay-response ratios and time ratios for all rats (Figure 1 and Figure 2). The data were close to the plane. These figures show that the residuals, the distance from each data to the best-fitting plane, were small. These figures also show that the residuals appear to be randomly distributed with respect to the values of the independent variables. Finally, these figures show that the slope of the plane was not equal to zero along the ratio of the ratios of magnitudes of reinforcers, which means that the ratio of ratios of magnitudes influenced allocations of stay responses and of time; however, the slope along the ratio of the sums of magnitudes of reinforcers was not equal to zero only for the ratio of stay responses, indicating that the ratio of the sums of magnitudes influenced stay-response ratios but not time ratios.
Fig. 1
Fig. 1
The logarithm of the ratio of the stay responses plotted as a function of the logarithm of the ratio of the ratios of the magnitudes and the logarithm of the ratio of the sums of the magnitudes. The plane shows the best fitting plane, by least-squares, (more ...)
Fig. 2
Fig. 2
The logarithm of the ratio of the times plotted as a function of the logarithm of the ratio of the ratios of the magnitudes and the logarithm of the ratio of the sums of the magnitudes. The plane shows the best fitting plane, using least-squares, to the (more ...)
These observations were confirmed by the results of the regressions using Equation 5 (Table 3). Equation 5 accounted for greater than 80% of the variance for all rats except Rat 819’s stay-response ratio. The value of l, sensitivity to the ratio of the ratio of stay to switch magnitudes, was greater than zero for both the ratio of stay responses and the ratio of times for all rats. The value of m, sensitivity to the ratio of the sums of stay and switch magnitudes, was greater than zero for all rats’ ratios of stay response and for time ratios for just two rats (Rats 814 and 824). Stay responding for Rats 814, 817 and 818 was biased towards the right alternative and time ratios for Rats 814, 817, 818 and 819 also were biased towards the right alternative. Rat 824’s stay response and time ratios were biased towards the left alternative.
Table 3
Table 3
This table shows the results of least-squares linear regression using data from all conditions using the stay/switch model (Equation 5), and the generalized matching law (Equation 7).
The generalized matching law described the response and time allocations when using the symmetrical arrangements of magnitudes of reinforcers (Figure 3 and Figure 4). For each rat, the response and time ratios from the symmetrical conditions form an approximately straight line. Thus, any failure of the generalized matching law to describe the data using other arrangements of stay and switch magnitudes of reinforcers cannot result simply from using two pairs of schedules.
Fig. 3
Fig. 3
The logarithm of the ratio of responses plotted as a function of the logarithm of the ratio of obtained magnitudes of reinforcers. The solid line represents the best fitting line, by least-squares, using Equation 7, to the data from all conditions. The (more ...)
Fig. 4
Fig. 4
The logarithm of the ratio of times plotted as a function of the logarithm of the ratio of obtained magnitudes of reinforcers. The solid line represents the best fitting line, by least-squares, using Equation 7, to the data from all conditions. The range (more ...)
Before using the results of this experiment to evaluate the generalized matching law, I need to develop the equation for describing choice among different magnitudes of reinforcers for staying and switching. When applying the generalized matching law to symmetrical magnitudes of reinforcers one takes the frequency of each reinforcer type multiplied by its magnitude (e.g., Mt1 * Rt1 + Mw2 * Rw2; Rachlin & Baum, 1969b; Davison & McCarthy, 1988, which produces,
equation M6
(6)
To show that this equation is the correct application of the generalized matching law, consider Equation 6 when applied to symmetrical conditions (traditional concurrent procedures). The sum of the number of stay and switch reinforcers is the same at both alternatives, so they can be factored out of the numerator and denominator and then they cancel. Because the magnitudes are equal we can substitute Mn, the magnitude obtained at each alternative. The resulting ratio, M1 / M2, is the generalized matching law applied to magnitudes of reinforcers (Rachlin & Baum, 1969a; Davison & McCarthy, 1988). Because the frequency of stay and switch reinforcers obtained at each alternative was the same, they cancel from the equation, leaving,
equation M7
(7)
This equation was used for regressions using both symmetrical and nonsymmetrical arrangements.
Using arrangements that were not symmetrical decreased the precision of the descriptions by the generalized matching law. Considering all data points for each rat, Figure 3 and Figure 4 show for Rat 814 and 824’s response ratios and Rat 816 and 817’s time ratios, that the data points fall roughly on the same straight line as in the symmetrical conditions but there is more variability. However, for Rats 816, 817 and 818 the response allocations are systematically above and for Rat 819 it is below a line for symmetrical conditions. For Rats 814, 818, 819 and 824 the time allocations are systematically below a line for symmetrical conditions. The generalized matching law adequately described (r2 > .80) response allocations from all conditions only for Rat 817. However, it adequately described the time allocations from all conditions for all but Rat 814 (Table 3). For the results of all rats, the sensitivity to reinforcer allocation was low (< 0.60). The low slopes could result from not using a COD or from using non-independent scheduling of reinforcers. Time ratios were consistently more sensitive to reinforcers ratios than were response ratios. There was no consistent bias, although several rat’s response or time allocations were biased, usually towards the right alternative (Table 3).
The stay/switch model provided better descriptions of the rats’ response allocation data. Examining Figure 3 shows that, in the Necessary Condition, all 12 deviations from the best-fitting line were in the direction predicted by Equation 5. The deviations first were towards one alternative and then the other alternative. Examining this figure also shows that, in the Sufficient Conditions, 10 of the 12 the deviations from the best-fitting line were in the direction predicted by Equation 5. These deviations were closer to indifference. Finally, for the Unsymmetrical Conditions, 6 of the 12 deviations were in the direction predicted by Equation 5. The deviations were closer to indifference. Combining the results from these three arrangements shows that, using a binomial test, the deviations were significantly in the direction predicted by Equation 5 (28 of 36 conditions; p < .01).
The models were also compared by plotting the obtained (stay) response and time ratios as a function of the ratios predicted by the generalized matching law and the stay/switch model. The diagonal dashed line shows the locus of perfect predictions. This figure shows that the predictions by the stay/switch model of the stay-response and time ratios are closer to the obtained values than the predictions of the generalized matching law. The difference is especially noticeable for the necessary, sufficient and unsymmetrical arrangements. Although a similar difference is evident for the time ratios, it is much smaller.
Because the stay/switch model described the stay-response and time ratios, the following analyses assessed whether changing the ratio of the magnitudes of obtained reinforcers was necessary or sufficient for changing response and time ratios. Changing the ratio of the magnitudes of obtained reinforcers was not necessary for changing the response and time ratios. For the necessary conditions, the ratio of magnitudes of reinforcers remained unchanged yet response and time ratios, for each rat, first favored one alternative and then the other alternative. For response allocations, the ranges did not overlap (triangles, Figure 3). For time allocations of Rats 814, 816, 819, and 824, the ranges did not overlap (triangles, Figure 4).
Changing the ratio of the magnitudes of obtained reinforcers was not sufficient for changing the response ratios but was sufficient for changing time ratios. That is, in the sufficient conditions, response ratios but not time ratios remained unchanged as the ratio of magnitudes of reinforcers favored one alternative and then the other alternative. For Rats 814, 816, 817, and 819 the ranges of the response ratios from the two sufficient conditions overlapped as the ratio of the magnitudes of reinforcers changed (squares, Figure 3). Changing the ratio of the magnitudes of reinforcers was sufficient for changing the time ratio for each rat (squares, Figure 4). For each rat, the ranges of time ratios did not overlap.
The primary purpose of this experiment was to assess whether the stay/switch model could describe behavior allocations when the magnitudes of reinforcers were varied. The stay/switch model provided adequate descriptions of allocations of stay responses for five rats and of time allocations for all six rats. Although the stay/switch model did not adequately describe Rat 819’s responding, its description was better than the description by the generalized matching law‥ The descriptions of time allocations by the stay/switch model were adequate but no better than the descriptions by the generalized matching law. The failure of the stay/switch model to provide better descriptions of time allocations is not surprising given the lack of influence of the ratio of the sums of the magnitudes of reinforcers earned at each alternative. The ratio of the sums of reinforcers can be omitted when either they have no influence on response or time allocations, when symmetrical arrangements are used.
Because the stay/switch model adequately described the data, an additional purpose of this experiment was to assess whether changing the allocation of the magnitudes of obtained reinforcers, as in the generalized matching law, was necessary or sufficient for changing response and time allocations. If changing the ratio of obtained reinforcers was necessary then the response ratios will remain unchanged as the obtained magnitude ratio remains changed. The triangles in Figure 3 and Figure 4 indicate that the response ratios changed even though ratios of the obtained magnitudes of reinforcers did not change. They were not necessary for changing the ratio of responding. For Rats 814, 819 and 824, changing the ratio of the magnitudes of reinforcers was not necessary for changing time ratios but this result was not obtained for time ratios of Rats 816, 817 and 818. Only the time allocation data from Rats 816, 817 and 818 weaken the conclusion that changing the ratios of obtained magnitudes of reinforcers was not necessary for changing preference.
Changing the ratios of obtained magnitudes of reinforcers and finding no change in the response or time ratios means changing the ratio of obtained magnitudes of reinforcers was not sufficient to change behavior ratios. The response ratios for Rats 814, 816, 817, and 819 remained unchanged (Figure 3); the response ratios for rats 818 and 824 changed as did the time ratios for each rat (Figure 3 and Figure 4). This suggests that the changing obtained ratio of magnitudes of reinforcers was sufficient for changing response ratios but not time ratios.
Analyses using the stay/switch model showed that the ratio of the sums of the magnitudes of reinforcers earned for staying and switching had no influence on time allocations (Figure 4, Table 3) and only influenced the response allocations of Rats 814 and 824. This lack of effect was surprising given the effect of this variable on response and time allocation in a previous experiment (MacDonall et al., 2006) and on response and time allocations in reanalyses of the data in MacDonall (2005). These previous experiments varied rates of earning stay and switch reinforcers at the alternatives and used independent scheduling of reinforcers. At present it is not clear whether varying magnitudes of reinforcers while keeping the rate constant, the use of non-independent scheduling of reinforcers, or the specific magnitudes used resulted in the ratios of the sums of reinforcers having little influence on time and response allocations. Further research is necessary to understand the influence of the ratio of the sums of the magnitudes on preference.
Models of learning that have included reinforcement have focused on obtained reinforcers. As the present and previous (MacDonall, 2005; MacDonall et al., 2006) results show, it may be more useful to focus on earned reinforces. Because all reinforcers are both earned and obtained this change will have little impact when modeling the absolute rate, or any other measure of responding, of one operant. For example, Catania and Reynolds (1968) recorded the response rate maintained by different rates of (obtained) reinforcers and Herrnstein (1970) used these data for developing a model of the strength of a single operant. Because just one response was available, the same results will be obtained by using earned reinforcers instead of obtained reinforcers. This distinction is important only when using various complex schedules, such as, concurrent schedules in which one schedule is not available for the entire session.
Overall, the results of the present experiment show that the stay/switch model is a viable alternative to the generalized matching law. The stay/switch model provided better descriptions of response allocations. In addition, changing the ratio of obtained reinforcers, as in the generalized matching law, was neither necessary nor sufficient for changing the response ratio.
Fig. 5
Fig. 5
The left panels show the obtained response (top) and time (bottom) ratios plotted as a function of the ratios predicted by Equation 7 (generalized matching law). The right panels show the ratio of stay-response (top) and time (bottom) ratios plotted as (more ...)
Author note
I am grateful to Jonathan Galente for building and maintaining the equipment used in this research. I thank Anthony Benners, Ian Hayes, Gigi McGraw and Danielle Siwek for assistance in data collection. I also thank Randy Grace and two anonymous reviewers for very helpful suggestions on earlier versions of the manuscript. This research was supported in part by a Faculty Research Grant from Fordham University. Preparation of this manuscript was supported, in part, by National Institute of Mental Health Grant MH81266. Portions of these data were presented at the 2007 meeting of the Society for the Quantitative Analysis of Behavior.
Footnotes
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
  • Baum WM. On two types of deviation from the matching law: Bias and undermatching. J. Exp. Anal. Behav. 1974;22:231–242. [PMC free article] [PubMed]
  • Catania AC, Reynolds GS. A quantitative analysis of the responding maintained by interval schedules of reinforcement. J. Exp. Anal. Behav. 1968;11:327–383. [PMC free article] [PubMed]
  • Davison M, Baum W. Every reinforcer counts: Reinforcer magnitude and local preference. J. Exp. Anal. Behav. 2003;80:95–129. [PMC free article] [PubMed]
  • Davison M, McCarthy D. The Matching Law: A Research Review. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc; 1988.
  • Landon J, Davison M, Elliffe D. Choice in a variable environment: effects of unequal reinforcer distributions. J. Exp. Anal. Behav. 2003;80:187–204. [PMC free article] [PubMed]
  • Herrnstein RJ. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 1961;4:267–272. [PMC free article] [PubMed]
  • Herrnstein RJ. On the law of effect. J. Exp. Anal. Behav. 1970;13:243–266. [PMC free article] [PubMed]
  • Keller JV, Gollub LR. Duration and rate of reinforcement as determinants of rate of responding. J. Exp. Anal. Behav. 1977;28:145–153. [PMC free article] [PubMed]
  • Landon J, Davison M, Elliffe D. Choice in a variable environment: effects of unequal reinforcer distributions. J. Exp. Anal. Behav. 2003;80:187–204. [PMC free article] [PubMed]
  • MacDonall JS. Synthesizing concurrent interval performance. J. Exp. Anal. Behav. 2000;74:189–206. [PMC free article] [PubMed]
  • MacDonall JS. Earning and obtaining reinforcers under concurrent interval scheduling. J. Exp. Anal. Behav. 2005;84:167–183. [PMC free article] [PubMed]
  • MacDonall JS. Some effects of overall rate of earning reinforcers on run lengths and visit durations. Behav. Process. 2006;73:13–21. [PubMed]
  • MacDonall JS, Goodell J, Juliano A. Momentary maximizing and optimal foraging theories of performances on concurrent VR schedules. Behav. Process. 2006;72:283–299. [PubMed]
  • Neuringer AJ. Effects of reinforcement magnitudes on choice and rate of responding. J. Exp. Anal. Behav. 1967;10:417–422. [PMC free article] [PubMed]
  • Rachlin H, Baum WM. Response rate as a function of amount of reinforcement for a signalled concurrent response. J. Exp. Anal. Behav. 1969a;12:11–16. [PMC free article] [PubMed]
  • Rachlin H, Baum W. Choice as time allocation. J. Exp. Anal. Behav. 1969b;12:861–874. [PMC free article] [PubMed]
  • Rachlin H, Green L, Tormey B. Is there a decisive test between matching and maximizing? J. Exp. Anal. Behav. 1988;50:113–123. [PMC free article] [PubMed]
  • Stubbs DA, Pliskoff SS. Concurrent responding with fixed relative rate of reinforcement. J. Exp. Anal. Behav. 1969;12:887–895. [PMC free article] [PubMed]
  • Sugrue LP, Corrado GS, Newsome WT. Matching behavior and the representation of value in the parietal cortex. Science. 2004;304:1782–1787. [PubMed]
  • Todorov JC. Interaction of frequency and magnitude of reinforcement on concurrent performances. J. Exp. Anal. Behav. 1973;19:451–458. [PMC free article] [PubMed]
  • Todorov JC, Hanna ES, Bittencourt de Sa MCN. Frequency versus magnitude of Reinforcement: new data with a different procedure. J. Exp. Anal. Behav. 1984;41:157–167. [PMC free article] [PubMed]