PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
 
J Exp Anal Behav. 2010 March; 93(2): 247–260.
PMCID: PMC2831661

The Identities Hidden in the Matching Laws, and Their Uses

Abstract

Various theoretical equations have been proposed to predict response rate as a function of the rate of reinforcement. If both the rate and probability of reinforcement are considered, a simple identity, defining equation, or “law” holds. This identity places algebraic constraints on the allowable forms of our mathematical models and can help identify the referents for certain empirical or theoretical coefficients. This identity can be applied to both single and compound schedules of reinforcement, absolute and relative measures, and to local, global and overall rates and probabilities. The rate matching equations of Hernstein and Catania appear to have been approximations to, and to have been evolving toward, one form of this algebraic identity. Estimates of the bias and sensitivity terms in the generalized ratio and logarithmic matching models are here held to be averaging artifacts arising from fitting procedures applied to models that violate or conceal the underlying identities.

Keywords: matching, law, compound schedules, concurrent schedules, identity, tautology, behavioral economics, foraging

The success in the late 1960s and early 1970s in predicting relative response rates on concurrent interval schedules led to attempts to predict absolute response rates on schedules of reinforcement in general. For the case of a single response class, Herrnstein (1961) suggested a simple proportionality of the form

equation image
1

where b and r are the mean response and reinforcement rates respectively. Catania (1963) suggested a power function which for the single response case reduced to the form

equation image
2

Herrnstein (1970) stated that the simple proportionality did not appear to work well empirically and proposed a hyperbolic function

equation image
3

where r0 was assumed to represent sources of reinforcement other than those specified by the experimenter and k was assumed to be the asymptotic rate attainable for that response in the absence of competition from other sources. Catania (1973) proposed a mathematically similar equation

equation image
4

Where k was a coefficient of proportionality, C was assumed to represent an inhibitory effect of reinforcement upon responding, and Σ ri was the total rate of the known sources of reinforcement.

Each of these formulations treats rate of reinforcement as the primary variable influencing responding.

It will be shown below that in both single-response and many choice-response situations Equation 1 is the more generally correct; that is, that the exponent is unity and that the obtained response and reinforcement rates (and their ratios) are directly proportional, but that the multiplier is an identifiable variable rather than a hypothetical constant.

Simple Schedules, Absolute Measures

At the end of a typical operant experiment one has three totals at hand—the duration of the session or schedule component of interest (T), the number of Behavioral responses that occurred therein (B), and the number of Reinforcements delivered (R). From these three raw totals we frequently derive three additional variables (or their reciprocals): response rate b  = B/T (or a mean IRT, T/B); rate of reinforcement r  = R/T (or a corresponding obtained mean VI value T/R); and probability of reinforcement given a response p  = R/B (or a corresponding obtained mean VR value, B/R). We make these conversions in order to generalize across sessions or experiments having different absolute totals. Whether the three original numbers (B, R and T) are viewed as free and independent of one another, the three derived variables (b, r and p) are ratios sharing common terms, hence they are interdependent. If two of the variable pairs are known the third is strictly determined. Thus we can write

equation image
5a

where the denominator is the “conditional” probability of reinforcement given a response and all values are obtained as opposed to programmed.

The word-equation 5a conceals from easy view that it is equivalent to the algebraic identity

equation image
5b

This identity will of course hold for all values of B, R and T; and for the grand totals on any simple, compound or second-order schedule of reinforcement. It will also hold, and can be evaluated, for the local totals on those compound schedules where one can legitimately determine each component's duration and counts (e.g. chain, multiple, concurrent, but not single-operandum conjunctive or alternative schedules, where it is not possible to assign a response exclusively to one component over the other). It would hold equally well for dubious or inappropriate cases such as noncontingent time schedules or the differential reinforcement of “other” behavior (where the reinforcement probabilities given B and Not-B are equal, or zero and nonzero respectively).

Two rearrangements of this identity are well known. One, often used in the present context, places R/T on the left and is typically used to derive the rate of reinforcement generated by ratio schedules. The other, often used in the context of behavioral economics, places B/R on the left and is used to derive the response “cost” imposed by interval schedules (where cost equals 1/p). Moving B/T to the left is trivial mathematically but perhaps significant perceptually or interpretively.

One important point this makes is that, because things equal to the same thing are equal to each other, any theoretical equation that accurately describes or purports to describe or predict response rate must also equal Equation 5a. The two equations may then be solved simultaneously to yield additional relationships, or to reveal arithmetic incompatibility.

Equation 5a can be abbreviated to the form

equation image
5c

which may be compared to Equation 1, or both sides of Equation 5b can be multiplied by T to give an equation in number and probability, rather than rate:

equation image
6

The identity can also be inverted and rewritten in terms of obtained mean interresponse times, variable-interval values, and variable-ratio values as

equation image
7

Although this last equation would seldom be useful in practice it can be used to make a somewhat contrived point. In any single experimental session where the experimenter imposes a particular VI (or VR) schedule, there is a post hoc VR (or VI) sequence that would have produced exactly the same recorded result. An uninformed observer, or the subject, would be unable to tell which was in effect. Zeiler (1977) has addressed this issue in the context of control by direct and indirect variables.

Although these equations are simple algebraic tautologies, that does not necessarily mean that they are trivial. Many, but not all, of the equations or laws of physics are equally circular (e.g. E  = IR, f  = ma) as can be shown by reexpressing them in their basic units and dimensions. Such laws are sometimes called defining equations.

Equations 5 through 7 may be considered descriptive but not explanatory in the colloquial sense. They say more about the particular behavioral measures we have come to use for parsing our subject matter than about behavior per se. Nevertheless, the identity can be useful in a number of ways ranging from the philosophical or heuristic to the practical:

  • 1)
    The identity can be interpreted as a reminder that even on the simplest schedule we do not have a single dependent variable but a dependent variable pair. This is most clearly seen in the case of simple ratio or interval schedules (e.g. Equation 7) where the experimenter has complete or effectively complete control over one variable while the subject controls the other two. If it is known that only one of the dependent variables is directly influenced by an experimental manipulation and that the other dependent variable merely follows as it mathematically must, then the common practice of recording and attending to only one of the two variables may be logically justifiable, albeit potentially misleading. In any single isolated experiment the two dependent variables will be inextricably confounded and we may arbitrarily decide to call one of them the relevant consequent and the other an irrelevant concomitant. Across a series of experiments, however, it may be possible to separate the two still-confounded effects based on differential manipulations and findings and/or a preferred criterion of “orderliness” such as linearity, invariance, or goodness-of-fit.The practice of attending to only one of the two dependent variables becomes less defensible if both variables are directly influenced by an experimental manipulation while simultaneously constrained to obey the numerical identity. We tend to describe such cases as exhibiting strong interaction. Possible analogies here might be simultaneous manipulation of both sensitivity and bias in Signal Detection experiments or of speed/accuracy tradeoffs in reaction time studies—two areas where double dependent measures have proven fruitful. Attending to rate while ignoring probability carries risks.
  • 2)
    The identity suggests a potentially stronger parallel than usually recognized between the phenomena identified with “rate matching” and those historically identified with “probability learning”, or with those more recently considered in behavioral economics.
  • 3)
    The identity can also be interpreted as showing that, as independent variables, neither rate of reinforcement nor probability of reinforcement is logically prior to the other. Both may act concurrently. Which if either of the two variables is behaviorally the more potent or pertinent may differ with the situation, but the identity still suggests an inverse relationship.
  • 4)
    The identity can also be used simply for calculating the “missing variable” in an ongoing experiment, an armchair experiment, or from a published report. This is the most frequent use of Ohm's Law, for example.
  • 5)
    The identity can also serve as a useful check on our theoretical models, since they may not contradict it. As an illustration it will be used below, first to ask whether several different matching models are mathematically allowed, disallowed, or forced. Finally, we will return to a single-response case combining the identity with an empirical finding.

Compound Schedules, Absolute Measures

On a two-ply (two-component) compound schedule we can distinguish at least 10 different absolute rates grouped into three distinct sets: Local rates, e.g. R1/T1, B2/T2; global or grand rates, e.g. (B1 + B2)/(T1 + T2); and what have come to be called overall rates, e.g. R2/(T1 + T2). The logically completing fourth set, a kind of “underall rate”, e.g. (R1 + R2)/T1, has not been deemed useful.

Parallel distinctions can be made for at least six different local, global, and overall conditional probabilities of reinforcement, for example, (R1 + R2)/(B1 + B2). Each of these rate and probability measures can be related to the others through appropriate algebraic manipulation. The identity of Equation 5c holds in a straightforward manner for both the local and global measures. That is, the local (global) response rate will equal the local (global) reinforcement rate divided by the local (global) probability of reinforcement. Overall rates and probabilities, being to some extent unsymmetrical mixed measures, lead to their own mixed identities. Thus absolute overall response rate could be written as either of the following

equation image
8a

equation image
8b

where the latter may have some potential theoretical usefulness.

Compound Schedules, Relative Measures

Having expressions for the various absolute rates in each component of a compound schedule allows us to compute various relative rates as well. It has been traditional practice to use overall rates for such computations, in which case all temporal terms cancel, leaving a dimensionless ratio that resembles a relative number or frequency rather than a relative rate. Thus relative overall response rate becomes

equation image
9

In the matching literature the term “rate” is still applied, perhaps because all numbers are normalized to a common session duration, hour, minute or other temporal unit. In other contexts and disciplines it might be more appropriate and conventional to treat this quantity as a probability, but the now historical terminology will be followed here.

Using the above definitions and identities, various expressions for relative overall response rates can now be derived. Beginning with Equation 6, and using subscripts to denote the two components, we have B1  = R1/p1 and B2  = R2/p2, hence

equation image
10

Multiplying both numerators and denominators by (p1 p2) simplifies to

equation image
11

Herrnstein (1970) used a formally equivalent equation in discussing matching under a concurrent ratio schedule, though the equation is a general identity true of all compound schedules. It is not identical to Herrnstein's (1961) matching relationship:

equation image
12

but reduces to it as a special case if and when p1  = p2, as noted by Revusky (1963). Thus matching is mathematically allowed but not forced. If relative rate matching does occur then local and global reinforcement probabilities will be equal, and vice versa. The two phenomena (rate matching and equal probabilities) are confounded correlates, of equal potential significance, with neither necessarily being the cause of the other. It also follows from Equation 11 that when relative rate matching does not occur (as in undermatching on concurrent schedules or behavioral contrast on multiple schedules) it must necessarily be due to differences in the two probabilities of reinforcement.

Equation 11 does not reduce to a general equation of the form

equation image
13

although for any isolated schedule pair, R1 R2, one can always find some value for k that forces numerical equality. However, by dividing the numerator and denominator of Equation 11 by p2 we get the somewhat more familiar form

equation image
14

where m  = p1/p2. This too is a general identity that must always hold.

For the case of a three-ply compound schedule the corresponding identity can be variously written as

equation image
15

where m j  = p 1/p j.

Equations 14 and 15 are written for the relative rates in component number 1, but corresponding equations will hold for components 2, 3, or n. For notational exactitude m should be doubly subscripted as m i j  = p i/p j where i is the component of interest selected for the numerator. Relative overall response rates for the generalized case of any n-ply compound schedule (multiple, concurrent or other) can then be written as either of the following identities

equation image
16a

equation image
16b

Application to Classic Rate Models

It will be convenient for what follows to rewrite the last identity as

equation image
17

where the right-hand summation is over all components except component number 1. This awkwardness could easily be avoided by assigning an m term of unity to the R1 term in the denominator and then moving both back within the summation, but it is left in the less orthodox form to facilitate comparisons with Herrnstein's (1970) theoretical equation

equation image
18

and with Catania's (1973) theoretical Equation 19 (before C k became a new k)

equation image
19

Although there are obvious differences between the last three equations, the similarities are the more interesting.

Herrnstein (1970) viewed m as an interaction term between components in multiple schedules, having a value between zero and one, to be estimated from the data. The present general account provides an explicit definition for m, having a value between zero and infinity, which can be calculated directly from the raw totals.

In the two-ply case Herrnstein (1970) began with the reasonable first assumption that the m terms were the same for both components, whereas in the present account m12 and m21 are seen to be reciprocally related—becoming the same only when the two probabilities are equal, and their ratios are unity. When unequal, the m terms could be viewed as representing a differential preference or bias toward one schedule, stimulus, or response of a pair.

The terms B and k in Equations 18 and 19 carry the dimensions of “responses” or “responses per unit time” (Catania, 1973; Herrnstein, 1974) whereas Equations 14–17 are written as dimensionless proportions or relative rates. The simplest resolution of this apparent difference is to assume that the term k represents the quantity (B1 + B2), or more generally Σ B j. Transposing k to the left-hand denominator makes the similarities among Equations 17–19 more obvious.

In matching experiments where the total rate of reinforcement is held constant the observed response total also appears to remain relatively constant, or to vary unsystematically (see Catania, 1963, Figure 2). A constant response total can also result if the organism responds at a fixed local rate that remains invariant with changes in schedule values or relative durations, as has been reported with pigeons (Blough, 1963; Stubbs & Pliskoff, 1969). The constancy of k has been assumed in several theoretical accounts, and these particular effects would tend to produce it, but it is not required for Σ B j in the identities, and many researchers have reported conditions under which it is not (see review by McSweeney, Melville & Whipple, 1983).

Fig 2
Data reanalyzed and replotted from Catania and Reynolds (1968) for the last 3 pigeons in their Figure 1. Left axis is the number of reinforcements per min divided by the number of responses per min. Intercept and slope constants from the linear regression ...

At this point we have rationalized and redefined the terms m and k in order to make the theoretical Equations 18 and 19 most easily consistent with what is algebraically required. The same approach also leads to questioning the necessity of the inhibitory term C in Equation 19 and the reinforcement term R0 in Equation 18.

In Equation 19 the C term in the denominator should have the dimensions associated with reinforcement—to prevent adding apples to oranges, and to cancel the corresponding dimensions of R in the numerator. But if C in the numerator also has these same dimensions then cancellation no longer occurs and we ultimately end up with the statement that a number equals a dimension. If C in the numerator does have the dimensions of reinforcement, then k cannot be a dimensionless coefficient, nor have the dimensions of responses or responses per time as in the Herrnstein (1970) account (and the above account), but must have the dimensions of responses per reinforcement (which, interestingly, is equivalent to 1/p in the above). Both Catania (1973) and Williams and Wixted (1994) discussed this dimensional necessity. It would seem less awkward and more parsimonious to discard C as unnecessary. Since Equation 19 already contains an m term for each R term within the summation, it then becomes algebraically and dimensionally compatible with the required identity of Equation 16b.

The term R0 in Equation 18 is assumed to represent unmeasured hypothetical reinforcers beyond those arranged by the experimenter. If there are cases where such intrinsic or extraneous sources of reinforcement must be considered, the formal equations require an additional m10 term on the right and a corresponding B0 term on the left, paralleling the third components in Equation 15. Identifying the physical referents for such terms and quantifying their values presents a number of problems and dangers one would hope to avoid. The most cautious and parsimonious approach might be to consider the possibility that R0 has served as a floating parameter to account for inaccuracies in the estimates of m and/or inconstancies in the values of k. Having explicit definitions for these terms makes it easier to evaluate this possibility.

If R0 were assumed to exist anyway, then the three required theoretical terms R0, m10, and B0 represent only two degrees of freedom since they too must fulfill the identity B0  = R0/p0. In the most general case, there will be no way of estimating p0 independently of R0 and B0. However, in the special case where overt matching occurs and probabilities are equal, then p0 should equal the empirically measurable value of p1, p2, etc. Unfortunately, this still leaves an infinite number of unmeasurable R0/B0 pairs that would fulfill the necessary relationship. This problem cannot be unequivocally resolved even by the, perhaps tempting, assumption of a constant k greater than the observed Σ Bj, such that B0  = k − Σ B j. Why? Any given n-ply schedule can be expressed as having only (n − 1) plies by discarding one set of real terms, or as having (n + 1) plies by adding appropriately selected imaginary terms, and in either case the equations will balance. Any value of k can be made to suffice. The existence of R0 cannot be formally confirmed or disconfirmed within this closed system.

It would be interesting to reexamine those studies where archival data permit comparing the assumed or computed m values with the obtained reinforcement probabilities, and the computed k values with the actual response sums. A computed k value less than any one of its contributing sums would seem fatal to the R0 hypothesis since it would indicate a negative number of B0 responses—a notion difficult to fathom or defend. (McSweeney et al., 1983, reported a number of cases where the computed R0 was negative, which equates to the same thing.)

The above derivations and analysis suggest that Herrnstein's (1970) and Catania's (1973) formulations in Equations 18 and 19 are incorrect in their details or parametric interpretations but are close approximations to the general identity of Equation 16b. That these investigators came so close, out of an unlimited number of alternative possibilities, is a tribute to their hypothetico-deductive skills.

Application to Ratio Models

In the above it has been shown how identities stemming from Equation 5 determine or constrain expressions relating responding and reinforcement, using relative overall rates as an example. An additional example will show how it applies to the alternative measures of response and reinforcement ratios.

In concurrent schedules the subject determines the time allocations T1 and T2, which usually differ, and the use of relative overall rates is conventional (following the discovery of matching). In multiple schedules the experimenter determines T1 and T2, which are usually equated rather than “matched” to the schedule values, and the use of relative local rates is conventional. (In a typical multiple schedule the overall rate B1/(T1 + T2) would equal B1/(2T1), or half the local rate.) However, by using rate ratios the temporal terms can be made to cancel in both cases allowing a common formulation (although possibly concealing an important underlying distinction).

For multiple schedules, Lander and Irwin (1968) proposed a nonmultiplicative ratio power function which Baum (1974) generalized to concurrent schedules and matching as

equation image
20

where a represents any differential bias and c represents sensitivity (‘censitivity’) to reinforcement. Equation 5b, on the other hand, leads to the identity

equation image

equation image
21

with an R ratio exponent of unity and a “bias” multiplier equivalent to m2,1 in the above. Many investigators have found cases where Equation 20 fits data more accurately when the exponent c differs from unity. These cases typically involve numerical fits computed across several schedule combinations of R1 R2 pairs. There are at least two ways that this result can arise.

First, if the investigator manipulates R1/R2 in order to observe its effects upon B1/B2 and c then, ideally, a should be held constant throughout. If it is not known that a represents the measurable quantity p2/p1 then there is no way of detecting its variation. Instead, one might assume and impose invariance by fiat. The resultant fit will therefore be inexact and it can be improved upon by a power transformation, a multiplicative correction factor, an added constant, or any other operation that introduces one or more free parameters. A nonunitary c implies a nonconstant a.

Second, if the experimental procedure allows or causes p2/p1 to become a direct or indirect function of R1/R2, then Equations 20 and 21 can hold simultaneously. As an example of an indirect function, assume that the organism “probability matches”, i.e. that it apportions its responses in the same manner as the obtained probabilities of reinforcement:

equation image
22

Note that this is not an identity but a hypothetical empirical finding. Substituting for p2/p1 in Equation 20 yields

equation image
23

Transposing the response terms on the right to the left gives

equation image
24

hence

equation image
25

Equation 25 is still equal to Equation 21, but the reinforcement probabilities have become indirect (and hidden) functions of the reinforcement rates. This can be revealed by rewriting Equation 25 as

equation image
26

If, instead, the reinforcement probabilities were equal to, or proportional to, some other power than 1/2, for example p1  = k R1C and p2  = k′ R2C then the (R1/R2) exponent would become 1-c with a  = k′/k. Many other p  = f (R) relationships are of course possible, yielding different (R1/R2) exponent values and nonunitary as. Identifying such relationships will require attending to measured probabilities as well as rates.

The Matching Law “Constants” as Averaging Artifacts

If an experiment employs, say, five different R1 R2 combinations, then Equation 21 will hold for each individual set of triple pairs. However, in most cases each different R1/R2 pair will generate a different B1/B2 pair and a different p2/p1 pair, hence a different local a value. That is, there is no forced invariance, constancy or even continuity across the five conditions. All can change.

Consider an experiment in which a subject exhibits bias but neither under- nor overmatching and c  = 1. If one fits and estimates the best constant a across all five pairs then the value of this a will usually fall somewhere between the obtained minimum and maximum values of the five p2/p1 pairs. That is, it will be some average, though not necessarily their arithmetic mean. It can be shown that if one fits Equation 20 using a least squares fitting procedure on the obtained values, then the fitted a will equal the geometric mean of the obtained p2/p1 values. If instead of the ratio form of the matching law one fits Baum's (1974) logarithmic form

equation image
27

using least squares, then the best fitting log a value will equal the arithmetic mean of the five log (p2/p1) pairs. When unlogged, this necessarily gives the same value for a as the geometric mean.

The same thing will result using a dataset where over- or undermatching occurs if one forces c to equal 1 and then refits. This will be demonstrated using data from Baum, Schwendiman and Bell (1999). This study exposed 4 pigeons to Conc VI VI schedules having a range of reinforcement rate ratios counterbalanced across two keys. When the data were conventionally plotted as log response ratios versus log reinforcement ratios (that is, of log [BLEFT/BRIGHT] versus log [RLEFT/RRIGHT]) the fitted lines showed minimal bias (log a ≈ 0) but noticeable undermatching (c < 1 ≈ 0.8), as most typically seen in other studies. When the ratios were instead dichotomized (subscripted) by preferred and nonpreferred schedule values, independent of key position, a different picture emerged with negative log bias and near matching (c ≈ 1). For 1 of their 4 pigeons they replotted the “Left/Right” data showing the function not as one continuous line of reduced slope but as two separate lines of near-unit slope and opposing bias. They then forced the two slopes to be exactly equal (at c  = 1) and solved for the “Lean and Rich” bias values separately, getting log(a) values of 1.55 and −1.31 respectively. Table 1 shows how these same values can be obtained by averaging the raw or logged data values without an explicit least squares fitting procedure.

Table 1
Data from Baum, Schwendiman and Bell (1999) for Bird 973 on 12 concurrent VI VI schedules.

The same thing will result using a dataset where the empirically obtained c is unity or near unity without being “forced”. Such data sets are rather rare but Table 2 presents one example where the raw data values are publicly available. This example is derived from the Appendix and Table 2 of Elliffe and Alsop (1996) for Bird 131 under one reinforcement/min, where the reported log bias value was −0.16 with a c of 1.01 (close enough to unity for the present purpose).

Table 2
Data from Elliffe and Alsop (1996) for Bird 131 on five concurrent VI VI schedules where near matching occurred (c  = 1.01).

Again, it is seen that the fitted log bias value equals the arithmetic mean of the log p2/p1 values.

Finally, consider the more common case of inexact matching. If one interprets the fitted a value as a true constant then, since it usually isn't, the fit will not be optimal and a better fit will result by introducing a free parameter c and estimating both parameters simultaneously. In this case the two fitted parameters will seldom be identifiable simple means of any of the raw data values. Table 3 gives an example derived from the appendix of Davison and Jones (1995) for Bird 21where the reported c was 0.54 and log a was 0.00 (a concurrent VI Extinction condition has been omitted to avoid division by zero).

Table 3
Data from Davison and Jones (1995) for Bird 21 on nine concurrent VI VI schedules where marked undermatching occurred (c  = 0.54).

In this case the arithmetic mean of the log p2/p1 values is not 0.00 (actually 0.0004 before rounding) but −0.095.

Interestingly, there is still some hidden underlying order. Equation 21 holds for each row of ratio triplets taken individually, as it must, but also applies to the means of the entire dataset. That is, the geometric mean of the n B1/B2 ratios equals the geometric mean of the n p2/p1 ratios times the geometric mean of the n R1/R2 ratios. Similarly, the arithmetic mean of the log B1/B2 ratios equals the sum of the arithmetic means of the two corresponding log ratios. The reader can verify this with the data in Tables 2 and and3.3. Neither of these mean calculations involves the parameter c. This shows that the behavioral identity underlies this result, that it is independent of the theoretical matching equation, and that it is not due to the fitting procedure, be it least squares or other.

The fitting procedure will, however, influence the value of c and a. Except for the case of strict matching, the identity of Equation 21 does not predict what the fitted values of c and a will be. That will be determined by the particular set of numbers generated by the subject and experimenter, and by the fitting procedure itself. Davison and Elliffe (2009) have shown that the least squares procedure underestimates the sensitivity parameter c. This is due to the variance, unequal variances, and asymmetrically skewed distributions among the response and reinforcement measures. The logarithmic transformation ameliorates some of these distortions and is preferable to the raw ratio fits, as Baum (1974, 1979, 1983), too, has discussed, but it does not remove them. A “best” fitting procedure is not yet known or agreed upon.

Because the generalized matching law of Equation 20 describes behavior well, the identity of Equation 21 implies that so might the equation

equation image
28

expressing the response ratios as a function of the probabilities rather than the rates of reinforcement, but with different bias and sensitivity values. Figure 1 compares plots of Equations 20 and 28 using the data in Table 3.

Fig 1
Data from Table 3 plotting log B1/B2 as a function of log R1/R2 (top) and as a function of log p2/p1 (bottom).

Graphs for the remaining 5 pigeons in the study showed this alternative relationship to be linear in log coordinates, with negative slopes, but with R-squares consistently smaller than with the conventional model (group mean R-square  = 0.83 versus 0.95). However, the slope values were consistently closer to “matching” (i.e. to −1.0). Slopes for the conventional model varied from 0.36 to 0.60 with a mean of 0.49. Slopes for the alternative model varied from −0.54 to −1.23 with a mean of −0.92. Bias values were small in both cases.

This finding would give a mixed answer to a simple question such as, “Which variable—rate or probability—is the more important, influential, operative, or causal?” It would appear that both variables are acting simultaneously but in different ways. Clarifying these differences would require a more molecular examination of the underlying factors leading to the particular obtained reinforcement rates and probabilities, which is not the main purpose of this article.

Application to Single Schedules and an Empirical Finding

Catania and Reynolds (1968) exposed pigeons to a number of single VI schedules ranging in value from 10.5 to 420 seconds. They plotted responses per min against reinforcements per hour and obtained monotonically increasing negatively accelerated functions. The identity of Equation 5c states that for each single VI value, rate of response b will equal rate of reinforcement r divided by the probability of reinforcement p. It does not make a specific prediction of how p varies with r across conditions if b also varies across conditions—linearly, nonlinearly or even independently of r. If b is free to vary, this relationship is not forced, but it can be determined empirically. Catania and Reynolds's data were reanalyzed after converting both r and b to rates per min so that their quotient would give the obtained p. The results are plotted on the left side of Figure 2.

The data were highly linear in these coordinates and over this range, hence they were fitted with a straight line of the form p  = α + β r. Substituting this expression for p into the identity gives the hyperbola.

equation image
29

Using the fitted intercept and slope constants, the predicted values for b are plotted with the obtained values for b on the right side of Figure 2.

Dividing both the numerator and denominator of this last equation by β gives

equation image
30

which can be compared to Herrnstein's theoretical model in Equation 3. Herrnstein's r 0 becomes α/β while 1/β closely approximates the asymptotic response rate. For example, in the top two panels of Figure 2 the asymptote appears to be around 75 responses per min and the fitted value for β is 0.0133, whose reciprocal is 75.2. The more exact predicted asymptote is found by evaluating the linear equation when probability p reaches one, and is (1 − α)/β or 75.1 responses per min. Substituting this correction complicates the equations somewhat while making little quantitative difference. With the units used here the absolute values for α are small. However, α cannot be zero, or the animal would always respond at its asymptotic response rate regardless of reinforcement rate.

The quantity α/β necessarily has the same dimensions as r, and in this example evaluates to 0.06 reinforcements per min, or 3.61 per hour. This is compatible with Herrnstein's theoretical hypothesis but I leave its interpretation to the reader.

Equations 29 and 30 suggest exploring which classes of experimental variables or “subject strategies” affect α and β differentially. Experimental variables might include reinforcement magnitude, deprivation level, or response effort. Subject strategies might include proportional changes in the postreinforcement pause, or changes in running rates, within-bout response rates or in the number and durations of between-bout pauses. On VI the subject has minimal control over r but through these can manipulate p.

The linearity between p and r, and the proportion of variance accounted for suggest the possibility that the obtained probability of reinforcement may be at least as effective a controlling variable as rate of reinforcement even on interval schedules, and allows the hypothesis that the hyperbolic relationship between response rate and reinforcement rate might be more derivative than primary.

Limitations and Extensions

The identity of Equation 5 is formulated only for the obtained values of the response rate, reinforcement rate, and the conditional probability of reinforcement; not for any of the many other variables known to characterize or influence behavior. Furthermore, it is based only on their totals or arithmetic means, and says nothing about their instantaneous, sequential or distributional characteristics.

This identity and those in Equations 14, 16 and 21 are purely quantitative, atheoretical molar descriptions of how the obtained values of these variables are related. They do not explain, for example, behavioral contrast or choice or why matching occurs, when it does. They do not address, and I will not here address, the vast literature on molecular theoretical models (momentary, local, or global maximizing, melioration, etc.) proposed as explanations for these phenomena. Neither do I here address the literature of behavioral economics where the same identities apply, with the variables usually inverted and renamed to represent the cost of responding or perhaps the cost of waiting (Collier, Johnson, & Mathis, 2002). However, the identities can serve as checks and guidelines for these molecular theoretical models, and perhaps aid in the interpretation of findings. The algebraic identities dictate that:

  • 1)
    Any quantitative theoretical model that predicts an absolute or local response rate b must also equal r/p. The two equations can then be solved simultaneously to yield other relationships.
  • 2)
    Any theoretical model that predicts a single B1/B2 ratio pair must also equal (p2/p1)*(R1/R2). If multiple pairs are considered simultaneously the result is more complex but a similar relationship holds for their geometric means.
  • 3)
    Any theoretical model that predicts a relative response rate B1/(B1 + B2) must also equal (R1 p2)/(R1 p2 + R2 p1), or equivalently R1/(R1 + m12 R2). If three (or n) components are involved then the summation forms of Equations 15 and 16 apply. However, any one of the components can be temporarily or permanently discarded or ignored and the relationship will hold for the other two (or n-1) components. That is, one can drop R3 (or R0, as is done in the ratio formulation) and the identities will still hold. It is mostly for the single response case that the empirically unmeasurable R0 has been considered theoretically appealing as a hypothetical variable.

Historically, most operant quantitative models regarded response rate as primarily a function of reinforcement rate (e.g., Herrnstein, Catania, Baum as cited herein). Some more recent models have considered various explicit or implicit reinforcement probabilities (e.g. the Stay/Switch model of MacDonall, 2009). The position taken here is that we need to consider and measure both variables. Behavior is presumed to be simultaneously sensitive to both and affects both through interaction and continuous feedback, at least until a stable state is reached.

Although Equation 5 is an equation that must hold, it is only one of many possible equations that may also hold. The identity should become increasingly useful as we discover or devise and incorporate additional equations relating its three variables to themselves or other variables. These additional equations may be empirical, theoretical, or mathematically definitional; and one may anticipate that some of them will be situation-specific rather than universal. Even in the absence of such additional equations the identity has conceptual and computational utility beyond the examples given here.

Acknowledgments

This material has been reviewed by the Walter Reed Army Institute of Research, and there is no objection to its presentation and/or publication. The opinions or assertions contained herein are the private views of the author and are not to be construed as official or as reflecting the views of the Department of the Army or the Department of Defense.

Appendix

VI data from Catania and Reynolds (1968) showing reinforcement rates, response rates, and the obtained probability of reinforcement p (as their ratio).ratio).

An external file that holds a picture, illustration, etc.
Object name is jeab-93-02-07-t04.jpg

Appendix VI data from Catania and Reynolds (1968) showing reinforcement rates, response rates, and the obtained probability of reinforcement p (as their ratio).ratio).

BirdR/hrR/minB/minpr/(α + β r)
2788.60.143410.003552.96
170.28366.840.004262.02
340.56771.150.008067.97
861.43374.350.019372.16
1712.85073.280.038973.63
3435.71774.690.076574.41
27990.15029.110.005228.06
170.28338.70.007339.07
340.56753.30.010650.13
861.43362.30.023060.49
1712.85061.460.046464.84
3435.71768.30.083767.30
28190.15043.680.003437.45
861.43362.390.023062.67
861.43363.20.022762.67
1712.85063.250.045165.22
3435.71767.050.085366.60
Note. Right column is the predicted response rate, where r is reinforcement rate and α and β are the intercept and slope constants shown in Figure 2.

REFERENCES

  • Baum W.M. On two types of deviation from the matching law: Bias and undermatching. Journal of the Experimental Analysis of Behavior. 1974;22:231–242. [PMC free article] [PubMed]
  • Baum W.M. Matching, undermatching, and overmatching in studies of choice. Journal of the Experimental Analysis of Bbehavior. 1979;32:269–281. [PMC free article] [PubMed]
  • Baum W.M. Matching, statistics, and common sense. Journal of the Experimental Analysis of Behavior. 1983;39:499–501. [PMC free article] [PubMed]
  • Baum W.M, Schwendiman J.W, Bell K.E. Choice, contingency discrimination, and foraging theory. Journal of the Experimental Analysis of Behavior. 1999;71:355–373. [PMC free article] [PubMed]
  • Blough D.S. Interresponse time as a function of continuous variables: A new method and some data. Journal of the Experimental Analysis of Behavior. 1963;6:237–246. [PMC free article] [PubMed]
  • Catania A.C. Concurrent performances: Reinforcement interaction and response independence. Journal of the Experimental Analysis of Behavior. 1963;6:253–263. [PMC free article] [PubMed]
  • Catania A.C. Self-inhibiting effects of reinforcement. Journal of the Experimental Analysis of Behavior. 1973;19:517–526. [PMC free article] [PubMed]
  • Catania A.C, Reynolds G.S. A quantitative analysis of the responding maintained by interval schedules of reinforcement. Journal of the Experimental Analysis of Behavior. 1968;11:327–383. [PMC free article] [PubMed]
  • Collier G, Johnson D.F, Mathis C. The currency of procurement cost. Journal of the Experimental Analysis of Behavior. 2002;78:31–61. [PMC free article] [PubMed]
  • Davison M, Elliffe D. Variance matters: The shape of a datum. Behavioural Processes. 2009;81:216–222. [PubMed]
  • Davison M, Jones M. A quantitative analysis of extreme choice. Journal of the Experimental Analysis of Behavior. 1995;64:147–162. [PMC free article] [PubMed]
  • Elliffe D, Alsop B. Concurrent choice: Effects of overall reinforcer rate and the temporal distribution of reinforcers. Journal of the Experimental Analysis of Behavior. 1996;65:445–463. [PMC free article] [PubMed]
  • Herrnstein R.J. Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior. 1961;4:267–272. [PMC free article] [PubMed]
  • Herrnstein R.J. On the law of effect. Journal of the Experimental Analysis of Behavior. 1970;13:243–266. [PMC free article] [PubMed]
  • Herrnstein R.J. Formal properties of the matching law. Journal of the Experimental Analysis of Behavior. 1974;21:159–164. [PMC free article] [PubMed]
  • Lander D.G, Irwin R.J. Multiple schedules: Effects of the distribution of reinforcements between components on the distribution of responses between components. Journal of the Experimental Analysis of Behavior. 1968;11:517–524. [PMC free article] [PubMed]
  • MacDonall J.S. The stay/switch model of concurrent choice. Journal of the Experimental Analysis of Behavior. 2009;91:21–39. [PMC free article] [PubMed]
  • McSweeney F.K, Melville C.L, Whipple J.E. Herrnstein's equation for the rates of responding during concurrent schedules. Animal Learning & Behavior. 1983;11:275–289.
  • Revusky S.H. A relationship between responses per reinforcement and preference during concurrent VI VI. Journal of the Experimental Analysis of Behavior. 1963;6:518. [PMC free article] [PubMed]
  • Stubbs D.A, Pliskoff S.S. Concurrent responding with fixed relative rate of reinforcement. Journal of the Experimental Analysis of Behavior. 1969;12:887–895. [PMC free article] [PubMed]
  • Williams B.A, Wixted J.T. Shortcomings of the behavioral competition theory of contrast: Reanalysis of McLean (1992) Journal of the Experimental Analysis of Behavior. 1994;61:107–112. [PMC free article] [PubMed]
  • Zeiler M. Schedules of reinforcement: The controlling variables. In: Honig W.K, Staddon J.E.R, editors. Handbook of operant behavior. Prentice-Hall; Englewood Cliffs, NJ: 1977. pp. 201–232.

Articles from Journal of the Experimental Analysis of Behavior are provided here courtesy of Society for the Experimental Analysis of Behavior