|Home | About | Journals | Submit | Contact Us | Français|
The emergence and maintainance of cooperation by natural selection is an enduring conundrum in evolutionary biology, which has been studied using a variety of game theoretical models inspired by different biological situations. The most widely studied games are the Prisoner’s Dilemma, the Snowdrift game and by-product mutualism for pairwise interactions, as well as Public Goods games in larger groups of interacting individuals. Here we present a general framework for cooperation in social dilemmas in which all the traditional scenarios can be recovered as special cases. In social dilemmas cooperators provide a benefit to the group at some cost, while defectors exploit the group by reaping the benefits without bearing the costs of cooperation. Using the concepts of discounting and synergy for describing how benefits accumulate when more than one cooperator is present in a group of interacting individuals, we recover the four basic scenarios of evolutionary dynamics given by (i) dominating defection, (ii) co-existence of defectors and cooperators, (iii) dominating cooperation and (iv) bi-stability, in which cooperators and defectors cannot invade each other. Generically, for groups of three or more interacting individuals further, more complex, dynamics can occur. Our framework provides the first unifying approach to model cooperation in different kinds of social dilemmas.
The question of how natural selection can lead to cooperation has fascinated evolutionary biologists since Darwin (Darwin, 1859, Hammerstein, 2003, Maynard Smith & Szathmáry, 1995, Trivers, 1971). Cooperation among relatives is explained by kin selection representing the idea that selfish genes lead to unselfish phenotypes (Frank, 1989, Hamilton, 1963). For the evolution of cooperation among genetically unrelated individuals various mechanisms have been put forward based on evolutionary game theory: cooperators form groups and thus preferentially interact with other cooperators (Sober & Wilson, 1998, Wilson & Sober, 1994); cooperators occupy spatial positions in lattices or networks and interact with their neighbors who are also cooperators (Hauert, 2001, Killingback et al., 1999, Nowak & May, 1992); optional interactions can stabilize cooperation (Hauert et al., 2002, Semmann et al., 2003); repeated games enable the emergence of direct reciprocity (Axelrod & Hamilton, 1981, Nowak & Sigmund, 1993); and reputation facilitates the evolution of cooperation via indirect reciprocity (Alexander, 1987, Nowak & Sigmund, 1998) or punishment opportunities (Sigmund et al., 2001).
The vast majority of models on the evolution of cooperation consider pairwise interactions: a cooperator meeting another cooperator obtains the reward R, but against a defector the cooperator is left with the sucker’s payoff S. In contrast, the defector exploits the cooperator and receives the temptation T , but when facing another defector each gets the punishment P . This terminology was introduced for the Prisoner’s Dilemma, which is defined by the payoff ranking T > R > P > S. Hence defection dominates cooperation because it is better to defect regardless of what the partner does. In terms of costs c and benefits b of cooperation, the Prisoner’s Dilemma describes situations where costs incur to the cooperator but benefits accrue exclusively to the partner, i.e. T = b, R = b – c, P = 0 and S = −c with b > c. This represents the most stringent form of a social dilemma.
Social dilemmas are defined as interactions in groups of individuals where groups of cooperators are better off than groups of defectors, but in any mixed group defectors outperform cooperators (Dawes, 1980). The fact that defectors exploit cooperators requires that defectors are better off in any mixed group than in the absence of cooperators, and conversely that cooperators are worse off in any mixed group than in the absence of defectors. In the context of pairwise interactions this requires R > P, T > S, T > P and R > S. These requirements are satisfied by the Prisoner’s Dilemma but there are three additional rankings possible.
The Snowdrift game (Sugden, 1986) (also known as the Chicken or Hawk-Dove game (Maynard Smith & Price, 1973)) is defined by T > R > S > P . To illustrate this game, consider two drivers on their way home that are caught in a blizzard and trapped on either side of a snowdrift. Each driver has the option to remove the snowdrift and start shoveling or to remain in the car. In contrast to the Prisoner’s Dilemma, the best choice now clearly depends on the other driver: if the other cooperates and starts shoveling, it pays to defect and remain in the car but if the other defects, it is better to shovel and get home than to wait for spring. Similar situations may occur whenever the act of cooperation creates a common good that can be exploited by others, i.e. whenever the benefits of cooperation accrue not only to the partner but also to the cooperator itself. For example, foraging yeast cells produce and secrete an enzyme in order to lyse their environment (Greig & Travisano, 2004). The resulting food resource represents a valuable common good prone to exploitation by other cells. However, despite the prospects of being exploited, a single cell may be better off (prevent starvation) by producing the enzyme if no one else does. This last twist relaxes the social dilemma to some extent.
The dilemma is completely relaxed if R > T > S > P holds. This refers to by-product mutualism where it is better to cooperate irrespective of the partner’s decision, i.e. cooperation dominates defection. However, note that in any mixed group, defectors are still better off than cooperators but at the same time the payoff of defecting individuals would increase upon switching to cooperation (because the individual draws a net benefit from its own cooperative act). The remaining ranking is R > T > P > S, for which the best choice again depends on the partner’s decision, but now it is best to aim for mutual decisions: defect if the other defects and cooperate if the other cooperates. The social dilemma presents itself as a coordination problem with mutual cooperation as the preferred outcome.
Cooperative interactions in groups of N individuals have received much attention in the context of Public Goods games (Fehr & Gächter, 2002, Kagel & Roth, 1995). In typical Public Goods experiments, individuals can make an investment into a common pool knowing that the experimenter will multiply the total investments and distribute it equally among all participants irrespective of their contributions. In essence, Public Goods games represent N-persons Prisoner’s Dilemmas (Dugatkin, 1997, Hauert & Szabó, 2003), and defection invariably dominates cooperation. As in pairwise interactions, maintenance of cooperation requires additional mechanisms such as iterated interactions (Boyd & Richerson, 1988, Hauert & Schuster, 1997) or local interactions in spatially structured populations (Hauert & Doebeli, 2004, Pollock, 1989). Group interactions under less stringent conditions have hardly been studied.
In evolutionary dynamics we consider infinitely large populations consisting of a fraction of cooperators x and 1 – x defectors. According to replicator dynamics (Hofbauer & Sigmund, 1998), changes in x are determined by the relative performance of cooperators as compared to defectors:
where fC, fD denote the average fitness, i.e. the average payoff, of cooperators and defectors, respectively. Here we present a general framework to study interactions of cooperators and defectors in groups of N players. Cooperators are defined as individuals that provide a benefit to all members of the group at some cost to themselves. In contrast, defectors attempt to exploit the common enterprise, avoid the costs and provide no benefit. We denote by PC(k) and PD(k), respectively, the payoff for a single cooperator and a single defector in a group that contains k cooperators. Note that for the payoff of a cooperator, PC(k), the cooperator is one of the k cooperators in the group. Thus, PC(k) is defined for k = 1,…, N, while PD(k) is meaningful for k = 0,…, N – 1. Also note that, in accordance with social dilemmas, PC(k) < PD(k) must hold for all k = 1, …, N – 1, so that in any given group each defector has a higher payoff than each cooperator.
In groups that are formed at random according to binomial sampling, the average fitness of cooperators and defectors is given by
Here is the probability that there are j cooperators among the N – 1 other individuals in a group of size N in which the focal cooperator or defector finds itself. Consequently, the formulas above represent weighted averages of the payoffs of a focal cooperator (focal defector) facing j cooperators among the N – 1 co-players.
In natural systems, the actual value of the benefits provided by cooperators may depend on the number of cooperators in the group. For example, in the case of foraging yeast cells, the benefit provided by the first cooperator may be critical for survival, whereas the value of additional food decreases until, eventually, more food becomes useless because the cells are saturated. Similarly, cooperators may produce enzymes for enzyme-mediated reactions. The efficiency of the reaction may exhibit a faster than linear increase with concentration such that additional enzyme production has an enhanced value (Fersht, 1977, Hammes, 1982). This leads to discounted or synergistically enhanced benefits based on the number of cooperators in groups of interacting individuals.
In order to model the synergistically enhanced as well as the discounted net value of accumulated cooperative benefits, we assume that defectors and cooperators, respectively, receive the following payoffs in a group with k cooperators:
Hence, the first cooperator provides a benefit b which is shared by all N members of the group (including itself), the second one increases everyone’s benefit by b/N · w, and so on, to the last of the k cooperators in the group providing an additional benefit of b/N · wk–1. The costs of cooperation c, however, incur only to cooperators. If w = 1, then all cooperators provide the same incremental benefit b/N. This corresponds to the traditional formulation of Public Goods games with PD(k) = rkc/N (or b = rc) where r denotes the multiplication factor of the common pool. If w < 1, then the benefits are discounted and the value of the benefits provided by each additional cooperator is lower than the previous one. If w > 1, then the benefits are synergistically enhanced, and each additional cooperator provides incremental benefits of increasing value. At this point it is important to note that neither discounting nor synergy involve temporal components, i.e. this does not refer to potential future benefits in iterated interactions, which is an entirely different line of research with strong roots in economics (Fudenberg & Maskin, 1986).
It is easy to see from these expression that the equation fC(x*) = fD(x*) has at most one solution x* in the open interval (0, 1), which, if it exists, is given by x* = [1 – (cN/b)1/(N–1)]/[1 – w] (see Fig. 1). Therefore, the replicator dynamics (see Eq. (1)) has at most one interior equilibrium. It follows that the dynamics can be determined by considering the invasion capabilities of cooperators and defectors, i.e. the stability of the two trivial equilibria x = 0 and x = 1.
Based on these calculations, we obtain a natural classification of the dynamics in N-player group interactions, which generates the four basic scenarios of evolutionary dynamics (Nowak & Sigmund, 2004):
Note that the conditions for successful invasions can be directly obtained from PC(k) and PD(k), independent of binomial (or any other) sampling. The reason for this is that if cooperators are rare, then they can only increase in abundance if a single cooperator in a group of N – 1 defectors has a higher payoff than a group of N defectors. Thus, cooperators can invade if PC(1) > PD(0). Likewise, if defectors are rare, they invade only if PD(N – 1) > PC(N), i.e. if a single defector in a group with N – 1 cooperators receives a higher payoff than a cooperator in a group of N cooperators. These conditions are sufficient to generate the above classification. Scenarios (i) and (ii), i.e. dominant defection and co-existence, are reminiscent of the distinction between ’strong’ and ’weak’ altruism (Charlesworth, 1979, Wilson, 1979) where the act of cooperation incurs either absolute costs to the cooperator (b/N < c) or only relative costs as compared to the performance of defectors (b/N > c).
The different dynamical regimes are summarized in Fig. 2 using phase diagrams. The figure highlights the fact that transitions between the qualitatively different types of evolutionary games can be generated by varying the parameters w, c/b, and N. Not surprisingly, synergy (w > 1) generally favours cooperation as compared to discounting (w < 1), whereas increasing the cost-to-benefit ratio c/b or the group size N generally favours defection (Fig. 2). For discounted benefits, w < 1, defection reigns for b/N < c whereas for b/N > c cooperators persist. In the latter case, the critical group size given by nc = 1 + log(cN/b)/ log(w) separates by-product mutualism and the (generalized) Snowdrift game: for N < nc cooperators dominate defectors, whereas for N > nc cooperators and defectors coexist (Fig. 2b). For synergistic interactions, w > 1, cooperation dominates for b/N > c whereas for b/N < c defectors dominate if N > nc but if N < nc the dynamics becomes bistable (Fig. 2b).
The suggested discount/synergy framework can be fully analyzed regardless of the group size N because there exists at most a single interior equilibrium x* . Since this already holds for N = 2 no qualitatively different scenarios are found for larger groups of interacting individuals (even though group size does have an effect on the dynamics, see Fig. 2). The analysis hinges on the fact that the value of the benefits provided by k cooperators in a group of N interacting players are captured in the single parameter w.
More generally, one could, for example, introduce N different parameters αj, j = 1,…, N describing these effects, so that PD(k) = α1 + … + αk–1 (and e.g. again PC(k) = PD(k) – c). It is easy to see that, in principle, such systems can have anywhere between zero and N – 1 interior equilibria because fC, fD are polynomials in x of degree N – 1. Moreover, there are up to 2N different dynamical scenarios: N different arrangements of interior equilibria with two alternative configurations each because the stability of adjacent equilibria must alternate.
To illustrate the possibility of multiple interior equilibria in a biologically motivated setting, we first consider a situation that includes interactions among defectors. Let us assume a vicious type of defector that not only avoids the costs of cooperation but additionally competes with other defectors for their share of the benefit. The strength of competition increases with the number of defectors in a group. For simplicity we neglect competition among cooperators because if all individuals compete, the overall benefit of cooperation would essentially decrease. Defector competition can be described by changing their payoff to
where 0 < u < 1 measures how the strength of competition increases with the number of defectors: the defector payoff decreases due to the presence of N – 1 – k other defectors in the group, and this decrease is more pronounced for smaller u. The cooperator payoff is assumed to be unchanged (see Eq. (3)). With this payoff structure, analytical solutions are no longer attainable in general, but Fig. 3 demonstrates that the existence of a second interior equilibrium point can lead to dynamic regimes not seen in the case without defector interaction. This is illustrated in Fig. 3a,b for N = 3 and b/N < c. Competition among defectors supports cooperators such that they survive over a broader range of parameters. Moreover, this allows for stable co-existence of cooperators and defectors for w < 1 and b/N < c, which is otherwise impossible (c.f. Fig. 2). Note that for b/N > c the qualitative dynamics with u < 1 is the same as with u = 1, i.e., as in absence of defector interactions.
Multiple interior equilibria also occur if cooperators and defectors use different discounting/synergy factors w and v, i.e. the value of the common good provided by the cooperators is different for the two types. This could occur for example if the two strategy types correspond to different physiological states such that cooperators and defectors differ in their efficiency in taking advantage of the common resource. We assume that the cooperator payoff remains the same as before (see Eqs. (3) and (4)), and that the defector payoff becomes
Only v = w allows for an analytical solution of fC = fD but in general there are again up to N – 1 equilibria in (0, 1). This case is illustrated in Fig. 3c,d for N = 3. For example, for b/N > c large v increase the domain of co-existence, whereas small v promote dominance of cooperation. More complicated scenarios, including coexistence of multiple stable interior equilibria, can be observed for larger N.
In this paper, we present a general framework for cooperation in evolutionary N-player games that encompasses and recovers traditional games such as the Prisoner’s Dilemma or Public Goods games as special cases. The basis of our framework is formed by the concept of discounting and synergy, which simply takes into account that the actual value of the benefits provided by cooperators may depend on the total number of cooperators in the group. Thus, with discounting the value of the benefit provided by the first cooperator in a group is b, but the value of the benefits provided by each additional cooperator is discounted by a factor w < 1 as compared to the previous cooperator. All cooperators pay a cost c. For example, this leads to the definition of an N-player Snowdrift game if the discounting factor w and the cost-benefit ratio cN/b are sufficiently small (Fig. 2). It is important to recall that discounting does not refer to potential future benefits but rather to the process of accumulating benefits provided by multiple cooperators. In the case of synergy, the value of the benefit provided by each additional cooperator is synergistically enhanced by a factor w > 1.
Viewing the traditional games from the perspective of this general framework emphasizes that the various scenarios - Prisoner’s Dilemma or Public Goods games, Snowdrift games, by-product mutualism, and bistability - are interconnected through variations of the continuous parameters w, c/b and the group size N, which seamlessly relates seemingly disparate biological situations. For example, the discomfort with the Prisoner’s Dilemma as the sole model for cooperation is increasing (Clutton-Brock, 2002, Heinsohn & Parker, 1995, West et al., 2002), but viewing cooperation in the framework of discounting opens up natural connections to related scenarios, such as the Snowdrift game. In this way, our framework could prove to be helpful in bridging the gap between theoretical advances and experimental evidence.
In experimental settings it is notoriously difficult to quantitatively assess the fitness of strategic/behavioral patterns. For example, sticklebacks inspect their predators preferably in pairs and are believed to be trapped in a Prisoner’s Dilemma (Milinski, 1987). However, despite tremendous efforts, only the payoff ranking T > R > S has been experimentally confirmed (Milinski et al., 1997). Consequently, it remains unresolved whether the fish indeed engage in a Prisoner’s Dilemma (requiring R > P > S) or rather in a Snowdrift game (R > S > P). In another example, a Prisoner’s Dilemma interaction has been shown to occur between RNA phages within host cells (Turner & Chao, 1999), but selection alters the payoff structure such that cooperative and defective phage strains coexist in a Snowdrift game (Turner & Chao, 2003).
Similarly, it has been argued that Prisoner’s Dilemma interactions occur in the aforementioned case of enzyme production in foraging yeast cells (Greig & Travisano, 2004). Despite the apparent connection to the Prisoner’s Dilemma game given by the possibility of cheating, such frequency dependent benefits may be better captured by the Snowdrift game: if cooperators abound, defection is dominant and selfish individuals exploit the accrued benefits but as cooperators become rare, the costly enzyme production, may provide sufficient advantage to the producing individual despite by-product benefits to others, such that cooperation becomes dominant. Indeed, Greig & Travisano (2004) report that cheating was beneficial only if a substantial fraction of the yeast population was cooperating, i.e., producing the enzyme.
Outside of biology, the study of social dilemmas has received particular attention by experimental economists and anthropologists (Fehr & Gächter, 2002, Henrich et al., 2001, Panchanathan & Boyd, 2004). Humans display an apparently irrational, high readiness to cooperate in Public Goods and Ultimatum games (Güth et al., 1982, Nowak et al., 2000), which confounds the basic rationality assumptions of homo oeconomicus. In both games, defection is dominant but the Ultimatum game adds aspects of punishment because it can be interpreted as a Prisoner’s Dilemma interaction followed by a round of (costly) punishment (Sigmund et al., 2001). Punishment and reputation have been identified as very potent promoters of human cooperation in social dilemmas (Fehr & Gächter, 2002, Milinski et al., 2002, Wedekind & Milinski, 2000). Such additional mechanisms can be easily incorporated into our framework. However, already in Public Goods interactions, where only the multiplication factor of the common good depends on the total amount invested, qualitatively different outcomes can be generated, which allow e.g. for co-existence of cooperators and defectors in a generalized Snowdrift game. In experimental settings, variations of the multiplication factor could test the sensitivity of human behavior to quantitative and qualitative changes of the interaction characteristics.
In summary, Snowdrift games can be considered as social dilemmas that are intermediate between Prisoner’s Dilemma games (or Public Goods games in larger groups) and by-product mutualism, which occur whenever ordinary selfish behavior benefits others (Brown, 1983, West-Eberhard, 1975). By-product mutualism has also been put forth to challenge the Prisoner’s Dilemma for explaining patterns of cooperation in natural populations (Connor, 1995, 1996, Dugatkin, 1996, Milinski, 1996). Our general theoretical framework for cooperation in social dilemmas seems capable of reconciling the different viewpoints and emphasizes that the different dynamical domains of social dilemmas are related by continuous changes in biologically meaningful parameters.