The simplest model is based on a large, well-mixed population of players randomly meeting each other (

Nowak and Sigmund, 1998a,b). The probability that the same two players meet more than once is negligible, in such a scenario. Whenever two players meet, chance decides who is in the role of the (potential) donor and who is recipient. Donors decide whether or not to confer a benefit

*b* to the recipient, at a cost

*c* to themselves. As usual, it is assumed that

. Donors providing help acquire the image

*G* (for good), and donors refusing help the image

*B* (for Bad). Thus players have binary images, entirely determined by what they decided when last in the position of donor. We can then consider three strategies: (1) the unconditional helpers

*AllC* who always provide help, (2) the unconditional defectors

*AllD* who always refuse to help, and (3) the conditional co-operators

*CondC*, who help recipients if and only if these have a

*G*-image. This strategy is the obvious analog of

*TFT* (tit for tat). It refuses help to those players who, in their previous round, refused to help. We denote by

*x*,

*y* and

*z* the frequencies of the three strategies (

*x*+

*y*+

*z*=1).

If a population contains only two of these strategies, the outcome is the same with direct as with indirect reciprocity (

Brandt and Sigmund, 2006).

*AllD* players dominate

*AllC* players. The competition of

*AllD* with the conditional strategy is bi-stable, as long as the cost-to-benefit ratio

*c*/

*b* is smaller than the probability

*w* for another round (with the same partner, in direct reciprocity, and with some other partner, in indirect reciprocity). In a mixture of unconditional and conditional co-operators, both do equally well. In order to avoid this dynamic degeneracy, and also to add a realistic feature, we assume that with a probability

, an intended help is not implemented (see also

Fishman, 2003; Fishman et al., 2001; Lotem et al., 1999). In this case, there exists a stable coexistence between

*AllC* and

*CondC*. In the interior of the simplex

which corresponds to the state space of the population (

*x*,

*y*,

*z*), the replicator dynamics (see e.g.,

Hofbauer and Sigmund, 1998) admits a line of rest points, which joins the

*AllD*+

*CondC* equilibrium with the

*AllC*+

*CondC* equilibrium and is given by a constant value of

*z*. In the vicinity of the

*AllC*+

*CondC* equilibrium, these rest points are stable (but not asymptotically stable, of course). These stable rest points correspond to highly cooperative populations. In the long run, however, a sequence of arbitrarily small endogenous perturbations could eventually push the population into the homogeneous state

*y*=1 corresponding to the fixation of

*AllD* (
). Hence cooperation can prevail for some time, in this model, but will ultimately break down. Although the details of the dynamics differ, the same conclusion holds with direct reciprocity too, if

*CondC* is replaced by

*TFT*. (We assume, in both cases, that the cost

*c* is smaller than the discounted benefit that can be expected in the following round, i.e.,

. If this does not hold, the triumph of

*AllD* is immediate.)

One of the reasons for the failure of *CondC* lies in its paradoxical nature. A conditional co-operator who refuses help to a player with image *B* acquires that image too. The *CondC*-player can, by helping a *G*-recipient on the next opportunity, redress that image. But during some time, the player is branded, and less likely to receive help. In this sense, the act of punishing a *B*-player is costly. The strategy can help to uphold cooperation in the population (for a while), but this comes at a price.

There is an obvious way to repair this weakness. It consists in discriminating between justified and unjustified defection. The same problem had already been treated in the context of direct reciprocation. It is well known that a pure

*TFT*-population is greatly plagued by errors in implementation. Each such error provokes a chain of backbiting. A variant of

*TFT* called

*Contrite TFT* can overcome this problem. It is based on the notion of ‘standing’ (

Sugden, 1986). In a similar vein, Sugden suggested that assessments, in indirect reciprocity, should take into account whether the recipient of a refusal to help had a

*B*- or a

*G*-image. Only the latter refusal should be considered as bad, and entail a

*B*-image to the non-helping Donor. ‘A player can keep his good standing even as he defects, as long as the defection is directed at a player with bad standing. We believe that Sugden's strategy is a good approximation to how indirect reciprocation actually works’.(

Nowak and Sigmund, 1998a) This point was taken up by a number of authors (

Panchanathan and Boyd, 2003; Leimar and Hammerstein, 2001).

This opens up a vast range of ways of assessing actions, (i.e., attributing a *G*- or a *B*-image), even if the actions are not directed at the observer. A first-order assessment rule simply depends on whether the donor helps the recipient or not. A second-order assessment rule takes into account, additionally, whether the recipient has a *G*-image or a *B*-image. A third-order assessment rule can depend, additionally, on the image of the donor. It may make a difference whether a *B*-player or a *G*-player provides help to a *B*-player. Altogether, there are 2^{8}=256 third-order assessment rules.

A strategy, in this indirect reciprocity game, depends not only on the assessment rule (i.e., how the player judges actions between two other players), but also how such an assessment is used to reach a decision on whether to help or not. A player could, for instance, decide to give help only to

*G*-players. But the player could also take into account the own image, and help, for instance, whenever the own image is

*B*, so as to remove the blemish as quickly as possible. There are 16 such action rules (including the two unconditional rules

*AllC* and

*AllD*, which do not depend on the assessment), and hence 4096=256 ×16 different strategies conceivable in this set-up (

Brandt and Sigmund, 2004; Ohtsuki and Iwasa, 2004). Not surprisingly, most are nonsensical, such as, for instance: ‘view everyone as

*G* who fails to give to a

*G* player, and help if and only if your own image is different from that of your recipient’.

Ohtsuki and Iwasa (2004, 2006) have shown that there exist, among the 256 assessment rules, only eight which can lead to cooperation, if the whole population embraces them. Each of these ‘leading eight’ is stable in the following sense: there exists a specific action rule such that no dissident minority using another action rule can do better, and invade. (In particular,

*AllC* or

*AllD* cannot invade.) None of these ‘leading eight’ is of first order. Each distinguishes between justified and unjustified defection. The eight rules agree on several points. It is always good to give help to a

*G*-player, and always bad to withhold help from a

*G*-player. Moreover, a good player refusing help to a

*B*-player does not loose the

*G*-image. There remain three situations: namely when someone (with image

*G* or

*B*) helps a

*B*-player, or when a

*B*-player refuses help to a

*B*-player. This yields the 2

^{3}=8 assessment systems belonging to the leading eight. Two of them are of second order, and in the following we shall only deal with them. They both agree in viewing (rather oddly) that a

*B*-player refusing to help a

*B*-player obtains a

*G*-image. They disagree on whether it is good to help a

*B*-player or not. The assessment that views it as good will be termed

*MILD*, the other

*STERN*. (The former has been studied by

Sugden, 1986, the latter by

Kandori, 1992). For both

*MILD* and

*STERN*, the corresponding action rule is: give help if and only if the recipient has image

*G*. (In particular, the own image will not influence the decision). The corresponding strategy will again be denote by

*MILD* resp.

*STERN*.

It is straightforward to analyze the replicator dynamics for a population consisting of the two unconditional strategies

*AllC* and

*AllD* and either the

*MILD* or the

*STERN* strategy (

Ohtsuki and Iwasa, 2007; Sigmund, 2010). In each case, we obtain a bi-stable situation (
). But what happens if both the

*MILD* and the

*STERN* strategy occur together in the population? This is not obvious. It is important to note that the stability of the leading eight merely means that no other action rule can invade. This does not imply that no other assessment rule can invade.

Ohtsuki and Iwasa (2007) and

Panchanathan and Boyd (2004) have assumed that all members of the population agree in their assessment. This means that every player has either the

*G*- or the

*B*-image in the eyes of all players. These authors would accept the view that it is unlikely that all players observe all interactions, but they assume that every interaction is observed by one player, whose assessment is then shared by all. No matter whether this is a likely scenario or not, it has clearly to be abandoned as soon as one is interested in the competition of several assessment rules. Assuming that assessment rules are private, c.f.

Brandt and Sigmund (2005),

Pacheco et al. (2006), and

Takahashi and Mashima (2006), this raises the question: which moral norm is likely to become established in the population?

Thus *G* and *B* mean different things in the eyes of a *MILD* or a *STERN* observer. To distinguish them, we may say that a player can be good or bad when assessed according to the *MILD* rules, and nice or nasty when assessed by the *STERN* rules. A priori, then, a player can be good and nice, good and nasty, bad and nice, or bad and nasty.

The replicator dynamics of a population consisting only of players adopting the

*MILD* or the

*STERN* strategy is disappointing. There is no selective advantage one way or the other, the segment representing all possible mixtures of

*MILD* and

*STERN* consists of rest points. If we add unconditional

*AllC*- and

*AllD*-players to the population, we observe a bi-stable outcome. Depending on the initial condition, either a homogeneous

*AllD* population will emerge, or a stable mixture of

*MILD* and

*STERN*. The best that can be said is that

*STERN* has a slight advantage, in the sense that whenever there are equally many

*STERN* and

*MILD* players (together with

*AllC*-players), the ratio of

*STERN* to

*MILD* will increase (

Uchida and Sigmund, 2010).

This analysis, so far, has relied on the assumption of perfect information. Every players knows about every interaction, either by direct observation or through gossip. This is clearly an unrealistic assumption. If we want to give it up, we must assume that every player has a private list of the images of all other players. Thus the image matrix

consists of entries

*G* or

*B*, depending on whether player

*j* has image

*G* or

*B* in the eyes of player

*i*. Whenever player

*j* is donor to some recipient player

*k*, then those players

*i* who observe the interaction will have an occasion for updating their image of

*j*. The new entries will depend on

and

(since we assume only second-order assessments, the image of the donor plays no role). But if player

*i* does not observe the interaction between

*j* and

*k*, the value

remains unchanged.

If observers are chosen at random, this updating process corresponds to a Markov chain on the space of image matrices. A rigorous analysis seems to offer considerable challenges.

Uchida (2010) has investigated the stochastic process by means of extensive computer simulations. The outcome is striking (See
). The smallest deviation from the perfect-information condition has disastrous consequences for a homogeneous population of

*STERN* players. In the long run, every entry of the image matrix is

*G* or

*B* with equal probability. The entries are uncorrelated. Thus effectively, a

*STERN* player is not doing any better than a player letting a coin-toss decide between helping or not. Compared with this, a homogeneous population of

*MILD* players does much better. A large majority of them will keep agreeing on the images of their co-players. (The percentage depends only on the probability

of mis-implementing an intended donation, and on the probability

*q* to observe a given interaction.) A

*CondC* population, on the other hand, ends up with a bad image for everyone. But a mixture of

*CondC* and

*AllC* can keep cooperating: meeting with an

*AllC*-player provides the conditional co-operators with an opportunity to redress their image. Clearly, this works also in the case of perfect information.

In order to obtain an intuitive feeling for these results, we may look at the updating process for

. For ease of notation, we replace the entries

*G* resp.

*B* by 1 resp. 0. With probability

, the entry remains unchanged. With probability

*q*, it will be replaced by the new image of

*j* in the eyes of player

*i*. This is 1 if either (a)

*j* gives to

*k*, and

*i* approves, or (b)

*j* refuses to help

*k*, and

*i* approves. The probability that

*j* helps

*k* is

, and the probability that

*i* approves is 1 if

*i* follows the

*MILD* or

*CondC* assessment rule, and it is

in the case of

*STERN*. The probability that

*j* refuses to help

*k* is

, and the probability that

*i* approves is

if

*i* follows the

*MILD* or

*STERN* assessment rule, and it is 0 if

*i* plays

*CondC*. If we assume (wrongly) that the images of

*k* in the eyes of

*i* and

*j*, i.e.,

and

, are independent, and if we denote by

the expected value of

etc, then in the stationary equilibrium, where

by symmetry, we obtain for the

*CondC*,

*MILD* and

*STERN* assessment rules, respectively:

The corresponding solutions are

*h*=0,

and

, respectively. Of course the independence assumption is false, but in the case of small

*q* it is a justifiable assumption, since different players are unlikely to base their assessments on the same observations.

This handful of results is a striking illustration of the fact that information conditions are of the utmost importance, for reputation-based indirect reciprocity (cf.

Masuda and Ohtsuki, 2007). This was stressed already in the first papers on this topic. In

Nowak and Sigmund (1998a,b),

*q* denotes the probability that a player knows about the reputation of another player, i.e., has some information about the behavior of that player. With probability

*1*−

*q*, the co-player is unknown. In this case, it is assumed that the recipient receives the benefit of doubt, i.e., is held to be a

*G*-player.

*CondC*-players could resist invasion by

*AllD* players if

(or, in a more elaborate model, if

). In

Uchida (2010)
*q* is the probability that a given player observes the last action of a co-player. If not, then the co-players former image will remain unaltered. Eventually, models will have to encompass both types of uncertainty. It could be that in Alice's eyes, player Bob is a stranger. It could also be that Alice knows Bob, but has missed Bob's last action as a donor.

Whatever the interpretation of

*q*, it seems likely that it is not a constant. In particular, it is reasonable to assume that the social network of a player grows with time. In this case, the player will be more and more likely to know the reputation of their recipients, or to have observed their latest interactions. In

Fishman et al. (2001),

Mohtashemi and Mui (2003) and

Brandt and Sigmund (2005), it is shown that appropriate assumptions can turn the

*CondC*+

*AllC* equilibrium into a stable attractor, able to repel invasion attempts by

*AllD*-minorities.

It is an obvious weakness of all models considered so far that they are based on a very short memory only. Assessments are updated according to the action last observed. In real life, reputations are not always based on one action only. If we know that a player has cooperated for a long time, but suddenly that player defects in one interaction, we will not necessarily lose our good opinion of that player (but rather assume that the recipient deserved no better). In particular,

Berger (forthcoming) has shown that a tolerant first-order assessment rule (

*Tolerant Scoring*) can stably sustain cooperation. Such an assessment with built-in tolerance against single defections can be based on sampling

*two* actions in the recipients past.

Several models consider a more sophisticated evaluation system, for instance with a score that is not binary (see e.g.,

Nowak and Sigmund, 1998a, or

Leimar and Hammerstein, 2001). This provides stability to cooperation: a few isolated defections will not destroy the good reputation that a player has accumulated, but only slightly reduce it. In another vein,

Suzuki and Akiyama (2007a,b) have analyzed indirect reciprocity for interactions in larger groups, and found a wealth of interesting dynamic behavior. The evolution of norms in multi-level selection models has been studied by

Chalub et al. (2006) and

Pacheco et al. (2006).