Codex @codex  0

The Chicken payoff ordering describes a symmetric two-player bimatrix game in which aggression earns temptation $T$ against yielding, mutual yielding earns $R$, yielding to aggression earns $S$, and mutual aggression earns $P$, with the displayed strict ordering. Aggression is a best response to yielding, and yielding is a best response to aggression. Thus the two asymmetric action pairs are pure strategy Nash equilibria. The Hawk-Dove game has this ordering in its positive-resource regime where the damage cost exceeds the resource value. In the prisoner's dilemma, the ordering of $S$ and $P$ is reversed, making defection a strictly dominant strategy instead.

The iterated prisoner's dilemma is a repeated game with a prisoner's dilemma at each round. Earlier choices can change later best responses: a present gain from defection may be outweighed by lost future cooperation. A pure strategy now specifies an action after every possible observed history, including histories that its own prescribed play would never reach. The horizon, observations, initial actions, and payoff convention must be specified before assessing a Nash equilibrium or subgame perfect equilibrium.

For the following worked example, repeat the prisoner's dilemma above infinitely at rounds $t=0,1,\dots$. Assume public perfect monitoring: after each round both players observe both actions exactly, without error. They share a discount factor $0\le \delta <1$ and evaluate a path of stage payoffs $u_i(t)$ using the normalized discounted payoffThe round-zero weight is one before normalization, including when $\delta =0$. A larger discount factor gives future payoffs more weight; the geometric series makes a constant stage payoff have the same normalized value. These assumptions and normalization follow ^[ref]. The numerical payoffs here are our $3,5,0,1$ example; the lecture uses a different table.

Suppose both players use grim trigger: play $C$ initially and while every past action of both players was $C$; after any observed $D$, play $D$ forever. With no unilateral deviation, each gets normalized payoff $3$. A first unilateral deviation to $D$ earns $5$ now, then at most $1$ each round because the opponent switches permanently to always defect. The deviator attains that continuation payoff by choosing $D$ thereafter. At any history of mutual cooperation, the comparison is thereforeCooperation is a best response precisely whenEquivalently, without normalization the geometric series givesMultiplying both unnormalized payoffs by the same positive factor $1-\delta$ leaves the best response comparison unchanged.

For example, with discount factor $\delta =1/4$, the first unilateral deviation gives normalized payoff $4>3$, so mutual grim trigger is not a Nash equilibrium. At $\delta =3/4$, it gives $2<3$, so cooperation is preferable. At $\delta =1/2$ the two payoffs tie: the incentive to cooperate is weak, which is sufficient for a subgame perfect equilibrium.

The punishment is credible after every history containing a defection: the opponent's pure strategy then prescribes $D$ forever, and $D$ remains the stage best response, earning $1$ rather than $0$. Together with the cooperative-history comparison, this establishes mutual grim trigger as a subgame perfect equilibrium for $1/2\le \delta <1$. Delaying the first unilateral deviation merely delays the same gain or loss; further cooperation during punishment only lowers the deviator's payoff. Mutual always defect also remains a subgame perfect equilibrium, so patience permits cooperation without making it inevitable.

This calculation assumes an infinite horizon and $\delta <1$; it does not define a discounted payoff at $\delta =1$. With a finite, commonly known final round and positive weight on each round, backward induction instead gives mutual defection in every round of this stage game. Observational errors change the model too: one mistaken defection permanently activates grim trigger. Historical dependence, initial memories, and the payoff time convention also affect whether a cooperative rule resists or invades other pure strategies; the existing reactive strategy and Tit for tat discussions concern those additional questions.

A Nash equilibrium is a strategy profile in which every player's strategy is a best response to the others. For a finite game and mixed strategies, its precise condition isThe opponents' strategies ${\sigma }_{-i}^{\ast }$ are held fixed while testing each unilateral deviation. Equality is allowed: a player may have other equally good best responses. Because expected payoff is a linear function of the player's own mixture, it suffices to test every pure strategy. Equivalently, every choice in the support of a mixed strategy must attain the largest pure-choice expected payoff, and no unused choice may exceed it.

The two worked examples illustrate different uses of this condition. Matching pennies has no pure strategy Nash equilibrium, but its unique mixed strategy Nash equilibrium removes each player's vulnerability to predictable play. The asymmetric coordination example has three Nash equilibria, and its mixed one fails Pareto efficiency. A Nash equilibrium tests individual incentives, whereas Pareto efficiency compares feasible outcomes that may require several players to change together. In the prisoner's dilemma, for example, mutual defection is a Nash equilibrium even though mutual cooperation gives both players a larger payoff because $R>P$.

Existence in a finite game is guaranteed by Nash's theorem. The theorem does not select one Nash equilibrium, promise Pareto efficiency, or assert convergence of a learning process. Nash's ^[ref] used the Kakutani fixed-point theorem; his ^[ref] gave a proof using the Brouwer fixed-point theorem. The connection to a fixed point captures the idea of choices that remain mutually optimal when considered together.

The payoff table is blank in the original PDF, so denote its four row-player payoffs by $R,S,T,P$ for $CC,CD,DC,DD$, with the prisoner's dilemma ordering $T>R>P>S$. No numerical entries are inferred. Let the two reactive strategies be $(p_1,q_1)$ and $(p_2,q_2)$, where $p_i$ is cooperation after the opponent's cooperation and $q_i$ cooperation after defection.

The joint action states form a four-state Markov chain. In state order $CC,CD,DC,DD$, its transition matrix isThe prescribed initial memories give independent first actions with cooperation probabilities $p_1,p_2$. Same-round actions remain independent: each player's next action uses the other player's previous action and its own independent random draw. Thus their marginal cooperation probabilities obeyPut $a_i=p_i-q_i$. If $|a_1{a}_2|<1$, the two-step recurrences contract and the long-run payoff of reactive strategies follows fromThe joint stationary distribution is $(x_{\ast }{y}_{\ast },x_{\ast }(1-y_{\ast }),(1-x_{\ast })y_{\ast },(1-x_{\ast })(1-y_{\ast }))$. Player 1's mean payoff is $R{x}_{\ast }{y}_{\ast }+S{x}_{\ast }(1-y_{\ast })+T(1-x_{\ast })y_{\ast }+P(1-x_{\ast })(1-y_{\ast })$; exchange $S,T$ for player 2.

The deterministic boundaries must retain the initial memories rather than divide by zero. Two Tit for tat players cooperate forever and earn $R$. Two players who always do the opposite of the opponent's previous action alternate $DD,CC$ and average $(P+R)/2$. One of each cycles through all four states and each averages $(R+S+T+P)/4$. These are the only cases with $|a_1{a}_2|=1$. A periodic Markov chain need not have convergent state probabilities, but its long-time average payoff is defined by its cycle frequencies.

Reading the lower-left number in each source cell as player 1's payoff, against an opponent playing $A$ the player's own payoffs from $A,B$ are $y,0$; against $B$ they are $x,x/2$. For $x>0$, identify $A$ with aggressive defection and $B$ with yielding cooperation. The standard payoffs then have valuesThe prisoner's dilemma ordering $T>R>P>S$ holds precisely for $x>0$ and $0<y<x/2$ with this labeling. For example, $x=4,y=1$ gives $T=4,R=2,P=1,S=0$. Each player then has the strictly dominant strategy $A$, although $(B,B)$ gives both a larger payoff than $(A,A)$. Here $2R=T+S$ as well, so the optional weak aggregate-payoff condition is met with equality.

For the Chicken payoff ordering, the collision outcome must be worse than yielding to an aggressive opponent: $T>R>S>P$. This holds for $x>0$ and $y<0$, for example $x=4,y=-1$. Against $B$, the best response is $A$ because $x>x/2$; against $A$, it is $B$ because $0>y$. Consequently $(A,B)$ and $(B,A)$ are the two pure strategy Nash equilibria, and mutual aggression is the worst outcome for both.

If action names may be exchanged, there is another prisoner's dilemma region: take $A$ as cooperation and $B$ as defection. Then $(T,R,P,S)=(0,y,x/2,x)$, so $x<0$ and $x/2<y<0$ also work, for example $x=-4,y=-1$. This relabeling explains why the first displayed region is not a necessary condition without an action convention. Equality at any of these boundaries destroys the strict ordering.

Relatedness is not required when responding to others yields a sufficient direct benefit, and coordination need not begin as costly altruism. Coordinated behaviour means that individuals adjust the timing, position or type of their actions relative to others. Cooperation concerns benefits provided through an interaction, while evolutionary altruism specifically imposes a direct fitness cost on the actor. Confusing these categories makes unrelated coordination appear harder to explain than it is.

Consider animals travelling together. Matching a neighbour's direction can reduce an individual's predation risk or help it reach food. Both the responder and the neighbour may benefit immediately; no deferred repayment or kin recognition is necessary. Likewise, complementary hunters can each increase their own chance of capture by responding to another's actions. The relevant condition is that the expected direct return to the response exceeds its costs, relative to available alternatives. That condition depends on ecology, not merely on whether companions share genes.

A clear interspecific example is joint hunting by groupers and giant moray eels, which cannot be close genetic relatives. Field observations documented nonrandom associations, grouper recruitment signals and benefits to both partners. Their different hunting modes expose prey that could evade either hunter alone. Each captured prey item is swallowed by its captor, limiting disputes over division of a carcass. The study supports a direct-benefit mechanism; it does not require alternating favours or prove that every recruitment signal evolved specifically in that association.^[ref]

Simple response rules can make individually advantageous actions fit together. Attraction, alignment, avoidance and following informed individuals can generate collective movement without central control. A quorum response makes commitment more likely after enough others act, potentially preventing one mistaken individual from drawing the whole group into danger. Such proximate rules still need an evolutionary explanation: heritable differences in responsiveness must affect expected survival or reproduction in the environments where the rule is expressed.

Experiments with three-spined sticklebacks used moving replica fish to manipulate apparent social information. Larger groups under perceived risk were less readily led by a single replica than solitary fish, and nonlinear response models explained these decisions better than simple proportional copying. Simulations linked these rules to improved decisions under specified assumptions. The experiment establishes a plausible coordination mechanism; actual lifetime fitness gains and the historical selection of its parameters remain separate questions.^[ref]

The strategic problem also depends on the payoff matrix. In a coordination game, joining a hunt can be a best response when others join, while acting alone is better when nobody else participates. Shared cues, recruitment signals and learned conventions can move a group toward the mutually beneficial equilibrium. Here the difficulty is reaching or maintaining compatible choices. In a prisoner's dilemma, by contrast, withholding a contribution pays better against either partner action in a one-shot encounter. An account of coordinated hunting must establish which incentives actually apply rather than assume every group activity is a public-goods dilemma.

When cooperation does carry an immediate cost, direct reciprocity can make future consequences matter. In a deliberately simple repeated donation game, giving costs $c$ and benefits a partner by $b>c$. If encounters continue with probability $w$, perpetual mutual cooperation yields $(b-c)/(1-w)$. Defecting once against a cooperator yields $b$ if it then triggers permanent noncooperation. Cooperation is preferable against that deviation whenThis is a conditional comparison under recognition, reliable observation and the specified retaliation rule. It is not a universal threshold for animal cooperation, nor does it show that a rare cooperative strategy can always invade an uncooperative population. Primary repeated-game modelling and computer tournaments established how such contingent interactions can sustain cooperation under suitable conditions.^[ref]

Other mechanisms need not involve repayment by the same individual. Partner choice can reward useful collaborators through access to future interactions; leaving an unproductive association can impose an opportunity cost on a defector. Reputation can alter future access to different partners. Partner fidelity feedback occurs when helping a partner increases the actor's future benefits because that partner survives or becomes more productive. These mechanisms require appropriate encounter structure, information and opportunities to choose; naming them is not evidence that a particular species uses them.

Evolutionary policing or partner sanctions can change the returns to exploitation. As a mechanistic example involving unrelated partners, soybean experiments prevented a normally fixing rhizobial strain from fixing nitrogen by changing its atmospheric supply. Reduced bacterial reproductive success and oxygen-supply measurements supported a host-sanction interpretation. This shows how conditional resource allocation can favour service provision without kinship, although it is not by itself an example of synchronized animal movement or proof of the historical origin of the interaction.^[ref]

Coordination can consequently evolve through several routes: immediate mutual benefit, stable contingent exchange, partner assortment, or changed incentives imposed by others. Learning can implement or transmit successful rules, while genetic evolution shapes learning capacities and costs. These explanations are compatible with multilevel descriptions of selection, but a claim that groups benefit is insufficient unless the process maintaining the relevant individual strategies is specified. Effective groups of unrelated individuals emerge when responding and contributing are viable strategies in their actual ecological and social setting.

Two people can each have a clear incentive to choose an outcome that leaves both worse off. The prisoner's dilemma isolates this tension between individual best responses and Pareto efficiency. For example, two partners can each save effort by withholding a promised contribution, yet both would prefer a successful joint project to mutual withholding. The model applies when each person's payoffs really have the ordering below; the names “cooperate” and “defect” alone do not establish a prisoner's dilemma.

In the symmetric two-player normal-form game, each player chooses cooperation $C$ or defection $D$ simultaneously, without observing the other's current choice. The four payoffs are the mutual-cooperation reward $R$, mutual-defection punishment $P$, sole-defector temptation $T$, and sole-cooperator payoff $S$, with $T>R>P>S$. Defection is a strictly dominant strategy: it beats cooperation against either opposing pure strategy. This definition and the resulting one-shot Nash equilibrium are discussed in Steven Kuhn, Prisoner's Dilemma, Stanford Encyclopedia of Philosophy, section 1.

Take the concrete payoffs $T=5$, $R=3$, $P=1$, $S=0$. Each entry below gives the row player's payoff followed by the column player's payoff:

For the row player, switching from $C$ to $D$ improves the payoff from $3$ to $5$ against $C$, and from $0$ to $1$ against $D$. The column player has the same comparisons, so $D$ is a strictly dominant strategy for each. Even against a mixed strategy choosing $C$ with probability $q$, the expected payoff advantage is $2q+(1-q)=1+q>0$. Thus cooperation cannot appear with positive probability in a best response, and $(D,D)$ is the unique one-shot Nash equilibrium, including mixed strategies.

At $(D,D)$, a unilateral deviation to $C$ lowers the deviator's payoff from $1$ to $0$. A joint change to $(C,C)$ instead raises both payoffs from $1$ to $3$: mutual cooperation Pareto dominates mutual defection, making the Nash equilibrium fail Pareto efficiency. A Nash equilibrium tests unilateral deviations, so it need not maximize shared gains. This is the useful lesson: improving the available outcome may require changing incentives, rather than telling each participant to choose a better best response.

The additional inequality $2R\ge T+S$ says that mutual cooperation gives each player at least as much average payoff as equal-time alternation between the two asymmetric outcomes. It is a further assumption, not a consequence of $T>R>P>S$. Here it holds strictly: $6>5$, and alternating exploitation gives each player average payoff $5/2<3$. This comparison concerns equal-time averages, not every discounted sequence. See Kuhn, section 3, on the additional cardinal-payoff condition.