Reactive strategy (game theory, $(p,q)=(C_C,C_D)$)

A reactive strategy cooperates with probability $p$ after the opponent cooperated and probability $q$ after the opponent defected. It ignores its own previous action. Independent behavioural randomization gives a four-state joint-action Markov chain. The reduced marginal recurrences depend on the prescribed initial action distribution.

Always defect chooses defection independently of the opponent's actions.

Always cooperate chooses cooperation independently of the opponent's actions.

Tit for tat cooperates initially and then repeats the opponent's previous action. Two such players cooperate forever without mistakes under this initial convention. It retaliates against defection and returns to cooperation after the opponent cooperates.

In a symmetric two-action stage game write $R,P,T_{\mathrm{tempt}},S$ for mutual cooperation, mutual defection, sole defection and sole cooperation payoffs. Suppose $T_{\mathrm{tempt}}\ge R$, $P\le R$ and $T_{\mathrm{tempt}}+S\le 2R$. Against an initially cooperative Tit for tat, every action sequence has limiting upper average payoff at most $R$.

For a deviator's actions $a_t$, put $a_{-1}=C$, $h(C)=0$ and $h(D)=T_{\mathrm{tempt}}-R$. The opponent at time $t$ plays $a_{t-1}$. Check the four transitions: $C$ to $C$ and $C$ to $D$ satisfywith equality; $D$ to $C$ uses $S\le 2R-T_{\mathrm{tempt}}$, and $D$ to $D$ uses $P\le R$. A telescoping sum gives the displayed finite-horizon bound. Divide by the horizon and let it increase. Mutual Tit for tat yields $R$ every round, so it is a Nash equilibrium under the long-run average payoff criterion. The argument is pathwise and therefore covers arbitrary deviations representable by finite-state machines, as well as action sequences outside that class.

For a reactive resident $(p,q)$ with $|p-q|<1$, let $z=q/(1-p+q)$ and $g(z)=R{z}^2+(S+T)z(1-z)+P(1-z{)}^2$. Its self-payoff and both ordered cross-payoffs against Tit for tat equal $g(z)$. At introduced Tit for tat frequency $ϵ$, its fitness difference is therefore the displayed expression. If $2R\ge S+T$, it is positive for $z<1$ because $R-g(z)=(1-z)[R-P+(R+P-S-T)z]$. Fully cooperative reached histories tie; the deterministic opposite-response strategy needs a separate cycle calculation. Initial advantage is order frequency rather than a positive invasion exponent at zero.

In an $L$-round iterated prisoner's dilemma, Tit for tat against always defect gets $S$ once and $P$ thereafter; the defector gets $T$ once and $P$ thereafter. Their self-payoffs are $R,P$. At introduced frequency $ϵ$, subtracting mixed-population payoffs gives $ϵ[(R-P)-(T+S-2P)/L]-(P-S)/L$. The displayed positive threshold applies when its denominator is positive and the threshold is less than one. Taking $L\to \infty$ first removes this finite entry cost, so the horizon and rare-mutant limits cannot be casually interchanged.

Under cooperative initial memories, matches involving only Tit for tat and always cooperate contain only cooperation, so all four payoffs equal $R$. The two-condition criterion for evolutionary stability has both differences zero. Thus Tit for tat is not an evolutionarily stable strategy against a strategy space containing unconditional cooperation, even if it is a Nash equilibrium. This counterexample is valid for finite horizons and discounting as well as infinite averages.

With initially independent actions, same-round independence is preserved by reactive updates: each next action uses the other independent previous action and fresh independent randomization. Marginal cooperation follows $x^{\prime }=q_1+(p_1-q_1)y$, $y^{\prime }=q_2+(p_2-q_2)x$. If the product of response slopes has modulus below one, these recurrences converge and the joint distribution is their product. Averaging $R,S,T,P$ over that distribution gives the long-time payoffs. At deterministic response slopes of modulus one, initial-memory-specific cycles must instead be averaged directly.