= Solution
The <payoff> table is blank in the original PDF, so denote its four row-player <payoffs> by $R,S,T,P$ for $CC,CD,DC,DD$, with the <prisoner's dilemma> ordering $T>R>P>S$. No numerical entries are inferred. Let the two <reactive strategies> be $(p_1,q_1)$ and $(p_2,q_2)$, where $p_i$ is cooperation after the opponent's cooperation and $q_i$ cooperation after defection.
The joint action states form a four-state <Markov chain>. In state order $CC,CD,DC,DD$, its transition <matrix> is
$$
\begin{pmatrix}
p_1p_2&p_1(1-p_2)&(1-p_1)p_2&(1-p_1)(1-p_2)\\
q_1p_2&q_1(1-p_2)&(1-q_1)p_2&(1-q_1)(1-p_2)\\
p_1q_2&p_1(1-q_2)&(1-p_1)q_2&(1-p_1)(1-q_2)\\
q_1q_2&q_1(1-q_2)&(1-q_1)q_2&(1-q_1)(1-q_2)
\end{pmatrix}.
$$
The prescribed initial memories give <independent> first actions with cooperation <probabilities> $p_1,p_2$. Same-round actions remain <independent>: each player's next action uses the other player's previous action and its own <independent> random draw. Thus their marginal cooperation <probabilities> obey
$$
x_{t+1}=q_1+(p_1-q_1)y_t,
\qquad y_{t+1}=q_2+(p_2-q_2)x_t.
$$
Put $a_i=p_i-q_i$. If $|a_1a_2|<1$, the two-step recurrences contract and the <long-run payoff of reactive strategies> follows from
$$
\boxed{x_*=\frac{q_1+a_1q_2}{1-a_1a_2},\qquad
y_*=\frac{q_2+a_2q_1}{1-a_1a_2}.}
$$
The joint <stationary distribution> is $(x_*y_*,x_*(1-y_*),(1-x_*)y_*,(1-x_*)(1-y_*))$. Player 1's <mean> <payoff> is $Rx_*y_*+Sx_*(1-y_*)+T(1-x_*)y_*+P(1-x_*)(1-y_*)$; exchange $S,T$ for player 2.
The deterministic boundaries must retain the initial memories rather than divide by zero. Two <Tit for tat> players cooperate forever and earn $R$. Two players who always do the opposite of the opponent's previous action alternate $DD,CC$ and average $(P+R)/2$. One of each cycles through all four states and each averages $(R+S+T+P)/4$. These are the only cases with $|a_1a_2|=1$. A periodic <Markov chain> need not have convergent state <probabilities>, but its long-time average <payoff> is defined by its cycle frequencies.
Back to article page
