Game theory

Game theory studies decisions whose consequences depend on what other decision makers choose. A bidder's gain depends on competing bids in an auction; a driver's travel time depends on other drivers in a congestion game. This strategic interdependence makes choosing well a different problem from optimization against a fixed environment: each participant's best response depends on the others' choices.

A normal-form game specifies the players, their available pure strategies, and their payoffs for each strategy profile. Larger payoffs mean more preferred outcomes for that player. Unless stated otherwise, a two-player table below lists $(u_1,u_2)$: the row player's payoff first, the column player's second. Each player maximizes their own expected utility; the numbers need not represent money, and comparing different players' numerical scales is unnecessary.

The central question is whether everyone's choices fit together. A Nash equilibrium consists of mutual best responses: no unilateral deviation improves a player's expected payoff. It can involve definite choices, or mixed strategies that deliberately randomize. Matching pennies shows why randomization can be essential; the coordination game shows how players can share an interest in coordinating while disagreeing over which outcome to choose. Such shared gains are absent from a two-player zero-sum game, where one player's gain is exactly the other's loss.

These ideas are useful both for analyzing incentives and for designing them. Mechanism design asks how changing the rules changes participants' best responses; evolutionary game theory studies how successful strategies spread. The mathematical appeal is that local comparisons of payoffs can reveal a globally consistent pattern of choices. Consistency alone does not imply Pareto efficiency, uniqueness, or that actual play will converge to a Nash equilibrium.

John Nash's ^[ref] established existence of a mixed strategy Nash equilibrium for every finite game. His ^[ref] developed the theory beyond two-player zero-sum games. The distinction between conflicting and shared interests is fundamental to the subject.

A game whose players' utilities are specified separately, without assuming a fixed sum. Improving one player's utility need not worsen another's. A finite perfect-information game can be solved by backward induction using entire utility tuples and each moving player's own coordinate, as in max-n search.

A turn-based game on a directed graph in which a player tries to force a designated target set. Its winning region is a least fixed point: add that player's vertices with some winning successor and the opponent's vertices with all successors winning. The finite-graph version uses the same construction as a winning-position attractor in an infinite game.

With disjoint terminal wins for two players and infinite plays valued as draws, compute both reachability attractors while keeping terminals fixed. Entry ranks give terminating winning strategies; positions outside both regions admit defence avoiding a forced loss. A cycle alone is not a certificate of a draw if it also offers a forceable winning exit.

A decision problem in which the same information set can be reached more than once along a single path, so the decision-maker does not know which visit is occurring. It violates perfect recall. A behavioural strategy uses a common probability at each visit and independent random draws, making terminal probabilities nonlinear in that probability.

For instance, let stopping at the first visit pay zero, continuing and then stopping pay eight, and continuing twice pay one. With continuation probability $p$, expected payoff is $8p(1-p)+p^2=16/7-7(p-4/7{)}^2$. Its maximum $16/7$ exceeds the best deterministic-plan payoff of one. Randomizing once between the deterministic plans cannot reproduce the two fresh decisions.

An assignment of an action probability distribution to every information set of a player in an extensive-form game. Randomization is fresh at each encounter. It differs from a mixed strategy, which chooses a complete deterministic plan once before play. Perfect recall is central to equivalence of their induced outcome distributions.

Consider a zero-sum game in which a player chooses a binary action, forgets it, and chooses again, while an opponent makes one hidden binary choice. The first player's payoff is one if all three choices agree and zero otherwise. The later information set merges both initial-action histories. This violates perfect recall without revisiting the same information set, unlike an absent-minded decision problem.

A behavioural strategy chooses the first and second actions with independent left probabilities $p,r$. Its security level payoff is $\min (pr,(1-p)(1-r))$, since the opponent selects the less favorable target action. This minimum is at most $\sqrt{p(1-p)r(1-r)}\le 1/4$, attained at $p=r=1/2$.

A mixed strategy instead samples the all-left or all-right complete plan with equal probabilities, guaranteeing $1/2$. Against an opponent who chooses each target equally often, each complete constant plan earns $1/2$ and each nonconstant plan earns zero, so no mixed strategy can guarantee more. The separation comes from correlation between a sampled plan's actions; fresh independent randomizations cannot retain it after the initial action is forgotten.

A player has perfect recall when the information sets never erase that player's previously available information or earlier chosen actions. With perfect recall, randomizing once over complete pure strategies and using suitable independent randomizations at information sets can induce the same outcome distributions.

Without perfect recall, one information set may be encountered twice on the same play path. A behavioural strategy then uses the same probabilities with fresh draws at both encounters; its outcome distribution need not be attainable by any mixed strategy over deterministic plans.

In a finite extensive-form game with perfect recall, each mixed strategy has a realization-equivalent behavioural strategy, and every behavioural strategy has a realization-equivalent mixed strategy. The latter direction samples all contingent actions independently in advance. For the former, condition the random complete plan on the player's own action and information history leading to an information set; perfect recall makes that own history common to its nodes. Conditional action probabilities then reproduce the terminal path factors. At an information set reached with zero probability under that player's own plan distribution, choose any action distribution. The result does not assert equivalence without perfect recall.^[ref]

An extensive-form game in which each information set contains a single decision node. Each acting player knows the entire preceding sequence of moves. Finite games with this property can be solved for a subgame perfect equilibrium by backward induction.

A collection of decision nodes that the acting player cannot distinguish when choosing an action in an extensive-form game. All nodes in an information set have the same acting player and available actions. A behavioural strategy assigns the same action probabilities throughout that set.

In an extensive-form game, a player has imperfect information when some decision information set contains more than one node. The player cannot distinguish those histories at the time of acting. This does not itself imply failure of perfect recall: a player can remember all their own earlier information and actions while never having observed an opponent's hidden choice. Conversely, merging histories in which the player's own earlier actions differ violates perfect recall.

A game described by a game tree, with the acting player and available actions specified at each decision node, payoffs at terminal nodes, and information sets indicating which decision nodes a player can distinguish. A pure strategy specifies an action at every information set belonging to that player, including ones not reached during realized play.

Two players alternate proposals for dividing a pie, with a fixed final round. Agreement ends the game; rejection shrinks the next available pie by a factor $0<\alpha <1$. Final rejection gives both players zero. Offers range continuously over the entire available pie. Backward induction in games determines immediate agreement and the proposer's share through the continuous-offer bargaining acceptance lemma and the finite-horizon bargaining parity effect. Discrete offer grids can change both the boundary behavior and the exact shares.

In finite-horizon alternating-offer bargaining, the proposer with $k$ rounds left retains fraction $P_k$ of the currently available pie. Terminal acceptance gives $P_1=1$. After rejection, the responder becomes proposer and can obtain fraction $\alpha P_{k-1}$ of the current pie. The continuous-offer bargaining acceptance lemma therefore gives $P_k=1-\alpha P_{k-1}$. Summing the resulting finite geometric series proves the displayed formula.

Odd horizons give shares above $1/(1+\alpha )$, decreasing toward it; even horizons give shares below it, increasing toward it. Between two available horizons $k_1<k_2$, choose $k_1$ if $k_1$ is odd and $k_2$ if $k_1$ is even. This concerns the initial proposer and immediate agreement, rather than waiting until the selected final round to obtain its share.

Let a proposer divide a pie of size $M>0$ by continuously offering its responder $q\in [0,M]$. Rejection gives fixed continuation payoffs $b$ to the proposer and $c$ to the responder, with $0\le c<M$ and $b<M-c$. Then a subgame perfect equilibrium must have the responder accept $q=c$ with probability one. The proposer offers exactly $c$, retaining $M-c$.

Sequential rationality forces rejection for $q<c$ and acceptance for $q>c$. If acceptance at $c$ had probability $a<1$, that offer would give the proposer $a(M-c)+(1-a)b<M-c$. Every larger offer gives less than $M-c$ and can be improved by a smaller offer still above $c$; every smaller offer gives $b<M-c$. The supremum $M-c$ would therefore be unattained. Randomizing offers cannot attain it either: the nonnegative gap from $M-c$ would be strictly positive almost surely and hence have positive expectation. No best response would exist, contradicting equilibrium. Acceptance at $c$ removes this gap and makes $q=c$ the unique optimal proposal. The assumptions $c<M$ and $b<M-c$ exclude degenerate indifference of the proposer.

In a finite perfect-information game without chance moves, solve terminal decision points first. At each earlier decision point, its player chooses an action maximizing its own payoff among the already solved continuations. Record a choice in every subgame, including those not reached by the root choice. This constructs a subgame perfect equilibrium. Tied choices may yield several equilibrium strategies and several outcomes. The name refers to Ernst Zermelo; the computational procedure is an application of backward induction.

Suppose a finite perfect-information game has no chance moves, and every player strictly ranks the possible outcome labels. Different terminal histories may have the same outcome label and thus the same payoff. Every subgame then has a unique subgame perfect equilibrium outcome, even though its equilibrium strategy need not be unique.

The proof is backward induction. Each child subgame has a unique continuation outcome. The moving player's strict ranking selects exactly one best label among those outcomes. Several actions can tie only by leading to that same label, which proves uniqueness at the current node. For a sequential election, the labels may be winning candidates: a voter can have several optimal ballots, all producing its same favourite achievable winner. If preferences have ties between different labels, the conclusion can fail.

Two strategies for the same player are realization-equivalent when they induce the same probability distribution on terminal histories against every fixed opposing strategy profile. The requirement is stronger than having equal expected payoff against one particular opponent. For a finite extensive-form game with perfect recall, Kuhn's theorem gives this equivalence between appropriate mixed strategies and behavioural strategies.

A finite game represented succinctly by propositional logic. Each player controls a disjoint collection of Boolean variables and chooses their truth values. Together those choices determine a Boolean valuation. A player's goal is a propositional formula, with payoff one if satisfied and zero otherwise. The control sets normally form a set partition of all variables.

A pure strategy strategy profile is a Nash equilibrium if every unsatisfied player remains unsatisfied after every change to that player's controlled variables. Checking a candidate can therefore require solving a Boolean satisfiability problem, even though evaluating its current payoffs is easy.

Given a succinct Boolean game and a complete pure strategy strategy profile, determine whether it is a Nash equilibrium. The problem is co-NP-complete for goal propositional formulas, even with one player. This differs from existential equilibrium checking in a Boolean game, which searches for some qualifying Nash equilibrium.

A negative instance has a short complexity certificate: a currently unsatisfied player and new truth values for that player's controlled Boolean variables that make the goal true. Evaluating the current and changed propositional formulas takes polynomial time, proving membership in co-NP.

For hardness, given a propositional formula $F(X)$, let one player control $X$ and a fresh variable $z$, with goal $z\wedge F(X)$. The all-false candidate has payoff zero. It is a Nash equilibrium exactly when $F$ is an unsatisfiable Boolean formula, since any satisfying assignment permits an improvement by also setting $z$ true. This is a polynomial-time many-one reduction from unsatisfiability. The fresh guard ensures that the candidate loses independently of the input formula's value there.

For nonnegative additive valuations, the requirement that each allocated item be assigned to an agent who values it strictly positively. This condition alone does not require allocating every item.

Let $S_i=\{j:v_{ij}>0\}$ and restrict agent $i$'s constraint domain to subsets of $S_i$, or to nonempty subsets when required. If $S_i\cap S_{\ell }=\varnothing$, the two bundles are automatically disjoint and each agent values the other's bundle at zero. Both directional envy-free allocation binary constraints are then automatic. Thus positive-support overlaps determine which pairs can still matter in the residual constraint graph.

An allocation of bundles $B_i$ is envy-free if each agent weakly prefers its own bundle to every other agent's bundle, evaluating both through its own value function. With an additive valuation,The reverse comparison uses $v_{\ell }$, so both directions are independent requirements. Ties are permitted. Empty bundles and unallocated items are separate modelling choices, not consequences of envy-freeness.

A bundle-value function for which an agent's value for a set of items is the sum of its item values. Different agents may assign different values to the same item. With nonnegative item values, a bundle containing no positively valued item has value zero. The formula excludes interactions such as a pair of items being valuable only when held together.

Allocation of resources among agents according to a specified fairness requirement. For indivisible items, each bundle is a set of whole items, and a complete allocation assigns every item exactly once. An envy-free allocation is one fairness criterion. Existence depends on the permitted allocations: indivisibility can prevent fairness that would be possible after splitting resources or permitting unallocated items.

An alternating grid-removal game in which choosing an occupied cell removes it and every occupied cell above and to its right. The player who removes the designated lower-left poison cell loses. Reachable occupied-cell sets are order ideals for coordinatewise order. A nonempty reachable position always contains the poison cell, and removing that cell empties the board. The monotone boundary representation of a grid order ideal supports compact board updates.

A game value lying strictly between $-r$ and $r$ for every positive rational number $r$, in the ordering of partizan games. Infinitesimal values can determine a sum whose numerical part is zero, while they cannot overturn a nonzero integer advantage.

The impartial game $\ast =\{0\mid 0\}$, so either player can move to zero. It is equivalent to a Nim heap of size one. It is incomparable with zero, and $\ast +\ast =0$ under the disjunctive sum of games.

A partizan game played on a graph attached to the ground. Left removes blue edges and Right removes red edges; after a removal, all edges disconnected from the ground disappear. The player with no legal move loses under the normal-play convention.

A Hackenbush position whose graph is a single path attached to the ground at one endpoint. Finite red-blue stalks have dyadic rational game values; the initial monochromatic run determines the integer part and subsequent colors determine successive binary fractions.

A number game is a recursively defined normal-play partizan game whose options are number games and every left option is smaller than every right option. Its value is the simplest surreal number between those options. Finite number games have dyadic rational values.

A two-player game on nine positions arranged in a three-by-three grid. Players alternate marking empty positions and stop when one has a full row, column or diagonal, or when the board is full. A game search must stop at the first win. Distinct move histories can reach the same position; canonical occupancy sets or bitboards make that equality available to memoization. Quotienting by board symmetry changes the requested output unless distinct oriented positions are subsequently restored.

Backward induction with an $n$-component utility vector. At a node controlled by player $i$, evaluate each child, choose a child maximizing coordinate $i$ and copy its entire vector. A fixed tie rule makes the returned vector well defined. By mathematical induction from terminal positions, the selected continuation is a best response at every node of a finite perfect-information game, giving a pure strategy subgame perfect equilibrium. The vector must be copied intact; separate coordinatewise maxima need not describe any attainable outcome.

Suppose every terminal pair satisfies $|U_A+U_B|\le ϵ$. At a B-node with immediate A-parent, let $\alpha$ be the A-coordinate of a completed alternative already retained by the parent. If a completed B-child supplies B-coordinate $y$, then every eventual B-selected pair $(u,v)$ has $v\ge y$ and hence $u\le ϵ-y$. Therefore $ϵ-y<\alpha$ permits discarding this B-option and skipping its remaining children. Equality is also safe if the parent keeps its existing option on a tie.

The inequality remains valid at completed subtrees because max-n search copies one terminal pair. If the observed pair is $(x,y)$, then $x+y\ge -ϵ$, so $u\le x+2ϵ$ and $x+2ϵ\le \alpha$ is a weaker sufficient test. At $ϵ=0$ this reduces to the usual MIN-node alpha-beta pruning cutoff. A discarded option is not assigned a fabricated exact pair; the proof uses a certified alternative at the immediate parent.

A game tree node whose successor is chosen according to a specified probability distribution. Its expected utility is a weighted sum of successor values, rather than the maximum or minimum used at a player's decision node.

Backward induction on a finite deterministic game tree of a two-player zero-sum game. Terminal payoffs are propagated by maximization at one player's decision nodes and minimization at the other's. With full information, the resulting choices give an optimal strategy against the opposing player's best response. A cyclic state graph requires a winning condition on infinite plays and a graph algorithm rather than unrestricted recursive unfolding.

The order in which legal child positions are searched. Good move ordering supplies useful alternatives early and increases alpha-beta pruning; poor move ordering can leave the worst-case time complexity of full minimax search unchanged.

The convention that a player unable to move loses. In a finite impartial game without draws, a position is winning exactly when it has a move to a losing position.

An impartial game is a game in which both players have the same legal moves from any given position. Under the normal-play convention, a player with no legal move loses. For finite games without draws, Grundy numbers encode positions for the Sprague-Grundy theorem.

An impartial game played under the normal-play convention.

Sprague-Grundy theory studies finite acyclic impartial games under the normal-play convention by assigning each position a Grundy number. The assignment recursively takes the minimum excludant of the values of all legal followers. The Sprague-Grundy theorem then reduces a disjunctive sum of games to the bitwise exclusive or of its component values. This framework explains both why a zero value is losing and how independently understood games can be combined without exploring the entire product of their state spaces.

Every finite acyclic impartial game under the normal-play convention has a Grundy number. The Grundy number of a sum of independent games is the bitwise exclusive or of their individual Grundy numbers, so the sum is losing exactly when that value is zero.

A subtraction game has a heap of nonnegative integer size. A legal move subtracts an allowed positive amount without making the size negative; normal play declares an empty move set losing.

In a disjunctive sum of games, each move changes exactly one component game. Under normal play, a player loses when no component has a legal move.

A heap game in which, after the first move, a player may remove at most twice the number removed on the preceding move. Under the usual first-move restriction of fewer than all counters, Fibonacci numbers and the Zeckendorf representation characterize the winning moves.

A game on two positive integers in which a move subtracts a positive multiple of the smaller from the larger without making it negative. The position is then reordered. This is an impartial game related to the Euclidean algorithm; the exact terminal and winning convention must be stated.

The value $g(P)=\mathrm{mex}\{g(Q):P\to Q\}$ of a game position, where mex is the smallest nonnegative integer missing from the values of its legal successors. Terminal positions have value zero. The Sprague-Grundy theorem explains why these values combine by exclusive or.

An impartial game played with heaps of counters: a move removes a positive number of counters from exactly one heap. Under the normal-play convention, a position is losing exactly when the bitwise exclusive or of its heap sizes is zero.

The bitwise exclusive or of a list of nonnegative integers. For heap sizes in ordinary Nim, a zero nim-sum characterizes a losing position under the normal-play convention.

Two people can each have a clear incentive to choose an outcome that leaves both worse off. The prisoner's dilemma isolates this tension between individual best responses and Pareto efficiency. For example, two partners can each save effort by withholding a promised contribution, yet both would prefer a successful joint project to mutual withholding. The model applies when each person's payoffs really have the ordering below; the names “cooperate” and “defect” alone do not establish a prisoner's dilemma.

In the symmetric two-player normal-form game, each player chooses cooperation $C$ or defection $D$ simultaneously, without observing the other's current choice. The four payoffs are the mutual-cooperation reward $R$, mutual-defection punishment $P$, sole-defector temptation $T$, and sole-cooperator payoff $S$, with $T>R>P>S$. Defection is a strictly dominant strategy: it beats cooperation against either opposing pure strategy. This definition and the resulting one-shot Nash equilibrium are discussed in Steven Kuhn, Prisoner's Dilemma, Stanford Encyclopedia of Philosophy, section 1.

Take the concrete payoffs $T=5$, $R=3$, $P=1$, $S=0$. Each entry below gives the row player's payoff followed by the column player's payoff:

For the row player, switching from $C$ to $D$ improves the payoff from $3$ to $5$ against $C$, and from $0$ to $1$ against $D$. The column player has the same comparisons, so $D$ is a strictly dominant strategy for each. Even against a mixed strategy choosing $C$ with probability $q$, the expected payoff advantage is $2q+(1-q)=1+q>0$. Thus cooperation cannot appear with positive probability in a best response, and $(D,D)$ is the unique one-shot Nash equilibrium, including mixed strategies.

At $(D,D)$, a unilateral deviation to $C$ lowers the deviator's payoff from $1$ to $0$. A joint change to $(C,C)$ instead raises both payoffs from $1$ to $3$: mutual cooperation Pareto dominates mutual defection, making the Nash equilibrium fail Pareto efficiency. A Nash equilibrium tests unilateral deviations, so it need not maximize shared gains. This is the useful lesson: improving the available outcome may require changing incentives, rather than telling each participant to choose a better best response.

The additional inequality $2R\ge T+S$ says that mutual cooperation gives each player at least as much average payoff as equal-time alternation between the two asymmetric outcomes. It is a further assumption, not a consequence of $T>R>P>S$. Here it holds strictly: $6>5$, and alternating exploitation gives each player average payoff $5/2<3$. This comparison concerns equal-time averages, not every discounted sequence. See Kuhn, section 3, on the additional cardinal-payoff condition.

A social game in which cooperation costs its actor $c$ and benefits its partner by $b$, with $b>c>0$. Mutual cooperation yields $b-c$ to each, unilateral defection yields $b$ to the defector and $-c$ to the cooperator, and mutual defection yields zero.

The iterated prisoner's dilemma is a repeated game with a prisoner's dilemma at each round. Earlier choices can change later best responses: a present gain from defection may be outweighed by lost future cooperation. A pure strategy now specifies an action after every possible observed history, including histories that its own prescribed play would never reach. The horizon, observations, initial actions, and payoff convention must be specified before assessing a Nash equilibrium or subgame perfect equilibrium.

For the following worked example, repeat the prisoner's dilemma above infinitely at rounds $t=0,1,\dots$. Assume public perfect monitoring: after each round both players observe both actions exactly, without error. They share a discount factor $0\le \delta <1$ and evaluate a path of stage payoffs $u_i(t)$ using the normalized discounted payoffThe round-zero weight is one before normalization, including when $\delta =0$. A larger discount factor gives future payoffs more weight; the geometric series makes a constant stage payoff have the same normalized value. These assumptions and normalization follow ^[ref]. The numerical payoffs here are our $3,5,0,1$ example; the lecture uses a different table.

Suppose both players use grim trigger: play $C$ initially and while every past action of both players was $C$; after any observed $D$, play $D$ forever. With no unilateral deviation, each gets normalized payoff $3$. A first unilateral deviation to $D$ earns $5$ now, then at most $1$ each round because the opponent switches permanently to always defect. The deviator attains that continuation payoff by choosing $D$ thereafter. At any history of mutual cooperation, the comparison is thereforeCooperation is a best response precisely whenEquivalently, without normalization the geometric series givesMultiplying both unnormalized payoffs by the same positive factor $1-\delta$ leaves the best response comparison unchanged.

For example, with discount factor $\delta =1/4$, the first unilateral deviation gives normalized payoff $4>3$, so mutual grim trigger is not a Nash equilibrium. At $\delta =3/4$, it gives $2<3$, so cooperation is preferable. At $\delta =1/2$ the two payoffs tie: the incentive to cooperate is weak, which is sufficient for a subgame perfect equilibrium.

The punishment is credible after every history containing a defection: the opponent's pure strategy then prescribes $D$ forever, and $D$ remains the stage best response, earning $1$ rather than $0$. Together with the cooperative-history comparison, this establishes mutual grim trigger as a subgame perfect equilibrium for $1/2\le \delta <1$. Delaying the first unilateral deviation merely delays the same gain or loss; further cooperation during punishment only lowers the deviator's payoff. Mutual always defect also remains a subgame perfect equilibrium, so patience permits cooperation without making it inevitable.

This calculation assumes an infinite horizon and $\delta <1$; it does not define a discounted payoff at $\delta =1$. With a finite, commonly known final round and positive weight on each round, backward induction instead gives mutual defection in every round of this stage game. Observational errors change the model too: one mistaken defection permanently activates grim trigger. Historical dependence, initial memories, and the payoff time convention also affect whether a cooperative rule resists or invades other pure strategies; the existing reactive strategy and Tit for tat discussions concern those additional questions.

A grim trigger is a history-dependent pure strategy in a repeated game that switches permanently to punishment after a trigger event. In the iterated prisoner's dilemma above, the trigger is any player's observed defection, and punishment is always defect. Unlike Tit for tat, grim trigger never restores cooperation after the opponent cooperates again. Whether mutual grim trigger is a subgame perfect equilibrium depends on the stage payoffs, discount factor, and monitoring assumptions; it is not an unconditional property of the rule. See Ozdaglar, lecture 15, slides 12–15.

A reactive strategy cooperates with probability $p$ after the opponent cooperated and probability $q$ after the opponent defected. It ignores its own previous action. Independent behavioural randomization gives a four-state joint-action Markov chain. The reduced marginal recurrences depend on the prescribed initial action distribution.

Always defect chooses defection independently of the opponent's actions.

Always cooperate chooses cooperation independently of the opponent's actions.

Tit for tat cooperates initially and then repeats the opponent's previous action. Two such players cooperate forever without mistakes under this initial convention. It retaliates against defection and returns to cooperation after the opponent cooperates.

In a symmetric two-action stage game write $R,P,T_{\mathrm{tempt}},S$ for mutual cooperation, mutual defection, sole defection and sole cooperation payoffs. Suppose $T_{\mathrm{tempt}}\ge R$, $P\le R$ and $T_{\mathrm{tempt}}+S\le 2R$. Against an initially cooperative Tit for tat, every action sequence has limiting upper average payoff at most $R$.

For a deviator's actions $a_t$, put $a_{-1}=C$, $h(C)=0$ and $h(D)=T_{\mathrm{tempt}}-R$. The opponent at time $t$ plays $a_{t-1}$. Check the four transitions: $C$ to $C$ and $C$ to $D$ satisfywith equality; $D$ to $C$ uses $S\le 2R-T_{\mathrm{tempt}}$, and $D$ to $D$ uses $P\le R$. A telescoping sum gives the displayed finite-horizon bound. Divide by the horizon and let it increase. Mutual Tit for tat yields $R$ every round, so it is a Nash equilibrium under the long-run average payoff criterion. The argument is pathwise and therefore covers arbitrary deviations representable by finite-state machines, as well as action sequences outside that class.

For a reactive resident $(p,q)$ with $|p-q|<1$, let $z=q/(1-p+q)$ and $g(z)=R{z}^2+(S+T)z(1-z)+P(1-z{)}^2$. Its self-payoff and both ordered cross-payoffs against Tit for tat equal $g(z)$. At introduced Tit for tat frequency $ϵ$, its fitness difference is therefore the displayed expression. If $2R\ge S+T$, it is positive for $z<1$ because $R-g(z)=(1-z)[R-P+(R+P-S-T)z]$. Fully cooperative reached histories tie; the deterministic opposite-response strategy needs a separate cycle calculation. Initial advantage is order frequency rather than a positive invasion exponent at zero.

In an $L$-round iterated prisoner's dilemma, Tit for tat against always defect gets $S$ once and $P$ thereafter; the defector gets $T$ once and $P$ thereafter. Their self-payoffs are $R,P$. At introduced frequency $ϵ$, subtracting mixed-population payoffs gives $ϵ[(R-P)-(T+S-2P)/L]-(P-S)/L$. The displayed positive threshold applies when its denominator is positive and the threshold is less than one. Taking $L\to \infty$ first removes this finite entry cost, so the horizon and rare-mutant limits cannot be casually interchanged.

Under cooperative initial memories, matches involving only Tit for tat and always cooperate contain only cooperation, so all four payoffs equal $R$. The two-condition criterion for evolutionary stability has both differences zero. Thus Tit for tat is not an evolutionarily stable strategy against a strategy space containing unconditional cooperation, even if it is a Nash equilibrium. This counterexample is valid for finite horizons and discounting as well as infinite averages.

With initially independent actions, same-round independence is preserved by reactive updates: each next action uses the other independent previous action and fresh independent randomization. Marginal cooperation follows $x^{\prime }=q_1+(p_1-q_1)y$, $y^{\prime }=q_2+(p_2-q_2)x$. If the product of response slopes has modulus below one, these recurrences converge and the joint distribution is their product. Averaging $R,S,T,P$ over that distribution gives the long-time payoffs. At deterministic response slopes of modulus one, initial-memory-specific cycles must instead be averaged directly.

Evolutionary game theory studies how a population's composition changes when the success of a pure strategy depends on which other pure strategies it encounters. The central feedback is a frequency-dependent payoff: becoming common changes the opponents a strategy meets, which can change whether it does well. In a coordination game, common behaviour can reinforce itself; in the coexistence example below, being rare is an advantage. This makes evolutionary game theory useful for asking which behaviours can persist and which initial compositions lead to different outcomes, without assuming that each participant solves an optimization problem.

In the basic model, a large, well-mixed population has finitely many inherited pure strategies; interactions sample opponents according to their population frequencies. Assume that a strategy's expected payoff determines its per capita net reproductive growth rate, with a common baseline allowed. These are modelling assumptions: reproduction must transmit the strategy, encounters must match the proposed sampling rule, and the conversion from payoff to growth must be justified for an application. The replicator equation below describes the resulting continuous-time frequencies. Taylor and Jonker's ^[ref] explicitly derive this growth model.

For learned behaviour, reproduction can instead be replaced by an assumed copying rule: an individual samples another individual uniformly and copies a better-performing pure strategy at a rate proportional to the positive payoff difference. At frequencies $x_i,x_j$, the net flow from $j$ to $i$ is $x_i{x}_j(f_i-f_j)$, because the two directional positive parts subtract to $f_i-f_j$. Summing over $j$ gives $x_i(f_i-\bar{f})$, the replicator equation. This derivation states a particular mechanism; an arbitrary rule for learning or imitation need not give the same dynamical system.

Three questions must be distinguished. A symmetric Nash equilibrium tests whether an individual can improve its expected payoff against the resident composition. An evolutionarily stable strategy tests whether each distinct rare mutant is outperformed after it is introduced. Dynamical stability tests the response of an actual dynamical system to small changes of composition, and asymptotic stability additionally requires nearby trajectories to converge. For a finite symmetric bimatrix game under the continuous-time replicator equation, every evolutionarily stable strategy is a symmetric Nash equilibrium and a locally asymptotically stable rest point; the converse from asymptotic stability to an evolutionarily stable strategy fails in general. See Ross Cressman and Yi Tao, The replicator equation and other game dynamics (2014), Theorems 1–2 and Example 1. These statements refer to stability on the full probability simplex, including perturbations introducing currently absent pure strategies.

A concrete experiment makes the frequency feedback visible. In 2009, Gore, Youk and van Oudenaarden competed two yeast strains: a producer of the enzyme invertase, which enables sucrose hydrolysis, and a mutant lacking the corresponding SUC2 gene. Released sugars can feed neighbours, so avoiding production can be advantageous. They used shaken 5 mL cultures with 5% sucrose at $30{}^{\circ }\mathrm{C}$, diluted daily after 23 hours of growth. Different labelled proteins allowed the two strains to be counted separately by flow cytometry; selective plating periodically checked the counts. The cooperator required externally supplied histidine, so restricting that amino acid imposed an adjustable growth disadvantage. See Snowdrift game dynamics and facultative cheating in yeast, pp. 253 and 255–256, Methods Summary.

Starting with rare producers and, separately, rare nonproducers tested both invasion directions. Under the reported coexistence conditions each rare strain increased; changing histidine or added glucose shifted the eventual composition. Their explanation used nonlinear growth benefits, rather than a fixed pairwise payoff matrix. This motivates testing both rarity regimes before inferring an evolutionarily stable strategy or fitting a replicator equation; it does not identify the illustrative tables below with measured payoffs. See Gore, Youk and van Oudenaarden, Figures 1–3.

An early motivation was understanding why animal contests can involve restraint even when escalation might win an individual encounter. Maynard Smith and Price's ^[ref] introduced the evolutionarily stable strategy in this setting. Taylor and Jonker's ^[ref] then studied explicit continuous and discrete dynamics. Static resistance to invasion and the paths populations follow are related questions, but they require separate definitions.

A rule prescribing different actions in different states or histories. Conditionality does not require conscious calculation.

The replicator equation says that a strategy grows in frequency when its expected payoff exceeds the population's current mean payoff. It is an ordinary differential equation for a composition, rather than for total population size. The subtraction of the mean is what makes relative success matter: even a type growing in absolute number can shrink as a fraction if other types grow faster.

Let $x_i(t)$ be the frequency of pure strategy $i$, with $n$ available pure strategies andIn a symmetric bimatrix game, let the payoff matrix $A$ have entry $a_{ij}$ equal to the payoff to strategy $i$ against $j$. The other player's payoff in that encounter is $a_{ji}$: symmetry between player roles does not require $A=A^T$. Uniform random encounters give the expected payoffs and their population meanAfter choosing the time unit so that the growth-rate scale is one, the continuous-time replicator equation isHere a payoff is measured on a scale proportional to net growth per unit time, not merely as an arbitrary preference ranking. More generally one can specify locally Lipschitz frequency-dependent payoffs $f_i(x)$ directly; the same equation with $\bar{f}={\sum }_i{x}_i{f}_i$ then defines a unique frequency trajectory. The linear pairwise model and its reproductive interpretation are given in Taylor and Jonker (1978), sections 2–3; the formulas here also follow by the following direct normalization.

Suppose the number $N_i$ of type $i$ obeys ${\dot{N}}_i=N_i[b(t)+f_i(x)]$, where the baseline $b(t)$ is identical for all types. Set $N={\sum }_i{N}_i>0$ and $x_i=N_i/N$. The quotient rule givesThus the common baseline cancels. Adding the same constant to every entry of $A$ leaves the replicator equation unchanged; multiplying $A$ by a positive constant changes only the speed along trajectories. The same payoff matrix can therefore describe frequency evolution at different growth-rate baselines and time scales.

The probability simplex is invariant in forward time. On ${\sum }_i{x}_i=1$,Normalization alone would not exclude negative frequencies. Nonnegativity follows as well: along a solution,An absent type remains absent, and a positive frequency remains positive at every finite time. For the fixed payoff matrix model, bounded rates on the compact probability simplex give global forward solutions. Frequencies can still tend to zero as $t\to \infty$. Each face is invariant, so this model has no spontaneous introduction of new pure strategies; mutation or outside introductions would require additional terms. For two present types the logarithm of their frequency ratio satisfies $\frac{d}{dt}\log (x_i/x_j)=f_i-f_j$, directly expressing the accumulated payoff advantage.

For two pure strategies, write the composition as $(x,1-x)$, where $x$ is the frequency of strategy 1, and letTheir payoff difference and the resulting scalar ordinary differential equation areIn the interior $0<x<1$, the sign of the payoff difference determines the direction of motion. The endpoints are always equilibrium points, and an additional interior equilibrium point exists precisely when the affine payoff difference has a zero strictly between them. At a simple interior zero $x_{\ast }$, $F^{\prime }(x_{\ast })=x_{\ast }(1-x_{\ast })\Delta f^{\prime }(x_{\ast })$: a negative value gives asymptotic stability, a positive value gives instability. At the endpoints $F^{\prime }(0)=b-d$ and $F^{\prime }(1)=c-a$, with stability understood relative to $[0,1]$. A zero derivative requires examining the actual flow rather than this strict sign test.