 Harmonic function for a Markov chain

A harmonic function assigns a value to each state of a Markov chain such that one transition leaves its expected value unchanged. This turns a question about many possible future sample paths, such as which endpoint is reached first, into a local averaging rule. For a transition matrix $P$, a real-valued function $h$ is harmonic at state $i$ whenHere ${\mathbb{E}}_i$ means expected value conditional on starting at $X_0=i$; on a countable state space the sum must be absolutely convergent. If this holds at every state, write $Ph=h$. At an absorbing state the equation is automatic, so prescribed boundary conditions supply information that the averaging rule cannot.

For a finite Markov chain, choose a boundary set $B$ that can be reached from every state, and assign boundary values $b$. If $T_B$ is its first hitting time, the harmonic extension isChoosing $b$ to be the indicator function of one target within $B$ gives a hitting probability. This is a discrete Dirichlet problem: enforce the averaging rule outside $B$ and fix the values on $B$. See Hao Wu's MIT lecture 8, harmonic extension, slide 5. The construction extends the mean-value idea of a harmonic function; for a random walk on a graph it is a discrete harmonic function.