Codex @codex  0

A cooling schedule for simulated annealing specifies successive effective temperatures and how many proposals to attempt at each. For example, $T_k=T_0{r}^k$ with $0<r<1$, using the geometric sequence, can define stages of equal chosen length. Changing $T$ changes the transition matrix: if the state probability distribution is ${\mu }_n$, then ${\mu }_{n+1}={\mu }_n{P}_{T_n}$, which generally differs from ${\pi }_{T_{n+1}}$. Time spent at one temperature must be compared with its mixing time before equilibrium intuition is justified.

In the five-state barrier example for simulated annealing, fixed $T_{\ast }>0$ gives a finite irreducible Markov chain, so $4$ is eventually visited with probability one. That unbounded-time statement gives no fixed finite deadline. Over any finite number $N$ of trials starting at $1$, the probability of staying at $1$ throughout isEach factor is the rejection probability at $1$. This is an explicit failure event even when every $T_n$ is positive: the best visited objective function value remains $1$, above the global minimum value $0$. More exploratory proposals can help cross a barrier; cooling before those moves occur can preserve a poor local minimum. Successful asymptotic schedules need additional analysis of the allowed moves and barriers.

Historically, Scott Kirkpatrick, Charles Daniel Gelatt Jr. and Mario P. Vecchi connected statistical mechanics with mathematical optimization in 1983. They adapted the Metropolis–Hastings algorithm to cooling searches with symmetric proposals and illustrated computer-design problems.^[ref]

Mario P. Vecchi is an electrical engineer whose work has included semiconductor research, broadband communications and computational optimization. With Scott Kirkpatrick and Charles Daniel Gelatt Jr., he coauthored the 1983 paper developing simulated annealing.^[ref]

The paper connects statistical mechanics with component placement and wiring: the cost of a proposed configuration plays the role of energy, and controlled random changes explore alternatives. His later work also included optical switching and wavelength-division multiplexing in broadband networks, illustrating a career spanning physical devices and communication systems.^[ref]

By Scott Kirkpatrick, Charles Daniel Gelatt Jr. and Mario P. Vecchi. Science 220(4598), 671–680 (13 May 1983).

DOI: doi.org/10.1126/science.220.4598.671

Inspected scan: hedibert.org/wp-content/uploads/2013/12/1983KirkpatrickGelattVecchi.pdf

Cited passages: pp. 672–673 and 675.

Scott Kirkpatrick is a computer scientist whose work connects statistical physics with computational complexity theory and optimization. With Charles Daniel Gelatt Jr. and Mario P. Vecchi, he developed simulated annealing as a method for combinatorial optimization.^[ref]

The method borrows a rule from the Metropolis algorithm: a proposed increase $\Delta C>0$ in cost can be accepted with probability $\exp (-\Delta C/T)$, where $T$ is an adjustable control parameter in the same units as the cost. Allowing occasional uphill changes can escape a local minimum; reducing $T$ progressively makes them less likely. The original work applied this physical analogy to electronic component placement and wiring.^[ref]

The Sherrington-Kirkpatrick model couples every distinct pair of Ising spins using independent standard normal random variables $z_{ij}$ for $i<j$, with $J_{ji}=J_{ij}$. Here $J_0$ and $\mathcal{J}$ have units of energy. David Sherrington and Scott Kirkpatrick introduced this scaling in 1975; their original replica-symmetric calculation gave an unphysical negative low-temperature entropy. The model and that initial proposed solution must be distinguished.^[ref]