Boltzmann's entropy formula ($S=k_B\log \Omega$)

Why does a gas spread through an available container? One part of the answer is counting: a nearly uniform macrostate can be realized by vastly more microstates than a macrostate with every particle crowded into one corner. Boltzmann's entropy formula turns that multiplicity into entropy. The useful insight is that a large entropy measures how many microscopic realizations fit a specified macroscopic description, rather than how untidy a picture looks.

For a specified macrostate $M$ containing $\Omega (M)$ accessible, equally likely microstates, writeHere $k_B$ is the Boltzmann constant, and $\ln$ is the natural logarithm, also written $\log$ in this article's title. The count $\Omega (M)$ is dimensionless; entropy has units of energy divided by temperature. Always specify the macrostate, the constraints and the model's microscopic resolution before counting. A microcanonical ensemble provides a familiar equilibrium setting for the equal-probability assumption. ^[ref]

The natural logarithm compresses an enormous count into a manageable number and makes products of counts into sums of entropies; see additivity of Boltzmann entropy. With this counting convention, one allowed microstate has zero Boltzmann entropy. This zero refers to the chosen model and resolution; it does not by itself assign an absolute entropy to a real material.

A single photon thermalized in a macroscopic body adds energy $hc/\lambda$. If its heat capacity is sufficiently large that the temperature change is negligible, the entropy increase is $hc/(\lambda T)$. The Boltzmann entropy then gives the displayed ratio of accessible microstates. The exponential can be enormous even though the deposited energy is macroscopically tiny.

The part of entropy associated with the number of distinct spatial or occupancy arrangements consistent with chosen macroscopic constraints. For equally weighted configurations, the Boltzmann entropy formula gives the displayed expression. Unequal probabilities require $-k_B{\sum }_i{p}_i\ln p_i$ over configurations. For $n$ indistinguishable empty sites among $M$ otherwise occupied sites, ${\Omega }_{\mathrm{conf}}=\left(\genfrac{}{}{0ex}{}{M}{n}\right)$. Vibrational or internal degrees of freedom can contribute additional entropy even at fixed site arrangement.

Randomly distributing $n$ species with equal fractions over $N$ sites gives $\Omega =N!/[(N/n)!{]}^n$. Stirling approximation gives $S=k_{B}N\ln n$, or $R\ln n$ per mole of sites. Configurational entropy alone does not decide whether a multicomponent alloy is a stable single thermodynamic phase.

Take $N=20$ distinguishable, labeled particles and two boxes of equal volume. Retain only whether each particle is in the left or right box: use a bit $0$ for left and $1$ for right. Each length-20 string is one coarse microstate of this toy model. Assume the labels make independent side choices with equal probability, so the $2^{20}=1048576$ strings are equiprobable. Equal volumes motivate these weights for noninteracting particles with no preference for either box; equal volumes alone would not establish them for an arbitrary interacting gas.

Now retain only the number $k$ in the left box as the macrostate. Choosing which $k$ labels are left gives a binomial coefficient, soThe factorials count assignments to labels; the probability of a macrostate sums the probabilities of its constituent microstates. This is the binomial distribution with parameters $20$ and $1/2$. The same binomial coefficient appears in the counting of distinguishable two-level systems, although here the macrostate records occupancy rather than energy. ^[ref]

The largest binomial coefficient occurs at $k=10$, the balanced macrostate. All-left means $k=20$ and supplies just one string. ThusThere are $184756$ balanced strings for every all-left string, even though each individual string has the same probability. Exact balance has about $17.6\%$ of the total probability; equilibrium reasoning concerns the many macrostates near balance too, rather than certainty of exactly ten particles on each side. The diagram compares the counts with their natural logarithms.

Four fair, independent coins give a smaller worked example with the same binomial distribution. If $k$ is the number of heads, the five macrostates $k=0,1,2,3,4$ contain respectively $1,4,6,4,1$ microstates. Their probabilities are respectively $1/16,4/16,6/16,4/16,1/16$. In particular, two heads is one macrostate containing six ordered coin sequences, with Boltzmann entropy $k_B\ln 6$ in this counting analogy. A prescribed ordered sequence has one realization. The next diagram enumerates the same sixteen strings as left/right assignments.

In Boltzmann entropy of a two-box occupancy, impose an initial restriction that all labels are left. There is one allowed coarse microstate. Remove that restriction, allowing all occupancy values: the allowed set is now the union of every occupancy macrostate, containing $2^N$ strings. If the final description assigns equal probability to this whole set, its Boltzmann entropy change isThe counting identity is the binomial theorem. For twenty labels, $\Delta S=20k_B\ln 2\approx 13.862944k_B$. This final description fixes neither box occupancy; it is coarser than the particular macrostate $k=10$, whose Boltzmann entropy is $k_B\ln 184756$. The two counts answer different questions, so substituting the balanced macrostate for the full allowed set would change the calculation.

For four labels, the initial all-left string is 0000; removing the restriction makes all sixteen strings available. The $k=2$ column has only six strings. Accordingly, the whole allowed set has Boltzmann entropy $k_B\ln 16=4k_B\ln 2$, while the balanced macrostate has Boltzmann entropy $k_B\ln 6$.

Removing a barrier changes the allowed set immediately, but it does not instantaneously randomize the actual probability distribution. Motion and equilibration must make those configurations accessible. The counting result illustrates the configurational contribution to free expansion; it does not describe the relaxation time or every contribution to the entropy of a real gas.

For two subsystems with independently specified macrostates, suppose every allowed microstate of $A$ can be paired with every allowed microstate of $B$. Their allowed set is a Cartesian product. With equal probability for each pair, Boltzmann's entropy formula givesFor example, take two independent copies of the toy model, each with two labels choosing between its two boxes. Each copy has four configurations and Boltzmann entropy $k_B\ln 4$. Combining their independent labels gives sixteen configurations and Boltzmann entropy $k_B\ln 16=2k_B\ln 4$. This explains why entropy adds even though microscopic counts multiply. ^[ref]

The product assumption matters. A constraint shared between the subsystems can forbid some pairs; for example, fixing only their combined energy generally requires summing over its possible divisions. See entropy maximization under thermal contact for how thermal equilibrium selects the dominant division. Statistical independence cannot be inferred merely from drawing a boundary between two parts.

The strings in Boltzmann entropy of a two-box occupancy are complete microstates only within the binary toy model. They retain a coarse-grained variable for each particle and omit its exact position within a box and its momentum. An actual classical gas instead occupies continuous phase space; obtaining a dimensionless count from a phase space volume requires a specified cell convention and the appropriate treatment of particle identity. A quantum state count must respect the allowed quantum states and quantum statistics. Thus $2^N$ is not a general count of microstates for a gas obeying quantum ideal-gas statistics. ^[ref]

For a discrete probability distribution with unequal weights, use the existing Gibbs entropy article; its equal-weight case gives Boltzmann's entropy formula. Shannon entropy uses the corresponding information measure, with the logarithm base setting its units. A count alone cannot replace the specified probability distribution, as the Boltzmann distribution examples demonstrate. ^[ref]

Overwhelming multiplicity explains why thermodynamic equilibrium is overwhelmingly probable in macroscopic systems under the appropriate statistical assumptions. It does not make low-entropy thermal fluctuations logically impossible. Nor does counting alone derive irreversible macroscopic evolution: one also needs dynamics, an initial preparation, and a reason the system samples the allowed microstates on the timescale of interest. ^[ref]

Historically, Boltzmann's 1877 paper counted microscopic arrangements, called complexions, and related the natural logarithm of their number to entropy. The Sharp and Matschinsky translation appeared in Entropy 17 (2015), pages 1971–2009, DOI 10.3390/e17041971. Its notation differs from ours: Boltzmann's permutability measure $\Omega$ is already logarithmic, whereas here $\Omega$ is the count. ^[ref] Josiah Willard Gibbs's Elementary Principles in Statistical Mechanics (1902) developed the statistical ensemble framework; Chapter XIV compares several statistical definitions of entropy with thermodynamics. ^{[ref] [ref]}