Solution (source code)

= Solution

Use normalized <Haar measure> $m$ on the <real torus>. The integer <matrix> $A$ and its integer inverse induce a <toral automorphism>; its determinant one preserves $m$. For $k\in\mathbb Z^n$, let $e_k(x)=e^{2\pi i k\cdot x}$. These <characters of a real torus> are an orthonormal basis, and
$$
e_k\circ T_A^j=e_{(A^T)^jk},\qquad
\int(e_k\circ T_A^j)\overline{e_\ell}\,dm
=\mathbf1_{\{(A^T)^jk=\ell\}}.
$$
Suppose no <eigenvalue> of $A$ is a <root of unity>. For $k\ne0$, the frequency $(A^T)^jk$ can equal a fixed $\ell$ at most once. Otherwise two such times would give $(A^T)^qk=k$ for some $q>0$, forcing $A^q-I$ to be singular and an <eigenvalue> of $A$ to have $q$th power one. Hence the displayed correlation eventually vanishes whenever a nonconstant character is involved. Constant characters give the product of their means.

Linearity gives mixing for finite <Fourier series>. To pass to arbitrary $f,g\in L^2(m)$, approximate them by trigonometric polynomials. Composition with $T_A^j$ is an <isometry> of $L^2(m)$, so the <Cauchy-Schwarz inequality> bounds the approximation errors uniformly in $j$. Let those errors tend to zero after taking the large-$j$ limit. Thus $\int(f\circ T_A^j)\overline g\,dm\to\int f\,dm\int\overline g\,dm$, which is the required <mixing measure-preserving transformation> property.

Conversely, if an <eigenvalue> is a <root of unity>, choose $q>0$ with $\det[(A^T)^q-I]=0$. This is an integer matrix, so its nullspace has a nonzero rational vector; clearing denominators gives $k\in\mathbb Z^n\setminus\{0\}$ with $(A^T)^qk=k$. The nonconstant character $e_k$ has mean zero, but
$$
\int(e_k\circ T_A^{jq})\overline{e_k}\,dm=1
$$
for every $j$. It cannot satisfy mixing. Therefore
$$
\boxed{T_A\text{ is mixing precisely when }\operatorname{spec}(A)\text{ contains no root of unity}.}
$$
<Hyperbolic toral automorphisms> are mixing, but hyperbolicity is not necessary: this proof excludes periodic integer frequencies, not every eigenvalue of modulus one.