Circulant matrix ($C_{nj}=c_{(n-j)\mathrm{mod}N}$)

A circulant matrix applies the same rule at every position of a finite ring: moving both the input and output indices one step leaves its entries unchanged. Its first column determines the whole matrix. For a length-$N$ vector $c$, index rows and columns by $0,\dots ,N-1$ and defineThe indices use modular arithmetic. Thus each column is the previous column shifted down one place, with the last entry wrapping around to the top. Multiplication is exactly circular convolution:This makes a circulant matrix a natural model for a periodic finite difference rule or a filter acting on a periodic record. Strang's MIT lecture develops this connection between circulant matrices, circular convolution and the discrete Fourier transform.^[ref]

For example, the first column $c=(2,-1,0,-1{)}^T$ givesHere the indices wrap around modulo four. Each output compares a value with its two neighbors; this is the Graph Laplacian of a four-vertex cycle graph. A constant input gives zero, while an alternating input gives a large response. The Fourier diagonalization of a circulant matrix makes this distinction exact: its four eigenvalues are $0,2,4,2$.

The plotted discrete Fourier modes are complex vectors, shown through their real and imaginary parts. Each entry has modulus one; the Euclidean norm of each length-four vector is two. In particular, modes $k=1$ and $k=3$ have the same real part but opposite imaginary parts, so they are distinct eigenvectors with the same eigenvalue.

The useful surprise is that every length-$N$ circulant matrix shares the same basis of eigenvectors, whatever its first column. One change of basis therefore simplifies an entire family of linear maps, instead of requiring a separate eigenvalue computation for each one. Bamieh's tutorial develops the discrete Fourier transform from precisely this simultaneous diagonalization problem.^[ref]

Use the negative-exponential forward discrete Fourier transform and the $1/N$-normalized inverse discrete Fourier transform:For the circulant matrix with first column $c$, the discrete Fourier mode $w_n^{(k)}=e^{2\pi ikn/N}$ is an eigenvector with eigenvalueIndeed, substitution into circular convolution givesThe columns of $F^{-1}$ are $w^{(k)}/N$, so the diagonalization of a matrix takes the formThe roots of unity give orthogonality of these eigenvectors; dividing each $w^{(k)}$ by $\sqrt{N}$ makes the basis orthonormal. Consequently, every complex circulant matrix is a normal matrix, including ones that are not Hermitian matrices.

The sign convention matters. For $c=(1,2,0,0{)}^T$, the circulant matrix isChanging the direction of the shift or the sign in the discrete Fourier transform changes which complex eigenvalue goes with which discrete Fourier mode. A real symmetric example alone would conceal that distinction.

For a linear system $Cx=b$, put $\hat{x}=Fx$ and $\hat{b}=Fb$. The equation becomes $N$ independent scalar equationsIf every ${\lambda }_k$ is nonzero, divide to find the unique vector $x$ and apply the inverse discrete Fourier transform. If some ${\lambda }_k=0$, a vector $x$ satisfying the linear system exists exactly when ${\hat{b}}_k=0$ at every such index; the corresponding ${\hat{x}}_k$ remain free. This is a concrete version of the Fredholm alternative in a finite-dimensional setting.

With an appropriate Fast Fourier transform, multiplication or a compatible solve takes $O(N\log N)$ arithmetic operations, instead of forming a dense matrix and applying general Gaussian elimination. The gain comes from the periodic structure; an arbitrary matrix does not have this shared basis of discrete Fourier modes.^[ref]

Consider the discrete linear system $Lx=b$ withThis is a finite periodic analogue of the Poisson equation, using the negative second finite difference with unit grid spacing. It is also a Graph Laplacian equation on a cycle graph; no continuum approximation is being claimed here.

Summing the four equations gives zero on the left, since each value occurs twice positively and twice negatively. A necessary condition is therefore ${\sum }_n{b}_n=0$. The Fourier diagonalization of a circulant matrix shows that this is also sufficient: the only zero eigenvalue of $L$ is the constant mode $k=0$, and ${\hat{b}}_0={\sum }_n{b}_n$.

For the displayed forcing, the discrete Fourier transform givesDivide only at the three nonzero eigenvalues, and choose ${\hat{x}}_0=0$ to impose zero arithmetic mean:Direct substitution gives $Lx=(1,0,-1,0{)}^T$. All other vectors satisfying the linear system are $x+a(1,1,1,1{)}^T$, because the kernel is the space of constant vectors. Thus balanced forcing determines the differences between values, while a reference level is an extra choice. This small model explains why a periodic Poisson equation needs both a compatibility condition and a normalization.