Write and . The double-coset Hecke algebra consists of -bi-invariant complex functions on supported on finitely many double cosets, with convolution
Each double coset has finitely many orbits under left multiplication by , by rational conjugation of finite-index modular subgroups. Thus the sum is finite and independent of representatives. The characteristic functions of the double cosets form its basis, and its right action on invariant modular forms is , using the determinant-normalized slash operator.
For a positive integer , let be all integral two-by-two matrices of positive determinant . It is -bi-invariant. Define to be its indicator function, equivalently the sum of the distinct double cosets it contains, each with coefficient one. This is the convention consistent with the requested formula. For composite , it need not be the single double coset of : for instance cannot lie in that double coset, since multiplying by unimodular integral matrices preserves the greatest common divisor of the entries.
Prove all the needed subgroup facts directly. For a finite-index subgroup , take to be the least positive first coordinate appearing in and the least positive second coordinate on its intersection with the second axis. Euclidean division then shows that the first-coordinate projection is and . Positivity follows, for example, because the finite quotient group kills a nonzero multiple of each coordinate vector. Choose and reduce modulo to . Every vector of has first coordinate a multiple of , and subtracting that multiple of leaves a multiple of . Therefore these two vectors form a basis of . The parameters are unique. Reducing the first coordinate modulo and then the second modulo gives precisely quotient representatives, so .
Apply this row Hermite normal form in rank two to the row lattice of an integral matrix with . Its row lattice contains , since , so it has finite index. Its two rows and the displayed two rows are bases of the same row lattice. The two inverse change-of-basis matrices have integer entries, so their determinants are integers whose product is one. Thus the change-of-basis matrix has determinant ; since both orientations are positive, its determinant is one. Thus each orbit under left multiplication by has exactly one of the determinant-n matrix representatives for Hecke operators
There are such representatives, proving finiteness as well as the formula. The subgroup argument is a proof of the relevant Hermite normal form, not an invocation of an unproved lattice classification.
Consequently the normalized Hecke operator is
It preserves : right multiplication by permutes the left-multiplication orbits in , giving invariance, and cusp holomorphy under rational slash operators gives the holomorphy of each term at every cusp.
For a triangular representative the determinant-normalized slash operator is . Summing the Fourier expansion of a modular form over kills every index not divisible by , by finite exponential orthogonality. Thus
The Fourier coefficients of a composite-index Hecke operator are therefore
In particular . If , comparison of the coefficients gives
Finally suppose and is a simultaneous eigenfunction. Constant coefficients force , so for all . Weight two cannot occur, by vanishing of weight-two level-one modular forms. For even , use the normalized Eisenstein series with its Fourier expansion of a normalized Eisenstein series
where is the Bernoulli number. Then is a cusp form with coefficients . The Fourier coefficient bound for a cusp form bounds these by , but at arbitrarily large primes their magnitude is . Since , the coefficient factor must vanish. All coefficients of then vanish, including its constant coefficient, so by its cusp expansion and the identity theorem. This proves the noncuspidal level-one Hecke eigenform characterization: