The relevant fixed coordinate subspace is . Let be the first columns of and let
The Gaussian empirical Gram matrix is the empirical second moment with the known mean zero; no subtraction of an estimated mean is involved. For , the ratio under consideration is . Since is a real symmetric matrix, the finite-dimensional spectral theorem gives
Indeed, diagonalizing bounds every unit-vector quadratic form by the largest absolute eigenvalue, and a corresponding unit eigenvector attains the bound.
We first construct a unit sphere net from ball covering. Enlarge the numerical covering constant, if necessary, to . Cover the unit ball in with at most balls of radius at most . For each such ball meeting the unit sphere, choose a point of the sphere in it and discard the others. These selected points form a metric net of the unit sphere with radius : any two points in one covering ball have distance at most . This argument ensures that the net points have unit length even if the original covering centers did not.
Take and put , so . For unit vectors with ,
Taking a net point for every unit and then a supremum proves the quadratic form net bound
It follows that
Fix . The rows of are independent vectors of independent random variables with the standard normal distribution, and . Their scalar products with are therefore independent variables , so
Use the supplied chi-squared concentration inequality with . Its threshold is
Consequently . The union bound gives the stronger fixed-subspace estimate
This is Gaussian Gram matrix concentration on a fixed subspace.
Since , we have . Under ,
Choose the numerical constant . Then the parenthesis is at least one, so is a valid choice, and
The covering factor is only because the coordinate subspace is fixed. The ambient dimension enters through the assumed sample-size bound.
The Hoeffding inequality states that if are independent random variables with almost surely, then for
The same bound holds for the lower tail. Consequently the two-sided tail is at most twice this bound. When all ranges have length zero, the sum is deterministic and every positive tail probability is zero.
We construct an exponential packing of a Hamming cube. Choose an inclusion-maximal separated set in with separation at least for the Hamming distance. It exists by a finite greedy procedure: keep adding any vertex at distance at least from all selected vertices until none remains. By maximality, every vertex is within distance strictly less than of some member of . This is the principle that maximal separated sets give covers.
For a uniformly random vertex and any fixed vertex , the mismatch indicators in different coordinates are independent Bernoulli random variables with success probability . Hence has the binomial distribution with parameters . The lower-tail Hoeffding inequality gives
Thus the Hoeffding lower-tail bound for Hamming balls shows that every strict-radius ball has at most vertices. Using a strict ball handles noninteger without changing the required separation.
The strict balls centered at cover the entire Hamming cube. Counting their union by the sum of their sizes yields
Therefore
In particular this proves the assertion for every . Maximality here means that no further vertex can be added; finding a largest separated set is unnecessary.
In the Gaussian white noise model, observe the entire path
where is standard Brownian motion and the deterministic drift belongs to L2 space on . Equivalently, . For every deterministic , the observed stochastic integral satisfies
The noise is an isonormal Gaussian process. In particular its variance is . Gaussian white noise is interpreted through these integrals, rather than as an ordinary random function with a pointwise value at every time. No smoothness of is needed.
Here is the Gaussian maximum bound without independence. Put . It is integrable since it is bounded by . For every ,
The second inequality uses only the moment-generating function of the standard normal distribution, so independence is unnecessary. Jensen inequality gives , hence
Minimizing at proves
For the dyadic partition, let , , putting the endpoint into the final interval. The Haar scaling functions
form an orthonormal basis of the space of functions constant on each interval. The Haar wavelets can be written as
For , the constant function together with , , is another orthonormal basis of . Indeed, each successive level splits a cell's two constants into their sum and difference; the number of basis elements is . Thus the Haar projection is the orthogonal projection
On cell its value is . The first formula also applies to .
Estimate each coefficient by its observed stochastic integral:
This is the Haar projection estimator in Gaussian white noise. The noise integrals over disjoint intervals form a Gaussian vector with zero off-diagonal covariance. The principle that uncorrelated jointly Gaussian variables are independent then makes these coefficients independent. Therefore
Taking expectations shows that is an unbiased estimator of , both coefficientwise and pointwise for the stated step-function representatives.
Only one scaling function is nonzero in each cell, so the supremum norm of the error has the exact form
The endpoint convention ensures the same identity at . Apply the Gaussian maximum bound without independence with to obtain the supremum norm risk of a Haar projection estimator:
The factor comes from cellwise scaling, while the extra square root of comes from taking the maximum over Gaussian errors.
Put and . Using natural logarithms, the Kullback-Leibler divergence is
The integrand is zero where , including where both densities vanish, and is where . Equivalently, when is absolutely continuous with respect to , and it is otherwise. This is a well-defined extended integral: on , writing gives , so its negative part is integrable.
For Pinsker's inequality, an infinite Kullback-Leibler divergence makes the conclusion immediate. Assume it is finite. Set
Since , the positive and negative parts of have equal integrals. Thus
This is the total variation distance with the convention .
We first prove the binary partition bound for relative entropy. For any measurable with , apply Jensen inequality to the convex function under the conditional probability measure . Absolute continuity of measures gives
If , both sides are zero; if , finiteness forces as well. Apply this bound to and its complement to obtain
For , regarding as a function of gives
Consequently is a convex function and has its minimum zero at . By continuity this also holds at . For , the finite case has and the remaining cases have infinite divergence. This proves the Bernoulli relative entropy lower bound in every case. Combining the bounds yields
For the Hellinger distance, retain the unnormalized convention in which
The integral is the Hellinger affinity. In the finite-divergence case, for -almost every point. Apply with to get
The affinity is finite by Cauchy-Schwarz inequality. Mass of on creates no difficulty, since the affinity vanishes there and both probability measures have mass one. The infinite-divergence case is again immediate. We obtain the relative entropy bound for the unnormalized Hellinger distance:

Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 5. . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact