Codex @codex  0

On the full Cartesian product $\Omega ={\mathbb{R}}^{\mathbb{N}}$ with its product sigma-algebra, take the coordinate random variables $X_j(x)=x_j$. A product-measurable set $A$ is a tail event if and only if membership is unchanged whenever two sequences differ in finitely many coordinates. The measurability condition is essential: invariance alone is not a license to assign a probability to an arbitrary subset.

For the forward implication, fix $m$. An event in $\sigma (X_{m+1},X_{m+2},\dots )$ cannot distinguish points with the same remaining coordinates. Since a tail event belongs to this sigma-algebra for every $m$, every finite change preserves membership.

For the converse, freeze the first $m$ coordinates to zero. The insertion mapis a measurable function for the product sigma-algebras. Thus $B_m=i_m^{-1}(A)$ is a measurable set. Invariance gives $A={\pi }_m^{-1}(B_m)$, where the coordinate projection ${\pi }_m$ discards the first $m$ entries. Hence $A\in \sigma (X_{m+1},X_{m+2},\dots )$ for every $m$. The same argument works on $\{0,1{\}}^{\mathbb{N}}$. On a general probability space, coordinates need not admit arbitrary replacements, so the defining intersection of sigma-algebras remains the reliable formulation.

Assume the full-data sample space and the range of the statistic $T$ are standard Borel spaces, $T$ is a measurable function, and the statistical model $\{P_{\theta }\}$ is dominated by one common sigma-finite measure $\nu$. Write $f_{\theta }=d{P}_{\theta }/d\nu$ for its densities. Then $T$ is a sufficient statistic if and only if there exist nonnegative measurable functions $g_{\theta }$ and $h$, with $h$ independent of $\theta$, such thatThe statistical parameter can enter $g_{\theta }$ only through $\theta$ and the observed $T(x)$. ^[ref]

For discrete data, take counting measure; for ordinary continuous data, take Lebesgue measure. Independence, identical sampling laws, smooth dependence on $\theta$, and a fixed positive support of a probability distribution are not requirements of this theorem. Parameter-dependent support must nevertheless be included in $f_{\theta }$, usually through an indicator function, and must fit the displayed factorization. The sufficient statistic for a symmetric uniform sample illustrates exactly this: its indicator function depends on the unknown scale through the observed absolute maximum. The null sets in the theorem concern versions of densities; the factors are not generally themselves normalized probability distributions.

For a discrete sample space, the cancellation behind the theorem is particularly transparent. On a group with positive probability, factorization givesConversely, multiply a common conditional probability mass function by $P_{\theta }(T=T(x))$ to obtain the required factorization. For continuous data, a sum over a group is not a density integral over the original Lebesgue measure: the group may have measure zero. Use the regular conditional distribution formulation and the Fisher-Neyman factorization theorem instead of that discrete calculation.

Let $1<p<\infty$ and let $f$ be a real or complex measurable function in Lp space $L^p((0,\infty ),dx)$, with Lebesgue measure $dx$. The Hardy operator, denoted here by $H$, takes the running averageHölder's inequality ensures ${\int }_0^x|f(t)|dt\le x^{1-1/p}\Vert f{\Vert }_p<\infty$, so every such average exists. Hardy's integral inequality statesThe real signed and complex extensions of the nonnegative statement follow from the triangle inequality $|Hf|\le H|f|$. Changing $f$ on a null set changes neither average nor Lp norm.^[ref]

Averaging over an expanding interval spreads a localized input across the whole positive half-line. Hardy's integral inequality bounds the total Lp norm of that spread. The Hardy operator is consequently a bounded linear operator on this Lp space. The factor $p/(p-1)$ bounds its operator norm; its $p$th power is the constant in the displayed integral bound. These are different constants: at $p=2$, the L2 norm bound is two and the squared L2 norm bound is four.

Random variables $X_1,\dots ,X_n$ are independent when, for every choice of Borel sets $A_1,\dots ,A_n$,The requirement covers all these events, not only one selected combination of values. Equivalently, the generated sigma-algebras are independent sigma-algebras, or the joint probability distribution is the product measure of the marginal distributions. For an infinite family, the same condition must hold for every finite subfamily. For random elements, replace Borel sets by measurable sets in their specified target spaces.

For discrete random variables, checking every entry of the joint probability mass function is enough:Summing this identity over the desired sets proves the event formulation. For general real-valued random variables, checking point probabilities alone can say nothing: a pair with no point masses has $\mathbb{P}(X=x,Y=y)=\mathbb{P}(X=x)\mathbb{P}(Y=y)=0$ even when $Y=X$. The all-set definition avoids that trap. See Gautam Iyer, Conditional Probability and Independence, definitions 12, 16 and 18 and proposition 19, with the continuous case treated in his ^[ref].

For a concrete comparison, let $X,Y$ be independent fair bits: each has a Bernoulli distribution with parameter $1/2$. Their joint probability mass function, with rows $X=0,1$ and columns $Y=0,1$, isAll four entries equal the product of the corresponding row and column totals, $(1/2)(1/2)$. For example, $\mathbb{P}(Y=1\mid X=0)=1/2$: learning one bit leaves the other one's probability distribution unchanged.

Now keep a fair $X$ but set $Y=X$. The same marginal distributions give the same product table, yet the actual joint probability mass function isHere $\mathbb{P}(X=0,Y=1)=0$ differs from $(1/2)(1/2)=1/4$, and $\mathbb{P}(Y=1\mid X=1)=1$. Thus matching marginal distributions does not establish independence. Grinstead and Snell formulate the discrete product criterion in Introduction to Probability, section 4.1, definition 4.4, printed page 143.

Independence is stronger than zero covariance. For square-integrable independent random variables, $\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]$, so they are uncorrelated random variables. The converse fails: take $X$ with a discrete uniform distribution on $\{-1,0,1\}$ and $Y=X^2$. Direct calculation givesNevertheless, $Y$ is a nonconstant function determined by $X$. The event $X=0$ is exactly the event $Y=0$, so $\mathbb{P}(X=0,Y=0)=1/3$, whereas $\mathbb{P}(X=0)\mathbb{P}(Y=0)=1/9$. The positive and negative contributions to the covariance cancel without erasing the statistical dependence.

The usefulness of independence is that a large joint probability distribution can be assembled from small pieces. For $n$ fair independent bits, each of the $2^n$ patterns has probability $2^{-n}$. Products also simplify expected values, and vanishing cross covariances simplify the variance of a sum. Separate measurable functions of independent random variables remain independent: each output event is already an input event, so the defining product rule still applies. These are consequences of a model's independence assumption; whether the assumption describes actual observations is a question for statistical inference.

A real or complex measurable function is integrable for a measure $\mu$ when its absolute value has finite integral. For a random variable, this is integrability with respect to its probability measure, and makes its expectation finite.

The law of the unconscious statistician computes the expected value of a measurable function of a random variable directly from the original probability distribution. If $g(X)$ is an integrable random variable, thenUse ${\sum }_{x}g(x)p_X(x)$ for a probability mass function, or ${\int }_{\mathbb{R}}g(x)f_X(x)dx$ for a probability density function. There is no need to derive the probability distribution of $g(X)$ first. The identity is the integration rule for a pushforward measure, and also holds with an infinite value when $g$ is nonnegative. ^[ref].

For $X$ taking $-1,2,8$ with probabilities $1/2,3/10,1/5$, choosing $g(x)=x^2$ gives $\mathbb{E}[X^2]=\frac{1}{2}+4\frac{3}{10}+64\frac{1}{5}=14.5$. Its variance is therefore $14.5-1.7^2=11.61$. Squaring the expected value would instead give $2.89$, because squaring does not commute with averaging.

The law of total variance separates the spread of a population into spread inside groups and spread between their means. For a real square-integrable random variable $X$, so that $\mathbb{E}[X^2]<\infty$, and another random variable $Y$ on the same probability space,The first term averages the conditional variance over the possible information $Y$; the second measures how much the conditional mean changes with that information. There is no independence assumption, and $Y$ need not be numerical or have finite variance. Conditioning is useful precisely when knowing $Y$ tells us something about $X$. See MIT tutorial on using conditional expectation and variance.

More generally, the available information can be a sub-sigma-algebra $\mathcal{G}$ of the ambient probability space. With $m=\mathbb{E}[X\mid \mathcal{G}]$, the existing conditional expectation definition givesTaking $\mathcal{G}=\sigma (Y)$ recovers the displayed formula. The conditional mean and conditional variance are themselves random variables: before learning the group, we do not yet know which group average or group spread will apply. Their outer expectation and variance turn these varying quantities into numbers.

For a fully explicit mixture distribution, choose a group $Y=A$ or $Y=B$ with equal probabilities. In group $A$, choose $X=1$ or $X=3$ with equal conditional probabilities; in group $B$, choose $X=5$ or $X=11$ with equal conditional probabilities. Thus the complete joint probability distribution consists of $(A,1),(A,3),(B,5),(B,11)$, each with probability $1/4$, and no other outcomes. The conditional means are $2$ and $8$. Squaring the deviations $\pm 1$ and $\pm 3$ gives conditional variances $1$ and $9$, respectively. The law of total expectation gives the overall mean $\mu =(2+8)/2=5$.

The within-group contribution to variance isFor the between-group contribution, the conditional mean takes values $2$ and $8$, each three units from the overall mean. Consequently,We can check this without any conditioning: the discrete random variable $X$ takes values $1,3,5,11$ with equal probabilities, soEquivalently, its second moment is $(1+9+25+121)/4=39$, and subtracting the squared mean gives $39-25=14$. These are population variances, with the stated probabilities as weights, rather than estimates using the corrected denominator of a sample variance.

The figure makes the two sources of spread visible. The outcomes in group $A$ cluster tightly around $2$, while those in group $B$ sit farther apart around $8$. Replacing every outcome by its conditional mean removes the internal spread but leaves the separation between groups. Conversely, subtracting each group's conditional mean aligns the two groups around zero. Averaging only the conditional variances would miss nine units of variance; retaining only the conditional means would miss five. Neither summary alone describes the whole population.

Two modifications clarify what each contribution measures. Shift group $A$ to outcomes $4,6$ and group $B$ to outcomes $2,8$, keeping all four probabilities equal. Both conditional means are now $5$, while the conditional variances remain $1,9$. The between-group contribution vanishes, and total variance is just $5$. Group membership still tells us how uncertain an outcome is, even though the best point prediction stays the same. Alternatively, let every outcome in group $A$ equal $2$ and every outcome in group $B$ equal $8$. Now both conditional variances vanish, and all nine units of total variance come from group separation. Knowing the group predicts the outcome perfectly. These limiting cases distinguish differences in average outcome from differences in internal spread; the original mixture distribution contains both.

The same calculation answers a prediction question. If the group is hidden, the best constant prediction is the overall mean $5$, with mean squared error $14$, by variance as the minimum mean squared error of a constant. If the group is revealed, predict $2$ for $A$ and $8$ for $B$. The errors become $-1,1,-3,3$, and their mean squared error is $5$. Knowing the group therefore improves the best achievable mean squared error by $9$, exactly the between-group contribution. This is an interpretation relative to the information available: the remaining uncertainty could shrink further if additional information were revealed.

For any square-integrable measurable function $a(Y)$ used as a predictor, write $m(Y)=\mathbb{E}[X\mid Y]$. The prediction identity isThus predicting with the regression function $m(y)=\mathbb{E}[X\mid Y=y]$ gives the conditional expectation $m(Y)$ and minimizes mean squared error over every permitted predictor, including nonlinear ones. This minimizes average squared error, not the error for each individual outcome. See Berkeley Statistics 201A notes, sections 36–37.

Why do the two pieces add cleanly? Put $R=X-m(Y)$. Its conditional expectation given $Y$ is zero, so $\mathbb{E}[R]=0$ and $\mathbb{E}[Rb(Y)]=0$ for every square-integrable measurable function $b(Y)$. In particular, $R$ and $m(Y)$ are uncorrelated random variables. Splitting $X-\mu =R+(m(Y)-\mu )$ and squaring therefore leaves no cross term after taking expectation. Geometrically, the conditional expectation is an orthogonal projection in the L2 space, and the law of total variance is its Pythagorean theorem for centered random variables.

In our example, the four products $R(m(Y)-5)$ are $3,-3,-9,9$, which average to zero. Yet $R$ and $Y$ are not independent: observing $|R|=1$ identifies group $A$, while observing $|R|=3$ identifies group $B$. This is a concrete instance of heteroscedasticity, with different conditional variances despite zero conditional means of the errors. A group can predict uncertainty even when it cannot predict the error's sign.

The law of total variance is useful whenever aggregated data conceal distinct groups. Changing mixture weights changes both contributions, so the weighting must match the population being described. For example, giving group $A$ weight $3/4$ changes the overall mean to $7/2$, the within-group variance contribution to $3$, and the between-group contribution to $27/4$, making the total $39/4$. Group differences account for spread and prediction gains; assigning them a causal explanation requires a separate causal inference argument.

For a nonnegative measurable function $f:X\to [0,\infty ]$ on a measure space, defineThe supremum may be infinite. A simple function assigns finitely many heights to measurable sets, so its integral is height times measure, summed over those pieces. This definition groups points by their values rather than requiring a partition into short intervals. See MIT lecture 11, Definition 108.

A real-valued function is Lebesgue integrable when ${\int }_X|f|d\mu <\infty$. Its integral is ${\int }_X{f}_{+}d\mu -{\int }_X{f}_{-}d\mu$, where $f_{+}=\max (f,0)$ is its positive part and $f_{-}=\max (-f,0)$. This finiteness requirement avoids subtracting two infinities. For functions taking values in the complex numbers, integrate the real part and imaginary part separately.

Consider the Dirichlet function $f=1_{\mathbb{Q}\cap [0,1]}$ on $[0,1]$. It is a simple function, andIts Riemann integral does not exist. For every partition of an interval $0=x_0<\cdots <x_n=1$, each nondegenerate $[x_{i-1},x_i]$ contains both rational numbers and irrational numbers, by their respective density of the rational numbers and density of the irrational numbers. Thus its infimum there is $0$ and its supremum is $1$. The lower Darboux sum is ${\sum }_{i}0(x_i-x_{i-1})=0$, whereas the upper Darboux sum is ${\sum }_{i}1(x_i-x_{i-1})=1$. Refinement never closes this gap. A finite scatterplot cannot represent this behavior: the assertion concerns every subinterval and every partition of an interval.

Another view builds the Lebesgue integral from horizontal layers. For any nonnegative measurable function, the layer cake representation statesBoth sides may be infinite. The right side integrates the measure of a superlevel set against Lebesgue measure in the height variable $t$. See Gautam Iyer's measure theory notes, Theorem 9.5.

For $f(x)=x^2$ on $[0,1]$ with Lebesgue measure, the superlevel set at $0\le t<1$ is $(\sqrt{t},1]$, of Lebesgue measure $1-\sqrt{t}$; for $t\ge 1$ it is empty. ConsequentlyAt height $t=1/4$, the layer extends from $x=1/2$ to $x=1$, a width of $1/2$. The picture shows why the same area can be accumulated vertically or horizontally. In a probability space, the identical formula expresses the expected value of a nonnegative random variable through its exceedance probabilities.

A proof outline avoids extra assumptions on the measure space. For a nonnegative simple function $s={\sum }_j{a}_j{1}_{A_j}$ with disjoint measurable sets $A_j$, the superlevel set has measure ${\sum }_{a_j>t}\mu (A_j)$. Integrating this finite sum gives ${\sum }_j{a}_j\mu (A_j)=\int sd\mu$. Next use approximation by nonnegative simple functions $s_n\uparrow f$. For every $t$, the superlevel sets $\{s_n>t\}$ increase to $\{f>t\}$. Continuity from below of a measure and the monotone convergence theorem on the height variable give the formula for $f$.

For integrable real-valued functions $f,g$ on a measure space $(X,\mu )$ and fixed real numbers $a,b$, their linear combination $af+bg$ is integrable, and the Lebesgue integral satisfiesThe same rule applies to any finite summation: ${\int }_X{\sum }_{j=1}^n{a}_j{f}_{j}d\mu ={\sum }_{j=1}^n{a}_j{\int }_X{f}_{j}d\mu$ for integrable functions $f_j$ and fixed scalars $a_j$. Finite signed values matter: integrability ensures all these integrals are finite, so no undefined $\infty -\infty$ occurs.

This is the distributive property of finite summation carried into integration: a fixed factor can be pulled out, and a finite sum can be integrated term by term. For simple functions, distribute the finite sums of value times measure; approximation extends additivity to nonnegative measurable functions, and splitting into a positive part and negative part gives the integrable signed case. See Lecture 12: Lebesgue Integrable Functions, the Lebesgue Integral and the Dominated Convergence Theorem (Proposition 122: linearity for integrable functions). For example, ${\int }_0^1(2x-3)dx=2{\int }_0^{1}xdx-3{\int }_0^{1}1dx=-2$. With a probability measure, linearity of integration gives linearity of expectation.

On a measure space of positive total measure, the L-infinity norm of an essentially bounded measurable function is $\Vert f{\Vert }_{\infty }={\mathrm{esssup}}_{\mu }|f|$. The absolute value matters: a constant function $f=-3$ has L-infinity norm three, even though its essential supremum is $-3$. The norm belongs to the equivalence class in Lp space; different representatives equal almost everywhere give the same answer. ^[ref].

For the function $g$ in essential infimum and essential supremum, $\Vert g{\Vert }_{\infty }=1$ under Lebesgue measure, whereas its supremum norm is four. Replacing the exceptional values by the ordinary values $x$ leaves every finite Lp norm unchanged as well: $\Vert g{\Vert }_p=({\int }_0^1{x}^{p}dx{)}^{1/p}=(p+1{)}^{-1/p}$. This tends to one as $p\to \infty$. That agreement explains why the endpoint with exponent $p=\infty$ uses an essential bound rather than a pointwise bound.

A finite probability space makes the distinction between zero and small measure explicit. Let $f(a)=1$, $f(b)=3$, and $f(c)=10$. Assign weights $\mu (a)=1/2-\epsilon$, $\mu (b)=1/2$, $\mu (c)=\epsilon$, where $0\le \epsilon <1/2$. For $1\le p<\infty$,When $\epsilon =0$, the singleton $\{c\}$ is a null set and $\Vert f{\Vert }_{\infty }=3$. For every $\epsilon >0$, however small, $\Vert f{\Vert }_{\infty }=10$. For example, $\epsilon =0.001$ gives $\Vert f{\Vert }_1=2.009$ and $\Vert f{\Vert }_2=\sqrt{5.099}$, yet the large-$p$ Lp norms approach ten. The ordinary pointwise supremum norm is ten in both cases.

Why can a rare value dominate? For a bounded function on a probability space, an almost everywhere bound $|f|\le M$ gives $\Vert f{\Vert }_p\le M$. If $0<t<\Vert f{\Vert }_{\infty }$, the set $A_t=\{|f|>t\}$ has positive probability, so $\Vert f{\Vert }_p\ge t\mu (A_t{)}^{1/p}$. As $p\to \infty$, the factor $\mu (A_t{)}^{1/p}$ tends to one. These two bounds squeeze the limit to $\Vert f{\Vert }_{\infty }$. The probability measure assumption keeps total measure finite; this explanation does not assert the same finite-$p$ integrability on arbitrary infinite-measure spaces.

An Lp norm measures the size of a measurable function by averaging a power of its absolute value with respect to a specified measure. For $1\le p<\infty$,when the integral is finite. The exponent $p$ controls how strongly large values influence the result. The Lp space identifies functions equal almost everywhere, so this becomes a norm on the corresponding equivalence classes. For $p=\infty$, use the essential supremum of $|f|$, as defined by the L-infinity norm. ^[ref].

For a two-coordinate vector $(3,4)$, treat the coordinates as a function on two inputs with counting measure. Then $\Vert (3,4){\Vert }_1=3+4=7$, $\Vert (3,4){\Vert }_2=\sqrt{9+16}=5$, and $\Vert (3,4){\Vert }_{\infty }=4$. The middle case is the Euclidean norm; the first adds the coordinate magnitudes, and the last takes their largest value. These are different answers to different questions about size, all satisfying the triangle inequality.

If instead each input has probability $1/2$, the same values give $\Vert f{\Vert }_1=3.5$ and $\Vert f{\Vert }_2=\sqrt{12.5}$, while $\Vert f{\Vert }_{\infty }=4$ remains unchanged. Thus the measure is not an optional label. On a probability space, the finite Lp norms are nondecreasing in $p$ whenever they are finite, by Hölder's inequality; the unnormalized two-coordinate example above decreases instead. Statements comparing different exponents must specify their measure convention.

Why are these sizes useful? If $e$ is an error function, its L1 norm measures aggregate absolute error, its L2 norm measures the square root of aggregate quadratic loss, and its L-infinity norm bounds error almost everywhere. Concentrating a large error on a set of small positive measure can give a modest finite Lp norm but a large L-infinity norm. The three-point example and diagram in L-infinity norm show this explicitly, including why a null set behaves differently from a rare event.

A positive linear operator on bounded measurable functions that preserves the constant function one. A Markov semigroup is a time-indexed family of such operators with the semigroup law.

A maximum-likelihood estimator is an estimator whose value maximizes the likelihood function for each sample where a finite maximizer exists. Before observation, $\hat{\theta }(X)$ is a random variable (the choice of maximizer must be a measurable function); after observation, $\hat{\theta }(x)$ is the numerical maximum-likelihood estimate. A formula must specify the allowed parameter set, including whether its endpoints belong to it.

For ten independent Bernoulli trials, write $K={\sum }_{i=1}^{10}{X}_i$. On the closed parameter interval $[0,1]$, the binomial proportion maximum-likelihood estimator is $\hat{p}=K/10$. Eight successes give the estimate $0.8$. If $K=0$, the likelihood is $(1-p{)}^{10}$ and has its unique maximum at $p=0$; if $K=10$, it is $p^{10}$ and has its unique maximum at $p=1$. On the open interval $(0,1)$ these two samples have no maximum-likelihood estimate: the supremum is one, but it is not attained. This is the same issue as likelihood supremum at an excluded boundary.

A complete continuous example uses independent and identically distributed random variables $X_i\sim N(\mu ,1)$ with unknown mean $\mu \in \mathbb{R}$ and known standard deviation one. Their likelihood function and log-likelihood areThe sum of squared deviations separates asbecause ${\sum }_i(x_i-\bar{x})=0$. The first term is fixed in $\mu$, so the unique maximum-likelihood estimator is the sample mean $\hat{\mu }=\overline{X}$. This also proves the equivalence with ordinary least squares for this model. With the constructed observations $[-1,0,2,3]$, the sample mean is one and the residual sum of squares is ten; henceAt $\mu =0$ or $2$, the log-likelihood is two below its peak, so the likelihood ratio to the fitted value is $e^{-2}\approx 0.135335$. The absolute likelihood at the peak is $(2\pi {)}^{-2}{e}^{-5}\approx 0.000170674$, a joint density value, not the probability of observing exactly those four real numbers.

The following figure compares that sample with a constructed sixteen-observation sample consisting of four repetitions of the displayed four values, modeled as sixteen independent draws. Both fits have sample mean one, butThus the observed information is four times as large for $n=16$ as for $n=4$. The distance from the peak at relative height $e^{-2}$ decreases from one to $1/2$: the width scales as $1/\sqrt{n}$. This is a comparison of two stipulated samples; merely copying recorded observations would not create new independent evidence.

Even uniqueness needs checking. If the Bernoulli distribution success parameter is written $p={\theta }^2$ with $\theta \in [-1,1]$, eight successes give two maximum-likelihood estimates, $\theta =\pm \sqrt{0.8}$, because both give the same probability distribution. The problem is a failure of identifiability. The special cases below show other ways an MLE can have bias or fail the usual asymptotic normality.

For measurable functions $f,g$ with values in the real numbers or complex numbers on the same measure space $(X,\mathcal{A},\mu )$, and $1\le p\le \infty$, membership in the same Lp space givesThus adding two functions cannot make their combined Lp norm exceed the sum of their separate Lp norms. The measure need not be finite or a probability measure. The hypotheses require finite Lp norms; $p=\infty$ uses the essential supremum of the absolute value, not an ordinary pointwise supremum. For complex numbers, the absolute value is the complex modulus. As usual, Lp spaces identify functions equal almost everywhere. ^[ref]

The Minkowski inequality makes the Lp norm a workable notion of size: it supplies the triangle inequality needed to turn $d_p(f,g)=\Vert f-g{\Vert }_p$ into a metric on Lp space. Applied to $(f-h)+(h-g)$, it says that travelling through an intermediate function $h$ cannot shorten the direct distance measured by this metric. Applied to two error functions, it gives a bound on the total error even when the errors reinforce one another; Minkowski inequality for error aggregation gives explicit examples.

The geometry is already visible in the Euclidean plane. Under counting measure on two coordinates, the closed unit balls for the L1 norm, Euclidean norm, and L-infinity norm are respectively $|x|+|y|\le 1$, $x^2+y^2\le 1$, and $\max (|x|,|y|)\le 1$. Every one is a convex set: if $u,v$ have Lp norm at most one, the Minkowski inequality and absolute homogeneity of a norm give $\Vert tu+(1-t)v{\Vert }_p\le t+(1-t)=1$ for $0\le t\le 1$. The Euclidean norm has no flat boundary segments, whereas the other two norms do. This is the distinction captured by a strictly convex normed space and developed in equality in Minkowski inequality.

A short Hölder's inequality proof outline handles $1<p<\infty$. First $|f+g{|}^p\le 2^{p-1}(|f{|}^p+|g{|}^p)$, so $f+g$ has finite Lp norm before any division. With conjugate exponents $p$ and $q=p/(p-1)$, set $h=|f+g{|}^{p-1}$; then $\Vert h{\Vert }_q=\Vert f+g{\Vert }_p^{p-1}$. The scalar triangle inequality followed by Hölder's inequality givesDivide if $\Vert f+g{\Vert }_p>0$; otherwise the desired bound is immediate. For $p=1$, integrate $|f+g|\le |f|+|g|$. For $p=\infty$, the almost everywhere bounds $|f|\le \Vert f{\Vert }_{\infty }$ and $|g|\le \Vert g{\Vert }_{\infty }$ give the result by the essential supremum. The same scalar triangle inequality works for real numbers and complex numbers.

For a nondecreasing real function $z$ on an interval $I$ and nonnegative measurable function $h$,The pushforward of $z^{\prime }(u)du$ is dominated by Lebesgue measure: on each interval, the integral of the derivative is at most the increase of $z$. This handles jumps and singular parts without a smoothness assumption.

Set $Y_n={\sup }_{m\ge n}|f_m|$. It is an extended-real measurable function and decreases to $Y={\mathrm{limsup}}_m|f_m|$. If $f_n\to 0$ almost surely, then $Y_n\to 0$ almost surely. For any $ϵ>0$, the decreasing sets $\{Y_n>ϵ\}$ have intersection contained, up to a null set, in the exceptional set for convergence. Continuity from above of the probability measure gives $P(Y_n>ϵ)\to 0$.

Conversely, if $Y_n\to 0$ in probability, then $P(Y>ϵ)\le P(Y_n>ϵ)\to 0$ for every positive $ϵ$. Taking rational $ϵ>0$ gives $Y=0$ almost surely, so $|f_n|\to 0$ almost surely. This proves both directions, even when some individual $Y_n$ take the value infinity.

A statistic is sufficient when the conditional distribution of the observation given it does not depend on the parameter. It is minimal sufficient when it is, up to null sets, a measurable function of every sufficient statistic. For a dominated family, the likelihood-ratio criterion identifies two observations precisely when their likelihood ratio is independent of the parameter.

The Rao–Blackwell theorem says that conditioning a square-integrable unbiased estimator on a sufficient statistic preserves unbiasedness and cannot increase variance. The Cramér–Rao bound for an unbiased estimator of scalar $\theta$ is ${\mathrm{Var}}_{\theta }\hat{\theta }\ge 1/I(\theta )$. It requires parameter-independent support, differentiability, interchange of differentiation and integration, and finite positive Fisher information. More generally the numerator is the square of the derivative of the estimand. The bound follows from $E_{\theta }[\hat{\theta }{\partial }_{\theta }\log f]=1$ and Cauchy–Schwarz.

For the uniform sample, the joint likelihood is ${\theta }^{-n}{1}_{\{0<x_i<\theta \forall i\}}$. Its dependence on the observations is through $T={\max }_i{X}_i$, proving sufficiency. Two such likelihoods have parameter-independent ratio exactly when their maxima coincide, proving minimality. The density of $T$ is $n{t}^{n-1}/{\theta }^n$ on $(0,\theta )$. Unbiasedness of $h(T)$ saysDifferentiating this absolutely continuous identity gives $h(t)=(n+1)t/n$ almost everywhere. Conversely that function is unbiased. Every finite-variance unbiased estimator, after Rao–Blackwellization, must therefore give this same function of $T$. Its variance cannot exceed that of any such estimator. ThusThe Cramér–Rao argument does not apply because the support depends on $\theta$; in particular differentiation of the support boundary invalidates the regular score identity.