Codex @codex  0

Suppose ${\hat{\beta }}_1$ and ${\hat{\beta }}_2$ minimize $Q$. Their midpoint is also a minimizer because $Q$ is a convex function. The group penalty is convex, while the squared Euclidean norm is strictly convex in the fitted value. If $X{\hat{\beta }}_1\ne X{\hat{\beta }}_2$, strict convexity would make the midpoint objective strictly smaller than the minimum. Hence $X{\hat{\beta }}_1=X{\hat{\beta }}_2$, so the fitted values are unique.

Let $0\le \lambda \le 1$ and put $P=\lambda P_1+(1-\lambda )P_2$ and $Q=\lambda Q_1+(1-\lambda )Q_2$. For each alphabet symbol $x$, apply the log-sum inequality to $a_1=\lambda P_1(x)$, $a_2=(1-\lambda )P_2(x)$ and the corresponding $b$ values. Summing over $x$ giveswhich is joint convexity in $(P,Q)$.

An element $u^{†}$ is a $J$-minimizing solution whenIt satisfies the source condition in variational regularization when there is a $w^{†}\in Y^{\ast }$ such thatwhere $\partial J(u^{†})$ is the subdifferential of the convex functional $J$.

For an integrable real random variable $X$, its cumulant-generating function is the extended-real convex functionwhere $\psi (\theta )=+\infty$ if the exponential moment diverges. Its Legendre transform of a cumulant-generating function is

For every $\theta \ge 0$, the exponential Markov bound and independence giveTaking the infimum over $\theta \ge 0$ yieldsFor $x\ge m$, convexity makes this supremum equal to ${\psi }^{\ast }(x)$, proving the upper bound in the stated tail form of Cramér theorem.

The pushforward measure of $\mu$ under $g$ is $g_{\ast }\mu$, defined for $B\in \mathcal{B}$ byThe Lebesgue decomposition theorem says that if $\nu$ and $\mu$ are sigma-finite measures on the same measurable space, then uniquely

Let $f:[0,\infty )\to \mathbb{R}\cup +\infty$ be convex with $f(1)=0$. If $\lambda$ dominates the probability distributions $P,Q$, with densities $p,q$, their f-divergence isusing the lower-semicontinuous perspective convention where $q=0$. This definition is independent of the dominating measure.

The data processing inequality for f-divergences statesTo prove it, take $\lambda =P+Q$ and let $\mathcal{G}=g^{-1}(\mathcal{B})$. If $p=dP/d\lambda$ and $q=dQ/d\lambda$, then the pullbacks of the densities of $g_{\ast }P$ and $g_{\ast }Q$ with respect to $g_{\ast }\lambda$ are respectively ${\mathbb{E}}_{\lambda }[p\mid \mathcal{G}]$ and ${\mathbb{E}}_{\lambda }[q\mid \mathcal{G}]$. The perspective $F(a,b)=bf(a/b)$ of a convex function is jointly convex. Conditional Jensen inequality therefore givesIntegration proves the claim.

The Squared Hellinger distance isIf $P,Q$ have densities $p,q$ with respect to a sigma-finite measure $\mu$, this becomes

Fix any probability distribution $Q$ and set $p_j=P_j(A_j)$, $q_j=Q(A_j)$, and $t_j=H^2(P_j,Q)$. Applying the data processing inequality to the indicator function of $A_j$ gives the Bernoulli Hellinger boundThe hinted inequality impliesThe function $t\mapsto \sqrt{t(1-t/4)}$ is concave on $[0,2]$. Since the $A_j$ form a set partition, ${\sum }_j{q}_j=1$, and Jensen inequality givesTaking the infimum over $Q$ proves the stated inequality.

We prove the symmetric derivative criterion for monotonicity using an antiderivative. Fix $a<u<v<b$ and defineThe fundamental theorem of calculus gives $F^{\prime }=f$ because $f$ is continuous. For each interior $x$,Since $f^s(x)>0$, the definition of symmetric derivative gives $\delta >0$ such that $f(x+t)-f(x-t)>t{f}^s(x)$ for $0<t<\delta$. Therefore, for $0<h<\delta$,

We next show that $F$ is a convex function. On any closed interval $[r,s]\subset (a,b)$ let $\ell$ be the affine function joining $(r,F(r))$ to $(s,F(s))$. If $F-\ell$ were positive anywhere, the extreme value theorem would give a positive maximum at an interior point $x$. For every sufficiently small $h>0$, that maximum would implyThe affine function contributes zero to this symmetric difference, contradicting the strict positive inequality above. Thus $F\le \ell$ on every such interval, exactly convexity.

For a differentiable convex function, secant slopes bound endpoint derivatives. Explicitly, the chord inequality at $u+h$ and at $v-h$, followed by $h\downarrow 0$, givesSo $f$ is nondecreasing. If $f(u)=f(v)$ for some $u<v$, monotonicity would make $f$ constant on $[u,v]$, forcing $f^s(x)=0$ for $u<x<v$, a contradiction. Consequently $\boxed{f(u)<f(v)\text{ for every }u<v}$: $f$ is strictly increasing. This proof requires no continuity of $f^s$ and never applies the ordinary mean value theorem to a merely symmetrically differentiable function.

Because all variables are positive, the product constraint is equivalent to ${\sum }_i\ln x_i=\ln a$. The Lagrange multiplier equations forare $1-\lambda /x_i=0$. Thus every $x_i$ equals $\lambda$, and the constraint requires $\lambda =a^{1/n}$. The candidate minimum is thereforeFor a global justification, put $y_i=\ln x_i$. The objective becomes the strictly convex function ${\sum }_i{e}^{y_i}$ on the affine constraint ${\sum }_i{y}_i=\ln a$. A stationary point of this restricted convex function is its unique global minimum, establishing the claim without presupposing the desired inequality.

Finally take $a={\prod }_i{x}_i$ and divide the minimum bound by $n$. This is the arithmetic-geometric mean inequality:with equality exactly when all the positive variables are equal.

For the ideal guaranteed-delay model, the total untreated yearly input at $250$ operating days isSince the discharge is a decreasing function of the residence time, the minimum is obtained at equality:This is $7.642$ half-lives and corresponds to $3.444$ L of carbon-dioxide-equivalent delay volume. The three-hour choice provides margin; operating every day instead would require at least $166.22$ minutes under the same assumptions.

This minimum is conditional on a guaranteed residence time. More generally the surviving fraction iswhere $E(t)$ is the residence time distribution. Because the exponential function is convex function, a spread of transit times gives more surviving radioactive activity than plug flow at the same mean time. A perfectly mixed continuously flowing vessel has a steady-state survival fraction $1/(1+\lambda V/Q)$, not $e^{-\lambda V/Q}$. Thus no unique minimum tank filling time follows without specifying the filling, retention and emptying schedule. The boxed minimum applies to the explicitly specified delay system, and must not be represented as approval of an unspecified tank.

Compact support of a measure gives finite second absolute moments for both probability measures. Since the source satisfies absolute continuity of measures with respect to Lebesgue measure, Brenier's theorem identifies the quadratic Kantorovich optimal transport problem with the Monge optimal transport problem:Here $S_{\#}\mu$ is the pushforward measure. The optimal transport plan is unique and equals $(\mathrm{Id},T{)}_{\#}\mu$. The optimal transport map is unique up to a $\mu$-null set, withThe gradient of this convex function is c-cyclically monotone for $c(x,y)=|x-y{|}^2$ on a full-measure set. In particular, summing the two supporting inequalities for the convex function gives $(T(x)-T(x^{\prime }))\cdot (x-x^{\prime })\ge 0$ wherever both gradients exist. No regularity of the target beyond the stated probability measure assumptions is needed.

Write $d\mu =f(x)dx$, with $f\ge 0$. Applying polar coordinates and Fubini's theorem gives, for bounded nonnegative measurable $h$,Also $\tilde{\mu }(\{0\})=\mu (\{0\})=0$ by absolute continuity of measures. Extending $g$ by zero on the negative half-line proves

Both radial pushforward measures have compact support of a measure, so Brenier's theorem on the line supplies a unique optimal transport plan and a transport map $\tau$, unique $\tilde{\mu }$-almost everywhere, withIn one dimension, the gradient of a convex function is a nondecreasing function. Nonnegativity holds $\tilde{\mu }$-almost everywhere because ${\tau }_{\#}\tilde{\mu }=\tilde{\nu }$ is carried by $[0,\infty )$. To choose a nonnegative, nondecreasing representative everywhere, let $F,G$ be the radial cumulative distribution functions and use the monotone rearrangementAt $F(r)=0$ set $\tau (r)=0$, and at $F(r)=1$ use the largest point of the support of a measure $\tilde{\nu }$. These choices preserve monotonicity and boundedness. Since $F$ is continuous, $F(R)$ has the uniform distribution on $(0,1)$ when $R$ has law $\tilde{\mu }$, and the quantile function $G^{-1}$ sends this uniform distribution to $\tilde{\nu }$. Thus this representative has the required pushforward measure, even if the target has atoms.

Choose the bounded, nonnegative nondecreasing function $\tau$ from part (d), and defineThe function $h$ is convex and nondecreasing. For $0\le a\le 1$, the triangle inequality, monotonicity, and convexity giveHence $U$ is a convex function. By the gradient of a radial function formula and the fundamental theorem of calculus,for $x\ne 0$ whenever $h^{\prime }(|x|)=\tau (|x|)$. A monotone function has at most countably many discontinuities, so part (d) makes the exceptional radii $\tilde{\mu }$-null. The origin is also $\mu$-null. Define $\mathcal{T}(0)=0$; thus $\mathcal{T}=\nabla U$ $\mu$-almost everywhere.

For every $Q$ in the orthogonal group, $\mathcal{T}(Qx)=Q\mathcal{T}(x)$, including at zero. Consequently ${\mathcal{T}}_{\#}\mu$ is invariant under the orthogonal group. Moreover $|\mathcal{T}(x)|=\tau (|x|)$ away from the null origin, and composition of pushforward measures givesPart (c) therefore gives ${\mathcal{T}}_{\#}\mu =\nu$. The Knott–Smith optimality criterion makes a transport map given by a convex function's gradient optimal for the quadratic transport cost. Uniqueness from Brenier's theorem yieldsSince directions agree, this also identifies the optimal value:

Use an independent copy $(x^{\prime },y^{\prime })$ of $(x,y)$ under the product measure $\Pi \otimes \Pi$. Write $q=y-x$, $q^{\prime }=y^{\prime }-x^{\prime }$, andSince the pushforward measure of $\Pi \otimes \Pi$ under $(z_t,z_t^{\prime })$ is ${\mu }_t\otimes {\mu }_t$, the interaction energy isTwo applications of the chain rule giveThe factor $1/2$ is retained. This classical identity requires differentiation under the integral sign to be justified, just as in part (b). Bounded Hessian matrix of $W$ is sufficient, since $|q-q^{\prime }{|}^2\le 2|q{|}^2+2|q^{\prime }{|}^2$ is integrable; compactly supported endpoints are another sufficient case. The printed quadratic-growth condition alone does not provide this domination.

The failure can occur for the interaction energy too. Take the even interaction potential $W(x)=1-\cos (2\pi x^3)$ and replace the endpoint probability measures in part (b) by $\frac{1}{2}{\delta }_0+\frac{1}{2}\mu$ and $\frac{1}{2}{\delta }_0+\frac{1}{2}\nu$. Keep the origin fixed and translate every other atom by one. This coupling is increasing on the line, hence is optimal: its assigned values extend to a nondecreasing transport map, which is the gradient of a convex function. Differences between translated atoms remain integers, so their mutual interaction vanishes. Only interactions with the origin contribute, giving $\mathcal{W}[{\mu }_t]=F(t)/4$, with $F$ from part (b). It therefore also lacks a finite classical second derivative at $t=1/2$.

Fix $x$, and write $p=x-\alpha G_{\alpha }(x)$ and $g=G_{\alpha }(x)$. The supplied subgradient relation gives $v=v_{\alpha }\in \lambda \partial \psi (p)$ with $g=\nabla f(x)+v$. The evaluation point in $\nabla f$ is $x$, whereas $v$ is a subgradient at the new point $p$.

Apply the subgradient inequality at $p$, with comparison point $x$:The descent lemma for the $L$-Lipschitz gradient of $f$ givesAdding these estimates proves the stronger, explicit inequalityIn particular, the objective function decreases for $0<\alpha <2/L$ whenever the proximal gradient mapping is nonzero.

To justify the requested big O notation, we must check that $g$, which depends on $\alpha$, stays bounded. Suppose $\partial (\lambda \psi )(x)$ is nonempty, and choose a fixed $w$ in this subdifferential. This holds for a finite-valued convex function on ${\mathbb{R}}^n$, including both norms in parts (a) and (b). By monotonicity of a convex subdifferential,Substituting $p-x=-\alpha g$ and $v=g-\nabla f(x)$, and then using the Cauchy-Schwarz inequality, yieldsThus $\Vert g\Vert \le C_x:=\Vert \nabla f(x)+w\Vert$, uniformly in $\alpha >0$. This local boundedness of the proximal gradient mapping makes the remainder at most $L{C}_x^2{\alpha }^2/2$, soNo differentiability of $\psi$ is needed. If an proper extended-real function is allowed as the regularization term, the subdifferential condition at $x$ is a qualification for this uniform remainder; the preceding explicit descent inequality remains the useful statement whenever its quantities are finite.

Condition on $X^k=x$, and set $r=x-x^{\ast }$, $g=\nabla f(x)$ and $h=\nabla f_{S_k}(x)$. Since the sampled index is independent of $X^k$ and has the discrete uniform distribution, $h$ is an unbiased stochastic gradient:Write ${\sigma }_k^2=\mathbb{E}[\Vert h-g{\Vert }^2\mid X^k=x]\le M$. Expanding $h=g+(h-g)$ and using the zero conditional mean of $h-g$ proves the stochastic gradient second-moment identityThe stochastic gradient descent update is $X^{k+1}=x-h/L$. Expanding the squared Euclidean distance and using the preceding identities givesThe subgradient inequality for the differentiable convex function $f$ gives $f(x)-f(x^{\ast })\le g^{T}r$. Combining this with (3),Removing the temporary notation for the conditioning point yieldsThe noise contributes twice in this argument: once in the supplied bound (3), and once in converting the expected squared step into the squared gradient. Each contribution is at most $M/(2L)$.

The printed distance needs a square. The native PDF, paper page 3, prints $\Vert x^0-x^{\ast }\Vert$ in the final bound, but the general conclusion from (5) contains $\Vert x^0-x^{\ast }{\Vert }^2$. We first prove the corrected statement and then show that the printed version can fail.

Take the expectation in (5), use the tower property, and sum over $j=0,\dots ,k-1$. The squared Euclidean distances form a telescoping sum:Discard the final nonnegative squared Euclidean distance. The smallest of $k$ numbers is at most their arithmetic mean, soThis is an $O(k^{-1})$ bound above the stochastic gradient noise floor $M/L$ for the best expected objective function value. By Jensen's inequality, it also bounds $\mathbb{E}f({\bar{X}}^k)-f(x^{\ast })$ for the averaged iterate ${\bar{X}}^k=k^{-1}{\sum }_{j=0}^{k-1}{X}^{j+1}$, because $f$ is a convex function.

To test the printed version, take one component function, $f_1(x)=f(x)=x^2/4$, with $L=1$, $x^{\ast }=0$ and $x^0=10$. This convex quadratic function has $\nabla f(x)=x/2$, hence it has a $1$-Lipschitz gradient. The stochastic gradient descent update is deterministic, its conditional variance is zero, and the assumed positive bound $M=1/10$ is valid. At $k=1$, the new iterate is $X^1=5$, soThus the printed inequality is false under its stated assumptions. The squared Euclidean distance in the corrected bound resolves the defect.

The subgradient inequality for the differentiable convex function $f$ givesThus $y$ is a global minimizer of $\varphi$ with value zero. Subtracting an affine function preserves convexity.

We must also justify using part (a) for $\varphi$, since its partial derivative can differ from that of $f$. First show that condition (6) for a singleton block implies the full coordinate Lipschitz continuity estimateIf ${\partial }_{i}f(z)\ne 0$, choose $\alpha =-t/{\partial }_{i}f(z)$ in (6). If it is zero but ${\partial }_{i}f(z+t{e}_i)\ne 0$, apply the same argument starting at $z+t{e}_i$ and reverse the displacement. If both partial derivatives are zero, the estimate is immediate. The availability of all real $\alpha$ in (6) justifies these choices.

Now ${\partial }_i\varphi (z)={\partial }_{i}f(z)-{\partial }_{i}f(y)$, so differences of this partial derivative along coordinate lines are unchanged. Hence $\varphi$ has the same coordinate Lipschitz continuity constant $L_i$ and satisfies the singleton version of (6). Apply part (a) to $\varphi$ with step size $1/L_i$:Combining with its already established nonnegativity provesThe coordinate estimate survives subtraction of the supporting affine function. This is the step that allows the arbitrary comparison point $y$ to be exploited in part (c).

The De la Vallée-Poussin criterion characterizes uniform integrability by existence of a nonnegative increasing convex function $\Psi$ with $\Psi (s)/s\to \infty$ and a uniform bound on $\int \Psi (|v_j|)$. On a finite interval this gives a weakly convergent subsequence in $L^1$ by the Dunford-Pettis theorem.

After discarding a finite prefix of the minimizing sequence if needed, put $H(t)=\Phi (t)-M\ge 0$. Its derivative sequence has $\int H(u_j^{\prime })\le C$. The superlinear bound directly gives uniform integrability: for $H(t)\ge A|t|$ outside $[-R_A,R_A]$,If the convex version of the criterion is used, choose increasing $R_m\to \infty$ so $H(t)\ge m^2|t|$ for $|t|\ge R_m$, and set $\Psi (s)={\sum }_{m\ge 1}(s-R_m{)}_{+}$. This is locally a finite sum, convex and increasing. It is superlinear because for fixed $N$ and $s\ge 2R_N$, $\Psi (s)\ge Ns/2$. If exactly $m$ terms are active, $\Psi (s)\le ms\le H(t)$ for $s=|t|$, so its integrals have the same bound. The criterion therefore supplies $u_{j_k}^{\prime }\rightharpoonup v$ in $L^1$.

Define $u(x)=\alpha +{\int }_a^{x}v$. Interval-indicator tests give $u_{j_k}(x)\to u(x)$ for every $x$, including $b$, so its endpoint values are correct and $u\in W^{1,1}$. The common derivative $L^1$ bound makes these functions uniformly bounded. Uniform integrability also gives common equicontinuity, since an increment is bounded by the derivative integral on the intervening interval. The Arzelà-Ascoli theorem yields uniform convergence to the same identified limit, and in particularHere the printed admissible space is $W^{1,1}$, so this compactness argument gives precisely the needed membership.

A convex function satisfies $F((1-t)x+ty)\le (1-t)F(x)+tF(y)$ for every $0\le t\le 1$ and every $x,y$.

We first establish the lower truncation identity as well. The pair $(-X,-Y)$ has the same hypotheses as $(X,Y)$, because conditional expectation is linear and negation preserves equality of probability distributions. Apply part (ii) to this pair and multiply by $-1$ to getFor every real $t$,Using this identity, part (ii), the lower truncation identity, and linearity of conditional expectation givesIn particular the argument does not assume that a function obtained by scalar clipping is a convex function or a concave function.

Write $C_a(t)=(t\wedge a)\vee (-a)$. The bounded random variables $C_a(X)$ and $C_a(Y)$ have the same probability distribution, and the displayed identity identifies the latter as the conditional expectation of the former. They are square-integrable, so part (i) gives $C_a(X)=C_a(Y)$ almost surely. Take $a=1,2,\dots$ and discard the countable union of exceptional null sets. At every remaining outcome, let $a\to \infty$ to obtainThus conditional expectation preserving a distribution forces equality even when only integrability is assumed.