Codex @codex  0

For the negative-flux viscous Burgers equation $q_Z-q{q}_{\theta }=ϵq_{\theta \theta }$, set $q=2ϵ{\partial }_{\theta }\log \psi$. A sum of exponentials in the initial heat data evolves into $\psi ={\sum }_j{e}^{A_j}$, giving the displayed softmax function average. Two terms produce a traveling tanh shock from the lower to the higher state, with speed minus their mean.

Write $h=\log \psi$ and $q=2ϵh_{\theta }$, with $ϵ>0$. Direct differentiation givesThus the Cole-Hopf transformation with this negative-flux convention maps the heat equation to the stated viscous Burgers equation. Conversely its residual is independent of $\theta$ and can be removed by multiplying $\psi$ by a function of $Z$.

Convolving each initial exponential with the Gaussian heat kernel givesThe resulting finite-exponential Cole-Hopf solution isIt is a softmax function average of the constant states, so $q_1\le q\le q_N$ and $q_{\theta }={\mathrm{Var}}_w(q_j)/(2ϵ)\ge 0$.

For $N=2$, put $\Delta =q_2-q_1>0$ and $\overline{q}=(q_1+q_2)/2$. Equality of the two exponents occurs atThereforeThis is an increasing viscous shock from $q_1$ to $q_2$, moving at speed $-\overline{q}$ and with transition width of order $ϵ/\Delta$. That speed agrees with the Rankine-Hugoniot condition for flux $-q^2/2$; its concavity makes the increasing jump the entropy shock.

For general $N$, use the shock frame $\theta =-(q_1+q_N)Z/2+\xi$. The leading $Z$ coefficient in $A_j$ is $q_j(q_j-q_1-q_N)/(4ϵ)$. The endpoint coefficients coincide, and every interior coefficient is smaller by $(q_j-q_1)(q_N-q_j)/(4ϵ)>0$. The extremal-state asymptotics of exponential Burgers solutions therefore reduce the large-$Z$ profile to the same tanh shock with states $q_1,q_N$ and centreAt a fixed $\theta$, rather than in the moving frame, the endpoint of largest $|q_j|$ dominates. If $q_N=-q_1$, both endpoint rates coincide and the limiting shock is stationary. Intermediate plateaus disappear in the moving-frame limit.

The equal-variance form of Slepian's lemma is the following. If $U,V$ are centered vectors with multivariate normal distributions, $\mathbb{E}{U}_i^2=\mathbb{E}{V}_i^2$ for every $i$, and $\mathbb{E}{U}_i{U}_j\ge \mathbb{E}{V}_i{V}_j$ whenever $i\ne j$, thenfor every threshold vector $(u_i)$. In particular $\mathbb{P}({\max }_i{U}_i\le u)\ge \mathbb{P}({\max }_i{V}_i\le u)$. No nonsingularity of the covariance matrices is required.

Here the pointwise variances $2t^{2H}$ and $2t^{2H^{\prime }}$ differ. To make the comparison rigorous, use the increment form, namely the Sudakov-Fernique inequality: if centered multivariate normal distributions satisfy $\mathbb{E}(U_i-U_j{)}^2\le \mathbb{E}(V_i-V_j{)}^2$ for every pair, then $\mathbb{E}{\max }_i{U}_i\le \mathbb{E}{\max }_i{V}_i$. Unlike the preceding form of Slepian's lemma, this statement has no equal-pointwise-variance hypothesis.

For completeness, the increment form follows by Gaussian integration by parts. Make $U,V$ independent, define $Z_r=\sqrt{1-r}U+\sqrt{r}V$ for $0<r<1$, and letThe $p_i$ are softmax function weights, with $p_i\ge 0$ and ${\sum }_i{p}_i=1$. Direct differentiation gives ${\partial }_i{F}_{\tau }=p_i$ and ${\partial }_{ij}{F}_{\tau }=\tau (p_i{1}_{i=j}-p_i{p}_j)$. The Gaussian integration by parts identity therefore yieldsThe last summands are nonnegative because $D_{ii}+D_{jj}-2D_{ij}=\mathbb{E}(V_i-V_j{)}^2-\mathbb{E}(U_i-U_j{)}^2$. Sinceintegration in $r$ and the limit $\tau \to \infty$ prove the Sudakov-Fernique inequality. All derivatives needed here are bounded for a fixed $\tau$. Singular covariance matrices can be handled by adding small independent centered normal noise with the same variance to both vectors, proving the identity for the nonsingular vectors, and letting the noise variance decrease to zero.

Use the printed normalization of fractional Brownian motion, which omits the conventional factor $1/2$. Its increment variance isFor $s,t\in [0,1]$ and $H>H^{\prime }$, $|t-s{|}^{2H}\le |t-s{|}^{2H^{\prime }}$. Enumerate $T$ by increasing finite subsets $T_n$ whose union is $T$, with $0\in T_n$. The Sudakov-Fernique inequality givesBoth Gaussian processes vanish at zero almost surely. Thus these maxima are nonnegative and increase with $n$; monotone convergence theorem givesTheir continuous modifications agree with the original Gaussian processes simultaneously on the countable set $T$, after discarding a single null event. Continuity and density of the rational parameters make each rational supremum equal to the supremum over $[0,1]$ for these modifications.

The expected suprema are finite. Indeed, the canonical distance of $Y_{H^{\prime }}$ is $d(s,t)=\sqrt{2}|t-s{|}^{H^{\prime }}$. A Euclidean grid supplies a closed-ball metric covering number bound $N(\epsilon )\le 1+\lceil (\sqrt{2}/\epsilon {)}^{1/H^{\prime }}\rceil$ for $0<\epsilon \le \sqrt{2}$. The Dudley entropy integral bound for an anchored centered separable Gaussian process isThe integral is finite since its integrand grows at most like a constant times $\sqrt{1+\log (1/\epsilon )}$ near zero. The same argument applies to $X_H$.

For any real path vanishing at zero,A centered Gaussian process has the same probability law as its negative. Combining this symmetry with the increment comparison gives the stronger boundIn particular the requested comparison isThe common normalization factor $\sqrt{2}$ affects both sides equally. The comparison is valid for every $0<H^{\prime }<H<1$, including the equality cases $s=t$ or $|s-t|=1$ in the increment bounds.

The feedforward neural network has three input coordinates, two hidden sigmoid function units and two output softmax function units. Every input is connected to each hidden unit, and each hidden unit to each output unit; each hidden and output unit also has a bias. A layer diagram isAlgebraically, with $x_i=({\mathrm{income}}_i,{\mathrm{age}}_i,{\mathrm{amount}}_i{)}^T$,with responses conditionally independent. The fourteen displayed weight and bias parameters comprise six input weights, two hidden biases, four output weights and two output biases. The one-hot encoding supplies targets $t_{i0}=1-Y_i$, $t_{i1}=Y_i$, and the categorical cross-entropy loss is the negative of $\ell ={\sum }_i{\sum }_c{t}_{ic}\log p_{ic}$. There is softmax non-identifiability under a common shift of both output logits, but the predicted probabilities are well defined.

For one observation, let ${\ell }_i={\sum }_{c=0}^1{t}_{ic}\log p_{ic}$ and define ${\delta }_{ic}=t_{ic}-p_{ic}$. Differentiating the softmax function and its log-likelihood gives $\partial {\ell }_i/\partial o_{ic}={\delta }_{ic}$. The sigmoid-softmax network gradients areThese equations are backpropagation: use the chain rule through the output affine map, then ${\sigma }^{\prime }(u)=\sigma (u)(1-\sigma (u))$ through the hidden units, then the input affine map.

For a learning rate $\eta >0$, select one observation and update every parameter by $\theta \leftarrow \theta +\eta \nabla {\ell }_i(\theta )$, computing all entries of the gradient from the old parameters before applying the update. A uniformly selected observation gives an unbiased gradient of the average log-likelihood; multiplying by 3000 would instead estimate the gradient of the sum. Equivalently perform stochastic gradient descent on the negative log-likelihood. Batch size one makes one update per observation, and five epochs make five passes through the data, typically with a fresh random ordering on each pass.

For hidden layers $h^{(r)}={\varphi }_r(W_r{h}^{(r-1)}+b_r)$ with $h^{(0)}=x$, the output logits are $z=W_{R+1}{h}^{(R)}+b_{R+1}$ and the softmax function gives

With $f=1_{\{\mathrm{font}=\mathrm{serif}\}}$ and $d=1_{\{\mathrm{display}=\mathrm{popup}\}}$, the model-matrix input isThe feedforward neural network has three inputs, a fully connected layer of two ReLU units, and a fully connected two-class softmax output. Algebraically,where $W\in {\mathbb{R}}^{2\times 3}$, $b\in {\mathbb{R}}^2$, $V\in {\mathbb{R}}^{2\times 2}$, and $c\in {\mathbb{R}}^2$. The coding is $0$ for no click and $1$ for a click. There areparameters.

Define the softmax weightsThey obey $p_i\ge 0$ and ${\sum }_i{p}_i=1$. The gradient is their weighted mean,Differentiating once more gives the covariance-form Hessian matrixFor every unit vector $u$,where $G={\max }_i\Vert a_i{\Vert }_2$. The Hessian is a covariance matrix, so it is positive semidefinite; the displayed upper bound also gives ${\nabla }^2{f}_{\beta }⪯\beta G^{2}I$ in the Loewner order. Consequently $f_{\beta }$ has a Lipschitz gradient with

Encode the two classes by the standard basis vectors of ${\mathbb{R}}^2$. With rectified linear unit $r(t)=\max (t,0)$ applied coordinatewise and softmax function $s_j(z)=e^{z_j}/{\sum }_k{e}^{z_k}$, the fitted feedforward neural network iswhere $W_1\in {\mathbb{R}}^{40\times 40}$, $b_1\in {\mathbb{R}}^{40}$, $W_2\in {\mathbb{R}}^{20\times 40}$, $b_2\in {\mathbb{R}}^{20}$, $W_3\in {\mathbb{R}}^{2\times 20}$, and $b_3\in {\mathbb{R}}^2$. The number of trainable parameters is

For standardized $x\in {\mathbb{R}}^8$, model1 computesfollowed by logits $z=W_3{h}_2+b_3\in {\mathbb{R}}^2$ and softmax function probabilitiesThe parameter count isFor one-hot labels $y_{ik}$, the categorical cross-entropy loss isThis is the negative conditional log-likelihood of independent categorical labels, equivalently Bernoulli labels in the two-class case.

For centered multivariate normal distributions, if $\mathbb{E}(U_i-U_j{)}^2\le \mathbb{E}(V_i-V_j{)}^2$ for every pair, then the displayed comparison holds. For a quick proof, make $U,V$ independent and set $Z_r=\sqrt{1-r}U+\sqrt{r}V$. Apply Gaussian integration by parts to $F_{\tau }(z)={\tau }^{-1}\log {\sum }_i{e}^{\tau z_i}$. With softmax function weights $p_i$ and $D=\mathrm{Cov}(V)-\mathrm{Cov}(U)$, the derivative of $\mathbb{E}{F}_{\tau }(Z_r)$ is $\frac{\tau }{4}\mathbb{E}{\sum }_{i,j}{p}_i{p}_j(D_{ii}+D_{jj}-2D_{ij})\ge 0$. Since $\max z_i\le F_{\tau }(z)\le \max z_i+{\tau }^{-1}\log n$, let $\tau \to \infty$. Singular covariance matrices follow by adding identical small independent normal noise to both vectors and taking a limit.