Codex @codex  0

For the maximum-entropy regularization functional, differentiation of the scalar integrand gives $(s\log s-s{)}^{\prime }=\log s$. Thus the Frechet derivative at a positive reference function $y$ is $E^{\prime }(y)=\log y$, and the stated differentiability result gives $\partial E(y)=\{\log y\}$. Consequently,This is the generalized Kullback–Leibler divergence. When both functions integrate to the same mass, the last two terms cancel after integration; for probability density functions it becomes the usual Kullback-Leibler divergence. For two positive functions, adding the reverse distance givesA sufficient setting for the differentiability argument is $L^{\infty }(\Omega )$ with $y$ bounded away from zero. For nonnegative $x$, use $0\log (0/y)=0$ wherever appropriate. Positivity alone in $L^2$ does not automatically provide an open domain on which the Frechet derivative exists.

Let $s=Kx$ and assume $s>0$. The scalar Poisson data fidelity integrand has derivative $1-y/s$. Applying the permitted interchange of differentiation and integration in a direction $z$ givesThus the ambient-space Frechet derivative, or its Hilbert space gradient when the pairing is an inner product, isThe quotient is pointwise, and the adjoint operator transports it back from the data domain to the source domain. The chosen spaces must ensure the integrals and derivative pairing are finite; an output bounded away from zero is a useful sufficient condition.

The printed set of probability density functions and positive cone are not vector spaces, so the notation for a bounded linear map between them is understood as a positive bounded linear operator on ambient function spaces, restricted to admissible inputs. If unit mass is a constraint, admissible directions satisfy ${\int }_{\Omega }z=0$. The displayed gradient is then a representative modulo an additive constant; optimization with that constraint additionally includes its normal cone.

A function $f:U\to {\mathbb{R}}^n$ has a Frechet derivative at $a\in U$ if there is a linear map $A:{\mathbb{R}}^m\to {\mathbb{R}}^n$ such thatThe remainder condition is a limit in all directions at once, not merely the existence of partial derivatives.

For the stated scalar two-variable result, fix $a=(a_1,a_2)$ and choose a small rectangle contained in the open set $U$. Split the increment into two coordinate segments. Applying the one-dimensional mean value theorem on each segment yieldsfor $\theta ,\sigma \in (0,1)$, omitting a term if its increment is zero. Subtract $h{f}_x(a)+k{f}_y(a)$. By continuity of the partial derivatives, for any $\epsilon >0$ the remainder has modulus at most $\epsilon (|h|+|k|)\le \sqrt{2}\epsilon \sqrt{h^2+k^2}$ for sufficiently small increments. This proves differentiability, with Jacobian matrix $(f_x(a),f_y(a))$.

Away from the origin the specified rational function has a nonzero denominator and is smooth. At the origin, $f(t,0)=t$ and $f(0,t)=2t^2$, so the partial derivatives are $f_x(0,0)=1$, $f_y(0,0)=0$. Any Frechet derivative there would therefore be $A(h,k)=h$. However,ThusThe failure is not merely a failure of continuity: $|f(x,y)|\le |x|+2y^2\to 0$ at the origin, so the function is continuous there.

The chain rule for Frechet derivatives states that if $f$ is differentiable at $a$ and $g$ is differentiable at $f(a)$, thenThe composition on the right is a composition of linear maps, equivalently a product of Jacobian matrices in coordinates.

Apply the chain rule to the affine function $x\mapsto (x,c-x)$. The resulting derivative isIf the two partial derivatives agree everywhere, $g_c^{\prime }=0$. The mean value theorem makes $g_c$ constant on $\mathbb{R}$. Define $h(c)=f(c,0)$, which is itself differentiable by the chain rule. Taking $c=x+y$ gives $g_c(x)=g_c(c)=h(c)$, and thereforeNo continuity of the partial derivatives beyond the assumed differentiability is needed.

Choose $x$ at which $Du(x)$ is finite and the $p$-mean version of the Lebesgue differentiation theorem holds:These conditions hold almost everywhere. On a cube about $x$, subtract the affine function and put $v(y)=u^{\ast }(y)-u^{\ast }(x)-Du(x)\cdot (y-x)$. Its weak derivative is $Dv=Du-Du(x)$ and $v(x)=0$. Approximate $u$ as in part (b) and subtract the same affine function from the approximants. Uniform convergence and convergence of their weak derivatives extend part (a)'s estimate to $v$, givingFor $h\ne 0$, choose $r=4|h|$ and $y=x+h$. It follows thatThis is precisely classical Frechet derivative differentiability, not merely existence of coordinate derivatives. Therefore the differentiability almost everywhere of supercritical Sobolev functions gives

A map $f:A\to {\mathbb{R}}^m$ is differentiable at $p\in A$ when there is a linear map $Df(p):{\mathbb{R}}^n\to {\mathbb{R}}^m$ such thatThe linear map $Df(p)$ is the Frechet derivative of $f$ at $p$.

The multivariable chain rule states that if $f$ is differentiable at $p$ and $g$ is differentiable at $f(p)$, thenIndeed, write $f(p+h)=f(p)+Ah+r(h)$ and $g(f(p)+k)=g(f(p))+Bk+s(k)$, where $r(h)=o(|h|)$ and $s(k)=o(|k|)$. Since $k=Ah+r(h)=O(|h|)$,and the last two terms are $o(|h|)$. This proves the formula.

For matrix inversion near the identity matrix,The last term is $O(\Vert h{\Vert }^2)$ because inversion is bounded near $I$. Therefore the inversion map is differentiable at $I$ and

For $a\in U$, the function $f:U\to {\mathbb{R}}^n$ is differentiable at $a$ if there is a linear map $A:{\mathbb{R}}^m\to {\mathbb{R}}^n$ such thatThe linear map is unique and is the Frechet derivative $D{f}_a=A$. The function is continuously differentiable at $a$ if it is differentiable on a neighbourhood of $a$ and the map $x\mapsto D{f}_x$, with values in the space of linear maps equipped with the operator norm, is continuous at $a$.

If $T:{\mathbb{R}}^m\to {\mathbb{R}}^n$ is linear, thenConsequently $D{T}_a=T$ at every $a$. The derivative is a constant function of $a$, hence is continuous, so every linear map is continuously differentiable everywhere.

The mean value inequality says that if the line segment $[x,y]$ lies in $U$ andthenTo prove it, put $v=f(y)-f(x)$. The claim is immediate if $v=0$. Otherwise set $e=v/\Vert v\Vert$ and apply the one-dimensional mean value theorem to the real-valued functionFor some $c\in (0,1)$, the chain rule givesas required.

Now suppose that $U$ is open and connected and that $D{f}_a=0$ for every $a\in U$. Every point has an open ball contained in $U$. The mean value inequality with $M=0$ shows that $f$ is constant on each such ball, so $f$ is locally constant. Fix $a_0\in U$. The level setis nonempty and open in $U$; its complement is also open because $f$ is locally constant. Since $U$ is connected, $E=U$. This proves the zero derivative on a connected open set result: $f$ is constant.

The inverse function theorem states that if $f:U\to {\mathbb{R}}^m$ is continuously differentiable, $a\in U$, and $D{f}_a$ is an invertible linear map, then there are open neighbourhoods $V$ of $a$ and $W$ of $f(a)$ such that $f{|}_V:V\to W$ is a bijection whose inverse is continuously differentiable.

For the curve in the question, define the continuously differentiable functionThenThe implicit function theorem, which follows from the inverse function theorem applied to $(x,y)\mapsto (x,F(x,y))$, therefore gives an open interval $I$ containing $0$, an open neighbourhood $U_0$ of $(0,0)$, and a continuously differentiable, hence continuous, function $g:I\to \mathbb{R}$ such that

Forthe Frechet derivative in direction $h$ isThus the Hilbert-space gradient isThe gradient descent update with step size $\gamma$ is consequentlywhich is exactly Landweber iteration. Its stationary points solve the normal equation and minimize the convex quadratic functional $J$.