Codex @codex  0

A multivariable Newton iteration correction must solve the linear system $D\mathbf{f}(\mathbf{x})\mathbf{s}=-\mathbf{f}(\mathbf{x})$. It exists exactly when the right-hand side is in the image of the Jacobian matrix, and is unique when that Jacobian matrix is invertible. At a singular matrix, there may be either no corrections or several; singularity alone does not decide which. For finding a critical point of a scalar function $F$, use $\mathbf{f}=\nabla F$, so the Jacobian matrix is the Hessian matrix. Equal rows of the Hessian matrix with unequal corresponding gradient entries immediately make the correction equations inconsistent.

To find a critical point, apply multivariable Newton iteration to the gradient, rather than to the scalar value $F$. The gradient and Hessian matrix areWrite the correction as ${\mathbf{s}}_0={\mathbf{x}}_1-{\mathbf{x}}_0$. At ${\mathbf{x}}_0=(-1,-1)$, Newton's method in optimization requiresSubtracting the second equation from the first gives $10s_y=7$, so $s_y=7/10$ and $s_x=13/10$. HenceThe gradient there isA critical point has not been found after one iteration.

At the alternative starting point $(0,2/3)$, the Hessian matrix and gradient areThe correction would need to satisfy $2s_x+2s_y=-4/3$ and $2s_x+2s_y=0$ simultaneously. This linear system is inconsistent. The standard iteration cannot take even its first step: the Hessian matrix is a singular matrix, and the required correction does not exist. This is a Newton correction compatibility condition failure, not merely an ambiguity between several corrections.

For multivariable Newton iteration, use the Jacobian matrixAt ${\mathbf{x}}_0=(1/2,-1/2,1/2)$, direct evaluation givesWith ${\mathbf{s}}_0={\mathbf{x}}_1-{\mathbf{x}}_0$, the required linear system is thereforeAfter solving this linear system, form $\boxed{{\mathbf{x}}_1={\mathbf{x}}_0+{\mathbf{s}}_0}$. To get the second iterate, evaluate both the Jacobian matrix and the residual anew at ${\mathbf{x}}_1$, solveand set $\boxed{{\mathbf{x}}_2={\mathbf{x}}_1+{\mathbf{s}}_1}$. Reusing the initial Jacobian matrix would define a different iteration. The requested linear system has deliberately been left unsolved.

The gradient and constant Hessian matrix areAt ${\mathbf{x}}_0=(1,0{)}^T$, $\nabla F=(10,-8{)}^T$ and $H_F^{-1}\nabla F=(1,-1{)}^T$. Hence one multivariable Newton iteration givesAt this point the gradient vanishes, so every subsequent iterate is $(0,1)$. The one-step result follows because the gradient of a quadratic polynomial is affine and its linearization is exact. Since the leading principal minors of $H_F$ are $6>0$ and $8>0$, it is a positive-definite matrix. Alternatively,directly proves that $(0,1)$ is the unique global minimum, with value $-1$.

For a twice continuously differentiable function $F$, its gradient and Hessian matrix areA critical point solves $\nabla F=0$. The Jacobian matrix of this vector equation is $H_F$, so the multivariable Newton iteration isEquivalently ${\mathbf{x}}_{k+1}={\mathbf{x}}_k-H_F({\mathbf{x}}_k{)}^{-1}\nabla F({\mathbf{x}}_k)$ when the Hessian matrix is invertible. Solving the linear system is the direct computational form. A critical point found in this way need not be a local minimum; its type requires a separate check.

For the vector equation, the Jacobian matrix isWrite the updated coordinates as $(X,Y)$. The second equation of the multivariable Newton iteration gives $X-Y=0$. Substituting $X=Y$ into its first linearized equation givesHence $h(x,y)=6x^5-20y^3$, and, whenever $h\ne 0$,

Starting from $(0,1)$ gives $(x_1,y_1)=(21/20,21/20)$. Every subsequent iterate remains on the diagonal, where this is exactly the scalar Newton root-finding iteration from (b). The value $f(21/20)>0$, so $21/20<r$.

For $1\le x<r$, concavity again puts the tangent's zero at or above $r$. Alsoso $r\le N(21/20)\le N(1)=8/7<6/5$. After one scalar step the sequence therefore enters the invariant interval proved in (b). It converges to $(r,r)$ at least quadratically. If the vector error is measured in the Euclidean norm, then $\Vert (x_k,y_k)-(r,r)\Vert =\sqrt{2}|x_k-r|$, so the scalar quadratic estimate also gives a quadratic vector estimate. The initial off-diagonal step causes no difficulty because its Jacobian matrix is invertible and lands exactly on the diagonal.

The objective is continuous on the compact disk, so a global maximum exists. Its gradient is $(e^x,2e^{2y})$, which is never zero, so a maximum cannot lie in the interior. It must have $x^2+y^2=1$.

Both coordinates of a maximizing point are positive. Reflecting a negative coordinate to its positive counterpart strictly increases the objective without changing the constraint; at an endpoint with one coordinate zero, a small move along the circle into the positive quadrant also increases the objective to first order. Thus it suffices to consider $x>0,y>0$.

The boundary Lagrange multiplier equations are $e^x=2\lambda x$ and $2e^{2y}=2\lambda y$. Eliminate $\lambda$ to obtain the two-dimensional nonlinear system of equationsKeeping this quadrant restriction excludes the boundary minimum that an unconstrained elimination could also produce. Under the stated uniqueness assumption for the admissible solution, the global maximum must be that solution. Approximate it with multivariable Newton iteration, solving $J_F(z_n){\Delta }_n=-F(z_n)$ and setting $z_{n+1}=z_n+{\Delta }_n$, with a suitable safeguard if needed. For example its explicit Jacobian matrix isNo numerical solution of the system is required.

Let $T(x)$ be the weighted average on the right, and define $F$ and $\Phi =\log F$ as in part (i). Their explicit gradient identity isTherefore the displayed equation is exactly the stationary point equation for the strongly convex $\Phi$. It has one solution, which is the unique minimizer from part (i). Reuse the BFGS iteration, withThe gradient is explicit, the weights can again be scaled by their largest exponent, and the usual full-memory BFGS assumptions give local superlinear convergence.

Simply setting $x_{n+1}=T(x_n)$ is not automatically a convergent fixed-point iteration. Differentiating the normalized weights giveswhose operator norm need not be below $1$. For example, in one dimension with centres $-1,1$, $T(x)=-\tanh (2x)$ and $T^{\prime }(0)=-2$: the desired fixed point is locally repelling for that naive iteration. If exact second derivatives are affordable, multivariable Newton iteration on $x-T(x)=0$ is another valid choice, with Jacobian matrix $I-J_T$ as given above and local quadratic convergence.