Codex @codex  0

The Hessian matrix describes how the slope of a function changes in different directions. For a real-valued function $f$ with continuous second partial derivatives on an open subset of ${\mathbb{R}}^n$, it isThe symmetry of second derivatives makes $H_f$ a symmetric matrix under these hypotheses. It is also the Jacobian matrix of the gradient: near $x$, a small displacement $h$ changes $\nabla f$ by $H_f(x)h+o(\Vert h\Vert )$. Thus the gradient measures the current slope, while the Hessian measures its local variation. ^[ref]

For example, take the quadratic formIts gradient and Hessian matrix areThe only critical point is the origin. Completing the square gives $q=(x+y{)}^2+2y^2$, so the origin is a strict global minimum. The eigenvalues $4-2\sqrt{2}$ and $4+2\sqrt{2}$ are both positive, recovering the same conclusion from the positive definiteness of the Hessian. Its eigenvectors identify the rotated directions in which the quadratic form has its smallest and largest bending.

For a fixed vector $v$, restriction to a straight line gives the directional interpretationThis is the quadratic form associated with the Hessian matrix. For a unit eigenvector $v$, it equals the corresponding eigenvalue. Opposite signs therefore mean that some directions bend upwards and others downwards. The second-derivative test turns this information into a classification when the gradient vanishes.

The same object is useful for computation. Newton's method in optimization chooses a displacement $h$ by solving $H_f(x)h=-\nabla f(x)$ when the Hessian matrix is an invertible matrix; this makes the gradient of the local quadratic model vanish. For the displayed $q$, the equation gives $h=-(x,y{)}^T$, so one exact step reaches the minimum from any starting point. For a general function, a Hessian with opposite-sign eigenvalues or a large condition number need not give a useful minimizing step, and the local minimum need not be a global minimum.

The name honors Otto Hesse. ^[ref] An older geometric use is the Hessian criterion for flexes of a plane cubic, connecting a determinant of second partial derivatives with inflection points of an algebraic curve. Hesse's 1844 paper Über die Wendepuncte der Curven dritter Ordnung is an early source on that geometric problem. ^[ref]

A Householder transformation is a reflection that leaves every direction in a chosen hyperplane fixed and reverses its normal direction. For a real unit vector $u$, its matrix isTo understand the formula, decompose any vector into its orthogonal projection onto the normal and a perpendicular component:Then $Hx=x_{\perp }-x_{\parallel }$. Subtracting twice the normal component changes its sign while retaining everything parallel to the mirror. A nonzero normal $v$ gives the equivalent formula $H=I-2v{v}^T/(v^{T}v)$, so normalization can be deferred.

For example, take $u=(2,1{)}^T/\sqrt{5}$. Its mirror is the straight line $2x_1+x_2=0$, andThe midpoint of these two vectors is $(-1,2{)}^T$, which lies on the mirror; their connecting segment is parallel to its normal. Both vectors have Euclidean norm five.

Several useful properties follow from this decomposition. The Householder reflection is a symmetric matrix and an orthogonal matrix:Reflecting twice restores the original vector; Euclidean norms and inner products are preserved. The eigenvalue is $-1$ along $u$ and $+1$ on $u^{\perp }$, so $\det H=-1$. A reflection reverses orientation even though it preserves lengths.

The computational payoff is that the mirror can be chosen to send a whole vector onto one coordinate axis, eliminating several entries at once. Householder QR decomposition repeats this operation on the remaining parts of successive columns. For a real symmetric matrix, applying the same Householder reflection on both sides gives a matrix similarity $HAH$ that preserves symmetry and eigenvalues; this is the basis of Householder tridiagonalization.

Alston S. Householder's 1958 paper Unitary Triangularization of a Nonsymmetric Matrix develops this elimination method. Its motivation includes reducing the arithmetic needed for triangularization while retaining the conditioning advantages of orthogonal transformations. ^[ref] Over the complex numbers, $I-2u{u}^{\ast }$ for a unit $u$ is Hermitian and unitary, with the star denoting the conjugate transpose; the coordinate-axis construction below is stated for real vectors.

For two distinct intersecting straight lines through the origin, with independent real linear equations $L_1=0$ and $L_2=0$, the nondegenerate centred hyperbolas having exactly those asymptotes areIndeed, the homogeneous quadratic form of a centred hyperbola vanishes in its two asymptotic directions and hence is a nonzero scalar multiple of $L_1{L}_2$. Its equation has no linear terms because the origin is its centre, and its constant term is nonzero by nondegeneracy. Conversely, the invertible linear transformation $(x,y)\mapsto (L_1,L_2)$ takes the displayed curve to $uv=c$, a hyperbola, and its ends approach $u=0$ or $v=0$. Distinct values of $c$ give disjoint level sets. In particular, the rectangular hyperbolas with asymptotes $y=\pm x$ are $x^2-y^2=c$, $c\ne 0$.

Two planes $n_1\cdot r=d_1$, $n_2\cdot r=d_2$ with independent normal vectors meet in a straight line with direction $n_1\times n_2$. Any difference of two points on both planes is perpendicular to both normal vectors, which proves the direction statement. A particular point follows by solving the two linear equations; adding a real multiple of the direction gives the parametric equation of a straight line. For parallel normal vectors, the planes are either coincident or disjoint instead.

For the first-order ordinary differential equation $y^{\prime }=f(x,y)$ defined on an open subset $D$ of the plane, an isocline of slope $k$ is the level setHere $k$ is a constant. At every point of this isocline, the direction field prescribes a tangent vector proportional to $(1,k)$ for a solution of a differential equation. It does not generally prescribe the tangent line of the isocline itself. The level set can have several components or fail to be a smooth curve, and points outside $D$ are excluded.

For example, take $y^{\prime }=x-y$ on the whole plane. Its isoclines are the parallel straight linesEach line has slope $1$, while the direction field along it has slope $k$. Thus the isocline $y=x$ carries horizontal segments even though the line itself rises. A function graph of a solution of a differential equation follows the segments, rather than following this line. Only the isocline with $k=1$, namely $y=x-1$, is also a solution of a differential equation.^[ref]

The first-order linear differential equation has the exact familyThe initial condition $y(x_0)=y_0$ selects $A=(y_0-x_0+1)e^{x_0}$ and hence a unique solution of a differential equation for every real $x$. The picture shows $y(0)=-1,0,1,2$. For $A=0$ the solution of a differential equation stays on the $k=1$ isocline; for other $A$ it moves through different isoclines. These formulas check the pictured trajectories by direct differentiation, independently of how the direction field was sampled.

The zero-slope isocline $f(x,y)=0$ is called a nullcline. Below the nullcline $y=x$ in this example, $y^{\prime }>0$; above it, $y^{\prime }<0$. For $A>0$, the solution of a differential equation meets it at $x=\log A$, where $y^{\prime }=0$ and $y^{\prime \prime }=1$, a strict local minimum. The nullcline is therefore a useful guide to turning points, although it is not itself a solution of a differential equation. In the equivalent planar autonomous differential equation $\dot{x}=1$, $\dot{y}=x-y$, it is the nullcline for the second component; there is no equilibrium point, because the first component never vanishes.

This makes isoclines useful even when no explicit solution of a differential equation is available: organize a direction field by common slopes, read where function graphs rise or fall, and identify possible turning points on nullclines.

The kernel of a linear map $T:V\to W$ is the vector subspace of inputs sent to zero. Linearity makes it a vector subspace: if $T(v)=T(w)=0$, then $T(av+bw)=0$ for all scalars $a,b$. A vector in the kernel represents a change that the map cannot detect, since $T(x+v)=T(x)$ for every input $x$.

For $T:{\mathbb{R}}^3\to {\mathbb{R}}^2$ given by $T(x,y,z)=(x+y,z)$, the zero-output equations are $x+y=0$ and $z=0$. ThusIncreasing the first input while decreasing the second by the same amount leaves the output unchanged. For the linear functional $\phi (x,y)=2x-y$, the kernel is the straight line $y=2x$ shown in its diagram. A linear map is injective exactly when its kernel is the zero vector subspace, because $T(v)=T(w)$ is equivalent to $T(v-w)=0$.

The kernel describes invisible changes, rather than all inputs that produce a specified nonzero output. If $T(x_0)=b$, the corresponding fiber of a function is the affine solution space of a linear equation $x_0+\ker T$. The difference of two inputs in that fiber belongs to the kernel; an input in that fiber need not. In particular, for $b\ne 0$ the fiber cannot be a vector subspace, since it does not contain the zero vector.^[ref]

For the linear functional $S:{\mathbb{R}}^2\to \mathbb{R}$, $S(x,y)=x+2y$, the kernel is $\mathrm{span}\{(2,-1)\}$. Each fiber $S^{-1}(\{c\})$ is the parallel straight line $(c,0)+\mathrm{span}\{(2,-1)\}$. The input vector space is two-dimensional, but the output is one scalar: a whole straight line of inputs collapses to one point of the output real line.

A linear functional is a linear map from a vector space $V$ to its scalar field $F$:Its output is one scalar, rather than an arbitrary vector in another space. In ${\mathbb{R}}^2$, $\phi (x,y)=2x-y$ gives $\phi (3,1)=5$, $\phi (1,2)=0$, and $\phi (4,3)=5$: adding the two inputs adds their measured values.

The level sets $\phi (v)=c$ visualize what the measurement retains and discards. For this example they are parallel straight lines $y=2x-c$. Moving parallel to $(1,2)$ changes the input but leaves $\phi$ unchanged. The zero level set is its kernel of a linear map, while the other level sets are translates of that vector subspace.

Inputs need not be geometric arrows. On the vector space of polynomials of polynomial degree at most two, evaluating at zero is a linear functional, as is taking the definite integral from zero to one. For $p(t)=a+bt+c{t}^2$, they give $p(0)=a$ and ${\int }_0^{1}p(t)dt=a+b/2+c/3$. Thus polynomial evaluation and integration are two different measurements of the same polynomial. Coordinate extraction is another measurement, developed under dual basis.

A linear functional is defined by linearity, not by continuity. In finite-dimensional real or complex normed vector spaces it is automatically continuous; in infinite dimension it need not be. The continuous dual space adds this requirement when small changes in the input should give small changes in the output.

The mean value theorem says that an average rate of change is attained as an instantaneous rate somewhere between the endpoints. Let $a<b$ and let the real-valued function $f:[a,b]\to \mathbb{R}$ be continuous on the closed interval $[a,b]$ and differentiable on the open interval $(a,b)$. Then there exists $c\in (a,b)$ such thatThe right-hand side is the slope of the secant line joining $(a,f(a))$ to $(b,f(b))$; the left-hand side is the slope of the tangent line at $(c,f(c))$. The derivative need not be continuous, and the function need not be differentiable at the endpoints. ^[ref]

For the polynomial $f(x)=x^2$ on $[1,3]$, the secant line has slope $(9-1)/(3-1)=4$. Since the derivative is $f^{\prime }(x)=2x$, the required interior point is $c=2$. The tangent line there is $y=4+4(x-2)=4x-4$, whereas the secant line is $y=1+4(x-1)=4x-3$. Their equal slopes make them parallel, but they are distinct straight lines: the theorem equates rates, not heights.

The practical strength is that the unknown point $c$ can disappear from an estimate. Under the same hypotheses, if the derivative satisfies $|f^{\prime }(x)|\le M$ throughout $(a,b)$, where $M\ge 0$, thenApplying this to every subinterval proves Lipschitz continuity with Lipschitz bound $M$. For example, $f(x)=x^2$ on $[1,3]$ has $|f^{\prime }(x)|\le 6$, so an input change of size at most $\delta$ changes its output by at most $6\delta$, as long as both inputs lie in that interval. A local derivative bound therefore controls finite changes without computing an integral. The sign of the derivative similarly gives the derivative criterion for monotonicity.

Taking $M=0$ shows that a function with zero derivative throughout a real interval is a constant function, provided it is continuous at any included endpoints. The interval condition is essential: on the domain of a function $(-2,-1)\cup (1,2)$, the function equal to zero on the left interval and one on the right interval has zero derivative everywhere. It is a locally constant function, yet not a constant function on the whole domain of a function. The mean value theorem cannot cross the gap.

A concise proof subtracts the secant line: set $m=(f(b)-f(a))/(b-a)$ and $h(x)=f(x)-f(a)-m(x-a)$. Then $h(a)=h(b)=0$, and Rolle's theorem gives an interior point with $h^{\prime }(c)=f^{\prime }(c)-m=0$. The geometric idea is to remove the average trend and find a horizontal tangent line in what remains.

A normal vector at a point of an embedded smooth manifold is orthogonal to every tangent vector in its tangent space, using the surrounding dot product. Its direction describes departure from the local tangent geometry. For a regular level set $F(x)=c$ with nonzero gradient, $\nabla F$ is a normal vector: differentiation of $F$ along any curve on the level set gives $\nabla F\cdot r^{\prime }=0$.

For a plane curve, the normal direction is a straight line perpendicular to its unit tangent vector; the two unit normals differ by sign. For a curve in three-dimensional Euclidean space, the orthogonal directions form a two-dimensional plane. A principal normal vector selects one of them using curvature, so it must be distinguished from an arbitrary normal vector.

The nullity of a linear map $T:V\to W$ is $n(T)=\dim \ker T$, the dimension of its kernel. It counts independent input directions that can change without changing the output. For a finite-dimensional vector space $V$, the rank-nullity theorem givesFor an $m\times n$ matrix $A$ of matrix rank $r$, this is $n-r$, using the number of columns, not the number of rows. This dimension identity is ^[ref].

The nullity answers a practical question about a linear system: once one input is known to produce the requested output, how much freedom remains? If $T(x_0)=b$, its fiber of a function over $b$ isIndeed, $T(x)=T(x_0)$ exactly when $T(x-x_0)=0$. Thus every nonempty fiber is an affine subspace of dimension $n(T)$, with the same direction space as the kernel. The kernel itself is the fiber over zero. Outputs outside the image of a linear map have empty fibers; the nullity does not guarantee existence, and we assign no solution-space dimension to an empty fiber here.

In the diagram, $F:{\mathbb{R}}^2\to {\mathbb{R}}^2$ has matrix $\left(\begin{array}{cc}1& 1\\ 0& 0\end{array}\right)$, matrix rank one and nullity one. Its kernel has basis $(1,-1)$, and the full solution of $F(x,y)=(c,0)$ is $(x,y)=(c,0)+s(1,-1)$. One free scalar moves the solution along a straight line; the output $(c,1)$ is impossible.

The existing three-input example $T(x,y,z)=(x+y,z)$ instead has rank two and nullity one. Its image of a linear map is all of ${\mathbb{R}}^2$, andEvery output has a solution straight line. In contrast, $U(x,y,z)=(x+y,0)$ has rank one and nullity two, with kernel basis $(1,-1,0),(0,0,1)$. Its reachable output $(a,0)$ has the solution affine subspacea plane, while $(a,b)$ with $b\ne 0$ has no solution. These two linear maps have the same input and output dimensions but different freedom and reachability. Computing the kernel separates what an output determines from what an extra constraint would have to fix. For either example, a basis of the kernel gives one independent free scalar per basis vector, which is why nullity measures freedom rather than the number of solutions.

Suppose three measurements constrain two unknowns:The linear system asks for $x_1=1$, $x_2=2$, and $x_1+x_2=6$, which are incompatible. A least-squares solution makes a balanced compromise. The matrix has full column rank, soThe fitted output and residual areThus the two direct measurements are each overshot by one, and the measured sum is undershot by one. The residual has Euclidean norm $\sqrt{3}$ and is orthogonal to the column space, the plane $z=y_1+y_2$ in output coordinates $(y_1,y_2,z)$.

For any perturbation $h\in {\mathbb{R}}^2$, the orthogonality of $r$ to $Ah$ givesSince the kernel of $A$ is trivial, every nonzero $h$ increases the residual. The least-squares solution is unique in this example. The formula using $A^{T}A$ is convenient for these small exact calculations; an actual numerical computation can use a QR decomposition or singular value decomposition to avoid the extra conditioning cost of the least-squares normal equations. For full column rank, the squared singular values of $A$ are the eigenvalues of $A^{T}A$, so its condition number in the Euclidean norm is ${\kappa }_2(A^{T}A)={\kappa }_2(A{)}^2$. ^[ref]

A straight line through a point $a$ in nonzero direction $t$ is the set of vectors $r=a+st$ for real parameter $s$. If $|t|=1$, changes in the parameter equal signed distances along the line. Two such lines intersect exactly when their two parameter formulas give a common vector for some choice of their parameters.

A plane through $r_0$ with nonzero normal vector $n$ consists of the position vectors $r$ satisfying $(r-r_0)\cdot n=0$. Thus $r\cdot n=d$, with $d=r_0\cdot n$. When $n$ is a unit vector, $d$ is the signed distance from the origin in the direction $n$; for an unnormalized normal vector, that distance is $d/\Vert n\Vert$. The displacement within the plane is perpendicular to $n$.

The cross product of the two normal vectors is $(2,6,-3)\times (5,2,-1)=(0,-13,-26)$, so their intersection of two planes has direction $(0,1,2)$. Eliminating $2y-z$ gives $x=2$, and substitution then gives $z=2y-2$. Thus the Cartesian constraints and parametric equation of a straight line areThere is no valid division by a zero $x$ component in a symmetric ratio form; retaining $x=2$ explicitly handles that component.

The matrix rank $r$ is the dimension of its column space, equivalently the dimension of the image of the corresponding linear map. The equation is solvable precisely when $\mathbf{b}\in \mathrm{col}A$, equivalently when the augmented matrix has the same rank as $A$. If ${\mathbf{x}}_0$ is one solution, every solution is ${\mathbf{x}}_0+\mathbf{u}$ with $\mathbf{u}\in \ker A$, since subtracting any two solutions gives a kernel element. The rank-nullity theorem gives $\dim \ker A=3-r$.

Consequently, for rank three there is one solution for every $\mathbf{b}$, namely $A^{-1}\mathbf{b}$. For rank two there are no solutions outside the column plane, and an affine solution space of a linear equation of dimension one when consistent. For rank one, consistency requires $\mathbf{b}$ to lie on the column line and the solution set is an affine plane of dimension two. For rank zero, $A=0$: the solution set is all of ${\mathbb{R}}^3$ if $\mathbf{b}=0$, and empty otherwise. These descriptions include homogeneous systems, whose solution spaces pass through the origin.

The sphere has normal $\nabla (x_1^2+x_2^2+x_3^2)=(0,2,2)$ at the specified point, so its tangent plane isA general straight line through the origin is $\mathbf{x}=t\mathbf{d}$, $t\in \mathbb{R}$, for a fixed nonzero direction $\mathbf{d}=(d_1,d_2,d_3)$. To express the intersection using a $3\times 3$ matrix, choose independent vectors $\mathbf{u},\mathbf{v}$ spanning ${\mathbf{d}}^{\perp }$. The line is exactly the intersection of $\mathbf{u}\cdot \mathbf{x}=0$ and $\mathbf{v}\cdot \mathbf{x}=0$, henceThe first two rows have rank two. The third is independent of them exactly when $(0,1,1)\cdot \mathbf{d}=d_2+d_3\ne 0$. In that case the rank is three and the unique intersection isGeometrically the line crosses the tangent plane transversely. If $d_2+d_3=0$, the matrix rank is two: its third row is a linear combination of the first two, but the corresponding right-hand-side combination would be zero rather than two. The system is inconsistent and there is no intersection. The line is parallel to the tangent plane and cannot lie in it, since it contains the origin while the plane does not. Thus no choice of a genuine direction produces an infinite intersection in this particular problem.

A pure positive cosmological constant in a flat universe gives constant $H=H_0$, the expanding de Sitter model. ThusThe redshift drift is a straight line through the origin with positive slope. Here the source and observer expansion rates are equal; the factor $1+z$ in the observer contribution makes every positive-redshift comoving source drift toward still larger redshift.

Write a straight line in Cartesian form $Ax+By+C=0$, where $A,B,C$ are real and $(A,B)\ne (0,0)$. With $z=x+iy$, set $c=(A+iB)/2$ and $r=C$. Using the complex conjugate,Thus the real linear equation of a line in the complex plane isConversely, writing $c=u+iv$ gives $2ux+2vy+r=0$, an ordinary straight line with normal vector $(u,v)$. The nonzero condition on $c$ matters: $c=0$ would give either the whole complex plane or the empty set.

Write $z=x+iy$ and $\alpha =p+iq$. The complex conjugates in the PDF giveSince $(p,q)\ne (0,0)$, the locus is the straight line through the origin perpendicular to the position vector of $\alpha$ in the complex plane. Equivalently,

In the parametric description, $a$ is a point of the straight line and the nonzero vector $b$ gives its direction. The real parameter runs through every point of that line. For the cross product description, take another cross product with $c$ and use the vector triple product identity:Hence every solution has the representationHere $a\cdot b=0$ and $|b|=|c|$, as required. Conversely,Thus the perpendicularity assumption is sufficient, and the solutions form a straight line parallel to $c$. When $d\ne 0$, the vector $d$ is perpendicular both to $c$ and to every position vector on the line; it is the normal to the plane through the origin containing that line. The point $a=c\times d/|c{|}^2$ is the perpendicular foot from the origin. Its distance from the origin is $|d|/|c|$, since $c\cdot d=0$.

For a general parametric straight line, minimize the squared Euclidean distanceThis strictly convex quadratic function has its unique minimum at $\lambda =b\cdot (y-a)/|b{|}^2$, giving the orthogonal projectionFor the cross product description, $a\cdot c=0$, so this becomes

For the two planes, put $s=m\cdot n$ and $c=m\times n$. Nonparallel unit normals give $|s|<1$ and $|c{|}^2=1-s^2$. Seek the perpendicular foot in their span, $a=um+vn$. Its two dot products impose $u+sv=\mu$ and $su+v=\nu$. Solving gives both requested descriptions of the intersection:andIndeed the vector triple product identity gives $x\times (m\times n)=m(x\cdot n)-n(x\cdot m)$. Conversely, since $m,n$ are linearly independent, equality to $\nu m-\mu n$ forces both plane equations. Thus neither description introduces extra points.

Use the Riemann sphere convention for evaluating a Möbius transformation at a pole and at infinity. When the three prescribed points are finite, takeIts zero is $w_2$, its pole is $w_1$, and substitution at $w_3$ gives one. The remaining three cases areAll coefficient determinants are nonzero because the points are distinct. There is only one such Möbius transformation: the composite of any two candidate maps, one inverted, fixes $\infty ,0,1$; fixing infinity makes it affine, and fixing zero and one makes it the identity.

For this question use the cross-ratio normalizationIn the finite case this is $(w_3-w_1)(w_4-w_2)/[(w_3-w_2)(w_4-w_1)]$, with the corresponding limits if a point is infinite. It is finite and distinct from zero and one, since $w_4$ is distinct from the other three points and $f$ is a bijection. Ordering conventions for the cross-ratio differ; the definition here is fixed by the specified images, rather than by importing another ordering formula.

A generalized circle means either a circle or a straight line completed by infinity. Let $K$ be the unique generalized circle through the first three points. Its image under $f$ is the unique generalized circle through $\infty ,0,1$, namely $\mathbb{R}\cup \{\infty \}$. Therefore $w_4\in K$ exactly when $f(w_4)$ is real. Conversely a real $f(w_4)$ lies on that extended real line, whose inverse image is $K$. This proves the real cross-ratio criterion for a generalized circle in both directions.

By i, a free choice of $x_3$ is possible precisely when $\lambda =5$ and $\mu =2$. ThusThe complete solution set of a linear system isThe direction vector spans the kernel, and the displayed straight line is the entire solution set of a linear system.