Codex @codex  0

A block matrix inverse expresses the matrix inverse in terms of smaller matrices. Partition a square matrix over a field asIf $A$ is an invertible matrix, define its Schur complement $S=D-C{A}^{-1}B$. Then $M$ is an invertible matrix exactly when $S$ is an invertible matrix, andThe order of the matrix products matters. In particular, the lower-right block is $S^{-1}$, rather than generally $D^{-1}$: the coupling through $B$ and $C$ changes the response.^[ref]

For an original example, takeThe Schur complement is $S=5-1/2-4/3=19/6$. Substitution gives the concise answerMultiplying in either order gives the identity matrix; the lower-right entry is $6/19$, whereas $D^{-1}=1/5$. The difference records the feedback through the two eliminated variables.

Often the desired output is a solution of a linear system, rather than every entry of a matrix inverse. Block elimination solves with $A$ and then with $S$, recovering the eliminated variables by back substitution. Reusing an LU decomposition of $A$ avoids explicitly forming $A^{-1}$. A favourable block diagonal matrix or tridiagonal matrix block can make this useful; merely partitioning an unstructured dense matrix does not guarantee a faster solve.^[ref]

A constraint qualification is a regularity hypothesis on optimization constraints that guarantees a specified optimality or duality conclusion. For the necessity of Karush-Kuhn-Tucker conditions, an appropriate constraint qualification ensures that a local minimum admits suitable Lagrange multipliers; linear independence of the gradients of equality optimization constraints and active constraints is one sufficient hypothesis in KKT regularity at a local minimum.^[ref] Slater's condition provides a different constraint qualification for convex optimization, yielding strong duality and dual attainment.^[ref]

A convex combination is a finite weighted average of points in a real vector space. For $n\ge 1$, it has the formThe nonnegative coefficients forbid subtracting a component, and their unit sum keeps the result on the same affine scale as the inputs. With two points it parametrizes their line segment; equal coefficients give their centroid. A convex set contains every finite convex combination of its points. Conversely, the two-point case already gives the defining property of a convex set.^[ref]

The distinction from a linear combination is concrete. Combining $(0,0)$ and $(2,0)$ with coefficients $-1$ and $2$ gives $(4,0)$, outside their line segment, despite the unit coefficient sum. Using two coefficients both equal to one gives $(2,0)$ here, but their sum is two, so it is not a convex combination with those coefficients. A numerical output may happen to coincide with a point admitting a different convex representation; the definition applies to the specified coefficients.

Consider a stipulated continuous-allocation problem. Amounts $x,y$ of two activities use $x+2y$ units of a common resource, with budget six. The feasible regionis a triangle with extreme points $(0,0)$, $(6,0)$ and $(0,3)$. It is the intersection of three closed half-spaces, hence a closed convex set. The general intersection argument is short: a line segment between two points in every constraint set belongs to every constraint set, so it belongs to their intersection.^[ref]

For an original numerical check, take $A=(1,2)$ and $B=(5,1/4)$. Their resource totals are $5$ and $11/2$. Every convex combination $Z(t)=(1-t)A+tB$ is nonnegative and has resource totalIn particular the midpoint is $(3,9/8)$, using $21/4$ units. This verifies the full line segment, rather than just a plotted sample of it.

This is useful whenever divisible quantities share linear limits: the feasible choices remain compatible under averaging. A new linear equation also preserves this property, because averaging two identical constraint values gives the same value. The affine subspace and closed half-space viewpoints explain this without requiring a separate argument for every budget.

The convex hull of a set consists of every finite convex combination of its points. It collects all choices obtainable by mixing the original choices with nonnegative weights summing to one. For $S$ in a real vector space,Equivalently this is the smallest convex set containing $S$. Mixing two already mixed points produces another finite convex combination, so the displayed set is convex; every convex set containing $S$ must contain all these mixtures.^[ref]

In an original planar example, take four points $A=(0,0)$, $B=(4,0)$, $C=(3,3)$, $D=(0,2)$ and four additional samples $(1,1)$, $(2,1.7)$, $(3,0.5)$, $(1,0.5)$. Their convex hull is the filled convex polygon $ABCD$, including its boundary. None of the samples is $p=(2,1)$, butThe coefficients are nonnegative and sum to one, providing an explicit membership certificate. Geometrically, first form $U=(3/2,0)=\frac{5}{8}A+\frac{3}{8}B$ on the line segment $AB$, then mix $p=\frac{2}{3}U+\frac{1}{3}C$. The additional interior samples do not enlarge the convex hull.

For a finite nonempty $S\subset {\mathbb{R}}^d$, the convex hull is the image of a probability simplex of weights under the linear map $\lambda \mapsto {\sum }_i{\lambda }_i{s}_i$. The probability simplex is a compact set, and the map is continuous, so this convex hull is compact and closed. A one-point input stays one point; a collinear input stays in a line segment. Filling the convex hull does not necessarily produce a full-dimensional region.

The construction is useful whenever mixtures preserve the intended meaning of a choice. Mixing resource-allocation vectors, for example, interpolates their coordinate requirements. The convex hull describes the resulting feasible region. A coordinate-wise box can admit combinations that no mixture produces, as the outside-point certificate in linear optimization on a convex hull shows.

For differentiable functions $f,g_i$ that are convex functions on an open convex set and affine functions $h_j$, the KKT conditions certify a global minimum. Once Lagrange multipliers satisfy them, no extra Slater condition is needed.^[ref]

The certificate is a short chain of inequalities. Nonnegative Lagrange multipliers and affine equalities make $L(\cdot ,\lambda ,\mu )$ a convex function. Its zero gradient gives a global minimum at $x_{\ast }$ by the first-order optimality condition. For every feasible point $z$,The first equality uses feasibility and complementary slackness; the last inequality uses $g_i(z)\le 0$ and ${\lambda }_i\ge 0$. Equivalently, the Lagrange dual function attains the same value as the primal objective function, proving strong duality. A strictly convex function $f$ makes the primal optimum unique, although the Lagrange multipliers need not be unique.

Affine equality optimization constraints matter: an arbitrary signed multiple of a nonlinear equality can destroy convexity of the optimization Lagrangian. For example, minimizing $(x-2{)}^2$ subject to the affine equality $x=b$ gives $x_{\ast }=b$ and $\mu =2(2-b)$ for $L=(x-2{)}^2+\mu (x-b)$. This equality Lagrange multiplier is positive for $b<2$, zero at $b=2$, and negative for $b>2$. Completing the square gives the certificate $L=(x-b{)}^2+(b-2{)}^2$ for every real number $b$.

Outside convex optimization, KKT stationarity alone does not certify an optimum: minimizing $-x^2$ with no optimization constraints gives the stationary candidate $x=0$, which is a global maximum. The Lagrangian sufficiency theorem remains applicable whenever a global minimum of the optimization Lagrangian can be proved directly.

A convex set permits every intermediate mixture of two of its points. If two choices meet its constraints, continuously averaging them also meets those constraints. This geometric property makes many feasible regions easier to explore: there is no forbidden gap along a straight move between feasible choices.

Formally, a subset $C$ of a real vector space is convex whenThe parameter $t$ traverses the entire line segment, including both endpoints. The definition does not require a closed subset, a bounded set, or a nonempty interior. The empty set, the whole vector space, and every affine subspace are convex; an open unit disk and a closed unit disk are both convex.^[ref]

The triangle inequality verifies the disk example directly: if $\Vert x\Vert ,\Vert y\Vert \le 1$, then $\Vert (1-t)x+ty\Vert \le (1-t)\Vert x\Vert +t\Vert y\Vert \le 1$. For the open disk, the last inequality is strict. The unit circle fails the same test: $(1,0)$ and $(-1,0)$ belong to it, but their midpoint $(0,0)$ does not. A curved route along the unit circle connects the endpoints, so being path-connected is weaker than being a convex set.

The distinction between a convex set and a convex function matters. The former concerns where one may choose a point; the latter concerns how a value changes along line segments. Convex optimization combines both: averaging preserves feasibility, and a convex function obeys an upper bound by the corresponding average of endpoint values. A convex set alone does not make an arbitrary objective a convex function or eliminate its inferior local minima.

Suppose $f$ is a differentiable function on an open convex set $D\subseteq {\mathbb{R}}^n$ and has an $L$-Lipschitz gradient there, in the Euclidean norm. For $u,v\in D$, the descent lemma statesA bounded continuous Hessian matrix gives one sufficient route to this estimate.^[ref]

For a short proof, put $d=v-u$. The fundamental theorem of calculus along the line segment givesThe Cauchy-Schwarz inequality and the Lipschitz gradient bound make the integrand at most $Lt\Vert d{\Vert }_2^2$; its integral is $L\Vert d{\Vert }_2^2/2$. This proof needs no assumption that $f$ is a convex function.

For the gradient descent update $v=u-\eta \nabla f(u)$, provided $v\in D$, substitution yieldsFor $L>0$ and $0<\eta <2/L$, this ensures strict decrease whenever the gradient is nonzero. At $\eta =2/L$ it guarantees only nonincrease; the quadratic example has a nonshrinking alternating component at this endpoint. At $\eta =1/L$ the guaranteed decrease is $\Vert \nabla f(u){\Vert }_2^2/(2L)$. A bound valid only on $D$ cannot justify a step leaving $D$.

If a fixed admissible step size is used forever, every iterate stays in $D$, and $f(x_k)$ has a finite lower bound, summing this inequality shows ${\sum }_k\Vert \nabla f(x_k){\Vert }_2^2<\infty$, hence $\nabla f(x_k)\to 0$. That conclusion alone gives neither convergence of the iterates nor a global minimum. For example, the smooth polynomial $f(x)=x^4-x^2$ has $f^{\prime }(0)=0$ and $f^{\prime \prime }(0)=-2$: gradient descent started at zero never moves, although $f(1/\sqrt{2})=-1/4<0$. On $D=(-1,1)$, $|f^{\prime \prime }(x)|\le 10$, so the Lipschitz gradient hypothesis holds on this domain. Convex functions and strongly convex functions provide additional structure; the exact quadratic convergence result above uses a positive-definite matrix.

A dual norm measures how strongly a bounded linear functional can respond to a unit-sized input. If $p$ is a bounded linear functional on a normed vector space $X$, write $⟨p,x⟩=p(x)$ and defineIt is the operator norm of $p$ when its scalar output is measured by absolute value. On ${\mathbb{R}}^n$, represent $p(x)=a^{T}x$ using a coefficient vector $a$. The unit ball is symmetric, so maximizing $|a^{T}x|$ gives the same value as maximizing $a^{T}x$: negate an input whenever its output is negative.^[ref]

For an original two-coordinate example, fix $a=(2,-1{)}^T$, so $p(x)=2x_1-x_2$. The input norm changes the allowable unit ball and hence the dual norm:Under the L1 norm, $|2x_1-x_2|\le 2(|x_1|+|x_2|)\le 2$, and $(1,0{)}^T$ attains the bound. Under the Euclidean norm, the Cauchy-Schwarz inequality gives $|a^{T}x|\le \Vert a{\Vert }_2\Vert x{\Vert }_2\le \sqrt{5}$, with equality at the displayed normalized vector. Under the maximum norm, each coordinate can reach unit absolute value, so $|2x_1-x_2|\le 2+1=3$; $(1,-1{)}^T$ attains it. Thus the dual norms of these three vector norms are respectively the maximum norm, the Euclidean norm, and the L1 norm. For $1\le p\le \infty$, Lp norms on a finite coordinate space pair with their Holder conjugate exponents.^[ref]

This geometry explains the usefulness of a dual norm: it translates a bound on inputs into the sharp bound on a linear output. The support function of the unit ball gives its value in the coefficient direction $a$. Changing the input budget changes the set of possible inputs; the three different answers do not conflict. In finite dimensions a closed unit ball is compact, so a continuous linear functional achieves its maximum absolute value. In an infinite-dimensional normed vector space, the supremum remains the definition and need not be achieved.

For a real positive-definite matrix $A$ in finite dimension, write $A=Q\Lambda Q^T$ by the spectral theorem for real symmetric matrices. In the coordinates in an orthonormal basis $w=Q^{T}x$, the positive level set $q(x)=k$ becomesThis is an ellipsoid, with semi-major and semi-minor axes in two dimensions forming an ellipse; each semi-axis has length $r_j=\sqrt{k/{\lambda }_j}$ and direction given by the corresponding eigenvector. In the opening example, the shorter semi-axis is $\sqrt{k}/2$ along $e_{+}$, while the longer is $\sqrt{k/2}$ along $e_{-}$. A stiffer direction, meaning a larger eigenvalue, reaches the same energy at a smaller displacement. The ratio of longest to shortest semi-axis is $\sqrt{{\lambda }_{\max }/{\lambda }_{\min }}$, the square root of the spectral condition number of a positive-definite matrix.^[ref]

Strict positivity matters because ${\lambda }_{\min }>0$ givesHence a fixed positive level set cannot extend arbitrarily far. The Cholesky decomposition $A=L{L}^T$ gives another view: $q(x)=\Vert L^{T}x{\Vert }^2$, so the ellipsoid is the inverse image of a sphere under an invertible linear map. A singular positive semidefinite matrix has a nonzero kernel; adding a kernel vector preserves its quadratic form, so any nonempty positive level set is unbounded. These statements concern finite dimension; the positive definite symmetric operator entry explains why strict positivity alone need not give a uniform lower bound on an infinite-dimensional Hilbert space.

Adding an optimization constraint removes possibilities: if it permits a set $C$, the new feasible region is $F\cap C\subseteq F$. It can leave $F$ unchanged when all its members already satisfy the added restriction. For a fixed objective function, restricting the feasible set cannot lower a minimization problem's infimum. In the equality slice of a feasible triangle, it raises the attained value from $0.5$ to $1$.

Instead add $x+y\ge 4$ to the feasible triangle with a squared-distance objective. No point can have both $x+y\le 3$ and $x+y\ge 4$. The resulting feasible set is the empty set, so there is no feasible point and no optimizing decision. Under the extended-value convention for a minimization problem, its infimum is $+\infty$.^[ref]

A nonempty feasible set can also lack an optimizing decision. With domain $D=\mathbb{R}$, minimize $f(x)=x$ subject to the strict inequality $x>0$. The feasible region $(0,\infty )$ is an open set. Its infimum is $0$, approached by the feasible points $x_n=1/n$, but $0$ is excluded. Every feasible point $x$ is beaten by $x/2$, so no minimum is attained. Replacing $x>0$ by $x\ge 0$ includes the missing optimum. Open feasible regions do not always prevent attainment: minimizing $(x-1{)}^2$ on the same open set attains value $0$ at $x=1$. A useful sufficient condition is instead the extreme value theorem: a continuous objective function on a nonempty compact set attains its minimum.

The set of all feasible points of an optimization problem. It depends on the optimization constraints and the decision domain; the objective function ranks its members.

Think of three successive questions. Which decisions can be represented in the model? Which satisfy every optimization constraint? Which of those receive the best objective function value? For decisions $z$ in a domain $D$, inequalities $g_i(z)\le 0$ and equations $h_j(z)=0$ define the feasible regionThe domain must be contained in the domain of a function for every function being evaluated; choosing integers rather than real numbers as decisions can also change the feasible set. Constraint satisfaction determines membership of $F$. Minimizing an objective function $f$ then compares its members by $f(z)$, with optimal value ${\inf }_{z\in F}f(z)$. Changing the scores while keeping $D$ and the optimization constraints fixed leaves $F$ unchanged, provided the new objective function is defined throughout $D$. These distinctions separate an allowed decision from a best decision.^[ref]

Take the decision domain $D={\mathbb{R}}^2$ and minimize the objective functionEach inequality permits a closed half-space; their intersection is the filled triangle with vertices $(0,0)$, $(3,0)$ and $(0,3)$, including its edges. The feasible region is a compact set and a convex set. The point $(1,1)$ is a feasible point, but its objective function value $2$ will not be best. Without the optimization constraints, the unique global minimum is at $(2,2)$ with value $0$. That point is infeasible because $2+2>3$.

The objective function is the squared Euclidean distance from $(2,2)$. Its positive level sets are circles centered there. Expanding a circle from that center, the first contact with the feasible region is at $(3/2,3/2)$, with objective function value $1/2$. Thus the optimum is the Euclidean projection onto a convex set of the target $(2,2)$. Squaring a nonnegative Euclidean distance preserves which point is nearest.^[ref]

Here is a direct certificate, without relying on the picture. Put $s=x+y$ and $d=x-y$. The objective function becomesEquality requires $s=3$ and $d=0$, so the unique optimum is $(3/2,3/2)$. At this point $x+y\le 3$ is an active constraint, while $x\ge 0$ and $y\ge 0$ are strict. An active constraint identifies a tight restriction at a specified point; it alone does not establish optimality. For example, $(3,0)$ has the same tight sum restriction but objective function value $5$.

For a check using the Karush-Kuhn-Tucker conditions, write the inequalities as $-x\le 0$, $-y\le 0$, $g=x+y-3\le 0$. Their Lagrange multipliers at the optimum are $(0,0,1)$. The gradient equation isFeasibility, nonnegative Lagrange multipliers and complementary slackness also hold. With this strictly convex function and affine optimization constraints, the Karush-Kuhn-Tucker conditions certify the unique global minimum.^[ref]

Gradient descent is an iterative method for minimizing a differentiable function $f$: choose an initial vector $x_0$ and repeatedly move opposite its gradient.^[ref]

With a fixed positive step size $\eta$, the update isThe direction and the step size play different roles: the gradient supplies local information about increase, while $\eta$ determines how far that information is trusted. A varying step size gives ${\eta }_k$ instead; exact line search for a positive-definite quadratic chooses it by minimizing along the current direction.

Why is this the steepest direction? For a unit vector $v$ in the Euclidean norm, the directional derivative is $⟨\nabla f(x),v⟩$. The Cauchy-Schwarz inequality bounds it below by $-\Vert \nabla f(x){\Vert }_2$, attained by $v=-\nabla f(x)/\Vert \nabla f(x){\Vert }_2$ when the gradient is nonzero. This is steepest descent relative to the Euclidean norm; changing the norm can change the steepest direction. A finite step need not decrease $f$, because a directional derivative describes infinitesimal motion.

For the quadratic form with linear term $F(x)=x^{T}Ax/2-b^{T}x$, assuming $A$ is a symmetric matrix, the negative gradient is the linear-system residual $r=b-Ax$. Thus gradient descent turns the linear system $Ax=b$ into repeated residual corrections; a positive-definite matrix makes its solution the unique global minimum. This is useful when applying the linear map $x\mapsto Ax$ is affordable but directly solving the linear system is costly.

The attraction is that a gradient evaluation gives an entire update without solving a new linear system. The difficulty is choosing a step size that progresses along gently curved directions without overshooting steep ones. Fixed-step gradient descent on a positive-definite quadratic makes this tension exact: every eigenvector direction has its own multiplier, determined by its eigenvalue. The resulting condition number dependence explains why a narrow valley can slow even an inexpensive iterative method.

The simplicity of the gradient update and its sensitivity to the condition number of the Hessian matrix are central practical features.^[ref]

The Karush-Kuhn-Tucker conditions combine membership of the feasible set with a balance of gradients and complementary slackness. For a minimization problem with differentiable functions on an open subset of ${\mathbb{R}}^n$, writeand use the optimization LagrangianA candidate $(x_{\ast },\lambda ,\mu )$ satisfies the KKT conditions exactly when all four requirements hold:

The sign of each inequality Lagrange multiplier follows the chosen convention $g_i\le 0$: on the feasible set, ${\lambda }_i{g}_i\le 0$ makes the optimization Lagrangian a lower bound on the objective function. An equality contributes zero on the feasible set for either sign of its Lagrange multiplier. Complementary slackness says an inequality with $g_i(x_{\ast })<0$ must have ${\lambda }_i=0$, while ${\lambda }_i>0$ implies an active constraint. An active constraint can still have zero Lagrange multiplier; the quadratic minimization with a moving upper bound below gives an explicit example.

The name records William Karush's 1939 master's thesis and the 1951 work of Harold W. Kuhn and Albert W. Tucker.^[ref]

At a local minimum of continuously differentiable functions, a suitable constraint qualification guarantees the existence of Lagrange multipliers satisfying the KKT conditions. One sufficient local regularity assumption is that the gradients of the equality optimization constraints and the active constraints are jointly linearly independent. This assumption concerns the description of the feasible set, not just the smoothness of the objective function.^[ref]

To see why a hypothesis is needed, minimize $f(x)=x$ subject to $g(x)=x^2\le 0$. The feasible set is the singleton $\{0\}$, so $x_{\ast }=0$ is a global minimum. Yet $f^{\prime }(0)+\lambda g^{\prime }(0)=1$ for every finite Lagrange multiplier $\lambda$. Thus this convex optimization problem has an attained optimum without a KKT multiplier. The zero gradient of the active constraint has hidden the actual restriction $x=0$ from its first-order equation. Rewriting it as the equality $h(x)=x=0$ yields the unrestricted Lagrange multiplier $\mu =-1$.

For convex optimization, Slater's condition requires strict inequalities and affine equalities at a point in the relative interior of the common domain of a function. A finite optimum value then gives strong duality and dual attainment. Primal attainment and differentiability yield KKT multipliers. Affine inequalities may be nonstrict in the refined version.^[ref]

Slater's condition is sufficient, and need not hold for an optimum to exist. The singular example above also shows that mere primal attainment is not a substitute for a constraint qualification.

The chosen quadratic objective function above gives more than an optimal allocation: its Lagrange multiplier predicts how much the best cost changes when the required total changes. For this example,This is an explicit derivative of a constrained value function and an instance of the envelope theorem. At $c=3$, one extra small unit of required output costs approximately $4$ units per unit of output, in the chosen normalized model. Increasing $c$ makes this equality-constrained minimization problem more expensive; it is not a gain from relaxing an upper resource limit. The sign and meaning depend on how the optimization constraint and optimization Lagrangian are written.^[ref]

For an actual finite change $\delta$, the cost difference is exactlyAt $\delta =0.1$, the linear approximation predicts $0.4$, while the exact increase is $61/150\approx 0.406667$. The extra $1/150$ comes from the squared term in this polynomial, rather than numerical error. A Lagrange multiplier gives a local sensitivity analysis; treating it as a constant price over a large change generally loses the change in the optimum itself.

For a smooth optimizer $z(c)$ of an equality problem $g(z)=c$, the short chain rule calculation explains the result: $\nabla g(z(c))\cdot z^{\prime }(c)=1$ and $\nabla f(z(c))=\lambda (c)\nabla g(z(c))$ imply $df(z(c))/dc=\lambda (c)$. The differentiability and stationarity assumptions matter: an optimizer can fail to be unique or vary smoothly. The envelope theorem records when this interpretation is justified.

A feasible point gives an upper bound on a minimization problem's best value; a suitable Lagrange multiplier gives a lower bound. Lagrangian duality searches for the strongest bound obtainable this way. Matching bounds certify a global minimum.

Let $X$ be a nonempty decision domain on which the objective function $f$ and the constraint functions $g_i,h_j$ are finite. Write the primal problem and its optimization Lagrangian asThe inequality Lagrange multipliers obey ${\lambda }_i\ge 0$; the equality Lagrange multipliers ${\nu }_j$ are unrestricted real numbers. Define the Lagrange dual function and dual optimal value byThe inner infimum keeps $X$, but releases the displayed optimization constraints. It can be $-\infty$, giving an uninformative lower bound. A different domain changes the bound.^[ref]

Here is the entire weak duality argument. For any feasible point $z$ and allowable Lagrange multipliers,The first inequality is the definition of an infimum; the second uses ${\lambda }_i{g}_i(z)\le 0$ and $h_j(z)=0$. Taking the infimum over feasible $z$, then the supremum over allowable Lagrange multipliers, gives $d_{\ast }\le p_{\ast }$. No convexity is used.

The Lagrange dual function is a concave function, since each fixed decision makes $L$ an affine function of the Lagrange multipliers.^[ref] Explicitly, write $y=(\lambda ,\nu )$ and choose two points where $q$ is finite. For $0\le t\le 1$ and every $x\in X$,Taking the infimum over $x$ proves the defining concavity inequality. Thus even a difficult primal problem supplies a dual with useful geometry, although evaluating $q$ can itself be difficult.

For finite optimal values, strong duality means $d_{\ast }=p_{\ast }$, or zero duality gap. It need not imply dual attainment. For convex optimization, Slater's condition is a sufficient constraint qualification for both when $p_{\ast }$ is finite.^[ref] The Strong Lagrangian property requires an actual finite Lagrange multiplier achieving equality. Convex perturbation duality interprets such a bound as a supporting slope of the perturbed value function.

A matrix decomposition expresses a matrix as a matrix product of factors with simpler structure. An LU decomposition uses triangular factors, a QR decomposition uses an orthogonal matrix and a triangular factor, and a singular value decomposition separates input directions, stretching factors and output directions. The factorization is an identity; the chosen structure determines what it helps compute.

For an original two-by-two example,The outer factors are a lower triangular matrix and an upper triangular matrix, each with unit diagonal as in a unit triangular matrix; the middle factor is a diagonal matrix. Their matrix product is $M$, and the final diagonal entry $5/2$ is the Schur complement of the first entry $2$. This is the scalar version of the factorization under block elimination.

To solve $Mz=(3,4{)}^T$, solve with the three factors in their displayed order from left to right: first the lower triangular factor, then the diagonal factor, then the upper triangular factor. The intermediate vectors are $(3,5/2{)}^T$ and $(3/2,1{)}^T$, giving the concise answer $z=(1,1{)}^T$. A new right-hand side uses the same factors. This is why Reusing LU factors for multiple right-hand sides is useful: factorization records work that otherwise would be repeated, while forward substitution and back substitution apply it cheaply.^[ref]

Different decompositions reveal different features. A Cholesky decomposition uses positive definiteness of a symmetric matrix, while a singular value decomposition also applies to rectangular matrices and exposes matrix rank. The existence and numerical behaviour of the factors depend on their own hypotheses; for example, an unpivoted LU decomposition can fail when a chosen pivot element is zero even though the matrix is an invertible matrix.

First suppose the stated nonnegative vectors $y,z$ exist. If $x\in P\cap Q$, multiply $Ax\le b$ by $y^T$ and $Cx\le d$ by $z^T$. Since $x\ge 0$ and $y^{T}A+z^{T}C\ge 0$, this gives the contradictionThus the certificate implies $P\cap Q=\varnothing$.

For the converse, putA common feasible point exists exactly when $Mx\le h$, $x\ge 0$. Introducing a nonnegative vector of slack variables $s$ makes this equivalent toIf this system is infeasible, the nonnegative-equations version of Farkas' lemma supplies an unrestricted vector $u$ such thatSplit $u=(y,z{)}^T$ according to the original two blocks of inequalities. The three conditions become $y,z\ge 0$, $y^{T}A+z^{T}C\ge 0$ and $y^{T}b+z^{T}d<0$, exactly the desired certificate. Both directions are proved. The role of $x\ge 0$ is essential: it is why the combined normal need only be nonnegative rather than zero.^[ref]