A partition of a sample space is a finite or countable family of pairwise disjoint events $B_i$ whose union is the whole sample space. For an integrable random variable, the partition form of the law of total expectation isZero-probability cells contribute zero and are omitted because elementary event-conditioning on them is undefined. The formula also holds with extended nonnegative values when $X\ge 0$. Integrability makes the signed series legitimate in the finite-expectation case.

For $\mathbb{P}(X=x)>0$, the conditional probability mass function isThe positive denominator makes the expression well-defined, and its sum over $y$ is one.

The joint probability mass function is $p_{X,Y}(x,y)=\mathbb{P}(X=x,Y=y)$ for $(x,y)\in S^2$. It is nonnegative and its sum over $S^2$ equals one. Summing over one coordinate gives the other variable's marginal distribution.

Gradient descent uses only the current gradient and a suitable step size; its additional storage is order $d$. It is attractive for very large $d$ or when low per-iteration cost matters most. For a smooth strongly convex function, standard step choices give linear convergence, with a rate sensitive to the condition number. Mere convexity gives a global-minimum interpretation but does not alone imply a linear rate; for nonconvex objectives the usual guarantee concerns stationary points.

BFGS uses successive gradients to update an approximation to inverse curvature, incorporating information that can correct poor scaling. Full BFGS uses order $d^2$ storage and update work. With suitable line searches and local smoothness and positive-curvature assumptions near a nondegenerate minimum, it has superlinear convergence. This often favours it in moderate dimensions when gradients are not extremely cheap relative to the extra work.

Neither gradient descent nor BFGS requires an explicitly evaluated Hessian matrix. Thus an expensive Hessian matrix is compatible with both, rather than a reason to reject BFGS. If an accurate Hessian matrix and its factorization are exceptionally cheap, Newton's method in optimization becomes an additional reasonable option. Convexity helps identify a stationary point as a global minimum; without it, both methods need suitable descent and curvature safeguards. The BFGS update learns curvature from the Quasi-Newton secant equation.^[ref]

Convexity of the components of a vector field is not the decisive criterion for a root-finding problem. Neither method becomes globally reliable merely because each component is convex.

Prefer the Newton root-finding iteration when the Jacobian matrix is available accurately and at reasonable cost. Solve $J({\mathbf{x}}_k){\mathbf{s}}_k=-\mathbf{f}({\mathbf{x}}_k)$ rather than explicitly forming an inverse matrix. Near a root with a nonsingular Jacobian matrix and a locally Lipschitz Jacobian matrix, the usual exact iteration has quadratic convergence.

Prefer Broyden's method when evaluating or differentiating the vector field makes a fresh Jacobian matrix expensive. It learns an approximate Jacobian matrix through secant updates using function differences. Under appropriate local regularity and initialization assumptions it has superlinear convergence, generally slower asymptotically than the quadratic convergence of exact Newton steps, but often with cheaper iterations.^[ref]

Dimension matters through storage and solves. A full dense Newton factorization costs order $d^3$ arithmetic operations; a dense Broyden approximation uses order $d^2$ storage, and maintained inverse or factorization updates can cost order $d^2$. A naive fresh dense solve can still cost order $d^3$. Sparsity can favour Newton, while very large $d$ can make either dense formulation unsuitable. Neither local rate guarantees convergence from an arbitrary starting point.

Use bisection on $f^{\prime }$ when the derivative is cheap and a bracket $f^{\prime }(a)<0<f^{\prime }(b)$ is easy to obtain. For a differentiable convex function, $f^{\prime }$ is nondecreasing, so such a bracket locates a global minimum, allowing a flat minimizing interval. Without convexity, first isolate a region containing the desired local minimum; arbitrary stationary points need not be minima.

Use golden-section search when reliable function values are available but derivatives are unavailable or expensive. It needs a bracket on which the objective is a unimodal function, commonly found by three ordered points whose middle value is below the two outer values. Those inequalities alone do not prove that the objective is a unimodal function. A convex function supplies the needed shape; for a nonconvex objective, isolate an interval on which it is a unimodal function.

Both methods have linear convergence of bracket width. Bisection contracts the width by $1/2$ per new derivative evaluation. Golden-section search contracts it by $(\sqrt{5}-1)/2\approx 0.618$ per new function evaluation after initialization, reusing one earlier value. Choose by evaluation cost and bracket availability as well as this contraction factor: comparing iteration counts alone can misidentify the faster method.

The origin is feasible and has objective zero, while a Euclidean norm is nonnegative. Thus the minimum is $0$, and the complete argmin isIf $A$ is injective, this set contains only the origin. If it has a nontrivial kernel, every feasible kernel vector is also a minimizer. No Lagrange multipliers are needed.

Interpret the norms as Euclidean norms and set $B=A^{\mathsf{T}}A$. This is a real symmetric matrix with ${\mathbf{x}}^{\mathsf{T}}B\mathbf{x}=\Vert A\mathbf{x}{\Vert }^2\ge 0$, hence a positive semidefinite matrix. Squaring the nonnegative objective does not change its argmin. Let ${\lambda }_{\min }$ be the smallest eigenvalue of $B$ and $E_{\min }$ its eigenspace.

First minimize on the unit sphere. The Lagrange multiplier function $L(\mathbf{x},\lambda )={\mathbf{x}}^{\mathsf{T}}B\mathbf{x}-\lambda ({\mathbf{x}}^{\mathsf{T}}\mathbf{x}-1)$ has stationarity condition $B\mathbf{x}=\lambda \mathbf{x}$. Multiplication by ${\mathbf{x}}^{\mathsf{T}}$ shows that the squared objective at such a point is $\lambda$. The spectral theorem for real symmetric matrices also gives, for every unit vector, ${\mathbf{x}}^{\mathsf{T}}B\mathbf{x}\ge {\lambda }_{\min }$, with equality exactly in $E_{\min }$. This proves global minimality, rather than merely finding stationary candidates.

If ${\lambda }_{\min }>0$, every point with $\Vert \mathbf{x}\Vert >1$ has a strictly smaller objective after rescaling to unit length. ThereforeIf ${\lambda }_{\min }=0$, the kernel of a linear map satisfies $\ker B=\ker A$, since $B\mathbf{x}=0$ implies $\Vert A\mathbf{x}{\Vert }^2=0$, and the converse is immediate. The minimum is then zero, attained precisely atIn this case minimizers need not have unit length. This distinction is essential for a rank-deficient $A$, including any real $m\times n$ matrix with $m<n$. The result is the minimum image norm outside a Euclidean unit ball, governed by the least singular value.

Since $\mathbf{a}\ne 0$, the line $\mathbf{x}=t\mathbf{a}/\Vert \mathbf{a}{\Vert }^2$ lies in the domain and satisfies ${\mathbf{a}}^{\mathsf{T}}\mathbf{x}=t$. The restriction of $l$ to this line is $t^3$, whose second derivative is $6t$: negative for $t<0$ and positive for $t>0$. It can be neither globally convex nor globally concave. Hence $l$ is neither. Equivalently, its Hessian matrix is $6({\mathbf{a}}^{\mathsf{T}}\mathbf{x})\mathbf{a}{\mathbf{a}}^{\mathsf{T}}$, changing the sign of its nonzero directional curvature across ${\mathbf{a}}^{\mathsf{T}}\mathbf{x}=0$.

The denominator is positive throughout the domain. Holding $y$ fixed, differentiate with respect to $x$:At the interior point $(1,1/2)$ this second partial derivative is $-2/9<0$, so the restriction to a horizontal line is locally strictly concave, ruling out convexity. At the interior point $(1,2)$ it is $4/9>0$, ruling out concavity. Restrictions of a convex function or a concave function to every line segment must retain that property. Consequently $k$ is neither convex nor concave on its stated domain.

Differentiating on the given domain yieldsThe denominator and $\ln x+1$ are positive there. Thus $j$ is strictly concave, and is not convex.

Suppose $n=p_1\cdots p_r=q_1\cdots q_s$ are two products of positive primes. If $r=0$, then $n=1$, and necessarily $s=0$, since any nonempty product of positive primes is at least $2$. Otherwise Euclid's lemma, applied repeatedly to the right-hand product, says $p_1$ divides one $q_j$. Since $q_j$ is prime, $p_1=q_j$. Reorder the right-hand factors and cancel this common nonzero factor. Repeating, or using induction on the number of factors, matches every factor. Neither side can have factors left over, because their product would then equal $1$. Thus the prime factorization is unique up to rearrangement.

If $p\mid a$, there is nothing to prove. Otherwise the only possible positive common divisors of the prime $p$ and $a$ show that $\gcd (p,a)=1$. By Bézout's identity, $up+va=1$ for some integers $u,v$. Multiplying by $b$ gives $b=upb+vab$. Both terms on the right are divisible by $p$, the second because $p\mid ab$. Hence $p\mid b$. This proves Euclid's lemma, including when either factor is zero or negative.

Use strong induction on the positive integer $n$. The case $n=1$ is represented by the empty product. For $n>1$, if $n$ is prime it is already a product of one prime. Otherwise $n=ab$ with $1<a,b<n$; the induction hypothesis gives prime factorizations for both $a$ and $b$, and concatenating them gives one for $n$. This proves existence in the fundamental theorem of arithmetic without presupposing uniqueness.

The Euclidean algorithm repeatedly replaces $(r_{i-1},r_i)$ by $(r_i,r_{i-1}-q_i{r}_i)$, where the new remainder lies between $0$ and $r_i-1$. Common divisors are unchanged by this subtraction, and decreasing positive remainders ensure termination. The last nonzero remainder is the greatest common divisor.

The extended Euclidean algorithm also tracks coefficients $r_i=u_{i}a+v_{i}b$, beginning with coefficient pairs $(1,0)$ and $(0,1)$ and applying the same subtraction to the pairs. This invariant produces Bézout coefficients for the last nonzero remainder. Here the divisions areBack substitution givesThus $\gcd (580,252)=4$, with $m=10$ and $n=-23$ in Bézout's identity.

The greatest common divisor of positive integers $a,b$ is the largest positive integer dividing both, denoted $\gcd (a,b)$. The set of positive common divisors contains $1$ and is finite, so this definition supplies a value.

Fix $b\in B$. Since $g\circ f$ is surjective, there is $a\in A$ with $g(f(a))=g(b)$. Injectivity of $g$ gives $f(a)=b$, proving that $f$ is surjective. All five conclusions are instances of injectivity and surjectivity under composition.

Suppose $g(b)=g(b^{\prime })$. Since $f$ is surjective, choose $a,a^{\prime }\in A$ with $f(a)=b$ and $f(a^{\prime })=b^{\prime }$. Then $(g\circ f)(a)=(g\circ f)(a^{\prime })$, so injectivity gives $a=a^{\prime }$ and therefore $b=b^{\prime }$. Thus $g$ is injective. The use of surjectivity is what makes every point of $B$ accessible to this argument.

If $f(a)=f(a^{\prime })$, applying $g$ gives $(g\circ f)(a)=(g\circ f)(a^{\prime })$. Injectivity of the function composition forces $a=a^{\prime }$. Hence $f$ is injective.

For any $c\in C$, surjectivity of $g\circ f$ supplies $a\in A$ with $g(f(a))=c$. Taking $b=f(a)\in B$ proves that $c$ has a preimage under $g$. Thus $g$ is surjective.