Past exam of the optimisation for data science course of the University of Oxford 2023 3 d Solution 2026-10-25
The supporting inequality in part 2(a) bounds below by an affine function plus a positive squared Euclidean norm, making it a coercive function. Its continuity gives a global minimizer , which is unique by strict convexity. Let , and . Here is the block size, so its role differs from the number of component functions in Question 2. Use randomized block coordinate descent with uniformly sampled blocks of size , independently at each update. Sampling uniformly from a fixed partition into equal-sized blocks gives the same calculation, since each coordinate has inclusion probability .
Let be a common upper bound on the block Lipschitz constants. Part (c) permits . With step size , part (a) givesSumming squared partial derivatives according to their inclusion probabilities,For a strongly convex function, part 2(a) also yields the strong convexity gradient inequalityIndeed, substitute into the supporting inequality and complete the square in : the expression is at least for every .
Combining the last three estimates gives a contraction in conditional expectation:By the tower property and repeated application,An optimization epoch counts coordinate partial derivative evaluations: it comprises block updates or one full gradient update. With sampling with replacement, an optimization epoch is this work measure; it need not visit every coordinate. Consequently the block method needsto bring the expected objective function gap below , for , up to integer rounding.
For the steepest descent method, use a global Lipschitz gradient constant . Part (c) with the full block permits . The descent lemma with step size , together with the strong convexity gradient inequality, givesAn exact line search along the negative gradient does at least as well in objective function decrease per update. Ignoring its additional work, the resulting epoch complexity isThe common coordinate-based guarantees favour blocks by a factor in epochs. In fact, with the conservative constants and , both methods have the same guaranteed contraction per update, while a block update uses only coordinate partial derivatives.
This compares the bounds justified by the given constants. For a particular objective function, sharper constants give the comparison against ; an actual speedup of is not universal. Computing a block coordinate gradient also need not cost exactly of a full gradient evaluation. The epoch complexity comparison assumes that coordinate partial derivative counts are a useful measure of work, and does not itself establish a wall-clock comparison.
The map is -Lipschitz continuous whenThe same finite Lipschitz constant must work for all pairs of points.
Let and let be a Lipschitz constant for . Sinceboth coefficients and are bounded and globally Lipschitz. The global existence theorem for stochastic differential equations with Lipschitz coefficients gives a unique global strong solution of a stochastic differential equation.
The same Itô formula cancellation as before givesFor , the stopped exponential is bounded between and . Continuity and strict positivity of giveThe Itô isometry therefore yieldsThe monotone convergence theorem proves . The bounded optional stopping argument is now identical to part (b)(iii), and givesPassing from level hitting to strict exceedances as in part (b)(iv),No global positive lower bound on was needed: compact-interval positivity suffices for each exit argument. The law is independent of the particular because the scale function is always : its infinitesimal generator satisfiesThis is the exponential scale function for a drift tied to diffusivity. Changing changes how quickly the path is traversed, but leaves these boundary probabilities and the maximum law unchanged.
Picard-Lindelöf theorem Created 2026-09-24 Updated 2026-10-14
Let be an open set and a continuous function. Suppose is locally Lipschitz in the state , with a Lipschitz constant uniform on a sufficiently small time-state neighborhood of each point. Then each determines a unique local solution of a differential equation to the displayed initial value problem, on an interval around . This is the local theorem in Ordinary Differential Equations and Dynamical Systems by Gerald Teschl.
For example, , , has the solution of a differential equation : its derivative is itself and its starting value is . The Lipschitz constant is , since . Thus uniqueness follows even before computing . Picard iteration below constructs this exponential function one polynomial at a time.
The Lipschitz condition has real content. For the continuous function involving a square root,each waiting time gives a different forward solution of a differential equation:The constant solution of a differential equation is another choice. At , delays give respectively. Before the join, both sides of the ordinary differential equation are zero; afterwards, . Both one-sided derivatives at are zero, so the join is valid. All these trajectories can also be extended by zero to . The failed Lipschitz condition is visible in the quotientFor this quotient is . Continuity still allows local existence by the Peano existence theorem; it cannot choose a waiting time. MIT's 18.034 Lecture 2, Example 2.3 develops the same waiting-time mechanism for a different ordinary differential equation, .
Failure of uniqueness and finite-time blowup are different obstructions
. Left: several delayed solutions of a differential equation of share . Right: the unique solution of a differential equation of , , has finite-time blowup at . The right vertical axis ends at ; the curve continues upward beyond the plot, and there is no finite endpoint value. Details.The useful promise is that an initial condition and a sufficiently regular local rule select one trajectory. It justifies comparing models and following trajectories without an arbitrary choice at each instant. A locally Lipschitz continuous rule need not keep that trajectory bounded: meets the theorem's hypotheses everywhere but has finite-time blowup. Longer existence is governed by the finite-dimensional continuation criterion.
Minimize over usingThe pointwise Euclidean projection onto a convex set sends a block to . The gradient of has Lipschitz constant , so finite-dimensional projected gradient descent converges for . On an unscaled square grid with , is safe. Reconstruct ; unused dual directions may make the minimizing nonunique while the primal minimizer is unique.
Randomized block coordinate descent 2026-10-25
A coordinate descent method that randomly selects a block and updates only its coordinates. Suppose each coordinate is selected with probability , every block has Lipschitz constant at most , and the objective function is a strongly convex function with modulus . The block descent lemma and the strong convexity gradient inequality giveFor uniform blocks of size in coordinates, updates constitute an optimization epoch, and the epoch complexity is . Here is the initial objective function gap, and the target is its expected gap at most . A coordinate-based common bound is ; actual constants can improve it.
Target: gradient-descent.bigb, under Gradient descent. Robust swing-foot clearance margin 2026-11-05
A robust swing-foot clearance margin is a guaranteed lower separation between a robot's swing foot and terrain over a specified crossing interval and set of allowed errors. It differs from clearance computed only for a nominal trajectory. The interval must exclude intended ground contacts when a strictly positive margin is required.
An original point-foot model lets the planned terrain be and the planned clearance be . Assume actual vertical foot error , horizontal error , and terrain-model height error bounded by . If has Lipschitz constant over the relevant neighborhood, thenThe actual clearance consequently satisfiesThis follows from the triangle inequality and covers all errors in the stated bounds. Their effects add in a worst-case calculation; combining them as independent random standard uncertainties would answer a different question.
For a local sloped-terrain example, take , , and . The total clearance allowance is . Guaranteeing a margin therefore requires nominal clearance at least throughout that crossing. A nominal gap gives only a lower bound and cannot certify the requested separation.
Finite path samples need another allowance. Suppose the nominal clearance as a function of path parameter is Lipschitz with constant , and every point lies within of a checked sample. If the smallest sampled clearance is , a clearance bound between trajectory samples givesWith per unit parameter and sample spacing , between-sample uncertainty costs another . The preceding example would then require sampled clearance at least for a certified margin.
Bounds must match the geometry. A vertical obstacle edge is not covered by a finite height-slope bound across its discontinuity; use the full configuration-space obstacle instead. Shared coordinate-frame errors can also cancel in a relative clearance, so separate component bounds can be conservative when their joint dependence is known.
The primary robotics formulation requires constraints along the full path and distinguishes planned from executed motion.[ref]
Robotic Systems by Kris Hauser. Chapter 8, Point-to-point path planning and Explicit vs implicit plans
A proposed test records terrain and foot geometry in one reference frame, validates the stated error bounds, and compares a continuous-path certificate with observed minimum separation. A passing sample grid without the intervening bound does not establish the same guarantee. Scalar clipping Created 2026-10-14 Updated 2026-10-25
The continuous map projecting a real number onto the closed interval , with . It equals below the interval, equals the input inside it, and equals above it. It is nondecreasing and has Lipschitz constant one.
Spatial pressure-map resolution below a sensor cell concerns which variations of a contact pressure field can actually be distinguished by spatial measurements. A smaller display pixel does not imply a smaller independently measured contact region. The response of a pressure sensor, its sampling positions and any imposed reconstruction assumptions must be specified.
For an original ideal array model, cell of known area reports an area-average pressure,It determines that cell’s load , but not every value inside it. For a more general linear response, with known sensitivity . The null space of a spatial pressure measurement consists of perturbations whose weighted integrals vanish for all measured channels. Nonnegative fields that differ by such a perturbation remain indistinguishable in that model.
Consider square cells of side aligned with , and compare fields independent of :Every cell contains one full cosine period, so both fields give every channel the identical value . Yet varies between and . With and , a perfectly uniform field and a field spanning – produce the same ideal map. On an aligned finite rectangular array, the added cosine also has zero first moment within each cell, so even the global resultant and center of pressure agree. This is an inverse-problem construction, not a claim that both profiles arise from the same elastic contact geometry.
Cell averaging has a calculable spatial response. For a one-dimensional centered interval of width , averaging a complex wave multiplies its value at the interval center byThe first zero at explains the erased cosine above. Sampling introduces further ambiguity; even translating this ideal array cannot recover a component erased by its aperture at every position.
A regularity assumption can supply a conditional bound. If has a validated spatial Lipschitz constant over a square cell, its center value differs from its area average by at mostEvery point in the cell is at most half the diagonal from its center, so integrating the Lipschitz inequality proves this bound. For and , it gives about . The array readings alone do not establish that slope bound.
Interpolation chooses a finer representation from the measured values. SciPy explicitly describes its array zoom operation as spline interpolation; it does not describe collecting new pressure data.[ref] A proposed apparatus test scans a known small indenter across a sensor array at controlled total load, comparing independent load measurements with channel responses. The indenter’s actual contact area, load spreading, sensor cross-coupling and repeatability must be measured before inferring a spatial response. Additional smaller-aperture measurements or justified mechanical constraints can improve identifiability; a smoother image alone supplies neither.
Stretch factor 2026-10-14
For an injective function that does not increase distances, one stretch-factor convention measures the largest compression that must be undone:If the sharp Lipschitz constant of is one, this equals bi-Lipschitz distortion. Fixing the scale matters: uniform compression has stretch factor two in this convention, but bi-Lipschitz distortion one. In a geometric spanner, the corresponding stretch factor is the largest graph-route distance divided by direct Euclidean distance; the graph-to-Euclidean identification does not increase distances. [ref]
https://people.scs.carleton.ca/~michiel/encyc_geometric_spanners.pdf. Gudmundsson, Narasimhan and Smid, Geometric Spanners, §1: the stretch-factor convention
. Strictly distance-decreasing map 2026-10-31
A strictly distance-decreasing map decreases the distance between every pair of distinct points. Unlike a contraction mapping, it need not have a single Lipschitz constant below one. Two distinct fixed points would contradict strict decrease, so there is at most one. Existence can fail even on a complete metric space: on , has no fixed point, while the mean value theorem and show strict distance decrease.
Uniformly continuous function Created 2026-09-24 Updated 2026-10-18
A uniformly continuous function has one input tolerance valid throughout its domain of a function. For , with , uniform continuity meansChoose the output tolerance first, then a positive valid for every pair of function inputs. It may depend on , the function and its domain of a function, but not on the pair’s location. The absolute values measure input and output distance.[ref]
Basic Analysis: Introduction to Real Analysis, Volume I by Jiří Lebl. §3.4, Definition 3.4.1: a single positive input tolerance for all pairs in the specified real domain
Compare the quantifier order with the epsilon-delta definition of continuity:Both lines end with . In the first, can depend on the point ; in the second, it is chosen before . Thus uniform continuity implies continuity at every point. Each requested gets its own shared tolerance.
For an exact example, take the polynomial function on the closed interval . For every pair of function inputs in that interval, polynomial factorization givesConsequently works everywhere on this domain of a function. With , the common tolerance is . Near the right endpoint, for example, and have input distance and output distance . The bound is sufficient; it does not claim the largest admissible input window at each point.
On the whole real line, the same polynomial function is a continuous function but fails uniform continuity. Take the sequences and , for positive integers . ThenFor any proposed , choose an integer . The inputs are closer than , yet their outputs violate the fixed tolerance . The sequences need not themselves have a finite limit: their mutual distance is what tends to zero. This is the obstruction captured by the sequential criterion for uniform continuity. Restricting the domain of a function to removes these escaping pairs.
A uniformly continuous function need not be a bounded function. The identity function on the real line has , so always works, although its image of a function is the unbounded real line. Uniform continuity controls changes between nearby inputs, not the overall size of outputs. Likewise, an unbounded domain of a function does not by itself prevent uniform continuity.
The square root supplies a different common tolerance on the interval . Assuming , put and . Since ,Interchanging the inputs when necessary provesThus guarantees uniform continuity on the entire domain of a function, including zero. For , any input distance below suffices. The pairs and both have input distance ; their output distances are respectively and . The common guarantee accommodates quite different local sensitivities.
This Hölder continuity bound allows a curved tolerance rule. It is weaker than a Lipschitz bound: at zero the output change divided by the input change isNo finite Lipschitz constant works near zero, even though uniform continuity does. A Lipschitz function does have uniform continuity, choosing when its Lipschitz bound is positive; when it is a constant function.[ref]
Basic Analysis: Introduction to Real Analysis, Volume I by Jiří Lebl. §3.4, Proposition 3.4.8 and Example 3.4.10: Lipschitz continuity implies uniform continuity; the square root on the nonnegative half-line separates the properties
Square-root input changes share an output tolerance even though amplification near zero is unbounded
. At left, the exact output changes for base points stay inside whenever and . The right panel computes the amplification for inputs : it grows without bound as decreases. These are calculated function graphs, with no experimental measurements. Details.For general metric spaces, replace the absolute values by the respective distance functions. The Heine-Cantor theorem gives an existence guarantee: every continuous function on a compact metric space is uniformly continuous. In particular, continuity on a finite closed interval provides a common tolerance, even when no simple formula for it is known.[ref]
Basic Analysis: Introduction to Real Analysis, Volume I by Jiří Lebl. §7.5, Definition 7.5.10 and Theorem 7.5.11: uniform continuity between metric spaces and the compact-domain theorem
For a short proof outline, fix . Continuity gives each point an open ball of radius within which outputs differ from by less than . The open balls of half these radii still cover the domain of a function. Compactness supplies a finite subcover. Take to be the smallest half-radius in this finite set. Two inputs closer than lie together in one of the original open balls, so the triangle inequality bounds their output distance by . The beautiful step is that the compact set turns local choices into finitely many choices, whose minimum stays positive.
A shared tolerance also makes uniformly continuous functions preserve Cauchy sequences: after choosing for , all sufficiently late input pairs are closer than , so all corresponding output pairs are closer than . Unlike a convergent sequence argument, this does not require an input limit inside the domain of a function. It is why uniform continuity is useful for extending a function toward missing endpoints and for the decay argument below.


