 Solution

The supporting inequality in part 2(a) bounds $f$ below by an affine function plus a positive squared Euclidean norm, making it a coercive function. Its continuity gives a global minimizer $x^{\ast }$, which is unique by strict convexity. Let $L_{\max }={\max }_i{L}_i$, $\ell =n/m$ and ${\Delta }_0=f(x^0)-f(x^{\ast })$. Here $m$ is the block size, so its role differs from the number of component functions in Question 2. Use randomized block coordinate descent with uniformly sampled blocks of size $m$, independently at each update. Sampling uniformly from a fixed partition into $\ell$ equal-sized blocks gives the same calculation, since each coordinate has inclusion probability $1/\ell =m/n$.

Let $\beta$ be a common upper bound on the block Lipschitz constants. Part (c) permits $\beta =m{L}_{\max }$. With step size $1/\beta$, part (a) givesSumming squared partial derivatives according to their inclusion probabilities,For a strongly convex function, part 2(a) also yields the strong convexity gradient inequalityIndeed, substitute $y=x^{\ast }$ into the supporting inequality and complete the square in $x^{\ast }-x$: the expression $\nabla f(x{)}^{T}d+\gamma \Vert d{\Vert }^2/2$ is at least $-\Vert \nabla f(x){\Vert }^2/(2\gamma )$ for every $d$.

Combining the last three estimates gives a contraction in conditional expectation:By the tower property and repeated application,An optimization epoch counts $n$ coordinate partial derivative evaluations: it comprises $\ell$ block updates or one full gradient update. With sampling with replacement, an optimization epoch is this work measure; it need not visit every coordinate. Consequently the block method needsto bring the expected objective function gap below $\epsilon$, for $0<\epsilon <{\Delta }_0$, up to integer rounding.

For the steepest descent method, use a global Lipschitz gradient constant $L_f$. Part (c) with the full block permits $L_f={\sum }_i{L}_i\le n{L}_{\max }$. The descent lemma with step size $1/L_f$, together with the strong convexity gradient inequality, givesAn exact line search along the negative gradient does at least as well in objective function decrease per update. Ignoring its additional work, the resulting epoch complexity isThe common coordinate-based guarantees favour blocks by a factor $n/m=\ell$ in epochs. In fact, with the conservative constants $\beta =m{L}_{\max }$ and $L_f=n{L}_{\max }$, both methods have the same guaranteed contraction per update, while a block update uses only $m$ coordinate partial derivatives.

This compares the bounds justified by the given constants. For a particular objective function, sharper constants give the comparison $\beta /\gamma$ against $L_f/\gamma$; an actual speedup of $\ell$ is not universal. Computing a block coordinate gradient also need not cost exactly $m/n$ of a full gradient evaluation. The epoch complexity comparison assumes that coordinate partial derivative counts are a useful measure of work, and does not itself establish a wall-clock comparison.