Past exam of the optimisation for data science course of the University of Oxford 2023 3 c Solution 2026-10-25
For arbitrary , write the Bregman divergencePart (b), with its supporting affine function constructed at this , implies the coordinate derivative bound by Bregman divergenceFor a nonempty block , set and . Sum the coordinate inequalities:Since in part (b) was arbitrary, interchange and to obtain the same left side bounded by . Adding the two inequalities and dividing by two givesNow require that have zero coordinates outside . The last dot product is then . By the Cauchy-Schwarz inequality,If , the desired estimate is immediate; otherwise divide by its Euclidean norm. We have proved the stronger block Lipschitz constant from coordinate smoothness estimateIn particular, take for any real . This proves (6), with a valid choiceThe least admissible Lipschitz constant is therefore no larger than this choice. For the empty block, the block coordinate gradient vanishes identically and its least nonnegative Lipschitz constant is zero. The proof uses only differentiability, convexity and coordinate Lipschitz continuity, with no assumed Hessian matrix.