= Solution
Here a dense $100000$-by-$100000$ <matrix> has $10^{10}$ entries, about $80$ GB even with eight-byte storage per entry. Use <limited-memory BFGS> on the negative of the objective, or equivalently its positive-definite approximate-metric form of <gradient ascent>. It stores only a fixed small number $\ell$ of <vector> pairs, costing $O(\ell d)$ memory rather than $O(d^2)$.
Let $G(x)=\sum_{i=1}^{100}e^{e^{-\|x-a_i\|^2}}$ and put $c_i(x)=e^{e^{-\|x-a_i\|^2}}e^{-\|x-a_i\|^2}$. Direct use of the <chain rule> gives
$$
\nabla G(x)=-2\sum_{i=1}^{100}c_i(x)(x-a_i).
$$
If $H_n$ denotes the positive-definite inverse-<Hessian matrix> approximation for $-G$, the iteration is
$$
x_{n+1}=x_n+\alpha_nH_n\nabla G(x_n),
$$
with a <line search> increasing $G$ and the usual safeguarded limited-memory updates. No exact <Hessian matrix> is required. With fixed memory and the usual bounded-metric and <line search> hypotheses, expect \b[local <linear convergence> near a nondegenerate strict maximum]; fixed-memory <limited-memory BFGS> does not generally inherit the superlinear theorem for full-memory <BFGS>.
Unlike the preceding objective, $G$ is not generally a <concave function>. Thus this is a method for finding a <local maximum>; comparison of different converged candidates or a global search is needed to certify a <global maximum> for arbitrary centres. The symbolic problem supplies no assumption that would justify an unconditional global-convergence claim.<Quasi-Newton Methods by NEOS Guide>[Final paragraphs: storing only recent secant <vectors>, linear-in-dimension memory, and the L-BFGS implementation]{ref}
Back to article page
