Solution (source code)

= Solution
{c}

We use the <spectral theorem> to construct a sufficiently accurate approximate <eigenpair>. Complete $x$ to an <orthonormal eigenbasis> $x,x_2,\ldots,x_n$ of the <symmetric matrix> $B$, with corresponding <eigenvalues> $\lambda,\lambda_2,\ldots,\lambda_n$. Simplicity of $\lambda$ means that every $\lambda_j-\lambda$ below is nonzero. Put
$$
a=x^TEx,\qquad
v=-\sum_{j=2}^n\frac{x_j^TEx}{\lambda_j-\lambda}x_j.
$$
The <orthogonal decomposition> of $Ex$ shows that $x^Tv=0$ and
$$
(B-\lambda I)v=-(E-aI)x.
$$
With $u_\varepsilon=x+\varepsilon v$ and $\mu_\varepsilon=\lambda+\varepsilon a$, direct expansion cancels every first-order term:
$$
\begin{aligned}
(B+\varepsilon E)u_\varepsilon-\mu_\varepsilon u_\varepsilon
&=\varepsilon\big((B-\lambda I)v+(E-aI)x\big)
+\varepsilon^2(E-aI)v\\
&=\varepsilon^2(E-aI)v.
\end{aligned}
$$
Also $\|u_\varepsilon\|_2^2=1+\varepsilon^2\|v\|_2^2\geq1$. Thus its <eigenpair residual> has <Euclidean norm> $O(\varepsilon^2)$ in <Big O notation>.

For completeness, a real <symmetric matrix> $M$ always has an <eigenvalue> within $\|(M-\mu I)u\|_2/\|u\|_2$ of any real $\mu$ and nonzero $u$. Indeed, writing $u=\sum_jc_jq_j$ in an <orthonormal eigenbasis> with <eigenvalues> $\theta_j$ gives
$$
\|(M-\mu I)u\|_2^2
=\sum_j(\theta_j-\mu)^2c_j^2
\geq\left(\min_j|\theta_j-\mu|\right)^2\|u\|_2^2.
$$
Apply this <symmetric eigenvalue residual bound> to $M=B+\varepsilon E$. There is an <eigenvalue> $\lambda_\varepsilon$ such that
$$
|\lambda_\varepsilon-(\lambda+\varepsilon a)|
\leq\varepsilon^2\|(E-aI)v\|_2,
$$
which proves
$$
\boxed{\lambda_\varepsilon=\lambda+\varepsilon(x^TEx)+O(\varepsilon^2).}
$$
This establishes the <first-order perturbation of a simple symmetric eigenvalue> without presupposing differentiability of the perturbed <eigenvector>. If $n=1$, the sum defining $v$ is empty and the displayed <eigenvalue> formula is exact.