Solution (source code)

= Solution

Give $\mathcal O(U)$ the <compact-open topology>, that is, uniform convergence on compact subsets. Put $R(z)=(z1-x)^{-1}$. Choose a finite oriented contour $\Gamma$ in $U\setminus\sigma(x)$ with winding number one on $\sigma(x)$ and zero outside $U$; boundaries of a finite union of sufficiently small rectangles around the compact spectrum provide such a contour. Define the <holomorphic functional calculus> by
$$
\boxed{\Theta_x(f)=\frac1{2\pi i}\int_\Gamma f(z)R(z)\,dz.}
$$
The integral is a <Bochner integral> along the piecewise smooth contour. It is independent of the admissible contour by the <Cauchy integral theorem>, since the resolvent is holomorphic off the spectrum. In particular, restricting $f$ to a smaller open spectral neighborhood leaves its value unchanged.

Linearity is immediate, and
$$
\|\Theta_x(f)\|\leq\frac{\operatorname{length}(\Gamma)}{2\pi}\max_{z\in\Gamma}\|R(z)\|\max_{z\in\Gamma}|f(z)|
$$
proves <continuity> for the compact-open topology. For the constant and coordinate functions, move the corresponding resolvent integrals to a large circle. The norm-convergent expansion $R(z)=\sum_{n\geq0}x^nz^{-n-1}$ there gives $\Theta_x(1)=1$ and $\Theta_x(u)=x$.

To check multiplication, take two nested admissible contours, with $z$ on the outer one and $w$ on the inner one. The <resolvent identity> gives
$$
R(z)R(w)=\frac{R(w)-R(z)}{z-w}.
$$
In the double integral for $\Theta_x(f)\Theta_x(g)$, the term with $R(w)$ integrates in $z$ to $f(w)R(w)g(w)$ by the <Cauchy integral formula>. The term with $R(z)$ integrates in $w$ to zero, because $z$ lies outside the inner region. Thus \b[$\Theta_x(fg)=\Theta_x(f)\Theta_x(g)$]. Nested regions can be chosen with closures inside $U$, so the argument also handles disconnected $U$ and contours with several boundary components.

For uniqueness, use the following form of the <Runge theorem>: rational functions whose poles lie outside an open set $U$ are dense in $\mathcal O(U)$ for uniform convergence on compact subsets; polynomials are included, with a pole at infinity allowed. This follows by a Runge exhaustion of $U$; no connectedness of $U$ is required. Any unital <algebra homomorphism> sending $u$ to $x$ must send $(u-a)^{-1}$ to $(x-a1)^{-1}$ for $a\notin U$. Consequently it agrees with the constructed map on every such rational function. <Continuity> and density give uniqueness. This is <continuity and uniqueness of holomorphic functional calculus>.

For the <holomorphic spectral mapping theorem>, first suppose $\mu\notin f(\sigma(x))$. The reciprocal $g=1/(f-\mu)$ is holomorphic on some smaller open neighborhood $V$ of $\sigma(x)$, even if it is not holomorphic on all of $U$. Restriction consistency and multiplicativity on $V$ show that $\Theta_x(g)$ is an inverse of $\Theta_x(f)-\mu1$. Thus $\sigma(\Theta_x(f))\subseteq f(\sigma(x))$.

Conversely, if $\lambda\in\sigma(x)$, then
$$
h(z)=\frac{f(z)-f(\lambda)}{z-\lambda}
$$
is holomorphic on $U$, with the singularity at $\lambda$ removed. Hence
$$
\Theta_x(f)-f(\lambda)1=(x-\lambda1)\Theta_x(h).
$$
In a commutative algebra, an invertible product has invertible factors. The displayed product cannot be invertible because $x-\lambda1$ is not. Therefore
$$
\boxed{\sigma(\Theta_x(f))=f(\sigma(x)).}
$$

For the operator conclusion, work over complex scalars and let $\sigma(T)=K_1\cup K_2$ be a partition into disjoint nonempty <compact sets>. To apply the preceding commutative-algebra result precisely, take the closed unital subalgebra of $\mathcal B(X)$ generated by $T$ and all its resolvents $(zI-T)^{-1}$, $z\notin\sigma_{\mathcal B(X)}(T)$. The generators commute, so this is a <resolvent-generated commutative algebra>. Its spectrum of $T$ is exactly the operator spectrum: every resolvent outside the latter was included, and an inverse inside the subalgebra would also be an inverse in $\mathcal B(X)$.

Choose disjoint open neighborhoods $U_1,U_2$ of $K_1,K_2$, and let $f$ equal one on $U_1$ and zero on $U_2$. The corresponding <Riesz projection> $P=f(T)$ satisfies
$$
\boxed{P^2=P,\qquad PT=TP,\qquad\sigma(P)=\{0,1\}.}
$$
The last spectrum is computed in the chosen commutative algebra and already ensures $P\neq0,I$. Its range $Y=PX=\ker(I-P)$ is closed, nonzero and proper. The commutation identity makes it invariant under $T$. This proves that a <disconnected spectrum yields a nontrivial invariant subspace>. Using all resolvents avoids inadvertently replacing the operator spectrum by the possibly larger spectrum in a polynomial-generated subalgebra.