Away from $x=\pm 1$, the sign of $\dot{x}=x^2-1$ is fixed within each of the three invariant strips, so $x(t)$ is strictly monotone. On $x=\pm 1$, the equation is $\dot{y}=\pm ky$, whose nonconstant solutions are exponential and hence not periodic. Therefore the monotone coordinate excludes periodic orbits, and there are

At an equilibrium point, $x^2-1=0$ and $kxy=0$. If $k\ne 0$, the only equilibria areTheir Jacobian matrix isAt $(1,0)$ its eigenvalues are $2,k$, and at $(-1,0)$ they are $-2,-k$. Hence for $k>0$, $(1,0)$ is an unstable node and $(-1,0)$ is a stable node; for $k<0$, both points are saddles.

If $k=0$, every point on each line $x=1$ and $x=-1$ is an equilibrium. The line $x=1$ is transversely repelling because the nonzero eigenvalue is $2$, while $x=-1$ is transversely attracting because the nonzero eigenvalue is $-2$; the tangential eigenvalue is zero on both nonisolated equilibrium lines.

The vector field is unchanged by reflection in the $x$-axis, $(x,y)\mapsto (x,-y)$; more generally, linearity of the second equation gives the scaling symmetry $(x,y)\mapsto (x,cy)$. It is also reversible under reflection in the $y$-axis:When $k=0$, $y$ is constant, so arbitrary translations $y\mapsto y+c$ are additional symmetries.

The process is adapted to its natural filtration. At each fixed time it has finite support and is therefore integrable. Conditional on $X_{n-1}=0$,Conditional on $X_{n-1}\ne 0$,The Markov property now givesso $(X_n)$ is a martingale.

The pair $(M_n,{\mathcal{F}}_n{)}_{n\ge 0}$ is a martingale if $({\mathcal{F}}_n)$ is a filtration, $(M_n)$ is an adapted process, every $M_n$ is integrable, and

Using the estimator from part (b) and the spectral decomposition of $X^{T}X$,ThusUnder the stated separation of scales, the shrinkage factor is approximately one for $i\le q$ and approximately zero for $i>q$. The principal-component shrinkage by ridge regression formula therefore reduces approximately tothe orthogonal projection of $Y$ onto the span of the $q$ normalized sample principal components with greatest variance.

Since $X=U{V}^T$ and $X^{T}X=V\Lambda V^T$,ThereforePut $w_i=u_i/\sqrt{{\Lambda }_{ii}}$. By part (c), the $w_i$ are normalized sample principal components forming an orthonormal set, andThis is the orthogonal projection onto a finite-dimensional subspace spanned by the $w_i$, so ${\hat{Y}}_{\mathrm{MLE}}$ is the unique closest point in that subspace to $Y$.

Because every column of $X$ has sample mean zero, every column $u_i$ of $U=XV$ also has sample mean zero. The spectral theorem for real symmetric matrices givesHence for $i\ne j$ the sample covariance is $n^{-1}{u}_i^T{u}_j=0$, while the sample variance of $u_i$ isThus the $u_i$ are pairwise uncorrelated sample principal components, ordered by decreasing sample variance.

Differentiating the ridge regression objective givesFor every nonzero $z$,so $X^{T}X+\lambda I_p$ is positive definite and hence invertible. The strictly convex objective therefore has the unique closed-form ridge regression estimator

The Gaussian likelihood is proportional toso maximum likelihood estimation is equivalent to ordinary least squares. If $p\le n$ and $X$ has rank $p$, then $X^{T}X$ is invertible and the normal equations giveIf $p>n$, then $\mathrm{rank}X<p$ and $\ker X\ne \{0\}$. A least-squares minimizer always exists, but it is not unique: by rank-deficient ordinary least squares, the complete set isan affine subspace of dimension $p-\mathrm{rank}X>0$. Thus there are uncountably infinitely many likelihood maximizers. In the generic full-row-rank case their dimension is $p-n$.

Let $\nu$ be an invariant distribution for the continuous-time Markov chain. Using $Q=D_g(P-I)$, its stationarity equation $\nu Q=0$ becomesThus the measure with weights ${\nu }_{i}g(i)$ is invariant for the irreducible positive-recurrent jump chain. Its invariant distribution $\pi$ is unique, so ${\nu }_{i}g(i)=c{\pi }_i$. Normalization gives the invariant-measure transfer between a jump chain and a CTMC formulaThe denominator is finite and nonzero because $\epsilon <g(j)<1/\epsilon$. Thus $\nu$ is a probability distribution and $(X_t)$ is positive recurrent.

A continuous-time Markov chain is irreducible if every state can reach every other state with positive probability. A state $i$ is recurrent if, after leaving $i$, the chain returns to $i$ with probability one; an irreducible chain is recurrent if one, and hence every, state is recurrent. A recurrent state is positive recurrent if its expected continuous first return time is finite, and an irreducible chain is positive recurrent when all its states are.

Conditional on $Y_m=i$, the holding time $V_m=U_m/g(i)$ has an exponential distribution of rate $g(i)$, and after that holding time the next state is $j$ with transition probability $p_{ij}$. The memoryless property of exponential random variables and the Markov property of $(Y_m)$ therefore make $(X_t)$ a continuous-time Markov chain.

Moreover $g(i)<1/\epsilon$, so $V_m\ge \epsilon U_m$. The strong law of large numbers gives ${\sum }_m{U}_m=\infty$ almost surely, hence $J_m\to \infty$ and the process is nonexplosive. Its Q-matrix isEquivalently, if $D_g$ is the diagonal matrix with entries $g(i)$, then