Past exam of the mathematics course of the University of Cambridge / 2003 / iii / Paper 43 (Time Series and Monte Carlo Inference)

Assume the target probability density function is supported where the proposal probability density function is positive and that the envelope constant is finite. Any $C\ge M$ satisfies $f(x)\le Cg(x)$, and integration gives $C\ge 1$. For each attempt draw independently $Y\sim g$ and $U\sim U(0,1)$. Accept $Y$ if $U\le f(Y)/(Cg(Y))$; otherwise discard it and repeat. Independent attempts give independent accepted values. A finite pre-generated list of independent proposal draws can be screened with independent uniforms in the same way, though the number retained is random. If $M=\infty$, this fixed-envelope rejection sampling method cannot be used with that proposal.

For any measurable set $A$, integrating the joint acceptance event in rejection sampling givesSetting $A$ equal to the whole support yields $P(\text{accept})=1/C$. ThusTo verify repeated attempts directly, the probability that the first accepted observation falls in $A$ isWith finite $C$, eventual acceptance has probability one. Applying the same argument after each success proves that the retained sequence consists of independent draws from the target.

The acceptance probability is $1/C$, soFor the given proposal probability density function $g$, every valid envelope must have $C\ge \sup f/g=M$. Rejection increases with $C$, so the optimal envelope is $C=M$, giving rejection probability $1-1/M$. This optimizes the constant for the fixed proposal; it does not assert that $g$ is the best possible proposal. For densities specified only almost everywhere, an essential supremum is sufficient.

Write $Z={\int }_a^{\infty }{x}^{\alpha -1}{e}^{-\beta x}dx$. A proper target requires $\beta >0$, and a proper proposal requires $b>0$. Normalization gives $f(x)=x^{\alpha -1}{e}^{-\beta x}/Z$ and the Pareto distribution proposal $g(x)=b{a}^b{x}^{-b-1}$ on $x>a$. Their ratio isThe derivative of its logarithm is $(\alpha +b)/x-\beta$. It is positive before $x_{\ast }=(\alpha +b)/\beta$ and negative after it. Since $a<\alpha /\beta <x_{\ast }$, the maximum occurs within the proposal support. Consequently the tight envelope for truncated gamma rejection sampling with a Pareto envelope isThe accompanying acceptance rule can be evaluated without $Z$:

Integrating the Pareto distribution proposal yields $G(x)=1-(a/x{)}^b$ for $x\ge a$. For an independent uniform random variable $U$, solve $G(Y)=U$ to obtain the inverse transform sampling ruleEquivalently $Y=a{U}^{-1/b}$ because $1-U$ is also uniformly distributed. In implementation choose $U$ strictly between zero and one so that the inverse is finite.

Every independent proposal/uniform pair succeeds with probability $p=1/M$. The number $K$ retained from a fixed budget of 100 attempts therefore has a binomial distribution, rather than a fixed value:Conditioned on the accept/reject indicators, each retained value still has the target probability density function; the random output count is a separate issue from the correctness of those retained draws.

The empirical distribution gives each observed value mass $1/n$, so its empirical distribution function isFor each of $B$ independent replications, draw $n$ values with replacement from this empirical distribution, recompute the same estimator ${\hat{\theta }}^{\ast (b)}$, and use their sample standard deviation:This estimates the conditional bootstrap standard error; dividing it by $\sqrt{B}$ would instead estimate simulation uncertainty in the bootstrap mean and is not the requested sampling standard error.

When the original observations are distinct, the first resampled index may be arbitrary. After $j$ distinct indices have been drawn, the next index avoids them with probability $1-j/n$. Multiplying conditional probabilities givesHere $P_{\ast }$ denotes probability conditional on the data.

For $0\le t\le \theta$, independence and the uniform distribution give $P(X_{(n)}\le t)=(t/\theta {)}^n$. Consequently, for fixed $x\ge 0$ and $n>x$,For $x<0$ the survival probability is one. Thus the scaled endpoint gap converges in distribution to a unit-rate exponential law; it is not exactly exponential at finite $n$.

The plug-in estimator principle replaces the unknown sampling distribution by its empirical distribution and evaluates its population functional at that fitted distribution. Here the fitted support endpoint is $\hat{\theta }=X_{(n)}$, also the maximum-likelihood estimate: the likelihood is ${\theta }^{-n}{1}_{\{\theta \ge X_{(n)}\}}$ and decreases over its allowed support. For an empirical bootstrap sample $Y_1,\dots ,Y_n$, the corresponding gap isAlmost surely the original maximum is unique. The bootstrap gap equals zero precisely when this observation is selected at least once, soThe true limiting exponential distribution has no atom at zero. This nonvanishing discrepancy proves bootstrap failure for a uniform endpoint: the bootstrap distribution cannot converge to the correct limit, even though the endpoint estimate itself is consistent. It fails to recreate the unsampled population tail beyond the observed maximum. A parametric bootstrap drawing from $U[0,X_{(n)}]$, rather than from the discrete empirical sample, reproduces the scaled-gap law here.

Write $v_i={\sigma }_i^2$ for each component variance and ${\tau }^2$ for the fixed prior variance denoted ${\sigma }^2$ in the source; these must not be confused. Let $\phi (x;\mu ,v)$ be the normal distribution probability density function and $D_j={\sum }_{i=1}^k{\omega }_i\phi (x_j;{\mu }_i,v_i)$. Independent observations and independent parameter priors give the posterior distributionIts support has $v_i>0$, ${\omega }_i>0$ and ${\sum }_i{\omega }_i=1$, with $\alpha ,\beta ,{\tau }^2,{ϵ}_i>0$. The component means have independent fixed-variance priors with a normal distribution, not means with variances proportional to $v_i$. The factor $e^{-\beta /v_i}$ follows from the stated inverse-gamma distribution. The later printed augmented expression instead has $e^{-\beta v_i}$ and an undefined prior mean; those are inconsistent with this specification. The intended model uses $e^{-\beta /v_i}$ and prior mean zero throughout.

Hold all parameters except ${\mu }_i$ fixed and propose ${\mu }_i^{\prime }={\mu }_i+\eta$ with $\eta \sim N(0,s_{\mu i}^2)$. This random-walk Metropolis algorithm has a symmetric proposal distribution. DefineCancellation of unchanged likelihood and prior factors gives the Metropolis–Hastings acceptance probabilityDraw an independent uniform and accept if it is at most $a_i$; otherwise retain the old value. Update each component in turn, recomputing the needed mixture contributions. On a logarithmic scale, compare $\log U$ with $\min (0,{\sum }_j\log (D_j^{\prime }/D_j)-[({\mu }_i^{\prime }{)}^2-{\mu }_i^2]/(2{\tau }^2))$ to avoid numerical underflow.

A valid symmetric proposal distribution is $v_i^{\prime }=v_i+\eta$ with $\eta \sim N(0,s_{vi}^2)$ on the entire real line. Reject immediately if $v_i^{\prime }\le 0$. Otherwise use the positive-parameter random walk with boundary rejection. Withthe Metropolis–Hastings acceptance probability isNo Jacobian determinant is needed for this additive proposal in $v_i$. Do not repeatedly redraw until $v_i^{\prime }>0$: that would create a state-dependent truncated proposal and require an additional Hastings correction. Alternatively a symmetric walk in $u_i=\log v_i$ respects positivity, but then the transformed target has a factor $v_i$; equivalently the acceptance ratio in variance coordinates has the extra factor $v_i^{\prime }/v_i$.

Choose a fixed Dirichlet distribution proposal $q(\mathbf{\omega })\propto {\prod }_i{\omega }_i^{{\eta }_i-1}$ with every ${\eta }_i>0$, independent of the current state, and propose the entire weight vector. This automatically respects the simplex constraint. With $D_j^{\prime }={\sum }_i{\omega }_i^{\prime }\phi (x_j;{\mu }_i,v_i)$, the Independence Metropolis–Hastings algorithm accepts withThis includes $q(\mathbf{\omega })/q({\mathbf{\omega }}^{\prime })$, not merely the posterior ratio. A particularly simple choice is ${\eta }_i={ϵ}_i$, the weight prior: its factors cancel and the ratio is just the mixture likelihood ratio. A proposal concentrated around a pilot estimate can mix better, provided it retains positive density throughout the relevant simplex. Both target and proposal densities use the same $k-1$ independent simplex coordinates, so this direct weight proposal has no extra transformation Jacobian determinant.

Use the data augmentation labels to separate the mixture into groups. Define $n_i={\sum }_j{1}_{\{z_j=i\}}$, $s_i={\sum }_{z_j=i}{x}_j$ and $Q_i({\mu }_i)={\sum }_{z_j=i}(x_j-{\mu }_i{)}^2$. The intended joint posterior distribution is proportional toSumming over all allocation labels recovers the marginal mixture posterior. The Gaussian mixture Gibbs updates with independent priors follow by collecting the terms in each parameter.

For ${\mu }_i$, the conditional exponent isCompleting the square givesFor $v_i$, the conditional density is proportional to $v_i^{-(\alpha +n_i/2)-1}\exp [-(\beta +Q_i({\mu }_i)/2)/v_i]$, henceThe independent prior on ${\mu }_i$ adds neither a $v_i^{-1/2}$ factor nor a ${\mu }_i^2/v_i$ penalty. Finally, combining the allocation counts with the Dirichlet distribution prior givesA Gibbs sampler successively draws from these conditionals, using the most recently updated values. An empty component has $n_i=s_i=Q_i=0$, so its mean and variance conditionals reduce to their proper priors without dividing by an undefined group sample mean.

Conditioned on the parameters, the data augmentation labels are independent. By Bayes' theorem, the conditional probability of component $i$ for observation $j$ isThis is the mixture responsibility. Draw an independent uniform random variable and select $i$ using the cumulative sums of $(r_{j1},\dots ,r_{jk})$. Refresh every label before recomputing the component sufficient statistics for the next Gibbs sampler sweep. Numerically, form the log scores $\log {\omega }_i-\frac{1}{2}\log v_i-(x_j-{\mu }_i{)}^2/(2v_i)$, subtract their maximum and exponentiate before normalization.

The marginal Metropolis–Hastings algorithm avoids storing allocation labels, but each component update changes every mixture likelihood term. Cache component contributions, calculate log-likelihoods stably and tune the proposal scale and random-walk Metropolis efficiency during a pilot run; then fix proposal scales or use a justified adaptive scheme. Means and variances may be strongly dependent, making separate random walks inefficient. Weight proposals must respect the simplex, variance proposals must respect positivity, and the displayed acceptance ratios must retain all relevant priors and proposal corrections.

The augmented Gibbs sampler uses elementary conditionals and typically costs order $nk$ for a label sweep, followed by sufficient-statistic updates. It avoids Metropolis rejections in these conditional draws, but does not remove slow mixing: ambiguous allocations can lock means, variances and labels into strongly coupled configurations. Empty components must be updated from their priors, not deleted automatically. The proper inverse-gamma scale $\beta >0$ suppresses arbitrarily small variances even though an unregularized Gaussian-mixture likelihood can be singular.

With exchangeable component priors, label switching creates $k!$ equivalent posterior modes. Ordering component means is an identification constraint, not an innocuous change unless the intended inferential model is made explicit. Predictive density and permutation-invariant summaries avoid arbitrary labelling; component-specific summaries require an appropriate relabelling convention. If the Dirichlet parameters differ between components, exact label symmetry may be broken, but substantial multimodality can remain.

Use dispersed initial states, multiple chains and diagnostics on several scientifically relevant functions. Report effective sample size of a Markov chain and Monte Carlo error estimates, not just a large iteration count. Burn-in can reduce initialization bias but does not prove convergence; thinning cannot by itself cure poor exploration. Closed-form Gibbs updates simplify computation, while reliable inference still requires adequate exploration of the posterior.

For minimization of an energy $E(\vartheta )$, simulated annealing at temperature $t>0$ targets the Boltzmann distribution ${\pi }_t(\vartheta )\propto e^{-E(\vartheta )/t}$ relative to a specified parameter reference measure. With proposal distribution $q({\vartheta }^{\prime }\mid \vartheta )$, accept withAt each temperature run the chain long enough to explore that law, then cool and retain the best state seen. For symmetric proposals every improvement is accepted, while a worsening move is accepted with probability $e^{-\Delta E/t}$. Equilibrium laws concentrate near global minima when their normalizing integrals are finite and the minima are suitably isolated. A particular rapid cooling schedule does not automatically inherit this guarantee.

Likelihood maximization uses energy $E=-\ell$, not $E=\ell$. The source's later wording “$f(p)$ equal to the log-likelihood” conflicts with its preceding minimization convention. The intended maximum-likelihood Boltzmann distribution is proportional to $e^{\ell /t}=L^{1/t}$. With the opposite sign, likelihood maxima would be disfavored; for binomial data with both successes and failures the density often would not even be integrable near the endpoints as $t$ decreases.

For the maximum-likelihood estimates, write $s={\sum }_{j=1}^m{x}_j$, $\bar{x}=s/m$ and $S={\sum }_j(x_j-\bar{x}{)}^2$. In the binomial distribution model with known positive integer $n$, the log-likelihood isDifferentiating gives $s/p-(mn-s)/(1-p)=0$, hence $\hat{p}=s/(mn)$. The second derivative is negative in the interior. If $s=0$ or $s=mn$, the maximum is at the corresponding endpoint. This model requires integer observations in $\{0,\dots ,n\}$; otherwise its likelihood is zero.

For the normal distribution model write $v={\sigma }^2$. Its log-likelihood isFor fixed $v$, the unique maximizing mean is $\bar{x}$. Substituting it and differentiating in $v$ gives $v=S/m$, so, assuming $S>0$,If $S=0$, the normal density likelihood grows without bound as $v\downarrow 0$ with $\mu =\bar{x}$; there is no finite interior maximum or ordinary finite AIC for that fit.

For Gaussian likelihood Gibbs annealing, use $L_N^{1/t}$ relative to $d\mu dv$. Completing the square and identifying an inverse-gamma distribution gives the Gibbs sampler conditionalsIntegrating out $\mu$ contributes a factor proportional to $v^{1/2}$, so the marginal is $v\sim \mathrm{InvGamma}(m/(2t)-3/2,S/(2t))$. The joint target is proper for $0<t<m/3$ when $S>0$; starting the Lebesgue-based annealing above this range would require a proper reference factor or bounded parameter domain. For sufficiently small $t$, its moments satisfyThus equilibrium draws concentrate at $(\bar{x},S/m)$ as $t\downarrow 0$.

There is also a direct convergence proof for the actual successive Gibbs sampler updates here. Choose deterministic $0<t_r\le m/10$ with $t_r\to 0$, start at a finite deterministic $v_0>0$, draw ${\mu }_r$ from the first conditional using $v_{r-1}$, then draw $v_r$ from the second using ${\mu }_r$. Conditional normal moments give $E[({\mu }_r-\bar{x}{)}^2\mid v_{r-1}]=t_r{v}_{r-1}/m$ and $E[({\mu }_r-\bar{x}{)}^4\mid v_{r-1}]=3t_r^2{v}_{r-1}^2/m^2$. The permitted inverse-gamma moments therefore yieldThe coefficient on the preceding first moment is at most $1/6$, and that on the preceding second moment is at most $1/8$. The remaining coefficients are uniformly bounded, so both moment sequences are bounded by iteration. Since $t_r\to 0$, the recursions then give $E[v_r]\to S/m$ and $E[v_r^2]\to (S/m{)}^2$. Also $E[({\mu }_r-\bar{x}{)}^2]=t_{r}E[v_{r-1}]/m\to 0$. HenceThis proves convergence in mean square, and thus in probability, to the normal maximum-likelihood estimates for this explicit cooling/update scheme, rather than merely assuming equilibration.

For binomial likelihood-power annealing, normalizing $p^{s/t}(1-p{)}^{(mn-s)/t}$ on $(0,1)$ givesIts mean is $(s+t)/(mn+2t)$, and its variance isThe mean tends to $s/(mn)$ and the variance tends to zero, proving concentration at the binomial maximum-likelihood estimate, including the endpoint cases. Independent exact draws from these beta laws are a simple annealing implementation within this model.

For model selection, the Akaike information criterion is $2d-2\hat{\ell }$ with $d_B=1$ and $d_N=2$. ThusUse the endpoint convention $0\log 0=0$. For a trans-dimensional annealing for penalized likelihood construction, target the disjoint model spaces with unnormalized densitiesrelative to $dp$ and $d\mu dv$, respectively. The energies are half the unoptimized AIC, $d-\ell$; their joint minimizer is the fitted model with smaller AIC. Keep constants such as $C_B$ and the normal density normalization, which do not cancel across models. The within-model conditional draws are unchanged by these constant penalties.

Here is a complete dimension-changing proposal. Select a binomial-to-normal move with probability $b>0$ and a normal-to-binomial move with probability $d>0$, both fixed; other steps can use the within-model updates. From $B,p$, draw $u\sim N(0,1)$ with density $h(u)=(2\pi {)}^{-1/2}{e}^{-u^2/2}$ and setThis bijection from $(p,u)\in (0,1)\times \mathbb{R}$ onto $({\mu }^{\prime },v^{\prime })\in \mathbb{R}\times (0,\infty )$ has inverse $p=(1+e^{-{\mu }^{\prime }}{)}^{-1}$, $u=\log v^{\prime }$. Its absolute Jacobian determinant is $v^{\prime }/[p(1-p)]$. The Binomial-normal reversible-jump annealing acceptance ratio is thereforeFor a reverse proposal from $N,\mu ,v$, compute $p^{\prime }=(1+e^{-\mu }{)}^{-1}$ and $u^{\prime }=\log v$ deterministically and accept withThese ratios are reciprocal at inverse states and enforce detailed balance on the dimension-matched augmented spaces. They specify the proposal density, model-selection probabilities and Jacobian explicitly. The map is a valid example, not a claim of optimal mixing. Positive-temperature beta draws stay in the open interval even when the limiting estimate is an endpoint. At each temperature ensure adequate model switching; retain the best fitted penalized likelihood encountered. If minima tie, limiting concentration need not choose a unique model.

There is a material measurement qualification. A binomial probability mass and an unrounded normal density use different observation reference measures. Their raw AIC values are not an invariant, scientifically calibrated comparison of the same recorded data. The expressions and jump algorithm above implement the formal density comparison suggested by the question, with its units fixed. For integer-recorded measurements, a compatible normal model instead assigns probabilities $P_{\mu ,v}(x-1/2<Z\le x+1/2)$, suitably adjusting boundaries or truncation if required. Then both models describe the same recorded outcomes relative to counting measure. Use those masses in the normal likelihood and the same reversible-jump formula; its simple normal MLE and conjugate Gibbs updates will generally no longer apply. This observation-model correction cannot be silently combined with the continuous-normal formulas.

Write $C_j={\sum }_{t=1}^T{y}_t\cos ({\omega }_{j}t)$ and $S_j={\sum }_{t=1}^T{y}_t\sin ({\omega }_{j}t)$. The discrete Fourier transform coefficient is $C_j-i{S}_j$. Alternatively, by the given orthogonality relations, the harmonic obtained by linear least-squares projection has coefficients $a_j=2C_j/T$ and $b_j=2S_j/T$. Thus the source's periodogram is the squared amplitude of a fitted harmonic, scaled as power per unit angular frequency:It measures how much of the centered record varies at frequency ${\omega }_j$. For odd $T$, Fourier Parseval identity givesThe positive-frequency spacing is $2\pi /T$, so summing the power ordinates times this spacing estimates the total variance. The factor $1/(\pi T)$ corresponds to the one-sided spectral density on $(0,\pi )$; the usual two-sided periodogram is half as large.

If the observations are Gaussian white noise with common variance ${\sigma }^2$, the $2m$ harmonic sums are jointly normally distributed. Their means are zero even with a nonzero constant series mean, because each sine and cosine sums to zero. Orthogonality givesUncorrelated coordinates of a multivariate normal distribution are independent. Therefore all $C_j/\sqrt{T{\sigma }^2/2}$ and $S_j/\sqrt{T{\sigma }^2/2}$ are independent standard normals, andSince the chi-squared distribution with two degrees of freedom has mean two and variance four,Here $f_{+}$ is the one-sided white-noise spectral density. Its exact unbiasedness is accompanied by a nonconcentrating distribution: along Fourier frequencies tending to any interior frequency, $I/f_{+}$ has the same nondegenerate exponential law at every $T$. It therefore does not converge in probability to one, proving failure of statistical consistency, not just nonvanishing variance.

For a general stationary process, exact finite-sample unbiasedness is not implied. Expanding the Fourier square instead giveswhere $\gamma$ is the autocovariance function. Under suitable short-memory regularity this tends to $f_{+}(\omega )$ at frequencies ${\omega }_j\to \omega$. The periodogram is thus generally asymptotically unbiased, while the exact assertion above is for white noise.

Use local periodogram smoothing to reduce variance: average $L_T$ neighboring ordinates with weights summing to one, where $L_T\to \infty$ but $L_T/T\to 0$. At an interior frequency the shrinking frequency window makes the bias tend to zero for smooth $f_{+}$, while averaging many weakly dependent ordinates makes the variance of order $1/L_T$. For white noise with equal weights, the ordinates are exactly independent and the variance is exactly $f_{+}^2/L_T$. For a general process, impose suitable short-memory and fourth-order dependence assumptions, such as a stationary Gaussian model with sufficiently summable autocovariances. These conditions, together with smoothness, give a consistent smoothed estimator. Boundary frequencies require an appropriately one-sided or reflected window.

Smooth spectral density alone is not enough. For example let $y_t=A{ϵ}_t$, where $A>0$ is a nondegenerate random variable constant throughout the record, independent of independent unit-variance Gaussian ${ϵ}_t$, with $E[A^2]<\infty$. The unconditional autocovariance is zero off lag zero and its one-sided spectral density is the smooth constant $E[A^2]/\pi$. Nevertheless a long local average estimates the realized $A^2/\pi$, not that deterministic unconditional value. This nonergodic example explains why a dependence/ergodicity condition accompanies the smoothing argument.

For the turning point test of randomness, the null is independent identically distributed continuous observations. Let $J_t$ be one if $y_t$ is a strict local maximum or minimum among $y_{t-1},y_t,y_{t+1}$, for $2\le t\le T-1$, and set $P_T={\sum }_t{J}_t$. Three ranks are equally likely; the middle observation is an extremum in four of the six orders, giving $E[J_t]=2/3$ and $\mathrm{Var}(J_t)=2/9$. ThereforeIndicators more than two positions apart depend on disjoint observations and are independent. For neighboring indicators, both are one precisely when four observations alternate up-down-up or down-up-down. By the probability-integral transform, use four independent uniforms. The first pattern has probabilityhere $b=y_t$, $c=y_{t+1}<b$, with the two outer inequalities having probabilities $b$ and $1-c$. The reverse pattern has the same probability, so $E[J_t{J}_{t+1}]=5/12$ and $\mathrm{Cov}(J_t,J_{t+1})=5/12-4/9=-1/36$.

At separation two, the triples share only one endpoint value $c$. Conditional on it, their other observations are independent. The probability that either triple's central observation is a turning point isIt follows that $E[J_t{J}_{t+2}]={\int }_0^{1}r(c{)}^{2}dc=9/20$, so this covariance is $9/20-4/9=1/180$. Combining all terms for $T\ge 4$ givesThe standardized count is approximately standard normal for large $T$, by a central limit theorem for bounded 2-dependent indicators. A two-sided randomness test rejects for large absolute values ofToo few turns suggests persistent movement, while too many suggests alternation. Exact rank-permutation calibration is possible at small $T$. Ties invalidate the continuous-rank probabilities unless their handling is specified. The test probes this feature of randomness; nonrejection does not establish independence.

For the usual causal, stable autoregressive moving-average model with zero-mean white noise, sufficient conditions for a stationary invertible representation areThe roots of $1-\varphi z$ and $1+\theta z$ then lie outside the closed unit disc. For a minimal representation these are the usual causality and invertibility root criteria for an ARMA model. A genuinely minimal ARMA(1,1) additionally has nonzero coefficients and no cancellation $\varphi +\theta =0$; if a coefficient vanishes the order is lower. Common factors can reduce the process to white noise, so inequalities on an unreduced representation should not be asserted as necessary for every reduced process. A noncausal stationary solution of an unstable autoregressive equation is a different representation, and does not give the requested innovation-based Wold coefficients below.

Using the backshift operator $B$, write $(1-\varphi B)X_t=(1+\theta B){ϵ}_t$. The absolutely summable geometric series givesThus the Wold representation isIts coefficients are square-summable, so the series converges in mean square and defines a weakly stationary process. Invertibility makes the closed spans of past observations and past innovations equal. Since the current driving white noise is orthogonal to all past innovations, it is orthogonal to all past observations and is the linear innovation of this causal invertible process.

The spectral density of an ARMA process follows from the squared frequency response of the linear filter of a stationary time series:This displayed density uses the usual two-sided convention. For the one-sided spectral density used in Question 5, multiply it by two and restrict to $0\le \omega \le \pi$.

Let ${\psi }_0=1$ and ${\psi }_j=(\varphi +\theta ){\varphi }^{j-1}$ for $j\ge 1$. Orthogonality of the driving white noise gives $\gamma (h)={\sigma }^2{\sum }_{j\ge 0}{\psi }_j{\psi }_{j+h}$ for $h\ge 0$. HenceFor $h\ge 1$, separate the $j=0$ term and sum the remaining geometric series:Negative lags satisfy $\gamma (-h)=\gamma (h)$. With $\varphi =0$, interpret the lag-one factor as one and later powers as zero, recovering the MA(1) covariance.