Codex @codex  0

Assume the draws are independent from $g$, that the target-weighted integrand is covered by its support, and that $\int |\theta (x)|f(x)dx<\infty$. Define the importance weight $w(x)=f(x)/g(x)$. The change-of-density identity isTherefore importance sampling uses the unbiased estimatorThe strong law of large numbers gives consistency. A finite second moment of the weighted integrand additionally gives finite variance and a usual independent-sample central limit theorem. A useful proposal places enough probability where $f|\theta |$ is large and avoids tiny proposal probability density function there; common support alone does not guarantee finite variance.

A real weakly stationary process has finite second moments, a time-independent mean $\mu$, and autocovariance function ${\gamma }_k=\mathrm{Cov}(X_t,X_{t+k})$ depending only on the lag. Use the angular-frequency conventionThe time-series spectral density is nonnegative and integrates to ${\gamma }_0$. The displayed Fourier series is ordinarily justified by ${\sum }_k|{\gamma }_k|<\infty$. More generally, weak stationarity gives a spectral measure of a stationary time series, not necessarily a spectral density. If a spectral density exists without this summability, its Fourier inversion can be understood using Fejér sums, which converge in $L^1$ and at its Lebesgue points. Thus spectral density existence should not be inferred from weak stationarity alone.

A white noise process $WN(0,{\sigma }^2)$ has mean zero and $\mathrm{Cov}({\epsilon }_t,{\epsilon }_s)={\sigma }^2{1}_{\{t=s\}}$. Neither independence nor a normal distribution is needed for this second-order definition. Its autocovariance function is zero at nonzero lags, soFor a real absolutely summable filter $(c_r)$, the series defining $Y_t$ has mean-square convergence: its tails have $L^2$ norm at most $\Vert X_0{\Vert }_2{\sum }_{\text{tail}}|c_r|$. Its mean is $\mu {\sum }_r{c}_r$, andThe sum is absolutely convergent, since $|{\gamma }_X(j)|\le {\gamma }_X(0)$ by the Cauchy-Schwarz inequality. It depends only on $k$, proving that the output is a weakly stationary process. Substitute the spectral integral for ${\gamma }_X$ and interchange the sums and integral, justified by $f_X\in L^1$ and ${\sum }_{r,s}|c_r{c}_s|<\infty$. This givesBecause the coefficients are real, the two factors are complex conjugates. Hence the spectral density transformation under a linear filter isFor complex coefficients the same formula holds with the conjugate-covariance convention.

Applying this result to the order-two moving-average model yieldsEquivalently, its nonzero autocovariances are ${\gamma }_0={\sigma }^2(1+{\theta }_1^2+{\theta }_2^2)$, ${\gamma }_1={\sigma }^2{\theta }_1(1+{\theta }_2)$ and ${\gamma }_2={\sigma }^2{\theta }_2$, together with ${\gamma }_{-k}={\gamma }_k$.

For the order-one moving-average process, invert its filter by a geometric series. The required coefficients areThey are absolutely summable for $|\theta |<1$. Direct cancellation giveswith mean-square convergence. Thus the inverse filter recovers exactly the original white noise process, including its variance; it is an invertible time-series representation.

For independent draws $X_i$ from a normalized proposal $g$, use the importance sampling identityThus the unbiased estimator and its variance areThe probability support condition is $g>0$ wherever $\theta f\ne 0$, and the variance formula requires a finite second moment. Since both probability densities are normalized, no self-normalization of weights is needed.

In the proposed example, normalize $g$ to $g(x)=3x^2$ on $(0,1)$. It is sampled by $X=U^{1/3}$ for uniform $U$. Taking $\theta (x)=x$ givesAs a check, ${\mu }_g=\frac{4}{\pi }{\int }_0^{1}x\sqrt{1-x^2}dx=4/(3\pi )$. The second moment of one weighted observation isso $\mathrm{Var}({\hat{\mu }}_g)=16/(9{\pi }^{2}n)$. The weight's singularity at zero does not make this variance infinite.

To prove the optimal proposal rule, apply the Cauchy-Schwarz inequality:Equality holds when the two factors are proportional almost everywhere, giving the minimum-variance importance distributionFor the present nonnegative integrand, normalization givesUnder this proposal, $xf(x)/g_0(x)=4/(3\pi )$ is constant, so the variance is exactly zero. If sampling is wanted, its distribution function is $1-(1-x^2{)}^{3/2}$, with inverse $X=\sqrt{1-(1-U{)}^{2/3}}$. In general the optimal proposal's normalizing constant can itself be the unknown integral, limiting this zero-variance construction's practical use.

For minimizing an energy $F(\theta )$, simulated annealing at temperature $T>0$ uses the Boltzmann distribution ${\pi }_T(\theta )\propto e^{-F(\theta )/T}$. At each temperature run a Metropolis–Hastings algorithm or Gibbs transition preserving that law, then reduce $T$. With a symmetric proposal, accept ${\theta }^{\prime }$ with probability $\min \{1,e^{-[F({\theta }^{\prime })-F(\theta )]/T}\}$: downhill moves are accepted and uphill moves remain possible. Keep the best state encountered. As temperature decreases, equilibrium mass favors global minima, provided the target is proper and the relevant concentration hypotheses hold. An arbitrary finite cooling run does not by itself prove global convergence.

Let $s={\sum }_{i=1}^m{x}_i$, $\bar{x}=s/m$ and $S={\sum }_i(x_i-\bar{x}{)}^2$. For the Poisson distribution model the observations must be nonnegative integers. Its log-likelihood isFor $s>0$, differentiation gives ${\ell }_P^{\prime }=-m+s/\lambda$ and ${\ell }_P^{\prime \prime }=-s/{\lambda }^2<0$. If $s=0$, the likelihood is decreasing and the maximizer is the boundary value zero. For the normal distribution model write $v={\sigma }^2>0$, givingFor each $v$, this is maximized at $\mu =\bar{x}$, and differentiating there gives $v=S/m$. HenceThe normal variance estimate has denominator $m$, not $m-1$. A finite positive normal maximum requires $S>0$; if all observations coincide, the likelihood is unbounded as $\mu =\bar{x},v\downarrow 0$. The normal annealing calculation below assumes $S>0$.

To maximize likelihood, choose the energy $-\ell$, so the annealing law is $L^{1/T}$, not $L^{-1/T}$. This sign is important in the Poisson request: literally inserting a positive log-likelihood into a minimizing Boltzmann law would give a nonnormalizable probability density function growing like $e^{m\lambda /T}$ at infinity. We use likelihood-power annealing with the maximizing sign.

For the normal distribution model, take base measure $d\mu dv$. The joint annealing probability density function isCompleting the square gives the Gibbs sampling conditional for $\mu$. Matching the conditional $v$ probability density function to the supplied inverse-gamma distribution convention givesIntegrating out $\mu$ contributes a factor $\sqrt{v}$, so the marginal law isThus this joint Boltzmann law is proper for $T<m/3$. Its variance marginal and the normal conditional both concentrate at the stated estimates as $T\downarrow 0$. For example, for $T<m/7$ its marginal mean and variance areAlso ${\mathbb{E}}_T(\mu -\bar{x}{)}^2=(T/m){\mathbb{E}}_{T}v\to 0$.

There is a direct convergence proof for the actual changing-temperature Gibbs algorithm, beyond convergence of its equilibrium probability densities. Start at a finite deterministic $v_0>0$, choose deterministic $0<T_n\le m/10$ with $T_n\to 0$, draw ${\mu }_n$ conditional on $v_{n-1}$, then draw $v_n$ conditional on ${\mu }_n$ using the two boxed distributions. Conditional normal moments giveThe inverse-gamma distribution first and second moments consequently yieldandIn the first recursion the coefficient of the preceding mean is at most $1/6$; in the second the coefficient of the preceding second moment is at most $1/8$. The means therefore stay bounded, and then so do the second moments. Letting $T_n\to 0$ in these formulas gives $\mathbb{E}{v}_n\to S/m$ and $\mathbb{E}{v}_n^2\to (S/m{)}^2$. Together with $\mathbb{E}({\mu }_n-\bar{x}{)}^2=T_n\mathbb{E}{v}_{n-1}/m\to 0$, this provesThis Gaussian likelihood Gibbs annealing convergence uses the explicit conditional structure and does not assume that one Gibbs sampling sweep is an exact equilibrium draw.

For the Poisson distribution model, the likelihood-power law with respect to $d\lambda$ has the normalized probability density functionsoThe mean tends to $\bar{x}$ and the variance to zero, proving convergence to a point mass at the Poisson maximum-likelihood estimate, including the zero-boundary case.

The Akaike information criterion is $2k-2\ell (\hat{\theta })$, counting one parameter for Poisson and two for normal. With $0\log 0=0$ and $S>0$, the two forms areandChoose the smaller criterion. Likelihood constants that differ between models, including the factorial and $2\pi$ terms, cannot be discarded.

For a fully specified trans-dimensional annealing for penalized likelihood, let $M=P$ or $N$ and defineUse the combined target, relative to counting measure on models and Lebesgue measure on their parameters,where $w_P,w_N>0$ are fixed reference weights. For $S>0$ and $0<T<m/3$ both model components are integrable. The factor $2T$ is a convenient temperature convention keeping the within-model laws exactly those already derived. The extra factor $e^{-k_M/T}$ makes the zero-temperature objective AIC rather than unpenalized likelihood.

Use, for example, probability $1/2$ for a model-change attempt in either model and probability $1/2$ for a within-model update. Within the Poisson distribution model draw a fresh gamma value from its boxed law; within the normal distribution model perform the boxed Gibbs sampling sweep. For a Poisson-to-normal proposal, draw $u\sim q(u)=N(0,{\tau }^2)$ with a fixed ${\tau }^2>0$, and setThe inverse normal-to-Poisson proposal is $\lambda =v$ with reverse auxiliary $u=\mu -v$. This dimension matching uses one auxiliary variable to augment the one-dimensional Poisson parameter space. The absolute Jacobian determinant isIf the forward model-jump selection probability is $b$ and the reverse probability is $d$, the reversible-jump Markov chain Monte Carlo acceptance ratio isAccept with probability $\min (1,R_{P\to N})$. For the reverse move compute $\lambda =v,u=\mu -v$ and useFor the suggested equal selection probabilities, $b=d=1/2$ and those factors cancel. The probability density function $q$ must still be included, despite symmetry of its normal distribution: it is an auxiliary draw in only the forward dimension-increasing proposal. Pairing a move with its inverse makes the accepted forward and reverse probability flows equal after the change of variables, proving detailed balance for each fixed-temperature model-switch kernel.

Cool through proper temperatures, allow enough model and parameter exploration at each stage, and retain the smallest observed $E_M$. The minima of the two energies are exactly the two AIC values above. Low-temperature equilibrium mass concentrates on the better criterion and its fitted parameters; if they tie, the minimizing set contains both models. Unlike the direct normal Gibbs argument, this multimodel exploration claim needs an appropriate cooling/mixing justification and is not guaranteed by an arbitrary short run.

Finally, an exact statistical comparison must put both likelihoods on the same observable and reference-measure convention. A Poisson distribution model is a probability mass function on counts, whereas the displayed normal distribution model is a continuous probability density function. The formulas above are the question's formal Gaussian working-likelihood comparison. A coherent exact normal approximation for count observations can instead use probabilities of rounding intervals; that changes its likelihood and generally loses the simple analytic normal estimates. This distinction matters when interpreting the numerical AIC difference, while leaving the dimension-matching and annealing construction applicable to properly specified likelihoods.

Interpret the barred velocities as second moments of a steady nonstreaming collisionless stellar system, and write ${\sigma }_r^2=\overline{v_r^2}$. In an isotropic model the three component dispersions agree, so the Spherical Jeans equation becomesPut $A={\rho }_0{r}_0^k$, so $\rho =A{r}^{-k}$. If this density supplies all of the Newtonian gravity and there is no central point mass, finite enclosed mass requires $k<3$ andIntegrate to an outer radius $b$ at which the pressure is $P_b=\rho (b){\sigma }_r^2(b)$. The exact Jeans-moment solution within the power-law region isand, for $k\ne 1$,For $k<1$ and $r\ll b$, the integral is dominated by outer radii. ThereforeThe coefficient cannot be determined from the local density cusp alone. Extending the power law to infinity gives a divergent pressure integral when $k<1$, so an outer scale or boundary condition is essential. For the usual shallow cusp $0<k<1$, the central velocity dispersion tends to zero even though the density diverges.

For $1<k<3$, inner radii dominate as $r/b\to 0$; equivalently one can send $b\to \infty$ with vanishing boundary pressure to obtainThus the central velocity dispersion decreases for $1<k<2$, is constant at $k=2$, and diverges for $2<k<3$. At the omitted threshold $k=1$, ${\sigma }_r^2=(P_b/A)r+2\pi GAr\ln (b/r)$. For $k\ge 3$ the assumed central density law has infinite enclosed mass, so the same unmodified model cannot determine a finite velocity dispersion. These boundary and domain restrictions are the boundary dependence of an isotropic self-gravitating cusp; specifying density without them does not specify a physical finite galaxy.

Let $p(r)=\rho (r){\sigma }_r^2(r)$ and use the velocity-anisotropy parameter $\beta =1-{\sigma }_{\theta }^2/{\sigma }_r^2$, with ${\sigma }_{\theta }^2={\sigma }_{\varphi }^2$. Here the tangential variance is per component; a definition using the sum of both tangential variances instead has an extra factor of two. The original PDF has this correct per-component ratio, which the TeX corrupts.

At a point along the line of sight, $r^2=R^2+z^2$. The radial direction makes an angle with the sightline satisfying ${\sin }^2\chi =R^2/r^2$. Spherical symmetry eliminates mixed moments, and projection givesUsing both sides of the galaxy and $dz=rdr/\sqrt{r^2-R^2}$,For this formula use $I$ as the projected density of the same tracer represented by $\rho$. If $\rho$ is a mass density and $I$ an observed surface brightness, a constant mass-to-light ratio rescales $I$ by that factor; otherwise replace $\rho$ by the luminosity density throughout the tracer equation. The gravitating $M(r)$ remains the total mass, not necessarily the tracer's mass.

The Spherical Jeans equation is $p^{\prime }+2\beta p/r=-\rho GM/r^2$. Eliminate $\beta p=-r[p^{\prime }+\rho GM/r^2]/2$ from the projected integrand:Rearranging provesNo constancy of $\beta$ was needed. Convergence of the projected moments and the steady spherical assumptions are required. This is the integral form of the projected spherical Jeans residual.

Although $\beta$ has disappeared explicitly, the unknown radial pressure $p$ still carries the orbital information. Thus the identity does not remove the mass--anisotropy--density degeneracy: projected brightness and one projected second moment do not uniquely determine both $M(r)$ and the radial/tangential partition. A central rise in velocity dispersion can reflect a compact central mass, radial orbital bias, or a changing mass-to-light ratio; flattening and departures from equilibrium add further ambiguities.

For example, within a point-mass-dominated region with tracer $\rho \propto r^{-k}$ and constant $\beta$, direct solution gives ${\sigma }_r^2=G{M}_{\bullet }/[(k+1-2\beta )r]$. For $k>1$ the projected velocity dispersion of a Keplerian cusp isThe beta function factors come from setting $r=R\sec \chi$ in the integrals along the line of sight. Different allowed $\beta$ generally change the mass needed to fit the same normalization; $k=2$ is a special cancellation, not the general case. Resolving the central black-hole sphere of influence and constraining orbital structure with higher velocity moments or additional kinematic directions strengthens a compact-mass inference. Brightness and velocity dispersion profiles alone do not establish a unique central black-hole mass, still less an event horizon.

Unmodified Hebbian learning reinforces correlated input and output but has no mechanism limiting synaptic strength. The second rule adds activity-dependent negative feedback and stabilizes the norm at an input-scale-independent value, provided the input keeps exciting the output. The third has a cubic weight penalty independent of the instantaneous output; its equilibrium norm instead depends on the strength of the input second moment.

For the third averaged rule, an equilibrium along eigenvector $i$ has radial linearized eigenvalue $-2{\lambda }_i/\tau$ and transverse eigenvalues $({\lambda }_j-{\lambda }_i)/\tau$. Thus a positive simple largest eigenvalue gives stable equilibria in its two opposite directions, whereas lower eigen-directions are unstable to larger components. With generic initial weights, it learns the dominant correlation direction, linking the rule to principal component analysis. These are idealized normalization and feature-extraction mechanisms, not claims that biological synapses implement these exact equations.

Write $R(v)={\int }_{\mathbb{R}}v(u{)}^{2}du$. The kernel density estimator with bandwidth $h>0$ isSufficient conditions are that $K$ be a nonnegative symmetric probability density function with $R(K)<\infty$ and finite nonzero second moment ${\mu }_2(K)=\int u^{2}K(u)du$, and that $h\to 0$, $nh\to \infty$. Symmetry and the second-moment assumption give $\int uK(u)du=0$. The expectation is a convolution:Taylor's theorem with integral remainder givesThe zeroth-order term integrates to $f(x)$, and the first-order term vanishes. The second derivative is bounded and continuous, so dominated convergence, with dominating function proportional to $u^{2}K(u)$, yieldsCombining this with the supplied leading variance gives the asymptotic mean squared errorWhen $f(x)>0$ and $f^{\prime \prime }(x)\ne 0$, its derivative is $-R(K)f(x)/(n{h}^2)+{\mu }_2(K{)}^2{f}^{\prime \prime }(x{)}^2{h}^3$. It changes sign exactly once, giving the pointwise optimal kernel bandwidthPositivity of $f(x)$ is needed for this positive interior optimum. The printed condition $f^{\prime \prime }(x)\ne 0$ alone does not ensure it. For instance, $f(x)=x^2\varphi (x)$ is a normalized bounded probability density function with bounded continuous square-integrable second derivative, but $f(0)=0$ and $f^{\prime \prime }(0)=2\varphi (0)>0$. At this point the displayed leading variance vanishes and the two-term AMSE has no positive interior minimizer; finer variance terms are needed to determine an appropriate bandwidth. The formal formula there gives zero, which does not satisfy the bandwidth assumptions.

For integrated error, the same expansion holds in $L^2$: translation continuity of $f^{\prime \prime }\in L^2$ applied to the integral remainder givesIndeed Minkowski's integral inequality bounds the norm of the difference, divided by $h^2$, by the integral of $u^{2}K(u){\int }_0^1(1-s)\Vert f^{\prime \prime }(\cdot -shu)-f^{\prime \prime }{\Vert }_{2}dsdu$; each translated difference tends to zero and is bounded by $2\Vert f^{\prime \prime }{\Vert }_2$. This proves the claimed integrated bias expansion without assuming a uniform pointwise remainder over the whole real line.

If $K_h(x)=h^{-1}K(x/h)$, direct integration of the variance givesThe last term is $O(n^{-1})$ because $f$ is bounded and integrable, hence square integrable, and convolution with a probability density function does not increase its $L^2$ norm. It is negligible compared with $1/(nh)$. Thus the asymptotic mean integrated squared error and its minimizer areHere $R(f^{\prime \prime })>0$: otherwise the continuous second derivative vanishes everywhere, making $f$ affine, impossible for a probability density function on the whole real line.

Now take the standard normal density $f=\varphi$. Differentiation gives ${\varphi }^{\prime \prime }(x)=(x^2-1)\varphi (x)$. Gaussian integration givesThe kernel constants cancel from the bandwidth ratio. At $x\ne \pm 1$,Put $t=x^2$. The logarithmic derivative of $e^{t/2}/(t-1{)}^2$ is $1/2-2/(t-1)$. On $0\le t<1$ the function increases, giving its minimum at $t=0$, with value one. On $t>1$ it decreases up to $t=5$ and then increases, giving its minimum $e^{5/2}/16<1$. Thus the global minimizers are $x=\pm \sqrt{5}$, and the normal-density local-to-global bandwidth ratio isThe ratio diverges at the two inflection points $\pm 1$, where the second-order pointwise bias approximation vanishes and needs a higher-order replacement.

For a prior $\Pi$ and loss $L$, the Bayes risk of a rule $d$ is $r(\Pi ,d)=\int {\mathbb{E}}_{\theta }L(\theta ,d(X))\Pi (d\theta )$. A Bayes decision rule minimizes this integrated risk. An Extended Bayes rule is a rule whose excess over the optimal Bayes risk can be made arbitrarily small by a suitable proper prior: for every $\epsilon >0$ some $\Pi$ has $r(\Pi ,d)\le {\inf }_{d^{\prime }}r(\Pi ,d^{\prime })+\epsilon$, with finite risks.

Condition on the observation and use iterated expectation. A rule is Bayes precisely when it chooses a minimizer of the posterior expected loss almost surely, up to the usual measurability requirement. Under squared Euclidean loss, with a finite posterior second moment, write $m=\mathbb{E}[\theta \mid X]$. The identityshows that the Bayes rule is the posterior mean.

The pair-birth rate here is the constant immigration rate $\lambda$, while deaths occur at total rate $\beta n$. This interpretation is fixed by the requested mean formula. With $P_n(t)=\mathrm{Pr}(N_t=n)$ and $P_j=0$ for $j<0$, the master equation isThe Markov jump-process generator acts as $Lf(n)=\lambda [f(n+2)-f(n)]+\beta n[f(n-1)-f(n)]$. Taking $f(n)=n$ gives $m^{\prime }=2\lambda -\beta m$, henceFor the second moment $s=\mathbb{E}{N}_t^2$, the two jumps give $s^{\prime }=4\lambda m+4\lambda -2\beta s+\beta m$. Differentiating $V=s-m^2$ therefore yields $V^{\prime }=4\lambda +\beta m-2\beta V$. Substitution of the mean and solution of this scalar linear equation giveThus $V(t)\to 3\lambda /\beta$, assuming $\beta >0$ and finite initial second moment. This is pair immigration with linear deaths; births at rate $\lambda n$ would be a different model and would not give the printed mean.

Use the positive-support convention for the geometric distribution and put $q=1-p$. Differentiating the convergent geometric series ${\sum }_{k\ge 0}{q}^k=(1-q{)}^{-1}$ givesConsequentlySubtracting the square of the expectation from the second moment yields

For the die, this is the coupon collector problem. When $r$ scores are still missing, let $W_r$ count rolls up to and including the next newly observed score. Every fresh roll has success probability $r/6$, so $W_r$ has a geometric distribution of parameter $r/6$. These six successive waiting times are independent: conditional on the entire past at the start of a stage, its waiting-time distribution depends only on the number $r$, not on the identities of the missing scores. Iterated conditioning therefore factors their joint distribution. The total number of rolls is $N={\sum }_{r=1}^6{W}_r$.

Using linearity of expectation givesUsing variance additivity for independent random variables, or the coupon collector waiting-time variance formula just derived by this decomposition, givesHence the requested standard deviation isThe hint rounds the sum of squares from $53.69$ to $53.7$; using that rounded value gives $\sqrt{39}\approx 6.245$, consistent at the precision of the hint. The exact computation preserves the slightly more precise value above.

Let $X$ be one score. It has the discrete uniform distribution on $1,\dots ,6$, so its expected value and second moment areThereforeFor independent throws, linearity of expectation and additivity of variance give $\mathbb{E}{Y}_n=7n/2$ and $\mathrm{Var}(Y_n)=35n/12$. The sample average consequently has expectation $7/2$ and variance $35/(12n)$.

The Chebyshev inequality follows by applying the Markov inequality to the nonnegative square of a centered random variable. It givesThis is at most $1/10$ if $n\ge 350/27$. Thus $n=13$ suffices, as does every larger integer. This is the smallest integer guaranteed by this particular Chebyshev inequality bound; it need not be the smallest integer satisfying the actual probability requirement.

Use the fixed common entry law from the printed setup: $Y=\sqrt{N}{X}_{ij}$ has mean zero and second moment $1$ for each upper-triangular entry, including the diagonal. LetSince $\mathbb{E}Y=0$, the removed part after centered truncation of a Wigner matrix isConsequently $\mathbb{E}{\Delta }_{ij}^2=N^{-1}\mathrm{Var}(Z_C)\le N^{-1}t(C)$. Symmetry gives $\mathrm{Tr}{\Delta }^2={\sum }_{i,j}{\Delta }_{ij}^2$. There are $N$ diagonal terms and $N(N-1)$ off-diagonal terms in this sum, soMirrored entries are counted twice, as they must be; independence of those mirrored entries is neither true nor needed for this expectation calculation.