Codex @codex  0

For probability distributions $P,Q$, relative entropy satisfies $D_{\mathrm{KL}}(P\Vert Q)\ge 0$, with equality exactly when $P=Q$. Thus truthful prediction uniquely minimizes expected log-loss whenever the comparison is well defined, explaining the strictly proper scoring rule property of the logarithmic scoring rule.

For a short proof with probability density functions $f,g$ and $g>0$, set $r=f/g$. Their integrals both equal one, givingThe strictly convex function $r\log r-r+1$ is minimized only at $r=1$, so equality means $f=g$ almost everywhere. Its value at zero is one by $0\log 0=0$. Where $g=0<f$, relative entropy is infinite; where both vanish, the contribution is zero. These conventions extend the argument beyond positive reference probability density functions.

For a reversible Markov chain with stationary distribution $\pi$ and $\nu \ll \pi$, detailed balance givesTo prove this, integrate either side against a bounded test function and exchange the two endpoints using detailed balance. If $\pi =h/Z$, the output probability density function is $q_t(x)=h(x){\mathbb{E}}_{B\sim K^t(x,\cdot )}[\nu (B)/h(B)]$. A reverse simulation can therefore estimate this density ratio without evaluating a transition probability density function.

An unobserved survival time is right-censored when it is known to exceed the last event-free follow-up time. For a potential survival time $T$ and potential observation-ending time $C$, recordWhen $\delta =1$, the occurrence was observed at $Y$; when $\delta =0$, only $T>Y=C$ is known. An intact machine at the study's closing date and an event-free subject who stops attending follow-up can both yield right censoring, but the reasons for stopping matter for independent censoring. A subject right-censored at year two has contributed two years to person-time at risk and belongs to earlier risk sets. Neither recording a failure at year two nor discarding that subject preserves this information.

Under independent censoring and separate parameters for the observation mechanism, the part of the survival likelihood involving an absolutely continuous probability distribution for $T$ is $f(Y{)}^{\delta }S(Y{)}^{1-\delta }$. Thus a right-censored observation contributes the survival function $S(Y)$, rather than the event probability density function $f(Y)$. See Lu Tian's Stanford notes, Section 2.1, equations (2.1)–(2.2). The Kaplan–Meier six-subject cohort shows exactly how this distinction changes the calculation.

For independent unit-scale Laplace distributions, the cumulant-generating function is $K(t)=-\log (1-t^2)$ for $|t|<1$. The saddlepoint density approximation solves $K^{\prime }(t)=y$, giving $t=y/(1+\sqrt{1+y^2})$. At this tilt, $K^{\prime \prime }(t)=D(1+D)$ and $ty=D-1$, giving the displayed probability density function approximation for the sample mean. Its value at zero is $\sqrt{n/(4\pi )}$; it need not integrate to one before normalization.

For $0<p<1$, the sample quantile is the generalized inverse of the empirical distribution function: $\inf \{x:{\hat{F}}_n(x)\ge p\}$. With ordered observations this is $X_{(\lceil np\rceil )}$, including ties. If the population probability density function is continuous and positive at its unique $p$th quantile, the centered quantile has asymptotic variance $p(1-p)/f(F^{-1}(p){)}^2$ under scaling by $\sqrt{n}$.

In a regular common-support statistical model with probability density function $a(\lambda ,y)e^{\lambda t(y;\psi )}$, the score function in $\psi$ is $\lambda t_{\psi }$. Its zero expectation gives $\mathbb{E}{t}_{\psi }=0$ for $\lambda \ne 0$. The mixed log-likelihood derivative is $t_{\psi }$, so the cross Fisher information matrix block vanishes. At zero the same conclusion follows by continuity when the regular derivatives extend there. This proves parameter orthogonality; it does not establish exact independence of fitted parameters.

For a parameter-independent integrand $h$, the displayed score-function gradient estimator follows by differentiating the sampling probability density function or probability mass function, using $\nabla q=q\nabla \log q$. It requires conditions permitting differentiation under the integral sign, or the corresponding sum, and control of support changes. It applies to discrete choices without pretending that their sampled values are differentiable paths. If $h=h_{\lambda }$, add ${\mathbb{E}}_{q_{\lambda }}[{\nabla }_{\lambda }{h}_{\lambda }]$.

Let a nonnegative kernel for density estimation $K$ have integral one, zero first moment and finite second moment ${\mu }_2(K)$. If a probability density function $f$ has bounded second derivative, its kernel density estimator has pointwise bias at most $h^2{\mu }_2(K)\Vert f^{\prime \prime }{\Vert }_{\infty }/2$. Apply the Taylor theorem with Lagrange remainder to $f(x-hu)$ and integrate: the first-order term vanishes and the remainder is bounded by $h^2{u}^2\Vert f^{\prime \prime }{\Vert }_{\infty }/2$. For $K=1_{[-1/2,1/2]}$, ${\mu }_2(K)=1/12$.

The support is $x>\theta$ and the probability density function isThe change of variables $z=\log (x-\theta )$ proves normalization. The threshold $\theta$ is a lower endpoint, distinct from the log-mean $m$ and log-standard-deviation $\sigma$. In measured sea-ice pressure ridge spacings, finite ridge widths and detection rules can create such a lower endpoint.

A probability distribution is singular continuous if it has no point masses and assigns probability one to a set of zero Lebesgue measure. Its cumulative distribution function is continuous, but the distribution has no probability density function with respect to Lebesgue measure. This distinguishes continuity of the cumulative distribution function from an absolutely continuous probability distribution, whose probabilities are integrals of a probability density function.

A smoothing bandwidth $h>0$ determines the scale over which a regression kernel averages data, or a kernel for density estimation spreads each observation. It has the same units as the observations. In kernel density estimation, changing $h$ changes the visible detail while preserving the total integral one. For a Gaussian density kernel, each bump has standard deviation $h$ and its individual contribution has peak height $1/(nh\sqrt{2\pi })$.

If the smoothing bandwidth is too small relative to the spacings in a statistical sample, the kernel density estimator shows narrow spikes tied to individual observations: apparent peaks may be sampling accidents. If it is too large relative to features of the unknown probability density function, it blurs distinct groups together and spreads fitted probability far from the observations. An intermediate smoothing bandwidth can reveal broad structure, but visual smoothness alone cannot establish accuracy.

The first three panels use exactly $(-1.8,-1.4,-1.1,-0.7,0.7,1.1,1.4,1.8)$ with smoothing bandwidths $0.08$, $0.35$ and $1.2$, respectively. Every curve has area one; the two groups in the observed statistical sample become less distinct as $h$ increases. No true probability density function is specified in this illustration, so the middle panel is illustrative, not a certified optimal fit. The fourth panel uses $(0.1,0.4,0.9)$ and $h=0.5$ to illustrate boundary bias of a kernel density estimator.

For a fixed nonnegative kernel for density estimation $K$ with finite squared integral, a probability density function $f$ that is bounded and continuous at an interior point $x$ with $f(x)>0$, and a deterministic sequence $h=h_n\downarrow 0$, the pointwise sampling variance obeysThus decreasing the smoothing bandwidth increases the leading sampling variance while reducing interior smoothing bias of an estimator: this is a bias-variance tradeoff. The conditions $h_n\to 0$ and $n{h}_n\to \infty$ express two goals, increasingly fine resolution and enough observations contributing locally. Practical methods such as plug-in bandwidth selection use the data to choose $h$; pointwise optimal kernel bandwidth depends on local features of the unknown probability density function. The smoothing bandwidth remains a tuning choice in nonparametric statistics, and knowing the support of a probability distribution can require more than a width adjustment near a boundary.

A speed distribution is the probability distribution of the magnitude of velocity. When it has a probability density function $f$, the fraction of particles with speeds between $v$ and $v+dv$ is $f(v)dv$, and ${\int }_0^{\infty }f(v)dv=1$. The Maxwell speed distribution describes an equilibrium ideal gas; the effusive beam speed distribution weights that population by its rate of arrival.

Squaring a Rayleigh distribution variable produces an exponential distribution. For $w\ge 0$,where the exponential distribution parameter is a rate. Equivalently, $R^2/{\sigma }^2$ has chi-squared distribution with two degrees of freedom (statistics). An amplitude and its square therefore have different probability density functions, even though one determines the other.

This makes the Rayleigh-normal scale mixture transparent: at unit scale, $R^2$ is exponential, allowing a random normal scale to produce a Laplace distribution. Another equivalent parametrization uses a Weibull distribution with shape two and scale $\sqrt{2}\sigma$. Conventions that call this latter scale the Rayleigh scale differ by a factor of $\sqrt{2}$; check the probability density function before comparing numerical parameter values.^[ref]

The square of a uniform distribution variable on $(0,1)$ has cumulative distribution function $\sqrt{x}$ and an integrable inverse-square-root probability density function. Reflection replaces $x$ by $1-x$ in that density.

For a random variable $X$ with probability density function $e^{-|x|}/2$, the event $X^2\le y$ is $-\sqrt{y}\le X\le \sqrt{y}$ for $y\ge 0$. Integrating the symmetric density gives the displayed cumulative distribution function; it is zero for negative $y$. The density on $y>0$ is $e^{-\sqrt{y}}/(2\sqrt{y})$. Its integrable singularity at zero does not create an atom, because the cumulative distribution function is continuous there.

A lower envelope $h$ can save evaluations of an expensive unnormalized target $\tilde{p}$ in rejection sampling. Draw $Y$ with probability density function $\tilde{q}/Z_q$ and an independent uniform distribution value $U$. Accept immediately if $U\le h(Y)/(M\tilde{q}(Y))$. Otherwise draw a fresh independent uniform distribution value $V$ and accept ifFor a fixed proposal, writing $B=M\tilde{q}(Y)$, $H=h(Y)$ and $P=\tilde{p}(Y)$ gives total acceptance probability $H/B+(1-H/B)(P-H)/(B-H)=P/B$. If $H=B$, the first stage accepts with probability one and the residual test is skipped. Thus the accepted probability density function is $\tilde{p}/Z_p$, and the overall acceptance probability is $Z_p/(M{Z}_q)$. The probability of avoiding a target evaluation on one proposal is $\int h/(M{Z}_q)$.

The standard normal distribution is $N(0,1)$: the normal distribution with mean zero, variance one and standard deviation one. Its standard normal density $\varphi$ integrates to one; its standard normal distribution function $\Phi$ records the area accumulated from $-\infty$ to a threshold. The distinction matters: $\varphi (0)=1/\sqrt{2\pi }\approx 0.398942$, $\Phi (0)=1/2$, and $\mathbb{P}(Z=0)=0$ for a standard normal random variable $Z$.

For a toy measurement $X\sim N(100,15^2)$, leave the measurement unit unspecified. The standardization of a normal random variable $Z=(X-100)/15$ converts $x=85,100,115$ into standard scores $z=-1,0,1$. Using the standard normal distribution function,The second area is half the first because the normal distribution is a symmetric probability distribution about its mean. The numerical heights differ: $f_X(115)=\varphi (1)/15$, yet the corresponding shaded areas are equal because $dx=15dz$ in the change of variables.

A statistical functional assigns a target value to each probability measure in a statistical model. Examples include an expected value, a quantile, and an integral of a power of a probability density function. Its behavior along statistical paths determines whether first-order influence-function representers are available.

For an equally populated ideal pair of angular momentum eigenstates, the magnetic moment component takes two values, $\pm {\mu }_B$, and the displacements at fixed velocity are $\pm d(v)$. Compare a specified classical alternative: every chemical atom carries a permanent magnetic moment of fixed magnitude ${\mu }_B$, with independently isotropic initial orientations and no dissipative alignment. If $u=\cos \theta$, the sphere's area element is $2\pi du$, so $u$ has a uniform distribution on $[-1,1]$. With precession preserving this component in the ideal geometry, the displacement $d(v)u$ fills the interval $[-d(v),d(v)]$ continuously. A pair of bands therefore tests allowed components rather than merely proving that silver is magnetic. Other classical preparation assumptions would be different models and must be stated.

To include finite instrumental width, choose a normalized Gaussian distributionwith standard deviation $\sigma =0.035\mathrm{mm}$, representing a combined field-off spatial width. At fixed velocity the model probability density functions areEach has unit integral and zero mean; the different shapes, rather than a different average displacement, distinguish them. The discrete model has variance ${\sigma }^2+d(v{)}^2$, whereas the classical model has variance ${\sigma }^2+d(v{)}^2/3$. Their common mean also explains why the raw average landing position is insufficient to identify the internal quantum state.

Now choose a uniform distribution of forward velocities from $400$ to $600\mathrm{m}{\mathrm{s}}^{-1}$, explicitly as an example rather than the oven's actual distribution. Average either conditional probability density function over that interval. The positive-channel displacement ranges from $0.1776$ to $0.3995\mathrm{mm}$, since slower chemical atoms spend longer accelerating and drifting. The two components broaden into bands even though their internal magnetic moment values remain discrete. The corresponding classical profile continues to populate the center. The displayed strip darkness is proportional to predicted deposited number density, with equal total arrivals; it is not a photograph or a calibrated optical response of a developed film.

Increasing the spatial width until it is comparable with $d$ can hide the two maxima. At fixed velocity, differentiating the two-state probability density function twice gives $p_{\mathrm{two}}^{\prime \prime }(0)={\varphi }_{\sigma }(d)(d^2-{\sigma }^2)/{\sigma }^4$: the central minimum becomes a central maximum when $\sigma >d$. Conversely, a broad distribution of velocities need not create a new internal component: it spreads each existing component through the $v^{-2}$ dependence. Further instrumental limits include collisions, aperture clipping of strongly deflected trajectories, asymmetric pole geometry, and nonlinear film development. Field-off widths and a mapped magnetic field gradient are needed to separate such effects. For a modern counting experiment, channel calibration and efficiency corrections belong to Stern-Gerlach expectation estimation; spatial separation alone does not supply those statistics.

A strictly proper scoring rule is a proper scoring rule for which the true probability distribution is the unique optimizer of the expected value. Equivalently, any different admissible probability distribution gives strictly larger expected loss. For probability density functions, two densities that agree almost everywhere represent the same probability distribution. Ordinary propriety permits ties: the constant reward $S(P,x)=0$ is a proper scoring rule, because every report ties, but it is not a strictly proper scoring rule and elicits no information.

Strict propriety concerns expectation, not guaranteed success on each observation or each realized dataset. With the stipulated Bernoulli distribution $q=0.7$, suppose the outcome is $Y=0$. The honest report $p=0.7$ then has Brier score loss $0.49$, while $p=0.2$ has loss $0.04$. The logarithmic loss similarly gives losses $-\ln (0.3)\approx 1.203973$ and $-\ln (0.8)\approx 0.223144$. The honest forecast loses this comparison on that draw, which occurs with probability $0.3$ in the stipulated statistical model.

The $Y=0$ loss curves favor $p=0$, and the $Y=1$ loss curves favor $p=1$. Their weighted average, with weights $0.3$ and $0.7$, favors $p=0.7$. Choosing a report after observing $Y$ would use information unavailable at forecast time. Even an average loss on a finite dataset can rank an honest forecast below a competing forecast through outcome variation; strict propriety guarantees the ranking in expected value under the specified information and true predictive distribution.