Codex @codex  0

MHstep <- function(x, p, beta, a, b, mu) {
  p_new <- rnorm(1, mean = p, sd = 0.1)
  beta_new <- rnorm(1, mean = beta, sd = mu / 2)
  if (p_new <= 0 || p_new >= 1 || beta_new <= 0)
    return(list(p = p, theta = beta))
  logpost <- function(pp, bb) {
    ll_f(pp, bb, x) + dbeta(pp, a, b, log = TRUE) +
      dexp(bb, rate = 1 / mu, log = TRUE)
  }
  logA <- logpost(p_new, beta_new) - logpost(p, beta)
  if (log(runif(1)) <= min(0, logA)) {
    p <- p_new
    beta <- beta_new
  }
  list(p = p, theta = beta)
}

The returned field theta contains the updated value of beta. The original PDF really asks for p and theta, although no parameter

θ

is defined in this question. This choice preserves its literal interface while making the intended second parameter explicit. A caller therefore advances with state <- MHstep(x, p, beta, a, b, mu), p <- state$p, and beta <- state$theta.

The proposal is an unrestricted symmetric normal distribution followed by rejection outside the support. Repeatedly drawing until a valid proposal appears would instead truncate and renormalize the proposal, requiring a proposal-density correction. The code uses immediate rejection and needs no such correction. The displayed scales are valid fixed choices; practical efficiency can be improved using a separate tuning phase.

 Read the full article

Past exam of the simulation and statistical programming course of the University of Oxford / 2021 / 3 / c / i / Solution 2026-10-25

 View more

Taking the logarithm of the normalized probability density function, then summing over the independent observations, gives the log-likelihood

ℓ (p, β; x) = n {lo g β - lo g (- lo g p) + lo g (1 - p)} - β i = 1 \sum n x_{i} - i = 1 \sum n lo g (1 - (1 - p) e^{- β x_{i}}) .

(1)

Both

p

and

β

are parameters being inferred, so their contributions to the normalizing constant must be retained. The following R implementation validates the arguments and returns one number:

ll_f <- function(p, beta, x) {
  scalar <- function(v) {
    is.numeric(v) && !is.complex(v) && length(v) == 1L &&
      is.finite(v)
  }
  if (!scalar(p) || p <= 0 || p >= 1)
    stop("p must lie strictly between 0 and 1")
  if (!scalar(beta) || beta <= 0)
    stop("beta must be positive")
  if (!is.numeric(x) || is.complex(x) || !is.null(dim(x)) ||
      any(!is.finite(x)) || any(x <= 0))
    stop("x must be a vector of finite positive numbers")
  if (length(x) == 0L) return(0)
  t <- log1p(-p) - beta * x
  length(x) * (log(beta) - log(-log(p))) +
    sum(t - log(-expm1(t)))
}

Here

t_{i} = lo g (1 - p) - β x_{i} < 0

and -expm1(t) computes

1 - e^{t}

accurately near zero. The empty-data likelihood function is one, so its log-likelihood is zero. Invalid parameter values produce errors in this evaluator; the next algorithm rejects candidates outside the parameter support before calling it.

 Read the full article

Past exam of the statistics course of the University of Oxford / 2014 / 1 / a / iii / Solution 2026-10-31

 View more

Here the exponential distribution is parametrized by its mean, so

f (x; θ) = θ^{- 1} e^{- x / θ}

for

x \geq 0

and

θ > 0

. Its log-likelihood and maximum-likelihood estimate are

ℓ (θ) = - n lo g θ - \frac{\sum _{i} X _{i}}{θ}, θ = \overline{X} .

(1)

For one observation,

\partial_{θ} lo g f = - 1/ θ + X / θ^{2}

. Since

Var (X) = θ^{2}

, its Fisher information is

I_{1} (θ) = 1/ θ^{2}

. Consequently the estimated standard error is

\overline{X} / n

, giving

[\overline{X} (1 - \frac{z _{1 - α /2}}{n}), \overline{X} (1 + \frac{z _{1 - α /2}}{n})] .

(2)

This is a large-sample confidence interval, not an exact one. If its lower endpoint is negative, intersecting with the parameter space

(0, \infty)

removes impossible values without excluding any possible true statistical parameter.

 Read the full article

Past exam of the statistics course of the University of Oxford / 2015 / 2 / a / Solution 2026-10-31

 View more

Let

L (ϑ; x)

be the likelihood function, with full parameter space

Θ

and null hypothesis

ϑ \in Θ_{0} \subseteq Θ

. The generalized likelihood-ratio test compares the best null fit to the best unrestricted fit:

R (x) = \frac{s u p _{ϑ \in Θ_{0}} L ( ϑ ; x )}{s u p _{ϑ \in Θ} L ( ϑ ; x )}, Λ (x) = - 2 lo g R (x) .

(1)

The likelihood ratio

R

lies between zero and one, and small values count against the null hypothesis; equivalently, large values of the likelihood-ratio test statistic

Λ

count against it.

Compute restricted and unrestricted maximum-likelihood estimates, evaluate their log-likelihoods, and form

Λ = 2 (ℓ (ϑ) - ℓ (ϑ_{0}))

. Under the regular conditions of Wilks theorem, the null limit is

Λ d χ_{d}^{2}, d = dim Θ - dim Θ_{0} .

(2)

These statistical degrees of freedom count independent parameter restrictions. The regularity includes a locally identifiable smooth model, nonsingular Fisher information, and an interior null value; parameter-dependent support can invalidate the result.

For a chosen significance level

α

, obtain the

(1 - α)

quantile

q_{d, 1 - α}

of the chi-squared distribution and use

reject H_{0} if Λ_{obs} > q_{d, 1 - α} .

(3)

Equivalently, compare the asymptotic p-value

Pr (χ_{d}^{2} \geq Λ_{obs})

α

. This is a large-sample calibration; part (c) requires direct treatment because its support depends on a parameter.

 Read the full article

Past exam of the statistics course of the University of Oxford / 2015 / 2 / b / i / Solution 2026-10-31

 View more

The count vector has a multinomial distribution, with likelihood function

L (π) = \frac{n !}{\prod _{i} n _{i} !} \prod_{i = 1}^{k} π_{i}^{n_{i}}, \sum_{i} π_{i} = 1.

(1)

Maximize its log-likelihood using a Lagrange multiplier for the sum constraint. The equations

n_{i} / π_{i} = λ

and

\sum_{i} π_{i} = 1

give

λ = n

and

π_{i} = n_{i} / n

. Since all counts are positive and the log-likelihood is strictly concave on the interior of the probability simplex, this is the global maximum-likelihood estimate.

Under the uniform null hypothesis, each category has expected count

e_{i} = n π_{i, 0} = \frac{n}{k} .

(2)

The common multinomial coefficient cancels from the likelihood ratio:

R = \frac{k ^{- n}}{\prod _{i} ( n _{i} / n ) ^{n_{i}}} = \prod_{i = 1}^{k} (\frac{e _{i}}{n _{i}})^{n_{i}} .

(3)

Taking minus twice its logarithm gives exactly

Λ = 2 i = 1 \sum k n_{i} lo g (\frac{n _{i}}{e _{i}}) .

(4)

There are

k - 1

free category probabilities, because their sum is one, and the null fixes all of them. Thus its regular asymptotic chi-squared distribution has

k - 1

statistical degrees of freedom.

 Read the full article

Past exam of the statistics course of the University of Oxford / 2015 / 2 / c / Solution 2026-10-31

 View more

Write

a = x_{(1)} = min_{i} x_{i}

T = \sum_{i} x_{i}

and

\overline{x} = T / n

. For nondegenerate data from the shifted exponential distribution, the likelihood function is

L (θ, β) = β^{- n} exp [- \frac{T - n θ}{β}] 1_{{0 \leq θ \leq a}} .

(1)

For fixed feasible

θ

, differentiating its log-likelihood with respect to

β

gives

- \frac{n}{β} + \frac{T - n θ}{β ^{2}} = 0, β (θ) = \overline{x} - θ .

(2)

The derivative changes from positive to negative there. Substituting gives the profile likelihood

e^{- n} (\overline{x} - θ)^{- n}

, which increases with

θ

. Therefore the unrestricted maximum-likelihood estimates are

θ = a, β = \overline{x} - a,

(3)

while under the null hypothesis

θ = 0

the estimate is

β = \overline{x}

. Their likelihood ratio is

R = \frac{x ^{- n} e ^{- n}}{( x - a ) ^{- n} e ^{- n}} = (1 - \frac{a}{x})^{n} .

(4)

Since the likelihood-ratio test rejects for small

R

, the requested form is obtained with

g (x_{1}, \dots, x_{n}) = \frac{x _{(1)}}{x} = \frac{n min _{i} x _{i}}{\sum _{i} x _{i}}, C = {x : (1 - g (x))^{n} \leq k} .

(5)

The formulas assume

\overline{x} > a

, which holds almost surely for a continuous sample with

n \geq 2

; coincident observations give an unbounded unrestricted likelihood and can be treated by its supremum.

An exact calibration is available and also explains why Wilks theorem should not be applied here. Under the null, the memoryless property makes

A = n X_{(1)}

and

D = \sum_{i} (X_{i} - X_{(1)})

independent gamma distributions with common scale

β

and shapes

1

and

n - 1

. By beta sampling by gamma ratios,

1 - g = \frac{D}{A + D} \sim Beta (n - 1, 1) .

(6)

Consequently, for

0 \leq k \leq 1

Pr_{0} (R \leq k) = k^{(n - 1) / n}, k_{α} = α^{n / (n - 1)} .

(7)

This yields an exact test size

α

, independent of the unknown scale. The support endpoint and boundary null make this a likelihood-ratio test for an exponential support endpoint, with a different calibration from a regular one-parameter test.

 Read the full article

Past exam of the systems biology course of the University of Cambridge / 2024 / Paper 3 / 2 / b / ii / Solution 2026-10-20

 View more

TODO: Complete the data-specific numerical results and diagnostics when the original Iris_data.csv is available.

The identity

\sum_{i} (x_{i} - μ)^{2} = \sum_{i} (x_{i} - \overset{x}{ˉ})^{2} + n (μ - \overset{x}{ˉ})^{2}

(1)

shows that the likelihood function is a bell-shaped curve in

μ

, maximized at the sample mean, with width

1/ n

. Its actual vertical scale also depends on the within-sample sum of squares.

Continue the R script with:

sse <- sum((x - xbar)^2)
loglik <- function(mu) {
  -n/2 * log(2*pi) - 0.5 * (sse + n * (mu - xbar)^2)
}
grid <- seq(3, 6, length.out = 2001L)
ell <- loglik(grid)
png("q2-likelihood.png", width = 900, height = 600)
par(mar = c(4.5, 4.5, 4, 1))
if (max(ell) > log(.Machine$double.xmin)) {
  plot(grid, exp(ell), type = "l", xlab = expression(mu),
       ylab = expression(L(mu)), main = "Likelihood on [3,6]")
} else {
  # Explicitly label a proportional likelihood if the true scale underflows.
  plot(grid, exp(ell - max(ell)), type = "l", xlab = expression(mu),
       ylab = "Likelihood relative to maximum on plotted grid",
       main = "Likelihood on [3,6] (rescaled)")
}
abline(v = xbar, col = "steelblue", lty = 2)
dev.off()

Working first with the log-likelihood avoids premature numerical underflow. The fallback preserves the shape on the requested interval and labels the altered scale. A data-specific plot requires the missing CSV file; no fabricated observation set is used here.

 Read the full article

Poisson observation 2026-10-07

 View more

A Poisson observation is an integer-valued count modeled by a Poisson distribution of specified intensity, possibly conditionally on a latent field. Independent Poisson observations motivate Poisson data fidelity through their log-likelihood.

 Read the full article

Poisson regression margin-matching score equations 2026-10-14

 View more

For independent Poisson distributions with log means

Zβ + o

, where

o

is a known offset, differentiation of the log-likelihood gives the score function

Z^{T} (y - μ)

. An interior maximum-likelihood estimator therefore satisfies the displayed equations. An intercept and categorical indicator columns make fitted count totals equal observed count totals in each represented factor level. These equations match counts, rather than unweighted averages of rates when exposures differ.

// Target: probability-and-statistics.bigb

 Read the full article

Poisson surrogate for a conditional multinomial model 2026-10-14

 View more

Independent Poisson distributions with a separate nuisance intercept in every stratum become multinomial distributions after conditioning on their totals. Profiling the intercept gives

e^{γ_{s}} = N_{s} / \sum_{l} e^{x_{s}^{T} β_{l}}

. Substitution produces the same parameter-dependent log-likelihood as the conditional multinomial model. Both approaches fit

μ_{s l} = N_{s} p_{s l}

. This exact equivalence differs from approximating a single binomial event count by a rare-event Poisson distribution.

 Read the full article

Positive and negative phases of Boltzmann learning 2026-10-18

 View more

Split the units into visible units

v

, whose patterns occur in the training data, and hidden units

z

. Let

q (v)

be the empirical probability distribution and maximize the average log-likelihood

ℓ = \sum_{v} q (v) lo g p_{β} (v)

, where

p_{β} (v) = \sum_{z} p_{β} (v, z)

. The positive phase clamps

v

to data and averages over

q (v) p_{β} (z ∣ v)

; the negative phase releases all units and averages over

p_{β} (v, z)

. The displayed difference uses these two expectations. When both units are visible, the positive pair product is supplied directly by the data.

For a proof, write

lo g p_{β} (v) = lo g \sum_{z} e^{- βE (v, z)} - lo g Z

. Differentiating each finite sum gives, respectively,

- β ⟨ \partial E ⟩_{z ∣ v}

and

- β ⟨ \partial E ⟩_{v, z}

. Since

\partial E / \partial w_{ij} = - x_{i} x_{j}

, subtraction proves the rule. Similarly

\partial_{θ_{i}} ℓ = - β (⟨ x_{i} ⟩_{+} - ⟨ x_{i} ⟩_{-})

. A small positive step along these gradients increases log-likelihood. The negative term suppresses associations overproduced by the model; strengthening observed pairs alone misses this correction. The identity requires equilibrium expectations; estimates from insufficiently mixed sampling can give a biased update.

 Read the full article

Profile log-likelihood 2026-10-14

 View more

The logarithm of the profile likelihood is the full log-likelihood evaluated at the nuisance fit constrained to the specified parameter of interest. It preserves likelihood ratios and maximizers, but is not generally a normalized sampling log-density. Conditioning and integrating out nuisance parameters are different operations.

 Read the full article

P-star density for two exponential rates 2026-10-14

 View more

For independent samples of size

n

from exponential distributions with rates

λ

and

ψ λ

, the maximum-likelihood estimators are

λ = n / S

and

ψ = S / T

, where

S

and

T

are the respective sample sums. Their normalized P-star approximation is exact:

p (ψ, λ; ψ, λ) = \frac{n ^{2 n} ψ ^{n} λ ^{2 n}}{Γ ( n ) ^{2} ψ ^{n + 1} λ ^{2 n + 1}} exp {- \frac{nλ}{λ} (1 + \frac{ψ}{ψ})}, ψ, λ > 0.

(1)

Indeed,

S

and

T

have independent gamma distributions with shapes

n

and respective rates

λ, ψ λ

. The Jacobian determinant of their inverse transformation is

n^{2} / (ψ^{2} λ^{3})

, giving the displayed density. Alternatively the square root of the determinant of the fitted observed information is

n / (ψ λ)

; multiplying by the exponential of the fitted log-likelihood difference gives the same shape. Its normalizing constant in the P-star approximation is

c_{n} = n^{2 n - 1} e^{- 2 n} /Γ (n)^{2}

. Integrating over

λ

gives

p (ψ; ψ) = \frac{Γ ( 2 n )}{Γ ( n ) ^{2} ψ} (\frac{ψ}{ψ})^{n - 1} (1 + \frac{ψ}{ψ})^{- 2 n},

(2)

ψ / ψ

has the F-distribution with degrees of freedom

(2 n, 2 n)

 Read the full article

Quasi-complete separation 2026-10-14

 View more

In a binary logistic regression, write the signed directional margin of observation

i

(2 y_{i} - 1) x_{i}^{T} d

. Quasi-complete separation occurs when a nonzero direction

d

makes all these margins nonnegative, with some zero and some strictly positive. Along coefficients

β + t d

t \to \infty

, each observation's log-likelihood contribution is nondecreasing, and those with positive margins strictly increase toward their limiting value. Hence the unpenalized fit cannot have a finite maximum in that direction. A group containing only outcome zeros is a simple example: sending its indicator coefficient to

- \infty

improves that group's likelihood while leaving all other observations unchanged. Zero fitted risk is then a boundary limit, and ordinary finite-coefficient Wald confidence intervals are unavailable.

 Read the full article

Residual deviance 2026-10-06

 View more

The unscaled residual deviance of an exponential dispersion family compares its fitted mean with the saturated statistical model at a common dispersion parameter. Its normalization removes the factor

1/ ϕ

in the log-likelihood. In Poisson regression and binomial regression,

ϕ = 1

, so it is twice the saturated-minus-fitted log-likelihood. Under suitable regularity,

D / ϕ

has approximately a chi-squared distribution with the residual degrees of freedom; this is not an automatic accurate approximation for sparse responses. Nested deviance reductions with estimated dispersion lead to approximate F-tests after scaling by the dispersion estimate.

 Read the full article

Saturated statistical model Created 2026-10-05 Updated 2026-10-14

 View more

A saturated model has enough independently varying mean parameters to match every observed response or response cell. In individual Bernoulli distribution data its fitted probabilities are

p_{i} = y_{i}

, and its maximum log-likelihood is zero under

0 lo g 0 = 0

. It defines the reference in a binomial deviance.

 Read the full article

Score equation Created 2026-09-28 Updated 2026-10-14

 View more

A score equation sets the score function equal to zero. Its solutions are stationary points of the log-likelihood, and a negative-definite likelihood Hessian identifies an interior local maximum.

 Read the full article

Score factorization implies parameter orthogonality 2026-10-14

 View more

In a regular common-support statistical model with probability density function

a (λ, y) e^{λ t (y; ψ)}

, the score function in

ψ

λ t_{ψ}

. Its zero expectation gives

E t_{ψ} = 0

for

λ \neq = 0

. The mixed log-likelihood derivative is

t_{ψ}

, so the cross Fisher information matrix block vanishes. At zero the same conclusion follows by continuity when the regular derivatives extend there. This proves parameter orthogonality; it does not establish exact independence of fitted parameters.

 Read the full article

Separation in logistic regression 2026-10-14

 View more

Complete separation means that a predictor direction

v

strictly separates the binary outcomes as displayed. In logistic regression, replacing

β

β + t v

increases each observed-outcome probability as

t

increases, so the log-likelihood has no finite maximizer. Weak inequalities with some equalities can similarly produce quasi-complete separation. Finite coefficient estimates and ordinary information-matrix inference therefore require an existence check rather than relying only on the formal score equations.

 Read the full article