A logistic regression can specify each binary response conditionally on recorded past outcomes and baseline covariates. A chain rule for probabilities gives the path likelihood as the product of the conditional Bernoulli distribution masses. This permits dependence within a person's history without treating their outcomes as unconditionally independent. The model must specify the initial history and which past features enter its conditional mean.
The accumulated number of previous successes may enter the conditional mean in a history-dependent logistic regression. Its coefficient exponentiates to the conditional success odds ratio per previous success. For a specified future path, update this cumulative predictor after each outcome before multiplying the conditional probabilities.
A previous outcome may affect a subsequent outcome after controlling both measured covariates and persistent unobserved heterogeneity. This is true state dependence. An observed lagged-outcome association alone does not establish it, because omitted individual propensities can create apparent persistence.
A regression for a categorical response with more than two possible outcomes, parametrizing log probability ratios relative to a baseline category by linear predictors. It preserves categories that a binary logistic regression would merge.
A logistic-normal count model takes and , where is a stationary unit-variance autoregressive process of order one with coefficient and the independent have normal distribution with variance . The marginal latent variance is , and its off-diagonal covariance is . Random success probabilities produce overdispersion relative to the binomial distribution.
A logistic model maps a real-valued score to the probability , equivalently . Logistic regression takes to be an affine function of the predictors.
Separation in a binary regression model occurs when an affine score assigns all observations from one class to one side of a threshold and all observations from the other class to the other side.
Complete separation means that every signed training margin is strictly positive for some affine score. In unpenalized logistic regression it causes coefficient norms to diverge and prevents a finite maximum-likelihood estimate.
At an interior maximum-likelihood fit of a logistic-regression model containing an intercept, the intercept score function isConsequently the sum of the fitted probabilities equals the number of observed successes exactly.
For a positive semidefinite trace ball , the quadratic-form hypothesis class isIts empirical logistic risk is convex as a function of .
L1-penalized logistic regression minimizes empirical logistic loss plus . Its subgradient optimality conditions set each logistic score coordinate equal to , where off zero and at zero.
For independent grouped counts , grouped-binomial logistic regression specifiesFitting the proportions with binomial weights gives the same likelihood.
With treatment coding, the intercept describes the reference factor level and each displayed factor coefficient is a contrast against that level. Changing the reference level reparametrizes the same fitted model: fitted probabilities and likelihood do not change, but coefficients representing different pairwise contrasts can have different standard errors and p-values.
A stochastic block model assigns each vertex of a graph to a class and, conditional on those classes, makes distinct edges independent random variables whose probabilities depend only on the classes of their endpoints.
For independent Bernoulli distribution upper-triangular entries of a symmetric zero-diagonal adjacency matrix of a graph, put . For a unit vector , has sub-Gaussian variance proxy at most , by the Hoeffding lemma. Hence . The volumetric bound for Euclidean metric nets gives a -metric net of the unit sphere with at most points. The quadratic form net bound gives on that net. The union bound gives ; integrating this tail proves the displayed expectation bound.
An additive class-effect logistic network model gives an edge joining classes and the log odds . A further coefficient multiplying represents a common within-class log-odds effect.
For independent binary edges with log odds , the natural statistics are the sum of the vertex degrees in each class and the total number of within-class edges. They form a minimal sufficient statistic whenever the natural parameter space contains an open subset of .
Articles by others on the same topic
Logistic regression is a statistical method used for binary classification problems, where the goal is to model the relationship between one or more independent variables (features) and a binary dependent variable (outcome) that can take on two possible values, typically represented as 0 and 1. Unlike ordinary linear regression, which predicts continuous outcomes, logistic regression predicts the probability that a given input falls into one of the two categories.