Conditional logistic model for longitudinal binary data Created 2026-10-07 Updated 2026-10-14
A logistic regression can specify each binary response conditionally on recorded past outcomes and baseline covariates. A chain rule for probabilities gives the path likelihood as the product of the conditional Bernoulli distribution masses. This permits dependence within a person's history without treating their outcomes as unconditionally independent. The model must specify the initial history and which past features enter its conditional mean.
A logistic regression fitted to ungrouped Bernoulli observations has residual deviance . Comparing it automatically with is generally invalid: the saturated dimension grows with , with one observation per fitted probability. If the true model has only an intercept and success probability , then , whereas . Fixed-dimensional nested-model likelihood-ratio tests can still use Wilks theorem under regularity. For absolute fit, use meaningful grouping, residual diagnostics, or a fitted-model parametric bootstrap.
For binary predictors , a logistic regression with linear predictor has conditional odds ratios at and at . Their ratio is . The same interaction can be computed from the four cell logit link values as . Independent positive event and nonevent counts in each cell give an estimated variance for this contrast equal to the sum of all eight reciprocal counts. An interaction on the odds ratio scale is not an interaction on the absolute-risk scale.
Logistic model Created 2026-09-24 Updated 2026-10-05
A logistic model maps a real-valued score to the probability , equivalently . Logistic regression takes to be an affine function of the predictors.
Logit 2026-10-14
The logit link maps a mean probability to its log odds: , . Its inverse is , used in logistic regression.
Multinomial logistic regression Created 2026-10-05 Updated 2026-10-07
A regression for a categorical response with more than two possible outcomes, parametrizing log probability ratios relative to a baseline category by linear predictors. It preserves categories that a binary logistic regression would merge.
Include an intercept and let be the chosen numerical feature vector. Logistic regression uses
The logistic function maps every real score into , allowing a Bernoulli distribution model for the label. Equivalently, the log odds are linear in the features:
This gives a smooth differentiable probability model with a linear decision boundary at any fixed probability threshold. The sigmoid output is a fitted conditional probability, not a categorical prediction until a threshold is chosen.
Use logistic regression when the response variable is binary, for example whether a lecturer missed a deadline. It relates the event's log odds to one or more predictors, producing fitted probabilities between zero and one. Exponentiating a coefficient gives an odds ratio for a specified predictor change, conditional on the other model terms. An odds ratio is not generally a ratio of probabilities; the corresponding probability change depends on the starting probability and other predictors.
Report the outcome coding, estimated associations with confidence intervals, and relevant model checks. Sparse events, complete separation and dependent observations can make ordinary estimates or inferential procedures unreliable; repeated observations may need a generalized linear mixed model. For prediction, evaluate calibration and discrimination on data not used to tune the model. An adjusted association in an observational study does not automatically identify a causal effect.
Use a hierarchical Bayesian model with binomial observation errors and a common distribution for the underlying risks:
Give a shared hyperprior, or parameterize them by mean and concentration and put priors on and . Conditional on these hyperparameters, the risks are independent and identically distributed; after integrating them out they are exchangeable random variables with shared uncertainty. The data estimate the overall risk level and between-hospital variation. Conditional Beta-binomial conjugacy gives
and full Bayes point estimates under squared-error loss are
An Empirical Bayes method instead estimates the hyperparameters from the marginal beta-binomial distribution likelihood
and inserts their fitted values in the conditional posterior means. Both approaches provide partial pooling, stronger for smaller hospitals. Full Bayes also propagates hyperparameter uncertainty into the credible intervals. If systematic covariates are needed, one can place exchangeable residual effects in a logistic regression model rather than assuming a common unconditional risk distribution.
The family-history table contains men, of whom have the response present, giving an overall observed fraction . The unadjusted fractions are with a positive family history and without one. Their crude odds ratio is ; this is not the covariate-adjusted effect from the fitted logistic regression.
The full model has an intercept and nine slopes, hence residual degrees of freedom. The binomial distribution dispersion is fixed at one. Although the software labels coefficient-to-standard-error ratios as t values, here they are asymptotic normal Wald statistics, not statistics with an estimated Gaussian residual variance and an exact Student reference. The full conditional coefficients provide evidence for tobacco, cholesterol, family history, behaviour score and age: their absolute Wald ratios are approximately . The other four ratios are smaller in magnitude, especially alcohol at about . These are conditional associations; a weak adjusted coefficient does not by itself establish absence of an underlying association, and multicollinearity can affect the estimates.
The null deviance is on degrees of freedom and the full residual deviance is on . Their difference has a nominal nine-degree-of-freedom likelihood-ratio test reference for the nine slopes jointly. The absolute residual deviance should not automatically be compared with for the reasons in part (i).
The shown stepwise selection by the Akaike information criterion follows a backward path. For individual binary observations the saturated log-likelihood is zero, so
where includes the intercept. Removing a one-parameter term improves AIC exactly when the increased deviance is less than two. The successive deletions are alcohol, adiposity, blood pressure and obesity. The corresponding values are
At the final step, every remaining single-term deletion increases AIC. This is a local stopping condition of the search, not a proof that it found the best subset among all possible models. Obesity is retained for several steps because its successive deletion costs differ after the other terms are removed; the procedure does not simply sort the initial Wald ratios.
For the selected fit, write for positive family history and otherwise. The fitted linear predictor is
Holding the other predictors fixed, a unit increase multiplies the odds by . In the same variable order, the unit odds ratios are
A ten-year age difference multiplies the fitted odds by , and ten behaviour-score points by . Family history has adjusted odds ratio about , with ordinary fitted-model confidence interval . The intercept is the log odds at zero numerical covariates and absent family history; that extrapolated baseline is not a typical man's risk. Odds ratios are not risk ratios.
The selected model's residual degrees of freedom are , consistent with its six coefficients. All five retained slopes have nominal normal-Wald values below in that model, but these tests and confidence intervals do not incorporate the prior model selection. Its improvement over the intercept-only fit is on five nominal degrees of freedom; it loses only deviance units relative to the nine-slope fit while using four fewer parameters. Selection optimism remains relevant to predictive performance. Check possible nonlinear continuous-predictor effects, influential observations and clinically motivated interactions; assess out-of-sample discrimination and agreement of predicted probabilities with observed frequencies by cross-validation or external data. The printed output alone does not supply those diagnostics, and the selected high-risk male population limits extrapolation to other populations. Nothing here establishes causal effects of these observational predictors.
For logistic regression, set and . Its Bernoulli log-likelihood, score function and Fisher information are
where . Solve by Fisher scoring or iteratively reweighted least squares. One scoring step is
Equivalently regress the working response on using weights . Iterate until the likelihood and coefficients stabilize, with rank and convergence checks. Separation in logistic regression can prevent a finite maximum: if a predictor direction strictly separates the zeros and ones, moving indefinitely along it improves the likelihood rather than producing a finite root.
Under the unrestricted model, each individual probability is estimated independently as . Every observed binary outcome then has probability one, so, using boundary limits,
The resulting residual deviance is . It is a legitimate likelihood contrast and can be used to compare fitted models. However, individual Bernoulli deviance need not have a chi-squared calibration: the saturated model has one boundary-valued parameter for each single observation, and its dimension grows with . The fixed-dimensional regularity behind Wilks theorem does not justify a reference here.
For a concrete demonstration, let the true model contain only an intercept and have success probability . Its fitted probability is , and
whereas a variable divided by converges to one. Thus treating the raw Bernoulli residual deviance as an ordinary absolute goodness-of-fit statistic can reject a correct model systematically. In contrast, deviance differences between fixed-dimensional nested logistic models have their usual asymptotic chi-squared reference under the relevant regularity assumptions. For absolute fit use meaningful replicated covariate groups and grouped-binomial logistic regression, or assess residual patterns and prediction on held-out data; a model-specific parametric bootstrap can calibrate a chosen statistic.
The first fit is a Bernoulli logistic-regression model for the probability of a positive response. Its fitted linear predictor is
and its fitted probability is . There are eight coefficients. The null degrees of freedom identify , and the residual degrees of freedom are .
Each slope is a change in the log odds per unit of its covariate, holding the others fixed. Exponentiating gives conditional odds ratios; for example a ten-unit glucose increase multiplies fitted odds by , and a ten-year age increase by . These are changes in odds, not multiplicative changes in probability, and the observational associations are not automatically causal effects. The intercept corresponds to all numerical covariates being zero, generally an extrapolation beyond meaningful subjects.
The standard errors come from the inverse Fisher information matrix of the logistic regression. With binomial dispersion parameter fixed at one, the displayed t ratios are asymptotic normal Wald statistics. In the full fit, glucose and the pedigree measure have clear individual evidence of nonzero slopes, with two-sided and . The body-mass and age terms are borderline, with and ; pregnancy count is weaker, . Blood pressure and skinfold have little individual evidence after adjustment. These labels should not be converted into a claim that the weaker predictors have no effect, or used as automatic model-selection rules.
A deviance residual here is
Its square is the observation's contribution to binomial deviance, so the squared residuals sum to the printed residual deviance. The five-number summaries describe the distribution of these signed residuals. The null deviance belongs to an intercept-only logistic regression; the full fitted deviance is . Four Fisher scoring iterations record convergence of the numerical likelihood fit.
The second model uses only glucose:
Its two coefficients leave residual degrees of freedom. A ten-unit glucose increase multiplies its fitted odds by . This differs from the adjusted multiplier , because the models condition on different covariates; both adjustment and noncollapsibility of the odds ratio can change logistic coefficients. The glucose-only model has deviance , so its maximized log-likelihood is lower than the full model's.
For these fixed-dimensional nested models, analysis of deviance and Wilks theorem give
Thus the additional six predictors collectively improve the glucose-only model, even though several individual Wald tests are not significant. These comparisons assume independent subjects, an appropriate logistic mean structure, full rank, and regular finite parameter estimates without separation.
It would be incorrect to infer that absolute fit is good merely because . Individual Bernoulli deviance need not have a chi-squared calibration: there is just one binary observation per saturated probability, and the saturated dimension increases with sample size. A simple counterexample is an intercept-only model with true probability , for which , not one. For absolute adequacy, inspect residual patterns and predicted-risk calibration, use meaningful grouped comparisons when appropriate, or calibrate a chosen fit statistic by parametric bootstrap. The regular nested-model tests above are not subject to that saturated-model dimensionality problem. Only the displayed summaries and a few example rows are supplied, so patient-level diagnostics or complete numerical refitting cannot be reconstructed from this page alone.
For a population question use a population-averaged logistic model for repeated binary outcomes. Code for the new treatment and for the control, with visit times months for . Let , , and
Different subjects are independent. Each response has a Bernoulli distribution with this marginal mean, but no independence of a subject's three responses is assumed. The linear time effect and absence of treatment-by-time interaction are substantive mean assumptions; if inappropriate they can be replaced by time indicators and interaction terms.
For complete data, set , , and , where has rows . A working exchangeable correlation matrix has unit diagonal and off-diagonal value ; set . The generalized estimating equation is
With correct means and independent subjects, a positive-definite working covariance matrix and regularity give consistency even if the working correlation is wrong. Write and . The fitted subject-level sandwich covariance matrix is
evaluated at the estimates. This inference is asymptotic in the number of independent subjects, not the number of visits.
For the actual incomplete data, define if visit is observed, with and monotone dropout. Let contain treatment, baseline covariates and responses observed before visit . Under sequential missing at random, specify
The assumption is that the retention decision, conditional on this history, does not additionally depend on unseen current or future outcomes. Fit the , for example by visit-specific logistic regressions. A simple valid choice uses working independence and the inverse-observation-weighted estimating equations for longitudinal dropout:
Terms with are zero and require no missing response. Given the full response vector, sequential missing at random makes . Consequently the weighted score has the same expectation as the complete-data score, proving its mean-zero property. Correct retention probabilities, positivity and a correctly specified marginal mean are required. Use cluster-robust uncertainty, accounting for fitted retention-model coefficients, or resample entire subjects and refit both models. Under missing completely at random, the simpler unweighted observed-response GEE can suffice; ordinary unweighted GEE is not valid under arbitrary history-dependent missing at random.
The treatment coefficient gives a population conditional-on-baseline odds ratio at each visit. To communicate an absolute population treatment benefit, standardize predicted probabilities over the full baseline population, rather than only completers:
A causal reading additionally uses the randomized controlled trial assignment, consistency of the treatment definition, no interference and the dropout assumptions; the statistical model alone does not supply those conditions.
The independence analysis is not justified without further assumptions. Three binary responses from the same subject generally remain correlated after conditioning on baseline covariates and treatment. The fitted independent-Bernoulli likelihood discards that correlation and gives incorrect model-based standard errors and tests if it is present. Positive within-subject correlation commonly makes those errors too small.
There is an important distinction between estimating the mean and estimating uncertainty. If the stated marginal logistic regression is correct and responses are fully observed, the independence score still has zero expectation. With independent subjects and regularity, its root can consistently estimate marginal coefficients despite within-subject dependence; a subject-level sandwich covariance matrix is then needed. True independence is a special case in which the original standard errors would be appropriate.
Dropout creates a separate issue. Restricting that score to observed responses need not preserve its zero expectation if observation depends on the response history or unseen depression outcomes. Therefore both point estimates and uncertainty may be wrong. Neither random treatment assignment nor baseline adjustment alone establishes an ignorable missingness mechanism. We need a model matched to the scientific estimand and explicit assumptions about missing data.
read.table(...,header=T) reads four grouped rows with column names. The displayed outcome counts give the observed risks in the indicated factor order. rbind in the first test pools over predisposition: the two rows represent no cannabis use and some use, with event/non-event counts and . Thus the pooled risks are and . This test concerns the marginal statistical association between use and outcome, not an association adjusted for predisposition.
Under independence, expected counts in the two-by-two contingency table are . The ordinary Pearson chi-squared statistic is . The printed is reproduced by the default two-by-two Yates continuity correction:
All absolute differences here exceed one half. Comparison with a chi-squared distribution on one statistical degree of freedom gives . There is strong evidence of marginal association. The simplified output omits the usual continuity-correction wording, but its numerical value shows which statistic was used.
attach makes the data columns available by name, and tot=with+without obtains group sizes . The grouped binomial regression fits , with response and prior weights . These weights tell R the binomial denominator; treating the four proportions as four equally precise individual observations would be wrong. The default logit link is
Let indicate some use and indicate predisposition. The additive logistic regression specifies . Its log-likelihood, including the binomial constants, is
Differentiation gives score equations and Fisher information matrix , where . Fisher scoring solves these equations, and the inverse fitted Fisher information matrix gives the reported approximate coefficient variances. The number of scoring iterations records numerical convergence, not degrees of freedom or statistical evidence.
The intercept corresponds to no use and no predisposition and gives baseline fitted risk . The additive coefficients imply common adjusted odds ratios:
They multiply the odds, not the risks. The Wald tests divide the estimates by their standard errors and use a standard normal distribution; both predictors have small two-sided p-values. Approximate confidence intervals for these odds ratios are and , obtained by exponentiating the log-scale intervals. The intercept test against zero asks whether the reference risk equals one half, and is not a test of either exposure association. The binomial dispersion is assumed to be one because ; the output does not demonstrate independence or rule out extra-binomial variation.
There are four grouped probabilities and three fitted coefficients, leaving one residual degree of freedom. The fitted risks are approximately . The binomial deviance compares this fit with the four unrestricted observed proportions:
The deviance goodness-of-fit test against has p-value . The common-odds-ratio fit is not rejected at , although the doubly exposed group's observed risk exceeds its fitted risk appreciably and suggests possible interaction.
The second formula adds . Its four coefficients fit all four probabilities, making a saturated statistical model with zero grouped residual degrees of freedom. The near-zero deviance is roundoff from exact interpolation; it supplies no remaining grouped lack-of-fit test. Unlike Q1's unreplicated Gaussian fit, however, these grouped responses represent many independent Bernoulli trials, and their binomial variances are specified by their sizes and probabilities. Coefficient standard errors remain available from binomial information without estimating dispersion from the zero residual degrees of freedom.
Indeed, let be the fitted cell logit link value. The saturated coefficients are
These calculations yield the printed estimates. The delta method gives , and independent groups make the variance of the sum of all eight reciprocal counts. Its square root is , reproducing the printed standard error.
The saturated logistic regression describes the conditional odds ratios as follows:
Thus the main coefficients now concern effects at the other factor's reference level. Logistic interaction as a ratio of odds ratios gives , with approximate confidence interval . The interaction Wald test has , while the nested likelihood-ratio test uses deviance difference on one statistical degree of freedom and has . The data suggest a larger odds association in predisposed subjects, but do not establish this interaction at the conventional level. This conclusion is specific to the odds scale; absolute-risk differences can vary even with no logit-scale interaction.
The Akaike information criterion is . The additive fit has and AIC ; the interaction fit has and AIC . The extra coefficient improves twice the log-likelihood by but incurs penalty , so AIC falls by only . It weakly favors the larger fit rather than decisively establishing an interaction, and AIC is not a p-value.
The marginal odds ratio and the fitted conditional odds ratios are different quantities. Predisposition is associated with both use frequency and outcome, so confounding is a substantive possibility; noncollapsibility of the odds ratio also means marginal and conditional odds ratios need not agree. This is an observational association analysis with only one adjustment factor. It does not by itself establish a causal effect, exclude other confounding, or validate the assumed independent binomial sampling.
The command fits independent grouped binomial distributions , using the supplied totals as trial counts. The logistic regression has the additive linear predictor
The reference category is adult females. The age and gender coefficients are conditional log odds ratios: the fitted odds for a child are multiplied by , and those for a male by .
The coefficients are maximum-likelihood estimates, found by Newton's method or iteratively reweighted least squares. With design matrix and , the estimated covariance matrix is ; the printed standard errors are square roots of its diagonal entries. Each Wald test compares a coefficient with zero using its estimate divided by its standard error, approximately a standard normal distribution under the null. The age and gender effects are strongly significant. The intercept test instead tests whether the reference probability is .
The fitted probabilities are about for adult females, for adult males, for female children, and for male children. Adult females have the largest fitted probability, about 4.48%. The residual deviance is on degree of freedom; comparison with a chi-squared distribution gives . There is no evidence here that an age–gender interaction is needed, under the assumed independent binomial distribution model.
Let indicate a depressed outcome and indicate the control arm. A logistic regression could specify
with containing prespecified baseline characteristics, including the stratification factor. Fit the coefficients by maximizing the likelihood for independent Bernoulli random variables . For equal covariate values the control-to-intervention odds ratio is . Thus an estimate produces the quoted adjusted ratio. An ordinary Wald confidence interval is obtained by exponentiating the endpoints of the coefficient's confidence interval.
The raw control-to-intervention odds ratio, using the actual counts, is
Using only the quoted percentages gives about , so both versions are approximately . The printed is not the usual whole-percentage rounding of ; using the counts avoids that small numerical inconsistency. The reversed intervention-to-control ratio is about .
The apparent surprise is a coding issue: with depression as the adverse outcome and intervention relative to control, protection would give an odds ratio below one. A ratio above one is favourable if the comparison is control relative to intervention, as above, or if the outcome has been recoded as absence of depression and intervention is compared with control. Both comparison and outcome must be specified. The adjusted ratio need not equal the crude one, even in a randomized trial: covariate adjustment and noncollapsibility of the odds ratio can change an odds ratio. Their close numerical values here do not prove that the regression adjustment was unnecessary.
Zero observed events do not mean that the underlying event probability cannot be estimated. For independent observations having a common Bernoulli distribution, the binomial likelihood with no events is , maximized at . A zero-event binomial upper confidence bound shows the remaining uncertainty: the one-sided 95% upper bound solves , giving
Thus the point estimate is zero, but a nonzero mortality risk is compatible with the observations. This bound is an illustration under independent equal-risk sampling; clustered data or unequal risks require their own uncertainty calculation.
The empirical unadjusted odds ratio is also defined:
Its log odds ratio is , so the ordinary normal approximation for a log-odds confidence interval is unavailable. Exact likelihood-based limits can still describe uncertainty; a specified continuity correction would instead yield a finite, method-dependent point estimate.
In an unpenalized logistic regression with a separate coefficient for elective exposure, every exposed death outcome is zero. Decreasing that coefficient toward increases the exposed observations' likelihood contributions and leaves the reference observations unchanged. This is quasi-complete separation: there is no finite ordinary maximum-likelihood estimate of the exposure coefficient, even after adding background predictors. It is reasonable to say that the usual finite adjusted coefficient could not be fitted; it is too strong to say that no risk estimate or statistical information is possible. A specified penalized likelihood or Bayesian inference with proper coefficient priors can provide a finite adjusted estimate, whose dependence on the regularization must be reported.
Write the logistic regression as
where denotes elective exposure and contains the measured background predictors. At fixed , changing from zero to one changes the log odds by , so the adjusted odds ratio is . If is the estimated coefficient covariance matrix, then
Use the clinic-cluster sandwich covariance matrix if that is how within-clinic dependence is allowed for. The interval is first calculated on the coefficient scale and then exponentiated; one does not add and subtract a standard error directly from the odds ratio. This is a large-sample Wald confidence interval; profile-likelihood limits are another option when the likelihood shape warrants them. If exposure interacts with background variables, there is no single common conditional odds ratio: its logarithm includes the corresponding interaction terms.
There is an apparent reversal, but not a mathematical contradiction. The crude odds ratio averages over the actual background distribution in each group, whereas the adjusted logistic regression compares groups at the same fitted background values. Strong confounding could make those distributions very different: for example, elective patients might disproportionately come from low-risk backgrounds or clinics, while the reference patients come from higher-risk backgrounds. The Simpson paradox can then reverse the marginal comparison even when every conditional comparison favours the same direction.
The noncollapsibility of the odds ratio can also make marginal and conditional odds ratios differ in magnitude. It does not, by itself, explain a sign reversal if both groups have the same covariate distribution and the common conditional effect has one sign: if for every , averaging over the same distribution of still gives a larger exposed risk. A genuine reversal therefore requires differing background distributions, heterogeneous effects or another change in what is being compared. The observed reversal is a reason to investigate these issues, not proof of a programming error.