Treat the two retrieved sets of positive tests as independent binomial samples, each of size 500, and use a two-sided test at significance level . The normal-approximation sample size for comparing two proportions, with equal numbers in each group, is
For , , this gives , or 356 per group. Thus 500 positives in each period are sufficient for 80% power under these assumptions.
More explicitly, a pooled two-proportion test rejects when
where pools the maximum-readout successes. At the proposed alternative, the difference has mean and standard deviation . The planning null critical difference is , giving approximate power
This compares the readout percentage among positive tests. It assumes comparable measurement rules and independent observations; a change in that percentage alone does not identify its causal explanation or the prevalence of drug use in the full tested population.
For pair , use conditionally independent counts and , with a common effect and a separate positive baseline rate for each physician. Equivalently the linear predictor is , with . This Poisson regression controls each physician's stable baseline propensity.
Condition on all twelve pair totals . The paired Poisson conditional likelihood has independent factors
This eliminates every exactly. It is more useful as a nuisance-elimination device than the unpaired calculation, which removes only one common baseline: many pair-specific rates are otherwise estimated from just two measurements each.
A possible concern is loss of information through conditioning on totals that are not ancillary for , especially with sparse pairs; a zero-total pair contributes no conditional information. There is an important qualification in this exact fixed-baseline Poisson model. Maximizing over gives , and substitution leaves precisely the same -dependent factor as the conditional likelihood. Hence there is no extra loss relative to ordinary profiling in this particular model; one should not automatically assert an incidental-parameter bias in its estimate of .
Additional baseline observations, a calibrated distribution of baseline rates, or informative hierarchical constraints can retain information in the pair totals. A Gamma–Poisson hierarchical model is one possible structured model, but merely giving it a completely free common mean does not automatically create extra information about : that mean can absorb the factor in the totals. Such remedies trade exact nuisance elimination for additional data or assumptions. Finally, paired sampling alone does not establish conditional Poisson independence or equidispersion: remaining within-physician dependence or overdispersion should be checked, and an appropriate joint count model or physician-level robust variance used if needed. The common multiplicative-effect assumption is also part of the analysis.
The solid Poisson regression profile and dashed negative binomial regression profile peak at essentially the same . The dashed profile is slightly wider and has heavier shoulders: it gives larger relative likelihood to values farther from the common maximizer. Thus the estimated mean ratio is stable, while the negative binomial model expresses slightly greater uncertainty.
The Poisson distribution constrains variance to equal the mean. A common negative binomial parametrization instead has , with additional shape ; it permits overdispersion due, for example, to unexplained between-physician heterogeneity. Profiling this extra parameter lets the data support greater count variability and usually reduces information about the mean contrast. In this two-group model, the fitted group means remain the sample means for fixed , which explains the nearly identical maximizers. The small separation of the profiles indicates a modest effect here, not overwhelming evidence for strong overdispersion. The comparison concerns the original PDF figure; no figure is reproduced.
Let and be the aggregate counts in the control and treated groups. Assuming independent physicians, their Poisson distributions have means and . Condition on . By independent Poisson conditioning,
The PDF's full table gives , and . Thus the Two-group Poisson ratio conditional likelihood is
Conditioning the individual observations instead gives a multinomial distribution with cell probabilities in the first group and in the second; its parameter-dependent likelihood is the same. No factor involves . The conditional estimates are and .
The mean ratio is , so its logarithm is exactly ; the baseline coefficient cancels. Under the stated asymptotic normal approximation, the Wald confidence interval for is . Exponentiation is monotone and therefore preserves the interval's coverage event. Hence
The standard error of is not needed for this confidence interval: the Poisson regression already supplies the standard error of the contrast . It would be incorrect to combine the two reported standard errors as if the mean ratio depended on both coefficients independently.
An orthonormal multiresolution analysis is a sequence of closed subspaces satisfying , , , and iff . A scaling function has integer translates forming an orthonormal basis of , and its scaled translates span . Thus contains information at resolution about .
The detail space is generated by the corresponding wavelet functions . The orthogonal splitting iterates to
Accordingly a square-integrable signal has a coarse-and-detail wavelet series,
and its wavelet projection onto keeps the coarse coefficients and details below level . Increasing adds finer-scale oscillations and localized features. On the real line there are infinitely many translates at each level; for a bounded observation interval, with an appropriate interval or periodic construction, the relevant approximation dimension satisfies . For the Haar wavelet on a unit interval this is exactly , corresponding to a separate mean on each dyadic subinterval.
This hierarchy gives a concrete meaning to wavelet approximation complexity. Coarse resolution uses few coefficients and can miss sharp features; fine resolution uses more coefficients and can fit noise. By orthogonality, projection bias is the energy of the omitted details,
For an orthonormal sequence model with independent coefficient noise of variance , estimating the retained coefficients without shrinkage gives
The first term decreases with resolution while the second increases, making the bias-variance tradeoff explicit. Equivalently, a fixed orthogonal regression projection of rank has effective degrees of freedom and prediction optimism under independent equal-variance errors.
Wavelets also permit local rather than only global complexity control. Thresholding small estimated detail coefficients, for example by soft thresholding, retains strong local structure while suppressing fine-scale noise elsewhere. A sparse retained coefficient set can represent localized features much more economically than keeping every coefficient through a high level. Its locations and scales matter as well as its count. If that set is selected using the same noisy responses, however, selection adds adaptivity: its raw coefficient count is not automatically its covariance-based degrees of freedom. Resolution or threshold tuning should therefore account for prediction error, using an appropriate risk estimate or cross-validation, rather than only decreasing training error.
Wavelet complexity is controlled by retained resolutions and coefficients; more detail reduces approximation bias while increasing estimation variance and sensitivity to noise.
Condition on the training covariates, and let . The fresh response has the same marginal distribution as and is independent of the complete training response vector, hence independent of . Assume the relevant second moments are finite. Expanding the squared errors gives
The first expectation averages over both training and new responses; the second averages over training responses only. Subtracting cancels both second-moment terms, leaving
Averaging over the design points proves the covariance formula for prediction optimism:
No linear fitting rule or Gaussian noise assumption is needed. The difference arises because a training response and its fitted value are dependent, whereas a fresh response and the fitted value are independent. Ordinary fitting often makes those covariances positive and the training error too favorable, explaining the name prediction optimism.
For example, if is a fixed linear smoother and , then . Thus
This links the optimism correction to effective degrees of freedom. For a data-selected fitting operator, its realized trace alone does not generally give the covariance penalty, since selecting the model uses the responses too.
For a binary hypothesis class , a set of distinct points is shattered if every one of its binary labelings is realized by some member of . The VC dimension is
with value infinity if arbitrarily large finite sets are shattered. For real-valued scores, apply this definition to the associated binary decision functions, for example their signs. The word “some” matters: shattering every set of that size is not required.
As an example, take the interval classifiers on the real line, . Two ordered points can receive labels , , or by choosing an interval away from both, around the first, around the second, or containing both. Hence the class shatters two points. No three ordered points can be shattered, since any interval containing the first and third also contains the middle, making labeling impossible. Thus the VC dimension of interval indicators is
This describes combinatorial classification capacity, rather than merely counting written parameters; the definition applies equally to nonlinear or infinite-parameter classes.
A random forest constructs an ensemble of classification and regression trees. For each tree, draw a bootstrap sample of the training observations. Grow the tree by recursive partitioning; at each node, draw a fresh random subset of predictor coordinates and choose the best split only among those coordinates, using a criterion such as Gini impurity. Trees are ordinarily grown deeply with a chosen minimum terminal-node size and without the pruning used to select a single small tree. Repeat for many independently randomized trees. For binary classification, predict by majority vote; for regression, average the tree predictions. The out-of-bag error predicts each observation using only trees whose bootstrap samples omitted it, furnishing an internal assessment of prediction error.
The strategies differ in how they form a useful ensemble. The forest uses resampling and random predictor subsets to decorrelate trees; averaging then reduces the variability of individual tree predictions. For illustration, if equally weighted predictions have common variance and pairwise correlation , the variance of their average is , showing why low inter-tree correlation matters. This identity describes a regression-prediction calculation; it is not an exact classification-error formula.
AdaBoost instead fits sequentially, with each round responding to errors of the accumulated score. It uses nonuniform observation weights and nonuniform final classifier weights derived from exponential loss; a forest's bootstrap sampling is ordinarily uniform and its final votes are ordinarily equal. Forest trees can be built in parallel once the data and tuning parameters are fixed, whereas each boosting stage depends on previous stages. Random forests commonly use deep trees; AdaBoost often uses shallow trees or stumps as weak learners, although its base class is not restricted to them. Reweighting persistent errors can make boosting sensitive to mislabeled points and outliers, while randomization and averaging often make a forest less sensitive; neither statement is a universal robustness theorem. Both methods require attention to ensemble size and tree complexity, and neither guarantees a better classifier for every dataset.
Random forests diversify and average randomized trees; AdaBoost sequentially fits errors and combines classifiers with loss-derived weights.
The exponential classification risk for the training sample is . At the start of round , define
Thus the observation weights arise directly from each point's current exponential loss. For a candidate base classifier and coefficient ,
where is its error under . For a fixed the bracket is increasing in , so the appropriate base fit minimizes weighted classification error. With fixed, differentiation gives
The second derivative is positive, hence the exponential-loss derivation of AdaBoost yields
The normalizer is . Consequently the next normalized weights again satisfy , proving the update by induction from uniform initial weights. The factor by which an error gains weight relative to a correct observation is .
This calculation also explains why repeated weak fits can improve training performance. Since the initial mean exponential loss is one, its value after rounds is . A misclassified point has and hence exponential loss at least one, giving the AdaBoost training error bound
The last inequality uses . This is a training guarantee under positive edges, not a guarantee that further rounds always improve prediction on new observations.
Use labels and base predictions . The binary AdaBoost algorithm starts with observation weights and score . For rounds :
  • Fit the weak learner to the training observations using weights , producing .
  • Compute its weighted error .
  • For , choose and update .
  • Reweight and normalize: , where is the sum of the unnormalized weights.
The final classifier is
Resolve a zero-score tie by a fixed convention. A classifier with error above may be sign-reversed if the base class permits it; otherwise it fails the weak-learning requirement and should be replaced. Error gives zero weight and no improvement. Error zero gives the limiting perfect-classifier step with infinite coefficient, so terminate with that classifier rather than computing the undefined finite update. Misclassified observations are multiplied by and correctly classified observations by , increasing the relative weight of errors for the next round. The procedure is sequential, and its number of rounds is a tuning parameter.
The ridge regression penalty is quadratic, whereas the Lasso uses an absolute-value penalty. To make the constants explicit, compare
Both trade goodness of fit against coefficient size, and both require sensible predictor scaling and tuning. Ridge continuously shrinks coefficient directions and generally retains all variables. The Lasso's nonsmooth penalty can set coefficients exactly to zero, so it performs variable selection as well as shrinkage. When , let . The distinction is especially transparent:
The second formula is soft thresholding. For general designs, the Karush-Kuhn-Tucker conditions for the Lasso give for a nonzero coefficient and for a zero coefficient. The latter interval permits exact zeros; the ridge equation instead is .
Ridge is especially useful for stable prediction when many predictors carry small effects or are highly correlated. Its strictly convex penalty gives unique coefficients for even in a rank-deficient design. It shares an effect among identical predictor columns: for a fixed coefficient sum the minimum quadratic penalty splits that sum equally. The Lasso can give a simpler sparse model when many effects are absent, but selection among strongly correlated predictors can be unstable. With identical columns, any same-sign split of a selected coefficient sum has the same absolute-value penalty and the same fitted values, so the coefficient vector need not be unique. This is a possible design degeneracy, not a claim that every Lasso fit is nonunique.
The penalties also encode different prior shapes: the Gaussian prior yields ridge, while independent double-exponential priors yield a Lasso posterior mode under Gaussian noise. A Lasso mode is not generally its posterior mean. Practically, sparse interpretability favors the Lasso, while stable distributed effects can favor ridge; prediction performance should be checked with cross-validation, and neither method universally dominates the other.
Use the Gaussian sampling model with known , and the independent prior with . A Gaussian prior alone is not enough to imply the ridge result without this likelihood assumption. The posterior distribution is proportional to
Maximizing it is equivalent to minimizing the ridge objective with . More explicitly, complete the square. The posterior precision, covariance matrix and mean are
Consequently the Gaussian posterior representation of ridge regression is
A tighter prior, smaller , implies stronger shrinkage; larger observation-noise variance at fixed prior scale also increases the penalty. The posterior covariance is , not the frequentist sampling covariance of the ridge estimator, which is . If an intercept is to remain unpenalized, give only the slopes this proper Gaussian prior and center the design and responses.
For an design matrix, distinguish the coefficient vector from the fitted-response vector. The ridge regression objective and its solution are
The gradient is . For , is a positive-definite matrix, so this is the unique minimizer even if is rank deficient. The expression with an additional leading is instead the -vector of fitted values,
This distinction corrects the dimensional mismatch in the printed coefficient-estimator notation.
The quadratic penalty discourages large coefficients, exchanging some bias for lower variance. If is a reduced singular value decomposition, with positive singular values , then
The principal-component shrinkage by ridge regression therefore shrinks directions with small most strongly. These are precisely the poorly determined directions created by multicollinearity, where ordinary least-squares coefficients can have large variance or fail to be uniquely determined. A positive penalty stabilizes their inversion and sets coefficients in the null space to zero. It does not make the predictors independent or provide a universal risk improvement for every signal. Because the penalty depends on coefficient units, predictors are ordinarily standardized before choosing by cross-validation. An unpenalized intercept in ridge regression is handled separately by centering; the displayed objective either has no intercept or applies the penalty to every component included in .
To isolate the bias-variance tradeoff of the smoothing step, fix the three neighbor locations and write , with independent zero-mean errors of common variance . At a target , the local estimate is
The bias and variance of a three-neighbour weighted smoother are
Consequently the variance is minimized by equal weights, , when it equals . It decreases as increases from zero to , and increases thereafter toward as . In particular, variance is not increasing over the entire interval .
Giving more weight to the closest location reduces the influence of more distant responses and often reduces smoothing bias. If the closest location is the target itself, , put . Then the squared bias is , and
Thus a locally flat signal favors equal averaging, while stronger local smoothing bias favors greater nearest-point weight. At a general target the bias is affine in , and its absolute value need not decrease: farther neighbors can cancel one another's approximation errors. Even without that cancellation, a smoothness-based worst-case bias bound decreases with because the first neighbor is closest. The exact tradeoff is therefore lower variance near balanced weights versus potentially lower bias near the closest-point fit. These formulas describe the elementary smoother; fitted partial residuals, centering and repeated backfitting can introduce dependence, so the variance of the final additive fit requires its full fitting operator rather than this single-step calculation.
There are two distinct identifiability issues. Without centering, constants can move between the regression intercept and the additive components. After centering, the decomposition still fails to be unique if there are nonzero allowed centered perturbations satisfying
Then replacing by leaves every fitted response unchanged. This nonlinear analogue of multicollinearity is concurvity. For example, if two predictor columns are identical, a centered function can be added to the effect of the first and subtracted from the effect of the second. Conceptually, the data observe their combined effect but cannot tell which predictor deserves the credit. A unique fitted sum can therefore coexist with nonunique individual effects. Smoothness penalties can distinguish some such decompositions, so a representation ambiguity is not automatically an ambiguity of every penalized estimator.
For a precise criterion for the backfitting algorithm with linear smoothers, put , , and . On the centered component space its fixed-point equations are the linear backfitting equations
A consistent system has more than one solution exactly when there is a nonzero centered tuple satisfying for all . In the two-component case, elimination gives
Thus
A common centered direction reproduced exactly by both smoothers, , gives the homogeneous perturbation and is a concrete instance. For example, two identical predictor columns with smoothers that reproduce the same centered linear trend have this property. Approximate concurvity yields an eigenvalue near one, causing poor conditioning and slow or unstable component estimation, but it does not constitute exact nonuniqueness. Failure of an iteration to converge is also distinct from the existence of multiple fixed points.
Write for the training values of each component in the additive regression model. Separate the regression intercept from the smooth components by imposing for every . Without these constraints, adding a constant to a component and subtracting it from the intercept gives exactly the same mean function. A population alternative is when those expectations exist. Additional identifiability of additive regression components requirements exclude redundant component directions, as discussed below.
The backfitting algorithm proceeds as follows:
  • Initialize and for all , or use another centered starting fit. Choose the smoother and its tuning parameters for each predictor.
  • Cycle through . Form the partial residual , using the newest available component estimates.
  • Apply the generic smoother in predictor : . Center its fitted values and replace the current component by . The same centering constant is subtracted from its function when predicting at a new predictor value.
  • Repeat complete cycles until the components and fitted sum change by less than the chosen numerical tolerance. Return .
Because every component remains centered, the least-squares update of the intercept remains throughout. The essential update is
For penalized least-squares smoothers these updates are coordinate descent on the additive fitting objective. For a completely arbitrary smoother, convergence is an additional property to establish, not a consequence of merely specifying the iteration. Centering removes the constant ambiguity but does not by itself remove concurvity.
Model B replaces independent tree parameters by multivariate normal random effects. It permits correlated asymptote, timing and slope variation and supplies partial pooling, a useful improvement if the five trees can be regarded as exchangeable.
The Wishart distribution prior is on the precision matrix , not the covariance matrix. In the WinBUGS convention, dimension three and degrees of freedom three give a proper but very dispersed prior. The induced inverse-Wishart distribution has no finite mean at those degrees of freedom. Thus this is not a neutral choice of covariance prior, and estimating six covariance components from only five trees is weakly informed. The broad mean priors and possible negative growth slopes also remain. Check prior-predictive curves and sensitivity, and consider separate, interpretable scale and correlation priors.
The resulting posterior means of the second and third parameters are quite similar across trees, motivating a simpler shared-shape model; their uncertainty means this similarity is evidence to explore, rather than a proof of exact equality.
The priors are proper but difficult to justify as practically weak information. Uniform bounds on log asymptotes cover fantastically small and large circumferences. The independent prior on the transformed location makes the centered fraction spend much of its mass near zero or one, while the slope prior permits decreasing growth. Independence also ignores likely relationships among curve parameters.
The prior on error precision is proper, but its near-zero shape creates a highly skewed prior with consequential mass near very small precision. It is not automatically harmless. Use biologically scaled priors for asymptotes and timing, a positive growth-rate prior, and a defensible prior on the error standard deviation such as a suitably scaled half-normal distribution. Check simulated trajectories with prior predictive checks and assess prior sensitivity.

Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 5. . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact