Principal component analysis replaces the original coordinates by orthogonal linear combinations, ordered so that the first few retain as much variance as possible. It gives a lower-dimensional summary for visualization or compression and reveals the main directions of variation. A direction of large variance need not be the direction most useful for classification.
Write . For a unit vector , the centered score has variance
Maximizing this Rayleigh quotient with a Lagrange multiplier gives
Thus a maximizing direction must be an eigenvector of the covariance matrix. To identify the maximum, use the spectral theorem for real symmetric matrices to choose an orthonormal basis of eigenvectors , with eigenvalues . Writing gives
Equality is attained at . Consequently the first population principal component is
For each subsequent principal component, maximize the same variance over unit coefficient vectors orthogonal to all the previously chosen ones. Restricting the expansion above to the remaining eigenvectors gives , then , and so on. Thus
The principal components are uncorrelated; they need not be independent unless additional distributional assumptions, such as joint Gaussianity, hold. Their total variance is . Signs of eigenvectors are arbitrary, and a repeated eigenvalue permits any orthonormal basis in its eigenspace. In sample principal component calculations, replace by the sample mean and by the sample covariance matrix.
The inequality in the printed rule is reversed. With the stated definition of , a larger score favors group one. For example, in one dimension take , , , equal priors and . The score is , and , whereas the printed rule assigns group two. The correct Bayes classifier follows from minimizing the error cost as follows.
Let be the region assigned to group one, let be its multivariate normal density, and let each misclassification cost , with zero cost for correct classification. The expected cost is
The permitted integral-minimization result therefore selects . Points of equality may be assigned to either group without changing the risk. Equivalently, the Bayes classifier chooses the larger posterior probability.
For positive priors and a common positive-definite covariance matrix, the normalization constants cancel in the likelihood ratio. Expanding the two quadratic forms gives
Thus the correct Gaussian Bayes classifier, choosing group one on ties, is
This proves the intended classification result with the necessary correction. Equal error costs remove from the decision, while prior odds determine the threshold.
Assume an independent Gaussian sample, positive-definite population covariance, and . Define the sample mean and unbiased sample covariance matrix by
The standard test is based on Hotelling's T-squared statistic
Reject the null hypothesis for large values. Under the null its exact distribution is
The factor uses the unbiased, divisor- covariance convention. Normal-sample theory makes independent of the residual covariance, whose whitened version has a Wishart distribution; this yields the stated F-distribution. The condition gives invertibility of almost surely and positive denominator degrees of freedom.
This is also the likelihood-ratio test. To see the connection, put and . The maximized covariance estimates under the unrestricted and null models are and , respectively. The determinant identity for a rank-one update gives
Thus the likelihood ratio is , decreasing in . The term best is understood here as this standard likelihood-ratio, affine-invariant procedure; a two-sided multidimensional alternative does not specify a single direction to optimize power against.
For the transformed sample , with the prescribed nonsingular coordinate transformation, we have
Consequently
This proves the affine invariance of Hotelling's statistic. The statistic, its p-value and its test decision are unchanged by a nonsingular affine transformation, provided the null mean is transformed with the data. Changes of units and rotations are included.
Take the coordinate-selection matrix
Then , and . The permitted result on a linear image of a multivariate normal vector therefore gives
This is the marginal distribution obtained by discarding the other coordinates. The conclusion holds also for a positive-semidefinite marginal covariance: in that case the Gaussian law may be degenerate and need not have a full-dimensional density. If the original covariance is positive definite, its principal block is positive definite as well.
Let . Independence of the standard normal distributions gives , , and joint density
Choose a matrix with , for example the lower-triangular factor from the Cholesky decomposition. It exists and is nonsingular because is a symmetric positive-definite matrix. Then the required simulation is
It has mean and covariance matrix , and every linear combination of its coordinates is Gaussian, so it has the required multivariate normal distribution.
For the multivariate normal density, apply the change of variables formula to . Its absolute Jacobian determinant is , and
Hence
The nonsingularity assumption is needed for a density with respect to -dimensional Lebesgue measure.
The accepted mutation parameters have marginal probability density function
Estimate this probability density function with a suitably smoothed histogram or a boundary-aware kernel density estimator from the accepted draws. On a parameter region where , divide the estimate by the known prior density and maximize:
The unknown is constant in and cancels. Maximizing the posterior density itself gives a posterior mode, not generally a maximum-likelihood estimate; the two coincide for a flat prior on a region containing the maximum. A prior excluding candidate maximizers cannot recover them through its posterior output. Boundary behavior also matters: for , the likelihood is decreasing in , so the unrestricted maximum is at .
The exact likelihood provides a useful independent check. Put . Its Laplace transform is , since is exponential of rate . The probability density function
has that same transform: substitute to obtain . Thus, expanding its binomial factor and integrating each term,
This is the segregating-site likelihood under the neutral coalescent. It can be maximized numerically to check the posterior-density estimate; the generating-function representation avoids cancellation in large alternating sums. For , it reduces to , whose maximum is at for .
Alternatively, differentiating the integrated likelihood gives
An interior maximum therefore satisfies . Conditional expectations of can be estimated by smoothing the accepted pairs near each candidate parameter value, yielding another simulation-based score equation. The simple estimator is the unbiased Watterson estimator from part (b), and is not generally the likelihood maximizer.
The rejection rule can use a tighter envelope than 1. For , differentiate . Its derivative changes from positive to negative at , so
Keep the same joint prior proposal but accept with probability . Multiplication of the acceptance function by this constant leaves the accepted probability density function unchanged. The improved rate is
For , , so this is a genuine improvement; for large , Stirling's formula gives . This is the best global constant envelope for an unrestricted positive prior proposal, because has support and can approach . For there is no improvement from this constant scaling. Further gains can come from proposals concentrating near , or integrating out the times first, but such changes require the appropriate proposal-density ratio and envelope rather than an uncorrected alteration of the sampling law.
The acceptance probability is the prior average of the Poisson likelihood:
Thus the expected number of proposals per accepted draw is . A numerical rate depends on the specified prior, sample size and observed count; it is not determined by alone.
For a more explicit one-dimensional expression, condition on . The probability generating function, integrating each independent exponential epoch, is
Let be its coefficient of . Then . This identifies the acceptance rate with a fully specified marginal likelihood average.
Use Bayesian rejection sampling for a segregating-site count. Independently propose and for , and calculate . Draw an independent uniform on and accept the whole proposed vector exactly when
Repeat after a rejection. The right side is a probability mass function value of a Poisson distribution and is therefore between zero and one. On acceptance, the probability density function of a proposed vector is its prior density multiplied by this acceptance probability, divided by the probability of acceptance. Part (c) shows that this is precisely . Independent proposals and uniforms produce independent posterior draws. Equivalently, one could simulate a count from a Poisson distribution with mean and retain the proposal when that count equals ; the uniform version avoids simulating that extra count.
Assume , , a proper prior density supported on nonnegative mutation rates, and independence of the mutation-rate prior from the neutral genealogy. Put
By Bayes' theorem, the joint posterior is
The normalizing constant is the prior predictive probability and must be positive. For , interpret the factor as 1 even at zero. If , there are no segregating sites: surely, so that case supplies no information about .
In the infinite sites mutation model, mutations occur as independent Poisson processes at rate per lineage per unit of the coalescent time used in part (a). Conditional on the genealogy, superposition over branches gives a Poisson distribution with mean . Since this depends only on , conditioning on total length alone gives
Each mutation occurs at a new site and below the common ancestor, so it contributes exactly one segregating site. The law of iterated expectation therefore gives, with ,
Here is fixed when taking the expectation. If it is also random under a proper prior with finite mean, the unconditional expectation is .
Use standard neutral coalescent time units, so each pair of ancestral lineages merges at rate 1. With lineages, there are possible pairs; consequently the successive waiting times are independent with
During this epoch there are branches, each of length . Hence the total branch length of a neutral coalescent is , and linearity of expectation gives
The root branch above the sample's common ancestor is excluded: its mutations would be shared by every chromosome and would not be segregating sites in the sample.
Under the original assumptions the extra affected child is impossible. A generation-3 woman can have at most one copy of the unique introduced disease allele, while an unrelated outside spouse has none. Their children cannot inherit two defective copies. Thus both linkage and no-linkage likelihoods of the enlarged data are zero, and the original single-entry model has no defined updated LOD score. At least one assumption must be relaxed: another introduced disease allele, a new mutation, a phenocopy, or the relationship of the outside spouse.
If the model is enlarged to allow an independent outside genetic carrier, the reported extra information contains disease status but no new genetic marker genotypes. In the simple extension with fixed outside heterozygous-carrier probability , the old disease observations force both generation-2 family ancestors to be genetic carriers. A further generation-3 sister inherits the disease copy with probability , independently of the existing offspring transmissions. Given that she and her spouse are genetic carriers, exactly one of their three children is affected with probability
The additional phenotype likelihood is therefore . It is the same at and , so it cancels and the LOD remains unchanged in this phenotype-only, independent-carrier extension. If the identity of the affected child is specified, the factor is , which also cancels. No such conclusion applies automatically if new genetic marker data are supplied or the revised pedigree founder model couples this branch to the old genetic marker evidence.
In general an enlarged data set changes the score by
after choosing a coherent model with positive likelihood. The additional affected child alone does not justify assigning it the autozygosity evidence of an affected child of a cousin marriage.
Write for the disease observations and for the genetic marker observations. The LOD score for complete linkage is
The disease marginal cancels because the single-locus disease segregation law does not depend on the genetic marker–disease recombination fraction. The ten genetic markers form one nonrecombining haplotype; they do not supply ten independent copies of the pedigree's segregation evidence.
There is a distinction between identity by state and identity by descent. The reported observation that affected subjects all have establishes the former. An exact genetic marker likelihood requires pedigree founder haplotype frequencies, or the actual pedigree founder genotypes if conditioning on them. Here is an explicit expression using independent pedigree founder haplotypes, with frequency for the observed haplotype , and linkage equilibrium between pedigree founder genetic marker and disease states. There are eight genetic marker copies in the four pedigree founders: the original couple and the two generation-2 outside spouses. For genetic marker inheritance configuration , let be the number of distinct pedigree founder copies ancestral to the sixteen chromosome copies in the eight selected subjects. Then
is the unlinked probability that all those copies have state . At complete linkage the affected subjects all inherit the disease pedigree founder copy twice, and the probability that this copy has genetic marker state is . Therefore
The denominator can be evaluated without an unspecified inheritance sum. Let for . If parental genetic marker genotypes contain copies of , the child count distribution is
For a generation-2 couple define
For a particular generation-3 parent with genetic marker copies, the chance it transmits to all its selected offspring is , which explains each factor. Conditional on the original founding couple, its two generation-2 siblings and their independent outside spouses give independent left and right contributions. Thus
Expanding gives
It satisfies , as it must for a monomorphic haplotype. This expression supplies a numerical LOD once is specified; the problem does not specify that frequency.
In the idealized limit where a matching rare haplotype identifies one ancestral copy, the leading coefficient is particularly simple. Dropping restrictions on the eight unaffected subjects, the probability that all selected subjects are autozygous for one common copy is
Accordingly as , and the affected-only IBD calculation gives .
If the genetic marker observations additionally establish the exact pattern of part (d), including that the eight unaffected subjects are not autozygous, its idealized IBD likelihood ratio is instead
This latter value requires the additional negative genetic marker observations. It cannot be inferred merely from homozygosity in the affected subjects, and neither simplified IBD score is an exact identity-by-state likelihood for an unspecified common haplotype.
Let denote the specified disease pattern. With one introduced disease-allele copy, no new mutations or phenocopies, and fully penetrant recessive inheritance, every affected subject must carry two descendants of that one copy. Since every sibship has an affected child, all the upstream genetic carrier transmissions described above are forced. Conversely, unaffected siblings cannot carry two disease copies. Thus implies the inheritance event at the causal genetic locus, and
The rarity of the unconditional pattern in part (d) does not make its posterior probability small after observing the pattern that forces it. If the location of the one disease copy is initially uniform among the four founding copies, the likelihood of is the same for each, so its posterior location remains uniform; for any named copy. If is already designated as the disease copy, its corresponding event has posterior probability 1.

Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 5. . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact