Past exam of the communication Theory course of the University of Oxford 2011 1 c Solution 2026-10-31
Write . For each row put ; the positive row entries form a probability distribution because the row sum is one. When , define the probability distributionPart (a), applied on this row support, gives . Its weighted sum expands asRows with contribute zero, since all in such a row vanish. Using the convention that zero-weight logarithmic terms are zero, the result isBoth stochasticity conditions have a role: row sums normalize the comparison distributions, while column sums preserve the total contribution from .
The accounting method assigns each operation a charge, its amortized cost, that can differ from its actual cost. Cheap operations save the excess charge as credits associated with the data structure or its entries. An expensive operation spends these previously saved credits. The essential invariant is that the credit balance never becomes negative.
If the initial balance is zero, operation has actual cost , charge , and leaves balance , thenThus a nonnegative credit balance makes the total charge an upper bound on the total actual cost for every operation sequence. In particular, a constant time charge proves a linear time total. The argument is a deterministic amortized analysis; it requires no probability distribution on the operations. A nonzero initial balance must be included as an additive term in the bound.
The two-interval approximation by one interval supplies an approximation guarantee for every input probability distribution . Merge overlapping target intervals, and write the target positive set as at most two disjoint ordered intervals . Put and . If there is only one nonempty component, the target already belongs to the interval hypothesis class and has zero misclassification risk. If the interval notation permits empty intervals and the target is empty, choose that same empty interval in .
In the two-component case, choosing the heavier component as the hypothesis loses only the lighter component, so its misclassification risk isAlternatively, choose the single interval spanning both components. This produces no false negatives, and its false positives lie in the gap. Its misclassification risk is at most . Both candidates are in , so there exists an interval withFor the last step, use if , and if . This avoids any assumption that an empty interval or the whole real line belongs to .
Use empirical risk minimization over intervals on independent labelled examples. The generalization result needed here is the VC uniform convergence theorem: for a binary hypothesis class of finite VC dimension , there is a universal constant such that, for any labelled-data probability distribution and , a sample sizeensures, with probability at least , that every hypothesis's empirical misclassification risk differs from its misclassification risk by at most . This theorem does not require the labels to be realizable by the hypothesis class.[ref]
Learnability and VC Dimension in MIT 6.790. Sections “Uniform Convergence implies Agnostic PAC Learnability” and “Fundamental Theorem of Learnability”
The interval hypothesis class has VC dimension two: any two ordered points can receive all four labellings, whereas an interval cannot label three ordered points . Take and choose an interval minimizing the empirical misclassification risk. On the simultaneous convergence event,ThereforeSince and are fixed constants, examples suffice, which is polynomial in as required by the paper's definition of weak PAC learning.
For completeness, empirical risk minimization over intervals is computationally effective. Sort the sample and group equal coordinates, since one interval cannot split identical inputs. Give each group weight equal to the number of positive labels minus the number of negative labels in that group. The number of empirical mistakes made by an interval is the total number of positives minus the total weight of the consecutive groups it contains. Select the consecutive block of maximum weight, or select a finite interval outside the sample if every nonempty block has negative weight. This enumerates all possible sample labellings by intervals and attains the minimum empirical misclassification risk; sorting and a maximum-block scan are sufficient. The time complexity is .
Write , , and . Correctly classified observations have and total -weight ; incorrectly classified observations have and total weight . The normalizing constant in the AdaBoost update is thereforeFor , the definition of givesIf observation is correct, its multiplier in the normalized update isif it is incorrect, the multiplier isThe corresponding denominators are exactly : they are when and when . ConsequentlyAs a check, the new total mass on correctly classified observations is , and the mass on incorrectly classified observations is . The update thus produces a probability distribution and assigns the two groups equal aggregate mass. When , and every individual weight is unchanged.
The displayed finite update assumes . The usual weak learner guarantees , but it can return a perfect classifier with ; then the logarithm is infinite and the normalized exponential formula is not defined. AdaBoost stops on that perfect hypothesis rather than executing this update. Similarly, the printed formula is not an ordinary finite update at .
Perfect secrecy means that observing the ciphertext changes no probability concerning the plaintext. For every distribution of the message , and every with ,Equivalently, for every pair the conditional probability distributions of the ciphertext are identical:The randomness here comes from independent key generation and any encryption randomness. The claim is information theoretic: unlimited computation cannot extract information about from beyond what was known beforehand.
Choose a positive integer-valued random variable with probability mass functionThese masses are positive. They sum to one because the partial fraction decompositionmakes their sum a telescoping series with value . Its expectation is another telescoping series:By contrast, since and for , its second moment satisfiesThe comparison uses the divergent harmonic series. To check the variance directly, for we have , so the corresponding tail of also diverges. ThusHere “unbounded” means infinite variance for this single probability distribution.
The independence of and permits factoring the expectation of the product . The moment-generating function of a binomial distribution from the previous part therefore givesThis is the moment-generating function of . Both moment-generating functions are finite on a neighbourhood of zero, so the uniqueness theorem for moment-generating functions identifies the probability distribution:The common success probability and independence are both used in this argument.
For a finite normal-form game, a mixed strategy is a probability distribution over player 's pure strategies. Independent randomization by the players defines an expected payoff . A strategy profile is a Nash equilibrium iffor every player and every alternative mixed strategy . By linearity in a player's own distribution, it is enough to check all pure deviations. Equivalently, every pure strategy in the support of a mixed strategy must be a best response to the other players' choices.
The phase oracle multiplies an unmarked basis vector by and a marked one by . Thus the state after the oracle isIts squared norm is . All computational outcomes would still have probability if measured at this point: the oracle has changed the relative phase, not the magnitudes of the amplitudes. The following reflection converts that phase information into a changed probability distribution through quantum interference.
This is a finite, discrete, partially observable, single-agent, sequential search problem with an unknown initial state. The cave is static while the agent deliberates. Given a complete world and an action, the described physical effects and ideal sensor readings are deterministic; ignorance of the world does not itself supply a probability distribution. The fixed monster and gold are environmental objects, not competing decision makers. A known action/sensor law with unknown object locations is different from having to learn an unknown search transition model.
Use partially observable search over belief states. A complete physical state can be writtenwhere is the agent’s room, the fixed monster room, the pit set, the fixed gold room, records whether the monster lives, whether the arrow remains, whether gold has been collected and whether the agent has exited or failed. The known room dimensions and exit are fixed data. Include death or entrapment as failure states. Movement, shooting toward an adjacent room, collection and departure have their physical meanings. In particular, a shot consumes the arrow, and kills the monster exactly when it targets the live monster’s room.
Let be this physical search transition model. For a surviving agent, an ideal observation is the tuple of three truth valuesrecording sound, breeze and glitter. Here denotes geometric adjacency and an indicator function. Initially contains all worlds compatible with the known facts and any initial observation. After action and observation , update the belief state byKnown success or failure observations are included as needed. No prior probabilities are required. A guaranteed-success goal test accepts only when every has exited carrying gold. A policy must use the same action in all worlds still indistinguishable to the agent.
An AND-OR search has OR choices for the agent’s next action and AND branches for every possible resulting observation with nonempty . Its output is a contingent plan, not merely a route. To demand safe success, reject any action whose possible outcomes include irreversible death or entrapment, and require a successful continuation on every observation branch. In an online execution the agent updates its belief state, follows the observation branch and can plan again. A strong plan need not exist: some admissible worlds have no safe route, and even solvable worlds can be observationally indistinguishable until a dangerous choice. Without an added prior and numerical rewards, expected-risk optimization is not determined by the description.
For a discrete memoryless channel with transition probabilities , the conditional probability of a length- output given a length- input factors asThe same transition probabilities apply at each use. Its channel capacity isThe unit is bits per channel use. For finite alphabets the maximum exists by continuity on the probability simplex; with more general alphabets one uses a supremum when appropriate.
An channel code consists of an encoder and a decoder . It represents each of messages by input symbols and recovers a message from the observed output symbols. Its code rate is bits per use. For each message , its conditional error probability is ; an average error also specifies the message probability distribution, while the maximum error takes the largest conditional error.
Write . Because is determined by , the chain rule for mutual information givesThis is the data processing inequality. It holds for every input probability distribution. Maximizing each side over that same set of input probability distributions gives ; processing cannot increase channel capacity.
A simple sufficient condition for equality is that be injective on the reachable output alphabet. Then can be recovered from , giving and hence for every input law. The input that achieves the original channel capacity therefore achieves it after processing as well. Values of outside reachable outputs are irrelevant.
Injectivity is sufficient rather than necessary. More precisely, for finite alphabets equality in capacity data processing holds exactly when there is an original capacity-achieving input law for which and have conditional independence given . Sufficiency follows from the displayed gap identity. For necessity, maximize at an input law ; if its maximum equals , then , so both inequalities are equalities. Processing may discard output details unrelated to the input without losing channel capacity.
Let and let be the error indicator random variable. Because is determined by , the chain rule for conditional entropy in its two orders givesBy conditioning reduces entropy, . On , knowing determines , so the corresponding conditional entropy is zero. On , at most symbols remain possible for . The maximum Shannon entropy of a probability distribution on at most symbols is . ConsequentlyFor an arbitrary function , repeat the argument with . This is still determined by ; when it is zero, , and when it is one, has at most possibilities for each . Thus Fano's inequality givesEvents of zero probability contribute zero to the conditional average, covering the error probabilities zero and one. If , the left side and error probabilities are zero; the trivial conclusion is handled separately instead of writing as an ordinary finite product.
The specified transition law is a binary erasure channel. For an input bit with , its Shannon entropy is the binary entropy . A non-erased output determines , whereas the erasure symbol leaves its probability distribution unchanged because the erasure event is independent of . ThusMaximizing the mutual information over the input probability distribution, with equality at , givesThis includes , a noiseless binary channel, and , a completely erased channel with zero channel capacity.
Past exam of the mathematics course of the University of Cambridge 2001 ia Paper 2 12F Solution Created 2026-09-24 Updated 2026-10-14
Fix any starting village and let be the pickup distance, the subsequent passenger distance. From each vertex, the four possible pickup distances are , each with probability . From each pickup village, the three other possible destinations have distances , each with probability .
Sharing the pickup village does not by itself give independence. Here it does hold because the conditional distribution of is the same for every pickup village: for all allowed ,Thus has the taxi distance convolution on a rectangular road cycle. Counting the twelve equally likely pairs givesNo other distances are possible. The probability distribution is the same for every starting village, so no assumption about the previous customer's destination is needed.
The separate moments are , , and . Independence therefore givesAs a check from the displayed probability mass function, and subtracting gives the same variance.
Past exam of the mathematics course of the University of Cambridge 2002 iii Paper 27 1 Solution Created 2026-10-03 Updated 2026-10-14
Use the exponent convention . A real Lévy process starts at , has stationary increments and independent increments, is stochastically continuous, and is taken in its càdlàg version. Its characteristic exponent of a Lévy process satisfies and is continuous. This convention has the opposite sign from the alternative , so .
For real , the exponential martingale of a Lévy process is integrable because is deterministic and finite. For , its conditional expectation with respect to the natural filtration isThe second equality uses both independent increments and stationary increments. Thus this is a complex martingale, meaning that its real and imaginary parts are martingales.
A probability distribution is an infinitely divisible distribution when, for every positive integer , it is the distribution of a sum of independent and identically distributed random variables. Their common distribution is allowed to depend on . For a Lévy process,and these summands are independent with the common law of . This proves the required infinite divisibility directly.
The Lévy–Khintchine theorem states that a real infinitely divisible distribution has characteristic function , where, for a unique triplet with the following fixed truncation,with , , a positive measure, , and . Conversely every such triplet gives an infinitely divisible distribution and a Lévy process, unique in law, whose time- characteristic function is . The Lévy measure need not be finite near zero. Taylor expansion makes the compensated integrand there, while the measure is finite away from zero, so the integral is well defined. The drift parameter depends on the truncation: using instead can alter by the contribution of atoms at . Here denotes Gaussian variance, not drift.
We now prove that continuity forces , using the suggested truncation rather than merely quoting a path classification. For , is finite. The truncated exponent is that of the Compound Poisson processwhere has rate and the marks have law when the rate is positive. The remaining exponent also has an admissible Lévy–Khintchine formula triplet. Hence construct independently a Lévy process with that exponent. The sum has the same finite-dimensional distributions as , and hence the same law on càdlàg path space. Continuity is therefore an almost-sure property of the sum if it is one of .
But independent compound Poisson jumps cannot cancel: conditional on the path of , its jump times are countable; the independent Poisson jump times have continuous distributions and avoid them almost surely. At each jump of , the sum has the same nonzero jump, of size at least . If , there is positive probability of such a jump on any fixed positive-length time interval, contradicting almost-sure continuity. Thus for every , and taking gives .
The characteristic function is now . If , the process has continuous paths and independent centered normal increments of variance , so it is standard Brownian motion on the original probability space. If , the increments are deterministic, and continuity together with equality at all rational times gives simultaneously for all almost surely. A Brownian motion on an enlarged space can be used in the zero-variance representation if necessary. Therefore
Past exam of the mathematics course of the University of Cambridge 2002 iii Paper 33 3 Solution Created 2026-10-03 Updated 2026-10-14
Use logarithms to base two, so all entropies are in bits. For a discrete probability distribution and a density operator with eigenvalues , the Shannon entropy and Von Neumann entropy are respectivelyUse . The second definition is basis-independent by the spectral theorem; for a density operator diagonal with probabilities , it agrees with the first.
The classical relative entropy, or Kullback-Leibler divergence, of from isIt is if some has . The quantum relative entropy iswhen the support of is contained in that of , and is otherwise. Matrix logarithms are taken on the corresponding positive supports. In a common eigenbasis this reduces to classical relative entropy; both divergences are nonnegative and are generally asymmetric.
To prove Fano's inequality, let the finite alphabet size be and let indicate an incorrect guess. Since is determined by , the chain rule for conditional entropy first givesExpanding in the opposite order givesConditioning reduces entropy, so , the binary entropy function. When , knowledge of determines and the conditional entropy is zero. When , can lie in at most the alphabet values other than . Entropy on values is at most : its deficit from that number is the nonnegative relative entropy from the uniform distribution. Averaging the two cases therefore givesCombining the two expansions provesThis proof of Fano's inequality via an error indicator works for any alphabet-valued guessing rule, not just an optimal one. If , there is no guessing uncertainty and the claim is simply ; that case is treated separately rather than writing .
Past exam of the mathematics course of the University of Cambridge 2002 iii Paper 38 4 Solution Created 2026-10-03 Updated 2026-10-14
A statistical functional is a map defined on a class of probability distributions. Its associated functional statistic is , where is the empirical distribution and each is a Dirac measure. For example, yields the sample mean. The influence function records the derivative in the direction of point contamination:It is a local, infinitesimal robustness diagnostic, not a description of arbitrary large contamination.
Suppose is differentiable in a sense strong enough to linearize it at the empirical distribution, and that its derivative in a signed direction is . Averaging the directions over gives the zero direction, so linearity implies . The resulting asymptotic linear representation isIf , the central limit theorem and the negligible remainder giveThus the asymptotic variance is . For a vector functional the integral is . Convergence of the actual finite-sample variance to this leading expression additionally requires control of second moments of the remainder, or an appropriate uniform integrability condition; convergence in distribution alone does not provide that. Existence of a contamination derivative by itself also does not guarantee the required empirical linearization.
Several complementary robustness measures follow. Gross-error sensitivity is . Under contamination , the first-order bias is , whose worst absolute coefficient is ; bounded influence therefore bounds infinitesimal gross-error bias. The local-shift sensitivity is , equal to when the influence function is smoothly differentiable. It measures the effect of moving a contaminant a small distance. The rejection point of an influence function is , using centered, standardized coordinates; it is infinite if there is no such cutoff. A finite cutoff completely rejects sufficiently extreme observations to first order. These measures address different aspects of robustness. The integral of squared influence controls asymptotic precision and efficiency, while breakdown point concerns finite amounts of contamination and is not determined by bounded influence alone.
For the quantile, assume , a unique solution , and a density continuous and positive at . Define quantiles of the contaminated distribution by generalized inversion, since a distribution with a point mass need not attain as an equality. If , its indicator is constant near for sufficiently small contamination, and differentiatingat zero gives . HenceAt the exceptional point , the generalized-inverse quantile remains exactly : below the contaminated cumulative distribution function is below , and its value at exceeds . Its actual derivative there is zero. The usual step-function expression is an almost-everywhere version, which is all that the asymptotic variance needs. Its mean is zero, and its second moment isTherefore the influence function of a quantile gives asymptotic variance and gross-error sensitivity . Its jump makes local-shift sensitivity infinite, and its nonzero constant tails make its rejection point infinite. If or the quantile is not unique, this regular derivation does not apply.
Past exam of the mathematics course of the University of Cambridge 2003 iii Paper 29 1 b Solution Created 2026-10-03 Updated 2026-10-14
Equality of the probability distributions implies equality of all integrals for which they are finite, in particular . Part (a), applied to this conditional expectation, therefore gives The second-moment assumption justifies the squared-norm argument; the next part removes it.
Past exam of the mathematics course of the University of Cambridge 2003 iii Paper 29 1 c Solution Created 2026-10-03 Updated 2026-10-14
Again put . For any fixed real , the random variableis nonnegative and integrable. The sign function of is bounded and -measurable, so conditioning and equality of the probability distributions giveThus almost surely. On this forces ; on it forces ; on it forces . In particular up to a null set. These two events have equal probability, again by equality of the probability distributions, so they agree up to a null set. ConsequentlyTaking establishes the suggested sign assertion, but this alone would not exclude unequal positive values. Apply the same argument simultaneously to the countable set of rational . Outside the union of their null sets, unequal and would have a rational strictly between them, giving opposite signs. This proves the conditional expectation preserving a distribution result:Only integrability was used; no finite second moment or hidden square-integrability assumption is needed.