Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 213 (Stochastic Networks)

An open migration process is a continuous-time Markov chain recording the populations $n=(n_1,\dots ,n_J)$ of finitely many colonies. Independent external Poisson processes bring individuals to colony $j$ at rate ${\nu }_j$. When that colony contains $k>0$ individuals, a departure opportunity occurs at total rate ${\varphi }_j(k)>0$, with ${\varphi }_j(0)=0$. The departing individual moves to colony $l$ with probability $p_{jl}$, or leaves the system with probability $p_{j0}=1-{\sum }_l{p}_{jl}$. Routing choices and the clocks are independent. A return to the same colony does not change the state.

The word open means that the routing matrix $P=(p_{jl})$ is transient: every individual eventually leaves, or equivalently the spectral radius of $P$ is less than one. The traffic equations of an open migration process, with row-vector convention, give the total arrival rates:These rates include migrations as well as external arrivals. On the active colonies assume ${\alpha }_j>0$ and the usual irreducible Markov chain condition. PutThe product form and its normalization condition areTo establish the product-form stationary distribution of an open migration process, first use its unnormalized weight $b(n)={\prod }_j{b}_j(n_j)$. Its ratios satisfy $b_j(k){\varphi }_j(k)={\alpha }_j{b}_j(k-1)$. The incoming external arrivals into $n$, divided by $b(n)$, contribute ${\sum }_j{\nu }_j{\varphi }_j(n_j)/{\alpha }_j$. Incoming exits contribute ${\sum }_i{\alpha }_i{p}_{i0}$. Incoming migrations from $i$ to $j\ne i$ contribute ${\alpha }_i{p}_{ij}{\varphi }_j(n_j)/{\alpha }_j$. Terms involving an empty destination are zero. Thus the full incoming rate divided by $b(n)$ isHere the traffic equations of an open migration process give the second equality, including ${\sum }_i{\alpha }_i{p}_{i0}={\sum }_j{\nu }_j$. The right side is precisely the total outgoing rate. This proves global balance for a continuous-time Markov chain; pairwise detailed balance is not required for general routing.

For finite rates at each finite population, the continuous-time Markov chain is nonexplosive Markov chain: its total population is bounded over a finite time interval by the initial population plus the finitely many external arrivals, so it visits a finite set on which all jump rates are bounded. Consequently finite $Z_j$ normalize the invariant weight to a stationary distribution. Under the stated irreducible Markov chain assumptions, this is the unique stationary distribution, and it is a positive recurrent Markov chain. Conversely, the unique invariant measure of an irreducible recurrent chain is proportional to $b$, so a positive recurrent Markov chain requires its normalization to be finite. A sufficient condition is ${\mathrm{liminf}}_{k\to \infty }{\varphi }_j(k)>{\alpha }_j$; the series criterion itself is the exact condition. Colonies with ${\alpha }_j=0$ have zero stationary population and can be removed from the active class.

For the single-server M/M/1 queue specialization, ${\varphi }_j(k)={\varphi }_j$ for $k>0$, each stationary colony has the geometric distribution of an M/M/1 queue:The traffic equations of an open migration process do not involve the service rates, so the ${\alpha }_j$ remain fixed during this service-capacity allocation in an open migration process. Write $A={\sum }_j{\alpha }_j$, $B=F-A$ and $b_j={\varphi }_j-{\alpha }_j$. A stationary allocation requires $B>0$, with $b_j>0$ and ${\sum }_j{b}_j=B$. The Cauchy-Schwarz inequality givesEquality holds exactly when $b_j$ is proportional to $\sqrt{{\alpha }_j}$. Hence, for positive arrival rates, the unique optimal service allocation and the minimum expected value of the total population areIf $F\le A$ and some arrival rate is positive, no stable service allocation exists. Zero-arrival colonies need no service in their stationary empty class: allowing zero service there gives the same formula restricted to active colonies. If strictly positive service is demanded even at inactive colonies, the displayed minimum is an infimum approached as that unused allocation tends to zero. If all arrivals vanish, the stationary mean population is zero for every allocation that drains the initial population.

An effective bandwidth measures how much capacity a random traffic source requires when unusually large demands matter. A mean-rate description loses burstiness; a peak-rate description can waste capacity when peaks are rare. The parameter $s>0$ in an effective bandwidth controls the emphasis placed on such bursts through the moment-generating function:Here $X$ is the traffic in a chosen time unit. If $X(t)$ instead measures traffic over length $t$, the corresponding rate is $\alpha (s,t)=\log \mathbb{E}[e^{sX(t)}]/(st)$. With independent stationary increments, this is independent of $t$ whenever the moment-generating function exists. More generally both the time scale and dependence structure matter.

The Chernoff bound follows directly from the Markov inequality, since $e^{sX}$ is nonnegative and $X\ge b$ implies $e^{sX}\ge e^{sb}$:Optimize over the admissible positive values of $s$ to obtainFor independent sources $X_1,\dots ,X_N$, moment-generating functions multiply and their effective bandwidths add. Thus a total capacity $c$ has the one-time overflow bound $\mathbb{P}({\sum }_i{X}_i\ge c)\le \exp \{s({\sum }_i{\alpha }_i(s)-c)\}$. In particular, for independent and identically distributed random variables,This is the positive-parameter part of the Legendre transform of a cumulant-generating function. The size of its exponent describes how a service margin reduces the chance of a large aggregate demand. A buffer-overflow event involves traffic over many intervals; a one-time Chernoff bound alone is not automatically a bound on that event. A time-dependent model and, for example, bounds on the union of relevant interval events are then needed.

Assume a finite moment-generating function in a neighborhood of zero. Differentiating the cumulant-generating function gives $K(0)=0$, $K^{\prime }(0)=\mathbb{E}[X]$ and $K^{\prime \prime }(0)=\mathrm{Var}(X)$. Therefore the small-parameter limit is the mean traffic rate:The variance term shows how an effective bandwidth charges extra capacity for burstiness. Also $K$ is a convex function with $K(0)=0$, so $K(s)/s$ is nondecreasing for $s>0$ in its domain.

For the other limit, let $M=\mathrm{esssup}X$, the essential supremum. If $M<\infty$, $X\le M$ almost surely gives $\alpha (s)\le M$. For every $a<M$, $p_a=\mathbb{P}(X\ge a)>0$, andTaking a lower limit and then $a\uparrow M$ proves the large-parameter limit is peak traffic:If $X$ has no finite upper bound and all positive exponential moments exist, the same lower bound for arbitrarily large $a$ makes the limit infinite. If positive exponential moments cease to exist at a finite parameter, the effective bandwidth is infinite beyond that domain, and the infinite limit is understood in this extended sense. For example, a normal distribution with mean $m$ and variance $v$ has $\alpha (s)=m+sv/2$: its unbounded upper tail has no finite peak. Thus effective bandwidth interpolates between mean and peak demand while permitting independent sources to be aggregated by addition.

In a loss network with fixed routing, calls of type $r$ arrive as independent Poisson processes with rates ${\nu }_r$. Every accepted call occupies $A_{jr}$ units of resource $j$ throughout its holding time. The link-route incidence matrix $A$ has nonnegative integer entries, $C$ is the resource-capacity vector, $n_r$ counts calls currently present, and $e_r$ is the $r$th coordinate unit vector. The feasible occupancy space isThe inequalities are coordinatewise. A call that would leave this space is rejected immediately, without queuing or changing $n$. With independent unit-rate exponential distributions for holding times, the memoryless property makes $n$ a continuous-time Markov chain, whose nonzero off-diagonal transition rates areThere are $n_r$ independent clocks of rate one for the second transition. Rejected arrivals are not jumps and do not appear in the off-diagonal transition rates.

Let $a(n)={\prod }_r{\nu }_r^{n_r}/n_r!$. For every feasible adjacent pair,Thus detailed balance for a continuous-time Markov chain gives the product-form stationary distribution of a loss network:In this convention $G(C)$ is the reciprocal normalizing sum. If every route consumes some resource, finite capacities make $S(C)$ finite. Even zero-resource routes cause no normalization difficulty: the sum is at most $\exp ({\sum }_r{\nu }_r)$. With positive arrival rates the reachable feasible class is an irreducible Markov chain, since calls can depart to the empty state and any feasible occupancy can be built from it. The stationary distribution is therefore unique on that class.

The vector $m=C-An$ records the free units of each resource. Its probability distribution ${\pi }^{\prime }(m)$ sums $\pi (n)$ over all occupancies with the same free resources. The map from $n$ to $m$ generally loses information: different call mixtures may consume exactly the same resources.

For ${\pi }^{\prime }(m)>0$, the conditional expectation can be computed by removing one call. The factorial identity $n_r\pi (n)={\nu }_r\pi (n-e_r)$ givesIndeed, $n\mapsto n-e_r$ is a bijection from the occupancies in this sum with $n_r>0$ to occupancies using $C-m-A{e}_r$ units. Conversely, adding one call to any such occupancy is feasible because its final resource use is $C-m$. Consequentlyand the conditional expectation is zero otherwise. Conditioning the identity ${\sum }_r{A}_{jr}{n}_r=C_j-m_j$ and multiplying by ${\pi }^{\prime }(m)$ now yieldsAlternatively, multiply the preceding unnormalized factorial identity by $A_{jr}$ and sum over $r$ directly; this proves the same formula even when ${\pi }^{\prime }(m)=0$, without dividing by a zero probability. This gives both the conditional expectation argument and the direct summation argument.

Use an ideal carrier-sense multiple access scheme on the interference graph. An active station finishes its transmission after an independent unit-rate exponential distribution waiting time. An inactive station has an independent attempt clock with an exponential distribution of rate $e^{{\theta }_r}$; an attempt succeeds only when all its neighbors are inactive. Attempts that are blocked do not change the state. Thus, on the independent sets represented by $S$, the continuous-time Markov chain has transition ratesThe empty independent set can be reached by successive deactivations, and any other independent set can be reached from it by successive activations. Hence this finite chain is an irreducible Markov chain. The weight $a_{\theta }(n)=e^{\theta \cdot n}$ satisfiesBy detailed balance for a continuous-time Markov chain, its unique stationary law isThis is a finite exponential family. Its natural parameter of an exponential family is $\theta$ and its mean parameter of an exponential family is the vector of active fractions.

The throughput region of an interference graph is exactly $\Lambda =\mathrm{conv}S$. To see the possibly less immediate inclusion, suppose a random feasible schedule has mean $v\ge \lambda$. Retain each of its active vertices $r$ independently with probability ${\lambda }_r/v_r$, interpreting zero coordinates as always deleted. A subset of an independent set remains an independent set; the thinned schedule has mean exactly $\lambda$. Thus $\lambda$ lies in the convex hull of $S$. The opposite inclusion follows from the definition using equality. Moreover $0,e_1,\dots ,e_R\in S$, so this convex hull has full dimension.

Now choose $\epsilon >0$ such that $\mu =\lambda +\epsilon 1$ remains in the interior of $\mathrm{conv}S$. Use interior moment matching in a finite exponential family. Consider the convex functionThere is a ball of radius $\delta >0$ around $\mu$ contained in $\mathrm{conv}S$. Maximizing a linear functional over the convex hull is the same as maximizing over its generating set, so, for every $\theta$,The last inequality uses the point $\mu +\delta \theta /\Vert \theta \Vert$ when $\theta \ne 0$. Hence $F$ is a coercive function and has a finite minimizer ${\theta }^{\ast }$. Differentiating the finite normalizing sum givesThis covariance matrix is a positive-definite matrix: a linear functional constant on all states of positive weight must be constant on $0,e_1,\dots ,e_R$, so its coefficients vanish. Thus $F$ is a strictly convex function; its minimizer is unique. At that minimizer the desired strict service margins areFor a backlogged station that transmits at unit speed whenever active, the long-run throughput is ${\mathbb{E}}_{\theta }[n_r]$, by the ergodic theorem for a finite continuous-time Markov chain. Therefore the ideal carrier-sense multiple access scheme can supply a strict service margin for every arrival vector in the interior of the throughput region of an interference graph. The local attempt rates can realize the whole interior capacity region in this sense. Boundary points may require parameters tending to infinity, and the argument gives no uniform delay or mixing-time bound near the boundary.

In congestion control, resource $j$ observes only its own aggregate load $y_j={\sum }_{r:j\in r}{x}_r$ and generates the nonnegative feedback price ${\mu }_j=p_j(y_j)$. User $r$ receives the sum $M_r={\sum }_{j\in r}{\mu }_j$ of prices along its route. Its desired increase $w_r$ competes with the congestion charge $x_r{M}_r$, while ${\kappa }_r$ sets the adjustment speed. This is local information: a resource needs the rates passing through it, and a user needs the feedback along its own route, rather than the complete network state.

There is a hypothesis to make explicit before claiming global convergence. If “increasing” means nondecreasing, the printed assumptions allow $p_j\equiv 0$, in which case ${\dot{x}}_r={\kappa }_r{w}_r>0$ and there is no equilibrium point. The intended theorem holds when every route contains at least one resource whose price is not identically zero, with nonempty routes and nonnegative, continuous, nondecreasing prices. Strictly increasing prices satisfy this condition automatically. We prove the theorem under this necessary qualification; without it the zero-price example disproves the literal claim.

Let $P_j(y)={\int }_0^y{p}_j(u)du$. Each $P_j$ is a nonnegative convex function. For positive rates define the primal congestion potentialThe weighted logarithmic utility is a strictly concave function, and subtracting the resource costs preserves strict concavity. Thus $U$ has at most one maximizer and at most one critical point on the positive orthant.

It also attains its maximum there. A nonzero nondecreasing price is bounded below by some positive constant at all sufficiently large loads; hence its primitive satisfies $P_j(y)\ge c_{j}y-K_j$ with $c_j>0$. Choose $c_j=0$ at zero-price resources. The route coverage assumption gives $c={\min }_r{\sum }_{j\in r}{c}_j>0$, soThis bounds all superlevel sets of $U$. Within a bounded set, approaching $x_r=0$ makes $U\to -\infty$. Consequently every nonempty superlevel set is a compact set contained in the positive orthant. A maximizing sequence stays in one such set, giving a unique interior maximizer $x^{\ast }$. Its critical-point equations areThe rate dynamics are the gradient flow with diagonal mobility ${\dot{x}}_r={\kappa }_r{x}_r{g}_r(x)$. Along any positive trajectory,with equality only at $x^{\ast }$. Thus $V(x)=U(x^{\ast })-U(x)$ is a strict Lyapunov function. The initial superlevel set traps the trajectory away from both infinity and the boundary, ensuring global continuation. Positive initial rates remain positive: variation of constants in ${\dot{x}}_r+{\kappa }_r{M}_r(t)x_r={\kappa }_r{w}_r$ makes this explicit. Starting with finite nonnegative rates also gives positive rates at every later positive time.

For completeness, mere continuity of $p_j$ suffices for convergence; one need not silently assume differentiable prices. On the trapping compact set, both the vector field and $D(x)={\sum }_r{\kappa }_r{x}_r{g}_r(x{)}^2$ are continuous and bounded. The trajectory satisfies a uniform Lipschitz condition in time, so $D(x(t))$ is uniformly continuous. Also ${\int }_0^{\infty }D(x(t))dt<\infty$, since $U$ increases to a finite limit. The decay of a nonnegative uniformly continuous integrable function applies: otherwise disjoint intervals of a fixed positive width would each contribute a fixed positive amount. Every accumulation point therefore has $D=0$ and is $x^{\ast }$. Compactness then proves the global convergenceThis is the dissipation argument behind the LaSalle invariance principle. If uniqueness of trajectories is desired under continuous prices, put $z_r=2\sqrt{x_r/{\kappa }_r}$. The transformed equation is $\dot{z}={\nabla }_{z}U(\kappa z^2/4)$. The transformed potential is concave: it is a sum of positive multiples of $\log z_r$, minus convex nondecreasing $P_j$ composed with nonnegative quadratic sums. Its gradient is monotone decreasing, so the squared distance between two solutions cannot increase. This proves uniqueness for positive initial data without a locally Lipschitz function hypothesis on $p_j$.

For the alternative price dynamics, restrict first to positive prices on resources used by at least one route, and discard unused resources. Define the dual congestion potentialIts domain has ${\mu }_j\ge 0$ and $M_r>0$ for every route. Strictly increasing $q_j$ make each $Q_j$ strictly convex, and $-\log M_r$ is convex. Thus $H$ is a strictly convex function. Each $q_j$ is eventually bounded below by a positive constant, so its integral grows at least linearly, dominating the negative logarithms as $\Vert \mu \Vert \to \infty$. If a route price tends to zero in a bounded set, $H\to \infty$. Therefore $H$ attains a unique minimum on its domain including the admissible boundary faces.

That minimum has every used-resource price positive. At a face ${\mu }_j=0$, all route prices are still positive and the inward derivative isHence the minimum cannot lie on that face. At the unique interior minimum,Since the price dynamics are ${\dot{\mu }}_j=-{\kappa }_j{\mu }_j{\partial }_{j}H$, their positive equilibrium points are exactly these critical points. There is one equilibrium in the positive price domain. An unused resource instead has its only nonnegative equilibrium at zero.

The positive-domain qualification matters here even with strictly increasing $q_j$. If zero prices are admitted, the multiplicative factor ${\mu }_j$ permits boundary equilibria of multiplicative price dynamics. Take one route using two resources, $w=1$, and $q_1(a)=q_2(a)=a$. Both $({\mu }_1,{\mu }_2)=(1,0)$ and $(0,1)$ are equilibria, with $x=1$, as is the positive equilibrium $(1/\sqrt{2},1/\sqrt{2})$, with $x=1/\sqrt{2}$. Thus the literal assertion of uniqueness on the whole nonnegative orthant is false. A boundary equilibrium need not satisfy load equals supplied capacity at its zero-price resources.

The relationship between the two intended equilibria is inverse resource response. The first system has ${\mu }_j=p_j(y_j)$; the positive equilibrium of the second has $y_j=q_j({\mu }_j)$. If $q_j=p_j^{-1}$ on the relevant ranges, these are the same resource equations, and both systems also impose $x_r{M}_r=w_r$. They therefore have the same rates and prices. For strictly increasing continuous $p_j$ with $p_j(0)=0$ and unbounded range, the ordinary inverses satisfy all the stated conditions on $q_j$. With arbitrary independently chosen $p_j$ and $q_j$, the equilibria need not agree. Under inverse response, $P_j$ and $Q_j$ are convex conjugates on the nonnegative half-line, and $H$ supplies the Lagrangian dual problem for maximizing the weighted logarithmic utility minus resource costs, up to constants independent of $\mu$.

Only routes with $n_r>0$ contribute to proportional fairness or to the weighted logarithmic utility; omit inactive routes from expressions with a rate in the denominator. Active routes have positive rates whenever a positive feasible allocation exists. For a feasible competitor $y$, the elementary logarithm inequality $\log u\le u-1$ givesA zero competitor rate on an active route has utility $-\infty$ and also cannot improve the objective. Therefore a proportionally fair allocation maximizes the logarithmic objective. Conversely, the feasible set is a convex set, and differentiating the objective along $x+t(y-x)$ at $t=0+$ shows that an optimizer satisfies the proportional fairness inequality. Strict concavity gives uniqueness on active coordinates; rates assigned to nonexistent flows are immaterial.

In the linear flow network, put $z_0=n_0{x}_0$ and $z_i=n_i{x}_i$ for active routes. A local route $i$ uses only resource $i$. If $n_i>0$ and $z_0+z_i<1$, increasing $x_i$ increases the objective without violating any other constraint. Thus each active local route saturates its resource:Write $L={\sum }_{i=1}^I{n}_i$ and $N=n_0+L$. For $n_0>0$ and $L>0$, substitute $z_i=1-z_0$ into the objective. Apart from constants its dependence on $z_0$ isIts derivative vanishes when $n_0/z_0=L/(1-z_0)$, and its second derivative is negative. Hence the per-flow through-route rate isIf $L=0<n_0$, the optimal through-route aggregate rate is one, so $x_0=1/n_0=1/N$ still holds. If $n_0=0<L$, each active local route has $x_i=1/n_i$, also given by the displayed local formula. When $N=0$ there are no flows and no departures. Inactive coordinates need not be assigned these undefined per-flow formulas.

Independent arrivals and exponential document sizes make this a flow-level network model. A document of type $r$ has residual-size hazard ${\mu }_r$ per unit of data transmitted; when its transmission speed is $x_r(n)$, its completion hazard per unit time is ${\mu }_r{x}_r(n)$. Thus the continuous-time Markov chain of flow counts has transition intensitiesFor $N>0$ these departure intensities are ${\mu }_0{n}_0/N$ for the through route and ${\mu }_{i}L/N$ for an active local route. Total arrival intensity is constant and total departure intensity is bounded by ${\sum }_r{\mu }_r$, ensuring a nonexplosive Markov chain.

Set ${\rho }_r={\nu }_r/{\mu }_r$, the offered traffic, and define the balance function of a flow-level networkThe binomial coefficients satisfyThese identities also cover states with only one kind of active route. They show that the weight $a(n)=\Phi (n){\prod }_{r=0}^I{\rho }_r^{n_r}$ satisfiesThus detailed balance for a continuous-time Markov chain proves that this is a reversible Markov chain and reduces the stationary distribution calculation to normalization.

Sum over $n_0$ first. The negative binomial series gives, for fixed local counts and $L={\sum }_i{n}_i$,All summands are nonnegative, so the Tonelli theorem permits exchanging the sums. The partition sum is thereforeIt is finite exactly when ${\rho }_0+{\rho }_i<1$ for every $i$, for $I\ge 1$. These are the resource-load conditions: resource $i$ receives the through-route offered traffic plus its local offered traffic. The normalized stationary distribution isWith positive arrival rates this is the unique stationary distribution of the irreducible feasible flow-count chain. Zero arrival rates restrict its closed class to the corresponding zero coordinates.

Summing out $n_0$ also shows that the local counts are independent, with geometric distributions on $0,1,\dots$:Hence their mean flow counts areThe binomial balance factor is essential: independent geometric counts for all routes would not give these departure rates or the correct resource-load normalization.