David Duvenaud coauthored the 2018 study of inference suboptimality with Chris Cremer and Xuechen Li. Their analysis distinguishes distribution-family limitations from a shared inference rule's shortcomings.[ref]
Inference Suboptimality in Variational Autoencoders by Chris Cremer and others. Title-page byline; Table1 and §3.1
An original diagnostic example makes that distinction numerical. Fix the generative model and one observation. Suppose its exact log evidence is , the globally best evidence lower bound within a chosen family is , and a recognition network returns a distribution with bound . The variational approximation gap is nat; the amortization gap is nat. They sum to the total -nat inference deficit.
These stipulated exact values isolate the diagnostic, rather than reproducing experimental measurements. Improving only the shared rule could recover at most nat within that family. Comparing a different generative model would change the target posterior distribution and would no longer isolate the same gap. This is a contribution of the joint paper; its byline does not specify individual responsibility for each derivation.
Articles by others on the same topic
There are currently no matching articles.