Efficient Inference and Learning with Intractable Posteriors? Yes, Please.

Add to your list(s) Download to your calendar using vCal

• Diederik P. Kingma (University of Amsterdam)
• Wednesday 14 October 2015, 11:00-12:00
• Engineering Department, CBL Room BE-438.

If you have a question about this talk, please contact Dr Jes Frellsen.

We discuss a number of recent advances in Stochastic Gradient Variational Inference (SGVI).

• Blending ideas from variational inference, deep learning and stochastic optimization, we derive an algorithm for efficient gradient-based inference and learning with intractable posteriors.

• Applied to deep latent-variable models with neural networks as components, this results in the Variational Auto-Encoder (VAE), a principled Bayesian auto-encoder. We show that VAEs can be useful for semi-supervised learning and analogic reasoning.

• Further improvements are realized through a new variational bound with auxiliary variables. Markov Chain Monte Carlo (MCMC) can be cast as variational inference with auxiliary variables; this interpretation allows principled optimization of MCMC parameters to greatly improve MCMC efficiency.

• When applying SGVI to global parameters, we show how an order of magnitude of variance reduction can be achieved through local reparameterization while retaining parallelizability. Gaussian Dropout can be cast as a special case of such SGVI with a scale-free prior. This variational interpretation of dropout allows for simple optimization of dropout rates.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

• Seminar
• All Talks (aka the CURE list)
• Biology
• CBL important
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge Neuroscience Seminars
• Cambridge University Engineering Department Talks
• Cambridge talks
• Centre for Smart Infrastructure & Construction
• Chris Davis' list
• Creating transparent intact animal organs for high-resolution 3D deep-tissue imaging
• Engineering Department, CBL Room BE-438
• Featured lists
• Guy Emerson's list
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• Joint Machine Learning Seminars
• Life Science
• Life Sciences
• ML
• Machine Learning @ CUED
• Machine Learning Summary
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Required lists for MLG
• School of Technology
• Simon Baker's List
• Stem Cells & Regenerative Medicine
• Trust & Technology Initiative - interesting events
• bld31
• dh539
• ndk22's list
• rp587
• yk373's list

Note that ex-directory lists are not shown.