Stein Discrepancy

Add to your list(s) Download to your calendar using vCal

• Sebastian Ober (University of Cambridge)
• Monday 28 January 2019, 14:00-15:30
• Engineering Department, CBL Room 438.

If you have a question about this talk, please contact jg801.

Recently, Stein discrepancy-based methods have become popular tools for machine learning applications, including verifying the convergence of MCMC [1], goodness-of-fit tests [2][3], and variational inference [4][5]. Although Stein’s method has been long-known, widespread use was limited by an intractable optimization over a difficult function space. However, the recent development of the kernelized Stein discrepancy (KSD) [2][3] has circumvented this difficulty. Our talk will give a theoretical introduction to the Stein discrepancy and KSD . We will then introduce two recent applications of the Stein discrepancy to machine learning problems. The first of these, Stein variational gradient descent (SVGD) [4], shows how to apply the KSD to variational inference. We conclude by discussing the Stein variational autoencoder (Stein VAE ) [5], which applies SVGD to VAE learning.

Papers that are important for the talk:

[2] Chwialkowski, K., Strathmann, H., and Gretton, A. A kernel test of goodness of fit. In ICML , 2016. https://arxiv.org/abs/1602.02964

[4] Liu, Q. and Wang, D. Stein variational gradient descent: A general purpose Bayesian inference algorithm. In NIPS , 2016. https://arxiv.org/abs/1608.04471

Additional Recommended reading:

[1] Gorham, J. and Mackey, L. Measuring sample quality with Stein’s method. In NIPS , pp. 226-234, 2015. https://arxiv.org/abs/1506.03039 —Note: we will only be discussing up to and including section 3

[3] Liu, Q., Lee, J., and Jordan, M. I. A kernelized Stein discrepancy for goodness-of-fit tests. In ICML , 2016. https://arxiv.org/abs/1602.03253 —Note: this is essentially the same paper as [2], which is what we will present

[5] Pu, Y., Gan, Z., Henao, R., Li, C., Han, S., and Carin, L. VAE learning with Stein variational gradient descent. In NIPS , 2017. https://arxiv.org/abs/1704.05155

This talk is part of the Machine Learning Reading Group @ CUED series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge University Engineering Department Talks
• Cambridge talks
• Centre for Smart Infrastructure & Construction
• Chris Davis' list
• Computational Continuum Mechanics Group Seminars
• Engineering Department, CBL Room 438
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Journal Clubs
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• ML
• Machine Learning Reading Group
• Machine Learning Reading Group @ CUED
• Machine Learning Summary
• Quantum Matter Journal Club
• Required lists for MLG
• School of Technology
• Simon Baker's List
• TQS Journal Clubs
• Trust & Technology Initiative - interesting events
• bld31
• custom
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.