University of Cambridge > Talks.cam > Machine Learning @ CUED > Annealing Between Distributions by Averaging Moments

  Browse A-Z

Log in

Cambridge users (raven)details Other usersdetails No account?details

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

Annealing Between Distributions by Averaging Moments

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Zoubin Ghahramani.

Many powerful Monte Carlo techniques for estimating partition functions, such as annealed importance sampling (AIS), are based on sampling from a sequence of intermediate distributions, which interpolate between a tractable initial distribution and an intractable target distribution. The near-universal practice is to use geometric averages of the initial and target distributions, but alternative paths can perform substantially better. We present a novel sequence of intermediate distributions for exponential families: averaging the moments of the initial and target distributions. We derive an asymptotically optimal piecewise linear schedule for the moments path and show that it performs at least as well as geometric averages with a linear schedule. Moment averaging performs well empirically at estimating partition functions of restricted Boltzmann machines (RBMs), which form the building blocks of many deep learning models.

This talk is part of the Machine Learning @ CUED series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

The obesity epidemic: Discussing the global health crisis Thinking Society: What is Life? Churchill College Phoenix Society Cambridge University Russian Society Talks Clare Politics Cambridge Futures

Other talks

A feast of languages: multilingualism in neuro-typical and atypical populations Part IIB Poster Presentations Art speak Aromatic foldamers: mastering molecular shape What has Engineering Design to say about Healthcare Improvement? TO A TRILLION AND BEYOND: THE FUTURE OF COMPUTING AND THE INTERNET OF THINGS - The IET Cambridge Prestige Lecture Glucagon like peptide-1 receptor - a possible role for beta cell physiology in susceptibility to autoimmune diabetes Constructing the virtual fundamental cycle 'Cambridge University, Past and Present' A transmissible RNA pathway in honeybees Uncertainty Quantification of geochemical and mechanical compaction in layered sedimentary basins
 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity