BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Pizza & AI April 2019 - Microsoft Research/University of Cambridge
DTSTART:20190426T163000Z
DTEND:20190426T180000Z
UID:TALK123142@talks.cam.ac.uk
CONTACT:Microsoft Research Cambridge Talks Admins
DESCRIPTION:*Speaker 1* - Andrey Malinin\n\n*Title* - This is the EnDD: En
 semble Distribution Distillation\n\n*Abstract* - Ensemble of Neural Networ
 k (NN) models are known to yield improvements \nin accuracy as well as rob
 ust measures of uncertainty. However\, ensembles come at high computationa
 l and memory cost\, which may be prohibitive for certain application. Prev
 iously\, the distillation of an ensemble into a single model has been inve
 stigated. Such approaches \ndecrease computational cost and allow a single
  model to achieve accuracy comparable to that of an ensemble. However\, in
 formation about the diversity of the ensemble\, which can yield estimates 
 of epistemic uncertainty\, is lost. Recently\, a new type of model\, calle
 d a Prior Network\, has been introduced\, which allows a single DNN to exp
 licitly model a distribution over output distributions conditioned on the 
 input by parameterizing a Dirichlet distribution. This work proposes an ap
 proach called Ensemble Distribution Distillation\, which allows distilling
  an ensemble into a single Prior Network model\, retaining both the improv
 ed classification performance as well as measures of diversity of the ense
 mble. The properties of Ensemble \nDistribution Distillation are investiga
 ted on a synthetic spiral dataset.\n\n\n*Speaker 2*- Yingzhen Li\n\n*Title
 * - Meta-Learning for Stochastic Gradient MCMC\n\n*Abstract* - Stochastic 
 gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popula
 r for simulating posterior samples in large-scale Bayesian modeling. Howev
 er\, existing SG-MCMC schemes are not tailored to any specific probabilist
 ic model\, even a simple modification of the underlying dynamical system r
 equires significant physical intuition. This paper presents the first meta
 -learning algorithm that allows automated design for the underlying contin
 uous dynamics of an SG-MCMC sampler. The learned sampler generalizes Hamil
 tonian dynamics with state-dependent drift and diffusion\, enabling fast t
 raversal and efficient exploration of neural network energy landscapes. Ex
 periments validate the proposed approach on both Bayesian fully connected 
 neural network and Bayesian recurrent neural network tasks\, showing that 
 the learned sampler out-performs generic\, hand-designed SG-MCMC algorithm
 s\, and generalizes to different datasets and larger architectures.\n\nThi
 s is a joint work with Wenbo Gong and Jose Miguel Hernandez-Lobato from th
 e University of Cambridge. The paper will be presented at ICLR 2019.\n
LOCATION:Auditorium\, Microsoft Research Ltd\, 21 Station Road\, Cambridge
 \, CB1 2FB
END:VEVENT
END:VCALENDAR