BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Adaptation through prior tails and deep neural networks - Ismael C
 astillo (Sorbonne Université)
DTSTART:20251031T140000Z
DTEND:20251031T150000Z
UID:TALK237511@talks.cam.ac.uk
CONTACT:Qingyuan Zhao
DESCRIPTION:We introduce a scalable Bayesian method to derive adaptation t
 o structural properties such as smoothness or intrinsic dimensions. The ma
 in idea is to define a prior distribution that combines tails that are `he
 avy’ together with small deterministic scaling factors. This produces a 
 `soft’ form of variable selection. A main algorithmic advantage is that 
 there is no need to explicitly design a variable selection prior in the fo
 rm of a hyper-prior. We illustrate this approach in nonparametric models: 
 regression\, density estimation and classification. \n\nWe examine the tra
 de-off between choice of the tails and choice of scalings in the prior. Wh
 ile polynomial tails (e.g. Student) can lead to full adaptation\, we also 
 show that lighter tails (e.g. Laplace or `1/p’-Weibull) still provide im
 proved rates compared to Gaussian tails and even full adaptation in an app
 ropriate limit of vanishing tail index ‘p’.\n\nWe then discuss the met
 hod in the context of ReLU neural networks. We consider an over-parameteri
 sed deterministic network architecture. When using iid Student-type priors
  on network weights\, the corresponding posterior distribution and its mea
 n-field variational counterpart enjoy fully adaptive (to both smoothness a
 nd structure) convergence rates. We finally discuss work in progress using
  lighter tails for priors on the weights\, and that connects to neural net
 work estimators that have been previously implemented in practice but for 
 which theoretical support is still limited.\n\nThis talk is based on joint
  works with Sergios Agapiou (Cyprus)\, Julyan Arbel (INRIA Grenoble) and P
 aul Egels (Sorbonne).
LOCATION:MR12\, Centre for Mathematical Sciences
END:VEVENT
END:VCALENDAR