BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Learning Deep Architectures - Yoshua Bengio\, University of Montre
 al
DTSTART:20090707T130000Z
DTEND:20090707T140000Z
UID:TALK19001@talks.cam.ac.uk
CONTACT:Microsoft Research Cambridge Talks Admins
DESCRIPTION:*Abstract* Whereas theoretical work suggests that deep archite
 ctures might be computationally and statistically more efficient at repres
 enting highly-varying functions\, training deep architectures was unsucces
 sful until the recent advent of algorithms based on unsupervised pre-train
 ing of each level of a hierarchically structured model. Several unsupervis
 ed criteria and procedures were proposed for this purpose\, starting with 
 the Restricted Boltzmann Machine (RBM)\, which when stacked gives rise to 
 Deep Belief Networks (DBN). Although the partition function of RBMs is int
 ractable\, inference is tractable and we review several successful learnin
 g algorithms that have been proposed\, in particular those using weights t
 hat change quickly during learning instead of converging. In addition to b
 eing impressive as generative models\, DBNs have made an impact by being u
 sed to initialize deep supervised neural networks. We present surprising e
 mpirical results regarding the visualization of the intermediate represent
 ations learned\, to help understand how these models learn to compose feat
 ures in a hierarchy of features. Finally\, in an attempt to understand the
  unsupervised pre-training effect\, we describe a large set of simulations
  exploring the apparently conflicting hypotheses that unsupervised pre-tra
 ining acts like a regularizer or that it helps optimizing a difficult non-
 convex criterion fraught with local minima. \n\n*Biography* Yoshua Bengio 
 (PhD`1991\, McGill University) is professor at the Department\nof Computer
  Science and Operations Research\, Universite de Montreal\, and\nCanada Re
 search Chair in Statistical Learning Algorithms\, as well as\nNSERC-CGI Ch
 air\, and Fellow of the Canadian Institute for Advanced\nResearch. He was 
 program co-chair for NIPS`2008 and is general co-chair for\nNIPS`2009.  Hi
 s main ambition is to understand how learning can give rise\nto intelligen
 ce.  He has been an early proponent of deep architectures and\ndistributed
  representations as tools to bypass the curse of dimensionality\nand learn
  complex tasks. He contributed to many machine learning areas:\nneural net
 works\, recurrent neural networks\, graphical models\, kernel\nmachines\, 
 semi-supervised learning\, unsupervised learning and manifold\nlearning\, 
 pattern recognition\, data-mining\, natural language processing\,\nmachine
  vision\, and time-series models.
LOCATION:Small public lecture room\, Microsoft Research Ltd\, 7 J J Thomso
 n Avenue (Off Madingley Road)\, Cambridge
END:VEVENT
END:VCALENDAR