BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//talks.cam.ac.uk//v3//EN
BEGIN:VTIMEZONE
TZID:Europe/London
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:BST
DTSTART:19700329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:GMT
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CATEGORIES:Inference Group
SUMMARY:A quick way to learn a mixture of exponentially ma
ny linear models - Geoffrey Hinton\, Canadian Inst
itute for Advanced Research &\; University of T
oronto
DTSTART;TZID=Europe/London:20090615T150000
DTEND;TZID=Europe/London:20090615T160000
UID:TALK17338AThttp://talks.cam.ac.uk
URL:http://talks.cam.ac.uk/talk/index/17338
DESCRIPTION:Mixtures of linear models can be used to model dat
a that lies on or\nnear a smooth non-linear manifo
ld.\nA proper Bayesian treatment can be applied to
toy data to determine\nthe number of models in t
he mixture and the dimensionality of each\nlinear
model but this neurally uninspired approach comple
tely misses\nthe main problem: Real data with many
degrees of freedom in the\nmanifold requires a mi
xture with an exponential number of components.\nI
t is quite easy to fit mixtures of 2^1000 linear m
odels by using a\nfew tricks: First\, each linear
model selects from a pool of shared\nfactors using
the selection rule that factors with negative val
ues are\nignored. Second\, undirected linear model
s are used to simplify\ninference and the models a
re trained by matching pairwise statistics.\nThird
\, Poisson noise is used to implement L1 regulariz
ation of the\nactivities of the factors. The fact
ors are then threshold linear\nneurons with Poisso
n noise and their positive integer activities are\
nvery sparse. Preliminary results suggest that the
se exponentially\nlarge mixtures work very well as
modules for greedy\, layer-by-layer\nlearning of
deep networks. Even with one eye closed\, they out
perform\nSupport Vector machines for recognizing
3-D images of objects from\nthe NORB database.\n
LOCATION:TCM Seminar Room\, Cavendish Laboratory\, Departme
nt of Physics
CONTACT:David MacKay
END:VEVENT
END:VCALENDAR