University of Cambridge > Talks.cam > Microsoft Research Cambridge, public talks > Do Deep Nets Really Need To Be Deep?

Do Deep Nets Really Need To Be Deep?

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Microsoft Research Cambridge Talks Admins.

This event may be recorded and made available internally or externally via http://research.microsoft.com. Microsoft will own the copyright of any recordings made. If you do not wish to have your image/voice recorded please consider this before attending

Currently, deep neural networks are the state of the art on problems such as speech recognition and computer vision. We show that by using a method called model compression that shallow feed-forward nets can learn the complex functions previously learned by deep nets and achieve accuracies previously only achievable with deep models. Moreover, in some cases the shallow neural nets can learn these deep functions using the same number of parameters as the original deep models. On the TIMIT phoneme recognition and CIFAR -10 image recognition tasks, shallow nets can be trained that perform similarly to complex, well-engineered, deeper convolutional architectures. Our success in training shallow neural nets to mimic deeper models suggests that there may be better algorithms for training shallow nets than those currently available. I’ll also briefly discuss work we’re doing to compress extremely large deep models and ensembles of deep models to “modest-size” deep models that fit on servers, and to “small” deep models that run on mobile devices.

This talk is part of the Microsoft Research Cambridge, public talks series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

 

© 2006-2019 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity