University of Cambridge > Talks.cam > Machine Learning Reading Group @ CUED > Stochastic optimization and adaptive learning rates

  Browse A-Z

Log in

Cambridge users (raven)details Other usersdetails No account?details

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

Stochastic optimization and adaptive learning rates

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Yingzhen Li.

Stochastic optimization is prevalent in modern machine learning, and the main purpose of this talk is to understand why it works. We will first briefly recap the history of stochastic approximation methods, starting from the famous Robbins and Monro paper. Then we will introduce the cost function minimization problem in machine learning context and show you how to prove the convergence of stochastic gradient descent to a local optima. We proceed the proof in three steps: continuous gradient descent, discrete gradient descent, and stochastic gradient descent. However the conditions for learning rates presented in the proof is not necessary. So in the second part of the talk we will discuss popular adaptive learning rates, and in particular we will do a short tutorial on online learning, to give people intuitions on the regret bounds. Finally we will have a live demo session on comparing different learning rates.

This talk is part of the Machine Learning Reading Group @ CUED series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Neonatal Neuroscience Seminars Other medicinal chemistry symposia Lees Knowles Lectures : Total War : The Soviet Union and the Eastern Front in a Comparative Framework Type the title of a new list here epiSTEMe dissemination event

Other talks

Kiwi Scientific Acceleration on FPGA Group covariance functions for Gaussian process metamodels with categorical inputs Cooperation, Construction, Coercion, Consent: Understanding the Role of Reimagined Urban Space within Nazi Germany and Fascist Italy Hypergraph Saturation Irregularities Domain Uncertainty Quantification
 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity