Online Meta-Learning

Add to your list(s) Download to your calendar using vCal

• Massimiliano Pontil, University College London
• Thursday 14 February 2019, 11:00-12:00
• Engineering Department, CBL Room BE-438..

If you have a question about this talk, please contact Carl Edward Rasmussen.

We study the problem in which a series of learning tasks are observed sequentially and the goal is to incrementally adapt a learning algorithm in order to improve its performance on future tasks. The tasks are sampled from a meta-distribution, called the environment in the learning-to-learn literature (Baxter 2000). We focus on linear learning algorithms based on regularized empirical risk minimization such as ridge regression or support vector machines. The algorithms are parametrized by either a (representation) matrix applied to the raw inputs or by a bias vector used in the regularizer. In both settings, we develop a computational efficient meta-algorithm to incrementally adapt the learning algorithm after a task dataset is observed. The meta-algorithm performs stochastic gradient descent on a proxy objective of the risk of the learning algorithm. We derive bounds on the performance of the meta-algorithm, measured by the average risk of the learning algorithm on random tasks from the environment. Our analysis leverages ideas from multitask learning and learning-to-learn with tools from online learning and stochastic optimization. In the last part of the talk, we discuss extensions of the framework to nonlinear models such a deep neural nets and draw links between meta-learning, bilevel optimization and gradient-based hyperparameter optimization.

This talk is part of the Machine Learning @ CUED series.

This talk is included in these lists:

• Seminar
• All Talks (aka the CURE list)
• Biology
• CBL important
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge Forum of Science and Humanities
• Cambridge Language Sciences
• Cambridge Neuroscience Seminars
• Cambridge talks
• Chris Davis' list
• Creating transparent intact animal organs for high-resolution 3D deep-tissue imaging
• Engineering Department, CBL Room BE-438.
• Featured lists
• Guy Emerson's list
• Hanchen DaDaDash
• Inference Group Summary
• Information Engineering Division seminar list
• Interested Talks
• Joint Machine Learning Seminars
• Life Science
• Life Sciences
• ML
• Machine Learning @ CUED
• Machine Learning Summary
• Neuroscience
• Neuroscience Seminars
• Neuroscience Seminars
• Required lists for MLG
• Simon Baker's List
• Stem Cells & Regenerative Medicine
• Trust & Technology Initiative - interesting events
• bld31
• dh539
• kt532's list
• ndk22's list
• ob366-ai4er
• rp587
• yk373's list

Note that ex-directory lists are not shown.