University of Cambridge > Talks.cam > Statistics > Data-driven calibration of linear estimators with minimal penalties, with an application to multi-task regression

  Browse A-Z

Log in

Cambridge users (raven)details Other usersdetails No account?details

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

Data-driven calibration of linear estimators with minimal penalties, with an application to multi-task regression

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Richard Samworth.

This talk tackles the problem of selecting among several linear estimators in non-parametric regression; this includes model selection for linear regression, the choice of a regularization parameter in kernel ridge regression or spline smoothing, the choice of a kernel in multiple kernel learning, the choice of a bandwidth for Nadaraya-Watson estimators, and the choice of k for k-nearest neighbors regression.

We propose a new algorithm which first estimates consistently the variance of the noise, based upon the concept of minimal penalty which was previously introduced in the context of model selection. Then, plugging our variance estimate in Mallows’ C_L penalty is proved to lead to an algorithm satisfying an oracle inequality. Simulation experiments show that the proposed algorithm often improves significantly existing calibration procedures such as 10-fold cross-validation or generalized cross-validation.

We then provide an application to the kernel multiple ridge regression framework, which we refer to as multi-task regression. The theoretical analysis of this problem shows that the key element appearing for an optimal calibration is the covariance matrix of the noise between the different tasks. We present a new algorithm for estimating this covariance matrix, based upon several single-task variance estimations. We show, in a non-asymptotic setting and under mild assumptions on the target function, that this estimator converges towards the covariance matrix. Then, plugging this estimator into the corresponding ideal penalty leads to an oracle inequality. We illustrate the behaviour of our algorithm on synthetic examples.

This talk is based on two joint works with Francis Bach and Matthieu Solnon:

S. Arlot, F. Bach. Data-driven Calibration of Linear Estimators with Minimal Penalties. arXiv:0909.1884

M. Solnon, S. Arlot, F. Bach. Multi-task Regression using Minimal Penalties. arXiv:1107.4512

This talk is part of the Statistics series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Zoology Cambridge University Railway Club Vascular Biology Research Seminars

Other talks

Discovering regulators of insulin output with flies and human islets: implications for diabetes and pancreas cancer Visual hallucinations in Parkinson’s disease - imbalances in top-down vs. bottom up information processing CANCELLED: The rise and fall of the Shopping Mall: dialogues on the relationship of commerce and city 160 years of occupational structure: Late Imperial China and its regions Active Subspace Techniques to Construct Surrogate Models for Complex Physical and Biological Models Louisiana Creole - a creole at the periphery Computing High Resolution Health(care) Cambridge-Lausanne Workshop 2018 - Day 1 An approach to the four colour theorem via Donaldson- Floer theory Vision Journal Club: feedforward vs back in figure ground segmentation The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age Developing an optimisation algorithm to supervise active learning in drug discovery
 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity