BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//talks.cam.ac.uk//v3//EN
BEGIN:VTIMEZONE
TZID:Europe/London
BEGIN:DAYLIGHT
TZOFFSETFROM:+0000
TZOFFSETTO:+0100
TZNAME:BST
DTSTART:19700329T010000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0000
TZNAME:GMT
DTSTART:19701025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CATEGORIES:Statistics
SUMMARY:Minimum L1-norm interpolators: Precise asymptotics
and multiple descent - Yuting Wei (University of
Pennsylvania)
DTSTART;TZID=Europe/London:20220424T140000
DTEND;TZID=Europe/London:20220424T150000
UID:TALK173318AThttp://talks.cam.ac.uk
URL:http://talks.cam.ac.uk/talk/index/173318
DESCRIPTION:An evolving line of machine learning works observe
s empirical evidence that suggests interpolating e
stimators --- the ones that achieve zero training
error --- may not necessarily be harmful. In this
talk\, we pursue a theoretical understanding for a
n important type of interpolators: the minimum L1-
norm interpolator\, which is motivated by the obse
rvation that several learning algorithms favor low
L1-norm solutions in the over-parameterized regim
e. Concretely\, we consider the noisy sparse regre
ssion model under Gaussian design\, focusing on li
near sparsity and high-dimensional asymptotics (so
that both the number of features and the sparsity
level scale proportionally with the sample size).
\n\nWe observe\, and provide rigorous theoretical
justification for\, a curious multi-descent phenom
enon\; that is\, the generalization risk of the mi
nimum L1-norm interpolator undergoes multiple (and
possibly more than two) phases of descent and asc
ent as one increases the model capacity. This phen
omenon stems from the special structure of the min
imum L1-norm interpolator as well as the delicate
interplay between the over-parameterized ratio and
the sparsity\, thus unveiling a fundamental dist
inction in geometry from the minimum L2-norm inter
polator. Our finding is built upon an exact charac
terization of the risk behavior\, which is governe
d by a system of two non-linear equations with two
unknowns.
LOCATION:MR12
CONTACT:Qingyuan Zhao
END:VEVENT
END:VCALENDAR