BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:[Special Statslab Seminar]  Scalable stochastic optimization and l
 arge-scale data - Michael W. Mahoney (ICSI and Department of Statistics\, 
 UC Berkeley)
DTSTART:20190508T130000Z
DTEND:20190508T140000Z
UID:TALK124819@talks.cam.ac.uk
CONTACT:HoD Secretary\, DPMMS
DESCRIPTION:Stochastic optimization is widely-used in many areas\, most re
 cently in large-scale machine learning and data science\, but its use in t
 hese areas is quite different than its use in more traditional areas of op
 erations research\, scientific computing\, and statistics.  In particular\
 , second order optimization methods have been ubiquitous historically\, bu
 t they are rarely used in machine learning and data science\, compared to 
 their first order counterparts.  Motivated by well-known problems of first
  order methods\, however\, recent work has begun to experiment with second
  order methods for machine learning problems.  By exploiting recent result
 s from Randomized Numerical Linear Algebra\, we establish improved bounds 
 for algorithms that incorporate sub-sampling as a way to improve computati
 onal efficiency\, while maintaining the original convergence properties of
  these algorithms.  These results provide quantitative convergence results
  for variants of Newton's methods\, where the Hessian and/or the gradient 
 is uniformly or non-uniformly sub-sampled\, under much weaker assumptions 
 than prior work\; and these results include extensions to trust region and
  cubic regularization algorithms for non-convex optimization problems.  Wh
 en applied to complex machine learning tasks such as training deep neural 
 networks\, empirical results demonstrate that these methods perform quite 
 well\, both in ways that one would expect (e.g.\, leading to improved cond
 itioning in the presence of so-called exploding/vanishing gradients) as we
 ll as in ways that are more surprising but more interesting (e.g.\, using 
 so-called adversarial examples to architect the objective function surface
  to be more amenable to optimization algorithms).\n\n\n
LOCATION:MR11\, Centre for Mathematical Sciences\, Wilberforce Road\, Camb
 ridge
END:VEVENT
END:VCALENDAR