Solving Large-scale Machine Learning Problems

Add to your list(s) Download to your calendar using vCal

• Vinod Kumar (IfM-DIAL, University of Cambridge)
• Thursday 27 June 2019, 11:30-12:30
• Lecture Theatre 1, Institute for Manufacturing, University of Cambridge.

If you have a question about this talk, please contact Shuya Zhong.

Big data is one of the major challenges in machine learning, which leads to slow training and scalability issues of models. In this work, we have identified problem formulation, problem solvers, optimization strategies and platform/framework utilization, as major areas to tackle the challenge. But out of these potential areas, recently, researchers have focused on stochastic approximation algorithms, coordinate descent algorithms, proximal algorithms and parallel & distributed algorithms to tackle the challenge. We have utilized the best of stochastic approximation and coordinate descent approaches to propose a batch block optimization framework (BBOF), which has been used with first and second order methods to solve the large-scale learning problems. But it has been observed that the stochastic approximation and coordinate descent, do not work well when combined together because the advantage is lost in extra overhead to implement BBOF due to double sampling, i.e., sampling of data points and that of features. We have proposed stochastic average adjusted gradient (SAAG-I, II, III and IV) methods, as variance reduction techniques to solve the large-scale problems. We have also proposed stochastic trust region Newton (STRON) method, which solves the Newton system inexactly to handle the large-scale problems. Moreover, simple sampling techniques have been proposed to improve the training time of models by reducing the data access time. We have provided theoretical analysis and empirical results which have proved efficacy of proposed methods against the existing techniques.

This talk is part of the DIAL seminars series.

This talk is included in these lists:

• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge talks
• Centre for Smart Infrastructure & Construction
• Chris Davis' list
• DIAL seminars
• IfM Seminars
• Interested Talks
• Lecture Theatre 1, Institute for Manufacturing, University of Cambridge
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.