Abstract

The problem of Large Vocabulary Continuous Speech Recognition (LVCSR) can be cast as a general problem of supervised learning where given some seen examples, the task is to learn the relationship between the input space and the output space from the data. Ideally, we wish to choose a prediction function that avoids rote memorization and instead generalizes the concepts that can be learned from a given set of utterances. This involves choosing an appropriate prediction function from a space of predictive functions that minimizes a risk measure over an adequately selected family of prediction functions. In practice, rather than consider a variational optimization problem over a generic family of prediction functions, we assume that the prediction function has a fixed form which in the context of LVCSR corresponds to Hybrid HMM -Deep Neural Networks models of different network topologies. The focus of this work is to investigate an effective coupling between the use of various optimisation methods and network topologies for effective training of large hours of speech data. In this work, we will particularly investigate optimisation methods that try to combine the best properties of batch and stochastic algorithms while making careful considerations towards the computational time and number of updates.