A trip down long short-term memory lane

Add to your list(s) Download to your calendar using vCal

• Petar Veličković (University of Cambridge)
• Tuesday 21 February 2017, 13:00-14:00
• Computer Laboratory, William Gates Building, Room SW01.

If you have a question about this talk, please contact Mariana Marasoiu.

Extending neural networks to handle sequential inputs of arbitrary lengths (such as time-series, statements in natural language, or mathematical expressions to evaluate) impedes the application of standard fully-connected or convolutional neural networks, which will typically require the input to be fixed in size. To alleviate the issue, recurrent neural networks introduce a learnable unit of computation, capable of processing inputs step-by-step in a length-independent manner. In fact, if feedforward neural networks were concerned with learning desirable functions that consume the input, recurrent neural networks may be seen as learning desirable programs that process the input.

In this lecture, I will introduce recurrent neural networks from first principles, and illustrate the many issues that arise with applying them naïvely. As a popular solution to those issues, the long short-term memory (LSTM) cell will be introduced, with a detailed intuitive and theoretical description of its mode of operation. This will be followed-up with strategies for tackling several kinds of sequential problems using LST Ms, and a survey of real-world applications of this model. Finally, a recently proposed non-recurrent model that shows promising results on sequential tasks will be outlined, to provide perspective for potential future trends in the field. No prior knowledge of neural networks or machine learning is needed, but an entry-level knowledge of supervised learning principles will be beneficial. Code examples that demonstrate how recurrent models can be constructed in only a few lines of Python will also be provided.

This talk is part of the Research Students Lecture Series series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge talks
• Computer Laboratory, William Gates Building, Room SW01
• Computing Education Research
• Department of Computer Science and Technology talks and seminars
• Guy Emerson's list
• Interested Talks
• School of Technology
• Trust & Technology Initiative - interesting events
• bld31

Note that ex-directory lists are not shown.