Visualizing and Understanding Recurrent Networks

Add to your list(s) Download to your calendar using vCal

• Andrej Karpathy, PhD student, Stanford University
• Thursday 06 August 2015, 12:00-13:00
• Cambridge University Engineering Department, LT1.

If you have a question about this talk, please contact rff22.

Any further queries regarding meeting, please direct them to Ankur Handa ah781@cam.ac.uk

Recurrent Neural Networks (RNNs), and specifically a variant with Long Short-Term Memory (LSTM), are enjoying renewed interest as a result of successful applications in a wide range of machine learning problems that involve sequential data. However, while LST Ms provide exceptional results in practice, the source of their performance and their limitations remain rather poorly understood. Using character-level language models as an interpretable testbed, we aim to bridge this gap by providing a comprehensive analysis of their representations, predictions and error types. In particular, our experiments reveal the existence of interpretable cells that keep track of long-range dependencies such as line lengths, quotes and brackets. Moreover, an extensive analysis with finite horizon n-gram models suggest that these dependencies are actively discovered and utilized by the networks. Finally, we provide detailed error analysis that suggests areas for further study.

This talk is part of the CUED Computer Vision Research Seminars series.

This talk is included in these lists:

• CUED Computer Vision Research Seminars
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge University Engineering Department, LT1
• Cambridge talks
• Chris Davis' list
• Information Engineering Division seminar list
• Interested Talks
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.