University of Cambridge > Talks.cam > Language Technology Lab Seminars > How to Pay Attention: Learning to Transfer Knowledge between Sentences and Tokens

How to Pay Attention: Learning to Transfer Knowledge between Sentences and Tokens

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Edoardo Maria Ponti.

Self-attention architectures allow models to dynamically decide which areas of the input should receive more focus. During the construction of text representations, attention weights also provide a way of quantifying the importance of different input areas. In this talk, we investigate how attention mechanisms can be turned into sequence labelers, opening up some new and interesting applications. These networks learn to predict labels for individual tokens, based only on sentence-level supervision, even without having seen any examples of sequence labeling. In addition, optimizing on the token level explicitly teaches the model where it should be focusing, leading to improvements in text classification. We will also discuss experiments with learning directly from the human cognitive signal, guiding the models to internally behave more like their users. The resulting architectures for text classification and sequence labeling are more accurate, more interpretable and make decisions in more predictable ways.

This talk is part of the Language Technology Lab Seminars series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

 

© 2006-2019 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity