BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Semiparametric Language Models - Dani Yogatama
DTSTART:20201112T140000Z
DTEND:20201112T150000Z
UID:TALK153712@talks.cam.ac.uk
CONTACT:Qianchu Liu
DESCRIPTION:Machine learning models work well on a dataset given enough tr
 aining examples\, but they often fail to isolate and reuse previously acqu
 ired knowledge when the data distribution shifts (e.g.\, when presented wi
 th a new dataset or very long context). In contrast\, humans are able to l
 earn incrementally and accumulate persistent knowledge to facilitate faste
 r learning of new skills without forgetting old ones.\n\nIn this talk\, I 
 will argue that obtaining such an ability for a language model requires si
 gnificant advances in how to represent\, store\, and reuse knowledge acqui
 red from textual data. I will present a semiparametric language model fram
 ework that separates computation (information processing) in a large param
 etric neural network and memory storage in a non-parametric component. I w
 ill show two instantiations of such a model. First\, I will discuss how to
  use it to allow a language model to continually learn new tasks without f
 orgetting old ones. Second\, I will present a language model architecture 
 that adaptively combines local context and global context to make more acc
 urate predictions.
LOCATION:https://teams.microsoft.com/l/meetup-join/19%3ameeting_NjYzNzQyOW
 UtZWM0MC00YzI5LWEwMzMtOGU0ODM4ZDU1NDcw%40thread.v2/0?context=%7b%22Tid%22%
 3a%2249a50445-bdfa-4b79-ade3-547b4f3986e9%22%2c%22Oid%22%3a%227c409a60-a41
 c-43c1-a95a-f00471773d03%22%7d
END:VEVENT
END:VCALENDAR