BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Talks.cam//talks.cam.ac.uk//
X-WR-CALNAME:Talks.cam
BEGIN:VEVENT
SUMMARY:Clustering Dynamics in Mean-Field Models of Transformers - Andrea 
 Agazzi (Universität Bern)
DTSTART:20250826T093000Z
DTEND:20250826T103000Z
UID:TALK234961@talks.cam.ac.uk
DESCRIPTION:\nTransformers are a central architecture in modern deep learn
 ing\, forming the backbone of large language models such as ChatGPT. In th
 is talk\, I will present a mathematical framework for studying how informa
 tion&mdash\;represented as "tokens"&mdash\;evolves through the layers of s
 uch neural networks. Specifically\, we consider a family of partial differ
 ential equations that describe how the distribution of tokens&mdash\;model
 ed as particles interacting in a mean-field way&mdash\;changes with depth.
 \nNumerical experiments reveal that\, under certain conditions\, these dyn
 amics exhibit a metastable clustering phenomenon\, where tokens group into
  well-separated clusters that evolve slowly over time. A rigorous analysis
  of this behavior uncovers a range of open questions and unexpected connec
 tions to analysis and geometry.\n\n&nbsp\;
LOCATION:Seminar Room 2\, Newton Institute
END:VEVENT
END:VCALENDAR