University of Cambridge > Talks.cam > Foundation AI > Positional encodings in LLMs

  Browse A-Z

Log in

University Account External (via Google)

🔐 Log In

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

ℹī¸ Information

Positional encodings in LLMs

Download to your calendar using vCal

If you have a question about this talk, please contact Pietro Lio .

Positional encodings are essential for transformer-based language models to understand sequence order, yet their influence extends far beyond simple position tracking. This talk explores the landscape of positional encoding methods in LLMs and reveals surprising insights about how these architectural choices shape model behavior.

We begin with the fundamental challenge: why attention mechanisms require explicit positional information. We then survey the evolution of encoding strategies, from sinusoidal approaches to modern techniques like RoPE, examining their architectural implications and trade-offs.

The talk delves into how these different encoding strategies fundamentally shape model architectures and representations. We analyze the specific limitations and trade-offs of each approach, examining how positional information propagates through transformer layers and influences the learned representations.

Watch it remotely

This talk is part of the Foundation AI series.

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

Average on weekdays: 9040.23466258 kWh Average on weekdays: 9028.58990536 kWh Seo

Other talks

Title abstract tbc External Seminar - Jenn Brophy TBC Title to be confirmed Psychiatry and Pain Medicine Disentangling preventative & ejective AGN feedback on a population basis with simulations, observations & machine learning Climate, uncertainty, and the slippery measurement of Antarctic ice
 

Š 2006-2025 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity