Generalization in deep learnin g remains poorly understood\, as neural networks fall outside the framewor k of classical statistical learning theory. To make progress on this quest ion\, research has focused on controlled tasks\, such as modular addition\ , as a testbed for generalization. On this task\, models exhibit grokking\ , i.e. a delayed onset of generalization after training loss has converged . Prior work has identified empirical regularities in learned representati ons associated with this transition\, but the mapping between representati on structure and generalization behavior remains empirical and descriptive . We lack a predictive theory of why and when generalization occurs. In th is work\, we provide such a predictive theory for the modular addition tas k. We introduce the notion of canonical representation of a task: the repr esentation determined by the target function prior to training which is ne eded for perfect generalization. For modular addition\, the canonical repr esentation is derivable explicitly from the group structure of the task. We then define \\representational deviation as the alignment of the learne d representation to the canonical representation. From this\, we derive th at generalization up to a chosen margin requires the representational devi ation to fall below a threshold. We finally provide a set of reproducible experiments which empirically confirm the above findings and offer a regul arizer to accelerate the grokking transition.
**M atthieu Tehenan is a PhD candidate in NLIP Group\, Department of Computer Science &\; Technology\, supervised by Prof Andreas Vlachos.**
LOCATION:SS03 Hybrid (In-Person + Online). Here is the Google Meet Link: h ttps://meet.google.com/cru-hcuo-rhu END:VEVENT END:VCALENDAR