Fields Mathematical AI Seminar

Understanding how neural networks learn structured representations remains a central challenge in deep learning theory. I will present Alternating Gradient Flows (AGF), a framework for analyzing feature learning in small-initialization regimes. Applied to modular addition, AGF explains the sequential emergence of Fourier features. I will then discuss sequential group composition, where two-layer networks learn irreducible representations one at a time but require width exponential in sequence length, while deeper networks exploit associativity to find efficient solutions. Together, these results connect optimization dynamics, representation theory, and the computational benefits of depth. Lastly, I will present ongoing work demonstrating how these works can help explain grid cells in the brain, neurons thought to support spatial navigation and path integration.

This talk is based on:

https://arxiv.org/abs/2506.06489 https://arxiv.org/abs/2602.03655 and an upcoming work soon to be put online.