Fields Mathematical AI Seminar

Neural networks trained on group operations exhibit striking structure: generalisation, regular learning curves and representations. The task's simplicity also offers analytical clarity impossible elsewhere, making it uniquely valuable for understanding learning dynamics. To benefit from these advantages, we must first understand the computations networks perform: we have shown that seemingly conflicting circuit descriptions all implement the same algorithm, an approximate Chinese Remainder Theorem, recovered via multiscale analysis. We further find that the hidden-layer features form geometrically equivalent manifolds (or projections of) across hundreds of independently trained networks, revealing the simple geometric structure of neuronal group representations.