Fields Mathematical AI Seminar

I'll discuss some results on the question of how \emph{semantic} relationships familiar to humans as \emph{geometric} relationships between their activations. The essential idea is that the exponentiated dot product form used by softmax distributions combined with the requirement that models can compose `causally seperable' concepts imposes non-trivial geometric structure. In particular, we will see that information geometry---particularly, the geometry of exponential family distributions---plays an important role.