Research Agenda
Our research is organized into distinct directions, each advancing our understanding of deep learning through the lens of singular learning theory.
Position Papers
High-level perspectives on our research program and its implications.
Foundations
Building the mathematical and statistical foundations of singular learning theory and its applications to deep learning.
Theory
Progress on singular learning theory, the S4 correspondence, structural Bayesianism, and more.
Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theory
October 14, 2025
We provide an extension of the MDL principle to singular models like neural networks and empirically test the predicted relationship between complexity and compressibility.
Programs as Singularities
April 10, 2025
Sampling
Improving the SGMCMC estimators and the science of hyperparameter selection.
Validation
Sanity checks on the basic scientific claims of SLT and developmental interpretability.
Interpretability
Developing principled approaches to understanding the internal structure and computation of neural networks.
Data Attribution
Applications of susceptibilities to influence functions, via the Bayesian influence function and loss kernel.
Influence Dynamics and Stagewise Data Attribution
October 14, 2025
We study the BIF as a tool for developmental interpretability and show that influence can change dramatically over the course of training, contrary to the classical view of training data attribution.
The Loss Kernel: A Geometric Probe for Deep Learning Interpretability
October 1, 2025
We study the kernel induced by the BIF as a tool for interpretability and show that this recovers ground-truth structure in the training distribution.
Bayesian Influence Functions for Hessian-Free Data Attribution
September 30, 2025
We introduce a Bayesian generalization of influence functions (the BIF) that scales to models with billions of parameters and beats strong baselines on retraining benchmarks.
Spectroscopy
Applications of susceptibilities to interpreting model internals.
Embryology of a Language Model
August 1, 2025
We study the susceptibilities introduced previously as a tool for developmental interpretability to study the embryology of a small language model over training.
Structural Inference: Interpreting Small Language Models with Susceptibilities
April 25, 2025
We introduce "susceptibilities" along with a framework for applying these measurements to discovering structure inside models ("structural inference") and validate this in a small language model.