News
A collection of posts written by various people associated with Developmental Interpretability (since before the agenda was conceived).
Event
The Australian AI Safety Forum 2024
Nov 07, 2024
Paper
Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient
Wang et al. Oct 04, 2024
Blog
Singular learning theory: exercises
Furman Aug 30, 2024
Event
ILIAD 2024
Aug 28, 2024
Blog
So you want to work on technical AI safety
Wang Aug 24, 2024
Paper
Loss landscape geometry reveals stagewise development of transformers
Wang et al. Jun 16, 2024
Blog
Stagewise Development in Neural Networks
Hoogland et al. Mar 20, 2024
Blog
Simple versus Short: Higher-order degeneracy and error-correction
Murfet Mar 11, 2024
Blog
Timaeus's First Four Months
Hoogland et al. Feb 28, 2024
Paper
Estimating the Local Learning Coefficient at Scale
Furman and Lau Feb 06, 2024
Paper
The Developmental Landscape of In-Context Learning
Hoogland et al. Feb 04, 2024
Blog
Generalization, from thermodynamics to statistical physics
Hoogland Nov 30, 2023
Blog
Learning coefficient estimation: the details
Furman Nov 15, 2023
Event
The 2023 Oxford Conference
Nov 05, 2023
Blog
Announcing Timaeus
Hoogland et al. Oct 22, 2023
Blog
You're Measuring Model Complexity Wrong
Hoogland and Wingerden Oct 11, 2023
Event
The 2023 Melbourne Hackathon
Oct 07, 2023
Event
The 2023 Amsterdam Retreat
Sep 18, 2023
Paper
Quantifying Degeneracy in Singular Models via the Learning Coefficient
Lau et al. Aug 23, 2023
Blog
DSLT 4. Phase Transitions in Neural Networks
Carroll Jun 24, 2023
Blog
DSLT 3. Neural Networks are Singular
Carroll Jun 20, 2023
Event
The 2023 Berkeley Conference
Jun 19, 2023
Event
The Primer
Jun 19, 2023
Blog
DSLT 2. Why Neural Networks obey Occam's Razor
Carroll Jun 18, 2023
Blog
DSLT 1. The RLCT Measures the Effective Dimension of Neural Networks
Carroll Jun 16, 2023
Blog
DSLT 0. Distilling Singular Learning Theory
Carroll Jun 15, 2023
Blog
Approximation is expensive, but the lunch is cheap
Hoogland Apr 19, 2023
Blog
Empirical risk minimization is fundamentally confused
Hoogland Mar 22, 2023
Blog
The shallow reality of 'deep learning theory'
Hoogland Feb 22, 2023
Blog
Gradient surfing: the hidden role of regularization
Hoogland Feb 06, 2023
Blog
Interview Daniel Murfet on Universal Phenomena in Learning Machines
Oldenziel Feb 06, 2023
Blog
Spooky action at a distance in the loss landscape
Hoogland Jan 28, 2023
Blog
Neural networks generalize because of this one weird trick
Hoogland Jan 18, 2023