devinterp Project
Toy Models of Superposition
Can we classify further transitions in toy models?
This might be the easiest place to quickly get interesting new results. Try variations of the Chen et al. set-up (e.g., 3 hidden dimensions, moving away from the high-sparsity regime, varying importance, etc.). See how this affects the development of these toy models.
Where to Begin
Before starting this project, we recommend familiarizing yourself with these resources:
Ready to contribute? Let us know in our Discord community . We'll update this listing so that other people interested in this project can find you.