Update

Arjun Panickssery took a crack at this project in the context of MNIST. See his writeup here.

Note: there’s still plenty more exploration to do. You can look at variants of this for other datasets. You can look at the consequences of deliberate interventions like adding label noise. Etc.

---

Background

MNIST is easy and quick to train. There’s a lot of room to explore how the learning coefficient varies as you change details of the MNIST set up — maybe you vary the number of tasks, or you add noise to some images, or you deliberately mislabel some samples…