Update

Arjun Panickssery took a crack at this project in the context of MNIST. See his writeup here.

Note: there’s still plenty more exploration to do. You can look at variants of this for other datasets. You can look at the consequences of deliberate interventions like adding label noise. Etc.

Background

MNIST is easy and quick to train. There’s a lot of room to explore how the learning coefficient varies as you change details of the MNIST set up — maybe you vary the number of tasks, or you add noise to some images, or you deliberately mislabel some samples…