Exploring Inverse Scaling
I recently submitted an entry for the Inverse Scaling Prize, and while it wasn’t selected, I think it still reveals some interesting properties of model scaling that are worth exploring (and are similar to those analyzed in one winning submission from Cavendish Labs). The goal of the competition is to identify machine learning tasks where …