MAD Benchmark

Mechanistic Architecture Design (in-short "MAD") represents a simple framework to accelerate the deep learning architecture design process. MAD uses simple synthetic tasks that can be implemented quickly and without much compute to predict how well new candidate architectures will perform at scale in sequence modeling. Each synthetic task is specifically designed to probe skills of a model relevant for squence modeling, such as compression and recall. Here, we host a set of benchmark datasets for MAD. For more details, see our accompanying GitHub repository: github.com/athms/mad-lab