EuroSAT Model Zoo: A Dataset of Diverse Populations of Neural Network Models - EuroSAT

Abstract

In the last years, neural networks have evolved from laboratory environments to the state-of-the-art for many real-world problems. Our hypothesis is that neural network models (i.e., their weights and biases) evolve on unique, smooth trajectories in weight space during training. Following, a population of such neural network models (refereed to as “model zoo”) would form topological structures in weight space. We think that the geometry, curvature and smoothness of these structures contain information about the state of training and can be reveal latent properties of individual models. With such zoos, one could investigate novel approaches for (i) model analysis, (ii) discover unknown learning dynamics, (iii) learn rich representations of such populations, or (iv) exploit the model zoos for generative modelling of neural network weights and biases. Unfortunately, the lack of standardized model zoos and available benchmarks significantly increases the friction for further research about populations of neural networks. With this work, we publish a novel dataset of model zoos containing systematically generated and diverse populations of neural network models for further research. In total the proposed model zoo dataset is based on six image datasets, consist of 27 model zoos with varying hyperparameter combinations are generated and includes 50’360 unique neural network models resulting in over 2’585’360 collected model states. Additionally, to the model zoo data we provide an in-depth analysis of the zoos and provide benchmarks for multiple downstream tasks as mentioned before.

Dataset

This dataset is part of a larger collection of model zoos and contains the zoos trained on EuroSAT. All zoos with extensive information and code can be found at www.modelzoos.cc.

This repository contains two types of model populations: the base model zoo ("eurosat_cnn_kaiming_uniform.zip"), as well as a collection of sparsified model zoos (filenames ending in "magn_XX.zip" or "ard.zip"). Zoos are trained with CNN models in configurations varying the seed only (seed), and sparsification is done through magnitude-based weight pruning ("magn_XX.zip") or  varational dropout ("ard.zip").

For more information on the zoos and code to access and use the zoos, please see www.modelzoos.cc.