Clustering Analysis of the Latent Space of Reinforcement Learning Agents on Atari

Results of the clustering analysis of the latent space of reinforcement learning agents trained on six Atari environments: Breakout, Mario Bros, Ms. Pac-Man, Pong, Space Invaders and Surround. For each environment, several layers of the agent's network are analysed (8 layers for Breakout, Mario Bros, Ms. Pac-Man and Surround; 6 layers for Pong and Space Invaders). The latent space of each layer is clustered with k-means for a number of clusters ranging from 2 to 20, using 3 surrogate policies. Key results: the three main metrics, all measured with the adjusted Rand index (ARI), are clustering universality, adjusted clustering universality and abstraction score. For a given environment and layer, they are located in:<environment>/latent_space_analysis/layer_<l>/result/clustering_universality_adjusted_rand_score.(pdf|txt)<environment>/latent_space_analysis/layer_<l>/result/adjusted_clustering_universality_adjusted_rand_score.(pdf|txt)<environment>/latent_space_analysis/layer_<l>/result/abstraction_score_adjusted_rand_score.(pdf|txt) Files:- 13 archives (one per environment; larger environments are split by groups of layers, e.g. ms_pacman_layers_1-3.zip). Extracting all archives into the same directory rebuilds the complete tree.- README.md: list of archives, detailed structure and description of all metrics. The videos of the agents are published in a separate record: https://doi.org/10.5281/zenodo.23019943 Code: https://github.com/Maxalaar/Clustering_Agent_Latent_Space