Dataset from: Selecting deep neural networks that yield consistent attribution-based interpretations for genomics

Deep neural networks (DNNs) have demonstrated great promise at taking DNA sequences as input and predicting a wide variety of functional activity. Post hoc attribution analysis has been employed to provide insights into the features learned by DNNs, often revealing patterns such as known motifs. However, attribution maps are noisy in practice to an extent that varies from model to model, even across DNNs that yield similar generalization performance. This makes it challenging to identify which high-performing DNN will provide trustworthy explanations. Here we propose a summary statistic that characterizes the consistency of learned features across a population of attribution maps which can be utilized as an additional criterion for model selection. We demonstrate the efficacy of this approach quantitatively using synthetic data and qualitatively with chromatin accessibility data. Together, this work advances our ability to select optimal DNNs that not only yield high generalization performance but also reliable attribution maps that will, in turn, accelerate scientific discovery in genomics.