Interpreting deep learning by establishing a rigorous corresponding relationship with the renormalization group on the Ising model

We focus on the interpretability of deep neural networks (DNNs). DNNs are currently the most widely used models in the field of artificial intelligence. However, they have been considered to lack interpretability. The most significant advantage of DNNs is that they can extract the features from big data effectively. The renormalization group (RG) is a kind of method in statistical physics. Physics research has shown that this method can effectively derive the macroscopic features from the microscopic characteristics of statistical physical systems. The coarse-graining procedure of the real-space RG is quite similar to the calculation in the forward propagation of DNNs. Inspired by this viewpoint, we consider establishing a corresponding relationship between the training process of the DNNs and RG. We propose a new framework to study the interpretability of DNNs using the RG method, and our main results are as follows: (1) Considering the input data and the main features extracted by a DNN as two statistical physical systems, we propose a rigorous correspondence relationship between the RG of input data and the training process of the DNN. Therefore, the DNN can be seen as a system artificially defined by its parameters, which carries out a non-canonical RG process on the data. (2) We prove that, when the input dataset is the one-dimensional Ising model, after the training process of the fully connected DNN, the limit of the coupling constant in the partition function of the network output is the same as the stable fixed point of the coupling constant calculated by the real-space RG on the input dataset. This shows that the ability of feature extraction of the DNN originates from the equivalence between their training process and the canonical RG. Also, both the DNN and RG extract the same macroscopic features from the data, which are the contents actually learned by the DNN.