Spectral Ergodicity in Deep Learning Architectures via Surrogate Random Matrices
收藏官方服务：
资源简介：
The new methodology combines approaches inspired from the Thirumalai-Mountain (TM) metric and metric rooted in Kullbach-Leibler (KL) divergence. The method is applied to a general study of deep and recurrent neural networks via the analysis of random matrix ensembles mimicking typical weight matrices of those systems. In particular, we examine different size circular random matrix ensembles: circular unitary ensemble (CUE), circular orthogonal ensemble (COE), and circular symplectic ensemble (CSE). Eigenvalue spectra and spectral ergodicity are computed for those ensembles as a function of connectivity in the system. The dataset and Python notebook are provided.
提供机构：
Zenodo创建时间：
2017-07-03
