Limiting laws and consistent estimation criteria for fixed and diverging number of spiked eigenvalues

In this paper, we study limiting laws and consistent estimation criteria for the extreme eigenvalues in a spiked covariance model of dimension p with the number of spikes k. Allowing both p and k to diverge, we derive limiting distributions of the spiked sample eigenvalues using random matrix theory techniques. Notably, our results are established under a general spiked covariance model, where the bulk eigenvalues are allowed to differ, and the spiked eigenvalues need not be uniformly upper bounded or tending to infinity, as have been assumed in the existing literature. Based on the above derived results, we formulate general estimation criteria that can consistently estimate k, while k can be fixed or grow at an order of k=o(n1/3). Our results are established under both Gaussian distributions and general distributions with finite fourth moments, with different growth rate conditions on k. The effectiveness of the proposed estimation criteria is illustrated through simulation studies and applications to three real-world data sets.