Data thinning for Poisson factor models and its applications

The Poisson factor model is a powerful tool for dimension reduction and visualization of large-scale count datasets across diverse domains. Despite the availability of efficient algorithms for estimating factors and loadings, existing methods either require prior knowledge of the number of factors, or resort to ad hoc criteria for its determination. This paper proposes a novel data-driven criterion called Information Criterion via Data Thinning (ICDT), leveraging the thinning property of the Poisson distribution. Unlike traditional data splitting, data thinning partitions the count matrix into training and validation sets while preserving both the distribution and the underlying data structure. Interestingly, the validation error can be decomposed into the training error plus a covariance penalty. A simple estimator of the covariance penalty is obtained, leading to the development of ICDT. The selection consistency of ICDT is derived when both the sample size and the number of variables diverge to infinity. The proposed methodology is extended to dimension reduction in regression by incorporating the response inversely into the Poisson factor model. Extensive simulated examples and two real data applications are used to evaluate the performance of ICDT and compare it with existing criteria.

泊松因子模型（Poisson factor model）是支撑多领域大规模计数数据集降维与可视化的有力工具。尽管现有高效算法可用于估计因子与载荷，但现有方法要么需要预先已知因子的数量，要么需借助特设准则来确定该数量。本文提出一种全新的数据驱动准则——基于数据稀疏化的信息准则（Information Criterion via Data Thinning, ICDT），其核心在于利用泊松分布的稀疏化性质。与传统的数据拆分方式不同，数据稀疏化在将计数矩阵划分为训练集与验证集的同时，能够保留数据的分布特性与底层结构。值得注意的是，验证误差可分解为训练误差与协方差惩罚项之和。通过推导出协方差惩罚项的简易估计量，最终构建出ICDT准则。当样本量与变量数均趋于无穷时，可证明ICDT具备选择一致性。本文还将所提方法拓展至回归降维场景，将响应变量反向融入泊松因子模型中。最后通过大量仿真实验与两个真实数据集应用，对ICDT的性能进行评估，并与现有准则展开对比。