"Differentially Private Zeroth-Order Methods for Scalable Large Language Model Fine-tuning"
收藏数据链接：
官方服务：
资源简介：
"This is the implementation for the paper Differentially Private Zeroth-Order Methods for Scalable Large Language Model Fine-tuning. In this paper, we investigate the potential of DP zeroth-order methods for LLM pretraining, which avoids the scalability bottleneck of SGD by approximating the gradient with the more efficient zeroth-order gradient. We propose DP zeroth-order stagewise method (DP-ZOSO) and DP zeroth-order stagewise pruning method (DP-ZOPO) with several pruning strategies and undertake a comparative analysis of these strategies."
提供机构：
IEEE DataPort创建时间：
2025-08-12
