Differentially Private Zeroth-Order Methods for Scalable Large Language Model Fine-tuning
收藏IEEE2026-04-17 收录
数据链接：
官方服务：
资源简介：
This is the implementation for the paper Differentially Private Zeroth-Order Methods for Scalable Large Language Model Fine-tuning. In this paper, we investigate the potential of DP zeroth-order methods for LLM pretraining, which avoids the scalability bottleneck of SGD by approximating the gradient with the more efficient zeroth-order gradient. We propose DP zeroth-order stagewise method (DP-ZOSO) and DP zeroth-order stagewise pruning method (DP-ZOPO) with several pruning strategies and undertake a comparative analysis of these strategies.
提供机构：
Zhihao Liu
