Differentially Private Zeroth-Order Methods for Scalable Large Language Model Fine-tuning
收藏IEEE2026-04-17 收录
下载链接：
https://ieee-dataport.org/documents/differentially-private-zeroth-order-methods-scalable-large-language-model-fine-tuning
下载链接
链接失效反馈官方服务：
资源简介：
This is the implementation for the paper Differentially Private Zeroth-Order Methods for Scalable Large Language Model Fine-tuning. In this paper, we investigate the potential of DP zeroth-order methods for LLM pretraining, which avoids the scalability bottleneck of SGD by approximating the gradient with the more efficient zeroth-order gradient. We propose DP zeroth-order stagewise method (DP-ZOSO) and DP zeroth-order stagewise pruning method (DP-ZOPO) with several pruning strategies and undertake a comparative analysis of these strategies.
提供机构：
Zhihao Liu
