Data for Novel Approaches to Stability for Enhanced Privacy-Preserving Machine Learning

Recently, machine learning models have seen considerable growth in size and popularity, lead-ing to concerns regarding dataset privacy, especially around sensitive data containing personal information.To address data extrapolation from model weights, various privacy frameworks ensure that the outputs ofmachine learning models do not reveal their training data. However, this often results in diminished modelperformance due to the necessary addition of noise to model weights. By enhancing models’ resistance tominor variations in input, their stability improves, leading to a reduction in the amount of noise necessarywhile still preserving privacy. This paper explores several techniques to improve stability and mitigate theadverse effects of privatization within the Probably Approximately Correct Privacy Framework in machinelearning, covering both neural networks and linear regressions. Neural network stability methods focuson varying clipping and pruning techniques, in addition to the novel tree-net applied to the context ofstability. Linear regression methods include sharing clipping techniques and introducing a novel group-based clipping method instead of batch-based clipping. Linear regression testing utilizes data embeddingto improve accuracy further and introduces dynamic baseline training, a new method of stability training.Using these methods, we enhance the test accuracy of a privatized Resnet20 on CIFAR-10 from 58.5% to72.5% while upholding the same level of privacy.