<p>Formal definition of threat models.</p>

Federated learning faces a fundamental privacy-utility-communication trilemma, and existing static defense mechanisms suffer from rigid adaptation and poor multidimensional coordination, leaving a critical gap in dynamic trade-off balancing. To address this, we propose DualMask, a cooperative optimization framework that integrates a client-side Adaptive Orthogonal Noise Canceler (AONC) with server-side Distributed Dueling Double Deep Q-Network (D3QN) scheduling and Particle Swarm Optimization (PSO)-based aggregation. The AONC module implements a triple-defense mechanism via orthogonal subspace projection: (1) layer-wise adaptive EMA-quantile clipping to mitigate threshold imbalance, (2) progress-aware noise decay that balances early-stage privacy with late-stage efficiency, and (3) directional tuning that dynamically adjusts parallel-to-orthogonal gradient ratios. On the server side, D3QN enables dynamic resource allocation across heterogeneous devices, while PSO fusion corrects non-IID aggregation bias through particle-swarm-based weight optimization. Experiments on CIFAR-10/100 and Shakespeare datasets demonstrate that DualMask achieves 5.2% higher accuracy (84.1% vs 79.4% in non-IID settings) and 34.4% faster convergence (210 vs 320 rounds) compared to FedAvg. Additionally, DualMask reduces the privacy budget from 4.5 to 2.8 and communication cost by 37.2% (45 MB vs 65 MB). This constitutes a significant Pareto improvement, substantially expanding the trilemma frontier. The code and data are available at https://github.com/zhou-weib/DualMask.git.

联邦学习（Federated Learning）面临着根本性的隐私-效用-通信三难困境，现有静态防御机制存在适配僵化、多维度协调能力不足的问题，在动态权衡平衡方面存在关键空白。为解决这一问题，本文提出DualMask框架——一种协同优化框架，集成了客户端自适应正交噪声消除器（Adaptive Orthogonal Noise Canceler, AONC）、服务端分布式对决双深度Q网络（Distributed Dueling Double Deep Q-Network, D3QN）调度器，以及基于粒子群优化（Particle Swarm Optimization, PSO）的聚合策略。
AONC模块通过正交子空间投影实现三重防御机制：(1) 逐层自适应指数移动平均（Exponential Moving Average, EMA）分位数裁剪，以缓解阈值失衡问题；(2) 进度感知噪声衰减，平衡早期训练的隐私保护与后期训练效率；(3) 方向微调，动态调整并行梯度与正交梯度的比例。
在服务端，D3QN可实现异构设备间的动态资源分配，而PSO融合模块则通过基于粒子群的权重优化，修正非独立同分布（non-IID）的聚合偏差。
在CIFAR-10/100与莎士比亚（Shakespeare）数据集上开展的实验表明，相较于联邦平均（FedAvg），DualMask的准确率提升5.2%（非独立同分布场景下为84.1% vs 79.4%），收敛速度加快34.4%（仅需210轮 vs 320轮）。此外，DualMask将隐私预算从4.5降至2.8，通信开销降低37.2%（45 MB对比65 MB）。这一成果实现了显著的帕累托改进（Pareto improvement），大幅拓展了三难困境的可行边界。
相关代码与数据集可在https://github.com/zhou-weib/DualMask.git获取。