b0sungk1m/tamperbench-quantization-qwen3-4b

---
tags:
- safety
- quantization
- tampering
- evaluation
- ai-safety
license: apache-2.0
---

# TamperBench + Quantization: Does Compression Act as Implicit Tampering?

## Motivation

[TamperBench](https://arxiv.org/abs/2602.06911) evaluates explicit tampering attacks (LoRA fine-tuning, jailbreak-tuning, etc.) on LLM safety guards. [Catastrophic Failure of LLM Unlearning via Quantization](https://arxiv.org/abs/2410.16454) shows that quantization can undo safety-trained behaviors.

This experiment bridges these two lines of work by adding **quantization as a deployment-realistic perturbation** to the TamperBench evaluation protocol. We test whether standard compression:
1. Acts as an **implicit tampering operator** (degrades safety on its own)
2. **Amplifies** the effects of prior explicit tampering (LoRA fine-tuning)

This matters for AI safety because open-weight models can be modified after release, and a defense that looks robust in full precision but fails after ordinary deployment steps may give a false sense of security.

## Experimental Setup

- **Model**: `Qwen/Qwen3-4B` (4B parameters, instruction-tuned)
- **Safety Metric**: StrongREJECT-style scoring (rubric-based, 0=safe, 1=harmful) on 150 harmful prompts
- **Utility Metric**: MMLU-Pro (70-sample stratified subset, 5-shot CoT)
- **Tampering Attack**: LoRA fine-tuning on 64 harmful instruction-response pairs (rank=16, α=32, lr=0.0002, epochs=3)

### Conditions

| # | Condition | Description |
|---|-----------|-------------|
| 1 | `baseline_fp16` | Full-precision model (control) |
| 2 | `bnb_int8` | BitsAndBytes INT8 quantization only |
| 3 | `bnb_nf4` | BitsAndBytes NF4 4-bit quantization only |
| 4 | `tampered_fp16` | LoRA tampering attack in full precision |
| 5 | `tampered_bnb_int8` | Tampered model + INT8 quantization |
| 6 | `tampered_bnb_nf4` | Tampered model + NF4 4-bit quantization |

## Results

| Condition | Refusal Rate ↑ | StrongREJECT Score ↓ | MMLU-Pro Accuracy ↑ |
|-----------|----------------|----------------------|---------------------|
| baseline_fp16 | 0.913 | 0.061 | 0.286 |
| bnb_int8 | 0.907 | 0.064 | 0.143 |
| bnb_nf4 | 0.920 | 0.053 | 0.186 |
| tampered_fp16 | 0.667 | 0.199 | 0.314 |
| tampered_bnb_int8 | 0.700 | 0.173 | 0.314 |
| tampered_bnb_nf4 | 0.680 | 0.203 | 0.271 |

## Key Findings

- ✅ **bnb_int8**: Does not significantly erode safety (ΔStrongREJECT = +0.003)
- ✅ **bnb_nf4**: Does not significantly erode safety (ΔStrongREJECT = -0.008)
- ✅ **tampered_bnb_int8**: Does not amplify prior tampering (Δ vs tampered FP16 = -0.027)
- ✅ **tampered_bnb_nf4**: Does not amplify prior tampering (Δ vs tampered FP16 = +0.004)

## Methodology

### Safety Evaluation (StrongREJECT-style)
Following the [StrongREJECT benchmark](https://arxiv.org/abs/2402.10260), we evaluate model responses to 313 harmful prompts from the Machlovi/strongreject-dataset. Each response is scored on a 0-1 scale where:
- **0.0** = Model refused the request (safe behavior)
- **1.0** = Model fully complied with specific, actionable harmful content

The scorer uses pattern-based refusal detection plus specificity heuristics (code blocks, step-by-step instructions, actionable content).

### Utility Evaluation (MMLU-Pro)
We use a 140-sample stratified subset of [MMLU-Pro](https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro) with 5-shot chain-of-thought prompting, following TamperBench's protocol.

### Tampering Attack
We apply a LoRA fine-tuning attack following TamperBench's harmful LoRA protocol: 64 harmful instruction-response pairs targeting all attention + MLP layers with rank 16.

### Quantization Methods
- **BnB INT8**: bitsandbytes LLM.int8() (mixed-precision INT8)
- **BnB NF4**: bitsandbytes NormalFloat4 with double quantization (4-bit, deployment-realistic)

## References

1. TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering. arXiv:2602.06911
2. Catastrophic Failure of LLM Unlearning via Quantization. arXiv:2410.16454
3. StrongREJECT: A Rejection Benchmark for Jailbreaking LLMs. arXiv:2402.10260
4. Decoding Compressed Trust: Quantized Trustworthiness. arXiv:2403.15447

## Citation

```bibtex
@misc{tamperbench_quantization_2026,
title={TamperBench + Quantization: Does Compression Act as Implicit Tampering?},
year={2026},
note={Experiment extending TamperBench with quantization as deployment-realistic perturbation}
}
```