Privacy-Threat-Aligned Test Suite and Experimental Materials for Evaluating LLM Guardrails

This record contains the supporting materials for the study on privacy-oriented evaluation of LLM guardrails. It includes a taxonomy-aligned privacy test suite with 600 prompts (300 harmful prompts and 300 matched benign controls), the implementation of the black-box testing strategies used in the study, and the corresponding experimental results. The materials support the analyses reported in the accompanying paper, including the evaluation of privacy-threat coverage and the comparison of observable guardrail behavior across different testing strategies and model variants. These materials accompany the paper “LLM Guardrails for Privacy by Design: A Taxonomy-Driven Evaluation of Privacy Threats.”