CaptchaBench: A Large-Scale Benchmark Dataset for Evaluating Adversarial Perturbation Methods against VLM-based CAPTCHA Recognition

CaptchaBench v1.1 is a large-scale benchmark dataset for evaluating adversarial perturbation methods against Vision-Language Model (VLM)-based Chinese CAPTCHA recognition. This version provides a self-contained stratified evaluation subset: - 4,000 clean CAPTCHA images with JSON metadata (2,000 per generator); - 16,000 adversarial PNG images with paired JSON annotations; - 2 generative pipelines (Illusion Diffusion ControlNet and SDXL); - 8 attack methods, each applied to the same 1,000 source samples per generator: ASPL, Glaze, AMP, XTransfer, AnyAttack, Nightshade, MMCoA, and CoA; - a normalized 16,000-record manifest, source-to-target matching files, and per-method attack metadata. The complete benchmark contains 840,000 base images. This release is the published evaluation subset, not the complete base-image corpus. Each adversarial image is stored as a lossless PNG with a same-stem JSON annotation. `metadata/manifest.jsonl` is the canonical cross-method index; its paths are relative to the archive root and it records the generator, method, modality, source/target pair, character label, and attack parameters. The code, manuscript materials, and package documentation are available at: https://github.com/Linzy19/CaptchaBench