CaptchaBench: A Large-Scale Benchmark Dataset for Evaluating Adversarial Perturbation Methods against VLM-based CAPTCHA Recognition

CaptchaBench v1.1 extends the representative evaluation subset from six to eight adversarial perturbation methods. This version preserves the v1.0 archive, which contains 4,000 clean CAPTCHA images and 12,000 adversarial images from the original six methods. It adds 4,000 adversarial images generated by two additional methods: - AnyAttack (image-only): 2 generators × 1,000 samples- CoA / Chain-of-Attack (image+text): 2 generators × 1,000 samples The complete evaluation subset now contains 16,000 adversarial PNG images:2 generators × 8 methods × 1,000 samples. Each adversarial image has a paired JSON annotation. The metadata archive includes a normalized 16,000-record manifest, source-target matching files, attack-run metadata, method parameters, and provenance information. Files added in v1.1:- CaptchaBench_16k_ID_AnyAttack.tar.gz- CaptchaBench_16k_ID_CoA.tar.gz- CaptchaBench_16k_SDXL_AnyAttack.tar.gz- CaptchaBench_16k_SDXL_CoA.tar.gz- CaptchaBench_16k_metadata.tar.gz- SHA256SUMS_v1.1 Code and full package documentation:https://github.com/Linzy19/CaptchaBench