Anti-Bot Adoption Index — WAF / anti-bot vendor scan of the Tranco top 1M

An independent anti-bot / WAF scan of the Tranco top 1,000,000 sites (run top1m-20260614). Every reachable homepage is passively fingerprinted to detect which managed anti-bot / WAF / bot-management vendor protects it. Headline: 998,497 sites probed; 818,614 reachable; 53.5% of reachable run a managed anti-bot / WAF. Top vendors (share of reachable): Cloudflare 45.0%, Google reCAPTCHA 7.5%, hCaptcha 3.5%, Cloudflare Turnstile 1.3%. The busiest sites run the least anti-bot (top-1k: 44.2%) and protection rises down the long tail (100k–1M: 53.6%). Files: top1m.jsonl.gz (998,497 rows), a 200-row uncompressed sample, headline aggregates, and a deeper multi-signal scan of the top 50k. One JSON record per line (domain, rank, reachable, protected, vendors, kinds, difficulty_band, block_reason, enforcement, final_url, status, probed_at). Methodology: passive homepage fingerprint from a datacenter IP — HTTP status, header names, Set-Cookie names only (never values, no PII), and challenge-shaped body markers vs documented public vendor fingerprints. One GET per homepage; no forms, no CAPTCHA solving. License: CC BY 4.0. Source: github.com/Crawlora-org/anti-bot-adoption-index-data; explorer: crawlora.net/anti-bot-index.