Piperic Open Datasets: Top 1 Million Websites & AI Crawler Registry

Two openly licensed datasets derived from an independent index of 50M+ live, content-validated websites. Top 1 Million Websites ranks domains by how many distinct domains link to them from their homepages — a link-evidence alternative to traffic-based popularity lists, resistant to link spam because each linking domain counts once. AI Crawler Registry covers 969 AI crawlers and AI-adjacent bots with operator, purpose (training vs. answer vs. agent), robots.txt compliance, user-agent patterns and verification hints, with per-row source attribution. Live versions: piperic.com/tools/top-sites and piperic.com/tools/ai-bots.