CrawlCheck AI-use declarations, top 10,000 (rights feed edition) rf-2026-11-04-b1fc4d
收藏官方服务:
资源简介:
What the top 10,000 sites declare about AI use: one signed line per site with its robots.txt groups for named AI agents, Content-Signal line and llms.txt presence, as read at read_at. Each edition is a signed manifest (sha256 per part, Ed25519 over the manifest) and gzip NDJSON parts. Spec: https://crawlcheck.io/spec/rights-feed. Edition rf-2026-11-04-b1fc4d. Canonical copy and verification: https://crawlcheck.io/spec/rights-feed. Archive URL: https://crawlcheck.io/api/v1/rights-feed/rf-2026-11-04-b1fc4d/manifest.json. Edition sha256: 1fade8929b87224777e3b9f9d756bac9fe4ea039a06adbe98eb57099a911154d. Every CrawlCheck dataset: https://crawlcheck.io/datasets.
提供机构:
Zenodo创建时间:
2026-11-04



