遇见数据集

CrawlCheck AI-use declarations, top 10,000 (rights feed edition) rf-2026-11-04-b1fc4d

收藏
Zenodo2026-11-04 更新2026-11-05 收录
官方服务:

资源简介:

What the top 10,000 sites declare about AI use: one signed line per site with its robots.txt groups for named AI agents, Content-Signal line and llms.txt presence, as read at read_at. Each edition is a signed manifest (sha256 per part, Ed25519 over the manifest) and gzip NDJSON parts. Spec: https://crawlcheck.io/spec/rights-feed. Edition rf-2026-11-04-b1fc4d. Canonical copy and verification: https://crawlcheck.io/spec/rights-feed. Archive URL: https://crawlcheck.io/api/v1/rights-feed/rf-2026-11-04-b1fc4d/manifest.json. Edition sha256: 1fade8929b87224777e3b9f9d756bac9fe4ea039a06adbe98eb57099a911154d. Every CrawlCheck dataset: https://crawlcheck.io/datasets.

提供机构:
Zenodo
创建时间:
2026-11-04
二维码
社区交流群
二维码
科研交流群
商业服务