遇见数据集

Linguabase: a public-domain semantic map of English

收藏
Zenodo2026-06-22 更新2026-06-28 收录
官方服务:

资源简介:

Linguabase is a large, sense-aware map of English word meaning, released to the public domain (CC0) by IDEA.org, a nonprofit. It is a set of flat tables describing how English words relate to one another: which words are associated, which are opposed, which share a root, what senses a word has, how to define or clue it, and how to filter it for an audience. More than 250 million word-to-word relationships across 13 tables, keyed on a ranked vocabulary of roughly 2 million words (with a curated 400,000-word “core” tier). Every table ships in two formats — .parquet (typed, columnar) and .tsv.gz (canonical, human-readable) — and, where applicable, in core and full tiers. This archive contains the complete release; see the included README.md and per-table READMEs for the full schema, column documentation, and load examples. Built for game makers, educators, researchers, generative artists, accessibility engineers, and anyone working with English at scale. Provenance. The product of over a decade of work by IDEA.org. Earlier foundations used pre-LLM methods (LDA, Word2Vec) and roughly 2.3 million supercomputer hours, drawing on sources such as WordNet, Roget’s Thesaurus, the NASA Thesaurus, and Library of Congress classifications. The most recent layers used frontier language models largely to validate, rank, characterize, and organize relationships at scale rather than to generate them. Caveats. Linguabase is a rich map, not ground truth; it was built at scale and may contain errors. The full tier includes a deliberately noisy low-familiarity tail you can threshold with a familiarity score. Because it aims to map all of English, it includes profanity and sensitive terms, with sensitivity/rating fields provided for filtering — filter before shipping to an audience. License: CC0 1.0 Universal — dedicated to the public domain. No attribution required, no royalties, commercial use welcome. Project home: linguabase.org · Contact: linguabase@idea.org

提供机构:
IDEA.org
创建时间:
2026-06-22
二维码
社区交流群
二维码
科研交流群
商业服务