Whalemini/grokipedia-v0.1-dump

---
license: other
license_name: combo-grok-2-cc
license_link: LICENSE
language:
- en
pretty_name: Grokipedia v0.1 Scrape
---

# Grokipedia v0.1 Scrape

This dataset represents a strctured, nearly-full point-in-time scrape of Grokipedia v0.1 as of the end of October / beginning of November 2025.

It also includes embeddings of 250-token semi-overlapping chunks of the Grokipedia corpus.

It was collected and initially used for Harold Triedman and Alexios Mantzarlis' November 2025 paper: "What did Elon Change? A comprehensive analysis of Grokipedia" ([arxiv](https://arxiv.org/abs/2511.09685)).

If you use this dataset, please cite it as follows

```bibtex
@misc{TriedmanManzarlisGrokipedia2025,
title = {What did Elon change? A comprehensive analysis of Grokipedia},
shorttitle = {What did Elon change?},
year = 2025,
month = nov,
number = {arXiv:2511.09685},
eprint = {2511.09685},
primaryclass = {cs},
publisher = {arXiv},
doi = {10.48550/arXiv:2511.09685},
}
```

## File Format

- **Format**: Newline-delimited JSON (NDJSON)
- **Encoding**: UTF-8
- **One JSON object per line**

## Root Object Schema

Each line in the file is a JSON object with the following structure:

```json
{
"title": string,
"url": string,
"data": ArticleData,
"scraped_at": string (ISO 8601 datetime)
}
```

### Root Level Fields

| Field | Type | Description |
|-------|------|-------------|
| `title` | string | The URL-encoded article title (e.g., "Yamaha_Venture_Royale") |
| `url` | string | Full URL to the Grokipedia article page |
| `data` | object | The article content and metadata (see ArticleData schema below) |
| `scraped_at` | string | ISO 8601 timestamp of when the article was scraped (e.g., "2025-10-29T19:33:13.650809") |

## ArticleData Schema

The `data` object contains the structured article content:

```json
{
"title": string,
"url": string,
"main_title": string,
"sections": Section[],
"paragraphs": string[],
"tables": Table[],
"references": Reference[],
"metadata": Metadata
}
```

### ArticleData Fields

| Field | Type | Description |
|-------|------|-------------|
| `title` | string | URL-encoded article title (same as root `title`) |
| `url` | string | Full URL to the article (same as root `url`) |
| `main_title` | string | Human-readable article title (e.g., "Yamaha Venture Royale") |
| `sections` | array | Array of section objects (see Section schema) |
| `paragraphs` | array | Array of plain text paragraph strings extracted from the article |
| `tables` | array | Array of table objects (see Table schema) |
| `references` | array | Array of reference objects (see Reference schema) |
| `metadata` | object | Article metadata (see Metadata schema) |

## Section Schema

Sections represent hierarchical content structure:

```json
{
"level": string,
"id": string,
"title": string,
"content": ContentBlock[]
}
```

### Section Fields

| Field | Type | Description |
|-------|------|-------------|
| `level` | string | Heading level (e.g., "h1", "h2", "h3") |
| `id` | string | URL-friendly section identifier (e.g., "yamaha-venture-royale") |
| `title` | string | Section heading text |
| `content` | array | Array of content blocks (see ContentBlock schema) |

## ContentBlock Schema

Content blocks within sections:

```json
{
"type": string,
"text": string
}
```

### ContentBlock Fields

| Field | Type | Description |
|-------|------|-------------|
| `type` | string | Content type (e.g., "paragraph", "p") |
| `text` | string | The text content of the block |

## Table Schema

Tables extracted from articles:

```json
{
"headers": string[],
"rows": string[][]
}
```

### Table Fields

| Field | Type | Description |
|-------|------|-------------|
| `headers` | array | Array of column header strings |
| `rows` | array | Array of row arrays, where each row is an array of cell strings |

## Reference Schema

Citations and references:

```json
{
"text": string,
"link": LinkObject | null
}
```

### Reference Fields

| Field | Type | Description |
|-------|------|-------------|
| `text` | string | Reference text/citation |
| `link` | object\|null | Link object (see LinkObject schema) or null if no link |

### LinkObject Schema

```json
{
"href": string,
"text": string
}
```

| Field | Type | Description |
|-------|------|-------------|
| `href` | string | URL of the reference link |
| `text` | string | Display text for the link |

## Metadata Schema

Article metadata:

```json
{
"has_edits": boolean,
"fact_check_timestamp": string | null
}
```

### Metadata Fields

| Field | Type | Description |
|-------|------|-------------|
| `has_edits` | boolean | Whether the article has been edited |
| `fact_check_timestamp` | string\|null | ISO 8601 timestamp of fact-checking, or null if not fact-checked |

## Example Record

```json
{
"title": "Yamaha_Venture_Royale",
"url": "https://grokipedia.com/page/Yamaha_Venture_Royale",
"data": {
"title": "Yamaha_Venture_Royale",
"url": "https://grokipedia.com/page/Yamaha_Venture_Royale",
"main_title": "Yamaha Venture Royale",
"sections": [
{
"level": "h1",
"id": "yamaha-venture-royale",
"title": "Yamaha Venture Royale",
"content": [
{
"type": "paragraph",
"text": "TheYamaha Venture Royaleis a large touring motorcycle..."
}
]
}
],
"paragraphs": [
"TheYamaha Venture Royaleis a large touring motorcycle...",
"The 1,198 cc (73.1 cu in)V4 enginefrom the Mk1 Venture..."
],
"tables": [
{
"headers": ["Year", "Colors"],
"rows": [
["2004", "Dull Red Metallic, Stardust Silver..."],
["2005", "Dark Purplish Red Cocktail, Black..."]
]
}
],
"references": [
{
"text": "https://www.venturerider.org/history/",
"link": {
"href": "https://www.venturerider.org/history/",
"text": "https://www.venturerider.org/history/"
}
},
{
"text": "Yamaha XVZ12L-N series Service Manual...",
"link": null
}
],
"metadata": {
"has_edits": false,
"fact_check_timestamp": null
}
},
"scraped_at": "2025-10-29T19:33:13.650809"
}
```

## Notes

- All string fields are UTF-8 encoded
- Arrays may be empty (`[]`)
- Some fields may be `null` (notably `link` in references and `fact_check_timestamp` in metadata)
- The `paragraphs` array contains plain text versions of paragraphs, which may overlap with content in `sections`
- Section IDs are typically URL-friendly (lowercase, hyphens instead of spaces)

## License

The [X AI Terms of Service](https://archive.ph/58VTS) dictates the licensing of this content.

Some of the content on Grokipedia is directly adapted from English Wikipedia and contains a reference to the Certain Grokipedia content may also be subject to [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/deed.en).

Other pages are subject to the [xAI Community License Agreement](https://huggingface.co/xai-org/grok-2/blob/main/LICENSE). For those pages, all rights reserved. Copyright © xAI 2025.

As dictated by the xAI Community License Agreement, I must state that this dataset is "Powered by xAI".

许可证：其他
许可证名称：combo-grok-2-cc
许可证链接：LICENSE
语言：
- 英语
规范名称：Grokipedia v0.1 爬取数据集

# Grokipedia v0.1 爬取数据集

本数据集为2025年10月末至11月初时点下的Grokipedia v0.1结构化近乎完整的快照爬取结果。

此外，本数据集还包含Grokipedia语料库中250个Token（Token）的半重叠分块的嵌入向量。

本数据集由Harold Triedman与Alexios Mantzarlis于2025年11月发表的论文《What did Elon Change? A comprehensive analysis of Grokipedia》（[arxiv链接](https://arxiv.org/abs/2511.09685)）首次收集并使用。

若使用本数据集，请按照以下格式引用：
bibtex
@misc{TriedmanManzarlisGrokipedia2025,
title = {What did Elon change? A comprehensive analysis of Grokipedia},
shorttitle = {What did Elon change?},
year = 2025,
month = nov,
number = {arXiv:2511.09685},
eprint = {2511.09685},
primaryclass = {cs},
publisher = {arXiv},
doi = {10.48550/arXiv:2511.09685},
}

## 文件格式
- **格式**：换行分隔JSON（NDJSON）
- **编码**：UTF-8
- **单行单JSON对象**：每行仅包含一个JSON对象

## 根对象Schema
文件中的每一行均为符合以下结构的JSON对象：
json
{
"title": string,
"url": string,
"data": ArticleData,
"scraped_at": string (ISO 8601 datetime)
}

### 根层级字段
| 字段 | 类型 | 描述 |
|-------|------|-------------|
| `title` | 字符串 | 经过URL编码的文章标题（例如："Yamaha_Venture_Royale"） |
| `url` | 字符串 | Grokipedia文章页面的完整URL |
| `data` | 对象 | 文章内容与元数据（详见下文的ArticleData Schema） |
| `scraped_at` | 字符串 | 文章爬取时的ISO 8601时间戳（例如："2025-10-29T19:33:13.650809"） |

## 文章数据（ArticleData）Schema
`data`对象包含结构化的文章内容：
json
{
"title": string,
"url": string,
"main_title": string,
"sections": Section[],
"paragraphs": string[],
"tables": Table[],
"references": Reference[],
"metadata": Metadata
}

### 文章数据字段
| 字段 | 类型 | 描述 |
|-------|------|-------------|
| `title` | 字符串 | 经过URL编码的文章标题（与根层级的`title`字段一致） |
| `url` | 字符串 | 文章的完整URL（与根层级的`url`字段一致） |
| `main_title` | 字符串 | 面向人类读者的文章标题（例如："Yamaha Venture Royale"） |
| `sections` | 数组 | 章节对象数组（详见章节Schema） |
| `paragraphs` | 数组 | 从文章中提取的纯文本段落字符串数组 |
| `tables` | 数组 | 表格对象数组（详见表格Schema） |
| `references` | 数组 | 引用对象数组（详见引用Schema） |
| `metadata` | 对象 | 文章元数据（详见元数据Schema） |

## 章节（Section）Schema
章节用于表示层级化的内容结构：
json
{
"level": string,
"id": string,
"title": string,
"content": ContentBlock[]
}

### 章节字段
| 字段 | 类型 | 描述 |
|-------|------|-------------|
| `level` | 字符串 | 标题层级（例如："h1"、"h2"、"h3"） |
| `id` | 字符串 | 符合URL规范的章节标识符（例如："yamaha-venture-royale"） |
| `title` | 字符串 | 章节标题文本 |
| `content` | 数组 | 内容块数组（详见内容块Schema） |

## 内容块（ContentBlock）Schema
章节内的内容块：
json
{
"type": string,
"text": string
}

### 内容块字段
| 字段 | 类型 | 描述 |
|-------|------|-------------|
| `type` | 字符串 | 内容类型（例如："paragraph"、"p"） |
| `text` | 字符串 | 内容块的文本内容 |

## 表格（Table）Schema
从文章中提取的表格：
json
{
"headers": string[],
"rows": string[][]
}

### 表格字段
| 字段 | 类型 | 描述 |
|-------|------|-------------|
| `headers` | 数组 | 列标题字符串数组 |
| `rows` | 数组 | 行数组，每个行为单元格字符串数组 |

## 引用（Reference）Schema
引用与参考文献：
json
{
"text": string,
"link": LinkObject | null
}

### 引用字段
| 字段 | 类型 | 描述 |
|-------|------|-------------|
| `text` | 字符串 | 引用文本/参考文献条目 |
| `link` | 对象|null | 链接对象（详见链接对象Schema），若无链接则为null |

### 链接对象（LinkObject）Schema
json
{
"href": string,
"text": string
}

| 字段 | 类型 | 描述 |
|-------|------|-------------|
| `href` | 字符串 | 引用链接的URL |
| `text` | 字符串 | 链接的显示文本 |

## 元数据（Metadata）Schema
文章元数据：
json
{
"has_edits": boolean,
"fact_check_timestamp": string | null
}

### 元数据字段
| 字段 | 类型 | 描述 |
|-------|------|-------------|
| `has_edits` | 布尔值 | 文章是否已被编辑 |
| `fact_check_timestamp` | 字符串|null | 事实核查的ISO 8601时间戳，若未进行事实核查则为null |

## 示例记录
json
{
"title": "Yamaha_Venture_Royale",
"url": "https://grokipedia.com/page/Yamaha_Venture_Royale",
"data": {
"title": "Yamaha_Venture_Royale",
"url": "https://grokipedia.com/page/Yamaha_Venture_Royale",
"main_title": "Yamaha Venture Royale",
"sections": [
{
"level": "h1",
"id": "yamaha-venture-royale",
"title": "Yamaha Venture Royale",
"content": [
{
"type": "paragraph",
"text": "TheYamaha Venture Royaleis a large touring motorcycle..."
}
]
}
],
"paragraphs": [
"TheYamaha Venture Royaleis a large touring motorcycle...",
"The 1,198 cc (73.1 cu in)V4 enginefrom the Mk1 Venture..."
],
"tables": [
{
"headers": ["Year", "Colors"],
"rows": [
["2004", "Dull Red Metallic, Stardust Silver..."],
["2005", "Dark Purplish Red Cocktail, Black..."]
]
}
],
"references": [
{
"text": "https://www.venturerider.org/history/",
"link": {
"href": "https://www.venturerider.org/history/",
"text": "https://www.venturerider.org/history/"
}
},
{
"text": "Yamaha XVZ12L-N series Service Manual...",
"link": null
}
],
"metadata": {
"has_edits": false,
"fact_check_timestamp": null
}
},
"scraped_at": "2025-10-29T19:33:13.650809"
}

## 注意事项
- 所有字符串字段均采用UTF-8编码
- 数组可为空（`[]`）
- 部分字段可为`null`（尤以引用中的`link`字段与元数据中的`fact_check_timestamp`字段为甚）
- `paragraphs`数组存储段落的纯文本版本，其内容可能与`sections`中的内容存在重叠
- 章节ID通常为URL友好格式（小写字母，以连字符替代空格）

## 许可证
本内容的授权规则受[X AI服务条款](https://archive.ph/58VTS)约束。

Grokipedia上的部分内容直接改编自英文维基百科，且部分Grokipedia内容可能同时受[知识共享署名-相同方式共享4.0国际许可协议（CC BY-SA 4.0）](https://creativecommons.org/licenses/by-sa/4.0/deed.en)约束。

其余页面受[xAI社区许可协议](https://huggingface.co/xai-org/grok-2/blob/main/LICENSE)约束，此类页面保留所有权利。版权所有 © xAI 2025。

根据xAI社区许可协议的要求，特此声明：本数据集“由xAI提供技术支持”。