stackexchange-duplicates

# Dataset Card for Stack Exchange Duplicates

This dataset contains the Stack Exchange Duplicates dataset in three formats that are easily used with Sentence Transformers to train embedding models. The data was originally extracted using the [Stack Exchange API](https://data.stackexchange.com/apple/query/fork/1456963) and taken from [embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data).
Each pair contains data from two Stack Exchange posts that were marked as duplicates. `title-title-pair` only has the titles, `body-body-pair` only the bodies, and `post-post-pair` has both.

## Dataset Subsets

### `title-title-pair` subset

* Columns: "title1", "title2"
* Column types: `str`, `str`
* Examples:
```python
{
'title1': 'what is the advantage of using the GPU rendering options in Android?',
'title2': 'Can anyone explain all these Developer Options?',
}
```
* Collection strategy: Reading the Stack Exchange Duplicates (title, title) dataset from [embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data).
* Deduplified: No

### `body-body-pair` subset

* Columns: "body1", "body2"
* Column types: `str`, `str`
* Examples:
```python
{
'body1': "I've been wondering about this for years. It seems like a pretty obvious question, so I'm surprised not to have found it addressed among the other Tolkien minutiae on this site. Hopefully I haven't missed it, but anyway, here goes... In Tolkien's Middle-Earth writings, Evil cannot create things, only twist and warp what already exists. Thus, Orcs are twisted Elves, Trolls are twisted Ents, etc. So then, what's the original source for Dragons? They look pretty original to me! The only template that seems even remotely possible is the Eagles, as they're both powerful fliers, but the connection seems very remote indeed. Also, as twisted copies Orcs and Trolls are markedly inferior to Elves and Ents respectively, but I'm not aware of any text describing Dragons as inferior to Eagles.",
'body2': 'All that I know of Smaug is that he (she?) came out of nowhere to attack and conquer Erebor. Where exactly did he come from? In fact, what are the origins of dragons? Did Ilúvatar create them or did they come from somewhere else?',
}
```
* Collection strategy: Reading the Stack Exchange Duplicates (body, body) dataset from [embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data).
* Deduplified: No

### `post-post-pair` subset

* Columns: "post1", "post2"
* Column types: `str`, `str`
* Examples:
```python
{
'post1': 'Hyperdrive vs Warp drive. Which is fastest? In Star Trek Warp Factor 36 is the highest FTL speed while in star wars I think the fastest canon was 0.4. So which is faster?',
'post2': "Is a warp drive faster than a hyperdrive? What's faster: a warp drive from Star Trek, or a hyperdrive from Star Wars?",
}
```
* Collection strategy: Reading the Stack Exchange Duplicates (title+body, title+body) dataset from [embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data).
* Deduplified: No

# Stack Exchange 重复帖子数据集卡片

本数据集提供三种格式的Stack Exchange重复帖子数据集，可便捷配合Sentence Transformers（句子Transformer）进行嵌入模型（embedding model）的训练。本数据集最初通过[Stack Exchange应用程序编程接口（API）](https://data.stackexchange.com/apple/query/fork/1456963)提取，源自[embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data)数据集。

每一组数据均来自两个被标记为重复内容的Stack Exchange帖子。其中`title-title-pair`仅包含帖子标题，`body-body-pair`仅包含帖子正文，`post-post-pair`则同时包含标题与正文。

## 数据集子集

### `title-title-pair` 子集

* 列名："title1"、"title2"
* 列类型：字符串型（`str`）、字符串型（`str`）
* 示例：
python
{
'title1': 'what is the advantage of using the GPU rendering options in Android?',
'title2': 'Can anyone explain all these Developer Options?',
}

* 采集策略：从[embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data)读取Stack Exchange重复帖子（标题对）数据集。
* 去重状态：否

### `body-body-pair` 子集

* 列名："body1"、"body2"
* 列类型：字符串型（`str`）、字符串型（`str`）
* 示例：
python
{
'body1': "I've been wondering about this for years. It seems like a pretty obvious question, so I'm surprised not to have found it addressed among the other Tolkien minutiae on this site. Hopefully I haven't missed it, but anyway, here goes... In Tolkien's Middle-Earth writings, Evil cannot create things, only twist and warp what already exists. Thus, Orcs are twisted Elves, Trolls are twisted Ents, etc. So then, what's the original source for Dragons? They look pretty original to me! The only template that seems even remotely possible is the Eagles, as they're both powerful fliers, but the connection seems very remote indeed. Also, as twisted copies Orcs and Trolls are markedly inferior to Elves and Ents respectively, but I'm not aware of any text describing Dragons as inferior to Eagles.",
'body2': 'All that I know of Smaug is that he (she?) came out of nowhere to attack and conquer Erebor. Where exactly did he come from? In fact, what are the origins of dragons? Did Ilúvatar create them or did they come from somewhere else?',
}

* 采集策略：从[embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data)读取Stack Exchange重复帖子（正文对）数据集。
* 去重状态：否

### `post-post-pair` 子集

* 列名："post1"、"post2"
* 列类型：字符串型（`str`）、字符串型（`str`）
* 示例：
python
{
'post1': 'Hyperdrive vs Warp drive. Which is fastest? In Star Trek Warp Factor 36 is the highest FTL speed while in star wars I think the fastest canon was 0.4. So which is faster?',
'post2': "Is a warp drive faster than a hyperdrive? What's faster: a warp drive from Star Trek, or a hyperdrive from Star Wars?",
}

* 采集策略：从[embedding-training-data](https://huggingface.co/datasets/sentence-transformers/embedding-training-data)读取Stack Exchange重复帖子（标题+正文对）数据集。
* 去重状态：否