augustoperes/mtg_text

---
task_categories:
- text-generation
language:
- en
size_categories:
- 10K<n<100K
---

# Magic the gathering dataset

This dataset contains text of all magic the gathering cards.
Example usage:

```python
from datasets import load_dataset

dataset = load_dataset('augustoperes/mtg_text')
dataset

# outputs:
# DatasetDict({
# train: Dataset({
# features: ['card_name', 'type_line', 'oracle_text'],
# num_rows: 20063
# })
# validation: Dataset({
# features: ['card_name', 'type_line', 'oracle_text'],
# num_rows: 5016
# })
# })
```

Elements of the dataset are, for example:

```python
train_dataset = dataset['train']
train_dataset[0]

# Outputs
# {'card_name': 'Recurring Insight',
# 'type_line': 'Sorcery',
# 'oracle_text': "Draw cards equal to the number of cards in target opponent's hand.\nRebound (If you cast this spell from your hand, exile it as it resolves. At the beginning of your next upkeep, you may cast this card from exile without paying its mana cost.)"}
```

# Example usage with Pytorch

You can easily tokenize, convert and pad this dataset to be usable in pytorch with:

```python
from transformers import AutoTokenizer

import torch
from torch.nn.utils.rnn import pad_sequence

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")

def tokenize(sample):
sample["card_name"] = tokenizer(sample["card_name"])["input_ids"]
sample["type_line"] = tokenizer(sample["type_line"])["input_ids"]
sample["oracle_text"] = tokenizer(sample["oracle_text"])["input_ids"]
return sample

tokenized_dataset = train_dataset.map(tokenize)

def collate_fn(sequences):
# Pad the sequences to the maximum length in the batch
card_names = [torch.tensor(sequence['card_name']) for sequence in sequences]
type_line = [torch.tensor(sequence['type_line']) for sequence in sequences]
oracle_text = [torch.tensor(sequence['oracle_text']) for sequence in sequences]

padded_card_name = pad_sequence(card_names, batch_first=True, padding_value=0)
padded_type_line = pad_sequence(type_line, batch_first=True, padding_value=0)
padded_oracle_text = pad_sequence(oracle_text, batch_first=True, padding_value=0)

return {'card_name': padded_card_name, 'type_line': padded_type_line, 'padded_oracle_text': padded_oracle_text}

loader = torch.utils.data.DataLoader(tokenized_dataset, collate_fn=collate_fn, batch_size=4)

for e in loader:
print(e)
break

# Will output:
# {'card_name': tensor([[ 101, 10694, 12369, 102, 0],
# [ 101, 3704, 9881, 102, 0],
# [ 101, 22639, 20066, 7347, 102],
# [ 101, 25697, 1997, 6019, 102]]),
# 'type_line': tensor([[ 101, 2061, 19170, 2854, 102, 0, 0],
# [ 101, 6492, 1517, 4743, 102, 0, 0],
# [ 101, 6492, 1517, 22639, 102, 0, 0],
# [ 101, 4372, 14856, 21181, 1517, 15240, 102]]),
# 'padded_oracle_text': [ommited for readability])}

```

原始信息汇总

Magic the Gathering 数据集

概述

该数据集包含所有 Magic the Gathering 卡牌的文本。

数据集信息

• 任务类别: 文本生成
• 语言: 英语
• 数据量: 10K<n<100K

数据集结构

数据集分为训练集和验证集：

• 训练集: 包含 20063 条数据
• 验证集: 包含 5016 条数据

数据特征

每条数据包含以下特征：

• card_name: 卡牌名称
• type_line: 卡牌类型
• oracle_text: 卡牌文本

示例数据

python {card_name: Recurring Insight, type_line: Sorcery, oracle_text: "Draw cards equal to the number of cards in target opponents hand. Rebound (If you cast this spell from your hand, exile it as it resolves. At the beginning of your next upkeep, you may cast this card from exile without paying its mana cost.)"}