alfredplpl/wikipedia-simple-ja-100k

---
dataset_info:
features:
- name: id
dtype: string
- name: url
dtype: string
- name: title
dtype: string
- name: text
dtype: string
splits:
- name: train
num_bytes: 26728437
num_examples: 106876
download_size: 0
dataset_size: 26728437
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
license: cc-by-sa-3.0
task_categories:
- summarization
language:
- ja
---
# Dataset Card for "wikipedia-simple-ja-100k"

# Original Dataset
- hpprc/wikipedia-20240101

# Procedure
- Exract the first line of the title from the dataset.
- Generate the answer by summizing the line using LLM:
- Input RAG-like prompt to CALM 2 7B Chat.
- Format the response.

# RAG-like Prompt
```python
f"""USER: {title}とはなんですか？次の文章を参考に一言でまとめてください。{text}
ASSISTANT: """
```

原始信息汇总

数据集概述

数据集信息

• 特征:
  • id: 字符串类型
  • url: 字符串类型
  • title: 字符串类型
  • text: 字符串类型
• 分割:
  • train:
    • 字节数: 26728437
    • 样本数: 106876
• 下载大小: 0
• 数据集大小: 26728437

配置

• 配置名称: default
  • 数据文件:
    • split: train
    • path: data/train-*

许可证

• cc-by-sa-3.0

任务类别

• summarization

语言

• ja