theblackcat102/anime-understanding-dataset

---
license: mit
configs:
- config_name: chainsawman
data_files:
- split: train
path: "chainsawman_dev.jsonl"
- split: validation
path: "chainsawman_val.jsonl"
- split: test
path: "chainsawman_test.jsonl"
- config_name: kurokonobasuke
data_files:
- split: train
path: "kurokonobasuke_dev.jsonl"
- split: validation
path: "kurokonobasuke_val.jsonl"
- split: test
path: "kurokonobasuke_test.jsonl"
- config_name: onepunch
data_files:
- split: train
path: "onepunch_dev.jsonl"
- split: validation
path: "onepunch_val.jsonl"
- split: test
path: "onepunch_test.jsonl"
- config_name: hellsing
data_files:
- split: train
path: "hellsing_dev.jsonl"
- split: validation
path: "hellsing_val.jsonl"
- split: test
path: "hellsing_test.jsonl"
- config_name: frieren
data_files:
- split: train
path: "frieren_dev.jsonl"
- split: validation
path: "frieren_val.jsonl"
- split: test
path: "frieren_test.jsonl"
- config_name: aot
data_files:
- split: train
path: "aot_dev.jsonl"
- split: validation
path: "aot_val.jsonl"
- split: test
path: "aot_test.jsonl"
- config_name: naruto
data_files:
- split: train
path: "naruto_dev.jsonl"
- split: validation
path: "naruto_val.jsonl"
- split: test
path: "naruto_test.jsonl"
- config_name: dr_stone
data_files:
- split: train
path: "dr_stone_dev.jsonl"
- split: validation
path: "dr_stone_val.jsonl"
- split: test
path: "dr_stone_test.jsonl"
- config_name: gundam_00
data_files:
- split: train
path: "gundam_00_dev.jsonl"
- split: validation
path: "gundam_00_val.jsonl"
- split: test
path: "gundam_00_test.jsonl"
- config_name: darling-in-the-franxx
data_files:
- split: train
path: "darling-in-the-franxx_dev.jsonl"
- split: validation
path: "darling-in-the-franxx_val.jsonl"
- split: test
path: "darling-in-the-franxx_test.jsonl"
- config_name: berserk
data_files:
- split: train
path: "berserk_dev.jsonl"
- split: validation
path: "berserk_val.jsonl"
- split: test
path: "berserk_test.jsonl"
- config_name: evangelion
data_files:
- split: train
path: "evangelion_dev.jsonl"
- split: validation
path: "evangelion_val.jsonl"
- split: test
path: "evangelion_test.jsonl"
- config_name: onepiece
data_files:
- split: train
path: "onepiece_dev.jsonl"
- split: validation
path: "onepiece_val.jsonl"
- split: test
path: "onepiece_test.jsonl"
task_categories:
- question-answering
language:
- en
tags:
- question-answering
- multi-choice
pretty_name: anime understanding dataset
size_categories:
- 1K<n<10K
---
# Anime Understanding Benchmark (WIP)

Evaluate anime knowledge found in existing LLMs. We hope to provide an easy to run evaluation on knowledge understanding in anime/manga. Better understanding in anime/manga knowledge should resulted in task such as waifu role play.

Any suggestion is open in discussion tab.

# Currently in the works

[] Eval on popular models such as gpt, hermes, dolphin, llama base model

[] Add more metadata regarding of anime/manga year span

[] Suggestions for more anime choices ideally diverse in terms of year of air and topic diversity.

[] Human inspection for any potential errors

## Data sources:

[One Piece](https://onepiece.fandom.com)

[Chainsaw man](https://chainsaw-man.fandom.com)

[Tokyo Ghoul](https://tokyoghoul.fandom.com)

[Naruto](https://naruto.fandom.com)

[Berserk](https://berserk.fandom.com/)

[Darling in the FRANXX](https://darling-in-the-franxx.fandom.com/)

[Shingeki no Kyojin - Attack on Titan](https://attackontitan.fandom.com)

[Evangelion](https://evangelion.fandom.com)

[Wikipedia](https://www.wikipedia.org/)

原始信息汇总

数据集概述

目标

• 评估现有大型语言模型（LLMs）中关于动漫知识的理解。
• 旨在提供一个易于执行的评估工具，用于测试对动漫/漫画知识的理解。

预期应用

• 通过提升对动漫/漫画知识的理解，预期能改善相关任务的表现，如角色扮演（例如，“waifu”角色扮演）。