llm-book/JGLUE

JGLUE是一个用于评估日本语言理解能力的基准数据集。该数据集是从零开始构建的，没有经过翻译，旨在测量日本语言的一般理解能力。JGLUE数据集涵盖了多种任务类别，包括多项选择、问答、句子相似度和文本分类等。

原始信息汇总

数据集概述

基本信息

• 数据集名称: JGLUE
• 语言: 日语
• 创建者: 众包和发现
• 许可证: Creative Commons Attribution 4.0 International License
• 多语言性: 单语
• 源数据集: 原始数据集
• 标签: MARC, STS, NLI, SQuAD, CommonsenseQA

任务类别

• 多选题
• 问答
• 句子相似性
• 文本分类

任务ID

• 多选题问答
• 开放领域问答
• 多类别分类
• 情感分类

引用

bibtex @inproceedings{kurihara-etal-2022-jglue, title = "{JGLUE}: {J}apanese General Language Understanding Evaluation", author = "Kurihara, Kentaro and Kawahara, Daisuke and Shibata, Tomohide", booktitle = "Proceedings of the Thirteenth Language Resources and Evaluation Conference", month = jun, year = "2022", address = "Marseille, France", publisher = "European Language Resources Association", url = "https://aclanthology.org/2022.lrec-1.317", pages = "2957--2966", abstract = "To develop high-performance natural language understanding (NLU) models, it is necessary to have a benchmark to evaluate and analyze NLU ability from various perspectives. While the English NLU benchmark, GLUE, has been the forerunner, benchmarks are now being released for languages other than English, such as CLUE for Chinese and FLUE for French; but there is no such benchmark for Japanese. We build a Japanese NLU benchmark, JGLUE, from scratch without translation to measure the general NLU ability in Japanese. We hope that JGLUE will facilitate NLU research in Japanese.", }

bibtex @InProceedings{Kurihara_nlp2022, author = "栗原健太郎 and 河原大輔 and 柴田知秀", title = "JGLUE: 日本語言語理解ベンチマーク", booktitle = "言語処理学会第 28 回年次大会", year = "2022", url = "https://www.anlp.jp/proceedings/annual_meeting/2022/pdf_dir/E8-4.pdf", note= "in Japanese" }

贡献者

• 数据集创建者: Kentaro Kurihara, Daisuke Kawahara, Tomohide Shibata
• 代码贡献者: Shunsuke Kitada

搜集汇总

背景与挑战

背景概述

JGLUE是一个日语自然语言理解基准数据集，支持多项选择、问答和句子相似性等多种任务，适用于日语语言模型的研究和评估。