asparius/Turkish-STSBenchmark

Turkish STS Benchmark数据集是英文STSb数据集的土耳其语翻译版本，用于评估土耳其语的语义相似性模型。该数据集最初来源于GitHub上的一个项目，并且用于抽象新闻摘要的评估。

原始信息汇总

Turkish STS Benchmark 数据集概述

数据集来源

• 原始数据来自 https://github.com/verimsu/STSb-TR/tree/main

引用信息

• 若使用此数据集，请引用以下论文：

  @inproceedings{beken-fikri-etal-2021-semantic, title = "Semantic Similarity Based Evaluation for Abstractive News Summarization", author = "Beken Fikri, Figen and Oflazer, Kemal and Yanikoglu, Berrin", booktitle = "Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)", month = aug, year = "2021", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2021.gem-1.3", doi = "10.18653/v1/2021.gem-1.3", pages = "24--33", abstract = "ROUGE is a widely used evaluation metric in text summarization. However, it is not suitable for the evaluation of abstractive summarization systems as it relies on lexical overlap between the gold standard and the generated summaries. This limitation becomes more apparent for agglutinative languages with very large vocabularies and high type/token ratios. In this paper, we present semantic similarity models for Turkish and apply them as evaluation metrics for an abstractive summarization task. To achieve this, we translated the English STSb dataset into Turkish and presented the first semantic textual similarity dataset for Turkish as well. We showed that our best similarity models have better alignment with average human judgments compared to ROUGE in both Pearson and Spearman correlations.", }

许可证

• 数据集遵循 CC-BY-SA-4.0 许可证。