figenfikri/stsb_tr

STSb-TR数据集是英文STS基准数据集的机器翻译版本，使用了Google Cloud Translation API进行翻译。该数据集的语言为土耳其语，属于单语言数据集，规模在1K到10K之间。数据集的任务类别为文本分类，具体任务包括文本评分和语义相似性评分。数据集的创建者包括众包和机器生成。

原始信息汇总

数据集描述

数据集概述

STSb-TR 数据集是使用 Google Cloud Translation API 对英语 STS 基准数据集进行机器翻译的版本。

数据集详情

• annotations_creators:
  • crowdsourced
• language_creators:
  • machine-generated
• language:
  • tr
• multilinguality:
  • monolingual
• pretty_name: Semantic Textual Similarity in Turkish
• size_categories:
  • 1K<n<10K
• source_datasets:
  • extended|other-sts-b
• task_categories:
  • text-classification
• task_ids:
  • text-scoring
  • semantic-similarity-scoring

引用

@inproceedings{beken-fikri-etal-2021-semantic, title = "Semantic Similarity Based Evaluation for Abstractive News Summarization", author = "Beken Fikri, Figen and Oflazer, Kemal and Yanikoglu, Berrin", booktitle = "Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)", month = aug, year = "2021", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2021.gem-1.3", doi = "10.18653/v1/2021.gem-1.3", pages = "24--33", abstract = "ROUGE is a widely used evaluation metric in text summarization. However, it is not suitable for the evaluation of abstractive summarization systems as it relies on lexical overlap between the gold standard and the generated summaries. This limitation becomes more apparent for agglutinative languages with very large vocabularies and high type/token ratios. In this paper, we present semantic similarity models for Turkish and apply them as evaluation metrics for an abstractive summarization task. To achieve this, we translated the English STSb dataset into Turkish and presented the first semantic textual similarity dataset for Turkish as well. We showed that our best similarity models have better alignment with average human judgments compared to ROUGE in both Pearson and Spearman correlations.", }