xwjzds/pretrain_sts_similarity

---
dataset_info:
features:
- name: input
dtype: string
- name: output
dtype: string
splits:
- name: train
num_bytes: 8335942
num_examples: 41191
download_size: 5350395
dataset_size: 8335942
---
Dataset Card for Sentence Paraphase Collections

Dataset Description Repository: Paper: DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM https://arxiv.org/abs/2310.15296

Leaderboard: Point of Contact: Weijie Xu

Dataset Summary Sentence_Paraphase is a combination of sentences paraphase tasks from various sources such as paraphase using ChatGPT, Paraphrase Adversaries from Word Scrambling (PAWS) and STS benchmark. We filtered out pairs that are detected as non english, too short or not have high similarity score.

Category Count Paraphrase 223241

Dataset Structure Data Instances An example of data as follows: {'input': 'U.S. prosecutors have arrested more than 130 individuals and have seized more than $17 million in a continuing crackdown on Internet fraud and abuse.', 'output': 'More than 130 people have been arrested and $17 million worth of property seized in an Internet fraud sweep announced Friday by three U.S. government agencies.'}

Data Fields The data fields are as follows:

input and output are paraphrase of a sentence or paragraph.

Dataset Creation Curation Rationale [More Information Needed]

Source Data Initial Data Collection and Normalization [More Information Needed]

Who are the source language producers? [More Information Needed]

Annotations Annotation process [More Information Needed]

Who are the annotators? [More Information Needed]

Personal and Sensitive Information [More Information Needed]

Considerations for Using the Data Social Impact of Dataset [More Information Needed]

Discussion of Biases [More Information Needed]

Other Known Limitations [More Information Needed]

Additional Information Dataset Curators [More Information Needed]

Licensing Information The dataset is available under the Creative Commons NonCommercial (CC BY-NC 4.0).

Citation Information @misc{xu2023detime, title={DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM}, author={Weijie Xu and Wenxiang Hu and Fanyou Wu and Srinivasan Sengamedu}, year={2023}, eprint={2310.15296}, archivePrefix={arXiv}, primaryClass={cs.CL} }

原始信息汇总

数据集概述

数据集信息

• 特征:
  • input: 类型为字符串
  • output: 类型为字符串
• 分割:
  • train: 字节数为8335942，样本数为41191
• 下载大小: 5350395字节
• 数据集大小: 8335942字节

数据集摘要

• 名称: Sentence_Paraphase
• 来源: 结合了来自不同来源的句子改写任务，如使用ChatGPT的改写、Paraphrase Adversaries from Word Scrambling (PAWS)和STS基准测试。过滤掉了非英语、过短或相似度得分不高的配对。
• 类别计数: 改写任务有223241个样本

数据结构

• 数据实例: json { "input": "U.S. prosecutors have arrested more than 130 individuals and have seized more than $17 million in a continuing crackdown on Internet fraud and abuse.", "output": "More than 130 people have been arrested and $17 million worth of property seized in an Internet fraud sweep announced Friday by three U.S. government agencies." }

• 数据字段:

  • input 和 output 是句子或段落的改写。

数据集创建

• 创建理由: [更多信息需要]
• 初始数据收集和规范化: [更多信息需要]
• 源语言生产者: [更多信息需要]
• 标注过程: [更多信息需要]
• 标注者: [更多信息需要]
• 个人和敏感信息: [更多信息需要]

使用数据集的考虑

• 数据集的社会影响: [更多信息需要]
• 偏见的讨论: [更多信息需要]
• 其他已知限制: [更多信息需要]

其他信息

• 数据集策展人: [更多信息需要]
• 许可信息: 数据集在Creative Commons NonCommercial (CC BY-NC 4.0)下可用。
• 引用信息: plaintext @misc{xu2023detime, title={DeTiME: Diffusion-Enhanced Topic Modeling using Encoder-decoder based LLM}, author={Weijie Xu and Wenxiang Hu and Fanyou Wu and Srinivasan Sengamedu}, year={2023}, eprint={2310.15296}, archivePrefix={arXiv}, primaryClass={cs.CL} }