pszemraj/flan-subsets-deduped

该数据集名为flan subsets: deduped，主要用于文本到文本生成任务。数据集包含多个配置，每个配置都有inputs和targets两个主要列，并且这些数据已经过去重处理。去重是基于inputs列进行的，并且过滤掉了内容少于5个字符的英文行以及任一列少于1个字符的行。此外，clean-text处理已应用于这两列。数据集还提供了去重命令的详细信息。

The dataset named flan subsets: deduped is primarily used for text-to-text generation tasks. It includes multiple configurations, each with inputs and targets as the main columns, and the data has been deduplicated. Deduplication is based on the inputs column, and rows with less than 5 characters in English or less than 1 character in either column have been filtered out. Additionally, clean-text processing has been applied to both columns. The dataset also provides detailed information on the deduplication command.