yguooo/summarize_from_feedback_tldr_3_filtered_oai_preprocessing_llama3_scene1

TL;DR SFT数据集是用于OpenAI的Summarize from Feedback任务的数据集，来源于Reddit TL;DR数据集。数据集包含多个特征列，如id、subreddit、title、post、summary等，这些列直接从原始数据集中提取。此外，还通过预处理脚本添加了一些新的列，如query、query_token、reference_response_token等，这些列用于处理文本的token化和长度限制。数据集分为train、validation和test三个部分，每个部分的大小和样本数量也在README中详细列出。

The dataset is extracted from Reddits TL;DR dataset, used for OpenAIs feedback summarization task. It includes multiple fields such as id (unique identifier for the post), subreddit (subreddit the post was taken from), title (title of the post), post (body of the post), summary (summary of the post), reference_response (reference response for the post), etc. Additionally, fields like query (length-limited summarization query) and query_token (tokenized version of query) are added through preprocessing scripts. The goal of the dataset is to train models to generate concise summaries of posts.