yguooo/summarize_from_feedback_tldr_3_filtered_oai_preprocessing_pythia_scene0_sheboygan

该数据集是用于OpenAI的Summarize from Feedback任务的TL;DR SFT数据集。数据集直接从OpenAI的GitHub仓库中获取，并包含了一些原始列和通过预处理脚本添加的列。原始列包括帖子的唯一标识符、所属的subreddit、标题、正文和摘要等。预处理脚本添加的列包括长度受限的查询、查询的token化版本、参考响应的token化版本等。数据集分为训练集、验证集和测试集，分别包含不同数量的样本和字节大小。This dataset is directly taken from Reddits TL;DR dataset for OpenAIs feedback summarization task. It includes the following columns: unique identifier for the post (id), subreddit the post was taken from (subreddit), title of the post (title), body of the post (post), summary of the post (summary), and reference response for the post (reference_response). The preprocessing script added the query (query), tokenized version of the query (query_token), tokenized version of the reference response (reference_response_token) and its length (reference_response_token_len), concatenation of the query and reference response (query_reference_response), its tokenized version (query_reference_response_token), and its length (query_reference_response_token_len).