Dataset For Generating Tl;Dr

This is the dataset for the TL;DR challenge containing posts from the Reddit corpus, suitable for abstractive summarization using deep learning. The format is a json file where each line is a JSON object representing a post. The schema of each post is shown below: author: string (nullable = true) body: string (nullable = true) normalizedBody: string (nullable = true) content: string (nullable = true) content_len: long (nullable = true) summary: string (nullable = true) summary_len: long (nullable = true) id: string (nullable = true) subreddit: string (nullable = true) subreddit_id: string (nullable = true) title: string (nullable = true) Specifically, the <strong>content</strong> and <strong>summary</strong> fields can be directly used as inputs to a deep learning model (e.g. Sequence to Sequence model ). The dataset consists of 3,084,410 posts with an average length of 211 words for content, and 25 words for the summary. <strong>Note : </strong>As this is the complete dataset for the challenge, it is up to the participants to split it into training and validation sets accordingly.本数据集为TL;DR挑战赛专用数据集，收录了Reddit语料库中的帖子，适用于基于深度学习的抽象式摘要生成任务。数据集采用JSON文件格式，每行均为一个代表单条帖子的JSON对象。各帖子的字段规范如下： - author：字符串类型，可空 - body：字符串类型，可空 - normalizedBody：字符串类型，可空 - content：字符串类型，可空 - content_len：长整型，可空 - summary：字符串类型，可空 - summary_len：长整型，可空 - id：字符串类型，可空 - subreddit：字符串类型，可空 - subreddit_id：字符串类型，可空 - title：字符串类型，可空 具体而言，content与summary字段可直接作为深度学习模型（例如序列到序列（Sequence to Sequence）模型）的输入。本数据集共计包含3084410条帖子，其中待摘要文本的平均长度为211词，摘要的平均长度为25词。 注意：本数据集为挑战赛官方完整数据集，参赛者需自行将其划分为训练集与验证集。