universeTBD/arxiv-bit-flip-cs.LG

--- dataset_info: features: - name: bit dtype: string - name: flip dtype: string - name: title dtype: string - name: categories dtype: string - name: abstract dtype: string - name: authors dtype: string - name: doi dtype: string - name: id dtype: string splits: - name: train num_bytes: 229044314 num_examples: 100039 download_size: 127335112 dataset_size: 229044314 configs: - config_name: default data_files: - split: train path: data/train-* --- # Dataset Card for "arxiv-bit-flip-cs.LG" This dataset contains "Bit-Flips," structured representations extracted from the abstracts of ArXiv papers, specifically in the category of `cs.LG` (Machine Learning). These Bit-Flips aim to encapsulate the essence of the research by identifying the conventional belief or 'status quo' the abstract challenges (Bit) and the counterargument or innovative approach that flips the Bit (Flip). ## Bit-Flip Concept A Bit-Flip serves as a two-part schema: * _Bit_: It identifies the conventional belief or assumption that the research implicitly or explicitly challenges. It is composed of three sentences that are logically connected. * _Flip_: It formulates the counterargument or innovative approach that flips the conventional belief or Bit. It also consists of three logically connected sentences. ## Data Collection The dataset focuses on the ArXiv category of `cs.LG` (Machine Learning). The dataset was created to understand the paradigm shifts or challenges to conventional wisdom that are presented in new research, encapsulated through the Bit-Flip schema. ## Methodology The data was processed using a Python script that performs the following steps: 1. The script generates a custom prompt based on each abstract, using a predefined template that explains the Bit-Flip concept. 2. An Azure model is used to generate a response to the custom prompt. 3. The response is parsed to extract a JSON-like structure containing the Bit and the Flip. 4. Each Bit and Flip is saved along with the title of the paper. 5. The script uses multithreading to speed up the data processing and can handle a batch of abstracts in each run. The processed data is saved in a CSV file.

原始信息汇总

数据集卡片 "arxiv-bit-flip-cs.LG"

数据集概述

该数据集包含从ArXiv论文摘要中提取的“比特翻转”结构化表示，特别关注cs.LG（机器学习）类别。这些比特翻转旨在通过识别摘要挑战的传统信念或“现状”（比特）以及翻转比特的反驳或创新方法（翻转）来概括研究的核心。

比特翻转概念

比特翻转是一个两部分的模式：

• 比特：它识别研究隐含或明确挑战的传统信念或假设。它由三个逻辑上相连的句子组成。
• 翻转：它构建了翻转传统信念或比特的反驳或创新方法。它也由三个逻辑上相连的句子组成。

数据收集

该数据集专注于ArXiv的cs.LG（机器学习）类别。该数据集旨在理解新研究中提出的范式转变或对传统智慧的挑战，通过比特翻转模式进行概括。

数据处理方法

数据使用Python脚本进行处理，步骤如下：

1. 脚本根据每个摘要生成一个基于预定义模板的自定义提示，解释比特翻转概念。
2. 使用Azure模型生成对自定义提示的响应。
3. 解析响应以提取包含比特和翻转的JSON样结构。
4. 每个比特和翻转与论文标题一起保存。
5. 脚本使用多线程加速数据处理，并可以一次处理一批摘要。处理后的数据保存在CSV文件中。

数据集信息

• 特征：
  • bit: 字符串
  • flip: 字符串
  • title: 字符串
  • categories: 字符串
  • abstract: 字符串
  • authors: 字符串
  • doi: 字符串
  • id: 字符串
• 分割：
  • train: 229044314字节，100039个样本
• 下载大小：127335112字节
• 数据集大小：229044314字节
• 配置：
  • default：
    • train：data/train-*