MCG-NJU/LongVPO-Training-Data

--- license: mit language: - en base_model: - OpenGVLab/InternVL3-8B pipeline_tag: video-text-to-text library_name: transformers tags: - multimodal task_categories: - video-text-to-text - visual-question-answering size_categories: - 10K<n<100K configs: - config_name: stage1 data_files: - split: train path: InternVL3_stage1_short2long_training.jsonl default: true - config_name: stage2 data_files: - split: train path: InternVL3_stage2_long_training.jsonl --- # LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization [\[📂 GitHub\]](https://github.com/MCG-NJU/LongVPO) [\[📜 Paper\]](https://arxiv.org/abs/2602.02341) [\[🤗 Model\]](https://huggingface.co/MCG-NJU/LongVPO-Stage2-InternVL3-8B) ## ⚙️ Training Methodology & Data The training process of LongVPO is divided into two progressive stages, utilizing curated datasets to enhance both grounded understanding and complex reasoning: * **Stage 1: Anchored Cues Optimization** * **Objective:** To anchor the model's attention to critical temporal events and prevent attention drift over long contexts. * **Data & Method:** Utilizes short-to-long video alignment data sourced from [LLaVA-Video-178K](https://huggingface.co/datasets/lmms-lab/LLaVA-Video-178K). The preference optimization leverages anchored temporal cues (e.g., specific timestamps or keyframes) to teach the model how to locate and extract relevant information accurately before generating an answer. * **Stage 2: Self-Reasoning Optimization** * **Objective:** To internalize the reasoning process, allowing the model to autonomously connect multiple events across the video without relying on explicit external cues. * **Data & Method:** Focuses purely on long-form video datasets, utilizing [Vript](https://huggingface.co/datasets/Mutonix/Vript). The model is trained to generate its own reasoning chains (self-reasoning) to deduce the correct answers, aligning its outputs with human preference for logical and comprehensive long-video comprehension. ## 📜 Citation If you find this work helpful, please consider citing our paper: ```bibtex @inproceedings{huang2025longvpo, title={Long{VPO}: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization}, author={Zhenpeng Huang and Jiaqi Li and Zihan Jia and Xinhao Li and Desen Meng and Lingxue Song and Xi Chen and Liang Li and Limin Wang}, booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems}, year={2025}, url={[https://openreview.net/forum?id=LKAp7Dknxf](https://openreview.net/forum?id=LKAp7Dknxf)} }许可证：MIT许可证 语言：英语 基础模型：OpenGVLab/InternVL3-8B 任务流水线标签：视频-文本转文本（video-text-to-text） 依赖库：Transformers库 标签：多模态（multimodal） 任务类别：视频-文本转文本（video-text-to-text）、视觉问答（visual-question-answering） 数据规模类别：10000 < 样本数 < 100000 配置项： - 配置名称：stage1 数据文件： - 拆分方式：训练集 - 文件路径：InternVL3_stage1_short2long_training.jsonl 为默认配置 - 配置名称：stage2 数据文件： - 拆分方式：训练集 - 文件路径：InternVL3_stage2_long_training.jsonl # LongVPO：面向长视频偏好优化的从锚定线索到自主推理 [📂 GitHub仓库](https://github.com/MCG-NJU/LongVPO) [📜 论文](https://arxiv.org/abs/2602.02341) [🤗 模型](https://huggingface.co/MCG-NJU/LongVPO-Stage2-InternVL3-8B) ## ⚙️ 训练方法与数据 LongVPO的训练过程分为两个递进阶段，通过精选数据集提升模型的锚定理解能力与复杂推理能力： * **阶段1：锚定线索优化** * **目标**：将模型的注意力锚定至关键时序事件，避免长上下文下的注意力漂移。 * **数据与方法**：采用源自[LLaVA-Video-178K](https://huggingface.co/datasets/lmms-lab/LLaVA-Video-178K)的长短视频对齐数据。偏好优化环节借助锚定时序线索（如特定时间戳或关键帧），指导模型在生成答案前精准定位并提取相关信息。 * **阶段2：自主推理优化** * **目标**：内化推理流程，使模型能够无需依赖显式外部线索，自主关联视频中的多个事件。 * **数据与方法**：仅面向长视频数据集，采用[Vript](https://huggingface.co/datasets/Mutonix/Vript)开展训练。模型被训练为生成自身的推理链（自主推理，self-reasoning）以推导正确答案，使其输出符合人类对长视频逻辑化、全面化理解的偏好。 ## 📜 引用 若您认为本工作对您有所帮助，请引用我们的论文： bibtex @inproceedings{huang2025longvpo, title={Long{VPO}: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization}, author={Zhenpeng Huang and Jiaqi Li and Zihan Jia and Xinhao Li and Desen Meng and Lingxue Song and Xi Chen and Liang Li and Limin Wang}, booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems}, year={2025}, url={[https://openreview.net/forum?id=LKAp7Dknxf](https://openreview.net/forum?id=LKAp7Dknxf)} }