text-2-video-human-preferences-seedance-1-pro

<style>

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.image-container img {
height: 150px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {
font-size: 12px;
font-weight: semi-bold;
}

</style>

# Rapidata Video Generation Seedance 1 Pro Human Preference

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="300" alt="Dataset visualization">
</a>

<a href="https://huggingface.co/datasets/Rapidata/text-2-image-Rich-Human-Feedback">
</a>

In this dataset, ~60k human responses from ~20k human annotators were collected to evaluate Seedance 1 Pro video generation model on our benchmark. This dataset was collected in roughtly 30 min using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale data annotation.

Explore our latest model rankings on our [website](https://www.rapidata.ai/benchmark).

If you get value from this dataset and would like to see more in the future, please consider liking it ❤️

# Overview

In this dataset, ~60k human responses from ~20k human annotators were collected to evaluate Seedance 1 Pro video generation model on our benchmark. This dataset was collected in roughtly 30 min using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale data annotation.
The benchmark data is accessible on [huggingface](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences) directly.

# Explanation of the colums

The dataset contains paired video comparisons. Each entry includes 'video1' and 'video2' fields, which contain links to downscaled GIFs for easy viewing. The full-resolution videos can be found [here](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences-seedance-1-pro/tree/main/Videos)

The weighted_results column contains scores ranging from 0 to 1, representing aggregated user responses. Individual user responses can be found in the detailedResults column.

# Alignment

The alignment score quantifies how well an video matches its prompt. Users were asked: "Which video fits the description better?".

## Examples

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>A 2D animated scene of a kindergarten teacher guiding excited children through a colorful art project, their hands busy with paints and smiles bright, infusing the classroom with joyful creativity.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">(Score: 85.9%)</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_4_0.gif" width=500>
</div>
<div>
<h3 class="score-amount">Sora </h3>
<div class="score-percentage">(Score: 14.1%)</div>
<img src="https://assets.rapidata.ai/sora_0004_1.gif" width=500>
</div>
</div>
</div>
<div class="container">
<div class="text-center">
<q>A sleek drone navigates through a dense rainforest, its sensors scanning vibrant flora and fauna. The camera follows closely, capturing the intricate details and dynamic motion beneath the canopy.</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">(Score: 17.31%)</div>
<img src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_58_0.gif" width=500>
</div>
<div>
<h3 class="score-amount">Veo 3 </h3>
<div class="score-percentage">(Score: 82.69%)</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/veo3_0058_0.gif" width=500>
</div>
</div>
</div>
</div>

# Coherence

The coherence score measures whether the generated video is logically consistent and free from artifacts or visual glitches. Without seeing the original prompt, users were asked: "Which video has more glitches and is more likely to be AI generated?"

## Examples

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">(Glitch Rating: 10.59%)</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_35_0.gif" width="500" alt="Dataset visualization">
</div>
<div>
<h3 class="score-amount">Pika </h3>
<div class="score-percentage">(Glitch Rating: 89.41%)</div>
<img src="https://assets.rapidata.ai/pika_0035_3716632439.gif" width="500" alt="Dataset visualization">
</div>
</div>
</div>
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">(Glitch Rating: 79.05%)</div>
<img src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_98_0.gif" width="500" alt="Dataset visualization">
</div>
<div>
<h3 class="score-amount">Ray 2 </h3>
<div class="score-percentage">(Glitch Rating: 20.95%)</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/ray2_0098_1.gif" width="500" alt="Dataset visualization">
</div>
</div>
</div>
</div>

# Preference

The preference score reflects how visually appealing participants found each video, independent of the prompt. Users were asked: "Which video do you prefer aesthetically?"

## Examples

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">(Score: 90.25%)</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_3_0.gif" width="500" alt="Dataset visualization">
</div>
<div>
<h3 class="score-amount">Alpha </h3>
<div class="score-percentage">(Score: 9.75%)</div>
<img src="https://assets.rapidata.ai/alpha_0003_2649066252.gif" width="500" alt="Dataset visualization">
</div>
</div>
</div>
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">(Score: 16.14%)</div>
<img src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_66_0.gif" width="500" alt="Dataset visualization">
</div>
<div>
<h3 class="score-amount">Veo 3 </h3>
<div class="score-percentage">(Score: 83.86%)</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/veo3_0066_0.gif " width="500" alt="Dataset visualization">
</div>
</div>
</div>
</div>

</br>

# About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.

# Other Datasets

We run a benchmark of the major video generation models, the results can be found on our [website](https://www.rapidata.ai/leaderboard/video-models). We rank the models according to their coherence/plausiblity, their aligment with the given prompt and style prefernce. The underlying 2M+ annotations can be found here:

- Link to the [Rich Video Annotation dataset](https://huggingface.co/datasets/Rapidata/text-2-video-Rich-Human-Feedback)
- Link to the [Coherence dataset](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Coherence_Dataset)
- Link to the [Text-2-Image Alignment dataset](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Alignment_Dataset)
- Link to the [Preference dataset](https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3)

# Rapidata 视频生成Seedance 1 Pro 人类偏好数据集

<a href="https://www.rapidata.ai">
<img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="300" alt="数据集可视化">
</a>

<a href="https://huggingface.co/datasets/Rapidata/text-2-image-Rich-Human-Feedback">
</a>

本次数据集共收集了约2万名标注人员的6万余份人类反馈，用于在我们的基准测试中评估Seedance 1 Pro视频生成模型。本数据集仅需约30分钟即可通过[Rapidata Python API](https://docs.rapidata.ai)完成采集，该工具面向所有用户开放，是大规模数据标注的理想方案。

可前往我们的[官网](https://www.rapidata.ai/benchmark)查看最新的模型排名。

若您从本数据集获益并希望后续获取更多同类资源，不妨为其点赞❤️

# 概述

本次数据集共收集了约2万名标注人员的6万余份人类反馈，用于在我们的基准测试中评估Seedance 1 Pro视频生成模型。本数据集仅需约30分钟即可通过[Rapidata Python API](https://docs.rapidata.ai)完成采集，该工具面向所有用户开放，是大规模数据标注的理想方案。基准测试的原始数据可直接在[Hugging Face](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences)获取。

# 字段说明

本数据集包含成对的视频对比样本。每条数据均包含`video1`与`video2`字段，其中存储了用于快速预览的压缩GIF文件链接。全分辨率视频可通过[此链接](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences-seedance-1-pro/tree/main/Videos)获取。

`weighted_results`（加权结果）字段的取值范围为0至1，代表汇总后的用户反馈结果。单条用户反馈则存储在`detailedResults`（详细结果）字段中。

# 对齐度

对齐度评分用于衡量生成视频与对应提示词（prompt）的匹配程度。本次调研向用户提出的问题为：「哪段视频更贴合给定的描述？」

## 示例

<div class="vertical-container">
<div class="container">
<div class="text-center">
<q>一段2D动画场景：幼儿园教师正带领兴奋的孩子们开展色彩丰富的艺术创作活动，孩子们的双手沾满颜料，脸上洋溢着笑容，整个教室充满了欢乐的创作氛围。</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">（得分：85.9%）</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_4_0.gif" width=500>
</div>
<div>
<h3 class="score-amount">Sora </h3>
<div class="score-percentage">（得分：14.1%）</div>
<img src="https://assets.rapidata.ai/sora_0004_1.gif" width=500>
</div>
</div>
</div>
<div class="container">
<div class="text-center">
<q>一架造型流畅的无人机穿梭于茂密的热带雨林中，传感器扫描着色彩斑斓的动植物。镜头紧随其后，捕捉树冠下错综复杂的细节与动态画面。</q>
</div>
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">（得分：17.31%）</div>
<img src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_58_0.gif" width=500>
</div>
<div>
<h3 class="score-amount">Veo 3 </h3>
<div class="score-percentage">（得分：82.69%）</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/veo3_0058_0.gif" width=500>
</div>
</div>
</div>
</div>

# 连贯性

连贯性评分用于衡量生成视频的逻辑自洽性，以及是否存在视觉伪影或画面瑕疵。本次调研在不展示原始提示词的前提下向用户提出问题：「哪段视频存在更多瑕疵，更有可能是AI生成的？」

## 示例

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">（瑕疵评分：10.59%）</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_35_0.gif" width="500" alt="数据集可视化">
</div>
<div>
<h3 class="score-amount">Pika </h3>
<div class="score-percentage">（瑕疵评分：89.41%）</div>
<img src="https://assets.rapidata.ai/pika_0035_3716632439.gif" width="500" alt="数据集可视化">
</div>
</div>
</div>
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">（瑕疵评分：79.05%）</div>
<img src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_98_0.gif" width="500" alt="数据集可视化">
</div>
<div>
<h3 class="score-amount">Ray 2 </h3>
<div class="score-percentage">（瑕疵评分：20.95%）</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/ray2_0098_1.gif" width="500" alt="数据集可视化">
</div>
</div>
</div>
</div>

# 偏好度

偏好度评分用于衡量参与者对单段视频的视觉吸引力评价，与原始提示词无关。本次调研向用户提出的问题为：「从审美角度出发，你更偏好哪段视频？」

## 示例

<div class="vertical-container">
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">（得分：90.25%）</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_3_0.gif" width="500" alt="数据集可视化">
</div>
<div>
<h3 class="score-amount">Alpha </h3>
<div class="score-percentage">（得分：9.75%）</div>
<img src="https://assets.rapidata.ai/alpha_0003_2649066252.gif" width="500" alt="数据集可视化">
</div>
</div>
</div>
<div class="container">
<div class="image-container">
<div>
<h3 class="score-amount">Seedance 1 Pro </h3>
<div class="score-percentage">（得分：16.14%）</div>
<img src="https://assets.rapidata.ai/seedance-1-pro-24-7-25_66_0.gif" width="500" alt="数据集可视化">
</div>
<div>
<h3 class="score-amount">Veo 3 </h3>
<div class="score-percentage">（得分：83.86%）</div>
<img style="border: 5px solid #18c54f;" src="https://assets.rapidata.ai/veo3_0066_0.gif " width="500" alt="数据集可视化">
</div>
</div>
</div>
</div>

</br>

# 关于Rapidata

Rapidata的技术使大规模人类反馈采集比以往任何时候都更加快速、便捷。请访问[Rapidata官网](https://www.rapidata.ai/)，了解我们如何革新AI开发中的人类反馈采集流程。

# 其他数据集

我们针对主流视频生成模型开展了基准测试，测试结果可在我们的[官网](https://www.rapidata.ai/leaderboard/video-models)查看。我们将根据模型的连贯性/合理性、与提示词的对齐度以及风格偏好度进行排名。底层包含200万余份标注数据的数据集可通过以下链接获取：

- 链接至[Rich Video Annotation数据集](https://huggingface.co/datasets/Rapidata/text-2-video-Rich-Human-Feedback)
- 链接至[Coherence数据集](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Coherence_Dataset)
- 链接至[Text-2-Image Alignment数据集](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Alignment_Dataset)
- 链接至[Preference数据集](https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3)