1k-ranked-videos-coherence

.horizontal-container {
display: flex;
flex-direction: row;
justify-content: center;
gap: 60px;
}

.image-container img {
max-height: 250px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
box-sizing: content-box;
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

# 1k Ranked Videos
This dataset contains approximately one thousand videos, ranked from most preferred to least preferred based on human feedback from over 25k pairwise comparisons. The videos are rated solely on coherence as evaluated by human annotators, without considering the specific prompt used for generation. Each video is associated with the model name that generated it.
The videos are sampled from our benchmark dataset [text-2-video-human-preferences-pika2.2](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences-pika2.2). Follow us to explore more benchmark datasets and stay updated on human responses to the latest text-to-image (T2I) and text-to-video (T2V) generation models.

## Examples
**Some of the most preferred videos**

Below are examples of the most coherent videos, as ranked by annotators. Note that the evaluation question was:
**"Which video has more glitches and is more likely to be AI-generated?"**
Therefore, a higher Elo score indicates worse performance. Notably, most of the top-performing videos come from the veo2 and ray2 models—aligning well with our [benchmarking observations](https://www.rapidata.ai/leaderboard/video-models).

**Some of the least preferred videos**

These videos received the lowest preference scores. They often feature unsettling, distorted, or dull visuals that negatively affect viewer perception.
Most of these clips originate from models such as hunyuan and pika2.2, which is consistent with their lower rankings on our [benchmark leaderboard](https://www.rapidata.ai/leaderboard/video-models).

Rank your own videos through [our API](https://docs.rapidata.ai)!

# About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.

# Other Datasets

We run a benchmark of the major video generation models, the results can be found on our [website](https://www.rapidata.ai/leaderboard/video-models). We rank the models according to their coherence/plausiblity, their aligment with the given prompt and style prefernce. The underlying 2M+ annotations can be found here:

- Link to the [Rich Video Annotation dataset](https://huggingface.co/datasets/Rapidata/text-2-video-Rich-Human-Feedback)
- Link to the [Coherence dataset](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Coherence_Dataset)
- Link to the [Text-2-Image Alignment dataset](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Alignment_Dataset)
- Link to the [Preference dataset](https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3)

.horizontal-container {
display: flex;
flex-direction: row;
justify-content: center;
gap: 60px;
}

.image-container img {
max-height: 250px; /* 设置所需高度 */
margin:0;
object-fit: contain; /* 保持图片宽高比 */
width: auto; /* 根据高度自动调整宽度 */
box-sizing: content-box;
}

.image-container {
display: flex; /* 将图片横向排列 */
justify-content: space-around; /* 均匀分布间距 */
align-items: center; /* 垂直居中对齐 */
gap: .5rem
}

.container {
width: 90%;
margin: 0 auto;
}

# 1000个排序视频数据集
本数据集包含约1000条视频，基于超过2.5万次成对比较的人类反馈，按偏好程度从高到低排序。所有视频仅以人类标注员评估的连贯性为评分依据，不考虑生成时使用的具体提示词。每条视频均关联其生成所用的模型名称。
本数据集的视频采样自我们的基准数据集 text-2-video-human-preferences-pika2.2（https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences-pika2.2）。欢迎关注我们，以了解更多基准数据集，并获取最新文本到图像（text-to-image, T2I）及文本到视频（text-to-video, T2V）生成模型的人类反馈研究动态。

## 示例
**高偏好度视频示例**
以下为标注员排序中连贯性最优的视频示例。需注意，本次评估的问题为：「哪条视频的瑕疵更多，且更像是AI生成的？」
因此，Elo评分越高代表模型表现越差。值得注意的是，表现最优的视频大多来自veo2与ray2模型，这与我们的基准测试观测结果（https://www.rapidata.ai/leaderboard/video-models）高度一致。

**低偏好度视频示例**
这些视频的偏好评分最低，通常存在令人不适、变形或画质黯淡的画面，会对观看体验造成负面影响。这类视频大多来自hunyuan、pika2.2等模型，这与它们在我们的基准测试排行榜（https://www.rapidata.ai/leaderboard/video-models）上的较低排名相符。

可通过我们的API（https://docs.rapidata.ai）为您自己的视频进行排序！

# 关于Rapidata
Rapidata的技术让大规模人类反馈收集工作比以往任何时候都更快捷、更易获取。请访问rapidata.ai（https://www.rapidata.ai/），了解我们如何革新AI开发中的人类反馈收集流程。

# 其他数据集
我们对主流视频生成模型开展了基准测试，测试结果可在我们的官网（https://www.rapidata.ai/leaderboard/video-models）查看。我们依据模型的连贯性/合理性、与给定提示词的对齐程度以及风格偏好对模型进行排序。相关的200万+标注数据可通过以下链接获取：
- Rich Video Annotation数据集链接：https://huggingface.co/datasets/Rapidata/text-2-video-Rich-Human-Feedback
- Coherence数据集链接：https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Coherence_Dataset
- Text-2-Image Alignment数据集链接：https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Alignment_Dataset
- Preference数据集链接：https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3