text-2-video-human-preferences-genmo-mochi-1

.vertical-container {
display: flex;
flex-direction: column;
gap: 60px;
}

.image-container img {
height: 150px; /* Set the desired height */
margin:0;
object-fit: contain; /* Ensures the aspect ratio is maintained */
width: auto; /* Adjust width automatically based on height */
}

.image-container {
display: flex; /* Aligns images side by side */
justify-content: space-around; /* Space them evenly */
align-items: center; /* Align them vertically */
}

.container {
width: 90%;
margin: 0 auto;
}

.text-center {
text-align: center;
}

.score-amount {
margin: 0;
margin-top: 10px;
}

.score-percentage {
font-size: 12px;
font-weight: semi-bold;
}

# Rapidata Video Generation Genmo Mochi-1 Human Preference

In this dataset, ~60k human responses from ~20k human annotators were collected to evaluate mochi-1 video generation model on our benchmark. This dataset was collected in roughtly 30 min using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale data annotation.

Explore our latest model rankings on our [website](https://www.rapidata.ai/benchmark).

If you get value from this dataset and would like to see more in the future, please consider liking it ❤️

# Overview

In this dataset, ~60k human responses from ~20k human annotators were collected to evaluate mochi-1 video generation model on our benchmark. This dataset was collected in roughtly 30 min using the [Rapidata Python API](https://docs.rapidata.ai), accessible to anyone and ideal for large scale data annotation.
The benchmark data is accessible on [huggingface](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences) directly.

# Explanation of the colums

The dataset contains paired video comparisons. Each entry includes 'video1' and 'video2' fields, which contain links to downscaled GIFs for easy viewing. The full-resolution videos can be found [here](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences-genmo-mochi-1/tree/main/Videos)

The weighted_results column contains scores ranging from 0 to 1, representing aggregated user responses. Individual user responses can be found in the detailedResults column.

# Alignment

The alignment score quantifies how well an video matches its prompt. Users were asked: "Which video fits the description better?".

## Examples

A hummingbird sipping nectar from vibrant flowers in a sunlit tropical garden, its wings a blur of motion amidst the blooming petals.

Mochi-1
(Score: 85.15%)

Alpha
(Score: 14.85%)

A fleet of autonomous ships gracefully maneuvering through a vibrant coral reef, their lights illuminating the underwater flora, navigating around colorful marine life with synchronized precision.

Mochi-1
(Score: 2.48%)

Pika2.2
(Score: 97.52%)

# Coherence

The coherence score measures whether the generated video is logically consistent and free from artifacts or visual glitches. Without seeing the original prompt, users were asked: "Which video has more glitches and is more likely to be AI generated?"

## Examples

Mochi-1
(Glitch Rating: 13.22%)

Pika
(Glitch Rating: 86.78%)

Mochi-1
(Glitch Rating: 84.64%)

Veo 3
(Glitch Rating: 15.36%)

# Preference

The preference score reflects how visually appealing participants found each video, independent of the prompt. Users were asked: "Which video do you prefer aesthetically?"

## Examples

Mochi-1
(Score: 68.51%)

Ray 2
(Score: 31.49%)

Mochi 1
(Score: 0.87%)

Pika
(Score: 91.23%)

# About Rapidata

Rapidata's technology makes collecting human feedback at scale faster and more accessible than ever before. Visit [rapidata.ai](https://www.rapidata.ai/) to learn more about how we're revolutionizing human feedback collection for AI development.

# Other Datasets

We run a benchmark of the major video generation models, the results can be found on our [website](https://www.rapidata.ai/leaderboard/video-models). We rank the models according to their coherence/plausiblity, their aligment with the given prompt and style prefernce. The underlying 2M+ annotations can be found here:

- Link to the [Rich Video Annotation dataset](https://huggingface.co/datasets/Rapidata/text-2-video-Rich-Human-Feedback)
- Link to the [Coherence dataset](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Coherence_Dataset)
- Link to the [Text-2-Image Alignment dataset](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Alignment_Dataset)
- Link to the [Preference dataset](https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3)

# Rapidata 视频生成 Genmo Mochi-1 人类偏好数据集

本数据集收集了约2万名人类标注者的总计约6万条人类反馈，用于在我们的基准测试中评估Genmo Mochi-1视频生成模型。本数据集通过[Rapidata Python应用程序编程接口（API）](https://docs.rapidata.ai)耗时约30分钟即可完成采集，该接口面向所有用户开放，非常适合大规模数据标注工作。

可访问我们的[官网](https://www.rapidata.ai/benchmark)查看最新的模型排名。

若您从本数据集获益并希望未来获取更多同类资源，欢迎为该数据集点赞 ❤️

# 概述
本数据集收集了约2万名人类标注者的总计约6万条人类反馈，用于在我们的基准测试中评估Genmo Mochi-1视频生成模型。本数据集通过[Rapidata Python API](https://docs.rapidata.ai)耗时约30分钟即可完成采集，该接口面向所有用户开放，非常适合大规模数据标注工作。
基准测试数据可直接在[Hugging Face](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences)获取。

# 字段说明
本数据集包含成对视频对比样本。每条数据均包含`video1`与`video2`字段，其中存储了用于便捷查看的压缩版GIF格式视频链接。全分辨率视频可通过[此链接](https://huggingface.co/datasets/Rapidata/text-2-video-human-preferences-genmo-mochi-1/tree/main/Videos)获取。
`weighted_results`（加权结果）字段包含0至1区间的得分，代表聚合后的用户反馈结果。单条用户反馈可在`detailedResults`（详细结果）字段中查看。

# 对齐度（Alignment）
对齐度得分用于量化视频与输入提示词的匹配程度。调研中向用户提出的问题为："哪段视频更贴合给定描述？"

## 示例
1. 一段描述：一只蜂鸟在阳光明媚的热带花园中吸食鲜艳花朵的花蜜，翅膀在盛开的花瓣间振翅成模糊的虚影。
- Mochi-1（得分：85.15%）
- Alpha（得分：14.85%）
2. 一段描述：一支自主航行船队在色彩斑斓的珊瑚礁中优雅穿梭，船灯照亮水下植被，以同步精准的姿态绕行多彩海洋生物。
- Mochi-1（得分：2.48%）
- Pika2.2（得分：97.52%）

# 连贯性（Coherence）
连贯性得分用于评估生成视频的逻辑自洽性，以及是否存在伪影或视觉瑕疵。在不展示原始提示词的前提下，向用户提出的问题为："哪段视频存在更多瑕疵，更像是AI生成内容？"

## 示例
1. - Mochi-1（瑕疵评分：13.22%）
- Pika（瑕疵评分：86.78%）
2. - Mochi-1（瑕疵评分：84.64%）
- Veo 3（瑕疵评分：15.36%）

# 偏好度（Preference）
偏好度得分用于反映参与者对各段视频的视觉美观程度评价，与原始提示词无关。调研中向用户提出的问题为："从审美角度而言，你更偏好哪段视频？"

## 示例
1. - Mochi-1（得分：68.51%）
- Ray 2（得分：31.49%）
2. - Mochi 1（得分：0.87%）
- Pika（得分：91.23%）

# 关于Rapidata
Rapidata的技术使大规模人类反馈采集工作比以往任何时候都更加快捷、易用。访问[rapidata.ai](https://www.rapidata.ai/)官网，了解我们如何革新AI开发领域的人类反馈采集流程。

# 其他数据集
我们针对主流视频生成模型开展了基准测试，测试结果可在我们的[官网](https://www.rapidata.ai/leaderboard/video-models)查看。我们依据模型的连贯性/合理性、与给定提示词的对齐度以及风格偏好对模型进行排名。总计超过200万条的底层标注数据可通过以下链接获取：
- 链接至[Rich Video Annotation数据集](https://huggingface.co/datasets/Rapidata/text-2-video-Rich-Human-Feedback)
- 链接至[Coherence数据集](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Coherence_Dataset)
- 链接至[Text-2-Image Alignment数据集](https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Alignment_Dataset)
- 链接至[Preference数据集](https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3)