datapointai/vibe-landing-page-arena

--- license: cc-by-4.0 task_categories: - image-classification - visual-question-answering language: - en tags: - human-preference - design - vibe-coding - pairwise-comparison - bradley-terry - web-design - ai-code-generation - landing-pages pretty_name: "Vibe Landing Page Arena" size_categories: - 1K<n<10K --- <img src="https://huggingface.co/datasets/datapointai/vibe-landing-page-arena/resolve/main/datapointlogo.png" alt="Datapoint AI" width="400"> # Vibe Landing Page Arena A large-scale human preference dataset for evaluating AI-generated landing page design quality. 36,000 pairwise judgments from 3,492 annotators comparing landing pages generated by Claude Code, Cursor, Lovable, and Replit across 100 prompts and 4 design dimensions. ## Overview | Metric | Value | |--------|-------| | Total judgments | 36,000 | | Unique annotators | 3,492 | | Prompts | 100 | | Business categories | 97 | | Design tones | 82 | | Tools compared | 4 (Claude Code, Cursor, Lovable, Replit) | | Evaluation dimensions | 4 (aesthetic, typography, layout, completeness) | | Judgments per matchup per dimension | 15 | | Tool pairs per prompt | 6 (all C(4,2) combinations) | ## How the data was collected 1. **100 detailed prompts** were written, each specifying a business name, brand description, page sections (hero, features, pricing, testimonials, etc.), color palette, typography, and design tone. 2. Each prompt was sent to **4 AI code generation tools**: Claude Code (Sonnet 4.6), Cursor (Sonnet 4.6), Lovable, and Replit. Each tool generated a single-file HTML landing page. 3. Full-page **screenshots** were captured at 1440x900 using Playwright. 4. All 6 possible tool pairs per prompt were served as **pairwise image comparisons** on the [Datapoint](https://trydatapoint.com) annotation platform. 5. For each comparison, annotators evaluated **4 dimensions independently**: aesthetic appeal, typography, layout, and completeness. 6. **Display order was randomized** per serving to eliminate left/right position bias. 7. Each matchup received **15 independent judgments per dimension**. ## Dataset Structure ### `comparisons` (2,400 rows) Each row is one aggregated comparison: one tool pair, one dimension, with screenshots, prompt text, and vote counts from 15 annotators. | Column | Type | Description | |--------|------|-------------| | `image_a` | image | Full-page screenshot of tool_a's generated landing page | | `image_b` | image | Full-page screenshot of tool_b's generated landing page | | `tool_a` | string | First tool in the pair | | `tool_b` | string | Second tool in the pair | | `prompt_id` | int | Prompt ID (1-100) | | `brand` | string | Business name from the prompt | | `category` | string | Business category (e.g., "SaaS", "fintech", "restaurant") | | `tone` | string | Design tone (e.g., "minimalist", "bold", "luxury") | | `prompt` | string | Full prompt text used to generate the landing page | | `dimension` | string | Evaluation dimension (see questions below) | | `dimension_question` | string | The exact question annotators answered | | `votes_a` | int | Number of annotators who preferred tool_a (out of 15) | | `votes_b` | int | Number of annotators who preferred tool_b (out of 15) | | `winner` | string | "A" (tool_a majority), "B" (tool_b majority), or "tie" | ### Evaluation Dimensions Each comparison was evaluated on 4 independent dimensions. Annotators answered one question per dimension: | Dimension | Question | |-----------|----------| | **aesthetic** | "Which design looks better at first glance?" | | **typography** | "Which has better font choices, sizing, and readability?" | | **layout** | "Which has better spacing, alignment, and visual flow?" | | **completeness** | "Which has more fully-built sections with no empty or broken areas?" | ### `prompts` (100 rows) | Column | Type | Description | |--------|------|-------------| | `id` | int | Prompt ID (1-100) | | `category` | string | Business category | | `tone` | string | Design tone | | `prompt` | string | Full prompt text | ### `screenshots` (400 images) Full-page screenshots of all generated landing pages (100 prompts x 4 tools), captured at 1440x900 viewport. ## Key Findings ### Overall Rankings (Bradley-Terry) | Rank | Tool | Strength | 95% CI | |------|------|----------|--------| | 1 | Cursor | 0.271 | 0.265 - 0.277 | | 2 | Claude | 0.269 | 0.263 - 0.274 | | 3 | Lovable | 0.262 | 0.256 - 0.267 | | 4 | Replit | 0.199 | 0.194 - 0.204 | The top 3 tools are **statistically indistinguishable** (Cursor vs Claude: p = 1.0; Claude vs Lovable: p = 0.14). Replit is significantly behind (p < 0.001). ### Dimension Specialization No single tool wins every dimension: | Dimension | #1 | #2 | #3 | #4 | |-----------|----|----|----|----| | Aesthetic | Lovable | Cursor | Claude | Replit | | Typography | Cursor | Claude | Lovable | Replit | | Layout | Lovable | Claude | Cursor | Replit | | Completeness | Claude | Cursor | Lovable | Replit | ### Category Specialization - **Lovable** ranks #1 in 35/97 categories (consumer brands, lifestyle, ecommerce) - **Claude** ranks #1 in 32/97 categories (professional services, enterprise, fintech) - **Cursor** ranks #1 in 17/97 categories (SaaS, tech, agency) - **Replit** ranks #1 in 13/97 categories (developer tools, compliance) ## Usage ```python from datasets import load_dataset # Load pairwise comparison judgments comparisons = load_dataset("datapointai/vibe-landing-page-arena", "comparisons") # Load prompts prompts = load_dataset("datapointai/vibe-landing-page-arena", "prompts") # Load screenshots screenshots = load_dataset("datapointai/vibe-landing-page-arena", "screenshots") ``` ### Reproduce the Bradley-Terry analysis ```python import numpy as np from scipy.optimize import minimize # Count wins per tool pair wins = {} for row in comparisons["train"]: a, b = row["tool_a"], row["tool_b"] if row["choice"] == "A": wins[(a, b)] = wins.get((a, b), 0) + 1 else: wins[(b, a)] = wins.get((b, a), 0) + 1 # Fit Bradley-Terry model tools = ["claude", "cursor", "lovable", "replit"] idx = {t: i for i, t in enumerate(tools)} def neg_log_likelihood(params): nll = 0.0 for (a, b), count in wins.items(): p = 1.0 / (1.0 + np.exp(params[idx[b]] - params[idx[a]])) nll -= count * np.log(max(p, 1e-10)) return nll result = minimize(neg_log_likelihood, np.zeros(4), method="L-BFGS-B", bounds=[(0,0)] + [(None,None)]*3) strengths = np.exp(result.x) / np.exp(result.x).sum() for tool, s in sorted(zip(tools, strengths), key=lambda x: -x[1]): print(f"{tool}: {s:.4f}") ``` ## Methodology - **Ranking model:** Bradley-Terry with 1,000 bootstrap iterations for 95% confidence intervals - **Significance testing:** Likelihood ratio tests between adjacent-ranked tools - **Position bias:** Verified negligible via BT model with position parameter (delta = -0.03, CI crosses zero). Display order randomized per serving. - **Annotator quality:** Platform uses calibration tasks with known gold-standard answers to compute annotator trust scores. 60% of calibrated annotators achieved perfect trust scores (1.0). ## Comparison to Related Work | | This dataset | [Vibe Design Arena v1](https://huggingface.co/datasets/datapointai/vibe-design-arena) | Verita AI Study | |---|---|---|---| | Prompts | 100 (controlled) | 60 (real-world apps) | 80 (controlled) | | Tools | 4 | 6 | 4 | | Dimensions | 4 | 1 | 4 | | Total judgments | 36,000 | ~53,000 | 1,260 | | Annotators | 3,492 | unknown | 5 | | Judgments per matchup | 15 per dimension | 30 | ~3 | | Position randomization | Yes | Yes | Not reported | | Statistical model | Bradley-Terry + bootstrap CI | Win rate | Bradley-Terry | ## License CC-BY-4.0 ## Citation ```bibtex @dataset{vibe_landing_page_arena_2026, title={Vibe Landing Page Arena: Human Preference Evaluation of AI-Generated Landing Page Design}, author={Datapoint AI}, year={2026}, url={https://huggingface.co/datasets/datapointai/vibe-landing-page-arena}, note={36,000 pairwise judgments across 4 tools, 100 prompts, and 4 design dimensions} } ``` ## Contact Built by [Datapoint AI](https://trydatapoint.com). Questions or feedback: [e-mail]许可证：CC-BY-4.0 任务类别： - 图像分类（image-classification） - 视觉问答（visual-question-answering） 语言： - 英语（en） 标签： - 人类偏好（human-preference） - 设计（design） - 氛围感代码生成（vibe-coding） - 成对比较（pairwise-comparison） - 布拉德利-特里模型（Bradley-Terry） - 网页设计（web-design） - AI代码生成（ai-code-generation） - 着陆页（landing-pages） 美观名称："氛围感着陆页竞技场（Vibe Landing Page Arena）" 规模类别：1000 < 样本量 < 10000  # 氛围感着陆页竞技场（Vibe Landing Page Arena） 这是一个用于评估AI生成着陆页设计质量的大规模人类偏好数据集，包含来自3492名标注者的36000条成对比较判断，基于100个提示词和4个设计维度，对比了Claude Code、Cursor、Lovable和Replit四款工具生成的着陆页。 ## 概述 | 指标 | 数值 | |--------|-------| | 总判断数 | 36,000 | | 唯一标注者数 | 3,492 | | 提示词数量 | 100 | | 商业类别数 | 97 | | 设计风格数 | 82 | | 对比工具数 | 4款（Claude Code、Cursor、Lovable、Replit） | | 评估维度 | 4个（美学、排版、布局、完整性） | | 每组对决每维度的判断数 | 15 | | 每个提示词对应的工具对数量 | 6组（所有C(4,2)组合） | ## 数据收集流程 1. **100个详细提示词**：每个提示词均指定了企业名称、品牌描述、页面板块（首屏、功能区、定价区、推荐语区等）、配色方案、排版规范与设计风格。 2. 将每个提示词输入**4款AI代码生成工具**：Claude Code（Sonnet 4.6）、Cursor（Sonnet 4.6）、Lovable与Replit，每款工具均生成单文件HTML格式的着陆页。 3. 使用Playwright工具捕获1440×900分辨率的全页截图。 4. 将每个提示词对应的所有6组工具对，以**成对图像对比**的形式部署在Datapoint标注平台（https://trydatapoint.com）上。 5. 针对每组对比，标注者需独立评估4个维度：美学吸引力、排版、布局与完整性。 6. 每次展示的图像顺序均随机化，以消除左右位置偏差。 7. 每组对决的每个维度均收到15条独立判断。 ## 数据集结构 ### `comparisons`（2400条数据） 每条数据对应一组聚合后的比较结果：包含一组工具对、一个评估维度，以及对应的截图、提示词文本与15名标注者的投票统计。 | 列名 | 数据类型 | 描述 | |--------|------|-------------| | `image_a` | 图像 | 工具a生成的着陆页全页截图 | | `image_b` | 图像 | 工具b生成的着陆页全页截图 | | `tool_a` | 字符串 | 工具对中的第一个工具 | | `tool_b` | 字符串 | 工具对中的第二个工具 | | `prompt_id` | 整数 | 提示词ID（1-100） | | `brand` | 字符串 | 提示词中指定的企业名称 | | `category` | 字符串 | 商业类别（例如"SaaS"、"金融科技"、"餐厅"） | | `tone` | 字符串 | 设计风格（例如"极简"、"醒目"、"奢华"） | | `prompt` | 字符串 | 用于生成着陆页的完整提示词文本 | | `dimension` | 字符串 | 评估维度（详见下文说明） | | `dimension_question` | 字符串 | 标注者需回答的具体问题 | | `votes_a` | 整数 | 偏好工具a的标注者数量（共15名） | | `votes_b` | 整数 | 偏好工具b的标注者数量（共15名） | | `winner` | 字符串 | 获胜方："A"（工具a获得多数票）、"B"（工具b获得多数票）或"平局" | ### 评估维度 每组对比均从4个独立维度进行评估，标注者需针对每个维度回答一个问题： | 维度 | 问题 | |-----------|----------| | **美学（aesthetic）** | "第一眼看上去哪个设计更出色？" | | **排版（typography）** | "哪个设计的字体选择、字号与可读性更优？" | | **布局（layout）** | "哪个设计的间距、对齐方式与视觉流畅度更优？" | | **完整性（completeness）** | "哪个设计的板块更完整，无空白或损坏区域？" | ### `prompts`（100条数据） | 列名 | 数据类型 | 描述 | |--------|------|-------------| | `id` | 整数 | 提示词ID（1-100） | | `category` | 字符串 | 商业类别 | | `tone` | 字符串 | 设计风格 | | `prompt` | 字符串 | 完整提示词文本 | ### `screenshots`（400张图像） 所有生成的着陆页的全页截图（100个提示词 × 4款工具），捕获分辨率为1440×900的视口截图。 ## 核心发现 ### 整体排名（布拉德利-特里模型，Bradley-Terry） | 排名 | 工具 | 强度值 | 95%置信区间 | |------|------|----------|--------| | 1 | Cursor | 0.271 | 0.265 - 0.277 | | 2 | Claude | 0.269 | 0.263 - 0.274 | | 3 | Lovable | 0.262 | 0.256 - 0.267 | | 4 | Replit | 0.199 | 0.194 - 0.204 | 排名前三的工具在统计学上无显著差异（Cursor与Claude对比：p=1.0；Claude与Lovable对比：p=0.14）。Replit的表现显著落后（p<0.001）。 ### 维度专项表现 没有任何一款工具在所有维度上均位列第一： | 维度 | 第1名 | 第2名 | 第3名 | 第4名 | |-----------|----|----|----|----| | 美学 | Lovable | Cursor | Claude | Replit | | 排版 | Cursor | Claude | Lovable | Replit | | 布局 | Lovable | Claude | Cursor | Replit | | 完整性 | Claude | Cursor | Lovable | Replit | ### 商业类别专项表现 - Lovable在97个类别中的35个类别中排名第一（消费品牌、生活方式、电商领域） - Claude在97个类别中的32个类别中排名第一（专业服务、企业服务、金融科技领域） - Cursor在97个类别中的17个类别中排名第一（SaaS、科技、代理机构领域） - Replit在97个类别中的13个类别中排名第一（开发者工具、合规领域） ## 使用方法 python from datasets import load_dataset # 加载成对比较判断数据 comparisons = load_dataset("datapointai/vibe-landing-page-arena", "comparisons") # 加载提示词数据 prompts = load_dataset("datapointai/vibe-landing-page-arena", "prompts") # 加载截图数据 screenshots = load_dataset("datapointai/vibe-landing-page-arena", "screenshots") ### 复现布拉德利-特里模型分析 python import numpy as np from scipy.optimize import minimize # 统计每对工具的获胜次数 wins = {} for row in comparisons["train"]: a, b = row["tool_a"], row["tool_b"] if row["choice"] == "A": wins[(a, b)] = wins.get((a, b), 0) + 1 else: wins[(b, a)] = wins.get((b, a), 0) + 1 # 拟合布拉德利-特里模型 tools = ["claude", "cursor", "lovable", "replit"] idx = {t: i for i, t in enumerate(tools)} def neg_log_likelihood(params): nll = 0.0 for (a, b), count in wins.items(): p = 1.0 / (1.0 + np.exp(params[idx[b]] - params[idx[a]])) nll -= count * np.log(max(p, 1e-10)) return nll result = minimize(neg_log_likelihood, np.zeros(4), method="L-BFGS-B", bounds=[(0,0)] + [(None,None)]*3) strengths = np.exp(result.x) / np.exp(result.x).sum() for tool, s in sorted(zip(tools, strengths), key=lambda x: -x[1]): print(f"{tool}: {s:.4f}") ## 研究方法 - **排名模型**：布拉德利-特里模型，通过1000次自助法迭代计算95%置信区间 - **显著性检验**：相邻排名工具间的似然比检验 - **位置偏差验证**：通过加入位置参数的布拉德利-特里模型验证偏差可忽略（delta=-0.03，置信区间包含0），每次展示的图像顺序均随机化 - **标注者质量**：平台使用带有已知标准答案的校准任务计算标注者信任得分，60%的校准后标注者获得了满分信任得分（1.0） ## 与相关工作的对比 | | 本数据集 | [Vibe Design Arena v1](https://huggingface.co/datasets/datapointai/vibe-design-arena) | Verita AI研究 | |---|---|---|---| | 提示词数量 | 100（受控设计） | 60（真实应用场景） | 80（受控设计） | | 对比工具数 | 4款 | 6款 | 4款 | | 评估维度数 | 4个 | 1个 | 4个 | | 总判断数 | 36,000 | ~53,000 | 1,260 | | 标注者数量 | 3,492 | 未知 | 5名 | | 每组对决的判断数 | 每个维度15条 | 30条 | ~3条 | | 图像顺序随机化 | 是 | 是 | 未提及 | | 统计模型 | 布拉德利-特里模型+自助法置信区间 | 获胜率 | 布拉德利-特里模型 | ## 许可证 CC-BY-4.0 ## 引用格式 bibtex @dataset{vibe_landing_page_arena_2026, title={Vibe Landing Page Arena: Human Preference Evaluation of AI-Generated Landing Page Design}, author={Datapoint AI}, year={2026}, url={https://huggingface.co/datasets/datapointai/vibe-landing-page-arena}, note={36,000 pairwise judgments across 4 tools, 100 prompts, and 4 design dimensions} } ## 联系方式 本数据集由Datapoint AI（https://trydatapoint.com）开发。如有疑问或反馈，请联系：[e-mail]