bananamark-dataset

# 🍌 Bananamark - Who has the best Bananas? <a href="https://www.rapidata.ai"> <img src="https://cdn-uploads.huggingface.co/production/uploads/66f5624c42b853e73e0738eb/jfxR79bOztqaC6_yNNnGU.jpeg" width="400" alt="Dataset visualization"> </a> Google released a model called Nano Banana. We had to know: is it actually good at generating bananas? Turns out, not as good as FLUX 2 or Seedream 4! ## What we did We tested 11 image generation models on various banana-themed prompts and collected over 20'000 human preferences using the [Rapidata Python API](https://docs.rapidata.ai). Annotators were asked a simple question: > **Which image has the better bananas?** <strong>Want to evaluate your own models? Check out [MRI](https://docs.rapidata.ai/mri/)</strong> ## Results Full leaderboard: [app.rapidata.ai/mri/benchmarks/692714af8b22f2eff170f664](https://app.rapidata.ai/mri/benchmarks/692714af8b22f2eff170f664) <a href="https://app.rapidata.ai/mri/benchmarks/692714af8b22f2eff170f664"> <img src="https://cdn-uploads.huggingface.co/production/uploads/672b7d79fd1e92e3c3567435/TCe2nhG1TryRsS2kWXK6H.png" alt="Bananamark Leaderboard"> </a> | | Model | Wins | Total Matches | Score | |----|------------------------------------|------|---------------|----------| | 1 | black-forest-labs/flux-2-flex | 3380 | 5859 | 1128.65 | | 2 | bytedance/seedream-4 | 1658 | 2988 | 1086.38 | | 3 | google/nano-banana-pro | 1711 | 3192 | 1058.92 | | 4 | google/nano-banana | 1953 | 3649 | 1057.67 | | 5 | black-forest-labs/flux-2-pro | 2514 | 4770 | 1055.79 | | 6 | black-forest-labs/flux-2-dev | 2981 | 5916 | 1039.09 | | 7 | qwen/qwen-image | 1447 | 2791 | 1030.80 | | 8 | ideogram-ai/ideogram-v3-quality | 1060 | 2185 | 976.65 | | 9 | tencent/hunyuan-image-3 | 1108 | 2464 | 926.62 | | 10 | recraft-ai/recraft-v3 | 942 | 2189 | 896.15 | | 11 | prunaai/wan-2.2-image | 1348 | 4201 | 743.28 | ## Takeaways - FLUX 2 Flex wins with a comfortable lead - Nano Banana, despite the name, only manages 3rd and 4th place - Black Forest Labs took 4 of the top 6 spots - Wan 2.2 needs some work on its banana game --- *Collected with [Rapidata](https://www.rapidata.ai). Run your own benchmarks with [MRI](https://docs.rapidata.ai/mri/).*