Codatta-dev/MM-Food

--- license: openrail task_categories: - image-classification - image-to-text language: - en size_categories: - 100K<n<1M --- ## Overview This project introduces a comprehensive food image dataset designed for computer vision tasks, particularly food recognition, classification, and nutritional analysis. The dataset aims to provide a reliable resource for researchers and developers to advance food AI applications including smart recipe recommendations, meal management, and health monitoring systems. - **Technical Report** - [MM-Food-100K: A 100,000-Sample Multimodal Food Intelligence Dataset with Verifiable Provenance](https://huggingface.co/papers/2508.10429) ## Motivation Tracking food intake is key to achieving health goals, but traditional food diaries are cumbersome. While new AI applications can quickly log meals with photos, their accuracy has significant shortcomings. Existing AI models perform poorly with diverse global foods; for example, calorie estimation for Asian dishes can have error rates as high as 76%. **Key Limitations of Existing Datasets:** - **Insufficient Food Diversity**: Lack richness in quantity, variety, and geographical coverage - **Monolithic Annotation Information**: Overly simplistic annotations focused only on food names without portion sizes or nutritional content - **Unrealistic Image Quality**: Highly curated images bearing little resemblance to real-world casual photos ## Dataset Contents The dataset consists of high-quality food images with detailed metadata in JSON format. Each record includes: - `image_url`: Link to the image file - `dish_name`: Main category or dish name (e.g., "Fried Eggs with Toast") - `food_type`: Source/context (e.g., "Homemade food," "Restaurant food") - `ingredients`: List of food ingredients (e.g., `["eggs", "bread", "olive oil"]`) - `portion_size`: Estimated weight of each ingredient (e.g., `["eggs:100g", "bread:50g"]`) - `nutritional_profile`: Detailed nutrition in JSON format including: - `calories_kcal`: Calories - `protein_g`: Protein - `fat_g`: Fat - `carbohydrate_g`: Carbohydrates - `cooking_method`: Preparation method (e.g., "Frying," "Stir-frying") - `camera_or_phone_prob`: Probability image is user-taken photo vs. online download - `online_download_prob`: Probability of online source - `food_prob`: Probability image contains food ## Key Statistics - **Number of Images**: 100,000 food images - **Food Type Distribution**: - Homemade food: 46,555 - Restaurant food: 35,461 - Raw vegetables and fruits: 9,357 - Packaged food: 8,354 - Others: 273 - **Camera/Phone Probability Distribution**: - 0.8: 47,879 - 0.7: 51,629 - 0.9: 200 - 0.85: 161 - 0.6: 131 ## Data Collection and Annotation Process **Three-Step Hybrid Approach:** 1. **Data Collection and Human Pre-annotation** - 1.2 million food images from Booster campaign (Codatta & Binance collaboration) - Filtered to 1 million high-quality images after cleaning - Professional annotation team performed pre-annotation: region, food name, category, brand, portion size 2. **Multi-model Automated Annotation** - Used GPT-4o and Qwen-max-latest for deeper automated annotation - Generated detailed information: food name, category, ingredients, cooking method, calories 3. **Human Secondary Evaluation and Quality Control** - Booster campaign users reviewed and corrected AI model outputs - Comprehensive verification of all key fields for accuracy ## Usage The dataset supports multiple applications: - **Food Recognition and Classification**: Train models for food type identification - **Nutritional Estimation**: Estimate nutritional content and dietary analysis - **Recipe Recommendation Systems**: Develop smart recommendation systems from food images - **Health Management and Monitoring**: Support wearables and mobile health apps - **Restaurant Automation**: Enable visual recognition for smart restaurants and robots - **Computer Vision Research**: Provide benchmarks for image recognition and fine-grained classification ## License and Open-Source Details - **Full Dataset**: 1 million images (private) - **Open-Source Subset**: 100,000 data entries - **License**: OpenRAIL-M (non-commercial use) - **Commercial Use**: Requires separate license agreement - **Contact**: [e-mail]许可证：OpenRAIL 任务类别： - 图像分类 - 图像到文本 语言： - 英语 规模类别： - 10万 < 样本数 < 100万 ## 概述 本项目发布了一款面向计算机视觉任务的高质量食品图像数据集，可应用于食品识别、分类与营养分析等场景。本数据集旨在为研究人员与开发者提供可靠的资源支撑，助力智能食谱推荐、膳食管理、健康监测系统等食品AI应用的迭代升级。 - **技术报告** - [MM-Food-100K：一款包含10万样本、可溯源的多模态食品智能数据集](https://huggingface.co/papers/2508.10429) ## 研究背景 追踪食物摄入是达成健康目标的核心环节，但传统饮食日记操作繁琐。尽管新兴AI应用可通过照片快速记录膳食，但其准确率存在显著短板。现有AI模型在处理多样化的全球食品时表现欠佳，例如亚洲菜品的热量估算误差率最高可达76%。 **现有数据集的核心局限：** - **食品多样性不足**：在样本数量、品类覆盖与地理维度上均缺乏丰富性 - **标注信息单一**：标注过于简化，仅涵盖食品名称，未包含分量或营养成分信息 - **图像质量脱离实际**：经过高度精修的图像，与现实场景下的随手拍摄照片差异较大 ## 数据集内容 本数据集包含高质量食品图像与JSON格式的详细元数据，单条记录包含以下字段： - `image_url`：图像文件链接 - `dish_name`：主品类或菜品名称（例如"吐司煎蛋"） - `food_type`：食品来源/场景（例如"自制食品""餐厅餐食"） - `ingredients`：食品原料列表（例如`["鸡蛋", "面包", "橄榄油"]`） - `portion_size`：各原料的估算重量（例如`["鸡蛋：100g", "面包：50g"]`） - `nutritional_profile`：JSON格式的详细营养信息，包括： - `calories_kcal`：热量（千卡） - `protein_g`：蛋白质（克） - `fat_g`：脂肪（克） - `carbohydrate_g`：碳水化合物（克） - `cooking_method`：制备方式（例如"煎制""快炒"） - `camera_or_phone_prob`：图像为用户实拍而非网络下载的概率 - `online_download_prob`：图像来源于网络的概率 - `food_prob`：图像包含食品的置信度 ## 关键统计数据 - **图像总数**：100,000张食品图像 - **食品类型分布**： - 自制食品：46,555条 - 餐厅餐食：35,461条 - 生鲜蔬果：9,357条 - 预包装食品：8,354条 - 其他：273条 - **相机/手机拍摄概率分布**： - 0.8：47,879条 - 0.7：51,629条 - 0.9：200条 - 0.85：161条 - 0.6：131条 ## 数据采集与标注流程 **三步混合标注流程：** 1. **数据采集与人工预标注** - 从Booster活动（Codatta与Binance合作项目）获取120万张食品图像 - 经过清洗后筛选出100万张高质量图像 - 专业标注团队完成预标注：涵盖区域、食品名称、品类、品牌与分量 2. **多模型自动化标注** - 采用GPT-4o与最新版Qwen进行深度自动化标注 - 生成食品名称、品类、原料、制备方式与热量等详细信息 3. **人工二次审核与质量管控** - Booster活动用户对AI生成的标注结果进行审核与修正 - 对所有核心字段进行全面验证以确保准确性 ## 应用场景 本数据集支持多类应用： - **食品识别与分类**：训练食品品类识别模型 - **营养估算**：实现营养成分估算与膳食分析 - **食谱推荐系统**：基于食品图像开发智能推荐系统 - **健康管理与监测**：为可穿戴设备与移动健康应用提供支撑 - **餐厅自动化**：为智能餐厅与服务机器人提供视觉识别能力 - **计算机视觉研究**：为图像识别与细粒度分类任务提供基准数据集 ## 许可证与开源详情 - **完整数据集**：包含100万张图像（私有版本） - **开源子集**：包含100,000条数据 - **许可证**：OpenRAIL-M（非商业使用） - **商业使用**：需另行签署许可协议 - **联系方式**：[e-mail]