Codatta-dev/MM-Food

---
license: openrail
task_categories:
- image-classification
- image-to-text
language:
- en
size_categories:
- 100K<n<1M
---

## Overview

This project introduces a comprehensive food image dataset designed for computer vision tasks, particularly food recognition, classification, and nutritional analysis. The dataset aims to provide a reliable resource for researchers and developers to advance food AI applications including smart recipe recommendations, meal management, and health monitoring systems.

- **Technical Report** - [MM-Food-100K: A 100,000-Sample Multimodal Food Intelligence Dataset with Verifiable Provenance](https://huggingface.co/papers/2508.10429)

## Motivation

Tracking food intake is key to achieving health goals, but traditional food diaries are cumbersome. While new AI applications can quickly log meals with photos, their accuracy has significant shortcomings. Existing AI models perform poorly with diverse global foods; for example, calorie estimation for Asian dishes can have error rates as high as 76%.

**Key Limitations of Existing Datasets:**
- **Insufficient Food Diversity**: Lack richness in quantity, variety, and geographical coverage
- **Monolithic Annotation Information**: Overly simplistic annotations focused only on food names without portion sizes or nutritional content
- **Unrealistic Image Quality**: Highly curated images bearing little resemblance to real-world casual photos

## Dataset Contents

The dataset consists of high-quality food images with detailed metadata in JSON format. Each record includes:

- `image_url`: Link to the image file
- `dish_name`: Main category or dish name (e.g., "Fried Eggs with Toast")
- `food_type`: Source/context (e.g., "Homemade food," "Restaurant food")
- `ingredients`: List of food ingredients (e.g., `["eggs", "bread", "olive oil"]`)
- `portion_size`: Estimated weight of each ingredient (e.g., `["eggs:100g", "bread:50g"]`)
- `nutritional_profile`: Detailed nutrition in JSON format including:
- `calories_kcal`: Calories
- `protein_g`: Protein
- `fat_g`: Fat
- `carbohydrate_g`: Carbohydrates
- `cooking_method`: Preparation method (e.g., "Frying," "Stir-frying")
- `camera_or_phone_prob`: Probability image is user-taken photo vs. online download
- `online_download_prob`: Probability of online source
- `food_prob`: Probability image contains food

## Key Statistics

- **Number of Images**: 100,000 food images
- **Food Type Distribution**:
- Homemade food: 46,555
- Restaurant food: 35,461
- Raw vegetables and fruits: 9,357
- Packaged food: 8,354
- Others: 273

- **Camera/Phone Probability Distribution**:
- 0.8: 47,879
- 0.7: 51,629
- 0.9: 200
- 0.85: 161
- 0.6: 131

## Data Collection and Annotation Process

**Three-Step Hybrid Approach:**

1. **Data Collection and Human Pre-annotation**
- 1.2 million food images from Booster campaign (Codatta & Binance collaboration)
- Filtered to 1 million high-quality images after cleaning
- Professional annotation team performed pre-annotation: region, food name, category, brand, portion size

2. **Multi-model Automated Annotation**
- Used GPT-4o and Qwen-max-latest for deeper automated annotation
- Generated detailed information: food name, category, ingredients, cooking method, calories

3. **Human Secondary Evaluation and Quality Control**
- Booster campaign users reviewed and corrected AI model outputs
- Comprehensive verification of all key fields for accuracy

## Usage

The dataset supports multiple applications:

- **Food Recognition and Classification**: Train models for food type identification
- **Nutritional Estimation**: Estimate nutritional content and dietary analysis
- **Recipe Recommendation Systems**: Develop smart recommendation systems from food images
- **Health Management and Monitoring**: Support wearables and mobile health apps
- **Restaurant Automation**: Enable visual recognition for smart restaurants and robots
- **Computer Vision Research**: Provide benchmarks for image recognition and fine-grained classification

## License and Open-Source Details

- **Full Dataset**: 1 million images (private)
- **Open-Source Subset**: 100,000 data entries
- **License**: OpenRAIL-M (non-commercial use)
- **Commercial Use**: Requires separate license agreement
- **Contact**: [e-mail]