mohamedashraff22/arabic-menus-dataset

--- task_categories: - image-to-text language: - ar tags: - arabic - ocr - egyptian - menus - vlm - image-text - multimodal pretty_name: restaurants menus --- # 🍽️ Arabic Restaurant Menus OCR Dataset A large-scale, high-quality multimodal dataset designed specifically for fine-tuning Vision-Language Models (VLMs) on zero-shot Information Extraction (IE) and Optical Character Recognition (OCR) tasks. This dataset focuses exclusively on authentic, culturally specific Arabic restaurant menus, containing localized formatting, dialectal food names, and dynamic pricing structures. ## 📊 Dataset Overview * **Total Images:** ~3,400 high-resolution menu images * **Language:** Arabic (Primary), English (Secondary/Bilingual items) * **Annotation Format:** LLaMA-Factory / ShareGPT compatible JSON * **Ground Truth:** Flattened, structured JSON arrays containing individual `name` and `price` pairs. --- ## 🏗️ How This Data Was Created Annotating thousands of highly complex Arabic menus manually is prone to human error and prohibitively expensive. This dataset's annotations were generated using an advanced multi-key pipeline powered by **Meta's Llama-4-Scout-17B** (via Groq). ### Quality Control Pipeline To ensure only the highest quality data made it into the final sets: 1. **Garbage Collection Threshold:** Any menu that did not contain at least 3 recognizable items with valid prices was automatically deleted from the dataset. 2. **Hallucination Prevention:** The pipeline explicitly forced the generation of a flattened JSON array, preventing nested hallucinations. 3. **Ghost-Image Cleanup:** The final JSON splits (`train.json`, `val.json`) were strictly validated against the physical image directory to prevent "file not found" errors during model training. --- ## 📂 Data Structure The dataset follows the standard **ShareGPT** mixed-modal format required by modern fine-tuning frameworks like `LLaMA-Factory`. ### Example Entry: ```json [ { "messages": [ { "content": "<image>\nExtract every food or drink item and its price from this menu image.\nKeep item names and prices exactly as written in the image.\nIf an item has multiple sizes, list each size as a separate entry.\nOnly include items that have a visible price. Skip anything without a price.\nReturn a JSON object with one key \"items\" containing an array. Each item has \"name\" (string) and \"price\" (string).", "role": "user" }, { "content": "{\"items\": [{\"name\": \"شاورما لحم\", \"price\": \"45\"}, {\"name\": \"شاورما دجاج\", \"price\": \"35\"}]}", "role": "assistant" } ], "images": [ "/images/menu_001.jpg" ] } ] ``` ### 🗂️ Splits * **`train.json`**: The primary training split. * **`val.json`**: Hold-out validation split for evaluating loss convergence. * **`images/`**: The physical `.jpg`, `.png`, and `.webp` files referenced in the JSON manifests. --- ## 🚀 How to Use (LLaMA-Factory) This dataset is plug-and-play with [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory). Simply append this rule to your `dataset_info.json`: ```json "arabic_ocr_menus": { "file_name": "train.json", "formatting": "sharegpt", "columns": { "messages": "messages", "images": "images" } } ``` ## ⚠️ Limitations & Bias * The dataset is primarily sourced from menus in the Middle East and North Africa (MENA) region. Distinct local dialects (e.g., specific terms used only in Egyptian vs. Gulf MENA menus) are represented as they appear natively. * While the annotations are highly accurate, they were heavily generated via an intermediary AI (Llama 4). Minor OCR hallucinations in highly blurry images may occasionally be present.