MechVQA

# MechVQA VQA SFT MechVQA VQA SFT is a bilingual visual question answering dataset for supervised fine-tuning on mechanical engineering drawings. This public release contains **13,515 question-answer records** paired with **3,371 unique, content-addressed images**. Assistant targets use `<think>...</think><answer>...</answer>` formatting. This repository is the **VQA-only SFT train/validation release** associated with the MechVQA project. The public evaluation benchmark is maintained separately in the [MechVQA code repository](https://github.com/xiaofengShi/MechVQA). ## Dataset Details | Item | Description | |---|---| | Dataset name | MechVQA VQA SFT | | Task | Mechanical-drawing visual question answering and multimodal SFT | | Languages | Chinese and English | | License | Apache License 2.0 | | Size | 13,515 records and 3,371 unique images | | Public release date | 2026-08-03 | | Curators | Qian Kou, Xiaofeng Shi, Yulin Li, Xiaosong Qiu, Xinyang Wang, Hua Zhou, and Dongxing Cao | ### Dataset Sources - **Project repository:** [xiaofengShi/MechVQA](https://github.com/xiaofengShi/MechVQA) - **Paper:** [MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding](https://arxiv.org/abs/2605.30794) - **Hugging Face mirror:** [XiaofengAlg/MechVQA](https://huggingface.co/datasets/XiaofengAlg/MechVQA) ## Dataset Contents | Split | Records | Unique images | |---|---:|---:| | Train | 12,749 | 3,126 | | Validation | 766 | 245 | | **Total** | **13,515** | **3,371** | The train and validation splits have no image-hash overlap. ### Files - `data/train.jsonl`: 12,749 training records - `data/val.jsonl`: 766 validation records - `images/`: 3,371 PNG or JPEG images stored under content-addressed paths - `dataset_info.json`: standalone LLaMA Factory dataset registration - `checksums.sha256`: SHA-256 manifest for the release package - `audit/`: build lineage, exclusions, image manifest, validation result, and data-quality report ### Languages, Capabilities, and Difficulty | Dimension | Train | Validation | Total | |---|---:|---:|---:| | Chinese | 8,912 | 547 | 9,459 | | English | 3,837 | 219 | 4,056 | | Recognition | 5,101 | 224 | 5,325 | | Reasoning | 2,384 | 73 | 2,457 | | Judging | 5,264 | 469 | 5,733 | | Easy | 5,207 | 347 | 5,554 | | Medium | 4,222 | 261 | 4,483 | | Hard | 3,320 | 158 | 3,478 | Records cover ten paper-aligned VQA subcategories: Anomaly Detection, Assembly Relationship, Consistency Judgment, Dimension & Annotation, Geometric Calculation, Identification & Counting, Item Localization, Projection & Multi-view, Structure Understanding, and Text & Table. ## Dataset Structure Each JSONL record contains three public fields: ```json { "messages": [ {"role": "user", "content": "<image>...question..."}, { "role": "assistant", "content": "<think>...reasoning...</think><answer>...answer...</answer>" } ], "images": ["images/ab/<sha256>.png"], "metadata": { "question_type": "VQA", "data_source": "...", "difficulty": "Easy|Medium|Hard", "capability": "Recognition|Reasoning|Judging", "subcategory": "...", "language": "中文|英文" } } ``` - `messages`: ShareGPT-style user and assistant turns. User content retains the `<image>` placeholder; assistant content contains the SFT target. - `images`: paths relative to the dataset repository root. Image paths are content-addressed using the underlying file SHA-256. - `metadata`: a small public projection for filtering and analysis. Internal audit fields and historical `original_q`, `original_a`, and `correct_answer` fields are intentionally excluded from the public JSONL. ## Downloading the Dataset ### ModelScope CLI ```bash pip install modelscope_hub modelscope download xiaofengalg/MechVQA \ --repo-type dataset \ --local-dir ./MechVQA ``` ### Python ```python from pathlib import Path from modelscope_hub import HubApi root = Path( HubApi().download_repo( "xiaofengalg/MechVQA", repo_type="dataset", local_dir="./MechVQA", max_workers=8, ) ) print(root) ``` To open a referenced image: ```python import json from pathlib import Path from PIL import Image root = Path("./MechVQA") with (root / "data/train.jsonl").open(encoding="utf-8") as handle: sample = json.loads(next(handle)) image = Image.open(root / sample["images"][0]) ``` ### LLaMA Factory Use the downloaded release directory as both `dataset_dir` and `media_dir`: ```yaml dataset_dir: /path/to/MechVQA media_dir: /path/to/MechVQA dataset: mechvqa_vqa_train eval_dataset: mechvqa_vqa_val template: qwen3_vl ``` The included `dataset_info.json` registers `mechvqa_vqa_train` and `mechvqa_vqa_val` with ShareGPT formatting. ## Dataset Creation ### Curation and Processing The release was built from reviewed VQA inputs using the following hard gates: 1. Keep only `question_type=VQA` records. 2. Keep only records whose quality status is `keep` or `fixed`. 3. Exclude the forbidden `mcmaster` data source. 4. Remove records whose exact image SHA-256 overlaps the official public MechVQA evaluation benchmark. 5. Remove train records that share an image with validation; validation has priority and no record is reassigned across splits. 6. Deduplicate exact message-plus-image training payloads, retaining the earliest record. 7. Materialize all images under relative, content-addressed paths and remove internal paths and audit-only fields from public records. | Curation stage | Train | Validation | Total | |---|---:|---:|---:| | Reviewed VQA input | 15,727 | 1,948 | 17,675 | | Quality status `keep` or `fixed` | 14,796 | 1,745 | 16,541 | | After official benchmark decontamination | 13,220 | 766 | 13,986 | | **Final release** | **12,749** | **766** | **13,515** | The release records 4,160 exclusions: 1,134 for quality status, 2,555 for official benchmark image overlap, 465 train-side cross-split image overlaps, and 6 exact payload duplicates. Detailed lineage and exclusion records are in `audit/lineage.jsonl` and `audit/exclusions.jsonl`. ### Validation The packaged audit reports: - 13,515 parseable public records - 3,371 referenced image files and no unreferenced images - zero train/validation image-hash overlap - zero exact image-hash overlap with the official public benchmark - no internal absolute paths or audit-only fields in public records - successful release validation on 2026-08-03 The two JSONL files have these release hashes: | File | SHA-256 | |---|---| | `data/train.jsonl` | `427d15a2a54a16bd5a4f3fb19e1476445f171d90a799eeada24ebebeffe110ee` | | `data/val.jsonl` | `8bf9dfe4e2e2e82cd08275cfe8ece5b84c121559a26a182d9b36d9aa5065028d` | ## Uses ### Intended Uses - Supervised fine-tuning of multimodal models for mechanical-drawing VQA - Research on recognition, reasoning, and judgment over engineering drawings - Controlled analysis or sampling by language, capability, subcategory, and difficulty - Reproduction and extension of the MechVQA training setup ### Out-of-Scope Uses - Treating generated answers or reasoning traces as certified engineering advice - Unsupervised use in safety-critical manufacturing, inspection, or design decisions - Using the training split itself as an unbiased evaluation benchmark - Assuming performance transfers to every drawing standard, language, industry, or image acquisition condition ## Bias, Risks, and Limitations - The distribution is uneven across languages, capabilities, subcategories, difficulties, and source collections. Chinese records are the majority. - Quality review reduces known errors but does not guarantee that every answer or reasoning trace is correct, complete, or optimally concise. - Benchmark decontamination uses exact image SHA-256 matching. It does not prove the absence of visually or semantically similar near-duplicates. - The dataset focuses on mechanical engineering drawings and should not be treated as representative of general multimodal reasoning. - Reasoning traces may teach stylistic artifacts in addition to useful domain reasoning. Users should evaluate both final-answer quality and trace quality. - The release audit targets data quality, schema, paths, duplication, and benchmark overlap; it is not a comprehensive privacy or legal review. Users should retain human expert review for safety-critical applications and report suspected data issues through the [project issue tracker](https://github.com/xiaofengShi/MechVQA/issues). ## Personal and Sensitive Information The dataset is designed around technical drawings rather than personal data, and no personal or sensitive attributes were intentionally collected. Because the audit was not a comprehensive privacy review, users who identify accidental sensitive content should report it through the project issue tracker. ## License The release is distributed under the Apache License 2.0. See `LICENSE` in this dataset repository for the complete terms. ## Citation If you use this dataset, please cite the accompanying MechVQA paper: ```bibtex @misc{kou2026mechvqabenchmarkingenhancingmultimodal, title = {MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding}, author = {Qian Kou and Xiaofeng Shi and Yulin Li and Xiaosong Qiu and Xinyang Wang and Hua Zhou and Dongxing Cao}, year = {2026}, eprint = {2605.30794}, archivePrefix= {arXiv}, primaryClass = {cs.CV}, url = {https://arxiv.org/abs/2605.30794} } ``` APA: > Kou, Q., Shi, X., Li, Y., Qiu, X., Wang, X., Zhou, H., & Cao, D. (2026). > *MechVQA: Benchmarking and enhancing multimodal LLMs on comprehensive > mechanical drawing understanding*. arXiv. https://arxiv.org/abs/2605.30794 ## Contact - ModelScope dataset: [xiaofengalg/MechVQA](https://modelscope.cn/datasets/xiaofengalg/MechVQA) - GitHub issues: [xiaofengShi/MechVQA](https://github.com/xiaofengShi/MechVQA/issues)