cad-edit-v2

# cadedit-v2 Standalone ECBA edit v2 dataset package for build123d CAD edit SFT and OPSD base construction. This folder is intended to be used as a working root. Run commands from this directory so that relative image and source paths inside the JSONL files resolve correctly. ## Contents - `data/ecba_edit_v2_real_remove_enabled/` - Source ECBA edit v2 JSONL: `all.jsonl`, `train.jsonl`, `val.jsonl`, `test.jsonl`. - Edit-stage audit summaries and template metadata. - `p1_sources/*.py`, referenced by JSONL metadata. - `data/ecba_edit_v2_real_remove_enabled_ms_swift/` - ms-swift SFT data in `messages/images` format. - `train.jsonl`, `val.jsonl`, `test.jsonl`, `stats.json`. - `data/ecba_edit_v2_real_remove_enabled_opsd/` - OPSD base records. - `opsd_base_train.jsonl`, stats, and invalid file. - `outputs/build123d_three_view_dataset/v1_10k_250seq_step_valid_resampled/drawings/` - PNG three-view drawings referenced by the SFT and OPSD JSONL files. - SVG/DXF/STEP artifacts are intentionally not included. - `outputs/build123d_three_view_dataset/v1_10k_250seq_step_valid_resampled/sources/` - P0 build123d source files referenced by JSONL metadata. - `scripts/` - ECBA edit generation, validation, export, ms-swift conversion, OPSD base construction, and data checks. ## Dataset Stats - ECBA edit records: `28,328` - SFT train/val/test: `22,785 / 2,851 / 2,692` - OPSD base train records: `22,785` - Image layout: `1` combined front/top/left PNG per record, with `1` `<image>` token. - PNG drawing files: `10,000` - P0 source files: `10,000` - P1 source files: `30,000` - Package files: `50,042` - Package size: about `1.63 GiB` All post-move checks passed: - SFT `train/val/test`: `invalid_records = 0` - OPSD base train: `invalid_records = 0` - Image token mismatches: `0` - Missing image paths: `0` - Prompt code leakage: `0` - Missing metadata P0/P1 source paths: `0` - Bad image count records: `0` - Bad image token count records: `0` ## Check Commands ```powershell python scripts\check_ms_swift_ecba_data.py --input_jsonl data\ecba_edit_v2_real_remove_enabled_ms_swift\train.jsonl --mode sft --preview 3 python scripts\check_ms_swift_ecba_data.py --input_jsonl data\ecba_edit_v2_real_remove_enabled_ms_swift\val.jsonl --mode sft --preview 3 python scripts\check_ms_swift_ecba_data.py --input_jsonl data\ecba_edit_v2_real_remove_enabled_ms_swift\test.jsonl --mode sft --preview 3 python scripts\check_ms_swift_ecba_data.py --input_jsonl data\ecba_edit_v2_real_remove_enabled_opsd\opsd_base_train.jsonl --mode opsd --preview 3 ``` ## Regenerate ms-swift SFT ```powershell python scripts\convert_ecba_edit_to_ms_swift_sft.py ` --input_jsonl data\ecba_edit_v2_real_remove_enabled\all.jsonl ` --output_dir data\ecba_edit_v2_real_remove_enabled_ms_swift ` --format messages ` --collapse_duplicate_images ` --overwrite ``` The packaged default is already collapsed: one combined front/top/left PNG and one `<image>` token per record. The `--collapse_duplicate_images` flag is kept in the command so regenerated outputs stay in that format even if the input JSONL was restored from an older three-token copy. ## Regenerate OPSD Base ```powershell python scripts\build_ecba_edit_opsd_base.py ` --input_jsonl data\ecba_edit_v2_real_remove_enabled\train.jsonl ` --output_jsonl data\ecba_edit_v2_real_remove_enabled_opsd\opsd_base_train.jsonl ` --collapse_duplicate_images ` --overwrite ``` This creates OPSD base records only. It does not create rollouts, gates, or teacher judgments. ## Format Notes SFT row shape: ```json { "id": "...", "images": ["..."], "messages": [ {"role": "user", "content": "<image>..."}, {"role": "assistant", "content": "[SRC]...[/SRC]\n[EDT]...[/EDT]"} ], "metadata": { "source_id": "...", "family": "...", "edit_type": "...", "subtype": "...", "transition_key": "...", "validation": {"ok": true, "failure_reason": ""} } } ``` OPSD base row shape: ```json { "id": "...", "images": ["..."], "instruction": "...", "student_prompt": "...", "teacher_prompt": "... [GT_SRC]...[/GT_SRC] ... [GT_EDT]...[/GT_EDT]", "gt_src": "...", "gt_edt": "...", "metadata": {...} } ```