SenseNova-U1-Training-Sample

# SenseNova-U1 Training Sample A tiny illustrative dataset accompanying the [**SenseNova-U1**](https://github.com/OpenSenseNova/SenseNova-U1) training code. It exists **only to smoke-test the training pipeline end-to-end** — every task type defined in `training/configs/` is represented by a handful of samples so the dataloader / packing / loss code paths all execute. It is **not** a usable training corpus. For real training, prepare your own jsonl annotations in the same schema and point `mm_data_path` at your own meta JSON. See the [upstream training README](https://github.com/OpenSenseNova/SenseNova-U1/blob/main/training/README.md#data) for the full data-prep contract. --- ## Quick start ```bash # Hugging Face CLI (recommended) huggingface-cli download \ SenseNova/SenseNova-U1-Training-Sample \ --repo-type dataset \ --local-dir SenseNova-U1/training/data ``` ```python # Or via the Python API from huggingface_hub import snapshot_download snapshot_download( repo_id="SenseNova/SenseNova-U1-Training-Sample", repo_type="dataset", local_dir="SenseNova-U1/training/data", ) ``` After download, the layout is exactly what the shipped configs and `sample_data_meta.json` expect — no path rewriting needed. --- ## Layout ``` . ├── sample/ # jsonl annotations (one sample per line) │ ├── sample_data_meta.json # top-level meta consumed by mm_data_path │ ├── t2i__curated_t2i_ge_512_lt_1024.jsonl # 16 samples │ ├── it2i__dresscode_dress_gt_22k.jsonl # 16 samples │ ├── interleave__interleave_v1.6.1_it2i_merged_shuffle_filtered.jsonl # 16 samples │ ├── interleave_gen__ppt_pipeline_forward_nojson_zh_batch22.jsonl # 16 samples │ ├── und__infographic_sft_20260402.jsonl # 16 samples │ ├── text__math__Maths-College.jsonl # 16 samples │ └── video__openvid_1m.jsonl # 2 samples ├── images/ │ ├── t2i/ # text-to-image targets │ ├── it2i/ # image-editing source/target pairs (DressCode-style) │ ├── mit2i/ # multi-input image editing │ ├── interleave_gen/ # interleaved text+image generation (slide pipeline) │ └── und/ # understanding (charts, OCR, network diagrams) └── videos/ # short video clips for video understanding ``` Roughly **~250 files / 680 MB** total. --- ## Task coverage Every task type the training framework supports is exercised by at least one jsonl in `sample/`: | Task key (in meta JSON) | `task` field | Example jsonl | Notes | |------------------------------------------|---------------------|--------------------------------------------------------------|---------------------------------------------| | `t2i__curated_t2i_ge_512_lt_1024` | `t2i` | `t2i__curated_t2i_ge_512_lt_1024.jsonl` | text → image, 512–1024 resolution bucket | | `it2i__dresscode_dress_gt_22k` | `it2i` | `it2i__dresscode_dress_gt_22k.jsonl` | single-image editing (DressCode dresses) | | `interleave__...it2i_merged_shuffle...` | `it2i` (multi-img) | `interleave__interleave_v1.6.1_it2i_merged_shuffle_filtered.jsonl` | multi-input image editing | | `interleave_gen__ppt_pipeline_forward...`| `interleave_gen` | `interleave_gen__ppt_pipeline_forward_nojson_zh_batch22.jsonl` | interleaved text + image generation | | `und__infographic_sft_20260402` | `image`(multimodal und) | `und__infographic_sft_20260402.jsonl` | chart / OCR / VQA understanding | | `text__math__Maths-College` | `text` | `text__math__Maths-College.jsonl` | text-only LLM continuation | | `video__openvid_1m` | `video` | `video__openvid_1m.jsonl` | video understanding | --- ## Schema ### `sample_data_meta.json` Top-level object; each key is a dataset name, each value points the loader at the annotation jsonl and the image/video root: ```json { "<dataset_name>": { "root": "data/images/<subdir>", // resolved relative to mm_data_path's parent "annotation": "data/sample/<file>.jsonl", "repeat_time": 10000, // upsample factor for mixing "length": 16, // number of samples in this jsonl "task": "t2i | it2i | interleave_gen | text | video", "data_type": "t2i_distillation_hf | edit_1turn_1image_hf | edit_1turn_mimage_hf | ...", "language": "en | zh | en&zh", "data_quality":"high | ...", "bucket_res": "ge_512_lt_1024 | ..." } } ``` ### Per-sample jsonl All annotations follow the same conversational shape used by `sensenovavl/data/dataset.py::preprocess_sensenovalm_v3_mm_chat`: ```json { "id": "...", "image": "subdir/file.jpg", // or [path, path, ...] for multi-image "conversations": [ {"from": "human", "value": "<image>\n<instruction>"}, {"from": "gpt", "value": "<image>"} // "<image>" placeholder for image outputs ], "width": [w1, w2, ...], "height": [h1, h2, ...] } ``` Task-specific extras (e.g., `cloth_image`, `categories`, `video`, system prompts for interleaved generation) are passed through verbatim. --- ## Licensing & provenance - **Annotations & layout** — released under **Apache-2.0**, matching the parent training code repository. - **Image / video content** — drawn from publicly-available sources used to illustrate the loader contract; each `image*` path retains the upstream attribution implied by its subdir name (`dresscode`, `openvid`, `curated`, `Maths-College`, etc.). If you intend to redistribute or use this content beyond pipeline smoke-testing, consult the original source's license. This is a **sample**: it is sized for "does the training loop start?" not for "can I learn anything?". Do not benchmark on it. --- ## Citation If you use the SenseNova-U1 training code in your work, please cite: ```bibtex @misc{sensenova2026neounify, title = {NEO-unify: Building Native Multimodal Unified Models End to End}, author = {SenseNova}, journal = {Hugging Face blog}, url = {https://huggingface.co/blog/sensenova/neo-unify}, year = {2026} } @article{sensenova2026sensenovau1, title = {SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture}, author = {Diao, Haiwen and Wu, Penghao and Deng, Hanming and Wang, Jiahao and Bai, Shihao and Wu, Silei and Fan, Weichen and Ye, Wenjie and Tong, Wenwen and Fan, Xiangyu and others}, journal = {arXiv preprint arXiv:2605.12500}, year = {2026} } ```