meamaturinlove221/vggt-human-prior-builder
收藏资源简介：
--- pretty_name: "VGGT Human Prior Builder" language: - en license: mit tags: - 3d - image - vggt - smpl-x - human-prior - 4k4d - zju-mocap - dataset-builder - preprocessing - reproducibility task_categories: - image-to-3d size_categories: - n<1K --- # VGGT Human Prior Builder **Conclusion first:** this repository is a **release-safe dataset builder**, not a processed-data release. It publishes the preprocessing structure used to adapt authorized multi-view human datasets to a VGGT human-prior workflow, while keeping 4K4D, ZJU-MoCap, DNA-Rendering, SMPL-X body models, RGB frames, masks, camera parameters, SMPL/SMPL-X parameters, derived depth maps, point maps, prior maps, and processed training packages outside the public repository.  ## Route position This repo sits between locally authorized human-centric multi-view data and a VGGT-style geometry model. Its job is to describe and scaffold a conversion layer that can produce VGGT-readable human-prior inputs and optional supervision targets **inside a user's local environment**. ```text Authorized local data │ ├─ 4K4D / DNA-style local assets ├─ ZJU-MoCap-style local assets ├─ RGB / masks / cameras └─ optional body annotations │ ▼ VGGT Human Prior Builder │ ├─ validate source tree ├─ build image-space prior maps ├─ optionally render local depth / point targets ├─ export VGGT-compatible schema └─ write source manifest and audit records │ ▼ Private local outputs ├─ inputs.npz ├─ targets.npz ├─ manifest.json └─ visual_audit/ ``` ## Why this repo exists Vanilla VGGT is a general feed-forward geometry model. In human-heavy scenes, especially when the final evidence must be a **human-main full-scene RGB point cloud with part of the environment preserved**, raw per-view geometry can be fragile around body topology, limbs, hands, feet, and occluded regions. The builder therefore records a conservative route: - use human masks, keypoints, body-part hints, or locally rendered SMPL-X supervision as **inputs / local targets**; - keep the VGGT model as a scene-geometry model rather than turning the public repo into a body-model data mirror; - separate data preprocessing, model supervision, and visual evidence gates; - make provenance, failure cases, and redistribution boundaries explicit. ## What changed compared with the original dataset usage ```text Original 4K4D / ZJU-MoCap style usage │ ├─ dataset-specific formats ├─ NeRF / 4D reconstruction / NVS-oriented pipelines └─ access controlled by upstream dataset agreements This builder │ ├─ keeps upstream data local and private ├─ defines a VGGT-compatible prior schema ├─ separates prior_maps from depth / point supervision ├─ records teacher / student boundaries ├─ records metric / visual / advisor gates └─ publishes scripts, schemas, configs, and synthetic examples only ``` ## Current public scope | Area | Status in this public repo | Redistribution rule | |---|---|---| | 4K4D / DNA-style route | scaffolded builder interface and schema | users must obtain data from official sources | | ZJU-MoCap-style route | scaffolded builder interface and schema | users must obtain data from official sources | | synthetic toy sample | included for schema smoke testing | safe to redistribute | | real RGB / masks / cameras | not included | do not upload | | SMPL / SMPL-X parameters or body models | not included | do not upload | | derived depth / point / prior maps from restricted data | not included | keep local unless explicitly permitted | | VGGT adapter training outputs | not included | audit separately before release | The public scripts are intentionally conservative. They provide the repository boundary, expected schema, manifest pattern, and safety checks. Dataset-specific parsing should be completed in a private worktree after the user has obtained the corresponding datasets and agreed to their licenses. ## Included files - `README.md` — this dataset card and route document. - `configs/` — conservative config templates for 4K4D-style and ZJU-MoCap-style local processing. - `scripts/` — local preprocessing entry points and synthetic toy generation. - `src/vggt_human_prior_builder/` — schema helpers and minimal prior-map utilities. - `manifests/` — expected file trees, output schema, and checksum templates. - `tools/audit_no_restricted_data.py` — release safety check that blocks restricted-looking files. - `examples/synthetic_toy_case/` — a tiny synthetic sample for testing the schema only. ## Not included This repository intentionally does **not** include: - 4K4D, DNA-Rendering, NHR, ENeRF-Outdoor, Mobile-Stage, or ZJU-MoCap data; - SMPL, SMPL-X, or other body-model assets; - real RGB frames, masks, camera files, or human annotations; - derived `.npz`, `.pkl`, `.ply`, `.npy`, depth maps, point maps, or prior maps from restricted datasets; - any file that would let this repo function as a mirror of an upstream human dataset. ## Output schema A local VGGT human-prior sample should follow this shape convention: | Key | Shape | Meaning | |---|---:|---| | `images` | `[V, H, W, 3]` | RGB inputs, float32 `[0,1]` or uint8 | | `prior_maps` | `[V, C, H, W]` | image-space human prior channels | | `prior_depths` | `[V, H, W]` | optional local human-region depth target | | `prior_points` | `[V, H, W, 3]` | optional local camera/world-space point target | | `prior_mask` | `[V, H, W]` | valid human-prior supervision mask | | `camera_intrinsics` | `[V, 3, 3]` | camera intrinsics | | `camera_extrinsics` | `[V, 4, 4]` | documented world-to-camera or camera-to-world convention | | `sample_id` | scalar string | local sample identifier | | `schema_version` | scalar string | schema version | See `manifests/output_schema.md` for the longer version. ## Minimal synthetic example The bundled example is synthetic and only checks that the schema is readable. It is not a benchmark, does not contain real identities, and should not be used as evidence of model quality. ```python from vggt_human_prior_builder.schema import load_vggt_prior_npz sample = load_vggt_prior_npz("examples/synthetic_toy_case/sample_vggt_human_prior.npz") print(sample["images"].shape) print(sample["prior_maps"].shape) ``` ## Local-only preprocessing entry points After installing the package and obtaining the corresponding datasets from official sources, users can run the local builders in their own private environment: ```bash python scripts/prepare_4k4d_vggt_prior.py \ --source-root /path/to/authorized/4k4d_or_dna_case \ --out-root /path/to/private_outputs \ --config configs/4k4d_prior.yaml ``` ```bash python scripts/prepare_zju_mocap_vggt_prior.py \ --source-root /path/to/authorized/zju_mocap_case \ --out-root /path/to/private_outputs \ --config configs/zju_mocap_prior.yaml ``` Generated outputs should remain private unless every upstream license explicitly permits redistribution of the exact derived artifacts. ## Evidence gates This repository follows a fail-closed interpretation of the VGGT + human-prior route: - **Teacher output is not student-model success.** Locally rendered SMPL-X or dataset-derived geometry can be used as supervision or reference, but it is not final VGGT evidence. - **Metric pass is not visual pass.** Loss values or projection metrics do not prove that the 3D point cloud is visually correct. - **Projection overlay is not 3D morphology.** A good 2D overlay can still hide a broken 3D body shape. - **Isolated human scatter is diagnostic only.** The target visual evidence is a human-main full-scene RGB point cloud with some environment context preserved. - **Controls matter.** If random, smoothed, no-SMPL, or teacher-only controls look similar to the proposed output, the result should not be promoted. ## Release safety checklist Before making the repository public or uploading a new revision: 1. Run `tools/audit_no_restricted_data.py` on the repository root. 2. Confirm that the file list does not contain real images, masks, cameras, SMPL/SMPL-X parameters, body models, point clouds, or generated training packages. 3. Confirm that examples are synthetic or independently redistributable. 4. Keep local generated artifacts outside this repository. 5. Document any new public artifact in the manifest before upload. ## Citation and upstream data Users must cite and follow the license or agreement of every upstream dataset and model used locally. This repository does not grant rights to 4K4D, ZJU-MoCap, DNA-Rendering, SMPL-X, or any other upstream assets. ```bibtex @misc{vggt_human_prior_builder, title = {VGGT Human Prior Builder}, author = {Xu, Jianghang}, year = {2026}, note = {Release-safe dataset-builder and preprocessing recipe. No upstream human dataset redistributed.} } ``` ## License The code and documentation in this repository are released under the MIT License. This license applies only to the files in this repository and does not apply to upstream datasets, body-model assets, or local generated outputs.
VGGT Human Prior Builder is a release-safe preprocessing tool and dataset builder designed to convert authorized multi-view human datasets into a VGGT-compatible format for human-prior workflows. It provides scripts, configuration templates, schema definitions, and synthetic examples, but does not include actual dataset content. Key functions include validating source data tree structures, building image-space prior maps, optionally rendering local depth/point targets, exporting VGGT-compatible schemas, and generating manifests and audit records. The tool does not include actual data from upstream datasets such as 4K4D, ZJU-MoCap, or DNA-Rendering, nor does it include SMPL-X body models, RGB frames, masks, camera parameters, or derived data. Users must obtain authorized data from official sources and use this tool in their local environment for preprocessing, with outputs remaining private.
