ABot-World-Explorer-4D

# ABot World Explorer 4D [](https://abot-world.amap.com) [](https://reactor.inc/abot-world) [](https://amap-cvlab.github.io/ABot-World/) [](https://arxiv.org/abs/2607.19191) [](https://github.com/amap-cvlab/ABot-World) [](https://huggingface.co/acvlab/ABot-World-0-5B-LF) [](https://huggingface.co/spaces/acvlab/abot-world-interactive) [](https://huggingface.co/datasets/acvlab/ABot-World-Explorer-500h) [](https://huggingface.co/datasets/acvlab/ABot-World-Explorer-4D) [](https://huggingface.co/papers/2607.19191) [](https://modelscope.cn/models/amap_cvlab/ABot-World-0-5B-LF) [](https://modelscope.cn/studios/amap_cvlab/abot-world-0) [](https://modelscope.cn/datasets/amap_cvlab/ABot-World-Explorer-500h) [](https://modelscope.cn/datasets/amap_cvlab/ABot-World-Explorer-4D)  ABot World Explorer 4D is a depth-enabled sample of the action-conditioned video data infrastructure described in [ABot-World-0](https://arxiv.org/abs/2607.19191). Its source manifest references 20 episodes and 181,561 EXR depth objects; the release preserves their bytes. ## Dataset facts | Item | Value | |---|---:| | Episodes | 20 | | Base source objects | 120 | | EXR depth objects | 181,561 | | Total source objects | 181,681 | | Semantic splits | None | | Depth representation | Absolute metric | | Depth unit / scale | `m` / `1.0` | | License | Apache-2.0 | Exact duration and timestamp-level RGB/action/pose/depth alignment are not claimed by this payload-only publication. Twenty episodes must not be confused with twenty depth frames. ## Layout and fidelity ```text meta/abot-world-explorer-poster.png metadata.jsonl data/<prefix>/<sample_id>/video.mp4 data/<prefix>/<sample_id>/annotations.tar data/<prefix>/<sample_id>/depth/shard-*.tar LICENSE README.md ``` `sample_id` is an anonymous, dataset-namespaced HMAC identifier. Source keys and OSS locations are not released. ## Preview The ModelScope **Preview** tab is backed by `metadata.jsonl`. It covers all 20 episodes and contains only each anonymous `sample_id` plus authenticated, commit-pinned links to the existing `video.mp4` and `annotations.tar` objects. It does not copy, rewrite, summarize, or otherwise modify captions, actions, or media payloads. Depth TAR archives remain ordinary repository payload and are intentionally not decoded by the table preview. ## Data file formats ### `video.mp4` and `annotations.tar` `video.mp4` preserves the source MP4 bytes without release-time transcoding. `annotations.tar` is an uncompressed POSIX USTAR archive containing `action.json`, `caption.json`, and the three files under `sparse/0/`. Treat these five members as one sample-level annotation package and validate their paths and hashes before extracting them. ### `action.json` `action.json` is a UTF-8 JSON object containing sequence metadata and a `frames` array. The released records use the following structure; the available controls and dataset-native numeric values may vary by sequence. | Field | JSON type | Description | |---|---|---| | `control_scheme` | string | Name of the source control convention. | | `original_fps`, `fps` | number | Source and sampled frame rates. | | `sample_stride` | integer | Sampling stride relative to the source sequence. | | `start_frame_index`, `end_frame_index`, `total_frames` | integer | Sequence/frame-range metadata. | | `thresholds` | object | Schema-specific control thresholds; it may be empty. | | `frames` | array of objects | One ordered control record per sampled frame. | Each `frames[]` object contains `frame_id` (string), `timestamp` (number), a `keys` mapping from control names to booleans, and four length-3 numeric vectors: `delta_translation_cam`, `delta_translation_cam_smooth`, `delta_euler_deg`, and `delta_euler_deg_smooth`. Rotation deltas are named in degrees; do not infer the translation or timestamp units, thresholds, or exact cross-modal alignment when this card marks them as unknown. Some source variants add per-frame translation/rotation threshold and release fields. Readers should accept additional fields and must not assume a fixed set of keys. ### `caption.json` `caption.json` is a UTF-8 JSON object with `perspective`, `scene_static`, and `narrative` strings plus a `dense_temporal` array. `dense_temporal` may be empty; all current audit canaries contain an empty array, so its item schema is not claimed here. Consumers must not assume at least one segment. String values are preserved. The release scan found no complete-token private source path identifier requiring replacement (`redacted_units=0`), and JSON keys are never rewritten. ### COLMAP pose files The three files under `sparse/0/` form one [COLMAP text sparse model](https://colmap.github.io/format.html#text-format) and must be interpreted together. Lines beginning with `#` are comments. - `cameras.txt`: one camera per line as `CAMERA_ID MODEL WIDTH HEIGHT PARAMS[]`. The parameter list depends on the [camera model](https://colmap.github.io/cameras.html). - `images.txt`: two lines per image. The first is `IMAGE_ID QW QX QY QZ TX TY TZ CAMERA_ID NAME`; the second is a repeated list of `(X, Y, POINT3D_ID)` observations and may be empty. The pose maps world coordinates to camera coordinates using a Hamilton quaternion. COLMAP camera axes are +X right, +Y down, +Z forward, and the camera center is `-R^T T`. - `points3D.txt`: one sparse point per line as `POINT3D_ID X Y Z R G B ERROR TRACK[]`; each track item is `(IMAGE_ID, POINT2D_IDX)`. A present but empty file is valid and means that the model contains no sparse 3D points. IDs are not guaranteed to be contiguous. See the official specifications for [`cameras.txt`](https://colmap.github.io/format.html#cameras-txt), [`images.txt`](https://colmap.github.io/format.html#images-txt), and [`points3D.txt`](https://colmap.github.io/format.html#points3d-txt). For all 20 4D episodes, the audited model is pose-only: `cameras.txt` contains one 1920 x 1080 `PINHOLE` camera, every `images.txt` observation line is empty, and `points3D.txt` has no point records. Image names run from `frame_000001.jpg` in order, and the image/depth counts match within each episode. This is a COLMAP-compatible camera-pose model, not a sparse point cloud; matching counts alone do not prove timestamp-level cross-modal alignment. ### OpenEXR depth files and organization Each `depth/<frame_key>.exr` member is an [OpenEXR binary image](https://openexr.com/en/latest/TechnicalIntroduction.html). OpenEXR stores named channels as `HALF` (16-bit floating point), `FLOAT` (32-bit floating point), or `UINT` (32-bit unsigned integer) samples; the header's `dataWindow` defines the valid pixel region. A file may contain one or more parts and each part has its own header and channels. See the official [channel description](https://openexr.com/en/latest/TechnicalIntroduction.html#image-channels-and-sampling-rates) and [file layout](https://openexr.com/en/latest/OpenEXRFileLayout.html#high-level-layout). A stratified audit of the first, middle, and last EXR from each of the 20 episodes found one consistent canary profile: | Property | Audited canary value | |---|---| | Version / layout | OpenEXR v2, single-part scanline image | | Resolution / windows | 1920 x 1080; matching data and display windows `(0, 0)-(1919, 1079)` | | Channels | `B`, `G`, `R`; each `FLOAT`, sampling 1 x 1; no `Z` or alpha channel | | Compression / line order | `PXR24` / increasing Y | Six decoded pixel canaries had identical values in `B`, `G`, and `R`, indicating that these files carry the same depth plane in all three channels. Readers may use `R` only after validating channel equality for the files they consume. `PXR24` reduces `FLOAT` data to 24-bit precision before compression and is therefore lossy relative to the original 32-bit values; see the official [compression specification](https://openexr.com/en/latest/TechnicalIntroduction.html#data-compression). The release nevertheless preserves every existing EXR byte: it does not perform additional normalization, clipping, quantization, conversion, or re-encoding. Absolute metric depth in metres with `scale_to_m=1.0` is the dataset semantic contract supplied by the authors; it is not encoded as custom EXR header metadata. Whether the measurement is camera-z depth or Euclidean range, and the meaning of zero or another invalid sentinel, remain explicitly unknown. EXRs are grouped per sample in deterministic, uncompressed `data/<prefix>/<sample_id>/depth/shard-*.tar` archives. Members are ordered by six-digit frame key (`depth/000001.exr`, `depth/000002.exr`, ...), a member is never split, and a shard never spans samples. The published payload does not include a separate frame/member Parquet index: inspect each TAR member list in archive order and use its six-digit frame key. Do not infer timestamp-level alignment or global temporal order from shard names alone. To inspect the original parts and channels with the official Python bindings: ```python import OpenEXR import numpy as np with OpenEXR.File("frame.exr", separate_channels=True) as exr: channels = exr.channels() depth_m = channels["R"].pixels if not all(np.array_equal(depth_m, channels[name].pixels) for name in "GB"): raise ValueError("B/G/R depth channels differ") ``` ## Selective download Use a full commit ID when reproducibility matters. The Preview index provides the anonymous `sample_id`; the corresponding payload paths are derived from that ID without consulting source keys or OSS locations. ```python from modelscope_hub.api import HubApi REPO_ID = "amap_cvlab/ABot-World-Explorer-4D" REVISION = "<full ModelScope commit ID>" api = HubApi() sample_id = "<sample_id from metadata.jsonl>" prefix = sample_id[:2] snapshot = api.download_repo( repo_id=REPO_ID, repo_type="dataset", revision=REVISION, allow_patterns=[ "LICENSE", "README.md", "metadata.jsonl", f"data/{prefix}/{sample_id}/video.mp4", f"data/{prefix}/{sample_id}/annotations.tar", f"data/{prefix}/{sample_id}/depth/*.tar", ], ) print(snapshot) ``` Absolute source timestamps and a frame/member index are not exposed. Stream each selected depth shard and validate exact regular-file member names before reading members. Plain `git clone` is not recommended for large payload download. ## Intended use and limitations Intended for metric-depth-aware world-model, multimodal alignment, action-conditioned video, controllable generation, and agent-learning research. It is not a calibrated sensor benchmark or a guarantee of physical/causal correctness. Pose units, axis conventions, invalid-depth sentinels, and exact cross-modal joins are defined only where the released audit supports them. ## Additional data access For additional data access or customized dataset requirements, please contact the AMAP Data Department directly at [[e-mail]](mailto:[e-mail]) ## Citation ```bibtex @misc{jiang2026abotworld0infiniteinteractiveworld, title={ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU}, author={Fan Jiang and Zhaoxu Sun and Mengchao Wang and Ziyu Zhu and Chiyu Wang and Yunpeng Zhang and Wenlin Liu and Yun Wang and Xue Zheng and Rui Sun and Junfeng Ni and Hongyu Pan and Zhongxu Sun and Fei Yu and Zengye Ge and Mengmeng Du and Nianfei Fan and Mingchao Sun and Yu Liu and Yongchang and Yanqing Zhu and Jiahang Wang and Ning Ying and Yuze Xuan and Di Yang and Zhicheng Liu and Zhe Gao and Tingbing Xu and Jiacheng Sui and Wenjin Yang and Junnan Lai and Shufeng Liu and Yuan Liu and Zheng Zhou and Yingliang Peng and Dawei Cao and Kaifeng Sheng and Yuxiang Cai and Fei Lu and Mu Xu and Ning Guo}, year={2026}, eprint={2607.19191}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2607.19191}, } ```