Penguin-Recap-I

# Penguin-Recap-I Penguin-Recap-I publishes recap metadata only. The repository does not contain image binaries. ## Included subsets | subset | collection | local source roots | expected records | | --- | --- | --- | ---: | | `datacomp_coyo_penguin` | DataComp + COYO Penguin recap | `datamultimodal/IMAGE/datacomp_1b, datamultimodal/IMAGE/coyo_700m` | 57,618,155 | | `sa1b_penguin` | SA-1B Penguin recap | `datamultimodal/IMAGE/SA-1B` | 9,254,501 | | `openimages_penguin` | OpenImages Penguin recap | `datamultimodal/IMAGE/openimages` | 1,709,646 | Expected total records: **68,582,302** ## Media access policy - `openimages_penguin`: keeps the relative image path and filename only. Users should obtain the image files from the official OpenImages release. - `sa1b_penguin`: keeps the relative image path and filename only. Users should obtain the image files from the official SA-1B release. - `datacomp_coyo_penguin`: stores the original image URL extracted from the sidecar JSON file next to each local image. ## Image download resources - OpenDataLab OpenImagesV6: https://opendatalab.com/OpenDataLab/OpenImagesV6/tree/main/raw - OpenDataLab SA-1B: https://opendatalab.com/OpenDataLab/SA-1B/tree/main/raw - Official Segment Anything release: https://ai.meta.com/datasets/segment-anything/ - Official OpenImages index: https://storage.googleapis.com/openimages/web/index.html For `openimages_penguin` and `sa1b_penguin`, use the exported `image_name`, `image_names`, `image`, and `image_refs` fields to map each row back to the corresponding original image file. For `datacomp_coyo_penguin`, each JSON entry includes `url` / `urls`, which can be used to download the image directly. ## Repository layout - `data/<subset>/*.jsonl.gz`: metadata shards used by the dataset viewer - `manifest/files.jsonl`: shard-level example counts and byte estimates - `manifest/skipped.jsonl`: skipped samples and the reason - `manifest/build_stats.json`: end-of-run summary ## Row schema Each row contains the normalized metadata below: - `sample_key`: stable public sample id - `subset`: Hugging Face subset/config id - `source`: source id - `original_id`: original annotation id, normalized to string - `image`: first relative image reference from the annotation - `image_refs`: full list of relative image references - `image_name`: first image basename - `url`: first URL for DataComp/COYO rows, otherwise `null` - `conversations`: full conversation list from the annotation - `prompt` / `response`: first human and first gpt turns - `annotation_metadata`: remaining annotation fields that were not promoted ## Loading ```python from datasets import load_dataset datacomp = load_dataset( "tencent/Penguin-Recap-I", "datacomp_coyo_penguin", split="train", streaming=True, ) sample = next(iter(datacomp)) print(sample["url"]) sa1b = load_dataset( "tencent/Penguin-Recap-I", "sa1b_penguin", split="train", streaming=True, ) sample = next(iter(sa1b)) print(sample["image_name"]) openimages = load_dataset( "tencent/Penguin-Recap-I", "openimages_penguin", split="train", streaming=True, ) sample = next(iter(openimages)) print(sample["conversations"][0]["value"]) ``` ## Citation ```bibtex @article{Penguin-VL, title={Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders}, author={Boqiang Zhang and Lei Ke and Ruihan Yang and Qi Gao and Tianyuan Qu and Rossell Chen and Dong Yu and Leoweiliang}, journal={arXiv preprint arXiv:2603.06569}, year={2026} } ```