finehardimages

# FineHARD ModelScope Transfer Package This package is for transferring the prepared FineHARD image data through ModelScope. Contents: - `images_shards/coyo_image_*.tar`: image shards. Each tar contains only JPG files under paths like `grit-12m/coyo_image_6/00068/000689772.jpg`. - `logs/`: packaging logs. - `SHA256SUMS`: checksums generated after packaging finishes. Not included: - all JSON files. - `grit-12m/**/*.json`: img2dataset metadata. - `FineHARD/json_files`: FineHARD annotations. - `down-grit-12m`: intermediate download output. - original parquet and zip files. - `url2key_jsons`: conversion-only mapping files. Annotation path note: FineHARD annotation items use paths like: ```text grit-20m/data-12m/coyo_image_6/00068/000689772.jpg ``` The transferred image tar files extract to: ```text grit-12m/coyo_image_6/00068/000689772.jpg ``` When loading annotations, replace this prefix: ```text grit-20m/data-12m/ -> grit-12m/ ``` Example extraction on target machine: ```bash mkdir -p /data/FineHARD cd /data/FineHARD for f in images_shards/coyo_image_*.tar; do tar -xf "$f"; done ```