varora/HIT

--- license: other license_name: max-planck license_link: https://hit.is.tue.mpg.de/license.html configs: - config_name: male data_files: - split: train path: male/train/*.gz - split: val path: male/val/*.gz - split: test path: male/test/*.gz - config_name: female data_files: - split: train path: female/train/*.gz - split: val path: female/val/*.gz - split: test path: female/test/*.gz tags: - SMPL - Tissues - Medical - Biomechanics - Human-Twins - Digital-Twins - Mesh - Bones - 3D - Classification - Occupancy - MRI - Segmentation --- ## Dataset Description - **Homepage:** [https://hit.is.tue.mpg.de/](https://hit.is.tue.mpg.de/) - **Repository:** [https://github.com/MarilynKeller/HIT](https://github.com/MarilynKeller/HIT) - **Paper:** [Coming Soon](Coming Soon) - **Point of Contact:** [Marilyn Keller]([e-mail]), [Sergi Pujades]([e-mail]), [Vaibhav Arora]([e-mail]) ### Dataset Summary The HIT dataset is a structured dataset of paired observations of body's inner tissues and the body surface. More concretely, it is a dataset of paired full-body volumetric segmented (bones, lean, and adipose tissue) MRI scans and SMPL meshes capturing the body surface shape for male (N=157) and female (N=241) subjects respectively. This is relevant for medicine, sports science, biomechanics, and computer graphics as it can ease the creation of personalized anatomic digital twins that model our bones, lean, and adipose tissue. Dataset acquistion: We work with scans acquired with a 1.5 T scanner (Magnetom Sonata, Siemens Healthcare) following a standardized protocol for whole body adipose tissue topography mapping. All subjects gave prior informed written consent and the study was approved by the local ethics board. Each scan has around 110 slices, slightly varying depending on the height of the subject. The slice resolution is 256 × 192, with an approximate voxel size of 2 × 2 × 10 mm. These slices are segmented into bones, lean, and adipose tissue by leveraging initial automatic segmentations and manual annotations to train and refine nnUnets with the help of human supervision. For each subject, we then fit the SMPL body mesh to the surface of the segmented MRI in a manner that captures the flattened shape of subjects in their lying positions on belly in the scanner (refer to Sec 3.2 in main paper for further details). Therefore for each subject, we provide the MRI segmented array and the SMPL mesh faces and vertices (in addition to the SMPL parameters). <img src="extras/hit_dataset.png" alt="alt text" width="300"> ### Supported Tasks and Leaderboards HIT fosters a new direction and therefore there aren't any exisiting Benchmarks. We encourage the use of the dataset to open up new tasks and research directions. ### Languages [N/A] ## Usage ### Quick use ```angular2html pip install datasets ``` ```angular2html from datasets import load_dataset # name in ['male', 'female'] # split in ['train', 'validation', 'test'] male_train = load_dataset("varora/hit", name='male', split='train') print(male_train.__len__()) print(next(iter(male_train))) ``` ### Visualize data Download `vis_hit_sample.py` from the repo or `git clone https://huggingface.co/datasets/varora/HIT` ```angular2html pip install datasets, open3d, pyvista ``` #### Visualize mesh and pointcloud ```angular2html python vis_hit_sample.py --gender male --split test --idx 5 --show_skin ``` <img src="extras/vis_script_output.png" alt="alt text" width="300"> #### Visualize tissue slice ```angular2html python vis_hit_sample.py --gender male --split test --idx 5 --show_tissue ``` <img src="extras/tissue_slice_frontal.png" alt="alt text" width="300"> ## Dataset Structure The dataset is structured as follows: ``` |- male |- train |- 001.gz |- 002.gz |- … |- 00X.gz |- val |- |- … |- 00X.gz |- test |- |- … |- 00X.gz |- female |- train |- 001.gz |- 002.gz |- … |- 00X.gz |- val |- |- … |- 00X.gz |- test |- |- … |- 00X.gz ``` ### Data Instances Each data instance (male/train/001.gz for example) contains the following: ``` { 'gender': str ['male', 'female'], 'subject_ID': str 'mri_seg': numpy.ndarray (None, 192, 256), 'mri_labels': dict {'NO': 0, 'LT': 1, 'AT': 2, 'VAT': 3, 'BONE': 4}, 'body_mask': numpy.ndarray (None, 192, 256), 'bondy_cont_pc': numpy.ndarray (None, 3), 'resolution': numpy.ndarray (N, 3), 'center': numpy.ndarray (N, 3), 'smpl_dict': dict dict_keys(['gender', 'verts_free', 'verts', 'faces', 'pose', 'betas', 'trans']) } ``` ### Data Fields Each data instance (male/train/001.gz for example) contains the following fields: - 'gender': "gender of the subject", - 'subject_ID': "anonymized name of the subject which is also the filename" - 'mri_seg': "annotated array with the labels 0,1,2,3", - 'mri_labels': "dictionary of mapping between label integer and name", - 'body_mask': "binary array for body mask", - 'body_cont_pc' "extracted point cloud from mri contours" - 'resolution': "per slice resolution in meters", - 'center': "per slice center, in pixels", - 'smpl_dict': dictionary containing all the relevant SMPL parameters of the subject alongwith mesh faces and vertices ('verts': original fit, 'verts_free': compressed fit ### Data Splits The HIT dataset has 3 splits for each subject type (male, female): train, val, and test. | | train | validation | test | |-------------------------|------:|-----------:|-----:| | male | 126 | 16 | 15 | | female | 191 | 25 | 25 | ## Dataset Creation ### Curation Rationale The dataset was created to foster research in biomechanics, computer graphics and Human Digital Twins. ### Source Data #### Initial Data Collection and Normalization We work with scans acquired with a 1.5 T scanner (Magnetom Sonata, Siemens Healthcare) following a standardized protocol for whole body adipose tissue topography mapping. All subjects gave prior informed written consent and the study was approved by the local ethics board. Each scan has around 110 slices, slightly varying depending on the height of the subject. The slice resolution is 256 × 192, with an approximate voxel size of 2 × 2 × 10 mm. These slices are segmented into bones, lean, and adipose tissue by leveraging initial automatic segmentations and manual annotations to train and refine nnUnets with the help of human supervision. For each subject, we then fit the SMPL body mesh to the surface of the segmented MRI in a manner that captures the flattened shape of subjects in their lying positions on belly in the scanner (refer to Sec 3.2 in main paper for further details). Therefore for each subject, we provide the MRI segmented array and the SMPL mesh faces and vertices (in addition to the SMPL parameters). #### Who are the source language producers? [N/A] ### Annotations #### Annotation process Refer to Sec 3 of the paper. #### Who are the annotators? Refer to Sec 3 of the paper. ### Personal and Sensitive Information The dataset uses identity category of gender: male and female. As the dataset intends to foster research in estimating tissues from outer shape which vary subsequently between the genders, the dataset is categorized as such. ## Considerations for Using the Data ### Social Impact of Dataset Today, many methods can estimate accurate SMPL bodies from images, and this dataset can be used to train models that can infer their internal tissues. As a good estimate of the body composition relates to health risks, HIT dataset could allow the estimation of health risks from a single image of a person. This is valuable as an early diagnostic tool when used with the persons knowledge, but could turn into a risk if it is used without consent. ### Discussion of Biases [N/A] ### Other Known Limitations Refer to Sec 3.3 of the paper ## Additional Information ### Dataset Curators The HIT dataset was curated by [Vaibhav Arora]([e-mail]), Abdelmouttaleb Dakri, Jürgen Machann, Sergi Pujades ### Licensing Information #### Software Copyright License for non-commercial scientific research purposes Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use the HIT data and software, (the "Data & Software"), including trained models, 3D meshes, images, videos, textures, software, scripts, and animations. By downloading and/or using the Data & Software (including downloading, cloning, installing, and any other use of the corresponding github repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Data & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this License. #### Ownership/Licensees The Software and the associated materials has been developed at the Max Planck Institute for Intelligent Systems (hereinafter "MPI"), University of Tübingen, and INRIA. The original skeleton mesh is released with permission of Anatoscope (www.anatoscope.com). Any copyright or patent right is owned by and proprietary material of the Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (hereinafter “MPG”; MPI and MPG hereinafter collectively “Max-Planck”), hereinafter the “Licensor”. #### License Grant Licensor grants you (Licensee) personally a single-user, non-exclusive, non-transferable, free of charge right: - To install the Data & Software on computers owned, leased or otherwise controlled by you and/or your organization; - To use the Data & Software for the sole purpose of performing non-commercial scientific research, non-commercial education, or non-commercial artistic projects; Any other use, in particular any use for commercial, pornographic, military, or surveillance, purposes is prohibited. This includes, without limitation, incorporation in a commercial product, use in a commercial service, or production of other artefacts for commercial purposes. The Data & Software may not be used to create fake, libelous, misleading, or defamatory content of any kind excluding analyses in peer-reviewed scientific research. The Software may not be reproduced, modified and/or made available in any form to any third party without Max-Planck’s prior written permission. The Data & Software may not be used for pornographic purposes or to generate pornographic material whether commercial or not. This license also prohibits the use of the Software to train methods/algorithms/neural networks/etc. for commercial, pornographic, military, surveillance, or defamatory use of any kind. By downloading the Data & Software, you agree not to reverse engineer it. #### No Distribution The Data & Software and the license herein granted shall not be copied, shared, distributed, re-sold, offered for re-sale, transferred or sub-licensed in whole or in part except that you may make one copy for archive purposes only. #### Disclaimer of Representations and Warranties You expressly acknowledge and agree that the Data & Software results from basic research, is provided “AS IS”, may contain errors, and that any use of the Data & Software is at your sole risk. LICENSOR MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE DATA & SOFTWARE, NEITHER EXPRESS NOR IMPLIED, AND THE ABSENCE OF ANY LEGAL OR ACTUAL DEFECTS, WHETHER DISCOVERABLE OR NOT. Specifically, and not to limit the foregoing, licensor makes no representations or warranties (i) regarding the merchantability or fitness for a particular purpose of the Data & Software, (ii) that the use of the Data & Software will not infringe any patents, copyrights or other intellectual property rights of a third party, and (iii) that the use of the Data & Software will not cause any damage of any kind to you or a third party. #### Limitation of Liability Because this Data & Software License Agreement qualifies as a donation, according to Section 521 of the German Civil Code (Bürgerliches Gesetzbuch – BGB) Licensor as a donor is liable for intent and gross negligence only. If the Licensor fraudulently conceals a legal or material defect, they are obliged to compensate the Licensee for the resulting damage. Licensor shall be liable for loss of data only up to the amount of typical recovery costs which would have arisen had proper and regular data backup measures been taken. For the avoidance of doubt Licensor shall be liable in accordance with the German Product Liability Act in the event of product liability. The foregoing applies also to Licensor’s legal representatives or assistants in performance. Any further liability shall be excluded. Patent claims generated through the usage of the Data & Software cannot be directed towards the copyright holders. The Data & Software is provided in the state of development the licensor defines. If modified or extended by Licensee, the Licensor makes no claims about the fitness of the Data & Software and is not responsible for any problems such modifications cause. #### No Maintenance Services You understand and agree that Licensor is under no obligation to provide either maintenance services, update services, notices of latent defects, or corrections of defects with regard to the Data & Software. Licensor nevertheless reserves the right to update, modify, or discontinue the Data & Software at any time. Defects of the Data & Software must be notified in writing to the Licensor with a comprehensible description of the error symptoms. The notification of the defect should enable the reproduction of the error. The Licensee is encouraged to communicate any use, results, modification or publication. #### Publications using the Data & Software You acknowledge that the Data & Software is a valuable scientific resource and agree to appropriately reference the following paper in any publication making use of the Data & Software. #### Commercial licensing opportunities For commercial uses of the Data & Software, please send email to [e-mail] This Agreement shall be governed by the laws of the Federal Republic of Germany except for the UN Sales Convention. ### Citation Information ``` @inproceedings{Keller:CVPR:2024, title = {{HIT}: Estimating Internal Human Implicit Tissues from the Body Surface}, author = {Keller, Marilyn and Arora, Vaibhav and Dakri, Abdelmouttaleb and Chandhok, Shivam and Machann, Jürgen and Fritsche, Andreas and Black, Michael J. and Pujades, Sergi}, booktitle = {Proceedings IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)}, month = jun, year = {2024}, month_numeric = {6}} ``` ### Contributions [N/A]

原始信息汇总

数据集概述

名称: HIT数据集

描述: HIT数据集是一个结构化的配对观测数据集，包含人体内部组织和体表的数据。具体来说，该数据集包含配对的全身体积分割（骨骼、瘦肉和脂肪组织）MRI扫描和SMPL网格，捕捉男性和女性受试者的体表形状。该数据集对于医学、运动科学、生物力学和计算机图形学具有重要意义，可用于创建个性化的解剖数字双胞胎，模拟我们的骨骼、瘦肉和脂肪组织。

数据获取: 使用1.5 T扫描仪（Magnetom Sonata，Siemens Healthcare）按照标准化协议进行全身脂肪组织地形图扫描。所有受试者均提供事先书面知情同意，研究获得当地伦理委员会批准。每个扫描包含约110个切片，切片分辨率为256×192，体素大小约为2×2×10 mm。这些切片通过利用初始自动分割和手动注释，借助人工监督训练和精化nnUnets进行骨骼、瘦肉和脂肪组织的分割。对于每个受试者，我们通过将SMPL身体网格拟合到分割的MRI表面，捕捉受试者在扫描仪中俯卧位置的扁平形状。因此，对于每个受试者，我们提供分割的MRI数组和SMPL网格面和顶点（以及SMPL参数）。

数据结构:

• 性别: 男性、女性
• 分割: 训练、验证、测试
• 文件路径: 性别/分割/*.gz

数据实例内容:

• gender: str [male, female]
• subject_ID: str
• mri_seg: numpy.ndarray (None, 192, 256)
• mri_labels: dict {NO: 0, LT: 1, AT: 2, VAT: 3, BONE: 4}
• body_mask: numpy.ndarray (None, 192, 256)
• bondy_cont_pc: numpy.ndarray (None, 3)
• resolution: numpy.ndarray (N, 3)
• center: numpy.ndarray (N, 3)
• smpl_dict: dict dict_keys([gender, verts_free, verts, faces, pose, betas, trans])

数据分割:

• 男性: 训练126个，验证16个，测试15个
• 女性: 训练191个，验证25个，测试25个

许可证: max-planck

标签: SMPL, Tissues, Medical, Biomechanics, Human-Twins, Digital-Twins, Mesh, Bones, 3D, Classification, Occupancy, MRI, Segmentation

使用方法

快速使用: python pip install datasets from datasets import load_dataset

示例: 加载男性训练数据

male_train = load_dataset("varora/hit", name=male, split=train)

print(male_train.len()) print(next(iter(male_train)))

可视化数据: python pip install datasets, open3d, pyvista python vis_hit_sample.py --gender male --split test --idx 5 --show_skin python vis_hit_sample.py --gender male --split test --idx 5 --show_tissue

数据集创建

采集和规范化: 使用1.5 T扫描仪（Magnetom Sonata，Siemens Healthcare）按照标准化协议进行全身脂肪组织地形图扫描。所有受试者均提供事先书面知情同意，研究获得当地伦理委员会批准。每个扫描包含约110个切片，切片分辨率为256×192，体素大小约为2×2×10 mm。这些切片通过利用初始自动分割和手动注释，借助人工监督训练和精化nnUnets进行骨骼、瘦肉和脂肪组织的分割。

注释过程: 参见论文第3节。

个人和敏感信息: 数据集使用性别（男性和女性）作为身份类别。由于数据集旨在促进从外部形状估计组织，这些组织随后在性别之间有所不同，因此数据集按此分类。

搜集汇总

数据集介绍

构建方式

HIT数据集是一项针对人体内部组织与体表形态配对观测的结构化资源。其构建基于1.5T磁共振扫描仪采集的全身脂肪组织分布图像，每份扫描包含约110层切片，体素大小约为2×2×10毫米。在数据预处理阶段，研究者通过初始自动分割与人工标注相结合的方式，训练并优化nnU-Net模型，将MRI切片精确分割为骨骼、瘦组织和脂肪组织。随后，针对每位受试者，采用SMPL人体网格模型拟合至分割后MRI的表面，以捕捉其在扫描仪中俯卧姿态下的扁平化体表形状。最终，每个数据实例同时包含三维分割数组和SMPL网格的顶点与面片信息，为多学科研究提供了高保真的数字化解剖学基础。

使用方法

用户可通过HuggingFace的datasets库便捷调用该数据集，例如使用`load_dataset("varora/hit", name='male', split='train')`加载指定性别和子集的数据。数据集按性别划分为训练、验证和测试子集，男性分别为126、16、15例，女性分别为191、25、25例。每个数据实例以压缩格式存储，包含性别标识、MRI分割数组、SMPL字典等字段。为便于可视化，官方提供了`vis_hit_sample.py`脚本，支持展示体表网格、点云及组织切片。研究者可基于此数据集开展内部组织推断、体表-组织关联建模等任务，推动无创健康评估与数字人体研究的发展。

背景与挑战

背景概述

在数字人体建模与生物力学领域，从体表形态推断内部组织结构的跨模态映射问题长期悬而未决。由马克思·普朗克智能系统研究所、图宾根大学及法国国家信息与自动化研究所的研究团队于2024年联合构建的HIT数据集，开创性地提供了398例（男性157例，女性241例）配对的全身体积分割MRI扫描与SMPL体表网格数据。该数据集的核心研究问题聚焦于建立人体表面几何与内部骨骼、瘦组织及脂肪组织之间的结构化关联，为个性化解剖数字孪生的生成奠定数据基础。其影响力跨越医学影像分析、运动科学及计算机图形学，通过发布标准化双模态数据，首次使得从单张人体图像估测内部组织分布的跨学科研究成为可能，推动了非侵入式健康风险评估技术的范式革新。

当前挑战

HIT数据集所面临的挑战具有双重维度。在领域问题层面，其核心难点在于体表与内部组织间存在高度非线性的映射关系——人体脂肪与肌肉的分布受遗传、年龄及代谢状态影响，仅依赖几何特征难以实现精准的跨模态推断；同时，现有深度学习方法在分割MRI中细粒度组织（如内脏脂肪与皮下脂肪的区分）时，仍受限于扫描分辨率（2×2×10 mm³）带来的边界模糊性。在数据集构建过程中，挑战体现在多阶段数据对齐的精密性上：需将受试者俯卧状态下因重力形变的体表SMPL网格与MRI分割体素进行空间配准，且半监督分割流程中依赖人工标注校正nnUNet的初始预测，导致标注成本高昂；此外，不同性别间组织分布的统计差异要求分别建模，进一步增加了数据划分与模型泛化的复杂性。

常用场景

经典使用场景

HIT数据集的核心应用在于建立人体表面形态与内部组织之间的映射关系。通过提供成对的全身MRI分割体数据（标注骨骼、瘦组织和脂肪组织）与SMPL人体网格模型，该数据集为从外部体表推断内部解剖结构开辟了新的研究范式。研究者可基于此训练回归模型，利用三维人体表面几何信息直接预测内部组织的体积分布与空间位置，实现非侵入式的体内组织估算。这一经典用法在人体数字孪生、运动生物力学分析及个性化医学建模领域具有里程碑意义。

解决学术问题

该数据集解决了人体解剖学与计算机图形学交叉领域的一个核心难题：如何从可观测的体表形状推断不可见的内部组织构成。传统上，人体内部组织的量化依赖MRI或CT等昂贵设备，而HIT通过提供大规模配对数据，使得基于外观的体内组织估计成为可能。这直接回应了健康风险评估、肥胖相关代谢疾病研究中的关键需求——从简单的人体测量或图像中获取脂肪、肌肉和骨骼的分布信息，从而推动流行病学筛查和临床前诊断方法的范式革新。

实际应用

在实际应用中，HIT数据集赋能了从单张照片或视频中估算人体健康指标的技术路径。例如，结合计算机视觉与深度学习，可开发出仅需普通摄像头即可评估体脂率、肌肉量及骨密度的移动端应用，服务于健身指导、营养规划与慢性病风险预警。在运动科学领域，基于该数据集的模型能实时分析运动员的肌肉与脂肪分布，优化训练方案；在临床场景中，它有望成为低成本、无辐射的代谢健康初筛工具，尤其适用于资源匮乏地区的流行病学调查。

数据集最近研究

最新研究方向

在数字孪生与人体生物力学的前沿交叉领域，HIT数据集通过配对全身MRI分割影像与SMPL表面网格，开创性地建立了人体内部组织（骨骼、瘦组织与脂肪组织）与外部体表形态之间的结构化映射关系。该数据集为个性化解剖数字孪生的构建提供了关键基准，推动医学影像分析、运动科学及计算机图形学向非侵入式健康评估方向发展。当前研究热点集中于利用该数据集训练深度学习模型，从单张人体图像直接推断内部组织分布，从而实现体脂率、骨密度等健康指标的远程估测。这一方向与可穿戴健康监测及远程医疗的爆发式需求高度契合，有望革新早期疾病筛查范式，但同时也引发了对隐私与伦理的深度思考——如何在无监督场景下防止数据滥用成为学界与产业界共同关注的议题。