Holasyb918/imagenet-1k-adm-crop-256

---
license: apache-2.0
task_categories:
- image-classification
tags:
- imagenet
- image-classification
- computer-vision
- diffusion
- adm
pretty_name: ImageNet-1k ADM Crop 256
size_categories:
- 1M<n<10M
dataset_info:
features:
- name: image
dtype: image
- name: label
dtype: int64
splits:
- name: train
num_examples: 1281167
- name: test
num_examples: 50000
---

# ImageNet-1k ADM Crop 256

This dataset is a preprocessed version of [ILSVRC/imagenet-1k](https://huggingface.co/datasets/ILSVRC/imagenet-1k) with all images center-cropped to **256×256** pixels using the ADM (Ablated Diffusion Model) algorithm.

## 🎯 Purpose

Optimized for training diffusion models and other generative models that require fixed-size square images.

## 📊 Dataset Details

| Split | Images | Files | Size (approx) |
|-------|--------|-------|---------------|
| train | 1,281,167 | 294 | ~38 GB |
| test | 50,000 | 28 | ~3.5 GB |

## 🔧 Processing Method

### Center Crop Algorithm (from ADM)

The center crop implementation follows the [guided-diffusion](https://github.com/openai/guided-diffusion) approach:

```python
from PIL import Image
import numpy as np

def center_crop_arr(pil_image, image_size):
"""
Center cropping implementation from ADM.
https://github.com/openai/guided-diffusion/blob/8fb3ad9197f16bbc40620447b2742e13458d2831/guided_diffusion/image_datasets.py#L126
"""
# Progressively downsample if image is much larger than target
while min(*pil_image.size) >= 2 * image_size:
pil_image = pil_image.resize(
tuple(x // 2 for x in pil_image.size), resample=Image.BOX
)

# Scale so shortest side equals target size
scale = image_size / min(*pil_image.size)
pil_image = pil_image.resize(
tuple(round(x * scale) for x in pil_image.size), resample=Image.BICUBIC
)

# Center crop to exact target size
arr = np.array(pil_image)
crop_y = (arr.shape[0] - image_size) // 2
crop_x = (arr.shape[1] - image_size) // 2
return Image.fromarray(
arr[crop_y : crop_y + image_size, crop_x : crop_x + image_size]
)
```

### Why this algorithm?

1. **Progressive downsampling**: Uses BOX filter for initial reduction, preserving image quality
2. **BICUBIC scaling**: High-quality interpolation for final resize
3. **Exact center crop**: Ensures consistent 256×256 output

## 📁 Data Structure

```
data/
├── train-00000-of-00294.parquet
├── train-00001-of-00294.parquet
├── ...
├── train-00293-of-00294.parquet
├── test-00000-of-00028.parquet
├── ...
└── test-00027-of-00028.parquet
```

## 📋 Schema

| Column | Type | Description |
|--------|------|-------------|
| image | Image | 256×256 RGB JPEG image |
| label | int64 | Class label (0-999 for train, -1 for test) |

## 🚀 Usage

### With 🤗 Datasets

```python
from datasets import load_dataset

# Load full dataset
dataset = load_dataset("Holasyb918/imagenet-1k-adm-crop-256")

# Load specific split
train_dataset = load_dataset("Holasyb918/imagenet-1k-adm-crop-256", split="train")
test_dataset = load_dataset("Holasyb918/imagenet-1k-adm-crop-256", split="test")

# Access data
for example in train_dataset:
image = example["image"] # PIL Image, 256×256
label = example["label"] # int, 0-999
```

### With PyTorch DataLoader

```python
from datasets import load_dataset
from torch.utils.data import DataLoader
from torchvision import transforms

# Load dataset
dataset = load_dataset("Holasyb918/imagenet-1k-adm-crop-256", split="train")

# Define transform
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]) # [-1, 1]
])

def collate_fn(batch):
images = torch.stack([transform(x["image"]) for x in batch])
labels = torch.tensor([x["label"] for x in batch])
return {"image": images, "label": labels}

# Create DataLoader
dataloader = DataLoader(dataset, batch_size=32, collate_fn=collate_fn, num_workers=4)
```

## 📜 License

This dataset follows the same license terms as the original [ImageNet](https://www.image-net.org/) dataset. Please ensure you comply with ImageNet's terms of use.

## 🙏 Acknowledgments

- Original dataset: [ILSVRC/imagenet-1k](https://huggingface.co/datasets/ILSVRC/imagenet-1k)
- Center crop algorithm: [OpenAI guided-diffusion](https://github.com/openai/guided-diffusion)

## 📝 Citation

If you use this dataset, please cite the original ImageNet paper:

```bibtex
@article{deng2009imagenet,
title={ImageNet: A large-scale hierarchical image database},
author={Deng, Jia and Dong, Wei and Socher, Richard and Li, Li-Jia and Li, Kai and Fei-Fei, Li},
},
year={2009}
}
```

---

_{📝 This README was generated with the assistance of AI (Claude).}

license: apache-2.0
task_categories:
- 图像分类
tags:
- ImageNet
- 图像分类
- 计算机视觉(computer vision)
- 扩散模型(diffusion model)
- ADM(Ablated Diffusion Model)
pretty_name: ImageNet-1k ADM Crop 256
size_categories:
- 100万 < 样本数 < 1000万
dataset_info:
features:
- name: image
dtype: 图像
- name: label
dtype: int64
splits:
- name: train
num_examples: 1281167
- name: test
num_examples: 50000

# ImageNet-1k ADM Crop 256 数据集简介
本数据集是[ILSVRC/imagenet-1k](https://huggingface.co/datasets/ILSVRC/imagenet-1k)的预处理版本，所有图像均通过ADM（消融扩散模型，Ablated Diffusion Model, ADM）算法被中心裁剪至**256×256**像素。

## 🎯 数据集用途
专为需要固定尺寸方形图像的扩散模型(diffusion model)及其他生成式模型(generative model)的训练优化设计。

## 📊 数据集详情
| 数据集划分 | 图像数量 | 文件数 | 近似大小 |
|-------|--------|-------|---------------|
| 训练集 | 1,281,167 | 294 | ~38 GB |
| 测试集 | 50,000 | 28 | ~3.5 GB |

## 🔧 预处理方法
### 中心裁剪算法（源自ADM）
该中心裁剪实现遵循[guided-diffusion](https://github.com/openai/guided-diffusion)的实现思路：
python
from PIL import Image
import numpy as np

def center_crop_arr(pil_image, image_size):
"""
Center cropping implementation from ADM.
https://github.com/openai/guided-diffusion/blob/8fb3ad9197f16bbc40620447b2742e13458d2831/guided_diffusion/image_datasets.py#L126
"""
# Progressively downsample if image is much larger than target
while min(*pil_image.size) >= 2 * image_size:
pil_image = pil_image.resize(
tuple(x // 2 for x in pil_image.size), resample=Image.BOX
)

# Scale so shortest side equals target size
scale = image_size / min(*pil_image.size)
pil_image = pil_image.resize(
tuple(round(x * scale) for x in pil_image.size), resample=Image.BICUBIC
)

# Center crop to exact target size
arr = np.array(pil_image)
crop_y = (arr.shape[0] - image_size) // 2
crop_x = (arr.shape[1] - image_size) // 2
return Image.fromarray(
arr[crop_y : crop_y + image_size, crop_x : crop_x + image_size]
)

### 该算法的优势
1. **渐进式下采样**：采用BOX滤波器进行初始尺寸缩减，有效保留图像质量
2. **双三次插值缩放**：使用高质量插值算法完成最终尺寸调整
3. **精准中心裁剪**：确保输出统一为256×256像素的图像

## 📁 数据结构

data/
├── train-00000-of-00294.parquet
├── train-00001-of-00294.parquet
├── ...
├── train-0293-of-00294.parquet
├── test-00000-of-00028.parquet
├── ...
└── test-0027-of-00028.parquet

## 📋 数据结构规范
| 字段名 | 数据类型 | 字段说明 |
|--------|------|-------------|
| image | 图像 | 256×256像素RGB格式JPEG图像 |
| label | int64 | 类别标签（训练集取值范围为0-999，测试集为-1） |

## 🚀 使用方法
### 使用🤗 Datasets库加载
python
from datasets import load_dataset

# 加载完整数据集
dataset = load_dataset("Holasyb918/imagenet-1k-adm-crop-256")

# 加载指定划分
train_dataset = load_dataset("Holasyb918/imagenet-1k-adm-crop-256", split="train")
test_dataset = load_dataset("Holasyb918/imagenet-1k-adm-crop-256", split="test")

# 访问数据
for example in train_dataset:
image = example["image"] # PIL图像，尺寸为256×256
label = example["label"] # 整数，取值范围0-999

### 使用PyTorch DataLoader加载
python
from datasets import load_dataset
from torch.utils.data import DataLoader
from torchvision import transforms

# 加载数据集
dataset = load_dataset("Holasyb918/imagenet-1k-adm-crop-256", split="train")

# 定义数据变换
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]) # 归一化至[-1, 1]区间
])

def collate_fn(batch):
images = torch.stack([transform(x["image"]) for x in batch])
labels = torch.tensor([x["label"] for x in batch])
return {"image": images, "label": labels}

# 创建DataLoader
dataloader = DataLoader(dataset, batch_size=32, collate_fn=collate_fn, num_workers=4)

## 📜 许可证
本数据集采用与原始[ImageNet](https://www.image-net.org/)数据集一致的许可证条款，请务必遵守ImageNet的使用协议。

## 🙏 致谢
- 原始数据集：[ILSVRC/imagenet-1k](https://huggingface.co/datasets/ILSVRC/imagenet-1k)
- 中心裁剪算法：[OpenAI guided-diffusion](https://github.com/openai/guided-diffusion)

## 📝 引用规范
若您使用本数据集，请引用原始ImageNet论文：
bibtex
@article{deng2009imagenet,
title={ImageNet: A large-scale hierarchical image database},
author={Deng, Jia and Dong, Wei and Socher, Richard and Li, Li-Jia and Li, Kai and Fei-Fei, Li},
year={2009}
}

_{📝 本README文档由AI（Claude）辅助生成。}