Birchlabs/openai-guided-diffusion-256-classcond-unguided-samples-50k

--- license: apache-2.0 pretty_name: OpenAI guided-diffusion 256px class-conditional unguided samples (50k) size_categories: - 10K<n<100K --- CleanFID: ``` FID: 18.9796, KID: 0.0145976 ``` Then another measurement (torchmetrics FID and CleanFID simultaneously): ``` torchmetrics FID: 19.3133 CleanFID FID: 19.1283, KID: 0.0147355 ``` Read from the webdataset (after saving it somewhere on your disk) like this: ```python from webdataset import WebDataset from typing import TypedDict, Iterable from PIL import Image from PIL.PngImagePlugin import PngImageFile from io import BytesIO from os import makedirs Example = TypedDict('Example', { '__key__': str, '__url__': str, 'img.png': bytes, }) dataset = WebDataset('./openai-guided-diffusion-256-classcond-unguided-samples-50k/{00000..00004}.tar') out_root = 'out' makedirs(out_root, exist_ok=True) it: Iterable[Example] = iter(dataset) for ix, item in enumerate(it): with BytesIO(item['img.png']) as stream: img: PngImageFile = Image.open(stream) img.load() img.save(f'{out_root}/{ix}.png') ``` Or from the HF dataset like this: ```python from datasets import load_dataset from datasets.dataset_dict import DatasetDict from datasets.arrow_dataset import Dataset from PIL.PngImagePlugin import PngImageFile from typing import TypedDict, Iterable from os import makedirs class Item(TypedDict): index: int tar: str tar_path: str img: PngImageFile dataset: DatasetDict = load_dataset('Birchlabs/openai-guided-diffusion-256-classcond-unguided-samples-50k') train: Dataset = dataset['train'] out_root = 'out' makedirs(out_root, exist_ok=True) it: Iterable[Item] = iter(train) for item in it: item['img'].save(f'{out_root}/{item["index"]}.png') ```许可证：apache-2.0 友好名称：OpenAI 引导式扩散模型（guided-diffusion）256像素类别条件无引导采样集（5万张） 样本量分类： - 10K < n < 100K --- CleanFID 指标结果： FID: 18.9796, KID: 0.0145976 另有一组同时使用torchmetrics FID与CleanFID的测量结果： torchmetrics FID: 19.3133 CleanFID FID: 19.1283, KID: 0.0147355 可在将数据集保存至本地磁盘后，通过如下代码从WebDataset读取： python from webdataset import WebDataset from typing import TypedDict, Iterable from PIL import Image from PIL.PngImagePlugin import PngImageFile from io import BytesIO from os import makedirs Example = TypedDict('Example', { '__key__': str, '__url__': str, 'img.png': bytes, }) dataset = WebDataset('./openai-guided-diffusion-256-classcond-unguided-samples-50k/{00000..00004}.tar') out_root = 'out' makedirs(out_root, exist_ok=True) it: Iterable[Example] = iter(dataset) for ix, item in enumerate(it): with BytesIO(item['img.png']) as stream: img: PngImageFile = Image.open(stream) img.load() img.save(f'{out_root}/{ix}.png') 亦可通过HF数据集按如下方式加载： python from datasets import load_dataset from datasets.dataset_dict import DatasetDict from datasets.arrow_dataset import Dataset from PIL.PngImagePlugin import PngImageFile from typing import TypedDict, Iterable from os import makedirs class Item(TypedDict): index: int tar: str tar_path: str img: PngImageFile dataset: DatasetDict = load_dataset('Birchlabs/openai-guided-diffusion-256-classcond-unguided-samples-50k') train: Dataset = dataset['train'] out_root = 'out' makedirs(out_root, exist_ok=True) it: Iterable[Item] = iter(train) for item in it: item['img'].save(f'{out_root}/{item["index"]}.png')

原始信息汇总

数据集概述

基本信息

• 名称: OpenAI guided-diffusion 256px class-conditional unguided samples (50k)
• 许可证: Apache-2.0
• 大小类别: 10K < n < 100K

性能指标

• FID: 18.9796
• KID: 0.0145976
• torchmetrics FID: 19.3133
• CleanFID FID: 19.1283
• CleanFID KID: 0.0147355

数据加载示例

• 从webdataset加载: python from webdataset import WebDataset from typing import TypedDict, Iterable from PIL import Image from PIL.PngImagePlugin import PngImageFile from io import BytesIO from os import makedirs

  Example = TypedDict(Example, { key: str, url: str, img.png: bytes, })

  dataset = WebDataset(./openai-guided-diffusion-256-classcond-unguided-samples-50k/{00000..00004}.tar)

  out_root = out makedirs(out_root, exist_ok=True)

  it: Iterable[Example] = iter(dataset) for ix, item in enumerate(it): with BytesIO(item[img.png]) as stream: img: PngImageFile = Image.open(stream) img.load() img.save(f{out_root}/{ix}.png)

• 从HF dataset加载: python from datasets import load_dataset from datasets.dataset_dict import DatasetDict from datasets.arrow_dataset import Dataset from PIL.PngImagePlugin import PngImageFile from typing import TypedDict, Iterable from os import makedirs

  class Item(TypedDict): index: int tar: str tar_path: str img: PngImageFile

  dataset: DatasetDict = load_dataset(Birchlabs/openai-guided-diffusion-256-classcond-unguided-samples-50k) train: Dataset = dataset[train]

  out_root = out makedirs(out_root, exist_ok=True)

  it: Iterable[Item] = iter(train) for item in it: item[img].save(f{out_root}/{item["index"]}.png)