FER-Universe/DiffusionFER

---
layout: default
title: Home
nav_order: 1
has_children: false
annotations_creators:
- no-annotation
language:
- en
language_creators:
- found
pretty_name: DiffusionFER
size_categories:
- n<500MB
source_datasets:
- original
license: cc0-1.0
tags:
- stable diffusion
- prompt engineering
- prompts
- research paper
- facial expression recognition
- emotion recognition
task_categories:
- text-to-image
task_ids:
- image-captioning
- face-detection
---

## Dataset Description

- **Homepage:** [DiffusionFER homepage](https://kdhht2334.github.io/)
- **Repository:** [DiffusionFER repository](https://github.com/kdhht2334/Facial-Expression-Recognition-Zoo)
- **Distribution:** [DiffusionFER Hugging Face Dataset](https://huggingface.co/datasets/FER-Universe/DiffusionFER)
- **Point of Contact:** [Daeha Kim](mailto:[e-mail])

### Summary

DiffusionFER is the large-scale text-to-image prompt database for face-related tasks. It contains about **1M(ongoing)** images generated by [Stable Diffusion](https://github.com/camenduru/stable-diffusion-webui-colab) using prompt(s) and other parameters.

DiffusionFER is available at [🤗 Hugging Face Dataset](https://huggingface.co/datasets/FER-Universe/DiffusionFER).

### Downstream Tasks and Leaderboards

This DiffusionFER dataset can be utilized for the following downstream tasks.
- Face detection
- Facial expression recognition
- Text-to-emotion prompting

In addition, the virtual subjects included in this dataset provide opportunities to perform various vision tasks related to face privacy.

### Data Loading

DiffusionFER can be loaded via both Python and Git. Please refer Hugging Face [`Datasets`](https://huggingface.co/docs/datasets/quickstart).

```python
from datasets import load_dataset

dataset = load_dataset("FER-Universe/DiffusionFER")
```

```bash
git lfs install
git clone https://huggingface.co/datasets/FER-Universe/DiffusionFER
```

### Pre-trained model

You can easily download and use pre-trained __Swin Transformer__ model with the `Diffusion_Emotion_S` dataset.

Later, Transformer models with the `Diffusion_Emotion_M` or `Diffusion_Emotion_L` will be released.

```python
from transformers import AutoFeatureExtractor, AutoModelForImageClassification

extractor = AutoFeatureExtractor.from_pretrained("kdhht2334/autotrain-diffusion-emotion-facial-expression-recognition-40429105176")

model = AutoModelForImageClassification.from_pretrained("kdhht2334/autotrain-diffusion-emotion-facial-expression-recognition-40429105176")
```

Or just clone the model repo

```bash
git lfs install
git clone https://huggingface.co/kdhht2334/autotrain-diffusion-emotion-facial-expression-recognition-40429105176
```

- Quick links: [huggingface model documentation](https://huggingface.co/docs/transformers/main/en/model_doc/swin#transformers.SwinForImageClassification)

### Sample Gallery

▼Happy


▼Angry


### Subsets

DiffusionFER supports a total of three distinct splits. And, each split additionally provides a face region cropped by [face detector](https://github.com/timesler/facenet-pytorch).
- DifussionEmotion_S (small), DifussionEmotion_M (medium), DifussionEmotion_L (large).

|Subset|Num of Images|Size|Image Directory |
|:--|--:|--:|--:|
|DifussionEmotion_S (original) | 1.5K | 647M | `DifussionEmotion_S/` |
|DifussionEmotion_S (cropped) | 1.5K | 322M | `DiffusionEmotion_S_cropped/` |
|DifussionEmotion_M (original) | N/A | N/A | `DifussionEmotion_M/` |
|DifussionEmotion_M (cropped) | N/A | N/A | `DiffusionEmotion_M_cropped/` |
|DifussionEmotion_L (original) | N/A | N/A | `DifussionEmotion_L/` |
|DifussionEmotion_L (cropped) | N/A | N/A | `DiffusionEmotion_L_cropped/` |

## Dataset Structure

We provide DiffusionFER using a modular file structure. `DiffusionEmotion_S`, the smallest scale, contains about 1,500 images and is divided into folders of a total of 7 emotion classes. The class labels of all these images are included in `dataset_sheet.csv`.
- In `dataset_sheet.csv`, not only 7-emotion class but also _valence-arousal_ value are annotated.

```bash
# Small version of DB
./
├── DifussionEmotion_S
│   ├── angry
│   │   ├── aaaaaaaa_6.png
│   │   ├── andtcvhp_6.png
│   │   ├── azikakjh_6.png
│   │   ├── [...]
│   ├── fear
│   ├── happy
│   ├── [...]
│   └── surprise
└── dataset_sheet.csv
```

- Middle size DB will be uploaded soon.
```bash
# Medium version of DB
(ongoing)
```

- TBD
```bash
# Large version of DB
(ongoing)
```

### Prompt Format

Basic format is as follows: "`Emotion`, `Race` `Age` style, a realistic portrait of `Style` `Gender`, upper body, `Others`".

- ex) one person, neutral emotion, white middle-aged style, a realistic portrait of man, upper body

Examples of format categories are listed in the table below.

| Category | Prompt(s) |
| --- | --- |
| `Emotion` | neutral emotion<br>happy emotion, with open mouth, smiley<br>sad emotion, with tears, lowered head, droopy eyebrows<br>surprise emotion, with open mouth, big eyes<br>fear emotion, scared, haunted<br>disgust emotion, frown, angry expression with open mouth<br>angry emotion, with open mouth, frown eyebrow, fierce, furious |
| `Race` | white<br>black<br>latin |
| `Age` | teen<br>middle-aged<br>old |
| `Gender` | man<br>woman |
| `Style` | gentle<br>handsome<br>pretty<br>cute<br>mature<br>punky<br>freckles<br>beautiful crystal eyes<br>big eyes<br>small nose<br>... |
| `Others` | 4K<br>8K<br>cyberpunk<br>camping<br>ancient<br>medieval Europe<br>... |

### Prompt Engineering

You can improve the performance and quality of generating default prompts with the settings below.

```
{
"negative prompt": "sketches, (worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, backlight, (duplicate:1.331), (morbid:1.21), (mutilated:1.21), mutated hands, (poorly drawn hands:1.331), (bad anatomy:1.21), (bad proportions:1.331), extra limbs, (disfigured:1.331), (missing arms:1.331), (extra legs:1.331), (fused fingers:1.61051), (too many fingers:1.61051), (unclear eyes:1.331), bad hands, missing fingers, extra digit",
"steps": 50,
"sampling method": "DPM++ 2M Karras"
"Width": "512",
"Height": "512",
"CFG scale": 12.0,
"seed": -1,
}
```

### Annotations

The DiffusionFER contains annotation process both 7-emotion classes and valence-arousal values.

#### Annotation process

This process was carried out inspired by the theory of the two research papers below.

- JA Russell, [A circumplex model of affect](https://d1wqtxts1xzle7.cloudfront.net/38425675/Russell1980-libre.pdf?1439132613=&response-content-disposition=inline%3B+filename%3DRussell1980.pdf&Expires=1678595455&Signature=UtbPsezND6w8vbISBiuL-ECk6hDI0etLcJSE7kJMC~hAkMSu9YyQcPKdVpdHSSq7idfcQ~eEKsqptvYpy0199DX0gi-nHJwhsciahC-zgDwylEUo6ykhP6Ab8VWCOW-DM21jHNvbYLQf7Pwi66fGvm~5bAXPc1o4HHpQpk-Cr7b0tW9lYnl3qgLoVeIICg6FLu0elbtVztgH5OS1uL6V~QhiP2PCwZf~WCHuJRQrWdPt5Kuco0lsNr1Qikk1~d7HY3ZcUTRZcMNDdem8XAFDH~ak3QER6Ml~JDkNFcLuygz~tjL4CdScVhByeAuMe3juyijtBFtYWH2h30iRkUDalg__&Key-Pair-Id=APKAJLOHF5GGSLRBV4ZA)
- A Mollahosseini et al., [AffectNet](https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=8013713&casa_token=C3QmhmiB6Y8AAAAA:1CiUll0bhIq06M17YwFIvxuse7GOosEN9G1A8vxVzR8Vb5eaFp6ERIjg7xhSIQlf008KLsfJ-w&tag=1)

#### Who are the annotators?

[Daeha Kim](mailto:[e-mail]) and [Dohee Kang](mailto:[e-mail])

## Additional Information

### Dataset Curators

DiffusionFER is created by [Daeha Kim](https://kdhht2334.github.io/) and [Dohee Kang](https://github.com/KangDohee2270).

### Acknowledgments

This repository is heavily inspired by [DiffusionDB](https://huggingface.co/datasets/poloclub/diffusiondb), with some format references. Thank you for your interest in [DiffusionDB](https://huggingface.co/datasets/poloclub/diffusiondb).

### Licensing Information

The DiffusionFER is available under the [CC0 1.0 License](https://creativecommons.org/publicdomain/zero/1.0/). NOTE: The primary purpose of this dataset is research. We are not responsible if you take any other action using this dataset.

### Contributions

If you have any questions, feel free to [open an issue](https://github.com/kdhht2334/Facial-Expression-Recognition-Zoo/issues/new) or contact [Daeha Kim](https://kdhht2334.github.io/).

原始信息汇总

DiffusionFER数据集概述

数据集基本信息

• 名称: DiffusionFER
• 语言: 英语
• 许可证: CC0-1.0
• 标签: stable diffusion, prompt engineering, prompts, research paper, facial expression recognition, emotion recognition
• 任务类别: text-to-image
• 任务ID: image-captioning, face-detection

数据集描述

• 概述: DiffusionFER是一个大规模的文本到图像提示数据库，专为面部相关任务设计。包含约1M（进行中）图像，由Stable Diffusion生成，使用特定的提示和其他参数。
• 可用性: 可通过Hugging Face Dataset访问。

下游任务和应用

• 下游任务:
  • 面部检测
  • 面部表情识别
  • 文本到情感提示
• 应用: 数据集中的虚拟主体提供了进行与面部隐私相关的多种视觉任务的机会。

数据加载

• 加载方式: 支持通过Python和Git加载。

  • Python示例: python from datasets import load_dataset dataset = load_dataset("FER-Universe/DiffusionFER")

  • Git示例: bash git lfs install git clone https://huggingface.co/datasets/FER-Universe/DiffusionFER

预训练模型

• 可用模型: 可下载并使用预训练的Swin Transformer模型，与Diffusion_Emotion_S数据集配合使用。
• 未来发布: 将发布与Diffusion_Emotion_M或Diffusion_Emotion_L配合使用的Transformer模型。

数据集结构

• 结构: 数据集采用模块化文件结构，最小规模的DiffusionEmotion_S包含约1,500张图像，分为7个情感类别文件夹。所有图像的类别标签包含在dataset_sheet.csv中。
• 文件结构示例: bash ./ ├── DifussionEmotion_S │ ├── angry │ │ ├── aaaaaaaa_6.png │ │ ├── andtcvhp_6.png │ │ ├── azikakjh_6.png │ │ ├── [...] │ ├── fear │ ├── happy │ ├── [...] │ └── surprise └── dataset_sheet.csv

提示格式

• 基本格式: "情感, 种族 年龄风格, 现实风格肖像, 性别, 上半身, 其他"。
• 示例: 一个人, 中性情感, 白人中年风格, 现实风格男性肖像, 上半身。

注释

• 注释内容: 数据集包含7个情感类别和valence-arousal值的注释。
• 注释过程: 受两篇研究论文启发进行。

数据集维护

• 维护者: Daeha Kim和Dohee Kang
• 联系方式: Daeha Kim

许可证信息

• 许可证: CC0 1.0 License
• 使用目的: 主要用于研究。