RyokoExtra/LFANIME

---
license: cc
tags:
- art
- anime
pretty_name: LFAnime
task_categories:
- image-classification
- text-to-image
---

# Dataset Card for LFANIME

A dataset of anime frames collected by KaraKaraWitch.

## Dataset Details

### Dataset Description

LFANIME, or Low-Framerate Anime, comprises frames from Japanese animation. The dataset serves dual purposes—facilitating fine-tuning of image diffusion models and functioning as a pre-training resource. Moreover, we anticipate its utilization in image classification.

Important Note: LFAnime is not intended for watching anime. To discourage this application, we have intentionally lowered the frame rate and excluded audio from the dataset.

- **Curated by:** KaraKaraWitch
- **Funded by [optional]:** N/A
- **Shared by [optional]:** N/A
- **Language(s) (NLP):** Nil. Primarily japanese, but no audio is included.
- **License:** CC

## Uses

A tar file compresses each "Episode," encompassing sequential anime frames. The dataset also incorporates chapters for episodes that have them. It's important to note that certain frame numbers may be absent intentionally.

### Direct Use

<!-- This section describes suitable use cases for the dataset. -->

We release this dataset for free in the hopes that it could be used for text to image generation and/or image classification.

### Out-of-Scope Use

Technically speaking, this dataset could be used to watch anime. However we do not recommend as such.
Additionally there could be unforseen usage that the author does not intend.

<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. -->

## Dataset Structure

<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->

Each tar file should generally follow this format `LFAnime-[T(Test),A(Alpha),B(Beta),R(Release)]-[Sequential Index]-[AnilistID]-[Episode]`

Each tar file should contain:
```
frame_[XXXX]_[detection_type]_[seconds (float)].jpg
kframes.log (scxvid keyframe log)
metadata.json (Selected frames + Detection metrics + Mode)
```

`detection_type` can be one of the following:
```
- key (KeyFrame)
- p_key (Previous Frame from Key Frame)
- inter (Inter frame)
```

## Dataset Creation

### Curation Rationale

<!-- Motivation for the creation of this dataset. -->

The emphasis has been on developing models for generating images from text, particularly in the realm of creating "anime"-style visuals.
Examples of such models include Waifu Diffusion and NovelAI's SD 1.x models. Regrettably, these models tend to converge, resulting in a consistent aesthetic.
While this aesthetic may appeal to many users, it poses a challenge when attempting to diverge from or fine-tune the ingrained visual style of most SD 1.x models.

### Source Data

<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). -->

#### Data Collection and Processing

<!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. -->

We've opted not to reveal the specific origins of the anime to establish a level of separation between the producers and this dataset.
Nevertheless, we can outline the processing steps as follows:

1. Extract frames from the mkv file, sampling every 10 frames per second.
2. Utilize scxvid to generate a timecode for identifying scene cuts.
3. Exclude frames that precede or follow a scene cut (considering potential inclusion of 1/2 frames at each scene cut).
4. Save the processed frames to a tar file.

#### Who are the source data producers?

We have decided not to disclose the exact sources.

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

As this dataset is a personal collection from KaraKaraWitch, it will have tendencies to generally not "Shonen" anime and will have female protagonists in general.

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users should be made aware of the risks, biases and limitations of the dataset.

## Citation [optional]

```
@misc{lfanime,
title = {LFAnime: A Low Framerate anime dataset.},
author = {KaraKaraWitch},
year = {2023},
howpublished = {\url{https://huggingface.co/datasets/RyokoExtra/LFANIME}},
}
```

## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the dataset or dataset card. -->

Anime:

> Anime (Japanese: アニメ, IPA: [aꜜɲime]) is hand-drawn and computer-generated animation originating from Japan. Outside Japan and in English, anime refers specifically to animation produced in Japan.[1] However, in Japan and in Japanese, anime (a term derived from a shortening of the English word animation) describes all animated works, regardless of style or origin. Many works of animation with a similar style to Japanese animation are also produced outside Japan. Video games sometimes also feature themes and artstyles that can be considered as "anime".
> - Wikipedia

### Contributions

- [@KaraKaraWitch (Twitter)](https://twitter.com/KaraKaraWitch) for gathering this dataset.
- [ChatGPT](https://chat.openai.com) rewording sentences in this datacard.

原始信息汇总

数据集卡片 for LFANIME

数据集详情

数据集描述

LFANIME，或称为低帧率动漫，包含来自日本动画的帧。该数据集有两个主要用途：促进图像扩散模型的微调和作为预训练资源。此外，我们还预计它将用于图像分类。

重要提示：LFAnime 不适用于观看动漫。为了防止这种应用，我们故意降低了帧率并从数据集中排除了音频。

• 由： KaraKaraWitch 策划
• 语言(s) (NLP)： 无。主要为日语，但不包括音频。
• 许可证： CC

用途

每个“剧集”包含一系列动漫帧，并以 tar 文件压缩。数据集还包括有章节的剧集。需要注意的是，某些帧号可能故意缺失。

直接用途

我们免费发布此数据集，希望它可以用于文本到图像生成和/或图像分类。

超出范围的用途

从技术上讲，此数据集可以用于观看动漫，但我们不建议这样做。此外，可能存在作者未预见的用途。

数据集结构

每个 tar 文件通常遵循以下格式 LFAnime-[T(Test),A(Alpha),B(Beta),R(Release)]-[Sequential Index]-[AnilistID]-[Episode]

每个 tar 文件应包含：

frame_[XXXX][detection_type][seconds (float)].jpg kframes.log (scxvid keyframe log) metadata.json (Selected frames + Detection metrics + Mode)

detection_type 可以是以下之一：

• key (KeyFrame)
• p_key (Previous Frame from Key Frame)
• inter (Inter frame)

数据集创建

策划理由

重点在于开发从文本生成图像的模型，特别是在创建“动漫”风格视觉效果的领域。例如，Waifu Diffusion 和 NovelAI 的 SD 1.x 模型。遗憾的是，这些模型往往收敛，导致一致的美学风格。虽然这种风格可能吸引许多用户，但在尝试偏离或微调大多数 SD 1.x 模型的固有视觉风格时，它构成了一项挑战。

源数据

数据收集和处理

我们选择不透露动漫的具体来源，以在制作方和此数据集之间建立一定程度的分离。尽管如此，我们可以概述处理步骤如下：

1. 从 mkv 文件中提取帧，每秒采样 10 帧。
2. 使用 scxvid 生成用于识别场景切割的时间码。
3. 排除场景切割前后的帧（考虑在每个场景切割处可能包含 1/2 帧）。
4. 将处理后的帧保存到 tar 文件中。

源数据生产者是谁？

我们决定不透露确切的来源。

偏差、风险和限制

由于此数据集是 KaraKaraWitch 的个人收藏，它通常不会偏向“少年”动漫，并且通常会有女性主角。

建议

用户应了解数据集的风险、偏差和技术限制。