ezipe/lichess_2023_janoct_shards

---
license: apache-2.0
---

## Installation
```
pip install datasets numpy zstandard
```

## Usage

i've given up on trying to get this to work natively with huggingface datasets. this is nice because it allows streaming (https://huggingface.co/docs/datasets/en/about_mapstyle_vs_iterable) and has functions like 'map' which easily parallelize operations over the dataset. maybe i'll try to get this working in the future, but for now it gets stuck as the download and extracted zstd files are not decompressible for some reason, and it has to rewrite the entire dataset into arrow first.

<!-- ```
from datasets import load_dataset

dataset = load_dataset("ezipe/lichess_2023_janoct_shards")

``` -->

or more slowly,

```
kconda install git-lfs
git lfs install
git clone https://huggingface.co/datasets/ezipe/lichess_2023_janoct_shards/
```

原始信息汇总

数据集概述

数据集名称

• 名称: lichess_2023_janoct_shards

数据集来源

• 来源: Hugging Face Datasets

许可证

• 许可证: Apache-2.0

安装要求

• 依赖安装: 需要安装 datasets, numpy, zstandard 库。

数据集加载方式

• 直接加载: 使用 datasets 库加载数据集。
• 通过Git LFS下载: 通过Git LFS安装并克隆数据集。