styal/filtered-python-edu

---
dataset_info:
features:
- name: blob_id
dtype: string
- name: repo_name
dtype: string
- name: path
dtype: string
- name: length_bytes
dtype: int64
- name: score
dtype: float64
- name: int_score
dtype: int64
- name: text
dtype: string
- name: is_english
dtype: bool
splits:
- name: train
num_bytes: 1226088650
num_examples: 1018270
download_size: 626200552
dataset_size: 1226088650
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---
Dataset created with this filter from Avelina/python-edu.
```
def my_filter(example):
score = example["score"] > 4.1
lengh = example["length_bytes"] < 3000 and example["length_bytes"] > 200
return score and lengh
```
A is_english boolean have been added with the model papluca/xlm-roberta-base-language-detection.