korean_textbooks

# Massive Korean synthetic dataset This dataset is a large-scale Korean artificial data set created using Gemini Pro. It was created using the methodology described in *Creation of synthetic textbook-quality datasets* in [Textbooks Are All You Need](https://arxiv.org/abs/2306.11644). ## Data overview **A subset of each dataset does not indicate the contents of that dataset.** **Further modification required before use this dataset for training.** **본 데이터셋은 바로 사용하기보다는 하고자하는 task에 맞추어 가공 후 사용을 권장드립니다. ex) 로컬 모델을 사용하여 QA 셋으로 변환** | subset | row count | link | + | |---|---|---|---| | tiny-textbooks | 395,985 | [nampdn-ai/tiny-textbooks](https://huggingface.co/datasets/nampdn-ai/tiny-textbooks) | | | ko_wikidata | 127,614 | [maywell/ko_wikidata_QA](https://huggingface.co/datasets/maywell/ko_wikidata_QA) | | | normal_instructions | 240,523 | [KonstantyM/science_qa](https://huggingface.co/datasets/KonstantyM/science_qa) | with science texts | | claude_evol | 239,102 | [Norquinal/claude_evol_instruct_210k](https://huggingface.co/datasets/Norquinal/claude_evol_instruct_210k) | used 250k files from that repo | | code-alpaca | 64,112 | [theblackcat102/evol-codealpaca-v1](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1) | original is a coding dataset, but generated data is not mainly a coding dataset | | helpsteer | 25,253 | [nvidia/HelpSteer](https://huggingface.co/datasets/nvidia/HelpSteer) | | | mmlu_abstract_algebra | 88,848 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_all | 97,765 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_anatomy | 97,463 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_astronomy | 97,347 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_business_ethics | 97,327 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_clinical_knowledge | 97,226 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_college_biology | 97,285 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_college_chemistry | 97,435 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_college_computer_science | 92,606 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_college_mathematics | 94,070 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_college_medicine | 95,156 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_college_physics | 97,452 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_computer_security | 97,212 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_conceptual_physics | 88,216 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_econometrics | 91,854 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_electrical_engineering | 87,826 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_elementary_mathematics | 89,307 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_formal_logic | 95,483 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_global_facts | 94,984 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_biology | 97,117 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_chemistry | 96,907 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_computer_science | 97,351 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_european_history | 97,222 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_geography | 97,261 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_government_and_politics | 97,311 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_macroeconomics | 97,400 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_mathematics | 97,396 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_microeconomics | 97,435 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_physics | 95,740 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_psychology | 80,626 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_statistics | 76,033 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_us_history | 79,322 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_high_school_world_history | 85,990 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_human_aging | 78,341 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_human_sexuality | 79,327 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | | mmlu_international_law | 78,989 | [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) | | ## When you find a problem If you find any issues with the dataset, please let us know in the discussion or send us a pull request.