aimamba/latvian-english-atomic-translation

--- language: - en - lv license: cc-by-4.0 task_categories: - translation tags: - latvian - english - translation - atomic-template - distillation size_categories: - 1M<n<10M --- # Latvian-English ATOMIC Translation Dataset Private dataset for distilling TildeOpen-30B into Qwen2 1.5B. ## Dataset Description 5,236,232 bidirectional Latvian↔English translation examples in ATOMIC chat JSONL format. ### Sources - OpenSubtitles (casual): 51.4% - Europarl (formal): 23.6% - WikiMatrix (encyclopedic): 18.5% - MUSE Dictionary: 3.5% - KDE4+GNOME+Ubuntu (technical): 2.9% - Tatoeba (short): 0.1% ### Format Each example is a chat-format JSONL entry: ```json {"messages": [{"role": "system", "content": "You are a Latvian-English translation assistant..."}, {"role": "user", "content": "Ko tu dari?"}, {"role": "assistant", "content": "What are you doing?"}]} ``` ### Splits - Train: 4,712,608 examples - Validation: 261,811 examples - Test: 261,813 examples ### Usage ```python from datasets import load_dataset ds = load_dataset("aimamba/latvian-english-atomic-translation", data_files={"train": "data/train.jsonl", "validation": "data/validation.jsonl", "test": "data/test.jsonl"}) ```