Neural-Navigator-Labs/wmt14_en-de
收藏官方服务：
资源简介：
--- language: - en - de task_categories: - translation pretty_name: SmolTrans EN-DE Clean Dataset size_categories: - 1M<n<10M --- # SmolTrans EN-DE Dataset High-quality English–German parallel dataset filtered using: - Language detection - HTML removal - Duplicate removal - LM perplexity filtering - Perplexity ratio filtering - Embedding cosine similarity ## Statistics - Total pairs: 4.5M - Median length: ... - Filtering removed: ~X% ## Format {"en": "...", "de": "..."} ## Usage from datasets import load_dataset ds = load_dataset("<your-username>/smoltrans-en-de") ## Notes Short sentences may show unstable perplexity values.
