dataset for paper-Why Sparse MoE Fails

This figure visualizes the core empirical finding from our experimental dataset: validation perplexity comparison between Dense Transformers and sparse Mixture-of-Experts (MoE) models across three data scales (100K, 500K, and 4.5M sentence pairs) on WMT14 En-De translation. Each point represents the mean of 5 random seeds with 95% confidence intervals. The dataset contains complete training logs, metrics, and statistical analyses demonstrating that dense models consistently achieve lower perplexity (better performance) than parameter-matched MoE models at all tested scales, contradicting widespread assumptions about MoE sample efficiency in neural machine translation.