eltokh7/stata-econ-bench

Stata计量经济学基准测试数据集包含250个自然语言计量经济学和统计学任务，每个任务可通过短Stata程序解决，并通过执行生成的代码与1250个隐藏数值测试用例（每个问题5个）进行评分。这是一个基于执行的基准测试：解决方案仅当运行后能在每个案例的容差范围内重现预期数值结果时才正确，而非字符串匹配。数据集涵盖主题包括面板/FE回归、IV/2SLS、MLE、时间序列（VAR/IRF、单位根）、生存分析、矩阵代数、不平等和集中指数、诊断（VIF、Durbin–Watson、Hausman、Chow）等。数据集分为两个难度层级：109个“dev”（开发/可见）问题和141个“sacred”（保留）问题，用于模型开发和泛化评估。每个问题以JSON对象表示，包含提示、测试用例和层级信息，测试用例包括设置代码、预期输出和容差。使用需要Stata安装以执行代码进行评分。The Stata Econometrics Benchmark consists of 250 natural-language econometrics & statistics tasks, each solvable with a short Stata program and graded by executing the generated code against 1250 hidden numeric test cases (5 per problem). This is an execution-based benchmark: a solution is correct only if running it reproduces the expected numeric result within a per-case tolerance — not by string match. Topics covered include panel/FE regression, IV/2SLS, MLE, time series (VAR/IRF, unit roots), survival, matrix algebra, inequality & concentration indices, diagnostics (VIF, Durbin–Watson, Hausman, Chow), and more. The dataset has two difficulty tiers: 109 dev (development/visible) problems and 141 sacred (held-out) problems for model development and generalization estimation. Each problem is represented as a JSON object with prompt, test cases, and tier, where test cases include setup code, expected outputs, and tolerance. Usage requires a Stata installation to execute code for grading.