遇见数据集

GSM8K Accuracy of Low-Bit Mixtral and Qwen3 GGUFs: Dataset and Reproducibility Artifact

收藏
Zenodo2026-09-29 更新2026-10-01 收录
官方服务:

资源简介:

A mixed-license dataset and reproducibility artifact covering six exact Mixtral and Qwen3 GGUF deployments on 100 shared GSM8K items. P4R1 scored Mixtral Q4/IQ2/IQ1 at 68/46/26 and Qwen at 91/97/96. The package contains privacy-sanitized item-level outcomes, deterministic paired-statistics checks, exact model and environment provenance, replication sources, and the accompanying technical report. The experiment is corrective and exploratory, not independent; the ladders are not precision-matched, and the results establish neither an architectural cause nor Qwen-tier equivalence. Benchmark text, model responses, and weights are excluded. AI assistance: Claude (Anthropic) assisted with evaluation-harness development and experiment orchestration. Claude and Codex (OpenAI) assisted with evidence auditing, analysis and figure code, citation checking, and manuscript drafting and revision. Matthew Schwartz directed the research and is responsible for its methods, results, interpretation, claims, citations, rights, and released artifacts.

提供机构:
Zenodo
创建时间:
2026-09-29
二维码
社区交流群
二维码
科研交流群
商业服务