tergite-chipcal
收藏资源简介:
tergite-chipcal benchmark chips v0.1(许可 Apache-2.0,pretty_name 为 tergite-chipcal benchmark chips v0.1)是一个面向强化学习/LLM agents 的模拟超导量子芯片校准基准数据集,也是 tergite-chipcal 多轮 agent 环境中的评估芯片集合。任务类别为 reinforcement-learning;标签包括 llm-agents、quantum-computing、superconducting-qubits、calibration、physics、benchmark。规模 n<1K,实际包含 300 个模拟芯片,约 2.0 MB。配置包括 2q1c、3q2c、5q4c,每个配置各 100 个芯片,并提供 test split JSONL:2q1c-test.jsonl、3q2c-test.jsonl、5q4c-test.jsonl。其目的是评估从零开始校准 CZ 门的 LLM agents:每个 episode 中,agent 获得一个未表征芯片,芯片由固定频率 transmon 量子比特和通量可调耦合器组成;agent 需找到读出谐振器和量子比特频率,再为每个耦合器校准 CZ 门。Agent 只能通过实验室测量交互,如 spectroscopy、chevrons、Ramsey phase scans、randomized benchmarking,每次测量消耗模拟实验室时间。评分标准是芯片所有耦合器上的最小 interleaved-RB fidelity。数据保存评估芯片的完整隐藏参数,使所有结果在同一批测试设备上报告;每个芯片均可校准,特权 oracle 认证每个耦合器至少可达 0.96 CZ fidelity,并附带每个耦合器的 ceiling。数据声明为仅模拟数据,没有物理设备行,使用 physics model v0.1,尚未与硬件验证,仓库 docs/physics.md 说明建模和简化,构建中使用了 AI 编码助手。规模细节:2q1c 为 2 qubits、1 coupler、turn cap 150、100 chips;3q2c 为 3 qubits、2 couplers、turn cap 250、100 chips;5q4c 为 5 qubits、4 couplers、turn cap 400、100 chips。每个 tier 保存从 3000000 开始的前 100 个 test seeds,且每个耦合器达到 0.96 ceiling;2q1c、3q2c、5q4c 分别拒绝了 2、3、13 个种子。三个 tier 的芯片 ceiling 中位数为 0.986、0.982、0.976。字段方面,每行一个芯片;device 字段包含 SI 单位的隐藏参数,包括量子比特频率、非谐性、T1/T2、三态读出混淆、耦合器 SQUID 参数、耦合强度、通量串扰,以及 charge-parity splits、telegraph defects、drift、calibrated-frame errors 等现实性噪声。ceiling 字段给出每个耦合器的 oracle 最佳 CZ fidelity 及达到该值的 bias、flux、脉冲频率、幅度、时长和局部相位。tier、seed、split 表示芯片大小、生成器种子和划分;generator、version 表示 chipcal-chips-v0.1 和数据集版本;manifest.json 保存 SHA-256 和选择规则。使用上,可通过 datasets 加载,例如 load_dataset(tergite/tergite-chipcal, 2q1c, split=test);运行 episode 需用 tergite-chipcal 包 import chipcal 重建带类型设备并校验 manifest,示例为 from chipcal import splits 后 splits.load(2q1c-test.jsonl)[0],也可用 python -m chipcal.splits verify . 校验并重新生成部分芯片。README 要求不要在这些芯片上训练;训练种子应来自 train split,范围 1000000–1999999(chipcal.tiers.SPLITS)。Agent 永远看不到隐藏参数,基准衡量从测量校准的能力。2q1c 基线主要结果:盲默认门策略 0.255、脚本化校准 0.853、Qwen3.8-27B 0.331、Gemma 4 31B 0.437、Gemma 4 31B + calibration playbook 0.945、NVIDIA Ising Calibration 1.5 为 0.000、特权 oracle 中位数 0.986;说明提升空间在 agent 而非物理。3q2c 和 5q4c 是相同评分下的更难设置。冻结芯片之外,生成器可程序化生成新芯片;tier 名称 <n>q-<difficulty> 可扩展到 2 到 64 个量子比特和三个难度预设,难度预设会削弱耦合、提高串扰并恶化读出和相干性;可生成训练集并可选由 oracle 认证为可校准,例如 python scripts/generate_chips.py --tier 12q-hard --n 500 --out chips 或 --tier 8q-extreme --n 50 --min-ceiling 0.96 --workers 64 --out chips。超过 10 个量子比特的 hard/extreme 芯片为实验性,可校准比例未表征。来源上,数据由 tergite-chipcal 模拟器从固定种子生成,不使用人类或硬件数据;测量节点和 CZ 校准链遵循 tergite-autocalibration,模拟器从 tergite-backend 的 pulse-level 模拟器扩展到 qubit-coupler-qubit 三能级 transmon。工作受 NVIDIA AI 量子校准工作启发,包括 QCalEval benchmark(S. Cao 等,arXiv:2604.25884,2026)和 NVIDIA Ising Calibration 模型,但未使用其代码、数据或提示。Tergite 由核心合作组开发,Chalmers Next Labs AB(CNL)管理维护;致谢 Chalmers Next Labs AB – WACQT Quantum Technology Testbed 及 Chalmers University of Technology 的 QTL、AQP、MC2。所有者 Chalmers Next Labs AB;创建日期 2026-10-26;许可 Apache-2.0。模型和产品名称属于各自所有者,仅用于识别,不暗示附属或认可。引用为 @software{tergite_chipcal_2026, title = {tergite-chipcal: a simulated superconducting-qubit chip-calibration environment for LLM agents}, author = {{Chalmers Next Labs AB}}, year = {2026}, version = {0.1.1}}。
The tergite-chipcal benchmark chips v0.1 (license Apache-2.0, pretty_name tergite-chipcal benchmark chips v0.1) is a simulated superconducting-qubit chip-calibration benchmark dataset for reinforcement learning and LLM agents, and it is the evaluation chip collection in the tergite-chipcal multi-turn agent environment. Task category is reinforcement-learning; tags include llm-agents, quantum-computing, superconducting-qubits, calibration, physics, benchmark. Size category is n<1K; it actually contains 300 simulated chips, about 2.0 MB. Configurations are 2q1c, 3q2c, 5q4c, each providing a test split JSONL file: 2q1c-test.jsonl, 3q2c-test.jsonl, 5q4c-test.jsonl. Background and purpose: the dataset evaluates LLM agents that calibrate CZ gates from scratch. In each episode the agent receives an uncharacterized chip made of fixed-frequency transmon qubits and flux-tunable couplers. The agent must find each readout resonator and qubit frequency, then calibrate CZ gates for each coupler; it can interact with the chip only through laboratory measurements such as spectroscopy, chevrons, Ramsey phase scans, and randomized benchmarking, each consuming simulated laboratory time. The scoring metric is the minimum interleaved-RB fidelity over all couplers on the chip. Data content and guarantees: the dataset stores full hidden parameters of evaluation chips so all results are reported on the same set of test devices. Each chip is calibratable; a privileged oracle certifies that each coupler can reach at least 0.96 CZ fidelity, and each chip comes with a per-coupler ceiling. The data are declared simulated only, with no row describing a physical device; it uses physics model v0.1, not yet validated against hardware, and docs/physics.md describes the modeling and simplifications. AI coding assistants were used during dataset construction. Configurations and size: 2q1c has 2 qubits, 1 coupler, turn cap 150, and 100 chips; 3q2c has 3 qubits, 2 couplers, turn cap 250, and 100 chips; 5q4c has 5 qubits, 4 couplers, turn cap 400, and 100 chips. Each tier stores the first 100 test seeds starting from 3000000; every coupler of these seeds reaches the 0.96 ceiling. The selection process rejected 2 seeds for 2q1c, 3 for 3q2c, and 13 for 5q4c. Median chip ceilings (minimum over couplers) are 0.986, 0.982, and 0.976 for the three tiers. Fields: each row is one chip. The device field contains hidden parameters in SI units (Hz, s, A): qubit frequencies, anharmonicities, T1/T2, three-state readout confusion; coupler SQUID parameters, coupling strengths, flux crosstalk; and realism noise such as charge-parity splits, telegraph defects, drift, and calibrated-frame errors. The ceiling field gives each coupler oracle-best CZ fidelity optimized over bias and pulse, including the bias, flux, pulse frequency, amplitude, duration, and local phase achieving that fidelity. tier, seed, and split fields indicate chip size, generator seed, and seed partition (test). generator and version fields indicate generator version chipcal-chips-v0.1 and dataset version. manifest.json stores SHA-256 for each file and selection rules. Usage: load with the datasets library, for example load_dataset(tergite/tergite-chipcal, 2q1c, split=test). To run an episode, use the tergite-chipcal package (import chipcal) to rebuild typed devices and verify files against the manifest; examples include from chipcal import splits, then splits.load(2q1c-test.jsonl)[0], and python -m chipcal.splits verify . checks checksums and regenerates the first few chips of each file. README explicitly asks not to train on these chips; training seeds should come from the train split with range 1000000–1999999 (chipcal.tiers.SPLITS). Parameter disclosure is a design choice: the agent never sees hidden parameters, and the benchmark measures calibration ability from measurements. Baseline results (2q1c): all baselines are evaluated on 100 chips, except Qwen is a pilot on the first 8 chips, and Gemma 4 31B (with or without playbook) and NVIDIA Ising Calibration 1.5 are on the first 10 chips. Verified entries are recorded in the leaderboard. Score/ceiling divides each final gate score by the best value reachable for that coupler. Each model turn is saved at tergite/tergite-chipcal-runs. Main results: discover-then-use blind-default-gate strategy scores 0.255 (95% CI 0.250–0.266), score/ceiling 0.259, calibration rate 0/100; scripted calibration routine scores 0.853 (0.807–0.892), score/ceiling 0.866, calibration rate 44/100; Qwen3.8-27B without policy prompt scores 0.331 (0.250–0.493), score/ceiling 0.337, calibration rate 0/8; Gemma 4 31B without policy prompt scores 0.437 (0.221–0.653), score/ceiling 0.444, calibration rate 2/10; Gemma 4 31B + calibration playbook (guided; undisclosed policy prompt) scores 0.945 (0.903–0.972), score/ceiling 0.960, calibration rate 8/10; NVIDIA Ising Calibration 1.5 (31B) without policy prompt has pending score, produced no reasoning tokens, scores 0.000 (0.000–0.000), score/ceiling 0.000, calibration rate 0/10; privileged oracle (physical ceiling) median 0.986 (0.961–0.996), score/ceiling 1, calibration rate 100/100. README notes every chip is calibratable, but the scripted routine calibrates only 44/100 and raw models at most 2/10, showing the improvement space is in the agent rather than physics. 3q2c and 5q4c tiers are harder settings under the same scoring, adding hidden flux crosstalk and weaker coupling. Extension beyond the frozen chips: the generator is programmatic and creates a new chip per seed; tier names <n>q-<difficulty> can scale to arbitrary sizes (2 to 64 qubits; chains up to 5 qubits, and grids above that, with each row having at most 5 readout feedlines) and three difficulty presets that weaken coupling, increase crosstalk, and worsen readout and coherence. Training sets can be generated and optionally certified as calibratable by the same oracle, for example python scripts/generate_chips.py --tier 12q-hard --n 500 --out chips, or python scripts/generate_chips.py --tier 8q-extreme --n 50 --min-ceiling 0.96 --workers 64 --out chips. Hard and extreme chips above 10 qubits are experimental and their calibratable proportion has not been characterized; exploration is welcome. Source and copyright: data are generated by the tergite-chipcal simulator from fixed seeds, using no human or hardware data. Measurement nodes and the CZ calibration chain follow tergite-autocalibration; the simulator starts from the pulse-level simulator in tergite-backend and extends to three-level transmons in qubit-coupler-qubit systems. The work is inspired by NVIDIA AI quantum calibration work, including the QCalEval benchmark (S. Cao et al., arXiv:2604.25884, 2026) and NVIDIA Ising Calibration models; their code, data, or prompts were not used. Tergite is developed by a core collaboration; Chalmers Next Labs AB (CNL) manages and maintains the project. Acknowledgments: Chalmers Next Labs AB – WACQT Quantum Technology Testbed, and the Quantum Technology Laboratory (QTL), Applied Quantum Physics Laboratory (AQP), and MC2 at Chalmers University of Technology. Owner: Chalmers Next Labs AB; creation date: 2026-10-26; license: Apache-2.0. Model and product names such as NVIDIA Ising Calibration, Gemma, and Qwen belong to their respective owners and are used only to identify the works and models involved, without implying any affiliation or endorsement. Citation: @software{tergite_chipcal_2026, title = {tergite-chipcal: a simulated superconducting-qubit chip-calibration environment for LLM agents}, author = {{Chalmers Next Labs AB}}, year = {2026}, version = {0.1.1}}.
tergite-chipcal benchmark chips v0.1 数据集总结
数据集概览
- 名称:tergite-chipcal benchmark chips v0.1
- 数据集页面:https://huggingface.co/datasets/tergite/tergite-chipcal
- 许可证:Apache-2.0
- 任务类别:reinforcement-learning
- 标签:llm-agents、quantum-computing、superconducting-qubits、calibration、physics、benchmark
- 大小类别:n<1K
- 总大小:2.0 MB
- 所有者:Chalmers Next Labs AB
- 创建日期:2026-10-26
- 引用版本:0.1.1
- 生成器版本:chipcal-chips-v0.1
数据集描述
- 该数据集包含 300 个冻结的、模拟的超导量子比特芯片,用于评估从零开始校准 CZ 门的 LLM agents。
- tergite-chipcal 是一个多轮 agent 环境。每个 episode 给 agent 一个未表征的芯片,由固定频率 transmon 和 flux-tunable coupler 组成。
- agent 需要找到每个读取谐振器和量子比特频率,然后校准每个耦合器上的 CZ 门。
- agent 只能通过实验室测量工作,例如 spectroscopy、chevrons、Ramsey 相位扫描和 randomized benchmarking,每种测量都消耗模拟实验室时间。
- 分数是芯片所有耦合器上的最小 interleaved-RB fidelity。
- 数据集保存评估芯片:每个测试芯片的完整隐藏参数,使所有结果都在相同设备上报告。
- 每个芯片都可校准:特权 oracle 证明其每个耦合器可实现至少 0.96 的 CZ fidelity,并且芯片的每耦合器 ceilings 随数据集提供。
- 仅包含模拟数据。没有任何行描述物理设备。物理模型 v0.1,未针对硬件验证;建模和简化内容见仓库文档中的 physics.md。
- 构建过程中使用了 AI coding assistants 的帮助。
配置与文件
| Config | test 文件 | Chip | Couplers | Turn cap | Chips |
|---|---|---|---|---|---|
2q1c |
2q1c-test.jsonl |
2 qubits | 1 | 150 | 100 |
3q2c |
3q2c-test.jsonl |
3 qubits | 2 | 250 | 100 |
5q4c |
5q4c-test.jsonl |
5 qubits | 4 | 400 | 100 |
- 每个 tier 包含前 100 个测试种子,从 3000000 起,且每个耦合器达到 0.96 ceiling。
- 选择过程拒绝了 2 个
2q1c、3 个3q2c和 13 个5q4c种子。 - 芯片 ceilings 为耦合器最小值,中位数分别为 0.986、0.982、0.976。
manifest.json包含每个文件的 SHA-256 和选择规则。
数据字段
每行对应一个芯片。
| 字段 | 描述 |
|---|---|
device |
SI 单位(Hz、s、A)下的隐藏参数:量子比特频率、非谐性、T1/T2、三态读取混淆;耦合器 SQUID 参数、耦合、通量串扰;现实噪声(电荷宇称分裂、电报缺陷、漂移、校准帧误差) |
ceiling |
每个耦合器:oracle 在 bias 和 pulse 上的最佳 CZ fidelity,以及达到该值所需的 bias、flux、脉冲频率、幅度、持续时间和局部相位 |
tier, seed, split |
芯片大小、生成器种子、种子划分(test) |
generator, version |
生成器版本(chipcal-chips-v0.1)、数据集版本 |
使用
python from datasets import load_dataset chips = load_dataset("tergite/tergite-chipcal", "2q1c", split="test")
运行 episodes 时,使用 tergite-chipcal 包重建 typed device(import chipcal),并对照 manifest 检查文件:
python from chipcal import splits tier, seed, device = splits.load("2q1c-test.jsonl")[0]
bash python -m chipcal.splits verify . # checksums, plus regeneration of the first chips of each file
- 不要在这些芯片上训练。训练种子应从 train split 抽取,范围为 1000000–1999999,见
chipcal.tiers.SPLITS。 - 参数按设计是公开的:agent 从不看到它们,benchmark 衡量的是从测量中进行校准。
基线(2q1c)
所有行在 100 个芯片上,除了模型:Qwen 试点在前 8 个,Gemma 4 31B(有/无 playbook)和 Ising Calibration 1.5 在前 10 个。已验证条目在 https://github.com/tergite/tergite-chipcal/blob/main/LEADERBOARD.md。
Score / ceiling 将每个最终门除以其耦合器允许的最佳值。每个模型轮次在 https://huggingface.co/datasets/tergite/tergite-chipcal-runs。
| Policy | Score: min final-RB F, mean (95 % CI) | Score / ceiling | Calibrated (F ≥ 0.95) |
|---|---|---|---|
| Discovery, then a blind default gate | 0.255 (0.250–0.266) | 0.259 | 0 / 100 |
| Scripted calibration routine | 0.853 (0.807–0.892) | 0.866 | 44 / 100 |
| Qwen3.8-27B, no strategy prompt | 0.331 (0.250–0.493) | 0.337 | 0 / 8 |
| Gemma 4 31B, no strategy prompt | 0.437 (0.221–0.653) | 0.444 | 2 / 10 |
| Gemma 4 31B + calibration playbook (guided: an unpublished strategy prompt) | 0.945 (0.903–0.972) | 0.960 | 8 / 10 |
| NVIDIA Ising Calibration 1.5 (31B), no strategy prompt, score pending: no reasoning tokens produced | 0.000 (0.000–0.000) | 0.000 | 0 / 10 |
| Privileged oracle (physics ceiling) | median 0.986 (0.961–0.996) | 1 | 100 / 100 |
- 每个芯片都可校准,但 scripted routine 校准了 100 个中的 44 个,原始模型最多校准 10 个中的 2 个:提升空间在 agent,而不是物理。
3q2c和5q4ctier 是更难的设置,使用相同分数。它们在耦合器之间增加了隐藏通量串扰,并具有更弱的耦合。
扩展与生成
- 生成器是程序化的:每个种子是一个新芯片,tier 名称
<n>q-<difficulty>可扩展到任意大小(2 到 64 qubits;链最多 5,超过后为 grid,其行是读取 feedlines,每行最多 5),以及三种难度预设:削弱耦合、提高串扰、恶化读取与相干性。 - 可生成训练集,并可选通过同一 oracle 认证可校准:
bash python scripts/generate_chips.py --tier 12q-hard --n 500 --out chips python scripts/generate_chips.py --tier 8q-extreme --n 50 --min-ceiling 0.96 --workers 64 --out chips
- 超过 10 qubits 的
hard和extreme芯片是实验性的:有多少可校准尚未被表征,欢迎探索。
来源
- 由 tergite-chipcal simulator 从固定种子生成,无人类或硬件数据。
- 测量节点和 CZ 校准链遵循 https://github.com/tergite/tergite-autocalibration。
- 模拟器始于 https://github.com/tergite/tergite-backend 中的 pulse-level simulator,并扩展到 qubit-coupler-qubit 系统中的三能级 transmon。
- 灵感来自 NVIDIA 在 AI for quantum calibration 方面的工作:QCalEval benchmark(S. Cao et al., https://arxiv.org/abs/2604.25884, 2026)和 NVIDIA Ising Calibration models。未使用其中任何代码、数据或 prompts。
- Tergite 由核心合作者开发;Chalmers Next Labs AB (CNL) 管理和维护该项目。感谢 Chalmers Next Labs AB – WACQT Quantum Technology Testbed,以及 Chalmers University of Technology 的 Quantum Technology Laboratory (QTL)、Applied Quantum Physics Laboratory (AQP)、MC2 的合作。
- Owner:Chalmers Next Labs AB · Created:2026-10-26 · License:Apache-2.0。
- 模型和产品名称(如 NVIDIA Ising Calibration、Gemma、Qwen)属于各自所有者。它们仅用于标识所提及的工作和模型;不暗示任何关联或背书。
引用
bibtex @software{tergite_chipcal_2026, title = {tergite-chipcal: a simulated superconducting-qubit chip-calibration environment for LLM agents}, author = {{Chalmers Next Labs AB}}, year = {2026}, version = {0.1.1} }




