d3-ultra-decision-index
收藏资源简介:
# d3-ultra: Decision Index 0.3 public runs Results of [vllm-sr/d3-ultra](https://huggingface.co/vllm-sr/d3-ultra) at revision `37fee01926f3de396620e794201b074fb427ac2d` on the complete Decision Index 0.3 public suite and the complete public vision suite, run with the unmodified kit [apolinario/decision-index](https://github.com/apolinario/decision-index) `25c6ec7` (edition 0.3). The model repository is private until the weights are released; it opens at that address, at that revision. Both runs used that revision's files. - **`runs/d3-ultra/`**: public index **71.57** (raw 78.59); 140,178 scoreable requests, all answered, 0 requests unsupported; `scores.json` says complete: true. It holds `scores.json`, `index.json`, `benchmark-summary.json`, `results.jsonl.gz` (the answers to every request), `environment.json`, `status.json`, `RUN_NOTES.md` (engine, settings, hardware) and `SHA256SUMS`. - **`runs/d3-ultra-vision/`**: public vision index **75.33**; all 12,210 rows answered; `vision-scores.json` says complete: true; images at native resolution capped at 1.6 MP. It holds `results.jsonl` (every answer), `vision-scores.json`, `environment.json`, `status.json`, `server.json` (the server's own `/health` and `/v1/models` replies), `model-files.json` (name and SHA-256 of every file the run loaded), `RUN_NOTES.md` and `SHA256SUMS`. - Re-scoring `results.jsonl.gz` with `decision_index score --edition 0.3 --engine d3-ultra@37fee01926f3de396620e794201b074fb427ac2d` reproduces `scores.json`, `index.json` and `benchmark-summary.json` (apart from `generated_utc`); re-scoring the vision `results.jsonl` with `decision-index vision score` reproduces `vision-scores.json`. - Hardware: AMD Instinct MI325X. Single-request latency on one AMD Instinct MI325X (one process, one request at a time after warm-up): median 95.2 ms per text request, 237.9 ms per request with an image.



