trie-workloads

# trie-workloads Agentic inference benchmark traces re-hosted for use with **evalscope perf**. The traces originate from [applied-compute/trie](https://github.com/applied-compute/trie) (Apache-2.0), released alongside the blog post *"Benchmarking Inference Engines on Agentic Workloads"* (Applied Compute, April 2026). Each file is a trace-replay workload: every line describes one multi-turn agent conversation by token-length sequences and tool-call wait times, enabling reproducible serving benchmarks that capture multi-turn KV-cache reuse and tool-stall behavior absent from single (P, D) benchmarks. ## Files | File | # traces | Source domain | | --- | --- | --- | | `agentic_coding_8k.jsonl` | 8192 | Coding-agent production traces, ~8k context | | `code_qa_8k.jsonl` | 8192 | Code Q&A traces, ~8k context | | `office_work_8k.jsonl` | 8192 | Office-work agent traces, ~8k context | ## Schema (one JSON object per line) | Field | Type | Meaning | | --- | --- | --- | | `input_prompt_length` | `int` | Initial user prompt length in tokens (turn 1 input) | | `num_turns` | `int` | Number of assistant-tool turns excluding the final assistant reply | | `assistant_response_length` | `list[int]` | Tokens emitted by the assistant on each turn | | `tool_call_output_length` | `list[int]` | Tokens injected as tool output before the next assistant turn | | `tool_call_latency` | `list[float]` | Seconds the client sleeps before sending the next turn (simulated tool-call wait) | | `final_assistant_response_length` | `int` | Tokens emitted in the final assistant message | The replay rule (per trace, one concurrency slot held throughout): ``` context_0 = synth_prompt(input_prompt_length) for i in range(num_turns): resp_i = generate(context_i, max_tokens=assistant_response_length[i]) sleep(tool_call_latency[i]) tool_i = synth_prompt(tool_call_output_length[i]) context_{i+1} = context_i + resp_i + tool_i generate(context_{num_turns}, max_tokens=final_assistant_response_length) ``` Requests must set `extra_body={"ignore_eos": true}` so `min_tokens == max_tokens` and the recorded length sequences are honored exactly. ## Usage with evalscope ```bash evalscope perf \ --model <served-model> \ --url http://localhost:8000/v1/chat/completions \ --tokenizer-path <hf-or-modelscope-id> \ --dataset trace_replay \ --subset agentic_coding_8k \ --multi-turn \ --parallel 24 \ --duration 3600 \ --ignore-eos \ --stream ``` `evalscope` downloads this dataset automatically from `evalscope/trie-workloads` when `--dataset trace_replay` is selected and `--dataset-path` is not provided. ## License Apache License 2.0 — copyright Applied Compute (2026). The original `LICENSE` file is included in this repository. When citing, please credit both the original blog and this re-hosted copy. ## References - Blog: [Benchmarking Inference Engines on Agentic Workloads](https://blog.appliedcompute.ai/) — Applied Compute, April 2026 - Source: <https://github.com/applied-compute/trie>