carnice-agent-trance-prompt-bank

# Carnice Agent Trace Prompt Bank This repository is a curated prompt bank for collecting agent traces. It is **not** a trace dataset by itself. It is the input side: prompts that can be run through an agent harness, then logged into traces with tool calls, observations, and final answers. The goal of this release is practical: - keep prompts that work well in an agent harness - remove prompts that assume hidden local state or user-private state - expand browser and long-horizon tasks enough to support serious trace collection ## What this contains The main release is `v4`: - `carnice_trace_prompt_bank_v4.jsonl` - full bank with routing metadata - `4,033` prompts - `carnice_trace_prompt_bank_v4.prompts_only.jsonl` - plain `prompt_id` + `prompt` - `carnice_trace_prompt_bank_v4.local.jsonl` - `1,983` local/non-web prompts - `carnice_trace_prompt_bank_v4.web.jsonl` - `2,050` web prompts - `carnice_trace_prompt_bank_v4.long_horizon.jsonl` - `483` harder long-horizon web prompts - `carnice_trace_prompt_bank_v4.fixtures.jsonl` - `25` fixture rows for prompts that need seeded local files - `carnice_trace_prompt_bank_v4.manifest.json` - counts and source breakdown ## Why this exists Most public prompt banks are not directly usable for agent trace collection. Common failure modes: - prompts assume a hidden repo or local file tree - prompts assume a private account, order, reservation, or inbox - prompts are benchmark-style but not harness-ready - browser tasks drift into auth/signup/checkout flows This dataset was built by curating and filtering several public sources into one bank that works better in a real agent harness. ## How it was built This release combines prompt material from multiple upstream sources and applies filtering plus routing labels. High-level process: 1. Start from a narrow local-first prompt core for coding, terminal, planning, and seeded fixtures. 2. Remove broad instruction-following and customer-support style prompt families that are not useful for trace collection. 3. Add filtered browser tasks from WebGym. 4. Add a smaller long-horizon slice from AssistantBench. 5. Remove tasks that require hidden user state, account login, signup, or private context. 6. Keep the result in a format that can be directly run through an agent harness. ## Source mix in `v4` From `carnice_trace_prompt_bank_v4.manifest.json`: - `local_gist/terminal_only.accepted`: `1,858` - `local_execute_code_probe`: `20` - `Danau5tin/terminal-tasks`: `105` - `microsoft/webgym_tasks`: `1,945` - `local_gist/browser_only.accepted`: `72` - `AssistantBench/AssistantBench`: `33` ## Row schema Main fields in `carnice_trace_prompt_bank_v4.jsonl`: - `prompt_id`: stable identifier - `prompt`: prompt text - `category`: coarse task family - `mode`: `local`, `web`, or `local_fixture` - `requires_web`: whether web access is expected - `requires_fixture`: whether seeded files are required - `source`: source family - `long_horizon`: whether the prompt was tagged as a harder long-horizon task ## How to use it 1. Choose a row from `carnice_trace_prompt_bank_v4.jsonl` or `carnice_trace_prompt_bank_v4.prompts_only.jsonl`. 2. If `requires_fixture=true`, seed the matching files from `carnice_trace_prompt_bank_v4.fixtures.jsonl`. 3. Run the prompt through your agent harness. 4. Save the resulting messages, tool calls, tool outputs, and final answer as the trace. ## Important note on licenses This release is a curated combination of upstream prompt sources. Source and attribution notes are in `SOURCES.md`. The overall release is marked `license: other` because it mixes multiple upstream sources with different licensing situations, including user-provided local prompt archives. Review the source notes before republishing or using this release in a stricter commercial/compliance setting.