CoDeTT

# CoDeTT: A Context-Aware Decision Benchmark for Turn-Taking Evaluation

## 🌐 Dataset Summary

CoDeTT is a benchmark dataset for turn-taking decision evaluation in full-duplex spoken dialogue systems.
It evaluates not only what action a model should take at the current moment, but also whether the underlying semantic intent is aligned.

Core action space (4 classes):
- `Maintain`
- `Stop & Listen`
- `Takeover`
- `Dismiss`

Fine-grained intent space:
- 14 scenario labels across two system states: `SystemSpeaking` and `SystemIdle`.

## 🧩 Task Definition

Given:
- Current system state
- Multi-turn dialogue history
- Current target input

Predict:
- Turn-taking action at the current timestamp (4-way)
- Optional fine-grained intent category (14-way)

## 🏷️ Label Taxonomy

### SystemSpeaking
- `Maintain`:
- `Backchannel`
- `Invalidation`
- `Side-talk`
- `Distraction`
- `Stop & Listen`:
- `Interruption`
- `Dismissal`
- `Collaboration`

### SystemIdle
- `Takeover`:
- `Completion`
- `Cooperation`
- `Dismiss`:
- `Incomplete`
- `Invalidation`
- `Dismissal`
- `Exclusion`
- `Side-talk`

## 🗂️ Data Structure (Full Directory Tree)

```text
Benchmark_Datasets
├── .cache
├── audios
│ ├── EN
│ │ ├── real
│ │ │ ├── System_Idle
│ │ │ │ ├── Dismiss
│ │ │ │ │ └── Incomplete
│ │ │ │ └── Takeover
│ │ │ │ └── Completion
│ │ │ └── System_Speaking
│ │ │ ├── Maintain
│ │ │ │ └── Backchannel
│ │ │ └── Stop_Listen
│ │ │ └── Interruption
│ │ └── syn
│ │ ├── SystemIdle
│ │ │ ├── Dismiss
│ │ │ │ ├── Dismissal
│ │ │ │ ├── Exclusion
│ │ │ │ ├── Incomplete
│ │ │ │ ├── Invalidation
│ │ │ │ └── SideTalk
│ │ │ └── Takeover
│ │ │ ├── Completion
│ │ │ └── Cooperation
│ │ └── SystemSpeaking
│ │ ├── Maintain
│ │ │ ├── Backchannel
│ │ │ ├── Distraction
│ │ │ ├── Invalidation
│ │ │ └── SideTalk
│ │ └── StopandListen
│ │ ├── Collaboration
│ │ ├── Dismissal
│ │ └── Interruption
│ └── ZH
│ ├── real
│ │ ├── System_Idle
│ │ │ ├── Dismiss
│ │ │ │ └── Incomplete
│ │ │ └── Takeover
│ │ │ └── Completion
│ │ └── System_Speaking
│ │ ├── Maintain
│ │ │ └── Backchannel
│ │ └── Stop_Listen
│ │ └── Interruption
│ └── syn
│ ├── SystemIdle
│ │ ├── Dismiss
│ │ │ ├── Dismissal
│ │ │ ├── Exclusion
│ │ │ ├── Incomplete
│ │ │ ├── Invalidation
│ │ │ └── SideTalk
│ │ └── Takeover
│ │ ├── Completion
│ │ └── Cooperation
│ └── SystemSpeaking
│ ├── Maintain
│ │ ├── Backchannel
│ │ ├── Distraction
│ │ ├── Invalidation
│ │ └── SideTalk
│ └── StopandListen
│ ├── Collaboration
│ ├── Dismissal
│ └── Interruption
└── jsonls
├── EN
│ ├── real
│ │ ├── SystemIdle_Dismiss_Incomplete.jsonl
│ │ ├── SystemIdle_Takeover_Completion.jsonl
│ │ ├── SystemSpeaking_Maintain_Backchannel.jsonl
│ │ └── SystemSpeaking_StopandListen_Interruption.jsonl
│ └── syn
│ ├── SystemIdle_Dismiss_Dismissal.jsonl
│ ├── SystemIdle_Dismiss_Exclusion.jsonl
│ ├── SystemIdle_Dismiss_Incomplete.jsonl
│ ├── SystemIdle_Dismiss_Invalidation.jsonl
│ ├── SystemIdle_Dismiss_SideTalk.jsonl
│ ├── SystemIdle_Takeover_Completion.jsonl
│ ├── SystemIdle_Takeover_Cooperation.jsonl
│ ├── SystemSpeaking_Maintain_Backchannel.jsonl
│ ├── SystemSpeaking_Maintain_Distraction.jsonl
│ ├── SystemSpeaking_Maintain_Invalidation.jsonl
│ ├── SystemSpeaking_Maintain_SideTalk.jsonl
│ ├── SystemSpeaking_StopandListen_Collaboration.jsonl
│ ├── SystemSpeaking_StopandListen_Dismissal.jsonl
│ └── SystemSpeaking_StopandListen_Interruption.jsonl
└── ZH
├── real
│ ├── SystemIdle_Dismiss_Incomplete.jsonl
│ ├── SystemIdle_Takeover_Completion.jsonl
│ ├── SystemSpeaking_Maintain_Backchannel.jsonl
│ └── SystemSpeaking_StopandListen_Interruption.jsonl
└── syn
├── SystemIdle_Dismiss_Dismissal.jsonl
├── SystemIdle_Dismiss_Exclusion.jsonl
├── SystemIdle_Dismiss_Incomplete.jsonl
├── SystemIdle_Dismiss_Invalidation.jsonl
├── SystemIdle_Dismiss_SideTalk.jsonl
├── SystemIdle_Takeover_Completion.jsonl
├── SystemIdle_Takeover_Cooperation.jsonl
├── SystemSpeaking_Maintain_Backchannel.jsonl
├── SystemSpeaking_Maintain_Distraction.jsonl
├── SystemSpeaking_Maintain_Invalidation.jsonl
├── SystemSpeaking_Maintain_SideTalk.jsonl
├── SystemSpeaking_StopandListen_Collaboration.jsonl
├── SystemSpeaking_StopandListen_Dismissal.jsonl
└── SystemSpeaking_StopandListen_Interruption.jsonl
```

## 📦 Quick Start

```bash
mkdir -p ./CoDeTT && lz4 -dc ./CoDeTT.lz4 | tar -x -C ./CoDeTT
```

## 📏 Evaluation Protocol

### Stage 1: Action Level
All models are evaluated in the unified 4-action space (Takeover / Maintain / Stop & Listen / Dismiss), enabling fair comparison across different model paradigms.

### Stage 2: Intent Level
Models with stronger semantic reasoning capabilities are further evaluated on direct prediction of all 14 fine-grained intent categories.

## 📊 Metrics

- `ACC` (Accuracy): Action-level prediction accuracy
- `SMR` (Semantic Misalignment Rate): The proportion of semantically incorrect intent predictions among action-correct samples

## 🚀 Use Cases

- Turn-taking evaluation for full-duplex voice assistants and speech agents
- Context understanding analysis for speech-language models in complex conversations
- Multi-speaker interaction and floor-management research
- Diagnosis of "action-correct but reason-wrong" behaviors
- Bilingual and cross-lingual turn-taking benchmarking (English/Chinese)

## 📚 Citation

If you use CoDeTT or reference its benchmark design, please cite:

```bibtex
@article{shen2026codett,
title={CoDeTT: A Context-Aware Decision Benchmark for Turn-Taking Evaluation},
author={Shen, Huan and Wang, Yingao and Huang, Shangkun and Zou, Wei and Chen, Yunzhang},
journal={arXiv preprint arXiv:2603.25434},
year={2026}
}
```

# CoDeTT: 面向轮次交互评估的上下文感知决策基准数据集

## 🌐 数据集概述
CoDeTT是一款面向全双工语音对话系统（full-duplex spoken dialogue systems）轮次交互决策评估的基准数据集。该数据集不仅可评估模型在当前时刻应采取的动作，还能检验其背后的语义意图是否匹配一致。

核心动作空间（共4类）：
- `维持当前动作（Maintain）`
- `停止并倾听（Stop & Listen）`
- `接管话语权（Takeover）`
- `终止交互（Dismiss）`

细粒度意图空间包含覆盖两种系统状态的14个场景标签：系统发言中（SystemSpeaking）与系统空闲时（SystemIdle）。

## 🧩 任务定义
输入：
- 当前系统状态
- 多轮对话历史
- 当前目标输入
输出：
- 当前时间戳下的对话轮次动作（4分类任务）
- 可选的细粒度意图类别（14分类任务）

## 🏷️ 标签分类体系
### 系统发言中（SystemSpeaking）
- `维持当前动作（Maintain）`:
- `反馈语（Backchannel）`
- `无效回应（Invalidation）`
- `旁支对话（Side-talk）`
- `干扰性发言（Distraction）`
- `停止并倾听（Stop & Listen）`:
- `插话（Interruption）`
- `终止交互（Dismissal）`
- `协作性回应（Collaboration）`

### 系统空闲时（SystemIdle）
- `接管话语权（Takeover）`:
- `完成接续（Completion）`
- `协作接续（Cooperation）`
- `终止交互（Dismiss）`:
- `未完成交互（Incomplete）`
- `无效回应（Invalidation）`
- `终止交互（Dismissal）`
- `排除性回应（Exclusion）`
- `旁支对话（Side-talk）`

## 🗂️ 数据结构（完整目录树）
text
Benchmark_Datasets
├── .cache
├── audios
│ ├── EN
│ │ ├── real
│ │ │ ├── System_Idle
│ │ │ │ ├── Dismiss
│ │ │ │ │ └── Incomplete
│ │ │ │ └── Takeover
│ │ │ │ └── Completion
│ │ │ └── System_Speaking
│ │ │ ├── Maintain
│ │ │ │ └── Backchannel
│ │ │ └── Stop_Listen
│ │ │ └── Interruption
│ │ └── syn
│ │ ├── SystemIdle
│ │ │ ├── Dismiss
│ │ │ │ ├── Dismissal
│ │ │ │ ├── Exclusion
│ │ │ │ ├── Incomplete
│ │ │ │ ├── Invalidation
│ │ │ │ └── SideTalk
│ │ │ └── Takeover
│ │ │ ├── Completion
│ │ │ └── Cooperation
│ │ └── SystemSpeaking
│ │ ├── Maintain
│ │ │ ├── Backchannel
│ │ │ ├── Distraction
│ │ │ ├── Invalidation
│ │ │ └── SideTalk
│ │ └── StopandListen
│ │ ├── Collaboration
│ │ ├── Dismissal
│ │ └── Interruption
│ └── ZH
│ ├── real
│ │ ├── System_Idle
│ │ │ ├── Dismiss
│ │ │ │ └── Incomplete
│ │ │ └── Takeover
│ │ │ └── Completion
│ │ └── System_Speaking
│ │ ├── Maintain
│ │ │ └── Backchannel
│ │ └── Stop_Listen
│ │ └── Interruption
│ └── syn
│ ├── SystemIdle
│ │ ├── Dismiss
│ │ │ ├── Dismissal
│ │ │ ├── Exclusion
│ │ │ ├── Incomplete
│ │ │ ├── Invalidation
│ │ │ └── SideTalk
│ │ └── Takeover
│ │ ├── Completion
│ │ └── Cooperation
│ └── SystemSpeaking
│ ├── Maintain
│ │ ├── Backchannel
│ │ ├── Distraction
│ │ ├── Invalidation
│ │ └── SideTalk
│ └── StopandListen
│ ├── Collaboration
│ ├── Dismissal
│ └── Interruption
└── jsonls
├── EN
│ ├── real
│ │ ├── SystemIdle_Dismiss_Incomplete.jsonl
│ │ ├── SystemIdle_Takeover_Completion.jsonl
│ │ ├── SystemSpeaking_Maintain_Backchannel.jsonl
│ │ └── SystemSpeaking_StopandListen_Interruption.jsonl
│ └── syn
│ ├── SystemIdle_Dismiss_Dismissal.jsonl
│ ├── SystemIdle_Dismiss_Exclusion.jsonl
│ ├── SystemIdle_Dismiss_Incomplete.jsonl
│ ├── SystemIdle_Dismiss_Invalidation.jsonl
│ ├── SystemIdle_Dismiss_SideTalk.jsonl
│ ├── SystemIdle_Takeover_Completion.jsonl
│ ├── SystemIdle_Takeover_Cooperation.jsonl
│ ├── SystemSpeaking_Maintain_Backchannel.jsonl
│ ├── SystemSpeaking_Maintain_Distraction.jsonl
│ ├── SystemSpeaking_Maintain_Invalidation.jsonl
│ ├── SystemSpeaking_Maintain_SideTalk.jsonl
│ ├── SystemSpeaking_StopandListen_Collaboration.jsonl
│ ├── SystemSpeaking_StopandListen_Dismissal.jsonl
│ └── SystemSpeaking_StopandListen_Interruption.jsonl
└── ZH
├── real
│ ├── SystemIdle_Dismiss_Incomplete.jsonl
│ ├── SystemIdle_Takeover_Completion.jsonl
│ ├── SystemSpeaking_Maintain_Backchannel.jsonl
│ └── SystemSpeaking_StopandListen_Interruption.jsonl
└── syn
├── SystemIdle_Dismiss_Dismissal.jsonl
├── SystemIdle_Dismiss_Exclusion.jsonl
├── SystemIdle_Dismiss_Incomplete.jsonl
├── SystemIdle_Dismiss_Invalidation.jsonl
├── SystemIdle_Dismiss_SideTalk.jsonl
├── SystemIdle_Takeover_Completion.jsonl
├── SystemIdle_Takeover_Cooperation.jsonl
├── SystemSpeaking_Maintain_Backchannel.jsonl
├── SystemSpeaking_Maintain_Distraction.jsonl
├── SystemSpeaking_Maintain_Invalidation.jsonl
├── SystemSpeaking_Maintain_SideTalk.jsonl
├── SystemSpeaking_StopandListen_Collaboration.jsonl
├── SystemSpeaking_StopandListen_Dismissal.jsonl
└── SystemSpeaking_StopandListen_Interruption.jsonl

## 📦 快速上手
bash
mkdir -p ./CoDeTT && lz4 -dc ./CoDeTT.lz4 | tar -x -C ./CoDeTT

## 📏 评估协议
### 阶段1：动作层级
所有模型将在统一的4类动作空间（接管话语权/维持当前动作/停止并倾听/终止交互）中进行评估，以实现不同模型范式间的公平对比。
### 阶段2：意图层级
具备更强语义推理能力的模型，将进一步在全部14个细粒度意图类别上开展直接预测评估。

## 📊 评估指标
- `ACC（准确率，Accuracy）`：动作层级预测准确率
- `SMR（语义失配率，Semantic Misalignment Rate）`：动作预测正确的样本中，语义意图预测错误的比例

## 🚀 应用场景
- 全双工语音助手与AI智能体（AI Agent）的轮次交互评估
- 复杂对话场景下语音语言模型的上下文理解分析
- 多说话人交互与话语权管理研究
- “动作正确但推理错误”行为的诊断分析
- 双语及跨语言轮次交互基准测试（覆盖英语/中文）

## 📚 引用规范
若您使用CoDeTT或参考其基准数据集设计，请引用以下文献：
bibtex
@article{shen2026codett,
title={CoDeTT: A Context-Aware Decision Benchmark for Turn-Taking Evaluation},
author={Shen, Huan and Wang, Yingao and Huang, Shangkun and Zou, Wei and Chen, Yunzhang},
journal={arXiv preprint arXiv:2603.25434},
year={2026}
}