ASTRA Synthetic Benchmark Dataset

Here’s a concise, copy-paste-ready Zenodo description you can use (with placeholders for the DOI and links you’ll add after deposit).

ASTRA Synthetic Benchmark Dataset (N=540)

ASTRA (Adaptive Socially-intelligent Team Reasoning Agents) Synthetic Benchmark Dataset is an open, reusable dataset designed to support trace-based evaluation of collaborative programming with socially differentiated AI agents. The dataset mirrors the logging schema and experimental structure of the ASTRA prototype, enabling researchers to develop and benchmark analytics methods for dialogue processes, participation balance in dyads, and programming-task outcomes without releasing sensitive learner data.

What the dataset contains

The dataset provides multi-granularity traces for three conditions:

alone_tutor: one learner with a Tutor agent

pair_tutor: two learners with a Tutor agent

pair_multiagent: two learners with Tutor + Facilitator agents (Facilitator prompts coordination and balanced participation)

It includes:

Turn-level dialogue logs (JSONL) with speaker roles (Student A/B, Tutor, Facilitator), timestamps, and lightweight tags (e.g., explanation/verification markers).

Task-level outcomes (CSV) for 1,440 task episodes (4 per session), including correctness, time-to-correct, retries, and verification proxies (e.g., tests run, edge-case mentions).

Session and participant metadata (CSV) supporting between-subjects comparisons and dyad-level analysis.

Task bank metadata (JSON) for 20 short Python tasks spanning list-processing (sentinel/rainfall-style), debugging, and filter–map–reduce patterns.

Derived metrics (CSV) with precomputed measures (e.g., participation imbalance, reciprocal engagement) plus a README and data dictionary describing schema and assumptions.

Intended use

ASTRA is intended for:

Method development in learning analytics, CS education, and human–AI interaction

Benchmarking models and metrics for collaboration, dialogue, and verification behaviour

Reproducible pipelines aligned to log-operational research questions such as:

interaction dynamics (turn-taking, explanation, verification, peer referencing)

participation-oriented equity proxies (balance and reciprocity in dyads)

performance proxies (correctness, time-to-correct, retries, testing behaviour)

Important note on interpretation

This release is synthetic: it simulates plausible distributions of behaviours consistent with the ASTRA schema and explicit generation assumptions. It should be treated as a benchmark for measurement feasibility and analysis sensitivity, not as observational evidence of real learners or causal effects.