UNTAD-QA: A Grounded and Adversarial Query Dataset for Retrieval-Augmented Generation (RAG) Evaluation on Indonesian Institutional Documents (v3)

A 170-query Indonesian-language dataset for evaluating Retrieval-Augmented Generation (RAG) systems on institutional university documents, with a taxonomy designed to test both answer grounding and refusal behavior. The dataset covers five official Universitas Tadulako (UNTAD) documents (academic curriculum guidelines, code of ethics, academic regulations, faculty guidelines, and research/community-service guidelines) and is stratified into 110 grounded queries (across five categories — factual lookup, definitional, procedural, eligibility criteria, and multi-hop reasoning — at three difficulty levels) and 60 adversarial queries (unanswerable, at two plausibility levels: high and low), letting evaluators measure not only whether a RAG pipeline retrieves the correct answer but whether it correctly abstains when no answer exists in the source documents. All queries underwent two independent rounds of expert validation (inter-annotator agreement reported separately). Released alongside: validation recap sheets (anonymized), closing validation reports (Round 1 and Round 2), and computed reliability statistics. This research was supported by the Skema Hibah Penelitian Fundamental Reguler BIMA program, funded by Indonesia's Ministry of Higher Education, Science, and Technology (Kemdiktisaintek), 2026.