airesearch/thai-bar-exam-judging

该数据集是一个泰语司法考试评分语料库，包含来自司法考试准备练习的匿名自由形式泰语法律文章。每篇文章由三位经过泰国律师理事会培训的考官进行评分，并对大约三分之二的答案提供了跨度锚定的内联评论。八名LLM考生在相同条件下参加了同一考试；他们的答案由相同的考官进行盲评。150个答案中的15个由两位非主要考官交叉评分，形成了包含3名评分者稳定性的子集，这是配套论文的基础。该语料库旨在支持LLM-as-judge与人类考官稳定性的研究，适合作为泰语法律NLP基准、跨度锚定批判建模数据集以及评分者间一致性研究的素材。

Anonymised free-form Thai legal essays from a bar-exam preparation exercise, with three Bar Council-trained examiners scoring every essay and span-anchored inline commentary on roughly two thirds of the answers. Eight LLM examinees took the same exam under the same conditions; their answers were graded blind by the same examiners. Fifteen of the 150 answers were cross-graded by the two non-primary examiners, producing the 3-rater stability subset that underlies the accompanying paper. This corpus was assembled to support a study of LLM-as-judge stability against human examiners. It is suitable as a Thai legal-NLP benchmark, a span-anchored critique-modeling dataset, and a substrate for inter-rater agreement research.