CLTBench: A multiple-choice question dataset on Brazilian labor law (CLT) from official examinations
收藏资源简介:
CLTBench is a dataset of 1,290 multiple-choice questions on Brazilian labor law (Consolidação das Leis do Trabalho – CLT), written in Portuguese and collected from official examinations held in Brazil between 2003 and 2026. It comprises 1,006 questions from public service entrance examinations (e.g., Regional Labor Courts and the Ministry of Labor and Employment) and 284 questions from the Brazilian Bar Examination (Exame de Ordem da OAB), prepared by four examination boards (FCC, FGV, ESAF and CESPE/CEBRASPE). All examinations required a higher-education degree. Public service examinations were selected when labor law (CLT) was listed in the official syllabus of the call for applications. For the Brazilian Bar Examination, labor-law questions were identified using commented editions of the exams, which classify each question by legal subject. Questions and official final answer keys were extracted semi-automatically from the original PDF documents published by the examination boards. Question structure (statement and options) and answer keys were then validated through peer review. Each record includes the question statement, the answer options, the correct answer, and metadata on the examination (exam, year, board, institution, position, exam type), as well as word and token counts. The dataset is provided in JSONL and CSV formats, accompanied by a README and a data dictionary, and is intended primarily as a benchmark for evaluating large language models on legal reasoning in Portuguese.



