reglab/housing_qa

HousingQA是一个评估大型语言模型（LLM）在不同州回答关于住房法律问题的能力的基准。该数据集包含三个子集：questions, questions_aux, 和 statutes。questions子集包含6853个关于各司法管辖区2021年住房法律的yes/no问题。questions_aux子集是一个更大的问题集，包含所有questions子集中的样本以及更多样本，但不包含法律条文注释，因此不能用于评估RAG。statutes子集包含大约170万条从Justia收集的法律条文，可用于评估完整的RAG管道。请注意，所有问题和法律条文仅准确到2021年，因此数据集中的内容不应被视为法律建议或用于提供法律建议。

HousingQA is a benchmark evaluating the ability of large language models (LLMs) to answer questions about housing law in different states. The dataset consists of three subsets: questions, questions_aux, and statutes. The questions subset contains 6853 yes/no questions about the housing law of each jurisdiction in 2021. The questions_aux subset is a larger set of questions containing all samples in questions and more, but without statute annotations, so it cannot be used for evaluating RAG. The statutes subset contains approximately 1.7 million statutes collected from Justia, which can be used to evaluate full RAG pipelines. Please note that all questions and statutes are accurate only as of 2021, and the content in the dataset should not be construed as legal advice or used to provide legal advice.