acruz/folktexts

folktexts数据集是基于美国社区调查（ACS）2018年公共使用微数据样本（PUMS）的问答数据集，旨在评估大型语言模型（LLMs）在不可实现任务上的校准能力。数据集包含多个任务，如预测收入、就业、公共健康保险覆盖等，每个任务都有自然的结果不确定性。数据集以自然语言问答格式提供，包括多项选择和数字问答格式。数据集的结构包括训练、验证和测试集，分别用于模型训练、超参数调整和性能评估。

The folktexts dataset is a suite of Q&A datasets derived from US Census data products, aimed at evaluating the calibration of large language models (LLMs) on tasks with natural outcome uncertainty. These tasks include predicting individual characteristics such as income, employment status, public health insurance coverage, mobility, and travel time. The datasets are available in both multiple-choice Q&A format and numeric Q&A format, and they include features mapped to natural text using the official ACS PUMS codebook. The README also describes the dataset structure, sources, uses, creation process, and provides a citation for the accompanying paper.