ARC_DPO_FewShot

 # Dataset Card for "ARC_DPOP_FewShot" [ARC](https://arxiv.org/abs/1803.05457) is a dataset that tests the level of understanding of science at approximately grade-school level. We focus specifically on the 'Challenge' subsection of ARC, the more difficult of the two subsections, which has been widely adopted as a measure of LLM reasoning and world understanding. We create a paired preference-ranked dataset from the train split of ARC-Challenge. The dataset is partitioned into questions which we take as our prompts x, and four choices of responses to each question of which only one is the correct answer. The correct response is taken as y_w and the incorrect responses are taken to be y_l; as there are three incorrect responses for every prompt, we repeat y_w multiple times for each prompt. The dataset is meant to be used to fine-tune LLMs (which have already undergone SFT) using the DPOP loss function. We used this dataset to create the [Smaug series of models](https://github.com/abacusai/smaug). See our paper for more details. The dataset contains 3357 training examples and 895 evaluation examples. See more details in the [datasheet](https://github.com/abacusai/smaug/blob/main/datasheet.md).