资源简介：
---
dataset_info:
features:
- name: id
dtype: string
- name: question
dtype: string
- name: question_concept
dtype: string
- name: choices
struct:
- name: label
sequence: string
- name: text
sequence: string
- name: answerKey
dtype: string
- name: rationale
dtype: string
splits:
- name: train
num_bytes: 21541
num_examples: 41
download_size: 19635
dataset_size: 21541
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---
# Commonsense QA CoT (Partial, Annotated) - PRELIMINARY
## Dataset Summary
This dataset is a human-annotated subset of randomly sampled question-answer entries from the CommonsenseQA dataset (tau/commonsense_qa).
The 'rationales' for each QA pair were created using a two-part method. First, Mixtral (mistralai/Mixtral-8x7B-Instruct-v0.1) was used
to generate 3 unique CoT (Chain-of-Thought) explanations. Next, human evaluation was applied to distill the random sampling down to
a cohesive set of question-answer-rationale triplets.
The working hypothesis, inspired by the research papers listed below, is that a diverse set of CoT rationales passed along
with the CommonsenseQA question-answer choices will provide accelerated commonsense reasoning performance on even a relatively
small model (<3B parameters).
Additional refinement and annotations to this dataset are to follow.
Background research and inspiration from the following papers:
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge (https://arxiv.org/abs/1811.00937)
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (https://arxiv.org/abs/2201.11903)
Specializing Smaller Language Models towards Multi-Step Reasoning (https://arxiv.org/abs/2301.12726)
Orca 2: Teaching Small Language Models How to Reason (https://arxiv.org/abs/2311.11045)
Large Language Models Are Reasoning Teachers (https://arxiv.org/abs/2212.10071)
Teaching Small Language Models to Reason (https://arxiv.org/abs/2212.08410)
## Dataset Structure
### Languages
The dataset is in English (`en`).
### Data Fields
- `id` (`str`): Unique ID.
- `question`: a `string` feature.
- `question_concept` (`str`): ConceptNet concept associated to the question.
- `choices`: a dictionary feature containing:
- `label`: a `string` feature.
- `text`: a `string` feature.
- `answerKey`: a `string` feature.
- `rationale`: a `string` feature.
### Data Example
```
{'id': '1fe48d12b6f6e4e38f4445f3ec60d5c5',
'question': 'What can happen to someone too sure of their learning?',
'question_concept': 'learning',
'choices': {'label': ['A', 'B', 'C', 'D', 'E'],
'text': ['growth',
'gaining knowledge',
'enlightenment',
'knowing more',
'overconfidence']},
'answerKey': 'E',
'rationale': 'When someone is too sure of their learning, they become '
'overconfident, thinking that they know everything. This can '
'prevent them from learning more, as they stop seeking new '
'knowledge and ideas. They might also miss out on '
'enlightenment, as they close themselves off to new '
'perspectives. Overall, their growth might be stunted, as they '
'stop challenging themselves and expanding their '
'understanding. So, out of the given choices, the most '
'appropriate answer is overconfidence.'}
```
### Source Data
- **Data:** https://huggingface.co/datasets/tau/commonsense_qa
- **Homepage:** https://www.tau-nlp.org/commonsenseqa
- **Repository:** https://github.com/jonathanherzig/commonsenseqa
- **Paper:** https://arxiv.org/abs/1811.00937
### Licensing Information
The dataset is licensed under the MIT License.
