ai2-adapt-dev/flan_v2_converted

This is a converted version of the Flan dataset into Tulu SFT training format.

The conversion script can be found in our [open-instruct](https://github.com/allenai/open-instruct/blob/main/scripts/data/sft/flan.py) repo.
The conversion took the following parameters:
- apply_keyword_filters: True
- apply_empty_message_filters: True
- push_to_hub: True
- hf_entity: ai2-adapt-dev
- converted_dataset_name: flan_v2_converted
- local_save_dir: ./data/sft/flan

The original FLAN dataset needs extensive efforts to be regenerated, so we are using [a reproduced version by the OpenOrca team](https://huggingface.co/datasets/Open-Orca/FLAN).More specifically, we only use their top level jsonl files, which is a subset of the original dataset.And by default, we only use the `cot_fsopt_data`, `cot_zsopt_data`, `niv2_fsopt_data`, `niv2_zsopt_data` `flan_fsopt_data`, `flan_zsopt_data`, `t0_fsopt_data` subsets.If you want to use more data, you can modify this script to load more data from their Huggingface repo.Please refer to their Huggingface repo [here](https://huggingface.co/datasets/Open-Orca/FLAN) and the [original FLAN v2 repo](https://github.com/google-research/FLAN/tree/main/flan/v2) for more information about this dataset and the license.