Highgroundbkk/Pangpuriye-generated_by_typhoon

--- license: cc-by-nc-2.0 task_categories: - table-question-answering language: - th - en tags: - code pretty_name: Thai-SQL_Question_generated_by_Typhoon size_categories: - 1K<n<10K configs: - config_name: default data_files: - split: train path: "data.json" --- # 🤖 [Super AI Engineer Development Program Season 4](https://superai.aiat.or.th/) - Pangpuriye House - Generated by Typhoon API  **Pangpuriye's House Dataset - Generated Dataset from [Typhoon API](https://opentyphoon.ai/)** This dataset is an output generated from the Typhoon API in the structure of SQL instruction for fine-tuning [Pangpuriye's LLM](https://huggingface.co/AIAT/Pangpuriye-openthaigpt-1.0.0-7b-chat). The dataset is set under cc-by-nc-2.0 license. ## Content The dataset consists of 16,125 rows of `input`, `instruction`, and `output` packed into a train set. - Each schema has its own CSV file as an `input`. - The `instruction` is the command that the Typhoon API receives as input. - The `output` is a SQL code. ## Uses The dataset is intended to be used as an instruction for fine-tuning table-based QA LLM. The instruction requires some processing before it can be utilized in the process. The following code is an example for using with the schema. ```python stream = client.chat.completions.create( model="typhoon-instruct", messages=[ { "role": "instruction", "content": """ Your task is to generate SQL plain-text schema Format: You won't explain or clarify your response. """, }, {"role": "user", "content": """Generate 1 random schema"""}, ], max_tokens=120, temperature=0.6, top_p=1, stream=False, ) ``` ## Call our dataset by `datasets` library The following code is an example of calling our dataset via the `datasets` library. ```python from datasets import load_dataset dataset = load_dataset("AIAT/Pangpuriye-generated_by_typhoon") ``` ## Acknowledgements The dataset is collectively stored by the members of Panguriye's house during the LLMs hackathon in Super AI Engineer Development Program Season 4. We thank the organizers of this hackathon, [OpenThaiGPT](https://openthaigpt.aieat.or.th/), [AIAT](https://aiat.or.th/), [NECTEC](https://www.nectec.or.th/en/) and [ThaiSC](https://thaisc.io/) for this challenging task and opportunity to be a part of developing Thai large language model. ## Citation Information ``` @misc{pipatanakul2023typhoon, title={Typhoon: Thai Large Language Models}, author={Kunat Pipatanakul and Phatrasek Jirabovonvisut and Potsawee Manakul and Sittipong Sripaisarnmongkol and Ruangsak Patomwong and Pathomporn Chokchainant and Kasima Tharnpipitchai}, year={2023}, eprint={2312.13951}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```数据集基本信息： - 许可证：CC-BY-NC-2.0 - 任务类别：表格问答（table-question-answering） - 支持语言：泰语、英语 - 标签：代码 - 数据集名称：Thai-SQL_Question_generated_by_Typhoon（Typhoon生成的泰语SQL问句） - 样本规模：1000<n<10000条 - 配置信息：默认配置，训练集数据文件为`data.json` 🤖 [Super AI工程师发展计划第4季](https://superai.aiat.or.th/) - Pangpuriye团队 - 由Typhoon API生成  **Pangpuriye数据集 - 由[Typhoon API](https://opentyphoon.ai/)生成的数据集** 本数据集是通过Typhoon API生成的输出，采用SQL指令结构，用于微调[Pangpuriye大语言模型（LLM）](https://huggingface.co/AIAT/Pangpuriye-openthaigpt-1.0.0-7b-chat)。本数据集采用CC-BY-NC-2.0许可证发布。 ## 数据集内容 本数据集包含16125条训练集样本，每条样本包含`input`、`instruction`和`output`三个字段。 - 每个数据表结构对应一个作为`input`的CSV文件。 - `instruction`为Typhoon API接收的输入指令。 - `output`为生成的SQL代码。 ## 数据集用途 本数据集旨在作为指令数据，用于微调基于表格的问答大语言模型。在实际使用前，需对该指令数据进行一定处理。 以下为配合数据表结构使用的示例代码： python stream = client.chat.completions.create( model="typhoon-instruct", messages=[ { "role": "instruction", "content": """ Your task is to generate SQL plain-text schema Format: You won't explain or clarify your response. """, }, {"role": "user", "content": """Generate 1 random schema"""}, ], max_tokens=120, temperature=0.6, top_p=1, stream=False, ) ## 通过`datasets`库调用本数据集 以下为通过`datasets`库加载本数据集的示例代码： python from datasets import load_dataset dataset = load_dataset("AIAT/Pangpuriye-generated_by_typhoon") ## 致谢 本数据集由Super AI工程师发展计划第4季大语言模型黑客松期间，Pangpuriye团队成员共同整理存储。 我们感谢本次黑客松的主办方：[OpenThaiGPT](https://openthaigpt.aieat.or.th/)、[AIAT](https://aiat.or.th/)、[NECTEC](https://www.nectec.or.th/en/)以及[ThaiSC](https://thaisc.io/)，为我们提供本次开发泰语大语言模型的挑战任务与实践机会。 ## 引用信息 @misc{pipatanakul2023typhoon, title={Typhoon: Thai Large Language Models}, author={Kunat Pipatanakul and Phatrasek Jirabovonvisut and Potsawee Manakul and Sittipong Sripaisarnmongkol and Ruangsak Patomwong and Pathomporn Chokchainant and Kasima Tharnpipitchai}, year={2023}, eprint={2312.13951}, archivePrefix={arXiv}, primaryClass={cs.CL} }