gruhit-patel/llama-omni-speech-instruct

Llama3.2 Omni Speech Instruct数据集旨在增强大型语言模型（LLM）的多模态能力，使其能够处理语音指令。该数据集由Stanford Alpaca Dataset和Libri Speech TTS Dataset两部分组成。首先，通过Deepgram的文本转语音API将Alpaca数据集的指令转换为语音；其次，将Libri Speech TTS数据集的音频特征与转换后的语音结合，形成最终的数据集。数据集包含输出、输入、指令、音频和类型等特征，用于训练LLM以适应语音输入。

This dataset is designed to enhance the multi-modal capabilities of large language models (LLMs) by extending their functionality to process speech instructions. The dataset is formed by combining the Stanford Alpaca Dataset and the Libri Speech TTS Dataset, converting text instructions into speech instructions and integrating existing audio data to create a new dataset. The dataset structure includes fields such as output, input, instruction, audio, and type, which are used to train LLM models to adapt to speech input signals.