H2O h2oGPT v0.2.0

Query and summarize your documents or just chat with local private GPT LLMs using h2oGPT, an Apache V2 open-source project.

- **Private** offline database of any documents [(PDFs, Excel, Word, Images, Video Frames, YouTube, Audio, Code, Text, MarkDown, etc.)](https://github.com/h2oai/h2ogpt/blob/main/docs/README_LangChain.md#supported-datatypes)
- **Persistent** database (Chroma, Weaviate, or in-memory FAISS) using accurate embeddings (instructor-large, all-MiniLM-L6-v2, etc.)
- **Efficient** use of context using instruct-tuned LLMs (no need for LangChain's few-shot approach)
- **Parallel** summarization and extraction, reaching an output of 80 tokens per second with the 13B LLaMa2 model
- **HYDE** (Hypothetical Document Embeddings) for enhanced retrieval based upon LLM responses
- **Variety** of models supported (LLaMa2, Mistral, Falcon, Vicuna, WizardLM. With AutoGPTQ, 4-bit/8-bit, LORA, etc.)
- **GPU** support from HF and LLaMa.cpp GGML models, and **CPU** support using HF, LLaMa.cpp, and GPT4ALL models
- **Attention Sinks** for [arbitrarily long](https://github.com/tomaarsen/attention_sinks) generation (LLaMa-2, Mistral, MPT, Pythia, Falcon, etc.)
- **UI** or CLI with streaming of all models
- **Upload** and **View** documents through the UI (control multiple collaborative or personal collections)
- **Vision Models** LLaVa, Claude-3, Gemini-Pro-Vision, GPT-4-Vision
- **Image Generation** Stable Diffusion (sdxl-turbo, sdxl) and PlaygroundAI (playv2)
- **Voice STT** using Whisper with streaming audio conversion
- **Voice TTS** using MIT-Licensed Microsoft Speech T5 with multiple voices and Streaming audio conversion
- **Voice TTS** using MPL2-Licensed TTS including Voice Cloning and Streaming audio conversion
- **AI Assistant Voice Control Mode** for hands-free control of h2oGPT chat
- **Bake-off** UI mode against many models at the same time
- **Easy Download** of model artifacts and control over models like LLaMa.cpp through the UI
- **Authentication** in the UI by user/password via Native or Google OAuth
- **State Preservation** in the UI by user/password

搜集汇总

背景与挑战

背景概述

H2O h2oGPT v0.2.0是一个开源项目，支持用户与本地私有GPT模型交互或处理多种格式的文档，具备高效并行处理和多样化模型支持的特点，同时提供包括UI操作、语音处理等丰富的功能。