rockship/quizgen

---
license: apache-2.0
---

# Quizgen LLM Dataset

This README provides an overview of the Quizgen LLM Dataset, a collection of curated and synthetic question-answer samples designed to facilitate training and testing for Large Language Models (LLMs) with a focus on quiz generation and agent-based workflows.

## Dataset Summary

The Quizgen LLM Dataset consists of **494 samples** across three main sources:

1. **AWS Quiz Samples**: 391 synthetic samples, created using the `GPT-4o-mini` model.
2. **Langfuse Workflow Samples**: 89 samples generated using the `Quizgen` agentic workflow, powered primarily by `OpenAI-4o`.
3. **Augmented Training Samples**: 14 samples created during training and testing, which have been re-integrated into the dataset to provide additional depth.

Each sample in the dataset includes question and answer pairs, along with metadata capturing the source and context.

### Dataset Structure

The dataset is stored in `.json` format, with each sample represented as a dictionary containing the following fields:

- **Question**: The main question prompt.
- **Answer**: The expected or model-generated answer.
- **Source**: The source of the sample (`AWS Quiz`, `Langfuse Workflow`, or `Augmented Sample`).
- **Context**: The synthetic context generated during the creation process for questions requiring additional information or contextual background.

An example sample entry:

```json
{
"question": "What is the primary use of Amazon EC2?",
"answer": "Amazon EC2 provides scalable computing capacity in the cloud.",
"source": "AWS Quiz",
"context": "Generated by GPT-4o-mini to simulate an AWS quiz environment."
}
```

## Sources and Data Generation

### 1. AWS Quiz Samples (391 Samples)
These samples were synthetically generated to simulate real AWS quiz questions and answers. Each sample question and answer pair is generated by `GPT-4o-mini` to replicate the style, content, and complexity found in AWS certification quizzes. Questions cover a range of AWS services, cloud fundamentals, and best practices, providing a realistic quiz experience.

### 2. Langfuse Workflow Samples (89 Samples)
The Langfuse samples were collected during the active development phase of Quizgen’s agentic workflow. This workflow involved an agent orchestrating LLM calls, with the primary model being `OpenAI-4o`. These samples are more dynamic and interactive in nature, with question prompts that adapt based on real-time user inputs. This set is useful for testing agentic workflows in quiz generation tasks, particularly where context and response variation are crucial.

### 3. Augmented Training Samples (14 Samples)
This subset includes additional samples that were created as part of initial training and testing phases for Quizgen LLM. These samples were re-integrated into the dataset to provide a broader range of question types and answers, enriching the dataset’s ability to handle complex, nuanced queries. They also offer additional examples for edge cases and scenarios that may not be covered in standard quiz samples.

## Usage and Applications

The Quizgen LLM Dataset is designed for use in training, fine-tuning, and evaluating LLMs and agentic architectures for quiz generation. Potential applications include:

- **Training and Fine-tuning**: Using the dataset to optimize LLMs to generate accurate, contextually appropriate quiz questions and answers.
- **Testing and Evaluation**: Assessing model performance on various quiz-related metrics such as accuracy, relevance, and consistency in an educational or certification-focused context.
- **Agentic Workflow Development**: Supporting agent-based architectures, where models are directed to generate question-answer pairs in real-time based on user input and task requirements. This is particularly beneficial for developing adaptive quiz generators.

### Suggested Workflow for Dataset Usage

1. **Data Loading**: Load the dataset in JSON format, which can be processed using common data science tools and libraries (e.g., `pandas`, `PyTorch`, `TensorFlow`).
2. **Preprocessing**: Format the questions and answers as required for specific model input formats. For example, tokenizing text for LLMs or structuring input as prompt-response pairs.
3. **Fine-tuning**: Train the model on the dataset’s question-answer pairs to improve the accuracy and contextual relevance of model-generated responses.
4. **Evaluation**: Use test splits or cross-validation to measure performance across various dimensions, such as accuracy, response coherence, and user satisfaction.
5. **Integration with Agentic Systems**: If used in an agent-based workflow, integrate the dataset samples to test real-time quiz generation based on dynamic context shifts or user interaction.

## License

The Quizgen LLM Dataset is provided for educational and research purposes only. Users are responsible for ensuring compliance with any licensing terms related to the use of OpenAI and AWS data sources if building upon this dataset for commercial or open-source projects.

## Contact and Contributions

For inquiries, improvements, or contributions, please contact the Quizgen LLM development team. Contributions are welcome via pull requests and feedback, as this project aims to create a robust dataset for quiz-based applications in language modeling.

---

This README outlines the structure, purpose, and usage of the Quizgen LLM Dataset, offering a foundation for those working on quiz generation and agentic workflows in language model contexts. Happy experimenting and training!

license: apache-2.0

# Quizgen 大语言模型数据集（Quizgen LLM Dataset）

本 README 文档旨在介绍 Quizgen LLM 数据集，这是一批经过精选的合成问答样本集合，旨在助力大语言模型（Large Language Model，LLM）的训练与测试，核心聚焦于测验生成与基于 AI 智能体（AI Agent）的工作流场景。

## 数据集概览

Quizgen LLM 数据集共包含 **494 个样本**，源自三大主要来源：
1. **AWS 测验样本**：391 个合成样本，由 `GPT-4o-mini` 模型生成。
2. **Langfuse 工作流样本**：89 个样本，通过以 `OpenAI-4o` 为核心算力的 `Quizgen` 智能体工作流生成。
3. **增强训练样本**：14 个样本，生成于模型训练与测试阶段，现已重新整合至数据集以补充内容深度。

每个数据集样本均包含问答对，以及用于记录样本来源与上下文的元数据。

### 数据集结构

本数据集以 `.json` 格式存储，每个样本均以字典形式呈现，包含以下字段：
- **Question（问题）**：核心问题提示词
- **Answer（答案）**：预设标准答案或模型生成的答案
- **Source（来源）**：样本来源类型，可选值为 `AWS Quiz`、`Langfuse Workflow` 或 `Augmented Sample`
- **Context（上下文）**：生成问题时配套的合成背景信息，用于为需要额外信息或上下文支撑的问题提供背景。

附带一个示例样本条目：
json
{
"question": "What is the primary use of Amazon EC2?",
"answer": "Amazon EC2 provides scalable computing capacity in the cloud.",
"source": "AWS Quiz",
"context": "Generated by GPT-4o-mini to simulate an AWS quiz environment."
}

## 数据来源与生成流程

### 1. AWS 测验样本（391 个样本）
本批次样本为模拟真实 AWS 测验场景的合成样本，所有问答对均由 `GPT-4o-mini` 生成，旨在复刻 AWS 认证测验的风格、内容与复杂度。问题覆盖各类 AWS 服务、云计算基础概念与最佳实践，可提供贴近真实认证测验的练习体验。

### 2. Langfuse 工作流样本（89 个样本）
本批次样本采集自 Quizgen 智能体工作流的活跃开发阶段。该工作流通过智能体编排大语言模型调用，核心算力模型为 `OpenAI-4o`。此类样本具备更强的动态性与交互性，问题提示词可根据实时用户输入进行适配调整，非常适合用于测试测验生成任务中的智能体工作流，尤其适用于需要考量上下文适配与响应多样性的场景。

### 3. 增强训练样本（14 个样本）
本子集包含首批 Quizgen LLM 训练与测试阶段生成的额外样本，现已重新整合至数据集，以拓展样本的题型与答案覆盖范围，提升数据集处理复杂、细微查询的能力。此外，该子集还补充了标准测验样本未覆盖的边缘场景与特殊案例示例。

## 使用场景与应用方向

Quizgen LLM 数据集专为测验生成相关的大语言模型与智能体架构的训练、微调与评估而设计，潜在应用场景包括：
- **训练与微调**：利用数据集优化大语言模型，使其能够生成准确且符合上下文逻辑的测验问答对。
- **测试与评估**：在教育或认证类场景中，从准确率、相关性与一致性等多维度评估模型在测验相关任务中的性能表现。
- **智能体工作流开发**：支撑基于 AI 智能体的架构开发，此类架构可根据用户输入与任务需求实时生成问答对，尤其适用于自适应测验生成系统的开发。

### 数据集使用推荐流程
1. **数据加载**：以 JSON 格式加载数据集，可通过主流数据科学工具与库（如 `pandas`、`PyTorch`、`TensorFlow`）进行处理。
2. **预处理**：根据目标模型的输入格式要求格式化问答对，例如为大语言模型进行文本 Token 化，或将输入结构整理为提示-响应对。
3. **微调训练**：基于数据集的问答对训练模型，提升模型生成答案的准确性与上下文贴合度。
4. **模型评估**：通过测试集划分或交叉验证，从准确率、响应连贯性与用户满意度等多维度衡量模型性能。
5. **智能体系统集成**：若用于智能体工作流场景，可整合数据集样本，测试基于动态上下文变化或用户交互的实时测验生成功能。

## 许可证

Quizgen LLM 数据集仅可用于教育与研究目的。若基于本数据集开展商业或开源项目，使用者需确保遵守与 OpenAI 及 AWS 数据源相关的所有许可条款。

## 联系方式与贡献途径

如有咨询、改进建议或贡献意愿，请联系 Quizgen LLM 开发团队。我们欢迎通过拉取请求与反馈进行贡献，本项目旨在为语言建模领域的测验类应用打造一套高质量数据集。

---

本 README 概述了 Quizgen LLM 数据集的结构、用途与使用方法，可为从事语言模型领域测验生成与智能体工作流相关研究的人员提供基础支撑。祝您实验与训练顺利！