Architecture for a Trustworthy Quantum Chatbot [dataset]

This dataset contains all the materials and results used in the development and empirical validation of C4Q 2.0. The files are organized into two main directories: bakend_testing and empirical_validation. Below is a brief overview of their contents: - app/ - Contains the full source code of C4Q at the state of the software at the time of submitting this paper, allowing for reproducibility and further development. The frontend’s node_modules directory is not included, but these dependencies can be generated by following the installation instructions provided in the README.md file. - README.md – A guide detailing how to locally set up and run C4Q. - bakend_testing/ - Contains data from the evaluation of C4Q’s backend components: *reportBackendC4Q2.0.html: An HTML report generated by running 189 tests on the backend of C4Q. *classLLM_20241110101327.pth_training_metrics.csv: A CSV file documenting the Classification LLM’s training and validation metrics, including training loss, validation loss, training accuracy, and validation accuracy for each epoch. *qaLLM_evaluation_metrics.txt: A text file listing exact match and F1 metrics per epoch for the QA LLM. - create_data/ – Includes the scripts used to generate and curate training data for the classification LLM and the QA LLM. - empirical_validation/ - Includes data from the empirical evaluation of C4Q against other chatbots: *Directories named by model (e.g., openai-o1/, deepseek-coder_33b/, deepseek-r1/, etc.): Each directory contains the raw answers produced by the respective model in response to our set of quantum computing and software engineering questions. *prompts.txt: A text file with the full list of prompts used during the empirical evaluation. *requirements_qiskit0.46.3: A requirements file for Python dependencies used to create an environment with a Qiskit versions <1.0.0, enabling code snippet testing under older Qiskit releases. *requirements_qiskit1.3.1: A requirements file for Python dependencies used to create an environment with a Qiskit versions >= 1.0.0, ensuring reproducible tests under newer releases. *results.xlsx: An Excel spreadsheet containing the empirical evaluation outcomes, including correct, incomplete, and incorrect answer rates for each model, under both Qiskit environments. *script_gates.sh: A shell script that automates prompting of OLAMA’s deepseek-coder:33b and starcoder2:15b models with gate-related quantum questions. *script_SE.sh: A shell script that automates prompting of OLAMA’s deepseek-coder:33b and starcoder2:15b models with software engineering problem questions.本数据集涵盖了C4Q 2.0开发与实证验证过程中使用的全部材料与结果。文件分为两个主要目录：bakend_testing与empirical_validation。以下是各目录内容的简要概述： - app/：包含提交本论文时C4Q软件的完整源代码，以保障可复现性与后续开发。前端的node_modules目录未包含在内，但可通过README.md文件中的安装说明生成相关依赖。 - README.md：一份指南，详细说明了如何在本地搭建并运行C4Q。 - bakend_testing/：包含C4Q后端组件的评估数据： * reportBackendC4Q2.0.html：一份HTML报告，由对C4Q后端执行189项测试生成。 * classLLM_20241110101327.pth_training_metrics.csv：一份CSV文件，记录了分类大语言模型（Classification LLM）的训练与验证指标，包括每个轮次的训练损失、验证损失、训练准确率与验证准确率。 * qaLLM_evaluation_metrics.txt：一份文本文件，列出了问答大语言模型（QA LLM）每个轮次的精确匹配率与F1值指标。 - create_data/：包含用于生成与整理分类大语言模型及问答大语言模型训练数据的脚本。 - empirical_validation/：包含C4Q与其他聊天机器人进行实证评估的相关数据： * 以模型名称命名的目录（例如openai-o1/、deepseek-coder_33b/、deepseek-r1/等）：每个目录均包含对应模型针对我们的量子计算与软件工程问题集生成的原始回答。 * prompts.txt：一份文本文件，收录了实证评估过程中使用的全部提示词。 * requirements_qiskit0.46.3：一份Python依赖需求文件，用于创建Qiskit版本低于1.0.0的环境，以支持在旧版Qiskit环境下测试代码片段。 * requirements_qiskit1.3.1：一份Python依赖需求文件，用于创建Qiskit版本≥1.0.0的环境，以保障在新版Qiskit环境下测试的可复现性。 * results.xlsx：一份Excel电子表格，包含实证评估结果，涵盖两种Qiskit环境下各模型的正确、不完整与错误回答占比。 * script_gates.sh：一个Shell脚本，可自动向OLAMA的deepseek-coder:33b与starcoder2:15b模型发送与量子逻辑门相关的问题提示。 * script_SE.sh：一个Shell脚本，可自动向OLAMA的deepseek-coder:33b与starcoder2:15b模型发送软件工程问题提示。