DCMS DLC Coatings Producing Assistant Testing Data

This dataset supports the research paper on the development of a specialized intelligent assistant for diamond-like carbon (DLC) coatings. The central hypothesis is that a Large Language Model (LLM) enhanced with a domain-specific Retrieval Augmented Generation (RAG) technique can significantly outperform base LLMs in handling complex technical tasks within the niche field of magnetron-sputtered DLC coatings. The data was gathered to train and rigorously evaluate this hypothesis. It represents a curated knowledge base of scientific publications and a series of technical queries and answers related to DLC coating processes, properties, and problem-solving. Each row in the dataset corresponds to a specific scientific document. The core data includes the original Filename, Title, and summaries of the source papers. The key generated fields are the Questions and answers and the Relevant papers retrieved by the RAG system for each query. The performance data is captured through multiple evaluation columns. For both the specialized Assistant (powered by either DeepSeek or GLM) and the Base LLMs, the dataset provides the model's generated answers, overall evaluation verdicts, and fractional scores indicating the rate of fully correct answers (A evals fraction) and partially correct answers (A,C evals fraction). The data shows a notable finding: the RAG-enhanced Assistant achieved a dramatically higher accuracy of 87% in responding to technical questions compared to the 25% accuracy of the base LLM, quantitatively demonstrating the value of domain-specific knowledge augmentation. Researchers can use this dataset to analyze the types of questions where the specialized system succeeds or fails, understand the relevance of the retrieved papers for accurate answering, and potentially use the question-answer pairs as a benchmark for developing their own domain-specific AI assistants in materials science.本数据集支撑一篇关于类金刚石碳（Diamond-like Carbon，简称DLC）涂层专用智能助手开发的研究论文。其核心研究假说为：搭载领域专属检索增强生成（Retrieval Augmented Generation，简称RAG）技术的大语言模型（Large Language Model，简称LLM），在磁控溅射DLC涂层这一细分领域的复杂技术任务处理中，性能可显著优于基础大语言模型。 本数据集的采集旨在训练并严格验证该假说，内容涵盖经精选整理的科学文献知识库，以及一系列与DLC涂层制备工艺、性能特性及问题解决相关的技术问答集合。 数据集中每一行对应一篇特定科学文献，核心数据字段包含原始文件名、文献标题与源论文摘要。新增生成的关键字段则包括各查询对应的问答对，以及RAG系统为各查询检索到的相关文献。 性能评估数据通过多列评估指标进行记录。针对分别基于DeepSeek与GLM构建的专用智能助手，以及基础大语言模型，数据集均提供了模型生成的回答内容、整体评估结论，以及表征完全正确回答占比（A evals fraction）与部分正确回答占比（A,C evals fraction）的分数值。 实验结果揭示了一项显著发现：经RAG增强的专用助手在技术问题响应中实现了87%的准确率，远高于基础大语言模型25%的准确率，定量证明了领域专属知识增强方案的应用价值。 研究人员可利用本数据集分析专用系统的优劣题型，理解检索文献对准确作答的相关性，也可将其中的问答对作为基准，用于开发材料科学领域的专属AI智能体（AI Agent）。