Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation using Knowledge Graphs and Large Language Models
收藏资源简介:
GenMCQ Reproducibility Dataset This compressed file contains a compact reproducibility bundle for the MCQ-generation and difficulty-estimation study. It is intended for readers who want to inspect the generated questions, inspect the difficulty-signal table, or rerun the LLM-based generation/audit pipeline. Dataset Description This dataset supports a proof-of-concept study on generating multiple-choice knowledge questions from text-derived knowledge graphs and estimating their empirical difficulty. The source corpus consists of saved Wikipedia text snapshots from popular open-domain pages. These documents were processed with GPT-4o and LangChain's graph-construction utilities to produce subject-predicate-object graph documents, which were then used to generate multiple-choice questions around high-centrality key entities. The bundle contains 156 candidate MCQs generated by the pipeline and a final 150-item analyzed subset used for the revised difficulty-estimation experiments. Six candidate MCQs were excluded from the final modeling set after a multi-model LLM answerability audit flagged them as weakly answerable from the stem and options alone. Each MCQ includes the question stem, four answer options, the correct answer label, the key entity, generation metadata, and an aggregate empirical difficulty label. The difficulty label is the observed incorrect-answer rate from the human study, computed at the item level; individual participant responses are not included. For each final MCQ, the dataset provides nine normalized difficulty signals used in the manuscript: reasoning form, helper-triple use, distractor depth, graph node-embedding similarity, stem-option text-embedding similarity, degree centrality, readability, above-largest-gap count, and the LLM Extra Fact diagnostic signal. The dataset also includes the graph components used to instantiate MCQ-generation prompts, the saved source Wikipedia snapshots, prompt templates, minimal scripts for reproducing KG construction and LLM audits, and compact reference outputs for the answerability audit. The dataset is designed for reproducibility and secondary analysis rather than as a broad educational benchmark. Its scope is common-knowledge Wikipedia content centered on popular/high-centrality entities, so results derived from it should not be interpreted as evidence of generalization to specialized domains, low-centrality concepts, or deployment-scale educational assessment without additional validation.



