"Large Language Models for Automated Bloom's Taxonomy Classification in Computer Science Assessment" (Supplementary Materials: Dataset and Reproducibility Code)

Paper Information Dataset and supplementary material of the paper: "Large Language Models for Automated Bloom's Taxonomy Classification in Computer Science Assessment" If you want to refer to this work, please cite this Zenodo entry as well as the published paper below: BIBTEX TBA Research Purpose This repository contains data, code, and resources used in research examining the effectiveness of Large Language Models (LLMs) for automated Bloom's Taxonomy classification of computer science assessment items. The project aims to: Establish a benchmark dataset of expert-annotated CS exam items across four Bloom cognitive levels (Remember, Understand, Apply, Analyze) Compare proprietary and open-weight LLMs on zero-shot Bloom classification Evaluate a multi-agent council architecture (divergence, dialectical review, convergence) using locally deployable models as a privacy-preserving alternative to commercial APIs Directory Structure /DATA Contains the annotated dataset and experiment outputs. data_full_with_annotations.csv — 75 unique items with question text, type, answers, and consensus ground truth (human_council) Experiment result files (JSON, CSV, Excel), metrics tables, and confusion matrices /PROMPTS Contains prompt templates used across experimental stages. Stage 1 (divergence) prompts for independent classification Stage 2 (dialectical review) prompts for blind peer-review Chairman prompts for final convergence Root scripts run_experiment.py — Council experiment (3-stage pipeline, local Ollama) query_remote_llms.py — Zero-shot classification via OpenRouter API config.py — Model list and backend settings council_optimized.py, llm_router.py, ollama_client.py, openrouter.py — Council and LLM backend logic make_latex_metrics.py, make_dataset_overview_table.py — LaTeX/metrics generation from results (optional)