Below Chance: Code and Derived Data for an Audit of the TRACE Continual-Learning Benchmark and a Census of Published Sequential Baselines

This record accompanies the article "Below Chance: Auditing Sequential Baselines on TRACE and Other Continual-Learning Benchmarks for Language Models" by Nont Kanungsukkasem, [submitted to / published in] Expert Systems. The article compares the final task scores published with the TRACE continual-learning benchmark (Wang et al., 2023, arXiv:2310.06762) with chance and with trivial predictors that use no model, reports a three-seed rerun (seeds 1234, 1235 and 1236) of TRACE's sequential LoRA baseline on Llama-2-7b-chat-hf using the released code, and reports a census of 39 published papers on four continual-learning benchmarks built from classification tasks. Contents - results/release/per_item_features_20260903/: per-item derived records for the rerun and the untuned model. Each record holds only booleans, integers and floats keyed by item index. - results/raw/: audit, analysis and control outputs, including every trivial-predictor score, and the census files: the screening record with every query and candidate (census_stage1_screening_20260929.json), the two independent transcriptions (census_read_*), their comparison, the third reading and the resolution rules with their reasons, the venue checks and the outcomes (census_outcomes_*). - scripts/: audit, analysis and consistency-checking code, including scripts/census/ for the census analysis and the appendix table. - paper_cl/CENSUS_PROTOCOL_2026-09-29.md: the census protocol, committed before any candidate was read. Not included: item-level prompts, model generations and gold labels, which are benchmark data; model weights and adapters; and the texts of the audited papers. Terminology: some file names and data fields use "floor" for what the article calls a trivial-predictor score. The names are kept so that the recorded hashes and checks remain valid. Licence: data and documents CC BY 4.0; code MIT.