Anonymous submission for double-blind review, supplementary to: "What Each Cell Means: Semantic Role Graphs for Structure-Aware Spreadsheet RAG" paper

Structure-Aware Chunk Extraction for Spreadsheet RAG via Learned Cell-Role Prediction

Data, trained models, evaluation results, and reproduction artifacts for the NeurIPS 2026 submission "What Each Cell Means: Semantic Role Graphs for Structure-Aware Spreadsheet RAG."

What is in this archive

Path

Description

labeled/

541 human-annotated spreadsheet-tab JSONs (~1 M cells with 13-class semantic roles) plus precomputed E5 text embeddings (*_embeddings.npz).

splits/

K-fold cross-validation definitions (seeds 42, 2137, 10042010; 5 folds) with explicit sheet_ids per fold for exact reproduction.

models/

120 trained checkpoints (8 architectures × 3 seeds × 5 folds): GAT, GCN, MLP, AdjTransformer, DualModalityGNN, SpatialEdgeTransformer, LayoutLMv3-frozen, LayoutLMv3-finetune.

rag_eval_dataset.json

298 question–answer pairs over 56 held-out spreadsheets for the RAG evaluation benchmark.

rag_eval_results.json

Aggregated RAG metrics (Recall@k, source recall, LLM judge, human judge) for all methods under hybrid RRF retrieval.

rag_eval_details.json

Per-query RAG answers, retrieved chunks, and LLM judge verdicts for every method–query pair.

rag_eval_human_judgments.json

18,932 human 1–5 ratings of RAG-generated answers (same rubric as the LLM judge).

rag_eval_structure_metrics.json

Per-method structure-vs-gold metrics at four granularity levels.

rag_eval_labeled/

46 human table-structure labels for the RAG hold-out sheets (oracle baseline only; not used for GNN training).

reddit_playwright_extraction.json

59 Reddit practitioner survey threads (r/Rag, r/LangChain, etc.) with tool mentions and pain-point annotations.

reddit_excel_rag_raw.json

Supplementary Reddit discussion data.

reddit_excel_rag_stats.json

Aggregated survey statistics (tool adoption, pain points, satisfaction).

inclusion_report/

Sheet inclusion/exclusion lists and metadata (--max-nodes 800 policy).

SHA256sums.txt

Per-file SHA-256 checksums for integrity verification.

Key numbers

541 labeled spreadsheet tabs, ~1 M cells with known semantic roles

13 cell-role classes (table header, column header, row header, value, aggregation, etc.)

8 model architectures, 3 seeds, 5 folds = 120 trained checkpoints

298 RAG evaluation queries over 56 held-out sheets

18,932 human judgment ratings

59 Reddit practitioner threads analyzed

Integrity verification

# Linux
sha256sum -c SHA256sums.txt

# macOS shasum -a 256 -c SHA256sums.txt

Reproduction

Full instructions are in REPRODUCIBILITY.md inside the archive. Quick start:

Unzip this archive into data/ inside the code repository.

pip install -r requirements.txt

bash scripts/paper/reproduce_all.sh — regenerates all paper figures and tables.

streamlit run dashboard.py — launches an interactive explorer for all data and results.

License

Code: MIT. Data: CC BY-SA 4.0 (inherited from Sheetpedia source sheets).