ORCCA-reg: Dataset Outputs

ORCCA-Reg: data and outputs Data and outputs for the submission Beyond the Registry: Ontology-Guided Discovery of Research Software across Five Million GitHub Repositories. Code: https://anonymous.4open.science/r/orcca-reg-msr27/ Unzip both archives at the root of the code repository. They recreate the folders the code reads and writes (data/, data_run2/, diagnostics/). The repository README maps every paper table to its script and output file (section Paper table → script → output map); file names here are identical. Domains and ontologies: math = MSC 2020, physics = PhySH, biology = EDAM. Files in this record File Content orcca-data.zip data/ (working data directory: Pipelines 1–6, LLM audit, paper statistics), diagnostics/, raw registry dumps orcca-data_run2.zip data_run2/: complete run of all Pipeline 2 configurations, including the external baselines expert_sheet_all150.xlsx Expert annotation sheet: 150 candidate repositories, judged blind (built with scripts/swh_expert_sheet.py) README.md This file orcca-data.zip Repository root File Content swmath_data.csv, bio_tools_data.json, ascl_github_links.json Raw registry dumps (swMATH, bio.tools, ASCL) from which the registry snapshots and ground truth are built data/: Pipeline 1 (ontologies) File Content MSC_2020.csv, physh.ttl, edam_ontology.owl Source ontologies {msc,physh,edam}_enhanced.owl OWL ontologies built by Pipeline 1 (input to OnT and Box²EL) {msc,physh,edam}_manifest.json, edam_manifest_meta.json Concept manifests (labels, hierarchy) {msc,physh,edam}_embeddings.h5 Concept embeddings (HDF5) {msc,physh,edam}_metadata.pkl Concept metadata (Python pickle) pipeline1_summary.json, run_manifest_pipeline1.json Ontology statistics (Table I) and run manifest data/: trained models Folder Content ont_models/<ontology>[__<tag>]/ OnT encoder fine-tuned on each ontology: final/ (model, tokenizer, eval_metrics.json, provenance), data/ (training and validation axioms), experiment/ (selected λ, validation results) box2el_models/<ontology>[__<tag>]/ Box²EL models with provenance __<tag> marks diagnostic variants (__ft, __galen, __msc, __drop, __fold); see Diagnostic runs in the repository README. data/: Pipeline 2 (alignment and discovery) File / folder Content results_<config>[__<tag>]/<domain>/ <config>_<domain>_repos.jsonl (scored repositories) and <config>_<domain>_topic_alignments.jsonl (topic → concept alignments). Configurations here: ont_only, box2el_struct, full, full_readme, and the tagged variants pipeline2_summary_<config>[__<tag>]_<timestamp>.json Run summaries; several timestamps for one configuration are successive runs run_manifest_pipeline2_<config>.json Run manifests data/: Pipeline 3 (registry evaluation) File / folder Content registries/ Frozen registry snapshots (GitHub links from swMATH, ASCL, bio.tools) and manifest gt_<domain>.json, gt_<domain>.hash Ground truth and its hash discoverable_<domain>.json Ground-truth repositories present in the corpus discoverable_math_cleaned_gt-math_boost-none.json Cleaned ground truth (math) pipeline3_evaluation.json, threshold_sweep_<config>.csv, threshold_sweep_all_models.csv Retrieval and ranking metrics (Table IV) sensitivity_betagamma_<domain>.{csv,pdf}, components_math.parquet β/γ sensitivity (Table VII) improved_evaluation_math/ Cleaned-GT evaluation, math (Table VI) github_baseline_<domain>.json GitHub-search baseline (Table V) top100_per_domain_new_candidates.{csv,json}, candidate_stratum_sizes.json, candidate_noise_filtered.csv Candidate export for the audits data/: Pipeline 4 (LLM audit, Table VIII) File / folder Content llm_judge_results.json, llm_judge_metrics.json, llm_judge_report.csv, llm_judge_results.provenance.json Dual LLM-judge verdicts, metrics and provenance audit_head/ Head-sample audit (top-ranked candidates): candidate export and judge outputs data/: Pipelines 5–6 and paper statistics File / folder Content defense3_outputs/ Stratified Box²EL analysis (RQ2) defense_E_outputs/ README re-ranking α sweep (RQ2) paper_stats.json, paper_stats_*.csv Funnel (Table II), channel overlap, concept coverage, matched GitHub recall, seed-topic coverage, Stage 0 count, corpus size diagnostics/ File Content results_<A>_vs_results_<B>_<domain>.json Comparisons of variant runs (scripts/diagnose_recall.py) probe_results.txt, hyperbolic_results.txt, unscored_topics.txt, unscored_diag.txt Outputs of additional encoder probes and of the analysis of unscored topics orcca-data_run2.zip: data_run2/ Same layout as data/. A complete Pipeline 2 run of all configurations in all three domains: ont_only, box2el_struct, hier_only, full, full_readme, readme_only, specter_flat, el_embed, codebert_readme, unixcoder_readme, graphcodebert_readme. It includes its Pipeline 1 manifests, ground truth, discoverable sets (cleaned versions for all three domains), OnT and Box²EL models, defense3_outputs/ and defense_E_outputs/. README-based configurations hold README scores (<domain>_readme_scores.json) instead of topic alignments. Not included Item Reason Identification repos.ndjson (GitHub topics dataset, 3,540,135,645 bytes) Third-party content, distributed separately SHA-256 62166452b6db0bd7da31f127b8fcc7f6fff9bf57c678ab557eb67c3a7c112f31 blobs-by-swhid.tar.zst (README texts addressed by SWHID) Third-party content; retrievable from Software Heritage by SWHID SHA-256 bb0713281f9a940335deb6c644c6dc6dadaad258a55110054cfaf82b3eed43f3 readme_cache/ (README embeddings, 36 GB) Regenerated by Pipeline 2 n/a Intermediate training checkpoints, run logs Not needed to check the results n/a Pretrained models (Hugging Face), with the revisions used: Model Revision allenai/specter 81cbfb43d4fc2e728d5b4201ce14987db8d0854c Hui97/OnT-MiniLM-L12-galen 562e87bad572d2d33b1b2a381e42bec4e8284c16 microsoft/codebert-base 3b0952feddeffad0063f274080e3c23d75e7eb39 microsoft/graphcodebert-base 2b0488a7bb0eefc7041f1bb2cad1ab26b0da269d microsoft/unixcoder-base 5604afdc964f6c53782a6813140ade5216b99006 sentence-transformers/all-MiniLM-L12-v2 a50ef00143b4d5391434df20ae11632588ac25be File formats .jsonl: JSON Lines (one record per line). .h5: HDF5. .parquet: Apache Parquet. .pkl: Python pickle (load only from trusted sources). Absolute paths in run summaries and manifests are replaced by <HOME> / <PROJECT> placeholders.