Replication package for the paper "The Content and Evolution of Machine Learning Model Repositories: An Empirical Study on Hugging Face and GitHub"

Cross-Platform Evolution of Hugging Face and GitHub Model Repositories This repository contains the scripts, datasets, and derived results used to study how machine-learning model repositories evolve across Hugging Face and GitHub. Repository layout . ├── configs/ # YAML configuration files used to run scripts reproducibly ├── data/ # Cleaned analysis datasets and intermediate CSV/JSON files ├── results/ # Tables, summaries, mined rules, and figures used in the paper ├── src/ # Python scripts grouped by research question ├── docs/ # Reproducibility commands and repository-structure notes ├── requirements.txt ├── .env.example ├── run_examples.sh └── README.md Research-question folders Folder Purpose rq0_dataset_construction Builds the study dataset, extracts GitHub links from Hugging Face model cards, collects issue/discussion/PR counts, and computes contributor-overlap statistics. rq1_file_inventory Inventories files in Hugging Face and GitHub repositories and maps files to analysis categories such as code, documentation, configuration, dependencies, model weights, and CI/CD. rq2_commit_change_analysis Extracts file-level commit histories, model-card section changes, UI metadata/tag changes, and commit-count distributions. rq3_cross_platform_temporal_analysis Merges Hugging Face and GitHub timelines, computes cross-platform delay events, and mines association rules between category changes across platforms. Setup Create a virtual environment and install dependencies: python -m venv .venv # Windows .venv\Scripts\activate # macOS/Linux source .venv/bin/activate pip install -r requirements.txt Some collection scripts use the GitHub API and Hugging Face Hub. Copy .env.example to .env and add tokens if needed: GITHUB_TOKEN=... HF_TOKEN=... Reproduction order The scripts are organized by research question. A typical execution order is: src/rq0_dataset_construction/ to build the model sample and collect model-level metadata. src/rq1_file_inventory/ to build Hugging Face and GitHub file inventories. src/rq2_commit_change_analysis/ to extract commit histories and model-card changes. src/rq3_cross_platform_temporal_analysis/ to compute temporal delay summaries and cross-platform association rules. The repository already includes the generated data/ and results/ files, so the scripts do not need to be rerun unless the dataset is being refreshed. Full command examples are provided in docs/reproducibility_commands.md. Example: python src/rq2_commit_change_analysis/classify_model_card_section_changes.py --config configs/rq2_model_card_section_changes.yaml Primary input/output paths can also be overridden directly with --input and --output where supported. Directory and file information This section lists the main files in each directory so that the repository is self-contained without a separate file-manifest CSV. Root files File Type Rows Size Description requirements.txt TXT — 139 B Python package requirements. .env.example EXAMPLE — 150 B Template for required environment variables and API tokens. .gitignore File — 151 B Git ignore rules for local caches, environments, and sensitive files. run_examples.sh SH — 722 B Small shell script with example analysis runs. configs/ File Type Rows Size Description rq0_collect_hf_models_with_github_links.yaml YAML — 86 B Config for collecting top Hugging Face models and GitHub links. rq0_contributor_overlap_statistics.yaml YAML — 816 B Config for contributor-overlap statistics. rq0_model_issue_discussion_stats.yaml YAML — 148 B Config for GitHub issue and Hugging Face discussion statistics. rq0_model_issue_pr_discussion_counts.yaml YAML — 153 B Config for issue, pull-request, and discussion counts. rq0_top_hf_models_commit_counts.yaml YAML — 94 B Config for top Hugging Face model commit counts. rq1_github_file_inventory.yaml YAML — 310 B Config for GitHub repository file inventory. rq1_hf_file_inventory.yaml YAML — 231 B Config for Hugging Face repository file inventory. rq2_compute_commit_distribution.yaml YAML — 309 B Config for commit-distribution summary. rq2_github_commit_file_change_inventory.yaml YAML — 646 B Config for GitHub file-level commit-change extraction. rq2_hf_commit_file_change_inventory.yaml YAML — 622 B Config for Hugging Face file-level commit-change extraction. rq2_model_card_section_changes.yaml YAML — 454 B Config for model-card section change classification. rq2_ui_metadata_tag_changes.yaml YAML — 537 B Config for UI metadata/tag change classification. rq3_commit_time_differences.yaml YAML — 560 B Config for cross-platform delay computation. rq3_cross_platform_association_rules.yaml YAML — 760 B Config for temporal basket association-rule mining. data/ data/rq0_dataset_construction/ File Type Rows Size Description github_contributors_deduplicated.csv CSV 9943 761.1 KB Deduplicated GitHub contributor list by model. hf_contributors_deduplicated.csv CSV 666 53.2 KB Deduplicated Hugging Face contributor list by model. hf_github_contributor_matches.csv CSV 248 32.6 KB Matched Hugging Face and GitHub contributors used for overlap analysis. hf_models_with_github_links.csv CSV 190 17.5 KB Hugging Face models from the top-1000 snapshot for which GitHub links were retrieved. hf_top_1000_downloads_commit_counts.csv CSV 1000 96.7 KB Top Hugging Face model snapshot with model-level commit-count information. hf_user_github_profile_links.csv CSV 955 103.0 KB GitHub profile links extracted from Hugging Face user profiles. model_issue_discussion_stats.csv CSV 190 22.8 KB Model-level GitHub issue counts and Hugging Face discussion counts. model_issue_pr_discussion_counts.csv CSV 190 26.9 KB Extended model-level issue, pull-request, and discussion counts. selected_models_dataset.csv CSV 148 12.6 KB Final validated model sample used for the analysis. data/rq1_file_inventory/ File Type Rows Size Description github_file_inventory_long.csv CSV 47589 6.4 MB Long-format GitHub repository file inventory with file categories. github_repositories_selected.csv CSV 77 5.5 KB Selected GitHub repositories retained for file-inventory analysis. hf_file_inventory_long.csv CSV 2786 227.5 KB Long-format Hugging Face repository file inventory with file categories. data/rq2_commit_change_analysis/ File Type Rows Size Description github_file_commit_history_until_2025_11_30.csv CSV 45733 18.5 MB GitHub file-level commit history up to the analysis cutoff date. hf_file_commit_history_until_2025_11_30.csv CSV 2705 483.9 KB Hugging Face file-level commit history up to the analysis cutoff date. model_card_changes_detailed.csv CSV 3262 490.1 KB Detailed model-card changes extracted from README/model-card history. model_card_changes_detailed.json JSON 1440 773.2 KB JSON version of detailed model-card change records. models_with_commit_counts.csv CSV 148 14.1 KB Model-level Hugging Face and GitHub commit-count dataset. section_taxonomy_mapping_used.csv CSV 1586 116.8 KB Mapping from model-card sections to section-level taxonomy categories. ui_metadata_field_mapping_used.csv CSV 32 806 B Mapping from UI metadata fields to metadata categories. data/rq3_cross_platform_temporal_analysis/ File Type Rows Size Description cross_platform_delay_events.csv CSV 1996 531.6 KB Detected cross-platform delay events between Hugging Face and GitHub. merged_commit_level_timeline.csv CSV 109870 38.4 MB Commit-level timeline used for temporal analysis. merged_commit_level_with_time_differences.csv CSV 243246 94.2 MB Merged commit-level timeline with computed time differences. merged_hf_github_commit_timeline.csv CSV 427607 91.5 MB Merged Hugging Face and GitHub commit timeline. normalized_commit_times.csv CSV 109870 43.9 MB Commit timestamps normalized with elapsed time fields. window_baskets.csv CSV 2107 415.0 KB Time-window baskets used for association-rule mining. window_commit_details.csv CSV 26145 8.7 MB Commit details for each temporal basket used in rule mining. results/ results/rq0_dataset_construction/ File Type Rows Size Description contributor_overlap_descriptive_stats.csv CSV 3 288 B Descriptive statistics for contributor overlap metrics. contributor_overlap_per_model_metrics.csv CSV 190 13.2 KB Per-model contributor overlap metrics. debug_github_alias_map.csv CSV 22286 1.8 MB Debug file for GitHub contributor alias mapping. debug_hf_alias_map.csv CSV 1673 165.3 KB Debug file for Hugging Face contributor alias mapping. debug_match_union_mapped.csv CSV 224 51.3 KB Debug file for union-mapped contributor matching. debug_matched_pairs.csv CSV 212 20.9 KB Debug file with matched contributor pairs. model_issue_discussion_summary.csv CSV 2 153 B Descriptive summary of GitHub issues and Hugging Face discussions. results/rq1_file_inventory/ File Type Rows Size Description github_file_category_counts_per_model.csv CSV 77 7.1 KB Per-model GitHub file-category counts. github_file_category_descriptive_stats.csv CSV 8 415 B Descriptive statistics for GitHub file categories. hf_file_category_counts_per_model.csv CSV 148 6.9 KB Per-model Hugging Face file-category counts. hf_file_category_descriptive_stats.csv CSV 8 418 B Descriptive statistics for Hugging Face file categories. results/rq2_commit_change_analysis/ File Type Rows Size Description commit_distribution_summary.csv CSV 2 120 B Descriptive statistics comparing Hugging Face and GitHub commit counts. github_category_commit_descriptive_stats.csv CSV 8 395 B Descriptive statistics for GitHub category-level commit changes. github_category_commit_stats_vs_total_commits.csv CSV 448 31.4 KB GitHub category-level commits compared with total repository commits. github_commit_collection_audit_until_2025_11_30.csv CSV 77 75.1 KB Audit log for GitHub commit collection up to the cutoff date. hf_category_commit_descriptive_stats.csv CSV 7 361 B Descriptive statistics for Hugging Face category-level commit changes. hf_category_commit_stats_vs_total_commits.csv CSV 600 36.6 KB Hugging Face category-level commits compared with total repository commits. hf_commit_collection_audit_until_2025_11_30.csv CSV 147 31.7 KB Audit log for Hugging Face commit collection up to the cutoff date. hf_file_commit_history_failures_until_2025_11_30.csv CSV 1 392 B Failed Hugging Face file-history collection records. model_card_change_extraction_debug_log.csv CSV 148 10.0 KB Debug log for model-card change extraction. section_taxonomy_change_counts_per_model_ignore_initial.csv CSV 148 12.0 KB Per-model section taxonomy change counts after ignoring initial additions. section_taxonomy_change_statistics_ignore_initial.csv CSV 13 1.2 KB Summary statistics for section taxonomy changes. ui_metadata_change_counts_per_model_ignore_initial.csv CSV 148 13.7 KB Per-model UI metadata change counts after ignoring initial additions. ui_metadata_change_statistics_ignore_initial.csv CSV 11 929 B Summary statistics for UI metadata changes. ui_metadata_mean_changes_ignore_initial.png PNG — 164.4 KB Figure showing mean UI metadata changes after ignoring initial additions. results/rq3_cross_platform_temporal_analysis/ File Type Rows Size Description all_association_rules.csv CSV 86 19.8 KB All mined association rules from temporal baskets. cross_platform_association_rules.csv CSV 3 913 B Filtered cross-platform association rules. cross_platform_association_rules_with_examples.csv CSV 14 51.5 KB Cross-platform association rules with supporting examples. cross_platform_delay_summary.csv CSV 2 271 B Descriptive summary of cross-platform delay events. cross_platform_association_heatmap.png PNG — 371.4 KB Heatmap visualization of cross-platform association rules. frequent_itemsets.csv CSV 105 11.0 KB Frequent itemsets mined from temporal baskets. src/ src/ File Type Default config Description runtime_config.py PY — Shared runtime helper used by scripts to read YAML configs and CLI overrides. src/rq0_dataset_construction/ File Type Default config Description collect_hf_models_with_github_links.py PY configs/rq0_collect_hf_models_with_github_links.yaml Collects top Hugging Face models and extracts GitHub repository links from model cards. collect_model_issue_discussion_stats.py PY configs/rq0_model_issue_discussion_stats.yaml Collects GitHub issue counts and Hugging Face discussion counts for selected models. collect_model_issue_pr_discussion_counts.py PY configs/rq0_model_issue_pr_discussion_counts.yaml Collects GitHub issue/PR counts and Hugging Face discussion counts. collect_top_hf_models_commit_counts.py PY configs/rq0_top_hf_models_commit_counts.yaml Collects commit-count information for the top Hugging Face model snapshot. compute_contributor_overlap_statistics.py PY configs/rq0_contributor_overlap_statistics.yaml Computes contributor overlap between Hugging Face and GitHub contributors. src/rq1_file_inventory/ File Type Default config Description github_file_inventory.py PY configs/rq1_github_file_inventory.yaml Builds a categorized file inventory for GitHub repositories. hf_file_inventory.py PY configs/rq1_hf_file_inventory.yaml Builds a categorized file inventory for Hugging Face repositories. src/rq2_commit_change_analysis/ File Type Default config Description classify_model_card_section_changes.py PY configs/rq2_model_card_section_changes.yaml Classifies README/model-card changes into section taxonomy categories. classify_ui_metadata_tag_changes.py PY configs/rq2_ui_metadata_tag_changes.yaml Classifies UI metadata and tag changes into metadata categories. compute_commit_distribution.py PY configs/rq2_compute_commit_distribution.yaml Computes descriptive statistics for Hugging Face and GitHub commit counts. github_commit_file_change_inventory.py PY configs/rq2_github_commit_file_change_inventory.yaml Extracts GitHub file-level commit-change inventory. hf_commit_file_change_inventory.py PY configs/rq2_hf_commit_file_change_inventory.yaml Extracts Hugging Face file-level commit-change inventory. src/rq3_cross_platform_temporal_analysis/ File Type Default config Description compute_cross_platform_commit_time_differences.py PY configs/rq3_commit_time_differences.yaml Computes delay events and temporal summaries across Hugging Face and GitHub. mine_cross_platform_association_rules.py PY configs/rq3_cross_platform_association_rules.yaml Mines frequent itemsets and association rules from temporal cross-platform baskets. docs/ File Type Rows Size Description repository_structure.md MD — 2.0 KB Notes explaining the repository organization and naming decisions. reproducibility_commands.md MD — 3.1 KB Command examples for reproducing each research-question analysis. Notes The repository uses config-based paths. Check configs/ before refreshing or rerunning a dataset collection step. The Hugging Face top-1000 model list was collected as a snapshot at the time of data collection. Hugging Face rankings, download counts, and model-card contents are dynamic, so rerunning the live API may produce a different candidate set. The paper analyzes the final validated set of 190 models for which usable GitHub repository data could be retrieved from the September 2025 snapshot.