Replication Package for "Beyond Comment Removal: How Coding Agents Address Self-Admitted Technical Debt"

# Replication Package for "Beyond Comment Removal: How Coding Agents Address Self-Admitted Technical Debt" ## Directories | Directory | Contents ||---|---|| `DataPreparation/` | Commit collection, SATD extraction, filtering, and sampling. || `RQ1/` | Manual labels and annotator agreement. || `RQ2/` | Classification inputs, prompts, predictions, and evaluation. || `RQ3/` | SATD re-additions, revert evidence, and figures. || `RQ4/` | Churn, undo events, commit intent, and relatedness. || `RQ4/output/churn_m1/` | Churn data and results. || `RQ4/output/history_slicing/llm_assessments/` | Model records and relatedness labels. | ## DataPreparation | Script | Purpose ||---|---|| `batch_clone.py` | Clone the study repositories. || `commit_languages.py` | Count programming languages in the collected commits. || `extract_commit_2k_remote.py` | Collect agent-associated commits and patches. || `filter_study_dataset.py` | Extract SATD removals and apply the study filters. || `sample_additional_records.py` | Select the manual-coding sample. || `satd_stage_languages.py` | Count languages at each filtering stage. | ## RQ1 | Script | Purpose ||---|---|| `compute_kappa.py` | Calculate agreement between annotators. || `label_distribution.py` | Count manual labels. | ## RQ2 | Script | Purpose ||---|---|| `build_population_inputs.py` | Build diff inputs or extract them from the input archive. || `classify_population.py` | Classify study inputs with the GPT API. || `evaluate.py` | Calculate classification metrics. || `evaluate_manual_sample.py` | Evaluate the complete prediction tables against the 350 manual labels. || `input_data.py` | Load classification inputs and construct prompts. || `method_context.py` | Extract enclosing function contexts. || `model_providers.py` | Connect to Bedrock classification models. || `run_satd_classifier.py` | Run model classification and record usage and costs. || `summarize_population.py` | Count the final population labels and their distribution by agent. || `taxonomy.py` | Validate classification labels and hierarchy. || `test_method_context.py` | Test function-context extraction. | ## RQ3 | Script | Purpose ||---|---|| `audit_deletion_presence.py` | Check deletion and re-addition links. || `classify_readd_intent.py` | Classify re-addition commit intent using CCS. || `collect_readd_metadata.py` | Collect deletion and re-addition authorship. || `git_snapshot.py` | Record and replay Git command outputs. || `readd_classify.py` | Identify revert evidence for re-addition events. || `readd_satd_same.py` | Find the first re-addition of each SATD. || `revert_oracle.py` | Check re-addition detection using Git. || `run_rq3.py` | Run or replay the RQ3 pipeline. || `satd_syntax.py` | Parse source definitions and comment lines. || `satdbailiff.py` | Parse SATD comments and match their identities. || `summarize_by_rq2.py` | Join resolution labels with re-addition evidence and generate flow figures. || `test_deletion_presence.py` | Test deletion and re-addition link checks. || `test_readd_classify.py` | Test revert detection. || `test_readd_metadata.py` | Test authorship parsing. || `test_readd_synthetic.py` | Test tracking on synthetic histories. || `test_summarize_by_rq2.py` | Test joins and flow counts. || `validate_satdbailiff.py` | Evaluate SATDBailiff parsing on the Java validation sample. | ## RQ4 | Script | Purpose ||---|---|| `analyze_rq4.py` | Analyze churn, follow-up changes, and undo events. || `churn_metrics.py` | Compute churn and reversal measurements. || `classify_commit_intent.py` | Classify follow-up commit batches using CCS. || `collect_later_signatures.py` | Identify agent signatures in follow-up commits. || `history_slicing.py` | Track changes to SATD-related code regions. || `hydrate_public_blobs.py` | Retrieve missing public Git objects. || `match_human_commits.py` | Select human commits by SATD count and commit size. || `prepare_commit_intent.py` | Prepare human follow-up commits as CCS inputs. || `recalculate_m1_churn.py` | Calculate M1 churn measurements. || `report_m1_churn.py` | Summarize churn rates and generate the figure. || `rework_by_author.py` | Count follow-up changes by author type. || `rework_metrics.py` | Measure individual follow-up changes. || `run_history_slicing_cache.py` | Track SATD-related code using cached histories. || `run_matched_humans.py` | Collect measurements for matched human commits. || `run_rq4.py` | Collect churn measurements. || `satd_syntax.py` | Parse source definitions and comment lines. || `summarize_binary_relatedness.py` | Summarize Related and Unrelated labels. || `summarize_history_slicing.py` | Summarize tracked regions and follow-up commits. || `test_churn_synthetic.py` | Test churn and reversal calculations. || `test_history_slicing.py` | Test code-region tracking. || `test_recalculate_m1_churn.py` | Test M1 churn calculations. |