Reassessing Function-Name Recovery in Stripped Binaries: Preprocessing, Metrics, and Reproducibility

Abstract: Recovering function names from stripped binaries remains a bottleneck in software maintenance, program comprehension, binary debugging, and security analysis. Although recent years have seen a wave of machine-learning-based techniques, the practical state of the art remains difficult to assess. Prior studies are confounded by three recurring problems: a widespread assumption that heavy manual preprocessing is needed to help tokenizers, even though such processing can erase domain-specific semantics or simplify labels in ways that inflate scores; evaluations that are not directly comparable because tools rely on different function-discovery backends or on permissive metrics such as per-token top-k; and severe reproducibility barriers caused by missing artifacts, undocumented bugs, and extreme computational cost. This paper reports a systematic reassessment of function-name recovery. We reproduce four representative state-of-the-art models over a shared raw corpus and retrain each under multiple preprocessing configurations, using controlled within-model comparisons to test whether manual segmentation and normalization are necessary. Across models, we find that these hand-engineered strategies often provide limited benefit over modern subword tokenizers and can silently discard useful semantic information. We further formalize how the per-token top-k formulation adopted in recent work differs from standard top-k evaluation, and show that it substantially overstates the performance an analyst would actually observe. Finally, we quantify what it costs to reproduce and adopt these tools in practice--the engineering effort, hardware demands, and defect repairs required to bring each pipeline to a working state. Based on these findings, we distill concrete best practices for more robust, comparable, and reproducible research on function name recovery.