The Influence of Code Comments on the Perceived Helpfulness of Stack Overflow Posts (Supplementary Material)

Question-and-answer platforms such as Stack Overflow have become an important way for software developers to share and retrieve knowledge. However, reusing poorly understood code can lead to serious problems, such as bugs or security vulnerabilities. To better understand the role of code comments in the perceived helpfulness of answers on Stack Overflow, we conducted an online experiment simulating a Stack Overflow environment (n=91). The results indicate that block comments in particular were perceived as significantly more helpful than uncommented source code, especially by novices. Novices also found code snippets with block comments more helpful than those with inline comments. Interestingly, other surface features, such as the position of an answer and its usefulness score, were considered less important. The content of Stack Overflow has been a major source for training large language models. Although AI-based coding assistants such as GitHub Copilot, which are based on these models, might change the way Stack Overflow is used, our findings have implications beyond this specific platform. First, the findings of this study may help improve the ongoing relevance of community-driven platforms like Stack Overflow that provide human advice and explanations of code solutions to complement AI-based support for programmers. Second, since chat-based tools can be prompted to generate code in different ways, knowing which properties influence the perceived helpfulness of code snippets is beneficial.诸如Stack Overflow（堆栈溢出）这类程序员问答平台，现已成为软件开发人员分享与检索专业知识的重要途径。然而，复用未充分理解的代码可能引发严重问题，例如程序缺陷（bugs）或安全漏洞（security vulnerabilities）。为探究代码注释对Stack Overflow平台问答回复感知有用性的影响机制，我们开展了一项模拟Stack Overflow环境的线上实验，共纳入有效样本91例（n=91）。研究结果显示，相较于无注释的源代码，块注释（block comments）能够显著提升回复的感知有用性，这一效应在新手开发者群体中尤为突出。新手开发者同样认为，带有块注释的代码片段（code snippets）比带有内联注释（inline comments）的代码片段更具帮助价值。值得注意的是，诸如回复发布位置、有用性评分这类表层特征，对感知有用性的影响相对有限。Stack Overflow的内容已成为训练大语言模型（Large Language Model, LLM）的核心数据源之一。尽管基于此类模型开发的AI编码助手（如GitHub Copilot）可能改变Stack Overflow的使用场景与模式，但本研究的结论对该平台之外的同类场景同样具有参考意义。其一，本研究结果有助于提升诸如Stack Overflow这类社区驱动型平台的持续价值，这类平台通过提供人工编写的开发建议与代码解决方案解释，来补充面向程序员的AI辅助工具。其二，鉴于对话式AI工具可通过提示指令生成不同形式的代码，明确哪些属性会影响代码片段的感知有用性，将具有重要的实际应用价值。