Data: HasBugs - Handpicked Haskell Bugs

<strong>HasBugs - Handpicked Haskell Bugs</strong> We try to provide a fresh dataset in the fashion of [Defects4J](https://github.com/rjust/defects4j) to evaluate tasks such as fault localization, program repair and test generation. Instead of just providing the commits, we learned from some of the datasets out there, which we address in the following: A failing CI does not mean a bug - atleast not certainly from the code. Hence, our bug-points are aknowledged by the projects through PRs and Issues. This also means that the *buggy version* of the datapoints usually compiles. Similarly, the Fix is what the maintainers consider fixed (even if that is removal of a feature). As research by [Martin Monperrus](https://link.springer.com/article/10.1007/s10664-016-9470-4) showed, not all found patches in program repair actually help despite passing the test-suite. To help with inspection of potential tasks, we look into the material provided (code, issues, project readme/architecture) to give a estimate of "what is buggy". We hope this helps to easier distinguish actually good result from e.g. overfitting. We express multiple bug and fix locations in the datapoint The aim of these changes is to provide a (small) dataset with bugs from real-world problems of which all steps are aknowledged by their maintainers (the bug, the test &amp; the fix).<br> We do not try to provide a dataset in the size that you can run deep learning on it - but we want to have a quality gold standard to benchmark modern tools against. <strong>Datapoint </strong> For every project, we provide The buggy version The fixed version The separated Test-Patch, tested Version A datapoint for analysis (see below) A single datapoint can contain the following attributes (required attributes are marked with *, but these stars are not in the actual json!): <pre><code class="language-javascript">{ "*id":42, "*repositoryurl": "https://github.com/ciselab/HasBugs", "*repository":"Ciselab/HasBugs", "*licence":"MIT", "*faultcommit":"xxx", "*fixcommit":"yyy", "ghc-version":"9.2.4", "buildframework":"Cabal", "testframeworks" : ["QuickCheck","Tasty"], "testpatch":true, "description": "Input Sanitazion missing - divides by zero", "categories" : ["Sanitazion", "DivideByZero"], "relatedissues" : ["https:www.github.com/ciselab/HasBugs/issues/1"], "relatedprs" : ["https:www.github.com/ciselab/HasBugs/issues/2"], "*faultlocations" : [ { "*startline": 5, "*endline": 15, "*file" : "./project/.../program.hs", "module" : "Math.Divison", "method": "divide" },{ "*startline": 25, "*endline": 35, "*file" : "./project/.../program.hs", "module" : "Math.Divison", "method": "divide" } ], "*fixlocations" : [ { "*startline": 5, "*endline": 20, "*file" : "./project/.../program.hs", "module" : "Math.Divison", "method": "divide" } ] }</code></pre> <strong>Inclusion Criteria </strong> Compatible with GHC &gt;=8.10 Projects run with a single `make test`, `cabal test` or `stack test` buggy version newer than 2018 bug and fix aknowledged as such by the maintainer(s) through Github Issues / PRs <br><strong>HasBugs——精选Haskell缺陷数据集</strong> 本数据集参考[Defects4J](https://github.com/rjust/defects4j)的构建范式，旨在为缺陷定位、程序修复以及测试用例生成等研究任务提供高质量的评测基准。相较于仅提供提交记录的数据集，本数据集汲取了现有同类数据集的经验教训，并作出如下改进：持续集成（Continuous Integration，CI）构建失败并不等同于代码存在缺陷——至少不能仅凭此判定。因此，本数据集的所有缺陷点均通过项目的拉取请求（Pull Request，PR）与议题（Issue）得到了项目维护者的确认。这也意味着，数据集中的**缺陷版本**通常可以正常编译。同理，所谓修复版本是指经项目维护者认可的修复结果（即便该修复只是移除了某项功能）。 正如[Martin Monperrus](https://link.springer.com/article/10.1007/s10664-016-9470-4)的研究所示，在程序修复领域，并非所有通过测试套件验证的补丁都能真正解决问题。为便于后续相关任务的验证分析，本数据集团队会梳理项目提供的各类资料（包括代码、议题、项目README与架构文档），对缺陷所在作出合理评估。我们希望此举能够帮助研究者更轻松地区分模型的真正有效结果与过拟合等情况。本数据集会在数据条目中标注多处缺陷与修复位置。本次构建的核心目标是打造一个规模适中的真实世界缺陷数据集，其中所有环节（缺陷确认、测试用例与修复方案）均得到了项目维护者的认可。 我们并未追求能够支撑深度学习训练的海量规模，而是旨在构建一套高质量的黄金基准集，用于对现代程序分析工具进行性能评测。 <strong>数据条目</strong> 针对每个项目，本数据集将提供以下内容：缺陷版本、修复版本、独立的测试补丁与经过验证的版本，以及用于分析的专用数据条目（详见下文）。 单个数据条目可包含以下属性（必填属性以星号*标记，本示例中的星号仅为说明使用）： <pre><code class="language-javascript">{ "*id":42, "*repositoryurl": "https://github.com/ciselab/HasBugs", "*repository":"Ciselab/HasBugs", "*licence":"MIT", "*faultcommit":"xxx", "*fixcommit":"yyy", "ghc-version":"9.2.4", "buildframework":"Cabal", "testframeworks" : ["QuickCheck","Tasty"], "testpatch":true, "description": "缺失输入清理——存在除零错误", "categories" : ["输入清理", "除零错误"], "relatedissues" : ["https:www.github.com/ciselab/HasBugs/issues/1"], "relatedprs" : ["https:www.github.com/ciselab/HasBugs/issues/2"], "*faultlocations" : [ { "*startline": 5, "*endline": 15, "*file" : "./project/.../program.hs", "module" : "Math.Divison", "method": "divide" },{ "*startline": 25, "*endline": 35, "*file" : "./project/.../program.hs", "module" : "Math.Divison", "method": "divide" } ], "*fixlocations" : [ { "*startline": 5, "*endline": 20, "*file" : "./project/.../program.hs", "module" : "Math.Divison", "method": "divide" } ] }</code></pre> <strong>入选标准</strong> - 兼容格拉斯哥Haskell编译器（Glasgow Haskell Compiler，GHC）版本≥8.10 - 仅需通过一条`make test`、`cabal test`或`stack test`命令即可完成测试 - 缺陷版本发布时间晚于2018年 - 缺陷与修复方案均通过GitHub议题（Issue）或拉取请求（PR）得到项目维护者的确认