Do you monitor CI Practices? I don't know. You tell me: A case study

In our previous studies have demonstrated that only executing automatic builds and checking test coverage have been considered to measure the level of Continuous Integration maturity in several projects. In this paper we seek for understand the the benefits and challenges of monitoring CI practices in day-to-day software development. We aim to evaluate the impact of monitoring seven CI practices in a real-world scenario on three public organizations in Brazil. We first developed a CI practices monitoring suite tool and conducted a multiple-case study applying a mixed-methods strategy, combining surveys, interviews, log data, and mining data from CI services. We verified organization’ interest in monitoring CI practices. Monitoring provided an overview of the organization’s CI status, not covered by other tools, motivated constant improvement in these practices, a perception of software quality, improve the communication and and it is easy to adopt. Conclusions: We recommend that companies adopt monitoring of CI practices and that CI services integrate this monitoring into their dashboards.

过往研究已证实，在多个项目中，仅通过执行自动构建与测试覆盖率检查，即可用于衡量持续集成（Continuous Integration, CI）的成熟度等级。本文旨在探究日常软件开发场景下，监控CI实践的收益与挑战。我们的目标是评估在巴西三家公共组织的真实业务场景中，监控七项CI实践所产生的影响。我们首先开发了一款CI实践监控套件工具，并采用混合研究方法开展多案例研究，整合了问卷调研、访谈、日志数据以及从CI服务中挖掘得到的多源数据。我们证实了各组织对监控CI实践的关注度：监控能够提供其他工具无法覆盖的组织CI状态概览，推动此类实践的持续优化，帮助形成对软件质量的认知，提升团队沟通效率，且易于落地实施。结论：我们建议企业采纳CI实践监控方案，并建议CI服务提供商将此类监控功能集成至其仪表盘中。