Monitoring Continuous Integration Practices in Industry: A Case Study

In our previous studies have demonstrated that only executing automatic builds and checking test coverage have been considered to measure the level of Continuous Integration maturity in several projects. In this paper we seek for understand the the benefits and challenges of monitoring CI practices in day-to-day software development. We aim to evaluate the impact of monitoring seven CI practices in a real-world scenario on three public organizations in Brazil. We first developed a CI practices monitoring suite tool and conducted a multiple-case study applying a mixed-methods strategy, combining surveys, interviews, log data, and mining data from CI services. We verified organization’ interest in monitoring CI practices. Monitoring provided an overview of the organization’s CI status, not covered by other tools, motivated constant improvement in these practices, a perception of software quality, improve the communication and and it is easy to adopt. Conclusions: We recommend that companies adopt monitoring of CI practices and that CI services integrate this monitoring into their dashboards.我们此前的研究证实，在多个项目中，通常仅将自动构建执行与测试覆盖率检查作为衡量持续集成（Continuous Integration, CI）成熟度水平的指标。本文旨在探究日常软件开发场景下，监控CI实践所能带来的收益与面临的挑战。我们的目标是，在真实场景中评估监控7项CI实践对巴西三家公共机构产生的影响。我们首先开发了一款CI实践监控套件工具，并采用混合方法策略开展多案例研究，结合了问卷调查、访谈、日志数据以及从CI服务中挖掘得到的数据。我们证实了各机构对监控CI实践存在浓厚兴趣。监控工作能够提供其他工具无法覆盖的机构CI状态全景，推动此类实践的持续优化，提升对软件质量的感知水平，改善团队沟通效率，且易于采用。结论：我们建议企业采用CI实践监控方案，并建议CI服务提供商将此类监控功能集成至其仪表板中。