Do you monitor CI Practices? I don't know. You tell me: A case study

In our previous studies have demonstrated that only executing automatic builds and checking test coverage have been considered to measure the level of Continuous Integration maturity in several projects. In this paper we seek for understand the the benefits and challenges of monitoring CI practices in day-to-day software development. We aim to evaluate the impact of monitoring seven CI practices in a real-world scenario on three public organizations in Brazil. We first developed a CI practices monitoring suite tool and conducted a multiple-case study applying a mixed-methods strategy, combining surveys, interviews, log data, and mining data from CI services. We verified organization’ interest in monitoring CI practices. Monitoring provided an overview of the organization’s CI status, not covered by other tools, motivated constant improvement in these practices, a perception of software quality, improve the communication and and it is easy to adopt. Conclusions: We recommend that companies adopt monitoring of CI practices and that CI services integrate this monitoring into their dashboards.