Estimating Usage Of Open Source Projects - Flutter Telemetry Case Study - MSR' 24

This dataset (CSV) was assembled to support analysis within a case study that will be published in the proceedings of the Mining Software Repositories conference (MSR ‘24) April 14-15 2024: Estimating Usage Of Open Source Projects.

This case study explored whether publicly available metrics could serve as proxies to estimate usage of an open source project. Using the Flutter project as our case study, we collected monthly proxy metrics from GitHub, StackOverflow and Slack to compare with Flutter’s monthly active user count over the same time period: January 2018 through February 2021.

All metrics correspond to the last day of the month and/or represent aggregate activity in that month. Data from GitHub shows aggregate activity counts across the entire Flutter GitHub organization (up to 33 repositories). 

Our specific metrics include:

Source

Metric

Aggregation method

Details

Flutter

Monthly Active Users (MAU)

Google internal tooling

Flutter users active in the last 30 days, collected on the last day of each month

GitHub

PullRequest Authors in month

GrimoireLab, hosted by bitergia.com

As Google governs changes to this code base, we excluded known Google employees in code change related metrics 

GitHub

Issues Created in month

GrimoireLab, hosted by bitergia.com

GitHub

Issue Authors in month

GrimoireLab, hosted by bitergia.com

GitHub

Fork events in month

gharchive.org

GitHub

Fork cumulative count at the end of month

gharchive.org

StackOverflow

Question Authors in month

GrimoireLab, hosted by bitergia.com

StackOverflow

Questions in month

GrimoireLab, hosted by bitergia.com

Slack

Claimed (cumulative) members at the end of the month

fluttercommunity.slack

Slack

Cumulative messages at the end of the month

fluttercommunity.slack

Tools: 

We used an instance of GrimoireLab hosted by Bitergia to aggregate GitHub PullRequest Authors, GitHub Issue Authors, GitHub Issues, across all repositories under the Flutter Organization, and StackOverflow Question Authors, and StackOverflow Questions for questions that mention Flutter. We used the Sorting Hat of feature GrimoireLab to identify Google employees in this sample.

GHArchive via BigQuery was used to count GitHub Star and Fork events across all repositories under the Flutter Organization

We pulled Slack activity directly from Flutter's slack channel dashboard