Impact of the COVID-19 pandemic on academic productivity

This data set was used to analyse changes in academic productivity due to the COVID-19 pandemic.

Here we describe how those data were collected. We downloaded the arXiv dataset hosted by Kaggle on 2022 June 6. This dataset is a JSON file (named `arxiv-metadata-oai-snapshot.json`) compressed into a zip file. We extracted this file and then executed `process.py` to create the following three files:

- `metadata.json`: titles, abstract, and affiliations of authors

- `authors.json`: author names

- `records.csv`: arXiv identifier, categories, number of authors and affiliations, abstract length

We excluded pre-prints from 2022 June when processing files. The `process.py` script also adds a 'dummy' line to the `records.csv` to make Python load arXiv identifiers as strings. These three files (`metadata.json`, `authors.json`, and `records.csv`) were used for this analysis.