ManyTypes4Py: A benchmark Python Dataset for Machine Learning-Based Type Inference

The dataset is gathered on Sep. 17th 2020. It has more than 5.4K Python repositories that are hosted on GitHub. Check out the file <strong>ManyTypes4PyDataset.spec </strong>for repositories URL and their commit SHA. The dataset is also de-duplicated using the CD4Py tool. The list of duplicate files is provided in <strong>duplicate_files.txt </strong>file. All of its Python projects are processed in JSON-formatted files. They contain a seq2seq representation of each file, type-related hints, and information for machine learning models. The structure of JSON-formatted files is described in <strong>JSONOutput.md</strong> file. The dataset is split into train, validation and test sets by source code files. The list of files and their corresponding set is provided in <strong>dataset_split.csv</strong> file. Name-based visible type hints for processed projects are stored in the <strong>extracted_visible_types </strong>folder. Notable changes to each version of the dataset are documented in <strong>CHANGELOG.md</strong>.