Quantity doesn't buy quality syntax with neural language models

This repository contains the 125 LSTM models analyzed in van Schijndel, Mueller, and Linzen (2019) "Quantity doesn't buy quality syntax with neural language models". Each archive contains 25 models trained on a specific number of training tokens. All models were trained to use the vocabulary in vocab.txt. The naming convention for each model is:<br> LSTM_[Hidden Units]_[Training Tokens]_[Training Partition]_[Random Seed]-d[Dropout Rate].pt Hidden Units: The number of hidden units per layer (there are two layers in each model) {100, 200, 400, 800, 1600}<br> Training Tokens: The number of tokens used to train each model {2m, 10m, 20m, 40m, 80m}<br> Training Partition: Five distinct training partitions were created for each amount of training data {a, b, c, d, e}<br> Random Seed: The random seed used to train each model*<br> Dropout Rate: All models used a dropout rate of 0.2 *A scripting bug led to a random seed of 0 for all models trained on less than 40 million tokens. This does not substantively affect the analyses since each model is distinct in terms of the model configuration or training data, so we opted to not retrain the models with unique random seeds to save time and computational resources.本仓库收录了van Schijndel、Mueller与Linzen（2019）在论文《神经网络语言模型的训练数据量并非语法质量的等价指标》（"Quantity doesn't buy quality syntax with neural language models"）中分析的125个长短期记忆网络（LSTM）模型。每个归档文件均包含25个基于特定训练Token数训练得到的模型，所有模型均使用vocab.txt中的词表完成训练。各模型的命名规范如下：<br>LSTM_[隐藏单元数（Hidden Units）]_[训练Token数（Training Tokens）]_[训练分区（Training Partition）]_[随机种子（Random Seed）]-d[丢弃率（Dropout Rate）].pt<br>隐藏单元数（Hidden Units）：指每层的隐藏单元数量（每个模型包含两层），可选取值为{100, 200, 400, 800, 1600}<br>训练Token数（Training Tokens）：指训练单个模型所用的Token总数，可选取值为{2m, 10m, 20m, 40m, 80m}<br>训练分区（Training Partition）：针对每种训练数据规模，共设置5个独立的训练分区，可选取值为{a, b, c, d, e}<br>随机种子（Random Seed）：训练模型时使用的随机种子*<br>丢弃率（Dropout Rate）：所有模型均采用0.2的丢弃率<br>*由于脚本存在漏洞，所有训练Token数少于4000万的模型均使用了随机种子0。鉴于每个模型在配置或训练数据上均存在差异，该问题不会对分析结果产生实质性影响，因此为节省时间与计算资源，我们未使用唯一随机种子重新训练这些模型。