Replication Data for: On Finetuning Large Language Models

A recent paper by Häffner et al. 2023 introduces an interpretable deep learning approach for domain specific dictionary creation, where it is claimed that the dictionary-based approach outperforms finetuned language models in predictive accuracy while retaining interpretability. We show that the dictionary-based approach's reported superiority over large language models, BERT specifically, is due to the fact that most of the parameters in the language models are excluded from finetuning. In this letter, we first discuss the architecture of BERT models, then explain the limitations of finetuning only the top classification layer, and lastly we report results where finetuned language models outperform the newly proposed dictionary-based approach by 27% in terms of R2 and 46% in terms of mean squared error once we allow these parameters to learn during finetuning. Researchers interested in large language models, text classification, and text regression should find our results useful. Our code and data are publicly available. Code to replicate \"On Finetuning Large Language Models.\"Häffner等人2023年的最新论文提出了一种面向特定领域词典构建的可解释深度学习方法，该研究声称基于词典的方法在预测精度上优于微调后的大语言模型（Large Language Model, LLM），同时保留可解释性。本文研究表明，该研究所称的基于词典方法相较大语言模型——特指BERT（Bidirectional Encoder Representations from Transformers）——的优越性，实则源于语言模型的绝大多数参数未参与微调。在本通讯中，我们首先探讨了BERT模型的架构，随后阐释了仅对顶层分类层进行微调的局限性；最后我们报告了实验结果：当允许语言模型的全部参数在微调阶段参与学习时，微调后的大语言模型在R²指标上较该新提出的基于词典的方法高出27%，在均方误差（mean squared error, MSE）指标上高出46%。关注大语言模型、文本分类与文本回归的研究人员将能从我们的研究结果中获益。本研究的代码与数据已公开可获取，包含复现《论微调大语言模型》一文所需的全部代码。