Beyond Embedding: Finetuning Transformer Models in Psychological Research

Psychologists frequently turn to natural language processing to convert texts into psychological constructs. The state-of-the-art technique for achieving that conversion is transformer models. Recently, Kjell et al. (2023) published the Text package with user-friendly functions to facilitate the adoption of transformers. With this package, researchers only need to perform two steps for extracting psychological constructs from texts: (1) transform texts into embeddings, and (2) train a model with those embeddings. In this review article, we examine the core functionalities of the Text package. Despite its usability and efficiency, we note that the Text package is not always the best option for researchers. Researchers might need to move beyond merely encoding text and instead consider finetuning the language models. We demonstrate that compared with the Text package, finetuning language models can be more effective and faster by a substantial margin.