Can You Help ChatGPT Get an “A” in Organic Chemistry? Teaching Effective Prompting of Large Language Models for Reaction Prediction

As generative artificial intelligence (AI) tools such
as large
language models (LLMs) become widespread, they are increasingly finding
applications in chemical sciences. Although LLMs have achieved impressive
performance in many chemistry tasks, optimal performance requires
proper use, including appropriate prompting techniques. Chemistry
students are not generally taught strategies for effective LLM usage,
especially for nonwriting tasks. Here we report an activity that introduces
organic chemistry students to the use of LLMs such as ChatGPT for
predicting the outcome of chemical reactions, specifically the types
of alkene addition reactions taught in introductory organic chemistry
courses. This activity exposes students to molecular representations,
digitization of chemical reactions, train-test splitting practices
for evaluating performance, and generalizable LLM prompting strategies,
namely, the Five “S” prompt-writing approach and in-context
learning. We tested this activity with chemistry students in the USA
and in Austria and evaluated the activity through anonymous pre- and
postlab surveys. Survey data revealed that students felt that they
achieved their learning goals and that the activity was enjoyable.
As chemistry students will inevitably interact with LLMs in their
future careers, it is important to teach best practices for the effective
and critical use of these tools in the context of chemistry.