Toward Automated Simulation Research Workflow through LLM Prompt Engineering Design

The advent of Large Language Models (LLMs) has created
new opportunities
for the automation of scientific research spanning both experimental
processes and computational simulations. This study explores the feasibility
of constructing an autonomous simulation agent (ASA) powered by LLMs
through prompt engineering and automated program design to automate
the entire simulation research process according to a human-provided
research plan. This process includes experimental design, remote upload
and simulation execution, data analysis, and report compilation. Using
a well-studied simulation problem of polymer chain conformations as
a test case, we assessed the long-task completion and reliability
of ASAs powered by different LLMs, including GPT-4o, Claude-3.5, etc.
Our findings revealed that ASA-GPT-4o achieved near-flawless execution
on designated research missions, underscoring the potential of methods
like ASA to achieve automation in simulation research processes to
enhance research efficiency. The outlined automation can be iteratively
performed for up to 20 cycles without human intervention, illustrating
the potential of ASA for long-task workflow automation. Additionally,
we discussed the intrinsic traits of ASA in managing extensive tasks,
focusing on self-validation mechanisms, and the balance between local
attention and global oversight.