A Systematic Literature Review on the Use of Large Language Models for Optimization Algorithms

Context: Large language models (LLMs) are becoming popular in nearly every industry, with their problem-solving capabilities, ease of use, and interpretable results touted for complex optimization problems. However, LLMs used for optimization problems are prone to hallucinations and unstructured search techniques. Meanwhile, optimization algorithms (OA) have been used for such problems for years but are often designed for specific use cases and require domain expertise. This motivated a new combined field of LLM-modified OA. Objective: Recent studies on LLM-modified OA reveal promising results in improving optimization algorithms’ capabilities but demonstrate poor standardization, use of metrics, and repeatability. Method: In this paper, we surveyed research published from January 2017 to November 2024 to uncover how researchers are using LLMs and optimization algorithms in combination and review the quality of LLM-OA research. Following an extensive manual filtering procedure and critical analysis process, we evaluated 41 relevant papers merging LLMs and optimization algorithms to answer five research questions. Results: Although, the majority (62.2\%) of studies use LLMs during the search process (in-the-loop), we also found an absence of repeatability measures, statistical validity tests, and accountability for LLM hallucination in LLM-OA research. Since \pct{29} of papers do not provide adequate statistical analysis and \pct{30} disregard hallucinations, we propose a standardized research methodology and highlight avenues for future research. Conclusions: LLMs for OAs remain largely unexplored, presenting several research opportunities.