Performance Comparison: NLP vs. LLMs.

This study investigates the potential of artificial intelligence, particularly Natural Language Processing and large-scale language models, to improve resource management and service access for individuals with autism in Alabama. The research aims to explore and evaluate the potential of AI-driven tools to address challenges in navigating complex datasets and supporting social work practices. We designed and tested AI systems, including general language models, domain-specific chatbots powered by advanced language models, and a Retrieval-Augmented Generation framework. A standardized set of queries was used to simulate real-world scenarios encountered by social workers engaged in autism service coordination. System performance was evaluated based on precision, recall, and response accuracy. Results demonstrated that the Retrieval-Augmented Generation (RAG) framework achieved superior performance compared to traditional NLP methods and general-purpose language models, attaining approximately 90–96% precision and recall across evaluated query types. RAG outperformed the domain-specific GPT-4 chatbot by approximately 5–12 percentage points in F1 score, with the largest gains observed for queries requiring geographic specificity, multiple constraints, or complex contextual understanding. The integration of domain-specific retrieval significantly enhanced the accuracy, contextual relevance, and usability of generated responses. The findings highlight the transformative potential of AI-driven tools in improving social work efficiency and enhancing healthcare equity. By streamlining care coordination and delivering accurate, contextually relevant information, these systems offer scalable solutions to improve access to autism-related services. Future research should focus on addressing data quality, minimizing biases, and ensuring ethical deployment to build trust and support widespread adoption of these tools.