Data Sheet 2_Empathy, bias, and data responsibility: evaluating AI chatbots for gender-based violence support.docx

Artificial Intelligence (AI) chatbots are increasingly deployed as support tools in sensitive domains such as gender-based violence (GBV). This study evaluates the performance of three conversational AI models—including a general-purpose Large Language Model (ChatGPT), an open-source model (LLaMA), and a specialized chatbot (AinoAid)—in providing first-line assistance to women affected by GBV. Drawing on findings from the European IMPROVE project, the research uses a mixed-methods design combining qualitative narrative interviews with 30 survivors in Spain and quantitative natural language processing metrics. Chatbots were assessed through scenario-based simulations across the GBV cycle, with prompts designed via the Systematic Context Construction and Behavior Specification method to ensure ethical and empathetic alignment. Results reveal significant differences in emotional resonance, response quality, and gender bias handling, with ChatGPT showing the most empathetic engagement and AinoAid offering contextually precise guidance. However, all models lacked intersectional sensitivity and proactive attention to privacy. These findings highlight the importance of trauma-informed design and qualitative grounding in developing responsible AI for GBV support.