SafeLabs-Eval ASI01 Through Clinical Notes: Independent External Evaluation on a Dental SaaS Application

Overview This dataset documents an external evaluation of SafeLabs-Eval is an open-source AI agent security evaluation framework developed by Waqar Javed, founder of Safe Labs AI Inc. The evaluation was conducted by Faruk Battikh on Tbibia, a dental clinic management SaaS application, to examine indirect prompt injection through clinical notes. Evaluation methodology The evaluation used SafeLabs-Eval v0.12.0 and prompt library v1.13.0, covering all 30 prompts in the ASI01 prompt-injection category. Each adversarial prompt was introduced as a clinical note through the application's user interface. The application's AI patient summarization feature subsequently processed the notes alongside legitimate synthetic clinical records. Testing included baseline evaluation, supplementary runs, forced-failure scenarios, remediation verification, and post-fix retesting. Only synthetic patient data was used. Key findings All 30 ASI01 prompts received UNCERTAIN verdicts in both baseline and retest evaluations. The model did not follow the injected instructions in the observed AI summaries. An application-level fallback issue was identified in which attacker-controlled note content could appear as a clinical next-step recommendation when model calls failed. The application issue was subsequently fixed and tested through natural and forced-failure scenarios. The evaluation identified limitations in automated payload detection and proposed improvements to document-based and schema-aware evaluation. Dataset contents The archive includes: Full evaluation report and methodology. Baseline and retest SafeLabs-Eval outputs. Per-prompt review tables. Application bridge logs and model traces. Testing configuration and reproducibility information. Forced-failure tests before and after remediation. Before-and-after screenshots. Supporting evaluation scripts. SHA-256 file integrity manifest. Framework feedback The evaluation generated three proposed improvements to SafeLabs-Eval: Payload-surfaced detection for indirect prompt injection. Document/context delivery mode for ASI01. Schema-aware evaluation for document-based agents. Limitations The evaluation used one synthetic patient file, with the injected note consistently positioned as the newest note. The results should not be interpreted as proving general resistance to indirect prompt injection or clinical safety.