StillMe: A Practical Framework for Building Transparent, Validated Retrieval-Augmented Generation Systems

StillMe is an open-source, transparency-first framework for building validated Retrieval-AugmentedGeneration (RAG) systems designed to address three critical issues in modern AI: black-box opacity,hallucination, and knowledge cutoff limitations.

Rather than modifying or retraining Large Language Models (LLMs), StillMe demonstrates thatcommercial LLMs can be transformed into ethical, auditable systems by surrounding them with atransparent architecture consisting of:

• Continuous Learning: Automated ingestion from trusted sources (RSS feeds, arXiv, CrossRef,  Wikipedia) every 4 hours, overcoming traditional LLM knowledge cutoff limitations.

• Multi-layer Validation Chain: Six validators (citation, evidence overlap, numeric consistency,  uncertainty, ethics, fallback) that enforce grounding, intellectual humility, and safety while reducing  hallucinations.

• Full System Transparency: Every response includes citations, validation results, and an audit trail.  All learning sources, metrics, and validation outcomes are fully visible through dashboards and APIs.

We evaluate StillMe on the TruthfulQA benchmark. On a 50-question subset, StillMe achieves 56%accuracy (vs. 52% for GPT-4 under identical conditions), 100% citation rate, and a 70.6% transparencyscore. On an extended 634-question evaluation, StillMe achieves 99.7% citation coverage and 70.9%transparency score, demonstrating the robustness of the validation chain across challenging subsets.

Unlike traditional approaches focusing on LLM interpretability, StillMe focuses on “system-leveltransparency”: making every step of the RAG pipeline visible and auditable. This makes StillMe apractical, deployable alternative to closed commercial AI systems.

The full codebase, dashboards, evaluation scripts, and deployment instructions are fully open-source:https://github.com/anhmtk/StillMe-Learning-AI-System-RAG-Foundation