A Dual Monitoring Framework
收藏资源简介:
A. Package Overview To ensure the reproducibility and verifiability of this study, we release a reproducibility package on Zenodo. After peer review concludes, or at the camera-ready stage, the DOI will be disclosed so that anyone can download the same artifacts and reproduce the results. This package consists of the experimental pipeline's database schema/dump, the experimental execution program (HTML+PHP), and a collection of SQL queries for reproducing the tables and figures reported in the paper. B. Runtime Environment The experimental pipeline was executed in the following environment. • OS: Windows 11 • Web stack: Apache 2.4.58, PHP 8.2.12 • Database: MariaDB 10.4.32 • Auxiliary scripts: Python 3.x C. Target LLMs (providers and model identifiers) The comparative experiment used the following six LLMs. • OpenAI: gpt-5 • xAI: grok-4 • Anthropic: claude-sonnet-4-5 • Google: gemini-2.5-flash • NAVER: HCX-005 • deepseek:deepseek-reasoner (evaluation-only LLM) D. Profanity Filtering Baseline API We applied the Profanity Filter API as a baseline for profanity detection. Although we initially sought to apply a national-level anti-profanity system, we were unable to secure research-use authorization for that service at the time of the experiment. We therefore substituted an external API with an equivalent structure. E. Data Collection Window and Request Control The period from prompt generation to model-specific response collection was 2026-02-06 to 2026-03-05, and the same prompt set was applied to all target models within this period to maintain comparability. Requests were executed through a web+API workflow, and the following controls were applied to minimize omissions caused by provider-specific rate limits and transient errors. • Minimum waiting time per model: 120 seconds • Maximum retries on error: 5 F. Experimental Scale (for Reference) This package is designed to enable reproduction and auditing of experimental logs at the following scale. • Prompts: 160 (8 speech-act types x 2 conditions: non-profanity/profanity) • Response collection: 160 x 5 target models x 3 repeated generations = 2,400 responses • LLM judgment (excluding self-judgment, with one additional evaluation-only LLM; 5 LLM judges per response): 12,000 behavioral safety labels + 12,000 age suitability ratings • Profanity Filter API verification: 2,400 cases • LLM self-judgment verification: 800 behavioral safety labels + 800 age suitability ratings • Human validation: 84 samples x 10 raters = 840 cases G. Safety/Ethics Note These data may include text that could cause discomfort, including aggressive expressions, profanity, and insults, because such material was necessary for research purposes. Harmful language and sensitive information, including personally identifiable information (PII) and secrets, have been removed from the public dump. The data should be used responsibly and only for research and verification purposes. H. Included Files and Their Roles 1) db_dump.sql (MariaDB dump) This MariaDB dump file contains the relational structure (DDL) used to store experimental logs, together with data from which sensitive information has been removed. Importing it reconstructs the full structure used to store and analyze prompts, model responses, LLM judgment logs, and human-validation records. For research-ethics and misuse-prevention reasons, harmful language in prompts and response text is masked. Quantitative logs, labels, and aggregates are provided without modification, thereby preserving reproducibility. [Major tables] • tb_prompt: prompt records based on 8 speech-act types x 2 conditions • tb_target_models: identifiers and metadata for five target LLMs • tb_llm_response: prompt-based response data • tb_behavior_safety_label: LLM-judged behavioral safety label data • tb_age_judgement: LLM-judged age suitability rating data • tb_human: metadata on external validation participants (de-identified experts) • tb_human_judgement: human-judgment data for 84 samples 2) program.zip (pipeline source code) This archive contains the HTML+PHP source code for the experimental pipeline, spanning prompt generation, Response Capture, the auxiliary baseline (API), LLM judgment (behavior/age), and database logging. Major scripts • /web_prompt.php: speech-act-based prompt generation and database storage • /web_llm_response.php: LLM Response Capture • /web_behavior_label_profanityapi.php: Profanity Filter API verification • /web_behavior_label_llmjudge.php: LLM judgment for behavioral safety labels • /web_age_label_llmjudge.php: LLM judgment for age suitability ratings 3) reproduce_paper_tables_figures.pdf This document is a collection of SQL queries for reproducing the tables and figures reported in the paper. I. Minimal Reproduction Procedure 1) Import db_dump.sql into MariaDB to restore the schema and the de-identified logs. 2) Deploy program.zip in a local web-server environment, and configure each API credential locally using the template config.php (the public package does not include secret keys). 3) If needed, execute the scripts either to regenerate the logs through the same procedure (re-collection) or to conduct analyses using the restored database. 4) Run the SQL queries in reproduce_paper_tables_figures.pdf to reproduce the core aggregated results reported in the paper's tables and figures.



