gladius-research

GLADIUS Research 是一个专注于自适应认知模型（ACM）研究的数据集，旨在探索模态不变性、跨模态认知几何和低于1亿参数的智能。GLADIUS是一种基于第一性原理设计的新型Transformer架构，最初在语言任务上进行测试，但其设计目标是作为通用的认知架构。该架构具有稀疏线性注意力（SLA²）、专家混合（MoE）、温/热记忆系统、Time2Vec时间编码和动态注意力分配的alpha路由等关键特性。数据集包含多个研究论文，涵盖渐进式扩展、细胞分裂、GPT-2蒸馏、时间序列能力、跨模态不变性发现等内容。关键发现包括跨模态不变性（早期层对模态不敏感）、认知距离谱（不同领域间的不变性强度差异）和温记忆作为新颖性检测器。架构细节包括12层、384隐藏维度、24头注意力、4专家MoE、16K BPE词汇表等。该数据集适用于AI研究、认知架构开发和跨模态学习任务。

GLADIUS Research is a dataset focused on the research of Adaptive Cognitive Models (ACM), aiming to explore modality invariance, cross-modal cognitive geometry, and sub-100-million-parameter intelligence. GLADIUS is a novel Transformer architecture designed based on first principles, initially tested on language tasks, while its design goal is to serve as a general cognitive architecture. This architecture features several key characteristics including Sparse Linear Attention (SLA²), Mixture of Experts (MoE), warm/thermal memory systems, Time2Vec temporal encoding, and alpha routing for dynamic attention allocation. The dataset contains multiple research papers covering topics such as progressive scaling, cell division, GPT-2 distillation, time series capabilities, and cross-modal invariance discovery. Key findings include cross-modal invariance (early layers are modality-insensitive), cognitive distance spectrum (differences in invariance strength across various domains), and warm memory acting as a novelty detector. Architectural details include 12 layers, 384 hidden dimensions, 24 attention heads, 4-expert MoE, 16K BPE vocabulary, and so on. This dataset is suitable for AI research, cognitive architecture development, and cross-modal learning tasks.