dgrx-systems/Spectra-Mamba-KAN-Hybrid-Language-Model
收藏Hugging Face2026-03-24 更新2026-03-29 收录
下载链接：
https://hf-mirror.com/datasets/dgrx-systems/Spectra-Mamba-KAN-Hybrid-Language-Model
下载链接
链接失效反馈官方服务：
资源简介：
---
license: mit
tags:
- architecture
- language-model
- mamba
- state-space-model
- KAN
- kolmogorov-arnold-networks
- research
- nlp
pretty_name: Spectra Architecture Reference
---
# Spectra: A Mamba-KAN Hybrid Language Model
A complete architectural, mathematical, and implementation reference for **Spectra** —
a hybrid language model that fuses **Selective State Space Models (Mamba)** with
**Kolmogorov-Arnold Networks (KANs)** in a parallel block design with learnable mixing weights.
**Key properties:**
- 🚀 **O(L) complexity** — eliminates the quadratic attention memory wall
- 🧠 **Constant inference memory** — SSM state replaces the growing KV cache
- 🔍 **Interpretable** — every KAN edge is a plottable univariate B-spline
- 📐 **339M parameters** at base configuration (d_model=768, 24 layers)
---
---
## Files
- [`Spectra_Architecture_fixed.pdf`](Spectra_Architecture_fixed.pdf) — full downloadable document
## Citation
```
@misc{spectra2024,
title = {Spectra: A Mamba-KAN Hybrid Language Model},
year = {2024},
note = {Architecture reference document, MIT License}
}
```
提供机构：
dgrx-systems
