Multi-Spectral Gaussian Splatting with Neural Color Representation

3D Gaussian Splatting (3DGS) has transformed novel-view synthesis from RGB images, yet remains restricted to the visible spectrum. Many applications, including agricultural monitoring, rely on multi-spectral imaging, where spectral camera alignment and scalability pose major challenges. We present MS-Splatting—a multi-spectral 3DGS framework enabling unified multi-view consistent reconstruction and rendering across both visible and invisible spectra. Our key component is a neural color representation that encodes per-primitive features shared across spectral bands, decoded through a shallow multi-layer perceptron into spectrum-specific radiance. By leveraging inter-band correlations, this formulation enhances detail while reducing memory consumption compared to independent band modeling via per-channel modeling with spherical harmonics. Our method enables accurate parallax-free novel-view vegetation index rendering for plant monitoring and enhances RGB novel view synthesis quality by exploiting details revealed through multi-spectral bands. Our evaluation demonstrates that MS-Splatting exceeds the current leading methods in both categories. In addition, we introduce a multi-spectral dataset from aerial captures covering outdoor environments, specifically designed for evaluating these applications. We will release our code and dataset to facilitate further research.