far-IR images for semantic segmentation

A long-standing problem in thermal imaging is the inherent assumption of a uniform and known emissivity across an entire image. Semantic segmentation of the materials in a thermal image can identify the pixel-wise emissivity, thus rectifying the spatially uniform emissivity assumption with no human intervention. We have created a multispectral thermal image dataset consisting of nine materials (acrylic, aluminum, bakelite, ceramic, cork, EVA, granite, maple, and silicone) at six different temperatures. Four unique neural networks (U-Net, modified-HybridSN, 3D-2D U-Net, and modified-SMFFNet) were tested to identify spectral-spatial and spatial features to semantically segment the test scenes of the dataset by material. The modified-SMFFNet performed the best with an overall accuracy of 84.5\%, an average accuracy of 82.1\%, and a Cohen's kappa score (x100) of 82.1 when evaluated against the ground truth. The resulting semantic segmentation of materials from the modified-SMFFNet was leveraged to allow unique emissivities for each pixel. Applying the surface radiative property correction decreased the average of all materials' root mean square error (RMSE) between the average surface temperatures and the corresponding thermocouple temperature from 8.2\textsuperscript{o}C to 4.3\textsuperscript{o}C. The range of the RMSE between the materials' average surface temperature and the thermocouple temperature decreased from 9.8\textsuperscript{o}C to 2.8\textsuperscript{o}C. The correction poorly models highly reflective materials in nonuniformly heated surroundings and composite materials with nonuniform surface emissivity.