Reconstruct Clear Image for High-Speed Motion Scene with Retina-Inspired Spike Camera - Supplementary Material

Supplementary material for the paper: 'Reconstruct Clear Image for High-Speed Motion Scene with Retina-Inspired Spike Camera' PAPER ABSTRACT: Conventional digital cameras typically accumulate all the photoelectric information within an exposure window to form a snapshot image. It requires the scene to be quite still during the exposure interval, otherwise it would result in blurry image for the moving objects. Recently, a retina-inspired spike camera has been proposed and shown great potential for recording high-speed motion scene. Instead of capturing the visual scene by a single snapshot, the spike camera records the dynamic changing process of light intensity continuously. Each pixel on spike camera sensor accumulates the incoming photons independently and persistently, and fires spikes once the dispatch threshold is reached, producing a continuous stream of spikes recorded at very high temporal resolution. To recover the dynamic scene from the captured spike data, we present an image reconstruction approach for spike camera. In order to generate high-quality reconstruction, we investigate the temporal correlation along motion trajectories and exploit it via an adaptive temporal auto-regressive model. In particular, we present a hierarchical motion-aligned temporal filtering scheme, combining short-term filtering with long-term filtering to take advantage of long-term temporal correlation with much lower model complexity. Experimental results demonstrate that the proposed method produces high-quality images for high-speed motion scenes, improving the reconstruction images in terms of both objective and subjective qualities. This dataset provides a set of results and demos for our work on spike camera image reconstruction. The work is submitted to IEEE Transactions on Image Processing.

论文《基于视网膜类脉冲相机的高速运动场景清晰图像重建》的补充材料

论文摘要：传统数码相机通常会在曝光窗口内累积所有光电信息以生成单帧快照图像，这要求拍摄场景在曝光时段内保持完全静止，否则运动目标会导致图像模糊。近年来，视网膜类脉冲相机（Spike Camera）被提出并展现出记录高速运动场景的巨大潜力。该脉冲相机并非通过单帧快照捕获视觉场景，而是持续记录光强的动态变化过程。脉冲相机传感器的每个像素会独立且持续地累积入射光子，一旦达到触发阈值便会产生脉冲，从而生成以极高时间分辨率记录的连续脉冲流。为从捕获的脉冲数据中恢复动态场景，我们提出了一种面向脉冲相机的图像重建方法。为生成高质量的重建结果，我们研究了运动轨迹上的时间相关性，并通过自适应时间自回归模型加以利用。具体而言，我们提出了一种分层运动对齐时间滤波方案，结合短时滤波与长时滤波，在大幅降低模型复杂度的同时充分利用长期时间相关性。实验结果表明，所提方法可为高速运动场景生成高质量图像，在客观与主观质量两方面均实现了重建图像的提升。本数据集为我们在脉冲相机图像重建领域的研究工作提供了一系列结果与演示样例。本论文已提交至《IEEE Transactions on Image Processing》（IEEE图像处理汇刊）。