FAST-EDI

<strong># Fast Event-based Double Integral for Real-time Robotics (Academic Use Only)</strong><br> <strong>## [Paper](</strong>https://arxiv.org/abs/2305.05925<strong>) | [Video: Youtube](</strong>https://www.youtube.com/watch?v=xzrHNA97wls<strong>) | [Video: Bilibili](</strong>https://www.bilibili.com/video/BV1qL411X7hc/?share_source=copy_web&amp;vd_source=2483c9488f1bd3f3478cf69bfca4d49e<strong>)</strong><br> <br> _{<br> Motion deblurring is a critical ill-posed problem that is important in many vision-based robotics applications. The recently proposed event-based double integral (EDI) provides a theoretical framework for solving the deblurring problem with the event camera and generating clear images at high frame-rate. However, the original EDI is mainly designed for offline computation and does not support real-time requirement in many robotics applications. In this paper, we propose the fast EDI, an efficient implementation of EDI that can achieve real-time online computation on single-core CPU devices, which is common for physical robotic platforms used in practice. In experiments, our method can handle event rates at as high as 13 million event per second in a wide variety of challenging lighting conditions. Its benefit has been demonstrated on multiple downstream real-time applications, including localization, visual tag detection, and feature matching.<br>}<br> <br> <br> <br> <strong>## Understanding the hardware bias of event camera</strong><br> [Tutorial of the bias](https://gitlab.com/inivation/inivation-docs/blob/master/Advanced configurations/User_guide_-_Biasing.md)<br> <strong>## Checking the hardware bias using the jAER</strong><br> One can use the jAER project to get a roughly estimated contrast from the bias currents that are estimated to be generated by the on-chip bias generator bias current ratios:[jAER homepage](https://github.com/SensorsINI/jaer)<br> <br> Discussion regarding the bias estimation:[discussion](https://groups.google.com/g/davis-users/c/68gp0zxTMUk/m/SpweyJKrDgAJ)<br> <br> and the physical model that build to estimate the threshold:[Paper](https://ieeexplore.ieee.org/document/7962235/)<br> <br> <strong>## Usage</strong><br> Test env:<br> ```<br> ubuntu 18.04 <br> dv-runtime 1.6.1 <br> dv-gui 1.6.0<br> ```<br> <br> 1. download the DV and install it following the tutorial [DV install guide](https://inivation.gitlab.io/dv/dv-docs/docs/getting-started.html)<br> 2. download the fast EDI code<br> ```<br> git clone https://github.com/eleboss/fast_EDI<br> ```<br> 3. compile the fast EDI module<br> ```<br> cd ./dv-module/fast-edi<br> cmake ./<br> make<br> ```<br> then you will see the `fedi_FEDI.so`, this is the file you need to add to the dv-gui.<br> <br> 4. configure the DV-GUI search path<br> ```<br> dv-gui<br> ```<br> find the `structure - add modules - modify module search path - add path` (this path leads to `fedi_FEDI.so`, for me is `/home/eleboss/Documents/fast_EDI/dv-module/fast-edi`). Then you can add the fast edi modules to the dv-gui and wires it in this way:<br> <br> <br> 5. download the test data [dataset](www.google.com), and play it. <br> <br> Performance tips: You can tune the contrast or use the jAER to estimate, or use EDI to optimize an accurate contrast.<br> <br> <strong>## Citation</strong><br> ```<br> @article{lin2023fast,<br> title={Fast Event-based Double Integral for Real-time Robotics},<br> author={Lin, Shijie and Zhang, Yingqiang and Huang, Dongyue and Zhou, Bin and Luo, Xiaowei and Pan, Jia},<br> booktitle={international conference on robotics and automation (ICRA)},<br> year={2023},<br> organization={IEEE}<br> }<br> ```<br> <br>

**# 用于实时机器人学的快速事件双积分法（仅学术用途）**
**## [论文](https://arxiv.org/abs/2305.05925) | [视频：YouTube](https://www.youtube.com/watch?v=xzrHNA97wls) | [视频：哔哩哔哩](https://www.bilibili.com/video/BV1qL411X7hc/?share_source=copy_web&vd_source=2483c9488f1bd3f3478cf69bfca4d49e)**

运动去模糊是一类关键的病态逆问题，在诸多基于视觉的机器人学应用中具有重要价值。近期提出的事件双积分法（Event-based Double Integral, EDI）为利用事件相机（event camera）求解去模糊问题提供了理论框架，并可生成高帧率清晰图像。然而，原始EDI主要面向离线计算设计，无法满足诸多机器人学应用的实时性需求。本文提出快速事件双积分法（Fast EDI），一种高效的EDI实现方案，可在单核CPU设备上完成实时在线计算——这也是实际物理机器人平台中常见的硬件配置。实验结果表明，我们的方法可在多种复杂光照条件下处理最高达1300万事件每秒的事件流。其优势已在多项下游实时应用中得到验证，包括定位、视觉标签检测与特征匹配。

**## 理解事件相机的硬件偏置**
[偏置教程](https://gitlab.com/inivation/inivation-docs/blob/master/Advanced configurations/User_guide_-_Biasing.md)
**## 使用jAER检测硬件偏置**
用户可借助jAER项目，基于片上偏置发生器的偏置电流比，估算得到近似对比度：[jAER主页](https://github.com/SensorsINI/jaer)
关于偏置估计的讨论：[讨论帖](https://groups.google.com/g/davis-users/c/68gp0zxTMUk/m/SpweyJKrDgAJ)
以及用于估算阈值的物理模型：[论文](https://ieeexplore.ieee.org/document/7962235/)

**## 使用方法**
测试环境：

Ubuntu 18.04
dv-runtime 1.6.1
dv-gui 1.6.0

1. 下载DV工具包并按照[DV安装指南](https://inivation.gitlab.io/dv/dv-docs/docs/getting-started.html)完成安装
2. 下载快速EDI代码：

git clone https://github.com/eleboss/fast_EDI

3. 编译快速EDI模块：

cd ./dv-module/fast-edi
cmake ./
make

编译完成后将生成`fedi_FEDI.so`文件，该文件需添加至dv-gui中。
4. 配置DV-GUI的模块搜索路径：

dv-gui

依次点击`结构 - 添加模块 - 修改模块搜索路径 - 添加路径`，将路径指向`fedi_FEDI.so`所在目录（例如本机路径为`/home/eleboss/Documents/fast_EDI/dv-module/fast-edi`）。随后即可在dv-gui中添加快速EDI模块，并按照如下方式连线：
5. 下载测试数据集[数据集](www.google.com)并运行播放。

性能优化提示：可调节对比度参数，或借助jAER进行估算，也可使用EDI方法优化得到精准对比度。

**## 引用格式**

@article{lin2023fast,
title={Fast Event-based Double Integral for Real-time Robotics},
author={Lin, Shijie and Zhang, Yingqiang and Huang, Dongyue and Zhou, Bin and Luo, Xiaowei and Pan, Jia},
booktitle={国际机器人与自动化会议（ICRA）},
year={2023},
organization={IEEE}
}