/Event-based-Vision-for-Robotics

[IEEE SPM 2020] Collect some papers about event-based Autonomous Driving & Event-based Robotic-Grasping.

MIT LicenseMIT

Awsome-Event-based-Vision-for-Robotics (Autonomous-Driving & Robotic-Grasping)

Collect some papers about event-based Autonomous Driving & Event-based Robotic-Grasping.

If you find some overlooked papers, please open issues or pull requests (recommended).

Papers

Survey papers

Year Paper Publication Code/Webpage
2020 Event-based Vision: A Survey TPAMI Code
2020 Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception IEEE Signal Processing Magazine Code

Datasets

Datasets about Autonomous Driving

Year Paper Publication Code/Webpage
2017 DDD17: End-To-End DAVIS Driving Dataset Int. Conf. Machine Learning Dataset
2018 N-CARS Dataset: A large real-world event-based dataset for car classification. CVPR Link
2018 The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception IEEE Robotics and Automation Letters (RA-L) Dataset , YouTube.
2019 DET: A High-resolution DVS Dataset for Lane Extraction CVPRW Code
2020 NeuroIV: Neuromorphic Vision Meets Intelligent Vehicle Towards Safe Driving With a New Database and Baseline Evaluations TITS -
2020 A Large Scale Event-based Detection Dataset for Automotive arXiv Code,1Mpx Detection Dataset
2020 DDD20 End-to-End Event Camera Driving Dataset: Fusing Frames and Events with Deep Learning for Improved Steering Prediction ITSC Dataset , More datasets
2021 TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset IROS -
2021 DSEC: A Stereo Event Camera Dataset for Driving Scenarios IEEE Robotics and Automation Letters (RA-L) Dataset, Code , Video
2022 PKU-DAVIS-SOD Dataset by National Engineering Research Center for Visual Technology Dataset -

Datasets about Robotic Grasping

Year Paper Publication Code/Webpage
2019 NeuroGrasp: Multimodal Neural Network With Euler Region Regression for Neuromorphic Vision-Based Grasp Pose Estimation IEEE Transactions on Instrumentation and Measurement (TIM) Dataset
2021 Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset Frontiers in Neurorobotics Dataset

Papers about Object Detection in Autonomous Driving

Year Paper Publication Code/Webpage
2019 Multi-Cue Event Information Fusion for Pedestrian Detection With Neuromorphic Vision Sensors Front. Neurorobot -
2019 [Mixed Frame-/Event-Driven Fast Pedestrian Detection ICRA -
2020 Learning to Detect Objects with a 1 Megapixel Event Camera NeurIPS
2021 N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras ICCV Suppl. Mat., Github Page, N-ImageNet Dataset.
2021 Fusion-based Feature Attention Gate Component for Vehicle Detection based on Event Camera IEEE Sensors Journal
2022 Fusing Asynchronous Event and Synchronization Frame Data with Multi-layer Feature Attention Guided Filtering Research Square
2022 Fusing Event-based and RGB camera for Robust Object Detection in Adverse Conditions ICRA
2022 Retinomorphic Object Detection in Asynchronous Visual Streams Proceedings of the AAAI Conference on Artificial Intelligence
2022 LDNet: End-to-End Lane Marking Detection Approach Using a Dynamic Vision Sensor TITS
2022 Asynchronous Spatio-Temporal Memory Network for Continuous Event-Based Object Detection IIEEE Transactions on Image Processing
2022 Traffic Sign Detection With Event Cameras and DCNN SPA
2023 GET: Group Event Transformer for Event-Based Vision ICCV Code
2023 Joint Intensity and Event Framework for Vehicle Detection in Degraded Conditions ICTIS
2023 RGB-Event Fusion for Moving Object Detection in Autonomous Driving ICRA Code
2023 Recurrent Vision Transformers for Object Detection with Event Cameras CVPR Code, YouTube. [Gen 1 Dataset]
2023 SODFormer: Streaming Object Detection with Transformer Using Events and Frames IEEE Transactions on Pattern Analysis and Machine Intelligence code

Papers about Semantic Segmentation in Autonomous Driving

Year Paper Publication Code/Webpage
2019 EV-SegNet: Semantic Segmentation for Event-based Cameras CVPRW PDF. Project page. Video pitch
2021 ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data IROS
2021 Moving Object Detection for Event-based vision using Graph Spectral Clustering ICCVW Code
2022 ESS: Learning Event-based Semantic Segmentation from Still Images ECCV Code
2022 Exploring Event-driven Dynamic Context for Accident Scene Segmentation IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS Code
2023 Event-Based Semantic Segmentation With Posterior Attention IEEE TRANSACTIONS ON IMAGE PROCESSING Code
2023 CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers IEEE Transactions on Intelligent Transportation Systems Code
2024 EISNet: A Multi-Modal Fusion Network for Semantic Segmentation with Events and Images IEEE Transactions on Multimedia Code

Papers about Autonomous Driving

Year Paper Publication Code/Webpage
2018 Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars CVPR PDF, Poster, YouTube.
2019 Event-Based Vision Enhanced: A Joint Detection Framework in Autonomous Driving ICME -
2019 Live Demonstration: Unsupervised Event-Based Learning of Optical Flow, Depth and Egomotion CVPRW Project page
2019 Slasher: Stadium racer car for event camera end-to-end learning autonomous driving experiments AICAS Code, Project page.
2020 Learning Monocular Dense Depth from Events IEEE Int. Conf. on 3D Vision (3DV) PDF, YouTube, Code, Project Page.
2020 EDDD: Event-based drowsiness driving detection through facial motion analysis with neuromorphic vision sensor IEEE Sensors Journal
2020 Learning to Exploit Multiple Vision Modalities by Using Grafted Networks ECCV
2020 NeuroIV: Neuromorphic Vision Meets Intelligent Vehicle Towards Safe Driving With a New Database and Baseline Evaluations TITS
2021 Combining Events and Frames Using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction IEEE Robotics and Automation Letters(RA-L)] PDF, Code, Project Page.
2021 EvDistill: Asynchronous Events To End-Task Learning via Bidirectional Reconstruction-Guided Cross-Modal Knowledge Distillation CVPR Code.
2022 Event-Based Fusion for Motion Deblurring with Cross-modal Attention ECCV code
2023 Multi-Modal Fusion for Sensorimotor Coordination in Steering Angle Prediction Engineering Applications of Artificial Intelligence) code
2023 Frame-Event Alignment and Fusion Network for High Frame Rate Tracking CVPR code
2024 FE-DeTr: Keypoint Detection and Tracking in Low-quality Image Frames with Events ICRA code

Papers about Robotic Grasping

Year Paper Publication Code/Webpage
2021 Event-Based Robotic Grasping Detection With Neuromorphic Vision Sensor and Event-Grasping Dataset Frontiers in Neurorobotics Dataset.
2021 NeuroGrasp: Multimodal Neural Network With Euler Region Regression for Neuromorphic Vision-Based Grasp Pose Estimation IEEE Transactions on Instrumentation and Measurement (TIM) Dataset.

Papers about Other tasks

Year Paper Publication Code/Webpage
2023 Non-Coaxial Event-guided Motion Deblurring with Spatial Alignment ICCV
2023 RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation ICCV Project page

arXiv papers

Year Paper Publication
2023 EvPlug: Learn a Plug-and-Play Module for Event and Image Fusion paper
2023 Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion paper
2023 Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks paper
2023 MetaBEV: Solving Sensor Failures for BEV Detection and Map Segmentation paper, Code,Project page.
2024 Scene Adaptive Sparse Transformer for Event-based Object Detection paper, code
2024 Event-assisted Low-Light Video Object Segmentation paper
2024 Bilateral Event Mining and Complementary for Event Stream Super-Resolution paper
2024 Retain, Blend, and Exchange: A Quality-aware Spatial-Stereo Fusion Approach for Event Stream Recognition paper, Code