Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:UT7QQAJErecord.jsonopen to challenge →
read the original abstract
Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously and produce event streams encoding the time, pixel position, and polarity (sign) of the intensity changes. Event cameras possess a myriad of advantages over canonical frame-based cameras, such as high temporal resolution, high dynamic range, low latency, etc. Being capable of capturing information in challenging visual conditions, event cameras have the potential to overcome the limitations of frame-based cameras in the computer vision and robotics community. In very recent years, deep learning (DL) has been brought to this emerging field and inspired active research endeavors in mining its potential. However, there is still a lack of taxonomies in DL techniques for event-based vision. We first scrutinize the typical event representations with quality enhancement methods as they play a pivotal role as inputs to the DL models. We then provide a comprehensive survey of existing DL-based methods by structurally grouping them into two major categories: 1) image/video reconstruction and restoration; 2) event-based scene understanding and 3D vision. We conduct benchmark experiments for the existing methods in some representative research directions, i.e., image reconstruction, deblurring, and object recognition, to identify some critical insights and problems. Finally, we have discussions regarding the challenges and provide new perspectives for inspiring more research studies.
This paper has not been read by Pith yet.
Forward citations
Cited by 8 Pith papers
-
DeLux: Cross-Modal Local Artifact Restoration in Video Using Neuromorphic Data
DeLux restores local lighting artifacts in RGB video by leveraging neuromorphic event data, outperforming RGB-only and event-guided HDR baselines with MS-SSIM over 0.99 and up to 88% artifact reduction.
-
Adaptive Control in Autonomous Driving via Real-Time Recurrent RL
Combines offline behavioral cloning with online Real-Time Recurrent RL fine-tuning on LrcSSM models to adapt autonomous driving policies to distribution shifts, validated in simulation and on a real 1:10-scale robot w...
-
A Hardware-Aware Open-Source Framework for Design Space Exploration of Mixed-Signal Spiking Neural Networks
An open-source PyTorch framework embeds calibrated floating-gate and ReRAM synapse non-idealities and mixed-signal neuron models directly into SNN training, enabling cross-layer design space exploration across accurac...
-
Brain-inspired spike-timing plasticity for reliable label-efficient event-camera vision
Local STDP modules enable label-efficient event-camera detection with 78.6% mAP on drone benchmarks and better drift handling than k-means.
-
Event-VLA: Action-Conditioned Event Fusion for Robust Vision-Language-Action Model
Event-VLA integrates event streams into VLA models through action-conditioned gated cross-attention to maintain performance in normal light while improving success rates under low-light and near-dark conditions.
-
EventCrab: Harnessing Frame and Point Synergy for Event-based Action Recognition and Beyond
EventCrab integrates frame and point networks with a joint representation space, SCL, and Hilbert-scan EPE to improve event-based action recognition by 5-7% on two datasets.
-
Memristor Technologies for Dynamic Vision Sensors: A Critical Assessment and Research Roadmap
A structured review concludes that end-to-end DVS-memristor integration for analog in-memory event-driven computing remains an open challenge at TRL 2-5, with half of surveyed applications resting on projections rathe...
-
A Systematic Survey on Event Camera Representation Learning
A survey that categorizes event camera representation learning into dense-based and sparse-based methods, examining design choices, benchmarks, and open problems.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.