Pith. sign in

REVIEW 3 major objections 5 minor 13 references

Labits: Layered Bidirectional Time Surfaces Representation for Event Camera-based Continuous Dense Trajectory Estimation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Each pixel records the time offset to the nearest past or future event at each probe time; the paper claims this preserves timing, features, and density, and that an active-pixel local flow module cuts trajectory end-point error by 49%.

desk verdict A genuinely new event representation with solid equal-size ablations, but the central-difference theory doesn't match the algorithm and the 49% headline is partly a model-size effect. read the letter →

arxiv 2412.08849 v1 pith:YZFIKEOI submitted 2024-12-12 cs.CV cs.AIcs.ET

classification cs.CVcs.AIcs.ET
keywords eventcamerasrepresentationtimesurfacesdensetrajectoryestimationactivepixellocalopticalflowBéziertrajectoriescontinuous-timemotionevent-basedvision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that how events are packed into dense tensors is itself a first-order limit on continuous-time trajectory estimation from event cameras. It introduces Labits, a layered bidirectional time-surface representation in which every pixel at each probe time stores the normalized time offset to the most recent past event or, failing that, to the next future event. The paper claims that this single construction simultaneously preserves microsecond-level timing, stable 2D edges, and uniform information density, which earlier representations did not. Coupled with a small U-Net that predicts active-pixel local optical flow from a single Labits layer, the method reports a 49% reduction in trajectory end-point error (TEPE 1.29 to 0.66) over DCT-RAFT on the MultiFlow dataset.

What carries the argument

The object that carries the argument is the Labits tensor $L \in \mathbb{R}^{B \times H \times W}$, built by Algorithm 1: for each probe time $\tau_i$, every pixel holds either the normalized time offset $(t_n - \tau_i)/\tau_{\text{range}}$ of the latest event in the past window $[\tau_i - \tau_{\text{range}}, \tau_i]$, or, if no past event exists, the offset of the earliest event in the future window $(\tau_i, \tau_i + \tau_{\text{range}}]$, or $-1$ when neither exists. Because the stored value is a time difference, local speed at active pixels is encoded as a spatial gradient of the Labits layer. The second carrying component is the Labits-to-APLOF Net, a U-Net with instance normalization that takes one Labits layer as input, masks inactive pixels with $|L| < \beta$, and outputs high- and low-resolution active-pixel local optical flow (APLOF); these features enter the RAFT-style trajectory predictor both as correlation features and as the initial hidden state of its ConvGRU.

What would settle it

Take a real event camera and a textured moving target whose true trajectory is measured independently (for example with high-speed motion capture), build Labits and APLOF from the recorded events, and compare the predicted instantaneous velocity with the measured velocity at each probe time; a systematic bias at edge pixels would indicate that the Eq. 6 supervision target is not a true local speed. Alternatively, evaluate the full pipeline on a real-event trajectory benchmark with dense ground truth; if the 49% TEPE advantage over DCT-RAFT shrinks substantially, the result is tied to MultiFlow's synthetic event statistics.

Watch

Extended reading notes

Core claim

The central claim is that a layered event representation reading time offsets in both directions—toward the latest past event and, where none exists, toward the next future event—can provide the three things dense trajectory estimation needs at once: fine-grained event timing, stable and sharp 2D features, and consistent information density across the whole interval. Prior time-surface variants keep only the most recent event per pixel and suffer temporal occlusion; voxel grids quantize time and blur it. Labits divides the interval into probe times and writes, at each probe, a normalized time difference to the nearest event within a backward/forward window of length $\tau_{\text{range}}$. The paper further claims that bidirectional search upgrades local speed estimation from backward difference (error $O(\delta t)$) to central difference (error $O(\delta t^2)$), and that the learned APLOF features inject this local speed into a RAFT-style Bézier trajectory predictor. On MultiFlow, the pure-event Labits-RAFT reduces TEPE by 29% (1.32 vs 1.85) and the events-plus-images version by 49% (0.66 vs 1.29), with comparable gains on TAE, EPE, and AE.

Load-bearing premise

The load-bearing premise is that the APLOF training target—the difference of cumulative ground-truth flows at plus and minus ten milliseconds—is a valid instantaneous local speed, and that the synthetic MultiFlow dataset's noise, texture, and motion statistics represent real event-camera streams closely enough for the 49% advantage to carry over.

Editorial extensions

If this is right

  • Switching from voxel grids to Labits within the same RAFT pipeline reduces events-plus-images TEPE from 1.16 to 1.01 (about 13%), and adding APLOF features brings it from 1.01 to 0.66 (about 35%), for a total 49% reduction relative to DCT-RAFT.
  • The full method reduces TEPE from 1.29 to 0.66 and TAE from 3.35 to 1.72 on MultiFlow compared with DCT-RAFT, with similar or larger reductions on two-view EPE and AE.
  • The APLOF net trained on Labits reaches a much lower supervised loss than the same net trained on voxel grids (0.084 vs 0.217 total loss), indicating Labits layers carry local speed information that voxel grids do not.
  • The representation is bin-size agnostic: one pretrained APLOF net transfers across time-bin sizes from 0.0125 s to 0.1000 s, with roughly 50% TEPE reduction over voxel grids at every tested bin size.
  • Bidirectional sampling fills pixels ahead of moving edges that past-only representations leave empty, increasing information density without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same layered bidirectional construction could be transferred to event-based feature tracking or frame interpolation, where intermediate motion cues, not just endpoints, matter; the paper reports gains concentrated in intermediate states, so those tasks would be natural testbeds.
  • The APLOF supervision requires ground-truth optical flow at plus and minus ten milliseconds; a self-supervised version that estimates local speed directly from event timestamps would test whether the learned module is genuinely extracting the representation's encoded speed or merely memorizing dataset-specific flow statistics.
  • Because bin size can be varied without retraining, one could generate Labits at adaptive probe times tied to local event density, spending more layers in fast-motion regions; this would trade storage for temporal resolution and is a direct extension of the paper's flexibility argument.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Labits, a layered bidirectional time-surface representation for event cameras, together with a Labits-to-APLOF module that estimates active-pixel local optical flow. The authors claim that this representation retains fine-grained temporal information, stable 2D features, and consistent information density, and that, integrated into a RAFT-style pipeline, it reduces trajectory end-point error (TEPE) on the MultiFlow dataset by 49% over DCT-RAFT, from 1.29 to 0.66. The main empirical evidence consists of comparisons against frame- and event-based RAFT variants, ablations that swap voxel grids for Labits while keeping the same 6.8M-parameter pipeline, one-way-versus-bidirectional Labits ablations, and a bin-size sensitivity study.

Significance. If the empirical claims hold, the paper would make a useful contribution by showing that event representation choice is a first-order factor in dense continuous-time trajectory estimation. The controlled ablation in Table 2 (replacing voxel grids with Labits in the same 6.8M-parameter DCT-RAFT pipeline reduces TEPE from 1.29 to 1.01) is a strong and creditworthy experimental design. The supplementary comparison against other layered representations and the bin-size study further support the practical value of the representation. However, the paper's central theoretical motivation, the claimed O(dt^2) central-difference advantage, is not realized by the algorithm as written, and the headline result is reported without statistical uncertainty or a matched-parameter comparison. These issues do not necessarily invalidate the empirical finding, but they require substantial revision before the claims can be accepted as stated.

major comments (3)
  1. [Section 3 / Algorithm 1 / Supplementary A.2] The central theoretical claim that Labits enables central-difference local speed estimation with error O(dt^2) is not supported by the algorithm as written. In Algorithm 1, lines 5-14, Tprev is overwritten by each past event in ascending order and therefore ends as the most recent past event; Tfuture is likewise overwritten and ends as the latest (farthest) future event in the bin; and line 14 selects Tprev whenever it is non-infinite. Consequently, every pixel stores at most one timestamp, never both f(x_n - dt) and f(x_n + dt), so the central-difference derivation in Eqs. 12-15 does not apply. Moreover, the future event stored is the farthest event in the future bin, not the 'next future event' described in the Figure 1 caption. The authors should either correct the pseudocode and the description so that the representation actually combines near-past and near-future events, or revise the theoretical motivation to match the implemented past-preferred, future-fallback behavior.
  2. [Table 2 / Supplementary Table 3] The headline 49% reduction compares the full 25.3M-parameter Labits-RAFT against the 6.8M-parameter DCT-RAFT, while the same-size ablation (Labits-RAFT without APLOF features, 6.8M parameters) gives TEPE 1.01, a 22% reduction. The 49% figure therefore conflates the representation change with a large increase in model capacity. The paper should report a matched-parameter comparison as the primary representation-only effect and clearly state the parameter counts alongside each result. In addition, no error bars or multiple-seed results are reported for any of the metrics; given that several ablation gaps are small (e.g., 0.72 vs 0.66 in Table 5), the significance of the improvements needs to be established.
  3. [Section 4.1, Eq. (6)] The APLOF ground-truth definition, A_tau(x_tau) = (O_{tau+}(x_start) - O_{tau-}(x_start)) * M_h(x), requires clarification. The cumulative flow O_tau is defined at the reference-time pixel x_start, whereas the label is assigned to x_tau = x_start + O_tau(x). If x_tau is not an integer pixel coordinate, the paper does not specify how the label is resampled or interpolated. Additionally, the target is a difference of cumulative flows over a 20 ms interval centered at tau, not an instantaneous velocity; the validity of this target as 'local optical flow' and the choice of the 10 ms offset are not justified or ablated. The authors should provide the exact label-generation procedure and an ablation or discussion of the offset size.
minor comments (5)
  1. [Introduction] The Introduction states that 'simply switching to Labits' yields a 13% improvement on TEPE, but Table 2 shows a 22% improvement for the matched-parameter comparison (1.29 to 1.01) and 17% for the event-only case (1.85 to 1.53). The percentage should be corrected for consistency.
  2. [Figure 1 caption] The caption says 'it searches for the next future event within dt,' but Algorithm 1 stores the latest future event within the bin when no past event exists. The caption and pseudocode should describe the same behavior.
  3. [Supplementary Table 6] The table header says 'DCF-RAFT' while the rest of the paper uses 'DCT-RAFT'; please correct the typo.
  4. [Various] There are several typographical and formatting issues, including 'utlizes' in the contributions list, 'V oxel' spacing in the text and tables, and 'groud truth' in Figure 9. A careful proofread is needed.
  5. [Section 6] The statement that Labits is 'the first to achieve this combination' is an overclaim; the paper does not provide a systematic survey to establish priority. Rephrase as a property of the proposed representation rather than a novelty claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central TEPE result is an external-data benchmark of a deterministic representation, and the self-citations are not load-bearing.

full rationale

The paper's central claim is an empirical comparison on the MultiFlow dataset against external baselines, not a derived prediction. Labits is a deterministic transform of event timestamps (Algorithm 1) with no parameters fitted to the evaluation metric and re-reported as a prediction. The APLOF ground truth in Eq. 6 is computed by finite differences of the same ground-truth flows that supervise the Bézier trajectory loss in Eq. 11; this is standard multi-task supervision rather than circularity, since the APLOF head outputs local speeds from a Labits layer and is only one feature injected into an independently supervised trajectory model. The two self-citations (V2CE and prior pose-tracking work) appear in background and definitions and supply none of the assumptions needed for the MultiFlow experiments. The Supplementary A.2 central-difference error analysis is not matched by Algorithm 1, because each pixel stores either a past or a future normalized timestamp (with future events overwritten to the farthest in the bin), so no Labits layer forms a true central difference; however, this is a correctness and consistency concern about the stated theory, not a reduction of the benchmark result to its inputs. The representation is ablated against one-way Labits and voxel-based counterparts, and the 49% TEPE improvement is measured on the held-out MultiFlow test set, so the central empirical claim retains independent content.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a handful of domain assumptions about event kinematics and the fidelity of the MultiFlow simulator, plus two tuned thresholds; no derivation in the paper is circular with its own conclusions. No new physical entities are introduced; Labits and APLOF are data structures and learned features, not postulated particles, forces, or conserved quantities.

free parameters (4)
  • Active pixel mask threshold beta = 0.3
    Equation 1: pixels with |L| < beta are considered active; tuned after fine-tuning, affects APLOF features and hence final TEPE.
  • Low-resolution APM threshold gamma = 0.125
    Equation 2: downsampled mask threshold; used for LR APLOF auxiliary head; tuned with beta.
  • Number of correlation Labits layers M and context layers N = M=25, N=41
    Implementation details: chosen by hand; results depend on temporal resolution of the representation.
  • Time bin duration tau_range = 0.0125 s (main result)
    Algorithm 1 bin size; ablation across 0.0125, 0.025, 0.05, 0.1 s shows performance degrades with larger bins.
assumptions (5)
  • domain assumption Events triggered by the same moving object within a short time window are closely tied to the object's instantaneous speed via v = dx/dt.
    Section 3 motivation; underlies the entire Labits design.
  • domain assumption The most recent event per pixel within a dt search window contains sufficient information for local speed estimation.
    Algorithm 1 keeps only one event per pixel per layer; all older events within the window are discarded.
  • domain assumption GT APLOF can be computed as A_tau(x_tau) = (O_{tau+}(x_start) - O_{tau-}(x_start)) * M_h(x) (Eq. 6).
    Assumes cumulative GT optical flow at plus and minus 10 ms can be differenced to approximate instantaneous local flow; this defines the supervision target for the APLOF net.
  • ad hoc to paper Bidirectional search yields central-difference accuracy O(dt^2).
    Supplementary A.2; the argument is standard finite differences, but Algorithm 1 stores either a past or future time per pixel, not both, so the central-difference claim does not directly apply to the representation.
  • domain assumption MultiFlow is representative of real event-camera streams.
    All experiments are on MultiFlow; no real-world dataset evaluation; synthetic-to-real transfer is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Labits: Layered Bidirectional Time Surfaces Representation for Event Camera-based Continuous Dense Trajectory Estimation." pith.science (2026). https://pith.science/paper/YZFIKEOI

@misc{pith2026241208849,
  author       = {Pith},
  title        = {Pith review of: Labits: Layered Bidirectional Time Surfaces Representation for Event Camera-based Continuous Dense Trajectory Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YZFIKEOI}},
  note         = {Machine review of arXiv:2412.08849}
}
read the original abstract

Event cameras provide a compelling alternative to traditional frame-based sensors, capturing dynamic scenes with high temporal resolution and low latency. Moving objects trigger events with precise timestamps along their trajectory, enabling smooth continuous-time estimation. However, few works have attempted to optimize the information loss during event representation construction, imposing a ceiling on this task. Fully exploiting event cameras requires representations that simultaneously preserve fine-grained temporal information, stable and characteristic 2D visual features, and temporally consistent information density, an unmet challenge in existing representations. We introduce Labits: Layered Bidirectional Time Surfaces, a simple yet elegant representation designed to retain all these features. Additionally, we propose a dedicated module for extracting active pixel local optical flow (APLOF), significantly boosting the performance. Our approach achieves an impressive 49% reduction in trajectory end-point error (TEPE) compared to the previous state-of-the-art on the MultiFlow dataset. The code will be released upon acceptance.

Figures

Figures reproduced from arXiv: 2412.08849 by the authors.

Figure 1
Figure 1. (a) Labits generation schematic: For a 1D event camera, at each pixel and probe time [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) Labits-RAFT architecture: Labits are used to generate correlation blocks, content fea [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Visualization of detailed inputs and outcomes from our model. It predicts instantaneous [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Trajectory predictions on the MultiFlow dataset by our proposed model and baseline methods. Ground truth Bezier trajectories are shown in red, while predictions are depicted in blue. ´ The background displays the ground truth optical flow to highlight moving objects. O…
Figure 5
Figure 5. Figure 5: Comparison of trajectory predictions: between baseline methods and our approach. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Comparison of Optical Flow Estimations: Each method includes a second column dis [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Visualization of APLOF and Labits in Trajectories: For each trajectory, Labits and APLOF [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Visualization of HR and LR APLOF with APM Ablation: For each trajectory, we display [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Visualization of Intermediate Prediction Results: We display dense, time-continuous pre￾diction outcomes for each trajectory within a sample window from the MultiFlow dataset Gehrig et al. (2024). The visualization includes ten intermediate instances of optical flow (O…
Figure 10
Figure 10. Figure 10: Visual Comparison of Labits in Various Scenarios: We generate Labits on three well￾known event camera datasets: DSEC Gehrig et al. (2021a), MVSEC Zhu et al. (2018a), and Multi￾Flow Gehrig et al. (2024). The time ranges and the number of bins are adjusted to accommodat…
Figure 11
Figure 11. Figure 11: Comparative Visualization of Event Representations: This figure presents event data [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Visual Comparison of Labits and One-way Labits: We visually compare Labits and One-way Labits generated from the same icon sample, as presented in [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 10 canonical work pages

  1. [1]

    Haste: multi-hypothesis asynchronous speeded-up tracking of events

    Ignacio Alzugaray and Margarita Chli. Haste: multi-hypothesis asynchronous speeded-up tracking of events. In 31st British Machine Vision Virtual Conference (BMVC 2020), pp

  2. [6]

    Lightweight event-based optical flow estimation via iterative deblurring

    Yilun Wu, Federico Paredes-Vall ´es, and Guido CHE de Croon. Lightweight event-based optical flow estimation via iterative deblurring. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 14708–14715. IEEE,

  3. [7]

    Towards anytime optical flow estimation with event cameras

    Yaozu Ye, Hao Shi, Kailun Yang, Ze Wang, Xiaoting Yin, Yining Lin, Mao Liu, Yaonan Wang, and Kaiwei Wang. Towards anytime optical flow estimation with event cameras. arXiv preprint arXiv:2307.05033,

  4. [8]

    V2ce: Video to continuous events simulator

    Zhongyang Zhang, Shuyang Cui, Kaidong Chai, Haowen Yu, Subhasis Dasgupta, Upal Mahbub, and Tauhidur Rahman. V2ce: Video to continuous events simulator. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 12455–12461. IEEE,

  5. [9]

    The multivehicle stereo event camera dataset: An event camera dataset for 3d perception

    Alex Zihao Zhu, Dinesh Thakur, Tolga ¨Ozaslan, Bernd Pfrommer, Vijay Kumar, and Kostas Dani- ilidis. The multivehicle stereo event camera dataset: An event camera dataset for 3d perception. IEEE Robotics and Automation Letters, 3(3):2032–2039, 2018a. Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, and Kostas Daniilidis. Ev-flownet: Self- supervised optical...

  6. [12]

    Detailed ablation studies focus on different bin configurations for Labits and V oxel Grid, specifically in the context of trajectory estimation

    Table 4: Comparison of APLOF Losses Model Total Loss LR APLOF L1 Loss HR APLOF L1 Loss Labits-to-APLOF Net 0.084 0.056 0.028 Labits-to-APLOF Net (One-way) 0.357 0.199 0.158 Table 5: Performance Metrics of Model Pipelines Method Input TEPE TAE EPE AE DCT-RAFT E+I 1.29 3.35 2.27 3.19 Labits-RAFT (Ours) E+I 0.66 1.72 1.08 1.45 Labits-RAFT (One-way) E+I 0.72 ...

  7. [13]

    Further- more, Table 8 details the performance metrics of the complete model pipeline that incorporates the V oxel-to-APLOF features and voxel

    between the Labits-to-APLOF net and the V oxel-to-APLOF net. Further- more, Table 8 details the performance metrics of the complete model pipeline that incorporates the V oxel-to-APLOF features and voxel. Furthermore, we deploy event representations that are computationally feasible and align well with the layered representation requirements. In this cont...

  8. [14]

    This comparison not only facilitates a thorough evaluation of each representation’s efficacy under consistent conditions but also validates the effectiveness of Labits

    These methods fit within our computational constraints and ensure fair comparisons across different tasks. This comparison not only facilitates a thorough evaluation of each representation’s efficacy under consistent conditions but also validates the effectiveness of Labits. Table 9: Comparison of more recent layered event representations applied to the R...

Show all 13 references
  1. [2017]

    U-net: Convolutional networks for biomed- ical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-assisted intervention– MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings, ...

  2. [2019]

    13 A S UPPLEMENTARY MATERIAL (b) DCT-RAFT (Voxel)(c) Oursw/oAPLOF(d) Ours(a) E-RAFT (Voxel) Figure 5: Comparison of trajectory predictions: between baseline methods and our approach. 14 (a) DCT-RAFT(Voxel)(b) Oursw/oAPLOF(c) Ours(d) GT Figure 6: Comparison of Optical Flow Esti...

  3. [2020]

    Dsec: A stereo event camera dataset for driving scenarios

    Mathias Gehrig, Willem Aarents, Daniel Gehrig, and Davide Scaramuzza. Dsec: A stereo event camera dataset for driving scenarios. IEEE Robotics and Automation Letters , 6(3):4947–4954, 2021a. Mathias Gehrig, Mario Millh ¨ausler, Daniel Gehrig, and Davide Scaramuzza. E-raft: Den...

  4. [2023]

    Raft: Recurrent all-pairs field transforms for optical flow

    12 Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pp. 402–419. Springer,

  5. [2024]

    Adaptive-spikenet: event-based optical flow estimation using spiking neural networks with learnable neuronal dynamics

    Adarsh Kumar Kosta and Kaushik Roy. Adaptive-spikenet: event-based optical flow estimation using spiking neural networks with learnable neuronal dynamics. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 6021–6027. IEEE,

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.