{"id":"0cd74f5e-35b9-427e-bb0d-3ea5861d934b","arxiv_id":"2412.08849","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A layered bidirectional time-surface representation plus a local-flow feature extractor reduces dense trajectory end-point error by 49% on the MultiFlow event-camera benchmark.","lead":"Event cameras record brightness changes at microsecond precision, and this paper proposes a new way to pack timestamp streams into layered images so moving-object trajectories can be estimated more accurately. Their method cuts trajectory end-point error by 49% on the MultiFlow benchmark relative to the previous best result, showing that event representation design matters as much as network architecture.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 stores only one of past/future timestamps per pixel and, for future events, keeps the farthest rather than nearest event in the bin, so the claimed O(dt^2) central-difference derivation in Supplementary A.2 is not realized by the representation as written.","rationale":"I agree with the reader's identification of the central-difference mismatch as the weakest load-bearing premise. The concern is more specific than 'no code provided': Algorithm 1 as printed is internally inconsistent with the stated design and with the derivation in A.2. This matters because the paper's narrative contribution is that bidirectional time surfaces yield higher-order accuracy via central differencing; if the representation never forms a central difference, that contribution is not established, and the empirical gains must be attributed to other factors (e.g., filling previously empty pixels). The equal-size ablation in Table 3 (6.8M params, 1.29 vs 1.01) and the one-way vs bidirectional ablation (0.72 vs 0.66) are reassuring that Labits and bidirectionality help in practice, but they do not rescue the theoretical claim. The missing code and absence of error bars further limit verification, but they are secondary to the concrete algorithmic contradiction. A CONDITIONAL verdict remains appropriate: the empirical result is plausible and partially controlled, but the representation's defining mechanism is not reproducible as written. I recommend no change to the reader's verdict.","tokens_in":17377,"tokens_out":9634,"duration_ms":98093,"concrete_test":"Implement Algorithm 1 exactly as written on a synthetic 1D event stream with constant velocity, placing events at the same pixel at t = tau_i - delta and t = tau_i + epsilon (both within one bin). Print L[i,y,x], Tprev, and Tfuture. If L equals the past normalized time and Tfuture equals the maximum (not minimum) future timestamp, the algorithm does not produce the nearest-future value or a central difference. Then re-run the bidirectional-vs-one-way ablation from Table 5 with a corrected implementation that stores the nearest future event when no past event exists; if the TEPE gap between 0.66 and 0.72 changes materially, the reported bidirectional benefit depends on which future event is stored.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's theoretical motivation for Labits, stated in Section 3 and Supplementary A.2, is that storing both near-past and near-future events at each probe time changes local speed estimation from backward difference (error O(dt)) to central difference (error O(dt^2)). Algorithm 1 does not do this. At each probe time tau_i (lines 5-14), it initializes Tprev and Tfuture, then updates each to the normalized timestamp of every event in the corresponding range; because events are processed in ascending timestamp order, Tprev ends as the most recent past event, but Tfuture ends as the latest (farthest) future event in (tau_i, tau_i + tau_range], not the 'next future event' promised in the Figure 1 caption. Moreover, line 14 selects Tprev whenever it is non-infinite, so a pixel that has both a past and a future event in the bin stores only the past value. Thus no pixel in any single Labits layer carries both f(x_n - dt) and f(x_n + dt), and the representation cannot form a central difference. The O(dt^2) error analysis in Eqs. 12-15 is therefore not connected to the algorithm. If an implementation instead stores the nearest future event, the pseudocode is wrong and the reported numbers are not reproducible from the manuscript; if it follows the pseudocode, the future channel encodes the farthest event, weakening temporal locality and the active-pixel mask (|L| < 0.3) will often exclude the very pixels it is meant to capture. Either way, the central theoretical claim about bidirectional search is unsupported, although the empirical 49% TEPE reduction might still survive through other mechanisms such as increased information density.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Labits, a layered bidirectional time-surface representation for event cameras, together with a Labits-to-APLOF module that estimates active-pixel local optical flow. The authors claim that this representation retains fine-grained temporal information, stable 2D features, and consistent information density, and that, integrated into a RAFT-style pipeline, it reduces trajectory end-point error (TEPE) on the MultiFlow dataset by 49% over DCT-RAFT, from 1.29 to 0.66. The main empirical evidence consists of comparisons against frame- and event-based RAFT variants, ablations that swap voxel grids for Labits while keeping the same 6.8M-parameter pipeline, one-way-versus-bidirectional Labits ablations, and a bin-size sensitivity study.","tokens_in":17738,"tokens_out":4660,"duration_ms":50183,"significance":"If the empirical claims hold, the paper would make a useful contribution by showing that event representation choice is a first-order factor in dense continuous-time trajectory estimation. The controlled ablation in Table 2 (replacing voxel grids with Labits in the same 6.8M-parameter DCT-RAFT pipeline reduces TEPE from 1.29 to 1.01) is a strong and creditworthy experimental design. The supplementary comparison against other layered representations and the bin-size study further support the practical value of the representation. However, the paper's central theoretical motivation, the claimed O(dt^2) central-difference advantage, is not realized by the algorithm as written, and the headline result is reported without statistical uncertainty or a matched-parameter comparison. These issues do not necessarily invalidate the empirical finding, but they require substantial revision before the claims can be accepted as stated.","major_comments":[{"comment":"The central theoretical claim that Labits enables central-difference local speed estimation with error O(dt^2) is not supported by the algorithm as written. In Algorithm 1, lines 5-14, Tprev is overwritten by each past event in ascending order and therefore ends as the most recent past event; Tfuture is likewise overwritten and ends as the latest (farthest) future event in the bin; and line 14 selects Tprev whenever it is non-infinite. Consequently, every pixel stores at most one timestamp, never both f(x_n - dt) and f(x_n + dt), so the central-difference derivation in Eqs. 12-15 does not apply. Moreover, the future event stored is the farthest event in the future bin, not the 'next future event' described in the Figure 1 caption. The authors should either correct the pseudocode and the description so that the representation actually combines near-past and near-future events, or revise the theoretical motivation to match the implemented past-preferred, future-fallback behavior.","section":"Section 3 / Algorithm 1 / Supplementary A.2"},{"comment":"The headline 49% reduction compares the full 25.3M-parameter Labits-RAFT against the 6.8M-parameter DCT-RAFT, while the same-size ablation (Labits-RAFT without APLOF features, 6.8M parameters) gives TEPE 1.01, a 22% reduction. The 49% figure therefore conflates the representation change with a large increase in model capacity. The paper should report a matched-parameter comparison as the primary representation-only effect and clearly state the parameter counts alongside each result. In addition, no error bars or multiple-seed results are reported for any of the metrics; given that several ablation gaps are small (e.g., 0.72 vs 0.66 in Table 5), the significance of the improvements needs to be established.","section":"Table 2 / Supplementary Table 3"},{"comment":"The APLOF ground-truth definition, A_tau(x_tau) = (O_{tau+}(x_start) - O_{tau-}(x_start)) * M_h(x), requires clarification. The cumulative flow O_tau is defined at the reference-time pixel x_start, whereas the label is assigned to x_tau = x_start + O_tau(x). If x_tau is not an integer pixel coordinate, the paper does not specify how the label is resampled or interpolated. Additionally, the target is a difference of cumulative flows over a 20 ms interval centered at tau, not an instantaneous velocity; the validity of this target as 'local optical flow' and the choice of the 10 ms offset are not justified or ablated. The authors should provide the exact label-generation procedure and an ablation or discussion of the offset size.","section":"Section 4.1, Eq. (6)"}],"minor_comments":[{"comment":"The Introduction states that 'simply switching to Labits' yields a 13% improvement on TEPE, but Table 2 shows a 22% improvement for the matched-parameter comparison (1.29 to 1.01) and 17% for the event-only case (1.85 to 1.53). The percentage should be corrected for consistency.","section":"Introduction"},{"comment":"The caption says 'it searches for the next future event within dt,' but Algorithm 1 stores the latest future event within the bin when no past event exists. The caption and pseudocode should describe the same behavior.","section":"Figure 1 caption"},{"comment":"The table header says 'DCF-RAFT' while the rest of the paper uses 'DCT-RAFT'; please correct the typo.","section":"Supplementary Table 6"},{"comment":"There are several typographical and formatting issues, including 'utlizes' in the contributions list, 'V oxel' spacing in the text and tables, and 'groud truth' in Figure 9. A careful proofread is needed.","section":"Various"},{"comment":"The statement that Labits is 'the first to achieve this combination' is an overclaim; the paper does not provide a systematic survey to establish priority. Rephrase as a property of the proposed representation rather than a novelty claim.","section":"Section 6"}],"recommendation":"major_revision","confidential_remarks":"The empirical core of the paper is likely salvageable: the controlled 6.8M-parameter ablation and the supplementary comparisons against other layered representations give credible evidence that Labits helps in this task. The main obstacle is the disconnect between the stated central-difference theory and the implemented algorithm, which undermines the conceptual contribution and reproducibility. I would also encourage the editor to require that the code be released or at least that the exact preprocessing be specified, because the Algorithm 1 details are central to the paper's claims and the current pseudocode is ambiguous about overwriting behavior. The lack of error bars and the unequal-parameter headline comparison are standard but important concerns for a benchmark claim of this magnitude."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou asked about Labits. The short version: it's a real empirical result from a new event representation, and the equal-size ablations are the strongest part. But the paper's theoretical motivation doesn't match its own algorithm, and the headline 49% is partly a model-size effect.\n\nWhat's actually new: Labits is a layered time-surface where each pixel stores either the most recent past event or, if none, the nearest future event at each probe time. That past-first/future-fallback design is a genuine twist; it fills empty pixels near moving edges and gives the network more information than a one-way time surface. The paper backs this with careful ablations: on the same 6.8M-parameter RAFT pipeline, replacing the voxel grid with Labits cuts TEPE from 1.29 to 1.01 (22%). The one-way vs bidirectional ablation (0.72 vs 0.66) shows the future-fallback itself helps, even if modestly. The APLOF module—a U-Net that regresses local flow differences from single Labits layers—is a reasonable idea, and the supplementary shows it behaves well. The paper also has an honest limitations section: Labits isn't a universal representation, it drops polarity and event density, and it's only tested on MultiFlow.\n\nThe soft spots, in order of severity. First, the central-difference claim is not what the algorithm does. Algorithm 1 stores Tprev whenever it exists, so no pixel carries both a past and a future event at a given probe time; for future events it keeps the farthest event in the bin, not the nearest. Supplementary A.2's O(dt^2) error analysis is therefore disconnected from the actual representation. The empirical gain may come from increased information density rather than central differencing—fine, but the paper should say that, and either fix the pseudocode or drop the claim. Second, the headline 49% is computed against a 6.8M-parameter baseline with a 25.3M-parameter model. The paper discloses this in the supplementary, but the abstract and intro don't. The equal-size comparison (22%) is the honest headline. Third, there are no error bars, no multiple seeds, no real-world test, and the active-pixel thresholds are tuned post hoc. These are manageable, but they need to be in there.\n\nBottom line: the representation has legs and the ablation methodology is decent, but the paper needs a major revision to align theory and algorithm and to present the parameter-controlled result. It deserves a serious referee. I'd take a skeptical-but-interested reviewer.","headline":"A genuinely new event representation with solid equal-size ablations, but the central-difference theory doesn't match the algorithm and the 49% headline is partly a model-size effect.","tokens_in":18315,"tokens_out":5559,"would_cite":true,"duration_ms":54652,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Each pixel records the time offset to the nearest past or future event at each probe time; the paper claims this preserves timing, features, and density, and that an active-pixel local flow module cuts trajectory end-point error by 49%.","keywords":["event cameras","event representation","time surfaces","dense trajectory estimation","active pixel local optical flow","Bézier trajectories","continuous-time motion","event-based vision"],"falsifier":"Take a real event camera and a textured moving target whose true trajectory is measured independently (for example with high-speed motion capture), build Labits and APLOF from the recorded events, and compare the predicted instantaneous velocity with the measured velocity at each probe time; a systematic bias at edge pixels would indicate that the Eq. 6 supervision target is not a true local speed. Alternatively, evaluate the full pipeline on a real-event trajectory benchmark with dense ground truth; if the 49% TEPE advantage over DCT-RAFT shrinks substantially, the result is tied to MultiFlow's synthetic event statistics.","tokens_in":17155,"feed_emoji":"⚡","tokens_out":13158,"duration_ms":115092,"temperature":0.7,"pith_summary":"This paper argues that how events are packed into dense tensors is itself a first-order limit on continuous-time trajectory estimation from event cameras. It introduces Labits, a layered bidirectional time-surface representation in which every pixel at each probe time stores the normalized time offset to the most recent past event or, failing that, to the next future event. The paper claims that this single construction simultaneously preserves microsecond-level timing, stable 2D edges, and uniform information density, which earlier representations did not. Coupled with a small U-Net that predicts active-pixel local optical flow from a single Labits layer, the method reports a 49% reduction in trajectory end-point error (TEPE 1.29 to 0.66) over DCT-RAFT on the MultiFlow dataset.","feed_headline":"Layered bidirectional time surfaces cut trajectory error by 49%","feed_subtitle":"A two-sided event representation preserves microsecond timing that other encodings lose.","key_machinery":"The object that carries the argument is the Labits tensor $L \\in \\mathbb{R}^{B \\times H \\times W}$, built by Algorithm 1: for each probe time $\\tau_i$, every pixel holds either the normalized time offset $(t_n - \\tau_i)/\\tau_{\\text{range}}$ of the latest event in the past window $[\\tau_i - \\tau_{\\text{range}}, \\tau_i]$, or, if no past event exists, the offset of the earliest event in the future window $(\\tau_i, \\tau_i + \\tau_{\\text{range}}]$, or $-1$ when neither exists. Because the stored value is a time difference, local speed at active pixels is encoded as a spatial gradient of the Labits layer. The second carrying component is the Labits-to-APLOF Net, a U-Net with instance normalization that takes one Labits layer as input, masks inactive pixels with $|L| < \\beta$, and outputs high- and low-resolution active-pixel local optical flow (APLOF); these features enter the RAFT-style trajectory predictor both as correlation features and as the initial hidden state of its ConvGRU.","core_discovery":"The central claim is that a layered event representation reading time offsets in both directions—toward the latest past event and, where none exists, toward the next future event—can provide the three things dense trajectory estimation needs at once: fine-grained event timing, stable and sharp 2D features, and consistent information density across the whole interval. Prior time-surface variants keep only the most recent event per pixel and suffer temporal occlusion; voxel grids quantize time and blur it. Labits divides the interval into probe times and writes, at each probe, a normalized time difference to the nearest event within a backward/forward window of length $\\tau_{\\text{range}}$. The paper further claims that bidirectional search upgrades local speed estimation from backward difference (error $O(\\delta t)$) to central difference (error $O(\\delta t^2)$), and that the learned APLOF features inject this local speed into a RAFT-style Bézier trajectory predictor. On MultiFlow, the pure-event Labits-RAFT reduces TEPE by 29% (1.32 vs 1.85) and the events-plus-images version by 49% (0.66 vs 1.29), with comparable gains on TAE, EPE, and AE.","pith_inferences":["The same layered bidirectional construction could be transferred to event-based feature tracking or frame interpolation, where intermediate motion cues, not just endpoints, matter; the paper reports gains concentrated in intermediate states, so those tasks would be natural testbeds.","The APLOF supervision requires ground-truth optical flow at plus and minus ten milliseconds; a self-supervised version that estimates local speed directly from event timestamps would test whether the learned module is genuinely extracting the representation's encoded speed or merely memorizing dataset-specific flow statistics.","Because bin size can be varied without retraining, one could generate Labits at adaptive probe times tied to local event density, spending more layers in fast-motion regions; this would trade storage for temporal resolution and is a direct extension of the paper's flexibility argument."],"forward_implications":["Switching from voxel grids to Labits within the same RAFT pipeline reduces events-plus-images TEPE from 1.16 to 1.01 (about 13%), and adding APLOF features brings it from 1.01 to 0.66 (about 35%), for a total 49% reduction relative to DCT-RAFT.","The full method reduces TEPE from 1.29 to 0.66 and TAE from 3.35 to 1.72 on MultiFlow compared with DCT-RAFT, with similar or larger reductions on two-view EPE and AE.","The APLOF net trained on Labits reaches a much lower supervised loss than the same net trained on voxel grids (0.084 vs 0.217 total loss), indicating Labits layers carry local speed information that voxel grids do not.","The representation is bin-size agnostic: one pretrained APLOF net transfers across time-bin sizes from 0.0125 s to 0.1000 s, with roughly 50% TEPE reduction over voxel grids at every tested bin size.","Bidirectional sampling fills pixels ahead of moving edges that past-only representations leave empty, increasing information density without retraining."],"supporting_citations":[{"why":"Supplies the MultiFlow dataset, the DCT-RAFT baseline and previous state of the art, and the Bézier trajectory refinement framework that Labits-RAFT builds on.","marker":"Gehrig et al. (2024)"},{"why":"Provides the RAFT correlation-and-refinement architecture that the pipeline adapts for trajectory prediction.","marker":"Teed & Deng (2020)"},{"why":"E-RAFT is one of the event-based baselines compared in Table 2 and establishes the voxel-based event flow approach.","marker":"Gehrig et al. (2021b)"},{"why":"Time-surface representations are the direct antecedent; Labits is a layered, bidirectional extension of the time-surface idea.","marker":"Lagorce et al. (2016)"},{"why":"TORE volumes motivate the layered time-surface direction and are the key comparison for preserving multiple timestamps per pixel.","marker":"Baldwin et al. (2022)"},{"why":"Defines the voxel grid representation used by DCT-RAFT and in the ablations that replace Labits.","marker":"Zhu et al. (2019)"},{"why":"U-Net is the backbone of the Labits-to-APLOF net that produces local optical flow features.","marker":"Ronneberger et al. (2015)"}],"fun_headline_variants":["Bidirectional time surfaces slash trajectory error 49%","Labits: event representation cuts error by half","Event camera trick: bidirectional surfaces boost tracking","Two-way time surfaces improve dense trajectory estimation","Layered event encoding reduces endpoint error 49%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the APLOF training target—the difference of cumulative ground-truth flows at plus and minus ten milliseconds—is a valid instantaneous local speed, and that the synthetic MultiFlow dataset's noise, texture, and motion statistics represent real event-camera streams closely enough for the 49% advantage to carry over.","fun_headline_variants_meta":{"raw":{"variants":["Bidirectional time surfaces slash trajectory error 49%","Labits: event representation cuts error by half","Event camera trick: bidirectional surfaces boost tracking","Two-way time surfaces improve dense trajectory estimation","Layered event encoding reduces endpoint error 49%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00012,"raw_usage":{"total_tokens":1096,"prompt_tokens":959,"completion_tokens":137,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":66}},"tokens_in":575,"tokens_out":137,"duration_ms":2172,"temperature":1.0,"reasoning_tokens":66,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:29:18.926564+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a real event camera and a textured moving target whose true trajectory is measured independently (for example with high-speed motion capture), build Labits and APLOF from the recorded events, and compare the predicted instantaneous velocity with the measured velocity at each probe time; a systematic bias at edge pixels would indicate that the Eq. 6 supervision target is not a true local speed. Alternatively, evaluate the full pipeline on a real-event trajectory benchmark with dense ground truth; if the 49% TEPE advantage over DCT-RAFT shrinks substantially, the result is tied to MultiFlow's synthetic event statistics.","supporting_citations":[{"cited_title":"Raft: Recurrent all-pairs field transforms for optical flow","cited_arxiv_id":null,"evidence_quote":"Provides the RAFT correlation-and-refinement architecture that the pipeline adapts for trajectory prediction."}],"review_version":1}