{"id":"1f3ba97b-187f-421d-9a60-90cfb6fe5dbf","arxiv_id":"2412.00133","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ETAP introduces the first event-only tracking-any-point network, trained on a new EventKubric synthetic dataset with a motion-invariance feature-alignment loss, and reports state-of-the-art results on the EDS and EC feature-tracking benchmarks and on TAP-style benchmarks (EventKubric, EVIMO2, E2D2).","lead":"ETAP is the first event-camera-only method for tracking arbitrary points in a scene, using microsecond event timing to keep tracking through fast motion and difficult lighting. On standard feature-tracking benchmarks it reports a 20% gain over the best prior event-only method and a 4.1% gain over the best frame-plus-event method, and it ships a new synthetic dataset and new ground truth.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"E2D2 fidget-spinner ground truth is self-generated under a hand-set threshold and a piecewise-constant angular-velocity model, so the claimed high-speed/HDR advantage is not independently validated.","rationale":"I agree with the reader's identification: the E2D2 GT generation is the least externally anchored piece of evidence. The paper provides no independent verification of the spinner GT (no motion capture, no high-speed camera, no manual labels), and the acceleration visible in Figure 10b contradicts the piecewise-constant model. Because the E2D2 experiment is the paper's main proof that frame-based tracking fails while event-only tracking succeeds in the high-speed/HDR regime, this is load-bearing. The EDS/EC feature-tracking results and EVIMO2 results are independent and remain credible; they would support the method even if the E2D2 demonstration weakens. I also note that the abstract's 136% figure uses the ablation model without FA-loss (Table 2: 0.556 versus 0.236 is 136%, whereas the final model gives 0.546, a 131% relative gain), but this is a reporting inconsistency rather than a threat to the central claim. Single-run metrics without error bars further argue for conditional acceptance. Overall, the concern does not overturn the reader's CONDITIONAL verdict; it reinforces it, so no change is needed.","tokens_in":18754,"tokens_out":5671,"duration_ms":56468,"concrete_test":"Regenerate the E2D2 GT for the fidget-spinner sequence using (i) a non-constant angular-velocity model, e.g. fitting a quadratic or exponential theta(t) to the detected valley times, and (ii) a sweep of the valley-detection L2 threshold over a plausible range such as 200-400. Re-run CoTracker (frames), E2VID+CoTracker, and ETAP on the same query points with the new GT. If the AJ gap (0.308 versus 0.183) and the qualitative failure of frame-based tracking persist across all GT variants and thresholds, the E2D2 claim is robust; if the gap shrinks below significance or reverses, the paper's high-speed/HDR demonstration needs independent GT, e.g. from a high-speed camera or manual annotations, before it can support the headline claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is the E2D2 fidget-spinner evaluation (Supplementary Section 8.1, Table 3). Ground-truth tracks are produced from event histograms: valleys of the L2-norm time series are detected with a hand-set threshold of 300, each valley is assumed to mark a completed third revolution, and angular velocity is assumed constant between valleys. The paper states that the spinner accelerates throughout the clip, so the piecewise-constant model is wrong by construction; the resulting positional error depends on the acceleration profile and on where the threshold places the valley times. Because ETAP and E2VID+CoTracker are both scored against this same synthetic GT, the reported 0.308 versus 0.183 AJ gap, and the qualitative claim that frame-based tracking fails while ETAP succeeds, is not independently validated. This matters because E2D2 is the paper's only real-world demonstration of the high-speed/HDR regime that motivates event-only TAP; if a corrected GT shrinks the gap, the central claim that events are a viable sole modality loses its most direct evidence, even though the EDS/EC and EVIMO2 results remain credible.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces ETAP, an event-only tracking-any-point (TAP) model. The method processes event stacks with a CoTracker-style transformer and is trained on a new synthetic EventKubric dataset produced by combining Kubric rendering with Vid2e/ESIM event generation. A time-inversion contrastive feature-alignment loss is proposed to make descriptors robust to motion direction. Experiments cover TAP on EventKubric, EVIMO2, and E2D2, and feature tracking on EDS and EC, reporting improvements over event-only and frame-event fusion baselines. Code and dataset are released.","tokens_in":18888,"tokens_out":6783,"duration_ms":58234,"significance":"If the reported results hold, the paper makes a strong contribution: it is the first event-only TAP approach, with released code and dataset, a systematic ablation study, and a dedicated experiment showing that the feature-alignment loss improves motion-invariance of descriptors. Cross-dataset generalization to different sensors and resolutions is demonstrated. The main caveat is the self-generated E2D2 ground truth, which directly supports the high-speed/HDR claim; this needs additional validation.","major_comments":[{"comment":"The E2D2 fidget-spinner ground truth is generated from event histograms using a hand-set L2-norm valley threshold of 300 and a piecewise-constant angular-velocity model between detected third-revolution valleys, yet the text explicitly states that the spinner accelerates throughout the clip. Because both ETAP and the baselines are scored against this synthetic GT, the reported AJ gap (0.308 vs 0.183) is not independently validated, and the qualitative claim that frame-based tracking fails while ETAP succeeds rests on this assumption. Please provide a sensitivity analysis over the threshold and an alternative acceleration-aware velocity model, or an independent manual annotation check, and temper the claims if the quantitative advantage is not robust.","section":"Supplementary §8.1, Table 3"},{"comment":"The abstract's '136% better' figure corresponds to ETAP without the FA-loss (0.556 vs 0.236 on EventKubric, a 135.6% improvement) rather than the final ETAP model (0.546 vs 0.236, a 131.4% improvement). Since the final model is the one presented as the method, please correct the headline number or explicitly state that the improvement is computed for the ablation without the feature-alignment loss.","section":"Abstract; §5.2, Table 2"},{"comment":"The 'E2Vid [52] + CoTracker [29]' baseline is an events-to-video pipeline rather than a pure event-only method; the text calls it an event-based baseline and the distinction matters because events are first converted to frames. Please clarify this in the main text and in the table caption so that readers can correctly interpret the comparison.","section":"§5.2, Table 2"}],"minor_comments":[{"comment":"The 'Varying dynamics' and 'static' configurations are not defined; please state whether 'static' means no camera motion, no object motion, or both, and explain how panning is incorporated.","section":"Table 5"},{"comment":"The notation 'w\\o' should be 'w/o' for readability.","section":"Tables 2-4"},{"comment":"Figure 10(b) truncates the L2-norm axis at 400, making the detected valleys difficult to inspect; a zoomed view around the threshold region would help the reader judge the reliability of the valley detection.","section":"Figure 10"},{"comment":"The time-inversion derivation relies on the approximation δτ~ ≈ δτ; please state this approximation explicitly when the feature-alignment loss is motivated in the main text.","section":"Supplementary §7"},{"comment":"The statement that ETAP surpasses the event-based baseline by '68% AJ' should specify that this is a relative improvement (0.308 vs 0.183), to avoid confusion with an absolute difference of 0.125.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"The E2D2 ground-truth issue is the primary obstacle. If the authors provide a sensitivity analysis and correct the abstract's 136% figure, I would support acceptance; the remaining issues are presentation-level. The paper is otherwise strong and well within the scope of the journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: ETAP is the first event-only TAP model that actually works across real datasets, and the claims mostly hold up. The core contribution is real: a tracking-any-point architecture that takes event stacks, a synthetic training pipeline (EventKubric) that beats MOVi-F by a meaningful margin, and a feature-alignment loss that explicitly tackles motion-dependent event features. The EDS/EC numbers are the strongest part: 0.704 feature age on EDS vs 0.589 for DDFT-E2VID and 0.676 for FE-TAP, so it wins clearly over prior event-only and even event+frame methods. Cross-dataset generalization to EVIMO2 and EC at different resolutions is credible, and the dedicated descriptor-similarity experiment (Table 7) gives direct evidence that the FA-loss does what it claims. I believe the paper.\n\nNow soft spots, in proportion.\n\nThe E2D2 fidget-spinner evaluation is the shakiest piece. Ground truth is self-generated from event-histogram valleys with a hand-set L2 threshold of 300 and a piecewise-constant angular-velocity assumption, even though the spinner accelerates throughout the clip. That is not independently validated, and the reported 0.308 AJ vs 0.183 for E2Vid+CoTracker could shift under a different threshold or a non-constant velocity model. Label that as a limitation and get a second GT source (e.g. a high-speed camera) before leaning on it as the high-speed/HDR proof.\n\nAlso: all metrics are single-run with no error bars or significance info; the abstract's 136% Jaccard figure matches the ablation without FA-loss (0.556 vs 0.236), not the final model (which is 131% and slightly worse in-domain); and Task 1 comparisons only pit ETAP against E2Vid+CoTracker, omitting the stronger event-based baselines (FE-TAP, DDFT) that appear in Task 2. These are addressable, not fatal.\n\nWho should read it: anyone working on event-based tracking, TAP, or high-speed robotic perception. The dataset and code release look useful even apart from the model. As a referee, I'd engage with it, ask for the E2D2 GT to be corroborated and for error bars, and accept after those revisions.\n\nRecommendation: serious referee, yes.","headline":"First event-only TAP that genuinely works across real datasets; the E2D2 ground truth is the one place to press, but the core claims hold.","tokens_in":19541,"tokens_out":2068,"would_cite":true,"duration_ms":17834,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An event-only tracker, trained on synthetic events, outperforms prior event-only and events-plus-frames methods and tracks points where frame-based trackers fail.","keywords":["event cameras","tracking any point","point tracking","event-based vision","synthetic event data","contrastive feature alignment","motion robustness","feature tracking"],"falsifier":"Rerun the E2D2 fidget-spinner experiment with independent ground truth, such as a high-speed camera, motion capture, or a checkerboard-encoded marker, and with a non-piecewise-constant velocity model; if ETAP's average-Jaccard margin over frame-based and event-to-frame baselines collapses or reverses when the assumed constant-velocity ground truth is replaced, the central claim fails. A second, easier check is to vary the valley-detection L2-norm threshold (300), fit a smooth angular-velocity model, and recompute the reported 0.308 average Jaccard.","tokens_in":18473,"feed_emoji":"🎯","tokens_out":7652,"duration_ms":62118,"temperature":0.7,"pith_summary":"The paper sets out to show that event cameras, which output asynchronous brightness-change spikes, can serve as the sole input for tracking arbitrary points in a video, a task previously dominated by frame-based tracking-any-point methods. It introduces ETAP, an event-only TAP model built by adapting a transformer point-tracker to event-stack representations, and trains it on EventKubric, a new synthetic dataset rendered at high frame rate and converted to events. The central result is that this event-only tracker outperforms the previous best event-only feature tracker by about 20% (feature age 0.704 vs 0.589), surpasses the best events-plus-frames TAP method by 4.1%, and improves average Jaccard by 136% over an event-to-frame baseline on the TAP benchmark. If correct, this matters because event cameras handle high-speed motion, motion blur, and high dynamic range where frame-based trackers fail, making reliable point tracks available in exactly the regimes that break conventional video. The paper also claims a new contrastive feature-alignment loss that makes event features invariant to motion direction, plus new ground truth for two real datasets.","feed_headline":"Event-only tracker beats frame-based trackers on fast motion","feed_subtitle":"First event-only TAP model beats prior event and event-plus-frame trackers, trained only on synthetic events.","key_machinery":"The load-bearing object is the event stack: each tracking timestep is represented by a 10-channel histogram of the most recent $N_e$ events, binned at hierarchical time resolutions, so asynchronous events become grid tensors a convolutional encoder and transformer can consume. Around this, the tracker uses the same iterative token-refinement mechanism as modern frame-based point trackers: tokens carry position offsets, per-point descriptors, spatial correlation features (inner products of a descriptor against a 7×7 neighborhood at four scales), and visibility flags, updated by alternating intra-point and temporal attention. The specifically new mechanism is the feature-alignment loss $\\mathcal{L}_{fa}$: for each training sample a time-inverted and randomly rotated duplicate is generated, and descriptors sampled from real and inverted tracks are matched after reversing time, rewarding cosine similarity so that learned features depend on scene structure rather than on the direction of motion. The third piece is the training data: EventKubric, generated by rendering 2-second physically based videos at 48 fps, adaptively upsampling so inter-frame motion is at most one pixel, and converting to events with a simulator under randomized contrast thresholds.","core_discovery":"On its own terms, the paper's discovery is that an event-only neural tracker can track arbitrary points, not just corners, robustly enough to beat both prior event-only methods and methods that combine events with frames. The model takes query points, builds multi-channel event stacks from the latest fixed number of events, extracts multi-scale feature maps, and refines point positions and appearance descriptors through a transformer with spatial correlations and visibility prediction, in the style of modern frame-based TAP trackers. Trained purely on synthetic EventKubric clips, it reports feature age 0.704 on EDS versus 0.589 for the best prior event-only method and 0.676 for the best events-plus-frames method; on the TAP benchmark it reaches average Jaccard 0.546 on EventKubric and 0.661 on EVIMO2, versus 0.236 and 0.531 for an event-to-frame reconstruction baseline. The second claimed discovery is that the motion-dependence of event features can be tamed by a feature-alignment loss: each training sample is duplicated under time inversion plus random rotation, and descriptors at corresponding track points are pulled together in cosine similarity, shrinking the inter-motion similarity gap from 0.377 to 0.067 in their measurement.","pith_inferences":["The feature-alignment idea is not limited to point tracks: the same time-inversion and rotation invariance principle could be applied to event representations for optical flow, segmentation, or recognition, wherever descriptor drift from motion direction is a problem.","Because the paper reports that query times during absence of motion produce poor descriptors, an event-only system in practice may need a motion-gating or reinitialization heuristic; a testable extension is to couple ETAP with an inertial or frame-based wake-up to handle static starts.","If event-only TAP sustains its margins on more diverse real data, downstream systems like visual odometry or robot manipulation could drop the frame stream entirely at high speed, saving bandwidth and power; this is a consequence the paper does not develop.","A direct stress test would be evaluating on slow, low-texture scenes where event data is sparse; the paper's limitation note suggests accuracy may degrade, and quantifying that would set the boundary of the method."],"forward_implications":["Event-only tracking can be deployed in high-speed and high-dynamic-range settings where frame-based trackers fail, including low-light recordings with severe motion blur.","The 20% feature-age gain over the best prior event-only method and 4.1% over the best events-plus-frames method suggest events alone carry enough information for long-range point correspondence.","The feature-alignment loss makes learned event features stable under opposite and perpendicular motion directions, closing most of the similarity gap between same-motion and different-motion descriptors.","The EventKubric pipeline, with high render frame rate and randomized contrast thresholds, transfers to real sensors of different resolutions and camera types without fine-tuning.","New ground truth tracks provided for EVIMO2 and E2D2 allow future event-based TAP methods to be compared directly on real high-speed data."],"supporting_citations":[{"why":"Defines the TAP task, the average-Jaccard and occlusion-accuracy metrics, and the point-track generation conventions ETAP adopts.","marker":"[13]"},{"why":"Supplies the physics-based rendering tool with which EventKubric training videos are generated.","marker":"[23]"},{"why":"Provides the video-to-events conversion workflow that turns rendered high-frame-rate video into event data.","marker":"[20]"},{"why":"Provides the event simulator used to synthesize events from the upsampled rendered frames.","marker":"[51]"},{"why":"Supplies the transformer tracker architecture and token-refinement loop that ETAP adapts to event stacks.","marker":"[29]"},{"why":"Provides the frame-reconstruction baseline used to turn events into images for a frame-based tracker comparison.","marker":"[52]"},{"why":"Defines the feature-age benchmark protocol and the prior data-driven event feature tracker that ETAP outperforms.","marker":"[44]"},{"why":"Defines the prior frame-event fusion TAP method that ETAP surpasses on feature tracking.","marker":"[38]"},{"why":"Supplies the E2D2 dataset used for the high-speed fidget-spinner experiment, for which the paper generates new ground truth.","marker":"[60]"},{"why":"Supplies the EDS dataset used as a main feature-tracking benchmark.","marker":"[26]"}],"fun_headline_variants":["Event-only TAP tracker beats event+frame methods by 4.1%","First event-only arbitrary-point tracker outperforms hybrid trackers","Feature-alignment loss tames motion-dependent event features","Event camera TAP: high-speed tracking without frames","Event-only point tracker supersedes frame-assisted rivals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's advantage over frame-based trackers in its headline high-speed demonstration is measured against self-generated ground truth for the E2D2 fidget spinner that assumes constant angular velocity between valleys detected by a hand-set threshold, even though the spinner accelerates; if that assumption is wrong, the quantified advantage changes.","fun_headline_variants_meta":{"raw":{"variants":["Event-only TAP tracker beats event+frame methods by 4.1%","First event-only arbitrary-point tracker outperforms hybrid trackers","Feature-alignment loss tames motion-dependent event features","Event camera TAP: high-speed tracking without frames","Event-only point tracker supersedes frame-assisted rivals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000286,"raw_usage":{"total_tokens":1730,"prompt_tokens":1040,"completion_tokens":690,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":608}},"tokens_in":656,"tokens_out":690,"duration_ms":21468,"temperature":1.0,"reasoning_tokens":608,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:26:42.214571+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the E2D2 fidget-spinner experiment with independent ground truth, such as a high-speed camera, motion capture, or a checkerboard-encoded marker, and with a non-piecewise-constant velocity model; if ETAP's average-Jaccard margin over frame-based and event-to-frame baselines collapses or reverses when the assumed constant-velocity ground truth is replaced, the central claim fails. A second, easier check is to vary the valley-detection L2-norm threshold (300), fit a smooth angular-velocity model, and recompute the reported 0.308 average Jaccard.","supporting_citations":[{"cited_title":"TAP-Vid: A benchmark for track- ing any point in a video","cited_arxiv_id":null,"evidence_quote":"Defines the TAP task, the average-Jaccard and occlusion-accuracy metrics, and the point-track generation conventions ETAP adopts."},{"cited_title":"Kubric: A scalable dataset generator","cited_arxiv_id":null,"evidence_quote":"Supplies the physics-based rendering tool with which EventKubric training videos are generated."},{"cited_title":"Video to Events: Recycling video datasets for event cameras","cited_arxiv_id":null,"evidence_quote":"Provides the video-to-events conversion workflow that turns rendered high-frame-rate video into event data."},{"cited_title":"ESIM: an open event camera simulator","cited_arxiv_id":null,"evidence_quote":"Provides the event simulator used to synthesize events from the upsampled rendered frames."},{"cited_title":"Co- Tracker: It is better to track together","cited_arxiv_id":null,"evidence_quote":"Supplies the transformer tracker architecture and token-refinement loop that ETAP adapts to event stacks."},{"cited_title":"Events-to-video: Bringing modern computer vision to event cameras","cited_arxiv_id":null,"evidence_quote":"Provides the frame-reconstruction baseline used to turn events into images for a frame-based tracker comparison."},{"cited_title":"Data-driven feature tracking for event cameras","cited_arxiv_id":null,"evidence_quote":"Defines the feature-age benchmark protocol and the prior data-driven event feature tracker that ETAP outperforms."},{"cited_title":"Event-aided direct sparse odometry","cited_arxiv_id":null,"evidence_quote":"Supplies the EDS dataset used as a main feature-tracking benchmark."}],"review_version":1}