{"id":"ad06ab59-1f92-450d-b369-e4278a757d2a","arxiv_id":"2412.01300","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MATE tracks any point from event cameras alone, using motion vectors extracted from time surfaces to guide matching, and reports higher accuracy and survival than video- and event-based baselines on four benchmarks.","lead":"This paper presents MATE, an event-camera system for tracking any point on a moving object that combines motion cues from event timestamps with appearance matching. It reports better accuracy and longer tracking than video-based trackers and prior event-based methods on four benchmarks, with fewer parameters.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sec. 4.1's GT-frequency-dependent time-surface window leaks oracle information into MATE's input; without a GT-free evaluation, the reported Survival50/δavg gains are not yet evidence for event-only tracking.","rationale":"The reader's weakest assumption is exactly the one I consider most load-bearing, so I agree. The central claim requires that MATE track points from event data alone under realistic conditions. Setting the temporal window using ground-truth event frequency violates that condition because the input time-surface values depend on the window. No experiment in the paper evaluates a GT-free protocol, and the supplementary ablation on input representation does not vary window length. The architecture itself is plausible: the MGM/VMA ablation in Table 3 shows consistent monotonic improvements, and the parameter count is genuinely lower (6.6M). However, the evaluation protocol and the missing ETAP comparison in Table 1 prevent accepting the 'outperforms all existing methods' claim as stated. The independent reproducibility problems, including no released code, no error bars, and the Table 2 detail where ETAP exceeds MATE on EC EFA (0.876 vs 0.875), reinforce CONDITIONAL rather than ACCEPT. Since my proposed fix matches the reader's requested condition, the verdict stays CONDITIONAL (UNCHANGED).","tokens_in":13591,"tokens_out":5029,"duration_ms":46219,"concrete_test":"Re-run MATE on Ev-PointOdyssey and Ev-Davis with the time-surface window length chosen by a fixed rule (e.g., constant 20 ms, or adaptive from the online event rate) instead of the ground-truth frequency, keeping the trained weights and all other evaluation settings identical. If Survival50 or δavg drops by more than a few percent relative, the headline gains are partly attributable to oracle windowing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Sec. 4.1 the authors write: 'The temporal window length is set according to the ground truth frequency.' The Time Surface is built by normalizing timestamps within this window, so the window length changes the actual values of the input representation. Setting it from ground-truth event frequency means every metric in Table 1 and Table 3 is produced with knowledge of a quantity that is unavailable when the method is deployed. This is not a benign hyperparameter choice: it directly shapes the motion cues consumed by the Motion-Guidance Module and the correlation features in the Variable Motion Aware Module. The paper gives no ablation with a fixed window, an event-rate-adaptive window, or any GT-free rule, so it cannot be determined whether the claimed 17.9% Survival50 improvement survives realistic use. A further unaddressed comparison is the omission of ETAP from the TAP benchmark (Table 1), which weakens the 'outperforms all existing methods' claim, but the oracle windowing is the more fundamental threat because it affects every reported number.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents MATE, an event-camera framework for tracking any point. Events are encoded as time surfaces, a motion-guidance module estimates kinematic vectors by local plane fitting, and a variable-motion-aware module builds temporally consistent correlation features; a transformer iteratively updates point positions. The authors simulate event versions of PointOdyssey and TAP-Vid-Davis, evaluate tracking metrics on those datasets and feature tracking metrics on the EC and EDS datasets, and report state-of-the-art results with 6.6M parameters. The main claims are a 17.9% Survival50 improvement over event-only TAP baselines and superiority over all existing trackers, including methods that combine events and video frames.","tokens_in":13798,"tokens_out":4516,"duration_ms":39364,"significance":"If the results hold, MATE would be a strong demonstration that event-only point tracking can compete with video-based TAP by using kinematic cues to resolve appearance ambiguity. The paper contributes a useful architectural decomposition: plane-fitting kinematic vectors, MLP-based correction, cross-attention temporal modeling, and component ablations. The simulated Ev-PointOdyssey and Ev-Davis datasets are a practical resource for a young subfield. The work is not merely incremental: it identifies a real failure mode of appearance-only event matching. However, the evaluation has gaps that must be closed before the claims can be accepted, most importantly an oracle-dependent time-surface window and missing comparisons against the only published event-based TAP method.","major_comments":[{"comment":"The sentence \"The temporal window length is set according to the ground truth frequency\" describes an oracle setting. Because the time surface normalizes timestamps within this window, the window length changes the actual values of the input representation consumed by the Motion-Guidance Module and the Variable Motion Aware Module; hence every result in Tables 1 and 3 is produced with a quantity that is unavailable when the method is deployed. No ablation with a fixed window, an event-rate-adaptive window, or any other ground-truth-free rule is reported. Please add such an experiment and state explicitly how the window length is chosen in practice.","section":"Sec. 4.1"},{"comment":"ETAP [14] is cited in Sec. 2.3 as an event-based TAP method but is absent from the TAP benchmark in Table 1. Without this comparison, the abstract's claims of improvement \"over event-only tracking of any point baseline\" and of outperforming \"all existing methods\" are unsupported. Add ETAP results on Ev-PointOdyssey and Ev-Davis, or restrict the claims to the methods actually compared.","section":"Table 1"},{"comment":"The statement that the method \"outperforms all three categories of existing trackers\" is contradicted by the table itself: on the EC dataset, ETAP reports EFA 0.876 while MATE reports 0.875, and several other margins are small enough to be within noise. Report variance or significance tests, and soften the \"outperforms all existing methods\" wording in the abstract and Sec. 4.3.","section":"Table 2 and Sec. 4.3"},{"comment":"The claimed 17.9% Survival50 improvement is relative to Ev-PIPs++ and only holds on Ev-PointOdyssey (0.560 vs. 0.475); on Ev-Davis the corresponding Survival16 gain is 0.928 vs. 0.874, which is a much smaller relative improvement. The abstract should state the comparison method and dataset explicitly rather than presenting the percentage as a universal result.","section":"Abstract"}],"minor_comments":[{"comment":"There are several typos and spacing issues, including \"Our apporach\" in Fig. 1 and \"MA TE\" instead of \"MATE\" in the title and body; a copyediting pass is needed.","section":"Throughout"},{"comment":"The additional ablations in the supplementary material are run on a baseline model without VMA and at a lower resolution (192x256); the main text should note these differences so readers do not compare them directly with Table 1.","section":"Supplementary Sec. 8"},{"comment":"For FE-TAP, the results are taken from the original paper rather than re-run under the same protocol; please state explicitly which settings differ and whether the comparison is apples-to-apples.","section":"Sec. 4.3"},{"comment":"The relation ∂Σ_e/∂x = (∂x/∂t)^{-1} = 1/v is dimensionally inconsistent unless v is understood as pixels per time unit; a short clarifying note would prevent confusion.","section":"Eq. (2)"}],"recommendation":"major_revision","confidential_remarks":"The oracle-window issue is the one I would want fixed before publication; it affects every reported number and is not addressed by any ablation. The missing ETAP comparison in Table 1 is also important for the paper's headline claims. The central idea is sound and the architecture is clearly described, so I see this as a major-revision situation rather than a rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is worth taking seriously: use kinematic vectors estimated from time-surface plane fitting to guide local matching in event-based point tracking. That is a natural fit for the event modality, and it is genuinely new for TAP even though the components (Benosman-style gradients, SVD plane fitting, PIPs-style iterative transformer matching) are known. The two simulated datasets are useful contributions in a field that lacks public event TAP benchmarks, and the ablations in Table 3 show each module earns its keep, with convincing jump in Survival_50 from adding MGM and VMA.\n\nThe soft spot is not small. Section 4.1 says the temporal window length is set according to the ground truth frequency. That quantity is unavailable at deployment, and it directly shapes the time-surface input consumed by both modules. This is not a harmless hyperparameter: changing the window changes the actual pixel values in the representation, so every number in Tables 1 and 3 is produced with oracle knowledge. The paper does not test a fixed window, an event-rate-adaptive window, or any GT-free rule. Until that ablation is done, the claimed 17.9% Survival_50 gain over the self-created baseline cannot be read as evidence for practical event-only tracking. The omission of ETAP from the TAP benchmark (Table 1) also weakens the 'outperforms all existing methods' claim, since ETAP is the only other published event-based TAP method and the feature-tracking table shows it is close. Also no code, data, or error bars, which makes the numbers hard to verify.\n\nThe math behind the event-rate versus speed relation is textbook and is used only as motivation, so no circularity there. The architecture is plausible, the writing is clear, and the parameter efficiency claim checks out. This is a solid technical idea with a visible evaluation gap. It deserves a serious referee, but the referee should ask for a GT-free window evaluation, an ETAP comparison on the TAP benchmark, and release of code/data before any acceptance.","headline":"The motion-guidance idea is sensible and the ablations back it, but the ground-truth-frequency-dependent time-surface window leaks oracle info into every reported metric, so the headline numbers are not yet trustworthy.","tokens_in":14330,"tokens_out":1165,"would_cite":false,"duration_ms":13022,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MATE reports that an event stream alone can track arbitrary points more accurately and over longer horizons than video-based and fusion baselines.","keywords":["event cameras","tracking any point","event-based point tracking","time surface","motion guidance","temporal consistency","feature tracking","simulated event datasets"],"falsifier":"Re-run the Ev-PointOdyssey and Ev-Davis evaluations with the time-surface window length selected without ground-truth frequency (for example, fixed, or adapted from observed event rates), and check whether $Survival_{50}$ and median trajectory error still beat Ev-PIPs++ and the video baselines in Table 1; if the gap shrinks or reverses, the central practical claim fails.","tokens_in":13395,"feed_emoji":"🎯","tokens_out":8783,"duration_ms":69401,"temperature":0.7,"pith_summary":"Trajectory tracking methods that work from ordinary video assume motion is smooth between frames, so they lose points during fast or nonlinear motion. This paper claims that event cameras, which emit sparse brightness-change spikes at microsecond resolution, contain enough continuous motion information to track arbitrary surface points more accurately and for longer than video-based methods. The proposed framework, MATE, adds two modules: a motion-guidance module that reads local kinematic vectors from the event time surface, and a variable-motion aware module that keeps appearance matching stable when object speed changes. On simulated event versions of two tracking-any-point datasets, MATE reports a 17.9% relative improvement in $Survival_{50}$ over the event-only baseline, and on standard feature-tracking benchmarks it reports the best results in its comparisons, including against methods that use both events and video. If these results hold, event-only tracking could replace frame-based pipelines in fast-motion and low-power settings.","feed_headline":"Event-only tracker beats video methods at point tracking","feed_subtitle":"MATE adds motion vectors from event time surfaces and a speed-aware module to cut tracking error across four benchmarks.","key_machinery":"The load-bearing objects are the time surface and two modules. The time surface is a per-pixel map that stores the timestamp of the most recent event, encoding motion history instead of intensity. The Motion-Guidance Module treats that map as a surface in $xyt$ space, fits a tangent plane to neighboring pixels with SVD, and uses its spatial gradients as kinematic vectors that estimate local motion tendency; an MLP reweights these vectors across time to correct ambiguities at object boundaries. The Variable Motion Aware Module takes local feature patches at times $t$, $t-2$, and $t-4$, aligns them under the kinematic vectors, fuses them by cross-attention, and passes them through a short-term temporal convolution plus a long-term temporal attention branch to produce correlation maps $C^{k}$. A transformer then takes the point positions $X^{k}$, kinematic vectors $V^{k}$, and correlation maps $C^{k}$ and iteratively predicts the displacement $\\Delta X$ that updates the trajectory.","core_discovery":"MATE's central claim is that an event stream alone can track arbitrary physical surface points more accurately and over longer horizons than video-based trackers, because events do not have a blind time between frames. The paper constructs simulated event versions of the PointOdyssey and TAP-Vid-Davis datasets, trains MATE on Ev-PointOdyssey, and reports the best numbers in its comparisons on four test sets: on Ev-PointOdyssey it reaches a $Survival_{50}$ of 0.560 against 0.475 for Ev-PIPs++ (a 17.9% relative gain) and a median trajectory error of 25.63 pixels, below the video-only PIPs++ at 26.95; on Ev-Davis it reaches 0.928 $Survival_{16}$ and an MTE of 3.47. On the EC and EDS feature-tracking benchmarks, it reports higher Feature Age than baselines that include event-plus-video methods such as FE-TAP and ETAP. The paper interprets this as evidence that motion guidance and temporal consistency, not appearance alone, should carry the matching signal for long-horizon point correspondence.","pith_inferences":["Editorial inference: the biggest unmeasured risk is outside-lab deployment; if the time-surface window cannot use ground-truth frequency, the reported 17.9% survival gain may shrink, so a natural next test is self-calibrated or fixed-window evaluation.","Editorial inference: the kinematic vectors from the time-surface plane fit are a general motion prior, so they could be reused by event-based optical flow or visual odometry systems, though the paper does not test those extensions.","Editorial inference: because both new TAP datasets are simulated, real-event sequences with noise, contrast-threshold variation, and sensor mismatch would be the sharper test of whether kinematic guidance stays stable; the paper does not provide that evidence.","Editorial inference: the reported comparison against FE-TAP and ETAP relies on numbers taken from their papers rather than re-runs, so a shared-code re-evaluation on the same splits would make the ranking more certain."],"forward_implications":["Event-only point tracking can in principle replace video input for tracking-any-point under fast motion, since events sample motion continuously instead of interpolating it across blind times.","Fusing events with video frames may not be required to reach top accuracy on the tested feature-tracking benchmarks; the event-only model reports higher Feature Age than the frame-event baselines.","The framework's small parameter count (6.6M) suggests event-based trackers can be deployed on lower-power platforms such as drones and embedded agents.","Long-horizon tracking improves when later time steps are weighted more heavily in the loss, so the training objective can directly shape survival time.","The time-surface plane-fit provides a cheap per-point velocity estimate that can support corridor searches during matching, reducing the appearance ambiguity that limits event-based trackers."],"supporting_citations":[{"why":"Supplies the PointOdyssey synthetic dataset used for training and for building Ev-PointOdyssey, and provides the PIPs++ video baseline that MATE must beat.","marker":"[44]"},{"why":"Provides the TAP-Vid benchmark that defines the tracking-any-point task and the source of the Ev-Davis evaluation set.","marker":"[9]"},{"why":"ETAP is the existing event-only tracking-any-point method whose survival and accuracy numbers MATE claims to exceed.","marker":"[14]"},{"why":"FE-TAP is the frame-event fusion tracker whose published Feature Age results serve as a hybrid baseline in the feature-tracking comparison.","marker":"[25]"},{"why":"DeepEvT establishes the data-driven event feature-tracking approach and the FA/EFA metrics used for the second task.","marker":"[27]"},{"why":"CoTracker3 is a strong video-based TAP baseline compared on Ev-PointOdyssey and Ev-Davis.","marker":"[20]"},{"why":"The EDS dataset provides real event-plus-video sequences with camera motion for feature-tracking evaluation.","marker":"[18]"},{"why":"The Event Camera dataset supplies the EC feature-tracking evaluation set and its asynchronous ground-truth feature points.","marker":"[30]"}],"fun_headline_variants":["Event-only tracker beats video methods at point tracking","MATE: event motion cues beat video tracking for point survival","Event tracker with motion guidance outlasts video on point tracking","17.9% better point survival: MATE uses event dynamics","No blind time: MATE event tracker wins point benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracy depends on setting the time-surface window length using the ground-truth event frequency; in a real deployment that frequency is unknown, and the paper does not test how well MATE performs without that knowledge.","fun_headline_variants_meta":{"raw":{"variants":["Event-only tracker beats video methods at point tracking","MATE: event motion cues beat video tracking for point survival","Event tracker with motion guidance outlasts video on point tracking","17.9% better point survival: MATE uses event dynamics","No blind time: MATE event tracker wins point benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000234,"raw_usage":{"total_tokens":1515,"prompt_tokens":984,"completion_tokens":531,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":448}},"tokens_in":600,"tokens_out":531,"duration_ms":4952,"temperature":1.0,"reasoning_tokens":448,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:29:41.854397+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Ev-PointOdyssey and Ev-Davis evaluations with the time-surface window length selected without ground-truth frequency (for example, fixed, or adapted from observed event rates), and check whether $Survival_{50}$ and median trajectory error still beat Ev-PIPs++ and the video baselines in Table 1; if the gap shrinks or reverses, the central practical claim fails.","supporting_citations":[{"cited_title":"Pointodyssey: A large-scale synthetic dataset for long-term point tracking","cited_arxiv_id":null,"evidence_quote":"Supplies the PointOdyssey synthetic dataset used for training and for building Ev-PointOdyssey, and provides the PIPs++ video baseline that MATE must beat."},{"cited_title":"Tap-vid: A benchmark for track- ing any point in a video","cited_arxiv_id":null,"evidence_quote":"Provides the TAP-Vid benchmark that defines the tracking-any-point task and the source of the Ev-Davis evaluation set."},{"cited_title":"ETAP: Event-based Tracking of Any Point","cited_arxiv_id":"2412.00133","evidence_quote":"ETAP is the existing event-only tracking-any-point method whose survival and accuracy numbers MATE claims to exceed."},{"cited_title":"Tracking Any Point with Frame-Event Fusion Network at High Frame Rate","cited_arxiv_id":"2409.11953","evidence_quote":"FE-TAP is the frame-event fusion tracker whose published Feature Age results serve as a hybrid baseline in the feature-tracking comparison."},{"cited_title":"Data-driven feature tracking for event cameras","cited_arxiv_id":null,"evidence_quote":"DeepEvT establishes the data-driven event feature-tracking approach and the FA/EFA metrics used for the second task."},{"cited_title":"Event-aided direct sparse odometry","cited_arxiv_id":null,"evidence_quote":"The EDS dataset provides real event-plus-video sequences with camera motion for feature-tracking evaluation."},{"cited_title":"The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and slam","cited_arxiv_id":null,"evidence_quote":"The Event Camera dataset supplies the EC feature-tracking evaluation set and its asynchronous ground-truth feature points."}],"review_version":1}