{"id":"d3308d8a-843a-480d-9558-bb49588a611e","arxiv_id":"2501.12482","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"TOFFE combines a trainable spiking speed-separation network with a small analog pose-and-direction network to estimate object flow from event cameras at lower energy and latency than prior methods.","lead":"TOFFE is a hybrid spiking and analog neural network that estimates the position, direction, and speed of fast-moving objects from event-camera data. It reports 5.7x to 8.3x lower energy and 4.6x to 5.8x lower latency than earlier event-based object detectors on edge and hybrid neuromorphic hardware.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Multi-object claim is structurally impossible: one OFPD (x,y,angle) output per speed bin cannot represent two objects sharing a bin; Fig. 6 overstates the method.","rationale":"The reader's weakest assumption correctly identifies the multi-object generalization. I agree with that assessment; it is the most load-bearing issue because it concerns the central functional claim rather than the efficiency measurement. The paper's efficiency results are plausible: Table IV compares dynamic energy and latency on the same hardware, and the hybrid Loihi-2/TX2 setup is a reasonable decomposition given that the OFPD ANN needs a conventional core. The accuracy numbers in Table III, however, are only reported on the authors' synthetic single-object/single-speed-per-input setup, and no baseline or error bars are given. The multi-object issue is more fundamental than the missing baselines: even with perfect OFS speed separation, two objects in one speed bin cannot be represented by the single-head OFPD. Thus the claim in Fig. 6 is not supported by the architecture. A conditional accept with a mandatory scope revision or multi-object extension is the appropriate outcome: the efficiency and single-object accuracy results may stand, but the paper must not claim general multi-object detection and tracking without either changing the output head or adding per-bin multi-object evaluation. This does not move the reader's verdict, so I mark it unchanged.","tokens_in":12070,"tokens_out":5437,"duration_ms":57285,"concrete_test":"Run TOFFE on a synthetic two-object sequence where both objects move at the same speed within one bin (e.g., two squares at 30 m/s, bin 2) from spatially separated positions. Compare the OFPD output for that bin against the two ground-truth centers and directions. Because the FC pose head emits a single (x,y) tuple, the expected result is one averaged (or one arbitrarily selected) center rather than two; such an outcome would directly invalidate the Fig. 6 multi-object claim. A simpler static check is to inspect the output dimension of the OFPD FC layer: a single 2-D pose output cannot encode two objects, which settles the structural limitation without running inference.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central functional claim (abstract, Fig. 6) is that TOFFE estimates pose, direction, and speed for many independently moving objects. The architecture, though, gives each speed bin one OFPD head whose output is a single fully-connected prediction: one (x,y) pose and one angle (Section III-D). The OFS front-end sorts events into N speed bins; if two objects fall in the same bin, their events are merged before OFPD, so that bin's head can return at most one centroid and direction. Multi-object inference is therefore not an untested empirical extension but a structural impossibility under the current output representation. Training (Section III-E) also uses single-speed inputs, so the network never sees multi-object inputs. No experiment with two objects in one speed bin is reported; Fig. 6 is the only evidence for multi-object operation. This matters because 'Object Detection and Tracking' and 'Object Flow' are inherently multi-object tasks, and real scenes routinely contain objects moving at comparable speeds. The efficiency numbers in Table IV are not affected, but the claim of per-object estimation for arbitrary scenes is. The paper should either explicitly scope its claims to one object per speed bin or replace the single-head OFPD with a set-valued/region output and evaluate on a multi-object benchmark.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes TOFFE, a hybrid SNN-ANN pipeline for estimating \"object flow\" (pose, motion direction, and speed) from event-camera data. The architecture first separates events into four speed bins using independently trained LIF-based SNN models (OFS), then passes each bin's events through a shared lightweight ANN (OFPD) that regresses a single (x,y) pose and a single motion angle per bin. The authors introduce a synthetic Gazebo-based dataset with high-rate ground truth, train OFS and OFPD in a supervised manner on single-object sequences, and report accuracy errors at several event time windows (Table III). They also measure dynamic energy and latency on CPU, RTX 3090-Ti, Jetson TX2, and a Loihi-2 + Jetson TX2 hybrid setup, reporting large efficiency gains over Adaptive-SpikeNet and DOTIE (Table IV). The central claimed capability, stated in the abstract and Fig. 6, is pose, direction, and speed estimation for many independently moving objects.","tokens_in":12361,"tokens_out":3868,"duration_ms":40268,"significance":"The efficiency comparison is a concrete strength: energy and latency are measured on real hardware with a clearly described methodology, and the hybrid Loihi-2 + Jetson TX2 measurement is a useful data point for neuromorphic edge deployment. The proposed split of speed separation (SNN) and pose/direction regression (ANN) is architecturally sensible for the single-object-per-speed-bin case, and the synthetic dataset with 20 kHz ground truth is a potentially reusable resource for high-speed event-vision research. However, the accuracy evaluation is narrow: it uses only the authors' synthetic dataset, contains no accuracy comparison against any baseline, and reports no error bars or repeated-run statistics. Moreover, the multi-object claim in the abstract and Fig. 6 is not supported by the architecture, which outputs exactly one pose and direction per speed bin. If scoped to one object per speed bin, the efficiency results stand; as written, the paper overclaims its functional capability.","major_comments":[{"comment":"The central claim that TOFFE estimates pose, direction, and speed for many independently moving objects is structurally inconsistent with the architecture. In Section III-D, each OFPD head outputs a single (x,y) pose and a single direction angle per speed bin, and in Section III-F the events for each speed bin are passed to one OFPD copy. If two objects move at speeds that fall in the same bin, their events are merged before OFPD, so the bin's head can return at most one centroid and one direction. The training setup (Section III-E) also uses inputs with a single moving object at a single speed, so the network never observes multi-object inputs. The manuscript should either explicitly restrict all claims to scenarios with at most one object per speed bin, or replace the single-head OFPD with a set-valued/region output and evaluate on a multi-object benchmark. As it stands, Fig. 6 overstates the method's capability.","section":"III-D, III-F, Fig. 6"},{"comment":"The accuracy evaluation is not sufficient to support the conclusion that TOFFE achieves \"performance comparable to state-of-the-art object detection and optical flow methods.\" Table III reports only TOFFE's own pose, direction, and speed errors on the authors' synthetic dataset; there is no accuracy comparison against DOTIE, Adaptive-SpikeNet, or any other baseline, no error bars over dataset splits or repeated training runs, and no real-world validation. A comparison on a shared benchmark (or a clearly justified reason why such a comparison is infeasible) is needed for the claimed functional performance.","section":"IV-A, Table III, Conclusion"},{"comment":"The reported speed error of 10.649 m/s at the optimal dt=500 is large relative to the speed-bin widths in Table II (bin 1 is 1–18 m/s, bin 2 is 18–42 m/s). This error is comparable to the width of bin 1, so a substantial fraction of speed estimates may be assigned to the wrong speed bin. The paper should report speed-classification accuracy (confusion matrix or per-bin accuracy) and discuss how the speed error affects the claimed speed-separation capability. Without this, the practical usefulness of the speed estimates for tracking is unclear.","section":"IV-A, Tables II and III"}],"minor_comments":[{"comment":"There are several typos: \"inforamtion\" should be \"information\", \"an frame\" should be \"a frame\", and the caption of Fig. 3a contains a duplicated \"for\" (\"trajectories for for TOFFE dataset\").","section":"III-C"},{"comment":"\"evetns\" should be \"events\" in the description of OFS training.","section":"III-E"},{"comment":"The post-processing step in Eq. (3)–(4) uses a closing operation followed by inversion to mask higher-speed events; the sentence describing it says \"The Closing operation ... incorporates dilation followed by erosion operations,\" which is the standard definition, but the text immediately after uses \"and above\" inconsistently (\"events corresponding to speed > speedk and above\"). Please clarify the ordering and the exact set of speeds removed.","section":"III-F"},{"comment":"The hybrid latency of 15.33 ms for TOFFE is described as \"approximately 65 frames per second\"; 1000/15.33 ≈ 65.2, so this is consistent, but the text should state whether this is per-inference latency or includes the post-processing overhead of the OFS masking steps, since that overhead is part of the inference pipeline.","section":"IV-B, Table IV"},{"comment":"The OFPD architecture is described as \"a two layered convolutional network followed by a fully-connected layer,\" but no kernel sizes, channel counts, or final activation details are provided. Sufficient detail should be given to reproduce the network.","section":"III-D"},{"comment":"The paper states the dataset is \"provided\" but does not give a URL or repository reference. Please include the release location or state clearly that it is available upon request.","section":"III-C"}],"recommendation":"major_revision","confidential_remarks":"The multi-object claim is load-bearing: the title, abstract, and Fig. 6 promise object detection and tracking for multiple objects, but the architecture's single-output-per-bin representation cannot represent two objects in the same speed bin. This is fixable either by scoping the claims to one object per speed bin (which would substantially reduce the paper's contribution) or by a non-trivial architectural change with new multi-object experiments. In addition, the accuracy evaluation currently compares against no baselines, despite the conclusion claiming parity with state-of-the-art methods. I recommend major revision rather than rejection because the efficiency measurements and the single-object-per-bin formulation are sound and could form the basis of a publishable paper after substantial re-scoping; however, if the authors wish to keep the multi-object claim, the experimental gap is significant."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick read on TOFFE. The strongest part is the hardware story. The energy and latency measurements on Jetson TX2 and Loihi-2+TX2 are concrete, with clear methodology (idle power subtraction, latency formula). The 5.7x/8.3x energy and 4.6x/5.8x latency improvements over DOTIE with clustering are plausible because TOFFE replaces DBSCAN with a tiny ANN head. That is a real engineering contribution.\n\nThe new algorithmic piece is making DOTIE's speed separation trainable, adding a direction head, and packaging pose plus discretized speed plus direction as 'object flow'. The synthetic dataset with 20 kHz ground truth is also useful, though it is simulator-only and no artifacts are released.\n\nSoft spots, in order. First, the multi-object claim. Section III-D gives each speed bin one OFPD head with a single fully-connected output: one (x,y) and one angle. The masking pipeline merges events from all objects in a bin before OFPD. Training is single-object per input (III-E). So Fig. 6, which shows many independently moving objects each getting pose, direction, and speed, cannot be realized when two objects share a speed bin. That is not an untested empirical extension; it is a representational limit. The paper should either explicitly scope to one object per bin or change the output head to a set-valued or region representation. As written, 'Object Detection and Tracking' overstates what is evaluated.\n\nSecond, accuracy is only on their own synthetic test split, with no baseline comparison, no error bars, no real event data. The speed error at dt=500 is 10.6 m/s on a bin that goes up to 500 m/s, so it is fine relative to bin width, but absolute error would be large for real tracking. Third, the baselines are both from the same lab. That is not disqualifying, but it means the efficiency comparison does not include independent event-based detectors. Fourth, no code or data is public, so the efficiency numbers are hard to reproduce.\n\nThe paper is worth a serious referee: the efficiency result is measured and the architecture idea is clearly described. But the multi-object claim must be fixed or scoped, and the accuracy evaluation needs at least one independent baseline or real event sequence. Recommend major revision rather than rejection.","headline":"TOFFE's efficiency numbers are believable and the trainable speed separation is a real step up from DOTIE, but the multi-object claim in Fig. 6 is structurally impossible with one output head per speed bin.","tokens_in":12858,"tokens_out":1958,"would_cite":false,"duration_ms":19854,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TOFFE estimates an object's position, direction, and speed directly from event-camera streams, using a hybrid spiking-and-convolutional network that cuts energy and latency several-fold versus prior event-based detectors.","keywords":["event-based vision","spiking neural networks","object detection","object tracking","object flow","neuromorphic hardware","edge computing","synthetic dataset"],"falsifier":"Run TOFFE on a synthetic or real event sequence in which two objects of different shapes move at speeds within the same speed bin but at different image locations; the OFPD head can emit only one pose and direction per bin, so the pipeline must either miss one object or produce a blended estimate, which a simple check of the two ground-truth poses would expose.","tokens_in":11910,"feed_emoji":"⚡","tokens_out":5710,"duration_ms":52544,"temperature":0.7,"pith_summary":"The paper claims that the full object detection-and-tracking task for high-speed edge robots can be reformulated as \"object flow\" — a per-object pose, direction, and speed estimate computed directly from sparse event-camera streams — and that this reformulation can be served by a hybrid network far cheaper than prior event-based detectors. TOFFE splits the problem into a speed-separation spiking network and a small convolutional pose-and-direction network, trains them on a new synthetic high-speed event dataset, and reports pose error around 5 pixels, direction error around 11 degrees, and speed error around 11 m/s at the best time window, while using 5.7-8.3x less dynamic energy and 4.6-5.8x lower latency than its nearest baselines on edge-grade hardware. If correct, the result would make event cameras practical for low-power, real-time navigation of small autonomous vehicles.","feed_headline":"Hybrid event network tracks fast objects at 5-8x lower energy","feed_subtitle":"TOFFE turns event streams into per-object pose, direction, and speed, enough for high-speed edge autonomy.","key_machinery":"The load-bearing mechanism is the speed-bin separation chain made of trained leaky integrate-and-fire spiking neurons. The OFS block uses LIF neurons with learnable firing threshold $v_{th}$ and leak $\\lambda$ (Eqs. 1-2) so that each OFS$_k$ fires for objects at speed bin $k$ and above, acting as a temporal high-pass filter; during inference the outputs are processed in descending speed order, each stage's binary event mask (built by morphological closing and inversion) removes already-assigned events from the input to the next stage. The resulting speed-separated event bins are fed to the OFPD network, a two-layer CNN plus fully-connected layer with separate pose $(x,y)$ and direction (angle) heads. This division lets each subnetwork stay tiny: the spiking network only discriminates temporal rates, and the ANN only reads spatial structure from already-isolated objects.","core_discovery":"In TOFFE's own terms, the central discovery is that object flow can be computed by a trainable division of labor: a single-layer spiking network with learnable leak and threshold acts as a velocity filter, separating events into four speed bins by exploiting the high-pass behavior of leaky integrate-and-fire neurons, and a lightweight two-layer convolutional network then predicts one pose and one motion direction per speed bin from the speed-separated event bins. The full system replaces per-pixel optical flow and clustering-based detection with sparse, speed-discretized object-level estimates, and the paper reports that on the Jetson TX2 and on a Loihi-2 plus Jetson TX2 hybrid setup the approach reduces dynamic energy by 5.7x/8.3x and latency by 4.6x/5.8x compared with Adaptive-SpikeNet and DOTIE baselines, at dt=500 achieving 5.355 pixels pose error, 10.769 degrees direction error, and 10.649 m/s speed error.","pith_inferences":["Editorial inference: The single-output-per-bin assumption is the natural stress point; a production deployment would need the OFPD head to either output a variable number of objects per bin or be preceded by a lightweight segmentation of the speed-separated events, otherwise multi-object scenes with matched speeds will silently drop objects.","The paper's error metrics are reported only on synthetic data; a real event camera would add noise, varying object textures, and ego-motion, so the accuracy and latency numbers should be re-measured on recorded real events before relying on them in a drone controller.","The speed-bin idea suggests a testable scaling prediction: if the number of speed bins grows, the sequential cascade of morphological masks and OFS stages should add roughly linearly to latency, so the architecture's sweet spot will be a small number of bins (as the four used here).","A direct extension the paper doesn't explore: using the direction head's angle for simple constant-velocity extrapolation of pose could turn the system into a tracker with zero extra computation, since the pose and direction are already aligned in time."],"forward_implications":["If TOFFE's speed-bin reformulation holds, object tracking on edge robots no longer needs dense per-pixel optical flow or iterative clustering; a per-object estimate is enough to feed downstream control loops, which is the main reason for the latency and energy reductions.","The same hybrid split (spiking temporal filter plus analog spatial network) could be applied to other event-stream tasks that decompose into a temporal-rate component and a spatial component, such as gesture recognition or landing-site selection.","The synthetic dataset's ground truth at 20 kHz means training and evaluation are no longer capped by the frame rate of traditional sensors, so the reported error numbers are the first benchmark for truly high-speed object flow.","On hybrid Loihi-2 plus Jetson TX2 hardware, TOFFE runs at roughly 65 inferences per second, which the paper argues is sufficient for real-world edge autonomy; if the OFPD part were also mapped to neuromorphic hardware, the same approach could go even lower in energy."],"supporting_citations":[{"why":"Supplies the learnable leaky integrate-and-fire neuronal dynamics and the discretized event-volume input representation that TOFFE's OFS and OFPD blocks build on.","marker":"[14]"},{"why":"DOTIE, the speed-separation spiking baseline that TOFFE makes trainable and reuses as its OFS block; also the clustering-based baseline that anchors the energy/latency comparison.","marker":"[15]"},{"why":"Spike-FlowNet, the prior hybrid SNN-ANN architecture that motivates splitting temporal spike processing from spatial analog decoding.","marker":"[29]"},{"why":"Defines the leaky integrate-and-fire neuron model whose membrane potential and firing rule (Eqs. 1-2) drive the OFS speed separator.","marker":"[40]"},{"why":"Source of the trainable threshold and leak parameters that turn the OFS layer into a learnable multi-speed filter.","marker":"[41]"}],"fun_headline_variants":["Event flow network cuts energy 5.7x, latency 4.6x","Hybrid SNN-ANN event tracker: 5x less energy, 5x less latency","TOFFE: speed-binned event flow for high-speed edge tracking","Bio-inspired event tracking with 8x energy savings on Loihi","Event-based object flow: 5.7x energy drop, 4.6x latency drop"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline assumes that at most one object ends up in any one speed bin at a time, because the pose-and-direction head returns just one $(x, y, angle)$ per bin; the paper never trains or tests the multi-object case, and two objects sharing a speed bin would violate the architecture's output capacity.","fun_headline_variants_meta":{"raw":{"variants":["Event flow network cuts energy 5.7x, latency 4.6x","Hybrid SNN-ANN event tracker: 5x less energy, 5x less latency","TOFFE: speed-binned event flow for high-speed edge tracking","Bio-inspired event tracking with 8x energy savings on Loihi","Event-based object flow: 5.7x energy drop, 4.6x latency drop"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000595,"raw_usage":{"total_tokens":2844,"prompt_tokens":1064,"completion_tokens":1780,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":680,"completion_tokens_details":{"reasoning_tokens":1670}},"tokens_in":680,"tokens_out":1780,"duration_ms":13623,"temperature":1.0,"reasoning_tokens":1670,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:08:31.678706+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run TOFFE on a synthetic or real event sequence in which two objects of different shapes move at speeds within the same speed bin but at different image locations; the OFPD head can emit only one pose and direction per bin, so the pipeline must either miss one object or produce a blended estimate, which a simple check of the two ground-truth poses would expose.","supporting_citations":[{"cited_title":"Adaptive-spikenet: event-based optical flow estimation using spiking neural networks with learnable neuronal dynamics,","cited_arxiv_id":null,"evidence_quote":"Supplies the learnable leaky integrate-and-fire neuronal dynamics and the discretized event-volume input representation that TOFFE's OFS and OFPD blocks build on."},{"cited_title":"Dotie-detecting objects through temporal isolation of events using a spiking architecture,","cited_arxiv_id":null,"evidence_quote":"DOTIE, the speed-separation spiking baseline that TOFFE makes trainable and reuses as its OFS block; also the clustering-based baseline that anchors the energy/latency comparison."},{"cited_title":"Spike-flownet: event-based optical flow estimation with energy-efficient hybrid neural networks,","cited_arxiv_id":null,"evidence_quote":"Spike-FlowNet, the prior hybrid SNN-ANN architecture that motivates splitting temporal spike processing from spatial analog decoding."},{"cited_title":"Lapicque’s introduction of the integrate-and-fire model neuron (1907),","cited_arxiv_id":null,"evidence_quote":"Defines the leaky integrate-and-fire neuron model whose membrane potential and firing rule (Eqs. 1-2) drive the OFS speed separator."}],"review_version":1}