{"id":"5b0b1cb1-bc23-4a27-8592-6c630afcebd1","arxiv_id":"2506.07375","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A multi-class collaborative detection and tracking framework that fuses multi-agent LiDAR and camera features, uses DINOv2 for re-identification, and adapts track lifetimes to object speed.","lead":"This paper introduces DINO-CoDT, a system for collaborative perception by connected vehicles that detects and tracks cars, pedestrians, and trucks together. It combines attention-based fusion across agents, visual re-identification with DINOv2, and velocity-adaptive track management, reporting improved results on two driving datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tracking gains may be artifacts of a more accurate detector; the REID module, as evidenced by its own ablation, increases ID switches and the claimed tracking improvements are not robust.","rationale":"The reader identified the REID module's dependence on raw image sharing as a key weakness and noted the IDSW contradiction, but did not fully center the fact that the ablation data directly refute the central tracking claim. The strongest claim in the paper is that the framework 'significantly outperforms' in tracking by reducing IDSW. Table V shows the complete model has more ID switches than the detector-only variant (3746 vs. 3316), and Table VI shows the REID module (DINOv2) yields essentially no sAMOTA improvement (23.43 vs. 22.98) while increasing IDSW compared to no REID in all but one ResNet variant. These internal inconsistencies are not mere noise; they demonstrate the tracking modules do not deliver their stated benefit in the very configuration the paper highlights. The detection gains are more credible because Table II is consistent and ablations show monotonic gains, but the paper's title and abstract emphasize collaborative detection AND tracking, and the tracking half is empirically unsupported and conceptually self-contradictory regarding bandwidth. Therefore, the paper should be rejected in its current form; the detection contribution is plausible but the central multi-task claim fails. This is a REJECT rather than CONDITIONAL because the core promise of IDSW reduction is directly contradicted by the paper's own tables, and no amount of additional analysis can reconcile the fact that the full model increases ID switches. The reader's verdict was CONDITIONAL, but the specific evidence in Table V is strong enough to warrant rejection unless retracted or corrected. I would however recommend the authors rerun tracking ablations with matched detection quality before resubmission, as the underlying detection method may be sound.","tokens_in":15112,"tokens_out":1885,"duration_ms":19573,"concrete_test":"Re-run the V2X-Real tracking experiments holding detection outputs fixed: (1) Feed the same detector (V2X-ViT) into the proposed tracking backbone with and without REID and VATM, reporting IDSW and sAMOTA per class; (2) Compare the full DINO-CoDT against a variant that replaces DINOv2 embeddings in REID with the same ResNet features used in the detector, and (3) verify whether raw camera images are actually transmitted in the bandwidth-limited setting by re-simulating communication with a 1x1 conv bottleneck and measuring whether REID can still crop patches. If the full system's IDSW remains higher than detector-only AB3DMOT, the tracking modules' claimed benefit is falsified.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central tracking claim—that DINO-CoDT reduces IDSW and outperforms baselines in tracking—is undercut by its own ablation. In Table V, the full system (A+B+C) reports IDSW=3746 for the pedestrian class, while the detector-only variant (A) reports IDSW=3316; the full tracking modules cause 430 additional ID switches. Table VI shows that adding the REID module with DINOv2 reduces sAMOTA relative to the no-REID baseline (23.43 vs. 22.98 is a slight increase, not reduction) and actually increases IDSW from 3316 to 3301 is a tiny reduction, but the R18 and R50 variants increase IDSW to 3823 and 3867. The claimed IDSW reduction (4016 to 3927) in the ablation narrative contradicts Table V's numbers for the full system. More fundamentally, the tracking comparison in Table III uses AB3DMOT on all detectors, but the 'Ours*' row shows that simply using the proposed detector already yields the largest tracking gains on most metrics, especially sAMOTA and MOTA for vehicles. The paper does not control for detection quality when attributing tracking improvements, and the only module-level evidence for the REID module shows it performs worse than detection-only tracking in terms of IDSW. Additionally, the REID module's reliance on raw multi-view images (Section IV-B-2) contradicts the detector's bandwidth compression rationale; if raw images are shared, the system already exceeds bandwidth limits, and if not, the module cannot operate as described. These issues make the central tracking claim fragile and likely not attributable to the proposed tracking modules.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DINO-CoDT, a multi-class collaborative perception framework comprising a Global Spatial Attention Fusion (GSAF) module for multi-agent detection, a Tracklet RE-IDentification (REID) module that uses DINOv2 visual embeddings to reduce identity switches, and a Velocity-based Adaptive Tracklet Management (VATM) module. The method is evaluated on the V2X-Real and OPV2V datasets, reporting large detection gains (e.g., +6.8% AP@0.3 on V2X-Real) and tracking improvements in sAMOTA and MOTA. The central claim is that this is the first multi-class collaborative detection-and-tracking framework and that it significantly outperforms existing methods in both tasks.","tokens_in":15440,"tokens_out":4980,"duration_ms":57807,"significance":"If the detection results are reproducible, the GSAF-based multi-scale fusion is a meaningful step for multi-class collaborative perception, and the use of DINOv2 for cross-agent re-identification is a plausible direction for reducing ID switches. The paper is also among the first to evaluate collaborative tracking on a multi-class real-world benchmark. However, the evidence for the tracking-specific contribution is currently weak: the ablation confounds detection and tracking improvements, and the paper's own data show that the full system increases IDSW relative to its detector-only variant. The significance is therefore conditional on a substantial re-analysis of the experimental results.","major_comments":[{"comment":"The ablation for Module A (GSAF) is confounded by the addition of the camera modality. The baseline V2X-ViT is a LiDAR-only detector, whereas the row with Module A uses an extended multi-modal fusion framework that includes ResNet+LSS image features. The reported +4.1 AP@0.3 for pedestrians therefore cannot be attributed solely to GSAF; it may be due to the inclusion of cameras. To support the detection claim, the authors should ablate GSAF within the same multi-modal detector (i.e., with camera features enabled but without GSAF) rather than against a LiDAR-only baseline.","section":"Section IV-E-1, Table V"},{"comment":"The paper's central tracking claim is that the REID module reduces IDSW, citing a reduction from 4016 to 3927 in the text. However, those numbers refer to the REID module added to the baseline in isolation, not to the complete system. In the full system (A+B+C), IDSW=3746, which is 430 higher than the detector-only variant A (IDSW=3316). Thus, when combined with the improved detector, the REID and VATM modules increase IDSW rather than reduce it. The text should report and discuss the full-system IDSW and the interaction between modules; the current narrative is internally inconsistent.","section":"Section IV-E-1, Table V"},{"comment":"The REID module's benefit is marginal. With DINOv2, sAMOTA increases from 22.98 to 23.43 and IDSW decreases by only 15 (from 3316 to 3301), while ResNet-18 and ResNet-50 variants increase IDSW to 3823 and 3867. Describing the DINOv2 result as a 'significant performance improvement' is an overstatement. The authors should provide effect sizes or significance tests, and discuss whether a 15-switch reduction is meaningful relative to the total number of ground-truth tracks.","section":"Table VI"},{"comment":"The REID module requires raw multi-view images from all connected agents at every frame; Section IV-B-2 states 'we use camera images from all connected agents to perform re-identification'. This appears to conflict with the detector's bandwidth-compression design, which shares compressed BEV features to limit communication. The paper does not quantify the bandwidth of transmitting raw camera images, nor does it explain how image sharing coexists with the stated bandwidth constraints. If raw images are not shared, the REID module as described cannot operate; if they are shared, the bandwidth argument for the detector is undermined.","section":"Section IV-B-2 and III-G"}],"minor_comments":[{"comment":"The header of Table III contains a duplicated 'AMOTA' column; the first should be 'sAMOTA' and the second 'AMOTA'.","section":"Table III"},{"comment":"The notation 'aTego' is confusing; please define the transformation matrix explicitly and explain how it differs across agents.","section":"Section III-G, Eqs. (5)-(6)"},{"comment":"When discussing Module B, the text says 'reduction in ID Switches from 4016 to 3927' without clarifying that this is the isolated B row relative to the baseline, not the full system. Please make this explicit to avoid confusion.","section":"Section IV-E-1"},{"comment":"Please state whether the qualitative examples in Fig. 5 are from the test split and whether the same hyperparameters were used as in the quantitative evaluation.","section":"Section IV-F, Fig. 5"},{"comment":"The conclusion reports '+5.38% on sAMOTA' without specifying that this improvement is for the pedestrian superclass only, as in Table V. Please qualify the claim.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The paper shares two co-authors with DINO-MOT [25], and the REID module is directly inspired by that work. The overlap is acknowledged and the current application is distinct enough that I do not see a circularity problem, but the authors should ensure that DINO-MOT's contribution is clearly separated in the related-work section. The main concern is that the experimental narrative overstates the tracking contribution; if the authors can provide a controlled ablation and revise the claims accordingly, the paper would be suitable for reconsideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my honest read. The genuinely new thing is the integration: a multi-class collaborative perception pipeline—vehicles, pedestrians, trucks—on V2X-Real, combining scale-aware attention fusion (GSAF), a DINOv2-based re-identification module, and velocity-adaptive tracklet management. That combination hasn't appeared before, and the headline detection numbers are credible and useful: +6.8 AP@0.3 and +8.3 AP@0.5 over V2X-ViT on V2X-Real, with consistent gains on OPV2V. Evaluation is against external benchmarks, the setup is described clearly enough to reproduce in spirit, and the detection claims are not obviously confounded in the way the tracking claims are.\n\nThe soft spots are mostly about attribution. Table V's detection ablation compares V2X-ViT, which is LiDAR-only in this setup, against a LiDAR+camera detector augmented with GSAF, so the +4.61 sAMOTA gain is not an isolated GSAF effect. The tracking modules themselves add almost nothing alone: REID gives +0.18 sAMOTA, VATM +0.16. Worse, the full-system row shows IDSW=3746 for the complete A+B+C model versus 3316 for the detector-only A—the opposite of the paper's claim that REID reduces IDSW from 4016 to 3927. Table VI confirms the weakness: DINOv2 REID reduces IDSW by only 15 relative to no REID (3301 vs 3316), and ResNet variants increase IDSW substantially. The Ours* row in Table III also shows that most of the tracking improvement over V2X-ViT comes from the better detector alone, not from the proposed tracking modules.\n\nBeyond that, Section III-F and Section IV-B-2 contradict each other on which classes use which motion model; the REID module requires raw multi-view images from all agents, which sits awkwardly with the detector's bandwidth-compression rationale; and there are no error bars, which matters when the module-level gains are under a point.\n\nNone of this falsifies the central detection claim. On balance, the paper should go to serious peer review, because it is the first multi-class collaborative tracking study on V2X-Real and the detection result is worth having. But a referee should demand a sensor-matched GSAF ablation, a detector-controlled tracking comparison, a consistent motion-model description, and a direct explanation of the IDSW reversal in the full system. I'd trust the detection half; the tracking half is unproven.","headline":"There is a real and useful multi-class collaborative detection result here, but the tracking contribution is not supported by the paper's own ablations; the paper deserves refereeing, but the tracking modules need a much tighter story.","tokens_in":15998,"tokens_out":3677,"would_cite":true,"duration_ms":43109,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a global spatial attention detector paired with DINOv2 re-identification and velocity-adaptive tracklet management can extend collaborative perception to multiple road-user classes at once.","keywords":["collaborative perception","multi-class object detection","multi-object tracking","DINOv2","tracklet re-identification","global spatial attention fusion","velocity-based tracklet management","V2X-Real dataset"],"falsifier":"Rerun the V2X-Real pedestrian experiment with identical detections and tracker settings but feed the REID module only compressed BEV features or only the ego agent's own camera images, with no images from other agents. The paper attributes the full-module IDSW drop from 4016 to 3746 and the sAMOTA gain from 18.37 to 23.75 to the three modules together; if withholding cross-agent images eliminates the IDSW reduction or most of the sAMOTA gain, the central claim that multi-agent image sharing drives the improvement would be falsified.","tokens_in":1883,"feed_emoji":"🚗","tokens_out":1799,"duration_ms":93886,"temperature":0.7,"pith_summary":"Collaborative perception has mostly been built around the vehicle superclass, so pedestrians, cyclists, and trucks are handled poorly or not at all. This paper tries to establish that a single framework can do multi-class collaborative 3D detection and tracking by combining a multi-scale global spatial attention fusion detector, a DINOv2-based tracklet re-identification module, and a velocity-adaptive tracklet management module. On the V2X-Real real-world benchmark it reports average AP@0.3 of 53.3 versus 46.5 for V2X-ViT and pedestrian sAMOTA of 23.75 versus 18.37 for the baseline, with ID switches down from 4016 to 3746; on OPV2V it also reports detection gains. If the claims hold, collaborative perception systems could keep stable identities for pedestrians and other vulnerable road users across agents, which matters for safe autonomous driving in mixed traffic.","feed_headline":"Multi-class collaborative perception beats vehicle-only baselines","feed_subtitle":"Reported gains: +6.8 AP@0.3 on V2X-Real and +5.38 pedestrian sAMOTA for mixed-traffic autonomy","key_machinery":"The detection side is carried by the Global Spatial Attention Fusion (GSAF) module: BEV features from all agents are split into scale-specific branches, processed with multi-scale self-attention using non-overlapping windows of size $w_s \\times w_s$, and recombined by an inter-agent adaptive weighting $\\beta_{m\\to n}(i,j)$ computed by a shared MLP. The tracking side is carried by the Tracklet REID module, which projects unmatched detections onto raw multi-view images from all agents via camera intrinsics and ego-to-camera transforms, crops the regions, embeds them with the 31st layer of DINOv2, and matches them against a tracklet look-up table by cosine similarity. The Velocity-based Adaptive Tracklet Management module sets each tracklet's maximum age as $A_t^j = A_c + \\alpha\\sqrt{v_x^2+v_y^2}$, so faster-moving objects are allowed to live longer, with class-specific Kalman motion models underneath.","core_discovery":"The central claim is that the vehicle-only ceiling in collaborative perception is not intrinsic, and that a single framework can handle multiple road-user classes simultaneously by pairing better fusion with appearance-based identity repair. On the paper's own terms, DINO-CoDT is the first framework to extend collaborative perception to multiple classes at once, and it establishes this by fusing LiDAR and camera BEV features with a global spatial attention fusion module, then repairing tracking associations with visual semantic matching from DINOv2 and managing tracklet lifetimes via a velocity-dependent max-age formula. On V2X-Real the full system reports AP@0.3 of 53.3 and AP@0.5 of 45.6 across vehicle, pedestrian, and truck superclasses, improving on V2X-ViT by 6.8 and 8.3 points; on the pedestrian split it reports sAMOTA 23.75 versus 18.37 and reduces IDSW from 4016 to 3746. On OPV2V, a vehicle-only simulated set, it reports AP@0.5 of 89.8 versus V2X-ViT's 85.9 and sAMOTA 57.98 versus 51.90, which the authors read as evidence that the multi-class design generalizes.","pith_inferences":["A natural extension not developed in the paper is to compress the raw image crops that the REID module requests, sending deep embeddings or sharply compressed patches only for unmatched detections; whether DINOv2 matching still works on those compressed inputs is a testable design choice.","The ablation as reported suggests the largest tracking gain comes from the GSAF detector itself, which lifts pedestrian sAMOTA by 4.61 points, while the REID and VATM modules each contribute under 0.2 points on their own; a fairer comparison of the tracking-specific modules would measure them against a stronger detector.","The same appearance-only DINOv2 layer-31 embedding could be reused for multi-camera pedestrian tracking across non-overlapping views outside V2X settings, since the matching is purely visual and does not require shared spatial coordinates.","VATM's linear velocity term could be replaced by learned per-class death policies or acceleration-aware rules; the paper only tunes the single scalar $\\alpha$ globally and reports marginal pedestrian gains."],"forward_implications":["A detector with GSAF can be plugged into existing BEV collaborative pipelines and improve small-object detection: the reported pedestrian AP@0.3 rises from the V2X-ViT baseline's 37.4 to 41.5.","A foundation-model embedding bank can serve as a cross-agent appearance memory, so ID switches in pedestrian tracking drop without task-specific fine-tuning, from 4016 to 3746 with all modules active.","Velocity-adaptive death thresholds imply that fixed max-age parameters are a bottleneck for multi-class tracking and that motion statistics should be used per class.","The same pipeline transfers to vehicle-only simulated data on OPV2V with gains over baselines, so adding multi-class support need not cost vehicle-only performance.","If the reported numbers hold, DINO-CoDT offers the first demonstration that collaborative perception can produce stable tracklets for pedestrians and trucks, not just vehicles."],"supporting_citations":[{"why":"supplies the main V2X-ViT baseline that DINO-CoDT must beat in detection and tracking.","marker":"[5]"},{"why":"provides the OPV2V simulated dataset used to test generalization.","marker":"[7]"},{"why":"provides the V2X-Real real-world multi-class benchmark, its superclass labels, and the baseline metrics.","marker":"[8]"},{"why":"provides the DINOv2 vision foundation model whose layer-31 embeddings power the REID module.","marker":"[13]"},{"why":"defines the AB3DMOT tracking-by-detection baseline and the sAMOTA, AMOTA, and IDSW metrics used in all comparisons.","marker":"[15]"},{"why":"defines the four-step collaborative detection pipeline of metadata sharing, feature extraction, compression, and fusion that DINO-CoDT extends with GSAF.","marker":"[21]"},{"why":"introduces the DINO-MOT look-up-table paradigm for pedestrian re-identification that the REID module follows.","marker":"[25]"},{"why":"supplies the ResNet plus GeneralizedLSS image-to-BEV fusion approach used for depth-aware camera features.","marker":"[29]"},{"why":"motivates using the 31st DINOv2 encoder layer for visual matching, which the REID module adopts for its embeddings.","marker":"[31]"}],"fun_headline_variants":["First multi-class collaborative perception framework","Vehicle-only ceiling broken for collaborative sensing","Multi-class tracking via vision foundation models","Detect and track all road users with DINO-CoDT"],"cache_read_input_tokens":18048,"weakest_assumption_plain":"The tracking story assumes every connected agent can send the ego agent raw multi-view camera images, or image crops, of unmatched detections in real time, since the REID module crops and embeds those images; if bandwidth or sensor asynchrony blocks that image sharing, the ID-switch reduction collapses.","fun_headline_variants_meta":{"raw":{"variants":["First multi-class collaborative perception framework","Vehicle-only ceiling broken for collaborative sensing","Multi-class tracking via vision foundation models","Detect and track all road users with DINO-CoDT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000139,"raw_usage":{"total_tokens":1212,"prompt_tokens":1055,"completion_tokens":157,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":671,"completion_tokens_details":{"reasoning_tokens":102}},"tokens_in":671,"tokens_out":157,"duration_ms":2754,"temperature":1.0,"reasoning_tokens":102,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:35:25.256568+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the V2X-Real pedestrian experiment with identical detections and tracker settings but feed the REID module only compressed BEV features or only the ego agent's own camera images, with no images from other agents. The paper attributes the full-module IDSW drop from 4016 to 3746 and the sAMOTA gain from 18.37 to 23.75 to the three modules together; if withholding cross-agent images eliminates the IDSW reduction or most of the sAMOTA gain, the central claim that multi-agent image sharing drives the improvement would be falsified.","supporting_citations":[{"cited_title":"Dino-mot: 3d multi-object tracking with visual foundation model for pedestrian re- identification using visual memory mechanism,","cited_arxiv_id":null,"evidence_quote":"introduces the DINO-MOT look-up-table paradigm for pedestrian re-identification that the REID module follows."},{"cited_title":"V2x-vit: Vehicle-to-everything cooperative perception with vision transformer,","cited_arxiv_id":null,"evidence_quote":"supplies the main V2X-ViT baseline that DINO-CoDT must beat in detection and tracking."},{"cited_title":"Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to- vehicle communication,","cited_arxiv_id":null,"evidence_quote":"provides the OPV2V simulated dataset used to test generalization."},{"cited_title":"V2x-real: a largs-scale dataset for vehicle-to-everything cooperative perception,","cited_arxiv_id":null,"evidence_quote":"provides the V2X-Real real-world multi-class benchmark, its superclass labels, and the baseline metrics."},{"cited_title":"AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metrics,","cited_arxiv_id":null,"evidence_quote":"defines the AB3DMOT tracking-by-detection baseline and the sAMOTA, AMOTA, and IDSW metrics used in all comparisons."},{"cited_title":"Where2comm: Communication-efficient collaborative perception via spatial confidence maps,","cited_arxiv_id":null,"evidence_quote":"defines the four-step collaborative detection pipeline of metadata sharing, feature extraction, compression, and fusion that DINO-CoDT extends with GSAF."}],"review_version":1}