{"id":"a56dd84e-6702-4f67-b93a-5b775f89c170","arxiv_id":"2412.06142","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"AgentAlign aligns camera and LiDAR features across vehicles and roadside sensors to keep cooperative perception accurate under realistic sensor misalignment and noise.","lead":"This paper presents AgentAlign, a system that helps fleets of self-driving cars share camera and laser data even when sensors are slightly misaligned due to vibration, weather, or calibration errors. It also introduces a simulated noisy benchmark, V2XSet-Noise, for testing such systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline V2X-Real result is contradicted by the paper's own ablation table: Table 1 reports the full model at 62.6/60.4 AP while Table 4 reports 49.9/38.4 on the same dataset, with no stated protocol difference, so the SOTA claim is not self-consistent.","rationale":"In good faith, the paper proposes a coherent and potentially useful architecture: CFAS builds a dense depth-variation representation to link sparse LiDAR with camera features, and HAFA adaptively weights modalities per agent. The V2XSet-Noise construction pipeline is also a reasonable contribution. However, the central claim is the reported state-of-the-art performance on V2X-Real, and that claim is undermined by the paper's own ablation. Table 1 and Table 4 both describe the full model on V2X-Real but differ by more than 12 AP points at AP@0.3 and more than 20 points at AP@0.5, with no stated explanation. The reader's REJECT verdict is therefore justified. I would place the load-bearing weight on this internal numerical contradiction rather than on the realism of the hand-crafted noise models: even if the noise models were field-validated, the headline result would remain unsupported while the two tables disagree. The absence of code, data, and error bars compounds the problem, but the contradiction alone is sufficient to prevent acceptance. Because the reader already reached REJECT, my assessment does not change the verdict.","tokens_in":16224,"tokens_out":3788,"duration_ms":37296,"concrete_test":"Request the exact configuration used for Table 1 and Table 4 (checkpoint, noise-corruption settings, evaluation split, and AP computation details), then run the trained full model on the official V2X-Real evaluation split under a single fixed protocol. If the result matches 49.9/38.4, Table 1 is misreported; if it matches 62.6/60.4, Table 4 is misreported or uses an unreported protocol difference. Either way, the authors must state which number is the official V2X-Real result and specify the difference between the two tables. This check can be completed with one evaluation script on the already-trained checkpoint.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3, Table 1 reports the complete AgentAlign model at 62.6% AP@0.3 and 60.4% AP@0.5 on V2X-Real, and the text explicitly claims this surpasses the prior SOTA. Section 4.4, Table 4 reports the same full model (Cam. + Depth Agg. + Depth Var. + HAFA) on the same dataset at 49.9% AP@0.3 and 38.4% AP@0.5. No difference in noise injection, evaluation split, checkpoint selection, or metric definition is stated anywhere in Sections 4.2-4.4 or the supplementary material. Both numbers cannot describe the same configuration under the same protocol, so the headline claim is internally contradicted by the paper's own ablation. This is not primarily a question of noise-model realism; even if every hand-crafted corruption in Section 3.5 were perfectly realistic, the paper must first establish which V2X-Real number is the actual result and why the other table exists. The promise to release code and data in the future does not resolve the contradiction, because no artifacts are available for independent verification.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AgentAlign, a cooperative perception framework that aligns multi-modal features across heterogeneous agents (vehicles and infrastructure) to be robust to sensor misalignment and multifactorial noise. The method introduces a cross-modality feature alignment space (CFAS) that densifies projected LiDAR depth and computes depth gradients, plus a heterogeneous agent feature alignment (HAFA) mechanism that learns an information-attention map to weight camera and LiDAR features. The authors construct a V2XSet-Noise dataset by applying hand-crafted noise models (calibration error, wind vibration, camera distortion, LiDAR motion distortion, time asynchrony, systematic error) to V2XSet and OPV2V, and evaluate on V2X-Real, noisy OPV2V, and V2XSet-Noise, reporting state-of-the-art AP numbers. The central claim is that AgentAlign achieves SOTA performance on real-world multifactorial noise benchmarks.","tokens_in":16464,"tokens_out":2290,"duration_ms":21366,"significance":"If the reported results are correct and reproducible, the work addresses a real and underexplored problem: fragile inter-agent sensor correlations under realistic misalignment. The proposed CFAS and HAFA modules are simple and potentially generalizable, and the V2XSet-Noise dataset could serve as a useful robustness benchmark. However, the paper currently provides no code, no data, no error bars, and no ablations on the real dataset that match the headline numbers, which severely limits the verifyability of the claims. The core architectural ideas deserve consideration, but the evidence presented is not yet at the standard required for publication.","major_comments":[{"comment":"The headline SOTA claim is internally contradicted by the paper's own ablation. Table 1 reports the complete AgentAlign model at 62.6% AP@0.3 and 60.4% AP@0.5 on V2X-Real, while Table 4 reports the full configuration (Cam. + Depth Agg. + Depth Var. + HAFA) at 49.9% AP@0.3 and 38.4% AP@0.5 on the same dataset. No difference in evaluation split, noise injection, checkpoint selection, or metric definition is stated. Both numbers cannot describe the same configuration under the same protocol. The authors must reconcile this discrepancy, explain any protocol difference, and report which number is the actual result for the full model.","section":"§4.3, Table 1 vs §4.4, Table 4"},{"comment":"The simulated noise evaluation is only as meaningful as the realism of the noise models. The parameters (2 Hz sinusoidal vibration, uniform calibration perturbations of ±0.5° and ±0.5 m, 0.1 s synchronization delay, systematic errors in ±0.1) are chosen by the authors without validation against field measurements or an established benchmark. Moreover, the method is explicitly designed to counter these specific noise types, so the V2XSet-Noise results do not provide independent evidence of real-world robustness. The real-world V2X-Real results would supply that evidence, but they are called into question by the Table 1/Table 4 contradiction.","section":"§3.5 and §4.2 (V2XSet-Noise)"},{"comment":"The comparison tables omit the strongest cited multi-modal cooperative perception methods. For instance, HM-ViT [26] and V2X-ViT [30], both cited in the paper, are not included in any comparison, and CooperFuse [37] is also absent. Additionally, no error bars or multiple-run statistics are provided, so the reported improvements (e.g., 62.6 vs 58.6 on V2X-Real) cannot be assessed for statistical significance. The authors should add comparisons to these relevant methods and report variance.","section":"§4.3, Tables 1–3"},{"comment":"The training protocol is underspecified. The paper states that the LiDAR and camera streams are trained separately and then HAFA is trained for 10 epochs, but it does not report the total training schedule, learning rates, batch size, optimizer, number of agents during training, or the exact data split (train/val/test) for V2XSet-Noise and V2X-Real. Without these details, the experiments are not reproducible, especially since code and data are promised only for future release.","section":"§4.1 (Implementation Details)"}],"minor_comments":[{"comment":"The heading reads \"Abalation Studies\" and should be corrected to \"Ablation Studies.\"","section":"§4.4 heading"},{"comment":"The heading \"Time Sychronization\" contains a typo; it should be \"Time Synchronization.\"","section":"§3.5 heading"},{"comment":"The phrase \"can be particularly series\" should be \"can be particularly serious.\"","section":"§3.5, Calibration Error"},{"comment":"The paper uses AP@0.3/0.5 for V2X-Real but AP@0.5/0.7 for V2XSet-Noise and OPV2V. The choice of different IoU thresholds across datasets should be justified, as it makes cross-dataset comparisons confusing.","section":"§4.2"},{"comment":"In the supplementary, Eq. (2) uses a notation with fXc and fXl that is not standard; this is likely a typographical artifact. Please define all symbols consistently.","section":"Supplementary, Eq. (2)"},{"comment":"Several references are incomplete or inconsistent (e.g., [5] formats authors as \"Wen Yang; Zheng Gong; ...\" with semicolons). The authors should verify the reference list.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The Table 1 vs Table 4 discrepancy is the kind of issue that can be resolved by a careful revision (e.g., stating that Table 4 is on a reduced evaluation range or with noisy data), but if it is not, the paper's central claim would be unsupported. I would also encourage the editor to require the authors to release at least the V2XSet-Noise generation pipeline and evaluation protocol to enable independent verification, as the paper currently promises future release without providing any artifact."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"AgentAlign targets a real gap: cooperative perception under realistic sensor misalignment. The problem framing is solid, and the proposed CFAS (projecting LiDAR to depth maps, densifying, and computing depth gradients) plus HAFA (per-agent adaptive weighting between camera and LiDAR features) is a sensible combination that goes beyond HM-ViT's single-sensor-per-agent setup. The new V2XSet-Noise dataset with controllable noise types is a good idea and would be useful if released.\n\nBut the paper has a load-bearing internal inconsistency that undercuts the headline SOTA claim. Table 1 reports the full model at 62.6/60.4 AP on V2X-Real; Table 4 reports the same full model (Cam + Depth Agg + Depth Var + HAFA) at 49.9/38.4 on the same dataset, with no difference in protocol stated. Both numbers cannot be correct. This is not a minor reporting issue—it determines whether the method is actually SOTA or merely competitive with a weaker baseline. The ablation trend (39.0 to 49.9) suggests the modules help, but against what reference? A reader cannot tell which V2X-Real result is the real one.\n\nOther soft spots: no error bars anywhere; comparison tables omit the strongest cited multi-modal baselines (HM-ViT is absent from Table 1); and the simulated noise parameters in Section 3.5 are hand-picked, with the method engineered to counter exactly those corruption types, so the V2XSet-Noise evaluation has a circularity smell—though V2X-Real would provide independent grounding if its numbers were trustworthy. Code and data are not released, which also prevents independent verification.\n\nOn the plus side, the depth-variation representation is a nice touch, and the HAFA gating mechanism is cleanly described. The problem is important and the direction is worth pursuing. But as it stands, the experimental presentation cannot support the paper's conclusions.\n\nRecommendation: send to peer review, but with the expectation that the authors must resolve the Table 1/Table 4 discrepancy, add error bars and full baselines, and release the noise pipeline. A serious referee should not have to guess which number is real.","headline":"A promising idea for misalignment-robust cooperative perception, but the paper's own V2X-Real numbers contradict each other and the SOTA claim is not self-consistent.","tokens_in":17055,"tokens_out":3055,"would_cite":false,"duration_ms":26104,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AgentAlign: aligning camera and LiDAR features across noisy agents restores cooperative perception, posting state-of-the-art scores on V2X-Real and V2XSet-Noise.","keywords":["cooperative perception","V2X","sensor misalignment","cross-modality feature alignment","LiDAR-camera fusion","multi-agent perception","noise robustness","3D object detection"],"falsifier":"Collect field measurements of calibration drift, vibration, and synchronization errors from a deployed infrastructure camera/LiDAR suite over several months, build a test set whose noise matches those measured distributions, and compare AgentAlign against standard intermediate fusion on it; if the method's margin shrinks to within a few AP points, the claimed real-world robustness is not established.","tokens_in":15968,"feed_emoji":"🚗","tokens_out":7021,"duration_ms":60093,"temperature":0.7,"pith_summary":"Cooperative perception lets connected vehicles and roadside infrastructure share sensor data, but in the real world each agent's cameras and LiDARs are misaligned by calibration drift, vibration, time-sync lag, and distortion, which breaks naive feature fusion. AgentAlign claims to fix this by aligning every agent's camera and LiDAR features in a shared cross-modality space and then letting a learned attention map decide, per agent, how much to trust each modality. On the real-world V2X-Real dataset the method reports 62.6 AP@0.3 and 60.4 AP@0.5, ahead of the previous 58.6 and 55.3, and it leads the new simulated V2XSet-Noise benchmark as well. If the result holds, V2X systems could tolerate imperfect calibration and synchronization instead of requiring perfect sensor alignment.","feed_headline":"AgentAlign tops noisy V2X detection benchmarks with 62.6 AP","feed_subtitle":"Cross-modal alignment plus attention gating restores detection under calibration, vibration, and sync lag.","key_machinery":"The cross-modality feature alignment space (CFAS) starts with LiDAR points projected into the camera image; the sparse depth map is densified by a 7-by-7 max-pooling window, and depth gradients in four directions are concatenated to form a depth variation map. The heterogeneous agent feature alignment (HAFA) module concatenates camera features with that depth variation map and applies a convolution-plus-activation to produce an information-attention map that gates which modality each agent relies on. The aligned features are then lifted into bird's-eye view using depth-based scattering and fused across agents with a transformer encoder, the same structure used in prior intermediate-fusion cooperative perception.","core_discovery":"The paper's central claim is that multi-sensor misalignment in multi-agent perception can be absorbed at the feature level rather than corrected at the sensor level. A cross-modality feature alignment space (CFAS), built by projecting LiDAR into dense depth maps and adding depth-gradient variation, gives camera and LiDAR features a common representation. A heterogeneous agent feature alignment mechanism (HAFA) then learns an information-attention map that dynamically reweights the modalities for each agent, down-weighting whichever stream is corrupted by noise. Under multifactorial noise this restores cooperative detection accuracy, and the paper reports state-of-the-art results on V2X-Real and on its introduced V2XSet-Noise dataset.","pith_inferences":["The paper's noise magnitudes are synthetic and untested against field measurements, so the practical size of the gain may depend on how well those ranges match a specific deployment site.","If HAFA learns to downweight a modality under misalignment, the same attention could be exposed as an interpretable sensor-health signal for diagnostics.","The method's benefit is demonstrated on detection only; tracking and prediction pipelines that consume these features might not inherit the full gain."],"forward_implications":["If the reported gains hold, infrastructure installation tolerances can be relaxed: cameras and LiDARs no longer need frequent manual recalibration to keep cooperative perception accurate.","The V2XSet-Noise benchmark provides a controlled way to test how any cooperative perception method degrades under calibration error, vibration, time-sync lag, distortion, and systematic bias.","The CFAS depth-variation representation is a reusable building block for any camera-LiDAR fusion task with sparse depth, not only V2X.","HAFA's gating could be read as a reliability estimate, potentially useful for deciding which agents' data to transmit under bandwidth constraints."],"supporting_citations":[{"why":"The baseline fusion pipeline and dataset whose method AgentAlign builds on and compares against.","marker":"[29]"},{"why":"The real-world benchmark where the paper reports its main state-of-the-art result.","marker":"[27]"},{"why":"The simulated V2X dataset that V2XSet-Noise corrupts with multifactorial noise.","marker":"[30]"},{"why":"The PointPillar backbone used to extract LiDAR BEV features.","marker":"[9]"},{"why":"The lift-splat-shoot depth estimation and BEV projection used to lift camera features.","marker":"[19]"},{"why":"The source for the calibration error range used in the noise model.","marker":"[36]"}],"fun_headline_variants":["AgentAlign: Feature-level alignment for misaligned multi-agent sensors","AgentAlign: Cross-modal alignment restores noisy V2X perception","AgentAlign: Dynamic reweighting of sensor streams for robustness","AgentAlign: Tops noisy V2X benchmarks with adaptive alignment","AgentAlign: Harnessing inter-agent correlations under misalignment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim rests on the assumption that the synthetic noise models used to build V2XSet-Noise (sinusoidal wind vibration at 2 Hz, uniform calibration perturbations inside ±0.5 degrees and meters, 0.1 s synchronization delay, and systematic errors inside ±0.1) match the distribution of real sensor misalignment in deployed vehicle-to-everything systems.","fun_headline_variants_meta":{"raw":{"variants":["AgentAlign: Feature-level alignment for misaligned multi-agent sensors","AgentAlign: Cross-modal alignment restores noisy V2X perception","AgentAlign: Dynamic reweighting of sensor streams for robustness","AgentAlign: Tops noisy V2X benchmarks with adaptive alignment","AgentAlign: Harnessing inter-agent correlations under misalignment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000376,"raw_usage":{"total_tokens":2001,"prompt_tokens":939,"completion_tokens":1062,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":973}},"tokens_in":555,"tokens_out":1062,"duration_ms":10232,"temperature":1.0,"reasoning_tokens":973,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:58:17.677497+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect field measurements of calibration drift, vibration, and synchronization errors from a deployed infrastructure camera/LiDAR suite over several months, build a test set whose noise matches those measured distributions, and compare AgentAlign against standard intermediate fusion on it; if the method's margin shrinks to within a few AP points, the claimed real-world robustness is not established.","supporting_citations":[{"cited_title":"Pointpillars: Fast encoders for object detection from point clouds","cited_arxiv_id":null,"evidence_quote":"The PointPillar backbone used to extract LiDAR BEV features."},{"cited_title":"Lift, splat, shoot: En- coding images from arbitrary camera rigs by implic- itly unprojecting to 3d","cited_arxiv_id":null,"evidence_quote":"The lift-splat-shoot depth estimation and BEV projection used to lift camera features."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The source for the calibration error range used in the noise model."}],"review_version":1}