{"id":"d3390454-f402-41aa-9c87-9ac9c827ec35","arxiv_id":"2411.13340","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"WHALES is a large simulated V2X dataset with 8.4 cooperative agents per scene and a communication-aware scheduling baseline that outperforms prior schedulers.","lead":"Researchers built a new simulated highway and city dataset, called WHALES, with 8.4 connected vehicles and road sensors cooperating per scene, plus communication limits. It provides a testbed for deciding which vehicles should share sensor data when bandwidth is tight, a task older datasets do not support.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CAHS's Eq. (4) reward uses 'detections' that are never specified as model outputs; if they are ground-truth labels, Table VI's scheduling gains are oracle-driven and deployment-infeasible.","rationale":"The dataset contribution is plausibly valuable: WHALES offers a simulated V2X benchmark with 8.4 cooperative agents per scene, public code, and a clear pathway for scheduling experiments that prior datasets cannot support. The most defensible central claim is the dataset's scalability and scheduling benchmark, and that part does not obviously fail. However, the CAHS performance claim in the abstract is a headline result, and it rests on an underspecified quantity in Eq. (4): whether the detections used for scheduling are the deployed model's own outputs or the dataset's ground-truth labels. The paper never resolves this, and the two interpretations have very different validity. If ground truth is used, the scheduler has oracle information during inference and the reported gains over MASS are not deployable; if model outputs are used, the scheduler is entangled with the detector's errors in a way that must be analyzed and reported. This is exactly the reader's weakest assumption, and I agree it is load-bearing. The reader's CONDITIONAL verdict is appropriate: the concern is specific, testable, and does not invalidate the dataset itself, but it does block full acceptance of the scheduling claim until resolved. I also note the absence of error bars in Table VI, which would matter especially if the scheduling gains are small; however, the oracle ambiguity is the more decisive issue. The proposed concrete test -- checking the released code and rerunning the relevant table row with both variants -- would settle the concern directly.","tokens_in":10804,"tokens_out":9057,"duration_ms":102712,"concrete_test":"Inspect the released WHALES code at github.com/chensiweiTHU/WHALES to determine whether the CAHS implementation computes B^{t-1}_i in Eq. (4) from the model's own detections at the previous frame or from ground-truth annotations. Then rerun the Single-Agent CAHS row of Table VI at 50m with both variants and at least three seeds; if the model-output variant loses the mAP margin over MASS (e.g., the 62.5 vs. 58.8 gap disappears), the improvement claim is unsupported as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that CAHS's selection signal R_{t-1}^i in Eq. (4) is genuinely computable at inference from each candidate's own perception. The paper defines B_t^i as 'bounding boxes detected by agent i' (Eq. 1) but never states whether B^{t-1}_i is the output of the trained detector or ground-truth annotations from CARLA. Section III-D says all labels are generated from ground-truth outputs, and the scheduling protocol in Fig. 4 does not specify which detector candidates use to compute the benchmark. If B is ground truth, CAHS has privileged access to object locations during inference, so the reported improvements over MASS in Table VI (e.g., CAHS 62.5 vs. MASS 58.8 mAP at 50m under Closest-Agent training) would not be achievable in a real system. If B is model output, the result is legitimate but depends on the detector's quality and failure modes, and the paper must state this and report variance across seeds. Without this clarification or code inspection, the central claim that CAHS improves perception over existing schedulers is not fully supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces WHALES, a simulated V2X cooperative-perception dataset with an average of 8.4 cooperative agents per scene, 2.01 million 3D annotations, and explicit communication metadata. It benchmarks stand-alone and cooperative 3D detection, and evaluates several agent-scheduling policies under a 2 MB/frame communication cap. The authors also propose the Coverage-Aware Historical Scheduler (CAHS), which selects a collaborator based on the number of previous-frame detections intersecting the ego's past perception range. The empirical claim is that CAHS improves perception performance over existing schedulers such as MASS, and that the dataset enables scheduling research not possible with prior 2- to 4-agent benchmarks.","tokens_in":11104,"tokens_out":3864,"duration_ms":48861,"significance":"If the central claims hold, WHALES is a useful community resource: it is the first V2X dataset explicitly designed for communication-aware scheduling at non-trivial agent density, and it ships publicly with code, benchmarks, and metadata for scheduling studies. The paper's strengths include the dataset scale and density, the communication-budget framing, the coverage of both V2V and V2I agents, and the inclusion of scheduling baselines. The proposed CAHS is simple and interpretable. However, the empirical support for CAHS is currently undermined by an ambiguity about whether its scheduling signal is available at inference time, and by an inconsistency between the formal definition of Full Communication and what Table VI actually measures. These issues are load-bearing for the paper's main claim that CAHS improves over state-of-the-art scheduling methods.","major_comments":[{"comment":"The reward R_{t-1}^i in Eq. (4) is computed from B^{t-1}_i, the 'bounding boxes detected by agent i,' but the paper never states whether these boxes are the output of the trained detection model or ground-truth annotations. Section III-D says all labels are generated from ground-truth outputs, and the scheduling protocol in Fig. 4 does not specify which detector (if any) candidates use when computing their benchmark. If B^{t-1}_i is ground truth, CAHS uses privileged information at inference and the gains over MASS in Table VI (e.g., 62.5 vs. 58.8 mAP at 50 m under Closest-Agent training) are not achievable in deployment. If B^{t-1}_i is model output, the method is legitimate but depends on detector failure modes and should be analyzed; the paper must also report variance across seeds. This point must be resolved before the scheduling claim can be accepted.","section":"§IV-D, Eq. (4)"},{"comment":"Eq. (1) defines Full Communication as the union P^t_full = ∪_i B^t_i of per-agent detected bounding boxes, which is a detection-set operation, not a fusion method with confidence scores and a detection head. However, the Full Communication rows and columns in Table VI report mAP numbers that clearly come from the VoxelNeXt cooperative detector with feature-level fusion of all agents. These are two different objects: a union of detections is an upper bound on recall, not an mAP estimate, and feature-level fusion is not the union operation. The manuscript should either replace Eq. (1) with a definition matching the experiments or explain why the union operation is the appropriate formalization; as written, the claimed 'upper bound' interpretation of the Full Communication baseline is unsupported.","section":"§IV-D, Eq. (1) and Table VI"},{"comment":"The text says agent scheduling involves 'unicast communication for raw-level fusion,' but the scheduling experiments use VoxelNeXt as the backbone, which performs feature-level fusion. This is a contradiction that also affects the communication-budget claim: the 2 MB/frame cap is stated in §IV-A, but no analysis is given of whether raw point clouds, intermediate features, or object detections are what is transmitted, and whether the cap is respected under any of the compared methods. The authors should state which transmission modality is assumed for each baseline and for CAHS, and should justify the 2 MB cap relative to that modality.","section":"§IV-D, first paragraph"},{"comment":"The experimental protocol for training and inference under different scheduling policies is not fully specified. It is not clear what 'training under Full Communication' means when the inference policy is, say, Single Random, or how many agents are used during training when the row is 'Multiple Random.' The mAP numbers in Table VI also lack standard deviations or any significance test, which matters because several differences are small (e.g., 65.3 vs. 65.1 vs. 65.2 in the multi-agent rows). The authors should define the training-time data distribution for each row and report multiple seeds for the main CAHS-versus-MASS comparisons.","section":"Table VI and §IV-D"}],"minor_comments":[{"comment":"The acronym expansion is inconsistent: the abstract gives 'Wireless enhanced Autonomous vehicles with Large number of Engaged agents,' while the introduction gives 'Wireless enHanced Autonomous vehicles with Large number of Engaged agentS.' Please standardize.","section":"Abstract and Section I"},{"comment":"The abbreviation 'CAV' is used without a definition; it should be spelled out at first use (e.g., connected/cooperative autonomous vehicle).","section":"Section III-A"},{"comment":"Fig. 4 is described as visualizing four scheduling stages, but the printed figure only contains a flow diagram; the captions for the four stages should be self-contained, and the figure should indicate what information is transmitted at each stage (location, reward, or raw data).","section":"Section III-C and Fig. 4"},{"comment":"The statement that WHALES 'bridges the gap between simulated and real-world V2X challenges' is stronger than the evidence: the paper does not include real-world validation or cross-dataset transfer experiments. I recommend softening this claim or adding a short discussion of the limitations of simulated data for this purpose.","section":"Section I and Conclusions"},{"comment":"WHALES is listed as having only 3 object categories, whereas several prior datasets have 8-23 categories. The paper should comment on whether the three-class taxonomy limits the scheduling benchmark's usefulness for tasks such as pedestrian- or cyclist-focused cooperation, especially because Table IV shows large performance drops for these classes at 100 m.","section":"Table I"},{"comment":"The reference for MMDetection3D (ref. [35]) is incomplete (it has only 'M. Contributors' and an arXiv-style URL), and the VoxelNeXt reference (ref. [34]) lacks the proceedings or page numbers. Please complete these entries.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The dataset and code release are potentially valuable, and the idea of a scheduling-oriented V2X benchmark is timely. The main risk is not the dataset itself but the evaluation of CAHS: if the scheduling reward uses ground-truth detections, the Table VI improvements are oracle results; if it uses model outputs, the method needs a more careful analysis. The Full Communication formalization also needs to be aligned with the experiments. I would be willing to re-review a revised version that clarifies these points and adds error bars or a small ablative study that replaces the ground-truth signal with the model's own detections."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi — quick read on WHALES. The dataset is the real contribution. It gives the community the first simulated V2X benchmark with an average of 8.4 agents per scene, V2I plus V2V, NLOS conditions, and explicit communication metadata for scheduling. That combination is genuinely absent from OPV2V, V2X-Sim, DOLPHINS, and the real-world sets, and 2M annotated boxes with dense agent interactions is a real asset. If the release matches the paper, it deserves to become a standard scheduling benchmark.\n\nThe CAHS scheduler is a simple coverage heuristic. As a baseline it is fine, and the handshake protocol is clearly described. But the experimental support for 'improves over SOTA' is weaker than the abstract implies. The biggest unresolved point is Eq. (4): R uses 'detections' from agent i at t-1, and the paper never says whether those are the trained model's outputs or CARLA ground-truth boxes. If they are ground truth, CAHS is using oracle information during inference and Table VI's gains are not deployable. If they are model outputs, fine, but then the paper has to say so and report variance across seeds, because the scheduler is entangled with detector failure modes. That ambiguity is load-bearing.\n\nTwo smaller issues. First, Eq. (1) defines Full Communication as a union of bounding boxes, but Table VI evaluates it with VoxelNeXt feature-level fusion; the mismatch should be reconciled. Second, there are no error bars anywhere, and some numbers in Table VI look like artifacts (e.g., 10.8 mAP at 100 m for Full Communication under Closest-Agent training). And the abstract's 'bridges simulated and real-world V2X challenges' is an overclaim; the paper provides no evidence for that.\n\nNone of this kills the dataset contribution. The scheduling ambiguity is fixable with a clear statement plus code release, and the error bars are standard revision material. The paper is coherent, the dataset design is thoughtful, and the citation pattern is fine. This deserves serious peer review, but the CAHS claims should not be accepted as-is. I'd bring it to reading group to debate the oracle question, and I'd cite the dataset if I work in this area.","headline":"WHALES is a genuinely useful dataset contribution; the scheduling experiments are real but the CAHS comparison needs clarification before its claims are taken at face value.","tokens_in":11580,"tokens_out":2390,"would_cite":true,"duration_ms":25140,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"WHALES is the first V2X dataset built for communication-aware agent scheduling, and its history-based scheduler improves detection over prior baselines.","keywords":["cooperative perception","V2X","agent scheduling","autonomous driving","3D object detection","dataset","communication constraints","CARLA simulation"],"falsifier":"Inspect the released code to see whether the set $B_{t-1}^i$ in Eq. (4) holds the detector's own outputs or ground-truth annotations, then rerun the single-agent scheduling experiments from Table VI using only the detector's outputs from the previous frame. If CAHS no longer beats the Closest Agent baseline, the reported improvement came from privileged information.","tokens_in":10641,"feed_emoji":"🚗","tokens_out":7146,"duration_ms":69267,"temperature":0.7,"pith_summary":"This paper tries to establish that the bottleneck in cooperative driving research is not fusion architectures but datasets too sparse to make agent scheduling matter. WHALES, a simulated V2X dataset with an average of 8.4 agents per scene and 2.01 million annotated 3D objects, is offered as the first resource that makes communication-aware scheduling a real benchmark task, complete with communication metadata and a bandwidth cap. Alongside it, the paper proposes CAHS, a scheduler that chooses collaborators by historical viewpoint overlap, and reports that it improves detection accuracy over existing schedulers such as MASS. A sympathetic reader would care because, if the claim holds, the field gains a standard testbed for deciding which agents to talk to under bandwidth limits, a problem every real V2X system must solve.","feed_headline":"New V2X dataset brings 8.4-agent scenes to cooperative driving","feed_subtitle":"WHALES adds 2.01M annotated 3D boxes and a coverage-aware scheduler that beats prior baselines.","key_machinery":"The load-bearing object is the dataset's agent density plus the CAHS handshake rule. In every frame each of the N agents is cyclically treated as the ego, yielding N ego-centric samples per frame, so scheduling decisions can be evaluated from every viewpoint. CAHS works in four steps: the ego broadcasts its location, each candidate computes its historical coverage reward $R_{t-1}^i = \\sum_{b \\in B_{t-1}^i} \\mathbb{I}(b \\cap R_{t-1}^E \\neq \\emptyset)$ — the number of its detections at time $t-1$ that intersect the ego's past perception range — and returns it, and the ego picks the argmax. This makes scheduling a parameter-free, one-step lookup rather than a learned policy, which is precisely what lets the paper isolate the value of dense-agent benchmarking.","core_discovery":"On its own terms, the paper claims that cooperative perception has been held back by datasets with only two to four agents per scene, because with so few collaborators agent selection is trivial and communication constraints cannot be studied. WHALES is presented as the first large-scale V2X dataset explicitly designed for communication-aware agent scheduling and scalable cooperative perception: it averages 8.4 cooperative agents per scene, contains 70K RGB images, 17K LiDAR frames, and 2.01 million annotated 3D objects, and includes communication metadata with a 2MB-per-frame budget. To go with it, the paper proposes CAHS, a scheduling baseline in which each candidate agent reports how many of its previous-frame detections overlapped the ego's past perception range, and the ego selects the agent with the largest overlap count. The paper reports that CAHS outperforms the learning-based MASS scheduler and often matches or beats full communication while transmitting far less data.","pith_inferences":["If the scheduling reward can be computed from model outputs rather than ground truth, the historical-overlap idea extends naturally to online settings where agents move and detections arrive asynchronously, and to camera-only agents that share the same ego-centric overlap criterion.","The cyclic ego assignment multiplies the effective sample count by N, so WHALES could also serve as a pretraining source for cooperative encoders before fine-tuning on real V2X data; the paper does not test this transfer.","The nonlinear peak at four agents suggests an information-theoretic framing: each added agent contributes marginal coverage but also noise and bandwidth cost, so an obvious next benchmark is a learned scheduler that predicts marginal gain per candidate rather than ranking by a fixed heuristic."],"forward_implications":["Agent scheduling becomes a first-class benchmark task: with 8.4 agents per scene, the number of possible cooperation sets is large enough that the selection strategy visibly changes mAP.","CAHS shows that a purely historical, communication-free heuristic can beat a learned scheduler such as MASS, so lightweight scheduling is viable under tight bandwidth budgets.","The four-agent peak in the ablation implies that indiscriminate full communication wastes bandwidth and can hurt accuracy; a scheduler should stop adding agents once marginal coverage gain turns negative.","The 50m and 100m detection benchmarks give future cooperative perception methods a common protocol to report against, including the 2MB-per-frame communication cap."],"supporting_citations":[{"why":"provides the OPV2V simulated V2V dataset that WHALES contrasts with, arguing that V2V-only and sparse-agent settings cannot support scheduling research.","marker":"[13]"},{"why":"represents DAIR-V2X, the real-world V2X dataset with only 2 agents per scene that motivates WHALES' denser design.","marker":"[11]"},{"why":"defines the MASS learning-based scheduler that CAHS is claimed to outperform in the Table VI evaluation.","marker":"[27]"},{"why":"supplies the VoxelNeXt backbone used as the detection model for all scheduling experiments.","marker":"[34]"},{"why":"provides the nuScenes annotation protocol and detection metrics (mAP, NDS) that WHALES adopts.","marker":"[16]"},{"why":"is the urban driving simulator on which WHALES' scene generation is built.","marker":"[25]"},{"why":"provides the F-Cooper feature-level fusion baseline used in the cooperative detection benchmark.","marker":"[36]"},{"why":"supplies the reinforcement-learning expert whose behaviors generate the agent trajectories in the dataset.","marker":"[29]"}],"fun_headline_variants":["WHALES: first V2X dataset for communication-aware scheduling","8.4 agents per scene: WHALES dataset pushes V2X cooperation","New V2X dataset includes 2.01M 3D boxes and a coverage-aware scheduler","CAHS scheduler beats baselines on new multi-agent V2X dataset","WHALES dataset: 8.4 agents, 2M objects, better scheduling"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The CAHS scheduler's advantage depends on the unstated assumption that the detections it uses to score agents at time t-1 are the model's own outputs; if the implementation instead uses ground-truth annotations, the reported scheduling gains would not transfer to real driving.","fun_headline_variants_meta":{"raw":{"variants":["WHALES: first V2X dataset for communication-aware scheduling","8.4 agents per scene: WHALES dataset pushes V2X cooperation","New V2X dataset includes 2.01M 3D boxes and a coverage-aware scheduler","CAHS scheduler beats baselines on new multi-agent V2X dataset","WHALES dataset: 8.4 agents, 2M objects, better scheduling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001088,"raw_usage":{"total_tokens":4553,"prompt_tokens":961,"completion_tokens":3592,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":3488}},"tokens_in":577,"tokens_out":3592,"duration_ms":25278,"temperature":1.0,"reasoning_tokens":3488,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:32:28.935307+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the released code to see whether the set $B_{t-1}^i$ in Eq. (4) holds the detector's own outputs or ground-truth annotations, then rerun the single-agent scheduling experiments from Table VI using only the detector's outputs from the previous frame. If CAHS no longer beats the Closest Agent baseline, the reported improvement came from privileged information.","supporting_citations":[{"cited_title":"Carla: An open urban driving simulator,","cited_arxiv_id":null,"evidence_quote":"is the urban driving simulator on which WHALES' scene generation is built."},{"cited_title":"F-cooper: Feature based cooperative perception for autonomous vehicle edge computing system using 3d point clouds,","cited_arxiv_id":null,"evidence_quote":"provides the F-Cooper feature-level fusion baseline used in the cooperative detection benchmark."},{"cited_title":"End-to- end urban driving by imitating a reinforcement learning coach,","cited_arxiv_id":null,"evidence_quote":"supplies the reinforcement-learning expert whose behaviors generate the agent trajectories in the dataset."},{"cited_title":"Mass: Mobility-aware sensor scheduling of cooperative perception for connected automated driving,","cited_arxiv_id":null,"evidence_quote":"defines the MASS learning-based scheduler that CAHS is claimed to outperform in the Table VI evaluation."},{"cited_title":"V oxelnext: Fully sparse voxelnet for 3d object detection and tracking,","cited_arxiv_id":null,"evidence_quote":"supplies the VoxelNeXt backbone used as the detection model for all scheduling experiments."},{"cited_title":"nuscenes: A multimodal dataset for autonomous driving,","cited_arxiv_id":null,"evidence_quote":"provides the nuScenes annotation protocol and detection metrics (mAP, NDS) that WHALES adopts."},{"cited_title":"Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to- vehicle communication,","cited_arxiv_id":null,"evidence_quote":"provides the OPV2V simulated V2V dataset that WHALES contrasts with, arguing that V2V-only and sparse-agent settings cannot support scheduling research."},{"cited_title":"Dair-v2x: A large-scale dataset for vehicle- infrastructure cooperative 3d object detection,","cited_arxiv_id":null,"evidence_quote":"represents DAIR-V2X, the real-world V2X dataset with only 2 agents per scene that motivates WHALES' denser design."}],"review_version":1}