{"id":"385078e4-0972-4a2d-9f91-d7760cf0e935","arxiv_id":"2501.15045","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new unsupervised driver attention predictor with knowledge embedding, uncertainty mining, and RoboMixup augmentation matches or beats several supervised baselines on clean and corrupted driving benchmarks.","lead":"An unsupervised system predicts where a self-driving car should focus by combining saliency models trained on everyday photos with traffic-object masks, needing no human driving-attention labels. The same paper adds a corrupted-weather benchmark and a mixup-style augmentation that the authors report cuts corruption-related degradation by roughly half on one of three test sets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim rests on an untested assumption that KEB-enhanced natural-scene pseudo-labels are adequate targets for driver attention; the paper never measures pseudo-label agreement with human gaze, especially for non-central/crash frames.","rationale":"The reader's weakest assumption identifies the same load-bearing point: the learning signal itself. The paper's abstract claims equivalence with fully supervised state-of-the-art on three public datasets and superior robustness, but the only supervision used in the target domain is pseudo-labels from free-viewing natural-scene saliency models, refined by a category-level boosting module. If those pseudo-labels do not approximate human driver attention for the objects and situations that matter, no amount of uncertainty mining, knowledge embedding, or augmentation can make the central claim true. The manuscript provides no direct evidence that the KEB-enhanced targets correlate with ground-truth gaze: all reported metrics are for the final trained model, which could be learning dataset-level regularities (central bias, road layout) rather than true attention. Table 7 demonstrates a large domain gap between natural and driving attention, which makes the sufficiency of the pseudo-label signal a genuine risk. The proposed test would settle the concern by measuring the targets themselves, separately on the non-central frames where safety-critical attention is most needed. This does not change the reader's verdict: the paper is promising and the clean-data results are competitive, but the conditional acceptance remains appropriate, with the requested revision being a direct validation of the pseudo-label targets and a clearer limitation statement. Secondary concerns about the CycleGAN filtering of DriverAttention-C and the abstract's selective reporting of Relative mCD on BDD-A-C are real but less fundamental than the adequacy of the unsupervised supervision signal.","tokens_in":21913,"tokens_out":13212,"duration_ms":132156,"concrete_test":"On the BDD-A and DADA-2000 test sets, compute KLD/CC between (a) raw ML-Net/UNISAL pseudo-labels and (b) their KEB-enhanced versions, against the human ground-truth maps, for all frames and for the high-central-bias-deviation subset (delta >= 3.0 using the paper's threshold protocol). If the KEB-enhanced pseudo-labels are not significantly closer to human gaze than raw pseudo-labels in the high-delta subset, the unsupervised targets are not adequate for safety-critical attention and the headline claim should be restricted to central-bias-dominated scenes or withdrawn.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's only supervision on driving data is pseudo-labels from natural-scene saliency models (ML-Net, UNISAL; Section 3.1). KEB (Section 3.2, Eq. 5) multiplies each pseudo-label by (M + alpha), where M is the Mask R-CNN mask for categories mined via Eq. (3). This suppresses all non-mined regions by a factor alpha = 0.3. The adequacy of the resulting targets is never checked against human attention: no experiment reports KLD/CC between the KEB pseudo-labels and ground-truth gaze maps. Table 7 shows the natural-to-driving domain gap is large (SALICON-trained APB achieves KLD 2.109 / CC 0.287 on BDD-A versus 1.036 / 0.657 when trained on BDD-A). If the boosted categories miss crash-relevant objects, or if the suppression of non-mined regions removes true attention mass in non-central and corner cases, the model is trained to predict the wrong distribution and the DADA-2000 and robustness results are not evidence for the claimed unsupervised capability. Table 6 shows only internal consistency of the training pipeline, not that the pseudo-label targets are close to true driver attention.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an unsupervised driver-attention prediction framework that trains only on pseudo-labels produced by natural-scene saliency models, without using human gaze labels from driving datasets. Three components are introduced: an Uncertainty Mining Branch (UMB) that fuses multiple pseudo-label sources with multi-scale features, a Knowledge Embedding Block (KEB) that uses Mask R-CNN to boost attention on traffic-relevant categories mined from pseudo-label statistics, and a RoboMixup data augmentation that combines soft-attention Mixup, a dynamic augmentation strategy, and random-crop regularization. The authors also introduce a corruption benchmark, DriverAttention-C, generated from BDD-A, DR(eye)VE, and DADA-2000, and report that the method matches or surpasses fully supervised state-of-the-art on clean data and improves robustness to corruption and central bias. The central claim is that driver attention can be predicted without traffic gaze labels while being more robust than several supervised baselines.","tokens_in":22213,"tokens_out":5769,"duration_ms":50964,"significance":"If the central claim holds, the paper would make a useful contribution: it relaxes the annotation bottleneck for driver-attention prediction and provides a new corruption benchmark (DriverAttention-C) with 115k frames, alongside an augmentation method (RoboMixup) that reduces central bias. The evaluation is extensive, covering three public datasets, a new benchmark, multiple corruption types, and several ablations. The availability of code and data is a strength. However, the significance is conditional on the adequacy of the pseudo-label learning signal, which is never directly validated against human gaze, and on a more careful reporting of the robustness claims, which are currently overstated relative to the full benchmark results.","major_comments":[{"comment":"The unsupervised learning signal is never validated against human gaze. The model trains exclusively on KEB-enhanced pseudo-labels from natural-scene saliency models, and Eq. (5) suppresses all non-mined regions by a factor α=0.3. Table 7 shows that the natural-to-driving domain gap is large (SALICON-trained APB yields KLD 2.109 / CC 0.287 on BDD-A versus 1.036 / 0.657 for BDD-A-trained APB). No experiment reports KLD/CC between the KEB pseudo-labels and the human gaze ground truth on held-out frames. Without such a check, the reported clean and robustness results cannot distinguish learning true driver attention from learning a shifted saliency prior. Please report pseudo-label-to-gaze agreement on each dataset, or an upper-bound ablation that replaces pseudo-labels with ground-truth labels.","section":"§3.1-§3.2, Eq. (5)"},{"comment":"The claimed reductions of 58.8% and 52.8% in relative corruption degradation are specific to BDD-A-C and are computed relative to the UAP baseline (Table 4: Relative mCD 0.412 KLD / 0.472 CC). On DR(eye)VE-C and DADA-2000-C, RUAP does not achieve the best Relative mCD; e.g., DADA-2000-C KLD: DADA 0.044 vs RUAP 1.137, and DR(eye)VE-C KLD: Multi-Branch 0.505 vs RUAP 0.652. The abstract and Section 5.3 should state the dataset and baseline for these headline numbers. The text's explanation that low Relative mCD by other methods reflects 'stable but poor performance' is post hoc; a formal criterion (e.g., Pareto comparison over clean KLD and degradation) is needed before discounting those competitors.","section":"Abstract; Tables 3 and 4"},{"comment":"The DriverAttention-C construction is under-specified and the reported counts are inconsistent. Section 4.1 reports a total of 115,332 frames and per-subset test counts 6,817 + 7,497 + 4,908 = 19,222, while Section 4.2 ends with 'filter out images with artificial stripes caused by inherent issues of the Cycle-GAN, resulting in 38,444 images.' Please clarify what 38,444 refers to, how many images were removed per corruption type, and whether filtering occurred before or after the train/test split. Because the filtering criterion is based on visual artifacts, it may introduce selection bias; the criterion and its effect on benchmark difficulty should be documented.","section":"§4.1-§4.2"},{"comment":"The uncertainty loss uses a scalar u_n defined as the spatial mean of the pixel-level uncertainty map U_n, yet the UMB is motivated as producing pixel-level uncertainty maps. This scalarization discards spatial information and conflicts with the design narrative. Either adopt per-pixel uncertainty in the loss (e.g., u_i) or justify the scalar aggregation. Additionally, the '≈' step in Eq. (9) needs a derivation; as written it is not evident that the approximation from the negative log-likelihood to LCE/u_n^2 + log(u_n) holds for arbitrary S and Y_n.","section":"§3.3-§3.4, Eqs. (8)-(10)"},{"comment":"No error bars or significance tests are reported for any fully-supervised baseline, while Ours is shown with ±. Because the claim of equivalence or superiority relies on KLD differences as small as 0.03-0.1, the comparison is not fully supported without variance estimates for the baselines. Please report variance over at least three runs of the main baselines, or a paired significance test, for the central clean-benchmark and robustness tables.","section":"§5.3, Tables 1-3"}],"minor_comments":[{"comment":"Typo: 'SALION' should be 'SALICON'.","section":"Table 7"},{"comment":"Typo: 'BDDA-C' appears in the paragraph after Table 3; it should be 'BDD-A-C'.","section":"§5.3"},{"comment":"Typo: 'DriverAttetnion-C' should be 'DriverAttention-C'.","section":"§4.1"},{"comment":"The notation 'µP · Mij / Mij' is confusing; it should be written as the mean of µP over the pixels of mask Mij.","section":"§3.2, Eq. (2)"},{"comment":"The text says ML-Net is selected as the baseline for mCD, but Table 4 uses UAP as the reference. Please state explicitly that different references are used in Tables 3 and 4.","section":"§5.1"},{"comment":"The phrase 'three public datasets and the proposed robustness benchmark' counts four items; consider 'three public datasets and the proposed corruption benchmark'.","section":"Abstract"},{"comment":"It is surprising that training on BDD-A pseudo-labels yields better DADA-2000 test KLD (1.677) than training on DADA-2000 pseudo-labels (1.711). A brief explanation of this cross-dataset result would help the reader assess the pseudo-label quality.","section":"Table 2"},{"comment":"The sentence 'We focus on the most frequent categories, as the remaining categories are likely to be misclassified due to their small proportion' is ambiguous; it suggests rare categories are noisy, but the mining then selects low-attention frequent categories. Please clarify.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely a solid extension of the authors' ICCV work, and the new benchmark and augmentation are potentially useful. However, the adequacy of the pseudo-label supervision is the crux: without a direct pseudo-label-to-gaze agreement study, the unsupervised claim is not fully supported. The abstract overstates the robustness result by omitting that 58.8%/52.8% are relative to the UAP baseline on BDD-A-C only. The journal should require the pseudo-label validation experiment and a rewriting of the robustness claims before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take: the paper is a real extension of the ICCV 2023 work. New pieces are RoboMixup and DriverAttention-C, a 115k-frame corruption benchmark built from BDD-A, DR(eye)VE, and DADA-2000. The dataset is the kind of resource people will use; the augmentation is plausible and ablated. The clean-set results are competitive, and the method does beat strong supervised baselines on BDD-A-C, the headline table. The authors are honest in the text that on DR(eye)VE-C and DADA-2000-C the relative mCD is not best; the abstract still makes a sweeping claim, which is the first thing a referee should fix.\n\nThe main scientific soft spot is the one the stress-test flags: the learning target is KEB-enhanced pseudo-labels from natural-scene saliency models, and the paper never measures how close those targets are to actual driver gaze. Table 7 shows the domain gap is large (SALICON-trained APB gets KLD 2.109 vs 1.036 on BDD-A). It is possible the boosted categories (pedestrian, stop sign, etc.) capture what drivers look at, but that is an assumption, and it is load-bearing. The robustness numbers on DADA-2000-C, a crash dataset, depend on the pseudo-labels containing crash-relevant objects; the paper doesn't check. This is not fatal, but it should be an explicit experiment: report KLD/CC between pseudo-labels (with and without KEB) and human gaze on validation splits.\n\nSmaller issues: no error bars on baseline tables (Tables 3, 4, 5), the closest unsupervised baseline [44] is not compared, and the CycleGAN weather generation includes an undisclosed filtering step that could bias the benchmark. These are fixable.\n\nOn circularity: the reader's concern is misplaced. Training uses pseudo-labels, evaluation uses human ground truth not used in training; the KEB category mining is self-referential but the final evaluation is external. Not circular, just under-validated.\n\nBottom line: deserves a serious referee. The new dataset alone justifies review, and the method is coherent and well-ablated. If I were the editor I'd send it out with a request to add pseudo-label-to-gaze validation and to temper the abstract. Would I cite it? The benchmark, yes; the method's claim, only after validation.","headline":"A useful robustness benchmark and a plausible unsupervised attention model, but the headline robustness claim only holds on one subset and the pseudo-label targets are never validated against human gaze.","tokens_in":22729,"tokens_out":1766,"would_cite":true,"duration_ms":16230,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Unsupervised driver attention prediction, built from refined pseudo-labels of natural-scene saliency models plus robust augmentation, matches or beats fully supervised methods on three benchmarks and is more robust to corruption and…","keywords":["driver attention prediction","unsupervised learning","pseudo-labels","saliency prediction","corruption robustness","central bias","data augmentation","autonomous driving"],"falsifier":"Run the method on DADA-2000 crash frames where the recorded driver fixation lands on an object that is neither salient in natural-scene models nor a member of the boosted COCO categories, such as a fallen cyclist, an animal on the roadway, or scattered debris; if the predicted attention map shows no peak at the human fixation on such frames and per-frame KLD and CC drop below the supervised baselines for those frames, the pseudo-label adequacy assumption fails. A direct quantitative check: compute the mean pseudo-label attention over ground-truth fixation pixels; the claim predicts that the boosted categories account for most of this mass, so measuring the residual mass on unboosted categories across the three datasets would settle the adequacy question.","tokens_in":21698,"feed_emoji":"🚗","tokens_out":14027,"duration_ms":102483,"temperature":0.7,"pith_summary":"Driver attention prediction normally requires expensive human gaze labels collected in vehicles or laboratories. This paper argues that those labels can be replaced entirely by pseudo-labels generated from off-the-shelf saliency models trained on natural images, provided the pseudo-labels are refined in three ways: mining uncertainty across several sources, embedding traffic-object knowledge from an instance segmenter, and training with a robust augmentation called RoboMixup. On BDD-A, DR(eye)VE, and DADA-2000, the resulting unsupervised model matches or beats fully supervised baselines, and on the new DriverAttention-C corruption benchmark it reduces relative corruption degradation by 58.8% and 52.8% while improving central-bias robustness by 12.4% and 11.4% in KLD and CC. If correct, this removes a major data bottleneck for self-driving attention and makes the predictor more reliable under adverse conditions.","feed_headline":"Driver attention, no gaze labels: matches or beats supervised","feed_subtitle":"Unsupervised model matches or beats supervised baselines and cuts corruption degradation by 58.8%.","key_machinery":"The load-bearing mechanism is the three-stage pseudo-label refinement pipeline. The Uncertainty Mining Branch (UMB) applies non-local self-attention across multi-scale encoder features and N pseudo-label maps to emit per-pixel uncertainty maps, and couples the predicted attention map to each pseudo-label through an uncertainty-weighted KL-divergence loss, where the predicted log variance weighs each pseudo-label's contribution. The Knowledge Embedding Block (KEB) multiplies each pseudo-label by a binary traffic-object mask produced by a frozen Mask R-CNN, with object categories selected automatically by picking frequent COCO classes whose mean pseudo-label attention is below a threshold, on the reasoning that natural-scene saliency models under-represent traffic objects. RoboMixup replaces the global scalar in Mixup with a pixel-level attention ratio between two samples, selects hard samples per batch by KL divergence to the average attention map, and combines random cropping with RegMixup's auxiliary-loss formulation to counter central bias.","core_discovery":"The central claim is that the supervision signal for driver attention need not come from human fixations at all. The paper shows that pseudo-labels produced by two natural-scene saliency models, ML-Net and UNISAL, can be transformed into training targets that rival ground-truth attention maps by combining three mechanisms: an Uncertainty Mining Branch that models disagreements among pseudo-labels as per-pixel uncertainty and optimizes a KL-divergence-based uncertainty loss, a Knowledge Embedding Block that boosts traffic-relevant object classes using a frozen Mask R-CNN segmenter, and RoboMixup, which blends images with attention-weighted pixel mixing, dynamic hard-sample selection, and random-crop regularization. The reported outcome is that the unsupervised model achieves KLD of 1.099, 1.901, and 1.677 and CC of 0.640, 0.510, and 0.488 on BDD-A, DR(eye)VE, and DADA-2000 respectively, comparable to or better than supervised state-of-the-art methods, and that on the six-corruption DriverAttention-C benchmark the RoboMixup variant lowers relative corruption degradation to 0.755 and 0.846 in KLD and CC on BDD-A-C, the best among all compared methods.","pith_inferences":["The recipe of multi-source pseudo-labels plus uncertainty mining, knowledge embedding, and robust mixup is domain-agnostic and could be applied to other sparse-label prediction tasks such as pedestrian intent or hazard detection, where gaze labels are equally scarce.","The knowledge-mining rule assumes that natural-scene saliency models uniformly under-respond to traffic objects; a natural test is whether boosting categories with low mean attention helps only because of the specific class distribution of MS-COCO, and whether the rule needs re-mining for traffic cultures with different object frequencies.","Because the robustness gain from dynamic augmentation, which selects samples with high KL divergence to the average map, is not disentangled from soft-attention mixing, an ablation of the selection rule alone would reveal which part of RoboMixup drives the 58.8% relative degradation reduction.","A reader should not expect the same unsupervised gains on novel crash-relevant objects such as debris or animals that are absent from both the saliency models' training data and the MS-COCO taxonomy; for such objects the pseudo-label signal may be missing entirely."],"forward_implications":["If the claim is correct, self-driving attention prediction no longer requires collecting human gaze data, which is expensive and prone to inconsistencies such as averaging eight drivers or relying on in-lab simulation.","The robustness numbers imply that attention prediction can tolerate common camera corruptions such as noise, blur, JPEG compression, fog, and snow without being trained on corrupted images, improving safety margins under adverse weather.","The central-bias results imply the model keeps attending to off-center objects such as pedestrians, cyclists, and fallen persons that are exactly the ones in accident scenarios, rather than collapsing to a road-center prior.","The DriverAttention-C benchmark provides a standardized way to measure robustness of attention predictors across noise, blur, digital, and weather corruptions.","The exploratory explainable-decision experiment suggests that predicted attention maps can be plugged into downstream driving-decision models to yield interpretable outputs."],"supporting_citations":[{"why":"Supplies the base uncertainty-mining and knowledge-embedding framework that this paper extends with RoboMixup and the corruption benchmark.","marker":"[19]"},{"why":"ML-Net, one of the two natural-scene saliency models whose outputs serve as pseudo-labels.","marker":"[32]"},{"why":"UNISAL, the second pseudo-label source; the combination ML-Net plus UNISAL yields the best results in the ablation.","marker":"[22]"},{"why":"BDD-A driving attention dataset used for evaluation and as the source of images for pseudo-label training.","marker":"[8]"},{"why":"DR(eye)VE in-car attention dataset used for evaluation.","marker":"[9]"},{"why":"DADA-2000 crash-scenario attention dataset used for evaluation and for cross-dataset pseudo-label transfer tests.","marker":"[10]"},{"why":"Supplies the corruption taxonomy and generation toolbox used to create the noise, blur, and JPEG corruptions in DriverAttention-C.","marker":"[13]"},{"why":"CycleGAN used to synthesize fog and snow weather corruptions for the benchmark.","marker":"[18]"},{"why":"Mask R-CNN, frozen and off-the-shelf, segments traffic-relevant objects for the knowledge embedding block; trained on MS-COCO.","marker":"[16]"},{"why":"RegMixup provides the auxiliary-loss formulation that RoboMixup adopts when combining mixup with random crop regularization.","marker":"[40]"}],"fun_headline_variants":["Unsupervised attention predicts driver focus as well as supervised","Gaze-free driver attention model rivals supervised accuracy","No human labels: unsupervised attention matches supervised","Driver attention with zero labels, robust to corruption","Unsupervised attention matches supervised, cuts corruption loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the enhanced pseudo-labels, natural-scene saliency outputs boosted on a mined set of common traffic-object categories, approximate what real drivers actually look at in safety-critical scenes; if those saliency models miss crash-relevant objects outside the boosted categories, the unsupervised training signal is blind to the very things that matter.","fun_headline_variants_meta":{"raw":{"variants":["Unsupervised attention predicts driver focus as well as supervised","Gaze-free driver attention model rivals supervised accuracy","No human labels: unsupervised attention matches supervised","Driver attention with zero labels, robust to corruption","Unsupervised attention matches supervised, cuts corruption loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000769,"raw_usage":{"total_tokens":3472,"prompt_tokens":1078,"completion_tokens":2394,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":694,"completion_tokens_details":{"reasoning_tokens":2321}},"tokens_in":694,"tokens_out":2394,"duration_ms":16046,"temperature":1.0,"reasoning_tokens":2321,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:41:09.356976+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the method on DADA-2000 crash frames where the recorded driver fixation lands on an object that is neither salient in natural-scene models nor a member of the boosted COCO categories, such as a fallen cyclist, an animal on the roadway, or scattered debris; if the predicted attention map shows no peak at the human fixation on such frames and per-frame KLD and CC drop below the supervised baselines for those frames, the pseudo-label adequacy assumption fails. A direct quantitative check: compute the mean pseudo-label attention over ground-truth fixation pixels; the claim predicts that the boosted categories account for most of this mass, so measuring the residual mass on unboosted categories across the three datasets would settle the adequacy question.","supporting_citations":[{"cited_title":"Unsupervised self-driving attention prediction via uncertainty mining and knowledge embedding","cited_arxiv_id":null,"evidence_quote":"Supplies the base uncertainty-mining and knowledge-embedding framework that this paper extends with RoboMixup and the corruption benchmark."},{"cited_title":"A deep multi-level network for saliency prediction","cited_arxiv_id":null,"evidence_quote":"ML-Net, one of the two natural-scene saliency models whose outputs serve as pseudo-labels."},{"cited_title":"Unified image and video saliency modeling","cited_arxiv_id":null,"evidence_quote":"UNISAL, the second pseudo-label source; the combination ML-Net plus UNISAL yields the best results in the ablation."},{"cited_title":"Predicting driver attention in critical situations","cited_arxiv_id":null,"evidence_quote":"BDD-A driving attention dataset used for evaluation and as the source of images for pseudo-label training."},{"cited_title":"Dr (eye) ve: a dataset for attention-based tasks with applications to autonomous and assisted driving","cited_arxiv_id":null,"evidence_quote":"DR(eye)VE in-car attention dataset used for evaluation."},{"cited_title":"Dada: Driver attention prediction in driving accident scenarios","cited_arxiv_id":null,"evidence_quote":"DADA-2000 crash-scenario attention dataset used for evaluation and for cross-dataset pseudo-label transfer tests."},{"cited_title":"Benchmarking neural network robustness to common corruptions and perturbations","cited_arxiv_id":null,"evidence_quote":"Supplies the corruption taxonomy and generation toolbox used to create the noise, blur, and JPEG corruptions in DriverAttention-C."},{"cited_title":"Unpaired image-to-image translation using cycle- consistent adversarial networks","cited_arxiv_id":null,"evidence_quote":"CycleGAN used to synthesize fog and snow weather corruptions for the benchmark."},{"cited_title":"Mask r-cnn","cited_arxiv_id":null,"evidence_quote":"Mask R-CNN, frozen and off-the-shelf, segments traffic-relevant objects for the knowledge embedding block; trained on MS-COCO."},{"cited_title":"Using mixup as a regularizer can surprisingly improve accuracy & out-of-distribution ro- bustness","cited_arxiv_id":null,"evidence_quote":"RegMixup provides the auxiliary-loss formulation that RoboMixup adopts when combining mixup with random crop regularization."}],"review_version":1}