{"id":"fb1b8329-8c78-48d6-940c-3145e29724ce","arxiv_id":"2604.20027","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Fine-tuning ViT-B/16 attention on human saliency maps induces three human-like attention biases and improves alignment on five metrics with no loss in classification accuracy on ImageNet, ImageNet-C, and ObjectNet.","lead":"This paper fine-tunes a Vision Transformer's self-attention weights on human eye-fixation saliency maps and shows the resulting model matches human attention patterns more closely on multiple metrics while preserving its original image-classification accuracy. A smart generalist might read it to understand whether biologically inspired priors can be added to powerful AI models without the usual performance trade-off.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Bayesian parity analysis for 'no cost' lacks reported sensitivity checks on priors and ROPE","rationale":"Reader's weakest assumption concerns isolation of semantic signal via shuffled saliency maps. That assumption governs whether alignment was successfully induced, but the headline claim under test is the absence of performance cost, whose sole quantitative support is the Bayesian parity result. The statistical robustness of that result is therefore the load-bearing point for the claim as stated.","tokens_in":1746,"tokens_out":329,"duration_ms":46380,"concrete_test":"From the paper's reported mean accuracies and standard errors on the three benchmarks, recompute Bayes factors for H0: |Δacc| < 0.5 % using a default Cauchy prior (scale 0.707) and a normal prior (sd = 0.01); confirm whether BF > 10 for equivalence holds under both.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim asserts that Bayesian parity analysis supplies decisive-to-very-strong evidence of equivalent classification accuracy (ImageNet, ImageNet-C, ObjectNet) after saliency fine-tuning. Equivalence Bayes factors are sensitive to the prior on the performance difference (e.g., normal vs. Cauchy) and the region of practical equivalence (ROPE) width. The abstract states the conclusion but supplies neither the exact model, BF values, prior specification, nor any robustness table. If a wider ROPE or heavier-tailed prior reduces the BF below the 'decisive' threshold, the 'at no cost' assertion no longer follows from the reported data.","agreement_with_reader":"disagree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that fine-tuning the self-attention layers of ViT-B/16 on human saliency fixation maps improves alignment with human attention patterns across five saliency metrics, induces three human-like biases (reversal of anti-human large-object bias toward small objects, amplified animacy preference, and reduced attention entropy), and that this alignment incurs no cost to classification accuracy. This is supported by comparison to a shuffled control (to isolate semantic signals) and Bayesian parity analysis showing decisive-to-very-strong evidence of equivalent performance on ImageNet, ImageNet-C, and ObjectNet. The same procedure applied to ResNet-50 degrades both alignment and accuracy, suggesting ViT self-attention is uniquely suited for this dissociation.","tokens_in":1901,"tokens_out":613,"duration_ms":25169,"significance":"If the empirical results and controls hold, the work is significant for demonstrating that biologically grounded attention priors can be instilled in transformers as an emergent property that enhances interpretability without performance trade-offs on in-distribution, corrupted, and out-of-distribution benchmarks. The architectural contrast with CNNs and the use of a shuffled control provide a concrete way to separate generic supervision from semantically relevant human biases, with potential implications for more human-aligned vision models.","major_comments":[{"comment":"The central 'at no cost' claim rests on Bayesian parity analysis providing decisive-to-very-strong evidence of equivalent accuracy across ImageNet, ImageNet-C, and ObjectNet. However, the abstract (and presumably the corresponding results section) reports neither the exact model specification, prior choices (e.g., normal vs. Cauchy on the performance difference), ROPE width, numerical Bayes factor values, nor any sensitivity/robustness checks. Equivalence Bayes factors are known to be sensitive to these choices; without them the strength of evidence cannot be verified and the claim does not fully follow from the reported data.","section":"Bayesian parity analysis (abstract and results)"},{"comment":"The shuffled control is presented as isolating semantically relevant signals from generic human supervision, yet the manuscript provides no quantitative comparison of how well the shuffled maps preserve low-level statistics (e.g., center bias, entropy) versus the original fixation maps. If the control fails to fully match these statistics, the attribution of bias induction specifically to semantic content is weakened.","section":"Methods (shuffled control description)"}],"minor_comments":[{"comment":"The abstract states that five saliency metrics were used but does not name them; explicitly listing the metrics (e.g., NSS, CC, etc.) would improve immediate readability.","section":"Abstract"},{"comment":"The claim that the procedure 'reversed the baseline's anti-human large-object bias' would benefit from a brief quantitative statement of the effect size or statistical test in the abstract or early results.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive comments, which have helped us improve the clarity and rigor of our manuscript. We address each of the major comments in detail below.","responses":[{"response":"We acknowledge that the manuscript does not provide the full specification of the Bayesian parity analysis. We will revise the Methods and Results sections to include the exact model specification, prior choices (normal prior on the performance difference), ROPE width, numerical Bayes factor values, and sensitivity/robustness checks under alternative priors. This will allow verification of the evidence strength for the equivalence claim.","revision_made":"yes","referee_comment":"[Bayesian parity analysis (abstract and results)] The central 'at no cost' claim rests on Bayesian parity analysis providing decisive-to-very-strong evidence of equivalent accuracy across ImageNet, ImageNet-C, and ObjectNet. However, the abstract (and presumably the corresponding results section) reports neither the exact model specification, prior choices (e.g., normal vs. Cauchy on the performance difference), ROPE width, numerical Bayes factor values, nor any sensitivity/robustness checks. Equivalence Bayes factors are known to be sensitive to these choices; without them the strength of evidence cannot be verified and the claim does not fully follow from the reported data."},{"response":"We agree that quantifying the preservation of low-level statistics in the shuffled control would strengthen the claim that it isolates semantic content. We will add this comparison to the Methods section, reporting metrics such as center bias and entropy for the original versus shuffled maps to demonstrate that the procedure primarily disrupts semantic alignment while largely preserving spatial statistics.","revision_made":"yes","referee_comment":"[Methods (shuffled control description)] The shuffled control is presented as isolating semantically relevant signals from generic human supervision, yet the manuscript provides no quantitative comparison of how well the shuffled maps preserve low-level statistics (e.g., center bias, entropy) versus the original fixation maps. If the control fails to fully match these statistics, the attribution of bias induction specifically to semantic content is weakened."}],"tokens_in":1512,"tokens_out":446,"duration_ms":35239,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that fine-tuning only the self-attention layers of ViT-B/16 on human fixation maps improves alignment on five saliency metrics, flips the model's large-object bias toward small objects, boosts animacy preference, and lowers attention entropy, all while accuracy stays statistically unchanged on ImageNet, ImageNet-C, and ObjectNet. The shuffled saliency control isolates the effect from generic supervision, and the same procedure on ResNet-50 hurts both alignment and accuracy instead. This highlights how the modular attention in transformers lets you adjust spatial priority separately from the rest of the model.","headline":"Fine-tuning ViT self-attention on saliency maps induces human-like biases at no accuracy cost on three benchmarks, but the decisive Bayesian parity claim rests on unreported prior and ROPE details.","tokens_in":2401,"tokens_out":203,"would_cite":false,"duration_ms":42521,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Fine-tuning vision transformers on human saliency maps induces human-like attention biases at no cost to accuracy.","keywords":["vision transformers","human saliency","attention alignment","model interpretability","fine-tuning","self-attention","cognitive biases","saliency metrics"],"falsifier":"Retraining with the shuffled maps yields the same alignment gains and bias changes as the real maps, or classification accuracy drops on ImageNet, ImageNet-C, or ObjectNet after fine-tuning with the real maps.","tokens_in":2628,"feed_emoji":"👁️","tokens_out":511,"duration_ms":30875,"temperature":0.7,"pith_summary":"The paper tests whether vision transformers can be aligned with human attentional patterns by fine-tuning their self-attention weights on maps of where people look at images. A shuffled control isolates whether the gains come from meaningful semantic signals rather than generic supervision. The tuned models improve on five saliency metrics and develop three human-like biases, including a reversal from large-object to small-object preference, stronger animacy focus, and reduced attention entropy. The same changes leave classification performance unchanged on ImageNet, ImageNet-C, and ObjectNet according to Bayesian analysis, whereas the identical procedure degrades both alignment and accuracy in a ResNet-50.","feed_headline":"Vision transformers gain human attention biases at no accuracy cost","feed_subtitle":"Fine-tuning on eye-fixation maps reverses large-object bias and strengthens animacy preference while preserving ImageNet, ImageNet-C and OOD","key_machinery":"Fine-tuning of ViT self-attention weights on human saliency fixation maps, isolated via comparison to a shuffled control.","core_discovery":"Fine-tuning the self-attention weights of ViT-B/16 on human saliency fixation maps, compared against a shuffled control, produces significantly higher alignment across five saliency metrics and induces three hallmark human biases: reversal of the baseline large-object preference toward small objects, amplified animacy preference, and lowered attention entropy. Bayesian parity analysis supplies decisive to very-strong evidence that these changes leave classification accuracy intact on ImageNet, ImageNet-C, and ObjectNet. The same procedure applied to ResNet-50 instead reduces both alignment and accuracy, indicating that the modular self-attention mechanism uniquely permits dissociation of spa","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["ViT-B/16 gains human biases from eye-fixation tuning at no accuracy cost","Saliency fine-tuning aligns ViT attention to humans without accuracy loss","Tuning ViT on human maps reverses object bias at no accuracy cost","ViT-B/16 saliency tuning induces human biases at no accuracy cost"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The human saliency fixation maps capture semantically relevant signals that the shuffled control can separate from nonspecific supervision effects.","fun_headline_variants_meta":{"raw":{"variants":["ViT-B/16 gains human biases from eye-fixation tuning at no accuracy cost","Saliency fine-tuning aligns ViT attention to humans without accuracy loss","Tuning ViT on human maps reverses object bias at no accuracy cost","ViT-B/16 saliency tuning induces human biases at no accuracy cost"]},"model":"grok-4.3","cost_usd":0.00959,"raw_usage":{"total_tokens":4225,"prompt_tokens":724,"num_sources_used":0,"completion_tokens":81,"cost_in_usd_ticks":95903000,"prompt_tokens_details":{"text_tokens":724,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3420,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":724,"tokens_out":81,"duration_ms":47520,"temperature":1.0,"reasoning_tokens":3420,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-10T02:02:43.375869+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Retraining with the shuffled maps yields the same alignment gains and bias changes as the real maps, or classification accuracy drops on ImageNet, ImageNet-C, or ObjectNet after fine-tuning with the real maps.","supporting_citations":[],"review_version":1}