{"id":"a5c0759e-b0a7-4ca7-9152-fb598b0bf72c","arxiv_id":"2606.12818","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Attribution methods localize anchoring signals in Qwen and Llama models; edge-level circuits transfer within a model but show sparse transfer from base to instruction-tuned variants.","lead":"The paper applies attribution-based circuit localization to find pathways inside 7B-8B language models that carry anchoring effects from irrelevant numbers in prompts. A smart generalist might read it to understand how cognitive biases manifest in model internals and whether post-training alters those pathways.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Logit-difference metric may be confounded by shared answer-option structure","rationale":"The reader's weakest assumption correctly identifies the foundational metric as the least secure step; the full-text validation details would be needed to evaluate its strength, but the concern location matches the abstract's own description of the metric.","tokens_in":1636,"tokens_out":284,"duration_ms":19542,"concrete_test":"Construct matched non-anchoring control prompts that reuse the exact same answer-option set and paragraph structure but omit or neutralize the numerical anchor; recompute the logit-difference and check whether it still varies systematically with the (absent) anchor value or with behavioral anchoring scores. If the metric remains elevated, the isolation fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the logit-difference (correct option minus anchor option) isolates anchoring behavior. The setup uses shared answer options across conditions, so the metric could capture option-position biases, general option preferences, or other prompt features rather than the numerical anchor effect. The abstract states that the metric 'tracks behavioral anchoring,' but without explicit controls separating anchor sensitivity from the shared-option structure, the recovered 'anchor-sensitive signal' for attribution may not be specific. This is load-bearing: all claims about edge-level vs. node-level fidelity and within-model transfer rest on this signal being veridically anchor-related.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript examines anchoring effects in language models' numerical reasoning using a controlled multiple-choice setup with shared answer options. It defines a logit-difference metric (correct option minus anchor option), states that this metric tracks behavioral anchoring, and applies attribution-based circuit localization on 7B-8B Qwen and Llama base and instruction-tuned models. Main results claim that edge-level attribution recovers the anchor-sensitive signal more faithfully than node-level methods, that low- and high-anchor circuits transfer strongly within a model, and that transfer is sparse across base and instruction-tuned variants.","tokens_in":1757,"tokens_out":321,"duration_ms":14469,"significance":"If the logit-difference metric is shown to isolate anchoring without confounds from shared option structure, the results would provide a mechanistic account of how anchoring-related decision signals are carried in LMs, with useful comparisons of edge vs. node attribution and effects of post-training. The work applies standard attribution methods to a new behavioral metric and reports within- vs. cross-variant transfer patterns.","major_comments":[{"comment":"Abstract and paragraph on metric definition: The claim that the logit-difference metric 'tracks behavioral anchoring' is asserted without reported quantitative validation numbers, error bars, ablation details, or explicit controls separating anchor sensitivity from shared answer-option structure (e.g., position biases or general option preferences). This is load-bearing for the central claims on edge-level fidelity and circuit transfer.","section":"Abstract and metric definition paragraph"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for highlighting the need for stronger validation of the logit-difference metric. We agree this is central to the claims and will expand the manuscript with the requested quantitative details, ablations, and controls.","responses":[{"response":"We accept the critique that the abstract and metric definition section present the validation claim without sufficient supporting numbers or controls. The full paper reports behavioral correlations between the logit-difference and anchoring effects, but these lack the explicit ablations, error bars, and option-structure controls requested. In revision we will add a dedicated validation subsection with: (1) quantitative correlation coefficients and error bars across prompt variants, (2) ablations that shuffle or remove the anchor while preserving option structure, and (3) controls for position bias and general option preference by comparing against non-anchor numerical prompts. These additions will directly support the downstream claims on edge-level attribution and circuit transfer.","revision_made":"yes","referee_comment":"[Abstract and metric definition paragraph] Abstract and paragraph on metric definition: The claim that the logit-difference metric 'tracks behavioral anchoring' is asserted without reported quantitative validation numbers, error bars, ablation details, or explicit controls separating anchor sensitivity from shared answer-option structure (e.g., position biases or general option preferences). This is load-bearing for the central claims on edge-level fidelity and circuit transfer."}],"tokens_in":1245,"tokens_out":294,"duration_ms":10563,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core contribution is a controlled application of attribution-based circuit finding to numerical anchoring in 7-8B Qwen and Llama models. They set up MCQs with shared answer options, define a logit difference between the correct choice and the anchor choice, claim it tracks the behavioral shift, then compare edge-level versus node-level methods and check transfer of the recovered circuits. Edge methods come out ahead and low/high-anchor circuits transfer within a model but show sparse transfer across base and instruction-tuned variants.\n\nWhat stands out is the direct comparison of edge and node attribution on this signal and the within-model versus cross-variant transfer results. Those are concrete observations rather than restatements of prior circuit work.\n\nThe setup is reasonable on its face and the authors are upfront that cross-variant transfer is less reliable. The main soft spot is the logit-difference metric itself. Shared answer options across conditions leave room for the metric to pick up position biases or general option preferences instead of anchor sensitivity alone. The abstract states that the metric tracks behavioral anchoring, but the strength of that validation determines how much weight the downstream claims about faithful recovery and pathway structure can carry. If the controls are thin, the edge-versus-node and transfer findings become harder to interpret as specifically about anchoring.\n\nThis work is aimed at people already doing mechanistic interpretability on biases inside current-scale models. It is solid enough on the experimental framing and the reported patterns to merit a serious referee, mainly to press on the metric validation and any ablations that separate anchor effects from the shared-option structure. I would send it to review rather than desk reject.","headline":"The paper applies edge attribution to localize anchoring in LMs and reports better recovery than node methods plus within-model transfer, but the logit-difference metric may still mix in option-position effects.","tokens_in":2241,"tokens_out":404,"would_cite":false,"duration_ms":16958,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Edge-level attribution recovers the anchor-sensitive signal in language models more faithfully than node-level methods.","keywords":["anchoring effects","language models","circuit localization","attribution methods","numerical reasoning","model variants","transfer"],"falsifier":"Ablating the localized circuits produces no measurable change in the model's preference for the anchor option over the correct option in the multiple-choice task.","tokens_in":2540,"feed_emoji":"","tokens_out":573,"duration_ms":23266,"temperature":0.7,"pith_summary":"The paper studies how irrelevant numbers in prompts create anchoring effects that shift language model judgments in numerical reasoning tasks. It sets up a controlled multiple-choice format with shared answer options and defines a logit-difference metric between the correct option and the anchor option to track this behavior. Attribution-based circuit localization applied to base and instruction-tuned models shows that edge-level methods identify the relevant pathways more accurately than node-level methods. Low-anchor and high-anchor circuits transfer strongly inside the same model but only sparsely between base and instruction-tuned variants. The results give a mechanistic account of where anchoring-related decision signals are carried inside the models.","feed_headline":"Edge attribution recovers anchoring signal more faithfully","feed_subtitle":"Low- and high-anchor circuits transfer within models but sparsely between base and instruction-tuned variants.","key_machinery":"The logit-difference metric comparing correct and anchor answer options, used to validate tracking of behavioral anchoring and combined with attribution-based circuit localization to identify pathways.","core_discovery":"Attribution-based circuit localization recovers the anchor-sensitive signal more faithfully with edge-level methods than with node-level methods. Low- and high-anchor circuits transfer strongly within a model, indicating shared pathway structure across anchor direction, while transfer across base and instruction-tuned variants is sparse, showing that post-training changes which pathways matter most.","pith_inferences":["Causal interventions on the localized circuits would be expected to reduce observed anchoring effects.","The same localization approach could be tested on other documented biases in language model reasoning.","Sparse transfer across variants implies that circuit findings require re-localization after instruction tuning."],"forward_implications":["Low- and high-anchor circuits share pathway structure across anchor direction within a model.","Post-training alters which pathways carry the anchor-sensitive signal compared with base models.","Edge-level attribution methods give a more faithful recovery of the signal than node-level methods.","Anchoring-related decision signals localize to identifiable circuits inside the models."],"fun_headline_variants":["Edge attribution recovers anchoring more faithfully than nodes","Shared low and high anchor circuits within models","Sparse transfer of anchoring circuits across model variants","Post training changes anchoring pathways in LMs"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The logit-difference metric between correct and anchor options faithfully isolates anchoring behavior without being confounded by the shared answer-option structure or other prompt features.","fun_headline_variants_meta":{"raw":{"variants":["Edge attribution recovers anchoring more faithfully than nodes","Shared low and high anchor circuits within models","Sparse transfer of anchoring circuits across model variants","Post training changes anchoring pathways in LMs"]},"model":"grok-4.3","cost_usd":0.006202,"raw_usage":{"total_tokens":2874,"prompt_tokens":571,"num_sources_used":0,"completion_tokens":53,"cost_in_usd_ticks":62024500,"prompt_tokens_details":{"text_tokens":571,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2250,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":571,"tokens_out":53,"duration_ms":16526,"temperature":1.0,"reasoning_tokens":2250,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T07:03:07.818028+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Ablating the localized circuits produces no measurable change in the model's preference for the anchor option over the correct option in the multiple-choice task.","supporting_citations":[],"review_version":1}