{"id":"dba1a885-c82d-4287-98fe-db99d3913b15","arxiv_id":"2607.09842","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Identity prompts produce direction-coded hidden-state geometry in a base model but magnitude-coded geometry in a multimodal instruction-tuned model; the shift is not seen in two other training regimes.","lead":"A four-model study finds that identity-specifying system prompts leave measurable geometric traces in a language model's hidden-state trajectory, and that the trace shifts from a direction-based to a magnitude-based signal after multimodal instruction-tuning. If the pattern holds, it suggests post-training changes the geometric format in which models encode identity — a possible handle for interpretability and alignment.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The magnitude-coding half of the central claim rests on the ||v1|| norm inversion (Table 4), which is never teacher-forced or template-controlled; if it is a prompt-format artifact, only a borderline, FDR-failing Euclidean p remains.","rationale":"After reading the manuscript, the central claim is the qualitative reorganization: in base Gemma, identity signal survives angular normalization (W1=0.034, p=0.002) and is thus direction-coded; in Gemma-4-E4B-it, it collapses under angular (p=0.439) while persisting under Euclidean (p=0.042) and the mean norm of v1 inverts. The reader's weakest-assumption analysis correctly isolates the norm inversion as the load-bearing piece. I agree with that assessment. The Euclidean p=0.042 is explicitly borderline, fails BH-FDR, and is qualified by the authors; the angular p=0.439 is a null, which by itself only shows an absence of directional coding. The only statistically robust piece of evidence for a positive magnitude-coding claim is Table 4, with pairwise p<0.001. But Table 4 is not controlled for template formatting, final-token position, or generated content; the teacher-forced protocol (Table 5) omits the norm statistic. The paper's own limitation section lists the content decomposition as approximate, but does not flag this missing control. The proposed test—teacher-forced norm comparison—would decide between identity-specific magnitude coding and a template artifact. If the inversion persists under teacher forcing, the central claim is substantially supported; if not, the 'magnitude' half collapses and the paper would need reframing as a suppression/absence result. Thus I would hold the reader's CONDITIONAL verdict: the paper should not be accepted as-is without this control. This is a productive concern, not a rejection: the base-model direction result is strong and the methodology is careful.","tokens_in":15584,"tokens_out":6117,"duration_ms":66137,"concrete_test":"Run the §4.1.4 teacher-forced protocol on Gemma-4-E4B-it with the same 30 shared neutral targets, but record the mean ||v1|| separately under the axis, generic, and vanilla system prompts (and report the final-prompt-token norm ||v0|| as a more direct probe). If the inverted ordering axis < vanilla < generic persists with generated token sequences held identical, the norm inversion is prompt-driven and the magnitude-coding interpretation survives. If the inversion disappears or reverses, Table 4 is a template/formatting artifact and the central claim must be revised. As a secondary check, echo the axis template with identity content removed (or a length-matched 2129-token generic) to test whether the inversion tracks identity content or template format.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's only non-borderline evidence that identity encoding is magnitude-coded in Gemma-4-E4B-it is the ||v1|| inversion in Table 4 (axis 138.9 < vanilla 195.3 < generic 211.5, Mann–Whitney p<0.001). The Euclidean Wasserstein separation that motivates the magnitude reading is itself marginal (p=0.042 at B=5000, Table 3) and fails BH-FDR (Table 9, rank 8). Yet Table 4's statistic is never put under the paper's own content-control: the teacher-forced protocol in §4.1.4 (Table 5) reports W1, ΔC01, and silhouette, but no ||v1||. The axis and generic prompts differ not only in identity content but in template wording, final-token position, and length (2129 vs 957 tokens); a formatting artifact (e.g., the axis prompt ending in a delimiter or special token that suppresses the pre-generation state norm) would reproduce the inversion without any identity-specific magnitude coding. The 'refutation of length as confound' (longest prompt, smallest norm) rules out monotonic length effects but not template effects. If the inversion is an artifact, the magnitude half of the central claim collapses, leaving only a direction-coded base model and an instruction-tuned model with no robust angular or Euclidean separation — a suppression story, not a direction-to-magnitude reorganization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks whether identity-specifying system prompts induce statistically distinguishable geometric fingerprints in transformer hidden-state trajectories, and how these depend on the post-training regime. Using four 7–8B open-weight models spanning four regimes (base, multimodal RLHF, RL distillation, SFT), it compares three prompt conditions via several geometric statistics, with the primary metric being the 1-Wasserstein distance between edge-wise distributions of Ollivier–Ricci curvature on k-NN trajectory graphs. The central claim is a direction-to-magnitude reorganization across the instruction-tuning boundary: in base Gemma-4-E4B the identity signal survives angular normalization (directional encoding), while in Gemma-4-E4B-it it collapses under angular normalization and is instead detected in Euclidean space and in the norm of the first generated state. The paper includes trajectory-level permutation tests, sensitivity sweeps, an FDR audit, and a teacher-forced content control.","tokens_in":15957,"tokens_out":5549,"duration_ms":62033,"significance":"If the direction-to-magnitude reorganization is real, it is a noteworthy qualitative finding about how instruction tuning changes the geometric substrate of prompt encoding, and the proposed W1-on-edge-Ollivier-Ricci statistic is a potentially useful methodological tool. The paper is unusually careful in its inferential discipline: permutation tests are trajectory-level, borderline p-values are re-run at B=5000, BH-FDR and Holm corrections are reported, k-NN and PCA sensitivities are swept, anisotropy is addressed via ABT projection, and teacher-forced controls isolate content-driven effects. The authors also disclose their limitations explicitly. My assessment is that the statistical reporting is honest and mostly sound, but the magnitude-coding half of the central claim currently rests on a norm inversion that has not been subjected to the paper's own content/template controls, while the only direct geometric evidence for it is a single FDR-failing p-value.","major_comments":[{"comment":"Table 4 is the principal evidence for magnitude coding of identity in Gemma-4-E4B-it: the axis prompt yields mean ||v1||=138.9 vs 211.5 for generic and 195.3 for vanilla. However, this statistic is never included in the teacher-forced protocol of §4.1.4/Table 5, and no template/format control is reported. The axis and generic prompts differ not only in identity content but in length (2129 vs 957 tokens), wording, formatting, and final-token position; the paper's 'refutation of length as confound' only rules out a monotonic length effect, not a template or special-token artifact. If the inversion disappears under teacher-forced shared targets or under a length-matched/format-matched control, the magnitude half of the central claim collapses. Please report ||v1|| in the teacher-forced setting and with at least one format-controlled generic variant.","section":"§4.1.2, Table 4"},{"comment":"The only non-norm geometric evidence for magnitude coding in the instruction-tuned model is the Euclidean axis-vs-vanilla W1 with p=0.042 at B=5000. As the paper correctly states, this fails BH-FDR at q=0.05 (rank 8, threshold 0.040). The paper nevertheless treats 'survives under Euclidean k-NN' as a central corroboration in §4.1.2 and §6. Given the FDR failure, this comparison is at best supportive; the central claim therefore depends even more heavily on the norm inversion of Table 4, whose controls are missing. Please either strengthen this comparison or explicitly restate the central claim as resting on the norm channel only.","section":"Table 3 and Table 9"},{"comment":"The teacher-forced control shows that the free-running Euclidean curvature signal W1(rho_axis,rho_vanilla) attenuates by ~94% (0.0288 to 0.0016) and becomes non-significant. Thus most of the IT-model curvature separation is content-driven, not prompt-driven. The ~30% 'prompt-driven component proper' is estimated only for the cosine statistic Delta C01 and is not shown to apply to the norm channel. Since the norm inversion is not teacher-forced, it could reflect differences in the content generated by the prompts rather than a prompt-driven geometric fingerprint. Please either report the teacher-forced ||v1|| or limit the magnitude-coding claim to free-running trajectories.","section":"§4.1.4, Table 5"},{"comment":"The paper's abstract and §1.4 state that the direction-to-magnitude reorganization is 'specific to the multimodal instruction-tuning regime,' but each regime is represented by a single model. As the authors acknowledge in §7, multiple models per regime are needed to support a regime-specificity claim. Without replication within a regime, the observed difference could be due to architecture, data mix, or model family rather than the post-training regime. Please temper the regime-specificity language in the abstract and introduction, or add additional checkpoints per regime.","section":"§7, Future work"}],"minor_comments":[{"comment":"The phrase 'normative substrate' appears to be a typo; the text elsewhere uses 'magnitude' or 'norm'. Please replace 'normative' with 'norm-based' or 'magnitude'.","section":"§1.4, H2 and §6"},{"comment":"The p-value for W1(rho_axis,rho_vanilla) in Gemma-4-E4B-it is reported as 0.042 in Table 2 and as 0.047 (refined to 0.042 at B=5000) in Table 3, while Table 5 lists 0.047 for what appears to be the same free-running comparison. Please harmonize these values and clarify which one is the headline p-value.","section":"Table 2 vs Table 3"},{"comment":"The generic prompt is called 'length-matched' but its length (957 tokens) is not matched to the axis prompt (2129 tokens). Consider describing it as 'length-controlled' or 'similarly long' to avoid confusion.","section":"§2.2"},{"comment":"The pooled-edge W1 uses ~60,000 curvature values per condition while the permutation test shuffles only 100 trajectories. The per-trajectory variant in Appendix C is a useful check, but the main text could explicitly note that the pooled statistic does not treat edges as independent samples for inference.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The paper is transparent and methodologically careful, and the authors have made a genuine effort to report borderline results honestly. My main concern is that the central direction-to-magnitude claim depends on the Table 4 norm inversion, which is not subjected to the teacher-forced or template controls that the paper itself uses elsewhere; the supporting Euclidean p-value is FDR-failing. These issues are fixable with additional experiments (teacher-forced norm, length/format-matched generic, multiple models per regime), so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing you should know: this is a well-designed, honestly-reported empirical study whose headline claim outruns its evidence. The authors do find a real in-sample pattern — in the base Gemma model identity prompts separate under angular k-NN, while in the instruction-tuned twin the separation shifts to Euclidean, and the norm of the first generated state is lowest for the longest (identity) prompt. That is an interesting observation. But the magnitude-coding half of the story rests on a norm statistic that is never subjected to the paper's own teacher-forced control, and the Euclidean Wasserstein p-value that motivates it is borderline (0.042) and fails BH-FDR. So the central 'direction-to-magnitude reorganization' is not yet established.\n\nWhat's genuinely good: the experimental discipline. Trajectory-level permutations, FDR audit, sensitivity sweeps over k and PCA dimension, ABT anisotropy corrections, and a teacher-forced shared-target control. The authors also state their limitations with refreshing honesty — including the one-model-per-regime design and the ~94% content-driven attenuation of the curvature signal. The W1-on-edges of Ollivier-Ricci curvature protocol is a new combination of existing tools, and it is cleanly presented. That methodological part deserves credit.\n\nThe soft spots, in proportion. The stress-test concern about Table 4 is valid: the norm inversion is never teacher-forced. The axis and generic prompts differ in template, final-token position, and length (2129 vs 957 tokens), so the inversion could be a formatting artifact. The 'longest prompt has the smallest norm' argument rules out monotonic length effects, but not template-specific effects. If the inversion does not survive a teacher-forced norm-side control, the magnitude-coding claim collapses, leaving only a suppression story in the IT model. The teacher-forced results in Table 5 already show that ~94% of the curvature signal is content-driven; applying the same protocol to ||v1|| is the obvious missing experiment. Also, regime-specificity rests on a single model per regime; the paper's own future-work section acknowledges this. The claim as stated is an empirical contingency, not a demonstrated property of multimodal RLHF.\n\nWho it's for: people working on persona vectors, activation norm statistics, and alignment monitoring. The methodological protocol is worth citing; the empirical finding should be treated as preliminary.\n\nRecommendation: send to peer review — the paper is careful enough and the question is important enough to justify referee time. Expect the norm claim to need substantial additional evidence (teacher-forced norm, template-matched controls, more models per regime) before it can be accepted as stated.","headline":"A careful, honest empirical paper whose headline direction-to-magnitude claim outruns its evidence — the norm inversion is never content-controlled, and the borderline Euclidean p fails FDR.","tokens_in":16419,"tokens_out":2594,"would_cite":true,"duration_ms":26238,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Multimodal instruction-tuning moves identity-specifying prompts from a direction-coded to a magnitude-coded geometric fingerprint in transformer hidden states.","keywords":["hidden-state geometry","Ollivier-Ricci curvature","Wasserstein distance","identity-specifying prompts","instruction tuning","direction vs magnitude encoding","transformer interpretability","trajectory analysis"],"falsifier":"Run the norm measurement with teacher-forced shared targets: condition an identity prompt and a generic prompt to generate the same neutral target sequences. If the identity prompt no longer yields the lowest norm of the first generated state—or if a length- and format-matched non-identity prompt does—the magnitude-coding claim is an artifact of template surface features rather than identity content.","tokens_in":15445,"feed_emoji":"📐","tokens_out":5318,"duration_ms":49355,"temperature":0.7,"pith_summary":"This paper asks whether identity-specifying system prompts leave a distinguishable geometric trace in transformer hidden-state trajectories once prompt length and generated content are controlled. Across a four-model panel spanning no training, multimodal RLHF, RL distillation, and SFT, the authors find a regime-dependent reorganization: in the untrained base model the identity fingerprint lives in the direction of hidden-state vectors, while in the multimodal instruction-tuned model it moves into the magnitude. The identity-vanilla separation collapses under angular (direction-only) comparison but survives under Euclidean comparison, and the norm of the first generated state is lowest for the identity prompt even though it is the longest prompt. The effect is specific to the multimodal instruction-tuning regime and is concentrated in the first generated token, with a teacher-forced content control attributing about 30 percent of the free-running cosine signal to the prompt itself. If correct, the finding implies that identity information in post-trained models may be carried by an overlooked channel—vector magnitude—and that the encoding substrate is not fixed across training regimes.","feed_headline":"Multimodal tuning moves identity coding into vector magnitude","feed_subtitle":"Hidden-state trajectories show the switch only under multimodal instruction-tuning, not other training regimes.","key_machinery":"The argument is carried by comparing the same point cloud of hidden states under two base metrics. Trajectories of 256 hidden states are turned into k-NN graphs; for each edge, Ollivier-Ricci curvature measures how much the random-walk neighborhoods of the two endpoints overlap under optimal transport, and the per-condition pooled distribution of edge curvatures is compared by the 1-Wasserstein distance. Running the same protocol with Euclidean distance versus angular (L2-normalized) distance isolates whether a geometric difference lives in vector magnitude or in direction. A teacher-forced control, in which different prompts are forced to generate the same target tokens, separates prompt-dr","core_discovery":"The central empirical claim is a qualitative reorganization of identity encoding across the instruction-tuning boundary. In the base-weight model, the axis-vs-vanilla separation under edge-wise Ollivier-Ricci curvature on k-NN trajectory graphs survives and strengthens under angular normalization (separation 0.034, permutation p=0.002), so identity is encoded in the direction of hidden states. In the multimodal instruction-tuned model, the same separation collapses under angular k-NN (p=0.439) but persists under Euclidean k-NN (p=0.042 at B=5000), and the mean norm of the first generated state is lowest for the identity prompt (138.9 vs 211.5 generic and 195.3 vanilla)—an inversion of the le","pith_inferences":["If norm-coding is genuine, hidden-state magnitude could be an unguarded channel: interventions that normalize or steer directions would leave identity content intact, and detection methods should add norm-based features.","The inversion is consistent with instruction-tuning learning to compress a long identity template into a single low-norm 'mode' at the first token; a layer-resolved trace should show where the norm drop is introduced, a direct next experiment.","The roughly 30 percent prompt-driven residual at the first token suggests identity conditioning acts like a switch that changes the initial state's norm; one testable consequence is that deliberately manipulating that norm could modulate identity-related behavior without changing token-level content.","Because only one model pair instantiates the multimodal regime, the regime-specificity claim would be stress-tested by repeating the angular-vs-Euclidean diagnostic on other base/tuned pairs; if the inversion fails to appear there, the finding narrows to this specific model family."],"forward_implications":["If the reorganization is real, identity-related information in instruction-tuned models is not stored along the directional axes that steering and probing methods usually search for; norm-based statistics would be needed to read it.","The length confound is directly refuted: the longest prompt produces the smallest first-state norm in the tuned model, so the magnitude signal tracks semantic content rather than prompt length.","The prompt's geometric effect is concentrated in the transition from the final prompt token to the first generated token and decays within roughly ten tokens; trajectory-level clustering mostly reflects generated content.","The direction-to-magnitude switch is specific to multimodal RLHF among the four regimes tested; RL distillation retains length-driven separation and SFT shows none, so the change cannot be attributed to instruction-tuning in general.","The methodological combination—Wasserstein distance on edge-wise Ollivier-Ricci curvature distributions—provides a graph-level way to compare full trajectory geometries, which the paper argues is independently useful."],"fun_headline_variants":["Multimodal tuning flips identity from direction to magnitude","Only multimodal tuning shifts identity from direction to magnitude","Identity coding moves to magnitude after multimodal tuning","Multimodal tuning, not RL or SFT, shifts identity to magnitude"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the lowered norm of the first generated hidden state under the identity prompt comes from its identity content and not from surface features such as prompt length, formatting, or the position of the last token, since the norm result was not checked under the prompt-to-content control the paper applies to its other signals.","fun_headline_variants_meta":{"raw":{"variants":["Multimodal tuning flips identity from direction to magnitude","Only multimodal tuning shifts identity from direction to magnitude","Identity coding moves to magnitude after multimodal tuning","Multimodal tuning, not RL or SFT, shifts identity to magnitude"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00104,"raw_usage":{"total_tokens":4289,"prompt_tokens":902,"completion_tokens":3387,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":3321}},"tokens_in":646,"tokens_out":3387,"duration_ms":67576,"temperature":1.0,"reasoning_tokens":3321,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T01:40:46.556569+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the norm measurement with teacher-forced shared targets: condition an identity prompt and a generic prompt to generate the same neutral target sequences. If the identity prompt no longer yields the lowest norm of the first generated state—or if a length- and format-matched non-identity prompt does—the magnitude-coding claim is an artifact of template surface features rather than identity content.","supporting_citations":[],"review_version":2}