{"id":"a4784fe6-c440-4553-b524-56c8263ece6c","arxiv_id":"2504.16871","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"LLM hidden states from the reading phase encode domain-specific signals that can route queries to better models, boosting average accuracy by 12.3% over a single fine-tuned model.","lead":"This paper shows that the internal activations of large language models, measured while the model reads a question, differ across domains like math, medicine, law, and humanities, and that these differences can be used to route queries to a better-performing model. The work offers a cheap way to pick which model should answer each question, which could reduce cost and improve accuracy in large-scale LLM services.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 12.3% routing gain and domain-trace clustering may be driven by surface statistics (length, options, template) rather than domain semantics; the paper never tests this confound.","rationale":"The reader's weakest assumption matches the main risk: the 'domain trajectory' is a 2L-dimensional summary (mean, std per layer) of one token, and the benchmarks are confounded with format and length. I agree this is the most load-bearing gap. A positive control of this kind does not require new model training or access to proprietary data beyond what the paper already uses, so it is a feasible check. I also note the evaluation lacks error bars and code, which amplifies the concern but is secondary. Because the paper's conditional verdict already identifies this, I recommend UNCHANGED: the claims should be treated as conditional on this confound being ruled out, or the claims should be weakened to 'format-sensitive traces.'","tokens_in":18026,"tokens_out":5721,"duration_ms":56794,"concrete_test":"Run the identical MLP routing experiment with input features computed from the raw query text instead of hidden states: token count, digit count, number of option markers (A/B/C/D), presence of 'Answer the following question'/'boxed', and prompt-template ID, plus first/second moments of token embeddings from a fixed random projection. If this surface-feature classifier matches or approaches the 0.395 average (or if adding hidden-state traces does not significantly improve it across 5 random splits with mean±std reported), then the hidden-state trajectories add no domain-semantic signal beyond formatting. As a control, also permute the hidden-state summaries within strata matched on token count, digit count, and template; if accuracy remains near chance, the observed clustering is surface-driven.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central hypothesis (Section 3) is that scalar mean/std traces of the last-token hidden states (Eq. 1-2, Section 5.1) are 'latent domain-related trajectories.' The load-bearing untested assumption is that these traces are informative about domain semantics and not about surface statistics correlated with domain. The five evaluation datasets differ on exactly those surface axes: MMLU/MEDMCQA/USMLE are multiple-choice with option markers, GSM8K/MATH are open-ended arithmetic with digits and 'boxed' answers, CaseHOLD is a long legal passage, and each source in Table 1 has a distinct prompt template. A per-layer scalar mean/std across all dimensions is precisely a summary that can be dominated by input length, positional norm growth, and formatting tokens, so the clustering in Figures 2-3 and the 12.3% improvement in Table 2 may reflect 'which benchmark am I looking at' rather than 'which domain is this query.' Section 6 even concedes traces may be an infusion of subdomains and may not generalize, but it does not address the surface-statistics confound. In addition, Table 2 has no error bars and no code is released, so the single reported improvement is not yet distinguishable from noise or implementation details.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether LLMs encode domain-specific knowledge in hidden states generated during the prefill phase. For each input query, the authors extract per-layer scalar mean and standard deviation of the last-token hidden state and call the resulting layer curves 'latent domain-related trajectories.' They report that these traces cluster by domain across multiple LLM architectures, remain stable under prompt variation and fine-tuning, and can be used for model routing. A multi-layer perceptron trained on the traces is claimed to improve average routing accuracy by 12.3% over a domain fine-tuned baseline on a set of closed and open-ended benchmarks. The paper includes a limitations section and several appendices with additional trace plots.","tokens_in":18307,"tokens_out":3701,"duration_ms":33003,"significance":"If the claims are substantiated, the paper would provide a lightweight, pre-generation signal for model selection and a new window into how autoregressive LLMs represent domain context. The multi-architecture comparison, the inclusion of open-ended tasks, and the comparison against semantic and token-based baselines are useful and go beyond much of the prior closed-ended probing literature. The paper also deserves credit for explicitly naming limitations, especially the possibility that traces are an infusion of subdomains rather than clean domain representations. However, the central empirical claims are currently undermined by evaluation leakage on MMLU, the untested surface-statistics confound in the scalar mean/std traces, and the absence of error bars or significance tests. These issues are fixable with additional experiments, but they are load-bearing for the paper's main conclusions.","major_comments":[{"comment":"The MLP router is trained on 4,000 random samples from the MMLU Base Pool, and Table 2 reports an MMLU column without any held-out split or overlap check. Because the training and evaluation distributions coincide, the MMLU row cannot support the headline claim of generalization, and it biases the average improvement. Please report results on an MMLU split that is disjoint from the 4,000 training samples (or remove MMLU and report the average over held-out datasets only).","section":"§5.3, Table 2"},{"comment":"The domain trace is a per-layer scalar mean and scalar standard deviation of the last-token hidden state, averaged over all dimensions. Such summaries can be dominated by surface statistics such as input length, digit density, option markers, and prompt template, and the evaluation datasets differ on exactly those axes (MMLU/MEDMCQA/USMLE are multiple-choice with option markers; GSM8K/MATH are open-ended with digits and boxed answers; CaseHOLD is long legal text). The paper does not test this confound. Section 6 even concedes that the clustering may reflect subdomain infusions and may not generalize. Please include controls, e.g., length- and format-matched queries across domains, or train the same router on surface features only and show that the hidden-state signal adds accuracy beyond those features.","section":"§3, §5.1, Eqs. (1)-(2)"},{"comment":"All routing accuracies are reported as point estimates from a single run, with no error bars, seeds, or significance tests. For large margins such as GSM8K the conclusion may survive, but for smaller margins such as USMLE (0.228 vs. 0.241) and CaseHOLD (0.487 vs. 0.492) the claim of consistent improvement is not supported. Please provide means and variances over multiple seeds, a paired significance test, and ideally release the code or per-seed results.","section":"Table 2, §5.4"},{"comment":"The claim that traces cluster by domain is supported only by visual inspection. Please quantify the separation, for example with inter-domain versus intra-domain distances, cluster purity, or cross-validated classification accuracy, and report whether the clusters separate by domain label or by dataset identity. Without such quantification, the 'trajectory' plots in Figures 2-3 are consistent with the router simply recognizing which benchmark or prompt template a query comes from.","section":"§5.2, Figures 2-3"}],"minor_comments":[{"comment":"The text says 'a subset of 5 different datasets (not seeing during training)' is selected, but Table 2 lists six datasets including MMLU, which was used for training. Please reconcile the count and clarify which columns are truly unseen.","section":"§5.3"},{"comment":"The caption says 'Further results for Llama-2B model', but the model list in Section 4 includes Llama2-7B and no Llama-2B model; this appears to be a typo.","section":"Figure 2"},{"comment":"The phrase 'similar behavir' should read 'similar behavior'.","section":"Appendix A.3"},{"comment":"The title contains a spacing typo, 'HOW LLM S CAPTURE', which should be corrected.","section":"Title and abstract"},{"comment":"The claim that layer 26 is a turning point and that using all 32 layers gives the best performance would benefit from the same multiple-seed reporting recommended for Table 2.","section":"§5.4, Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of an applied ML venue, and the core idea is worth exploring. My main concern is that the headline results are not yet separable from evaluation leakage and surface-statistics confounds. I believe the required fixes—held-out MMLU evaluation, surface-feature controls, and error bars—are feasible within revision. I also note that no code or data splits are provided, which makes reproducibility difficult."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my read on arXiv:2504.16871. The headline result — a 12.3% accuracy gain from routing queries to fine-tuned models based on per-layer mean/std of prefill hidden states — is plausible but not yet established. The paper's central hypothesis is that these scalar traces are 'latent domain-related trajectories,' but the load-bearing assumption that they capture domain semantics rather than surface statistics is never tested. The evaluation datasets differ on exactly those axes: MMLU/MEDMCQA/USMLE are multiple-choice with option markers, GSM8K/MATH are open-ended arithmetic with digits and boxed answers, CaseHOLD is long legal passages, and each source has a distinct prompt template. A per-layer scalar summary across all dimensions is precisely the kind of feature that can be dominated by query length and formatting tokens.\n\nWhat is genuinely new: using prefill hidden-state statistics as domain representations for model routing, particularly for open-ended generative tasks where prior probing work focused on closed-ended classification. The paper also shows consistent visual separation across Gemma, Phi, Llama, and Mistral, and reports that traces persist after fine-tuning and stabilize in deeper layers across prompt variations. To their credit, the authors include external datasets not seen during training, so the GSM8K/MATH gains (0.400→0.560, 0.057→0.144) are not a trivial restatement of the training signal. The limitations section is honest about subdomain infusion.\n\nSoft spots, in proportion: First, the surface-statistics confound is real and untested. A simple control — varying prompt templates within the same dataset, permuting option markers, or checking whether the router predicts length-based synthetic queries as a domain — would settle it. Second, Table 2 includes MMLU without a clear held-out split, and the MLP was trained on 4,000 MMLU samples; no error bars or significance tests are given. Third, no code is released, so the reported numbers cannot be checked. These are all fixable.\n\nWho this is for: researchers working on LLM serving, model routing, and activation-based interpretability. It is not a paradigm shift, but it is a cheap, practical idea with a plausible signal. I would send it to peer review rather than desk reject, with a request for the control analyses and a clearer evaluation setup.","headline":"Plausible routing signal from prefill hidden states, but the domain-versus-surface-statistics confound is untested; worth refereeing if the authors add controls and error bars.","tokens_in":18831,"tokens_out":3422,"would_cite":false,"duration_ms":27826,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Hidden states produced while an LLM reads a query encode its domain, and routing on those states beats a single fine-tuned model by 12.3%.","keywords":["latent domain-related trajectories","hidden states","prefill phase","model routing","model selection","LLM interpretability","domain-specific knowledge","MMLU"],"falsifier":"Build a control set of queries that are matched for length, token count, and prompt template and differ only in domain, then retrain the same MLP on their traces: if held-out routing accuracy falls to chance, the domain trajectories were formatting artifacts.","tokens_in":17861,"feed_emoji":"🧭","tokens_out":13719,"duration_ms":118794,"temperature":0.7,"pith_summary":"The paper asks whether large language models already know a query's domain before they start answering it. It claims that the hidden states generated during the prefill phase carry 'latent domain-related trajectories' that separate mathematics, biomedical, law, and humanities queries in a consistent way across several autoregressive model families. It further claims these trajectories survive prompt reformatting and fine-tuning, and that a classifier trained on them can route each query to the model that answers it best. On a six-dataset evaluation, that router raises average accuracy from 0.352 to 0.395 over a single domain-fine-tuned model, a gain the paper reports as 12.3%. The result matters because it offers a path to model selection and interpretation that reads the model's own context understanding rather than external labels or embedding similarity.","feed_headline":"Hidden-state domain traces lift LLM routing accuracy by 12.3%","feed_subtitle":"Prefill activations, not fine-tuning, pick the right model for math, medical, legal, and humanities queries.","key_machinery":"The load-bearing object is the latent domain-related trajectory: for each layer $l$, the mean $\\mu_l$ and standard deviation $\\sigma_l$ of the prefill hidden states, collapsed over the batch and dimension axes and computed from the last token. These two scalars per layer compress each query into a curve over layers. The paper's argument is that the shape of this curve separates queries by domain, stabilizes in deeper layers across prompt variations, and carries enough signal for a small multilayer perceptron (MLP) classifier to route a query to the most accurate model. The layer-reduction experiment shows that the signal becomes usable only after roughly layer 26 and is strongest with all 32 layers.","core_discovery":"The central claim is that an LLM's internal recognition of a query's domain happens during context understanding, before generation, and is visible in the per-layer statistics of its hidden states. Across Gemma, Phi, Llama 2, and Mistral, the per-layer standard deviation of prefill activations forms traces that cluster by domain, while the encoder model DeBERTa does not show the same pattern. The traces remain separated under different prompt templates and persist after fine-tuning, which the paper reads as evidence that they reflect stable properties acquired during pretraining rather than surface text features. When a small classifier is trained on these traces and used to route each query to the model with the highest performance on similar traces, it beats the domain-fine-tuned baseline on average and, in several cases, beats each specialist on its own domain; the paper attributes this to the fine-tuned models overfitting their training data and to hidden states retaining richer domain information. The evaluation covers both multiple-choice and open-ended tasks; the largest gains come on open-ended math benchmarks GSM8K and MATH.","pith_inferences":["The natural next test is cross-model transfer: if the trajectories encode domain semantics rather than model-peculiar formatting, an MLP trained on one LLM family should route queries for another family without retraining.","The observed maths-biomedical overlap suggests the clusters may follow reasoning style (structured and formal versus interpretive and narrative) rather than subject labels, which would explain why a specialist can lose to a generalist on its own domain.","A cheaper router could exploit the early-layer prompt sensitivity and deep-layer stability by reading only the final layers; the paper's own cumulative-layer experiment already locates the earliest useful cutoff at layer 26.","Richer summaries than a per-layer mean and standard deviation, such as layerwise covariance or per-token trajectories, might separate the overlapping maths and biomedical clusters further."],"forward_implications":["Router construction can read only the prefill phase, avoiding a full forward pass and generation for every candidate model.","A fine-tuned specialist is not automatically the best router choice for its own domain; the hidden-state trajectory can send a query to a generalist or another specialist.","Because deep-layer traces are stable across prompt templates, a router trained on one instruction style should keep working when users paraphrase or reformat queries.","The same trajectory signal works for multiple-choice and open-ended generation tasks, and the largest gains appear on open-ended math benchmarks.","Post-fine-tuning trace persistence implies a router can be built once from a pretrained model and kept when the underlying model is fine-tuned for specific tasks."],"supporting_citations":[{"why":"Supplies the MMLU subcategories and supercategory labels that define the four-domain Base Pool and the routing evaluation.","marker":"Hendrycks et al. (2021a)"},{"why":"Introduces Phi-3-mini, the main model whose prefill hidden states are compressed into trajectories and fed to the MLP router.","marker":"Abdin et al. (2024)"},{"why":"Provides Gemma-2B, one of the autoregressive families used to show that the domain traces are not specific to a single model.","marker":"Mesnard et al. (2024)"},{"why":"Provides Mistral-7B, adding a second 7B architecture to the cross-model consistency check.","marker":"Jiang et al. (2023a)"},{"why":"Provides Llama-2-7B and its chat variants, which the paper uses to test whether domain traces persist after fine-tuning.","marker":"Touvron et al. (2023)"},{"why":"Defines DeBERTa, the encoder baseline whose hidden states do not show the same domain separation and whose supervised classifier is a routing baseline.","marker":"He et al. (2021)"},{"why":"Defines the Semantic Router, the embedding-similarity routing baseline that the hidden-state classifier must outperform.","marker":"Labs (2024)"},{"why":"Supplies GSM8K, the open-ended math benchmark that drives the largest routing gains and serves as an out-of-distribution math pool.","marker":"Cobbe et al. (2021)"},{"why":"Supplies the MATH benchmark, a second open-ended math task where hidden-state routing improves on the fine-tuned baseline.","marker":"Hendrycks et al. (2021b)"},{"why":"Supplies CaseHOLD, the law-domain evaluation task that tests whether the hidden-state router transfers to held-out legal questions.","marker":"Zheng et al. (2021)"}],"fun_headline_variants":["Hidden-state domain traces route queries to the best LLM","Prefill activations pick the right model per domain","Domain traces from hidden states beat fine-tuned specialists","LLMs encode domain cues in prefill hidden states"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that one mean and one standard deviation per layer on the last token preserve domain information, rather than surface formatting cues such as query length, digit count, or prompt template.","fun_headline_variants_meta":{"raw":{"variants":["Hidden-state domain traces route queries to the best LLM","Prefill activations pick the right model per domain","Domain traces from hidden states beat fine-tuned specialists","LLMs encode domain cues in prefill hidden states"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1380,"prompt_tokens":894,"completion_tokens":486,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":422}},"tokens_in":510,"tokens_out":486,"duration_ms":4942,"temperature":1.0,"reasoning_tokens":422,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:54:18.111101+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a control set of queries that are matched for length, token count, and prompt template and differ only in domain, then retrain the same MLP on their traces: if held-out routing accuracy falls to chance, the domain trajectories were formatting artifacts.","supporting_citations":[],"review_version":1}