{"id":"246d0fcd-4152-454d-9ba3-93a8767eea69","arxiv_id":"2604.22411","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"LLMs show implementation-induced randomness even at T=0 that can be characterized as an effective background temperature T_bg estimated via an ideal reference system.","lead":"The paper introduces background temperature as a way to measure hidden randomness in LLMs that appears even when temperature is set to zero. This formalization could improve how we test and rely on AI outputs in practice.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Equivalence of implementation perturbations to temperature sampling lacks distributional justification","rationale":"The reader's weakest assumption matches the load-bearing modeling step exactly. The short-note format and pilot experiments do not resolve whether the perturbations obey the same probabilistic structure as temperature sampling, so the formalization remains conditional on that check.","tokens_in":1646,"tokens_out":338,"duration_ms":47289,"concrete_test":"For one model and prompt exhibiting T=0 divergence, estimate T_bg via the paper's protocol. Then, in a fixed deterministic implementation, generate 1000 samples at temperature T = T_bg and compute KL divergence between the resulting token distribution and the empirical distribution from 1000 T=0 runs with varying batch sizes. If KL > 0.5 nats or top-5 token overlap < 70%, the equivalence does not hold.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim defines T_bg as the effective temperature induced by implementation-dependent perturbations (batch-size variation, kernel non-invariance, floating-point non-associativity) even at nominal T=0, and relates it to T_n(I) in an ideal reference system. This requires that the stochastic process governed by inference environment I produces output distributions equivalent to those obtained by scaling logits before softmax in a deterministic model. However, the cited perturbation sources can induce structured, non-uniform variations (e.g., order-dependent rounding errors affecting only certain matrix multiplications) that do not match the entropy-increasing effect of temperature sampling across the full vocabulary. The empirical protocol for estimating T_n(I) therefore rests on an unverified assumption of distributional equivalence.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that LLMs produce divergent outputs even at nominal temperature T=0 due to implementation-dependent perturbations (batch-size variation, kernel non-invariance, floating-point non-associativity). It formalizes this by defining background temperature T_bg as the effective temperature induced by a stochastic perturbation process governed by the inference environment I, relates T_bg to an equivalent temperature T_n(I) in an ideal reference system, proposes an empirical protocol for estimating T_bg, presents pilot experiments across major LLM providers, and discusses implications for reproducibility, evaluation, and deployment.","tokens_in":1793,"tokens_out":521,"duration_ms":41368,"significance":"If the distributional equivalence between implementation perturbations and temperature sampling holds and the estimation protocol is validated, the framework could provide a useful quantitative lens for hidden nondeterminism in LLM inference, supporting more reproducible research and reliable production systems. The pilot experiments add initial empirical grounding by showing the effect across providers.","major_comments":[{"comment":"The core modeling step—that implementation perturbations induce an output distribution equivalent to temperature scaling of logits before softmax—is load-bearing for the definition of T_bg and the protocol using T_n(I), yet the abstract supplies no distributional justification or comparison (e.g., via KL divergence, entropy matching, or moment analysis). Sources such as order-dependent rounding can produce structured, non-uniform effects that do not replicate the entropy-increasing action of temperature across the full vocabulary.","section":"Abstract (definitions and relations)"},{"comment":"The empirical protocol for estimating T_n(I) is described at a high level but lacks concrete details on the matching procedure, metrics, or controls for confounding factors (e.g., how multiple runs at T=0 are compared to temperature sweeps in the reference system). Without these, it is impossible to assess whether the pilot experiments actually support the claimed equivalence.","section":"Abstract (empirical protocol and pilot experiments)"}],"minor_comments":[{"comment":"Clarify the precise mathematical relation between T_bg and T_n(I) upon first introduction; the current phrasing leaves open whether T_bg is defined as identical to T_n(I) or merely estimated by it.","section":null}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a short conceptual note whose central contribution rests on an unverified modeling assumption; if the authors cannot supply the missing distributional evidence within the current scope, the work may be better suited to a workshop or preprint rather than a full journal article."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on our short note. The comments correctly note that the abstract is high-level and could better support the core claims with additional clarification. We address each major comment below and will revise the manuscript accordingly to improve transparency without altering the note's scope or conclusions.","responses":[{"response":"We agree the abstract does not detail distributional comparisons. The manuscript defines T_bg operationally via the inference environment I and relates it to T_n(I) through empirical matching of output statistics (e.g., entropy or diversity measures) rather than assuming exact equivalence for every perturbation source. Structured effects such as non-associativity are treated as contributing to net effective randomness. We will revise the abstract to state explicitly that equivalence is operational and metric-based (entropy or KL matching), not a claim of identical mechanisms across all implementation artifacts. This preserves the framework while addressing the concern.","revision_made":"yes","referee_comment":"The core modeling step—that implementation perturbations induce an output distribution equivalent to temperature scaling of logits before softmax—is load-bearing for the definition of T_bg and the protocol using T_n(I), yet the abstract supplies no distributional justification or comparison (e.g., via KL divergence, entropy matching, or moment analysis). Sources such as order-dependent rounding can produce structured, non-uniform effects that do not replicate the entropy-increasing action of temperature across the full vocabulary."},{"response":"The referee correctly observes that the abstract omits protocol specifics. The full manuscript outlines the protocol as comparing variability from repeated T=0 runs under I against temperature sweeps in a reference system, using metrics such as output entropy and unique response rates, with controls including fixed prompts and averaging across trials. We will expand the abstract with a concise description of the matching procedure and metrics, and augment the pilot experiments section with further details on controls and the reference implementation. These changes will make the empirical grounding more transparent.","revision_made":"yes","referee_comment":"The empirical protocol for estimating T_n(I) is described at a high level but lacks concrete details on the matching procedure, metrics, or controls for confounding factors (e.g., how multiple runs at T=0 are compared to temperature sweeps in the reference system). Without these, it is impossible to assess whether the pilot experiments actually support the claimed equivalence."}],"tokens_in":1345,"tokens_out":500,"duration_ms":39512,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core contribution is a clean formalization of T_bg as the effective temperature from real-world inference quirks like batch-size effects and floating-point ordering, even when the model is set to deterministic decoding. They relate it to an ideal reference system governed by the inference environment and give a protocol to back it out empirically. That framing is useful because it turns a known annoyance into something measurable for reproducibility checks and deployment decisions. The pilot experiments on major providers are a reasonable starting point to show the idea in practice. What the paper does well is keep the definitions straightforward and tie the concept directly to the sources of nondeterminism already flagged in the Thinking Machines Lab work. It avoids overclaiming and focuses on a practical characterization rather than a broad theory. The main soft spot is the unexamined assumption that the output distribution from those implementation perturbations is close enough to what you get by simply scaling logits with temperature. Structured effects, such as order-dependent rounding in specific matrix multiplies, can create non-uniform shifts that do not increase entropy the same way across the vocabulary. Without seeing the actual estimation details or controls in the pilots, it is difficult to judge how well the protocol holds up. The paper is short and targeted, so it is best for readers who already work on LLM evaluation pipelines or inference reliability. It is not a foundational advance, but the idea is coherent enough on its own terms that a serious referee could usefully tighten the distributional justification and check the experiments. I would send it to peer review.","headline":"The paper defines background temperature T_bg to quantify implementation-induced nondeterminism in LLMs at nominal T=0 and sketches an estimation protocol, but the distributional match to temperature scaling is assumed rather than shown.","tokens_in":2245,"tokens_out":385,"would_cite":false,"duration_ms":27996,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Even at a nominal temperature of zero, large language models produce divergent outputs due to implementation-dependent perturbations that can be characterized as an effective background temperature.","keywords":["background temperature","large language models","nondeterminism","reproducibility","inference environment","temperature sampling","LLM evaluation","hidden randomness"],"falsifier":"Running the same input multiple times at T=0 on a fixed inference setup and finding that the output variability does not correspond to what the estimated T_n(I) would predict in the reference system.","tokens_in":2545,"feed_emoji":"🌡️","tokens_out":673,"duration_ms":32802,"temperature":0.7,"pith_summary":"This paper formalizes the hidden nondeterminism in LLMs that occurs even when temperature is set to zero for deterministic decoding. It introduces background temperature as the effective temperature caused by factors like batch size variation, kernel choices, and floating-point operations in the inference environment. The authors provide definitions linking this to stochastic perturbations and outline an empirical method to estimate it using an ideal reference system. Pilot experiments across major LLM providers illustrate the concept and its effects on output consistency. Understanding this allows for better control over reproducibility in model evaluations and deployments.","feed_headline":"LLMs retain hidden randomness at T=0 modeled by background temperature","feed_subtitle":"The effective temperature from implementation details explains output divergence even in deterministic mode and affects reproducibility.","key_machinery":"Background temperature T_bg, which captures the effective randomness from implementation perturbations at nominal T=0 and is estimated via the equivalent temperature T_n(I) in a reference system determined by the inference environment I.","core_discovery":"We introduce the notion of background temperature T_bg as the effective temperature induced by an implementation-dependent perturbation process that is observed even when the nominal temperature T is set to zero. This T_bg relates to a stochastic perturbation governed by the specific inference environment I, and can be estimated through the equivalent temperature T_n(I) of an ideal reference system. The formalization and estimation protocol are demonstrated via experiments on LLMs from major providers.","pith_inferences":["Standardizing the measurement of background temperature could enable fairer benchmarks across different hardware and software stacks.","This concept might extend to other generative AI models beyond LLMs where implementation noise affects outputs.","Future work could explore ways to reduce T_bg through more deterministic computing practices or software fixes.","The idea connects to broader issues in computational reproducibility in scientific computing."],"forward_implications":["Outputs from LLMs at T=0 will still vary based on the specific inference setup used.","Reproducibility requires controlling or measuring the background temperature induced by the environment.","Evaluation of models must consider this hidden variability to avoid misleading comparisons.","Deployment in production systems needs awareness of how different inference environments affect output stability.","The proposed estimation protocol can be used to quantify and mitigate these effects."],"fun_headline_variants":["Background temperature explains LLM randomness despite T=0","LLMs display hidden randomness at T=0 via background temperature","Background temperature formalizes hidden randomness in LLMs at T=0","Implementation perturbations yield background temperature at T=0 in LLMs"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The divergence observed at T=0 can be accurately represented as an equivalent temperature T_n(I) within an ideal reference model controlled by the inference environment I.","fun_headline_variants_meta":{"raw":{"variants":["Background temperature explains LLM randomness despite T=0","LLMs display hidden randomness at T=0 via background temperature","Background temperature formalizes hidden randomness in LLMs at T=0","Implementation perturbations yield background temperature at T=0 in LLMs"]},"model":"grok-4.3","cost_usd":0.009698,"raw_usage":{"total_tokens":4216,"prompt_tokens":620,"num_sources_used":0,"completion_tokens":67,"cost_in_usd_ticks":96978000,"prompt_tokens_details":{"text_tokens":620,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3529,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":620,"tokens_out":67,"duration_ms":34859,"temperature":1.0,"reasoning_tokens":3529,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-08T12:10:39.936495+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the same input multiple times at T=0 on a fixed inference setup and finding that the output variability does not correspond to what the estimated T_n(I) would predict in the reference system.","supporting_citations":[],"review_version":1}