{"id":"374455ac-5c1a-4ee8-9d99-bc3bfe741a8a","arxiv_id":"2607.17060","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Predictive Monte Carlo recovers a Bayes-filtered transformer's implicit prior and posterior over latent tasks from next-token rollouts alone; the task-diversity threshold and transient generalization appear in this recovered latent space.","lead":"This paper applies an existing Bayesian sampling method, Predictive Monte Carlo (PMC), to read the implicit prior and posterior a Bayes-filtered transformer has internalized, using only the model's next-token predictions. It demonstrates the approach on three simple task families and finds that two known in-context-learning phenomena appear in the recovered latent beliefs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PMC validity conditions are unverified; the OOD linear-regression mismatch leaves the central latent-space interpretation conditional on an assumption the paper explicitly does not test.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: PMC's interpretability rests on representation-theorem conditions that the authors explicitly do not verify. My stress-test agrees and goes one step further by pointing out that the paper's own 'agreement certifies' argument cannot cover the OOD regime where the method fails to match either reference, leaving the two explanations entangled. This is not a new inconsistency; it is the same unverified premise. The paper is transparent about the gap and frames the contribution as a tool conditional on future verification, which is why CONDITIONAL is an appropriate verdict rather than REJECT. My read does not alter the reader's judgment. A direct empirical check of the martingale/c.i.d. condition on the trained linear-regression BFT would be the decisive experiment: if the condition fails, the central interpretability claim is not supported; if it holds, the OOD mismatch becomes evidence of a genuinely non-reference posterior, which would strengthen the paper. Until such a test is run, the central claim remains conditional.","tokens_in":26604,"tokens_out":3752,"duration_ms":45583,"concrete_test":"Test the c.i.d./martingale condition on the linear-regression BFT, independent of reference posteriors. For a fixed prefix length n and token bucket A, compare the model's next-token distribution at the prefix, P_phi_n(A | y_{1:n}), with the average of the one-step-ahead distributions after drawing y_{n+1} from the model: E[P_phi_{n+1}(A | y_{1:n+1}) | y_{1:n}]. The difference is the conditional drift. Estimate it with many one-step rollouts over many random prefixes of lengths n=8, 16, 32, 64 and across all 256 discretized response bins. If the drift does not decay toward zero as n grows, asymptotic exchangeability fails for P_phi, and the OOD PMC mismatch cannot be interpreted as an implicit posterior. If the drift does decay, rerun the OOD posterior comparison with larger rollout length N to see whether the mismatch persists; persistence would indicate a genuine non-reference belief r","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that PMC recovers the BFT's implicit prior/posterior over the latent task. This requires P_phi—the joint law induced by the trained transformer—to satisfy the corresponding representation theorem: exchangeability for balls-and-urns and linear regression, and Markov exchangeability plus recurrence for Markov chains. Appendix C states plainly: 'We do not verify any of these conditions for the BFTs studied here.' This is not a minor gap: if the conditions fail, the latent task theta is not a well-defined function of the infinite path, and PMC samples are merely empirical statistics of generated rollouts, not draws from an implicit posterior. The unexplained out-of-distribution linear-regression mismatch is precisely where the ambiguity bites: the paper gives two candidate explanations—the model genuinely departs from both reference posteriors, or P_phi violates the PMC conditions—and explicitly says it cannot tell which is operative. The 'agreement certifies the PMC output' argument covers only regimes where a known reference is matched; it cannot certify the method in regimes where no reference matches, which is the regime that matters for interpretability. Without verifying the conditions, the central contribution—latent-space access to the model's beliefs—is conditional: the method may be recovering a useful projection of the model's predictive behavior, but it is not established that this projection is the model's internal belief. This is the load-bearing soft spot in an otherwise careful and honest paper.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes predictive Monte Carlo (PMC) as a black-box interpretability tool for Bayes-filtered transformers (BFTs), claiming to recover the implicit prior and posterior over the latent task using only next-token generation. The authors train BFTs on three synthetic families (balls-and-urns, linear regression, Markov chains), apply PMC, and compare the recovered latent distributions against memorizing (empirical prior) and generalizing (population prior) baselines using energy distance and sliced Wasserstein distance. They report that previously observed phenomena (task-diversity threshold, transient generalization) appear in latent space. The central validity requirement—that the transformer-induced joint law P_phi satisfies exchangeability or Markov exchangeability/recurrence—is explicitly not verified, and an out-of-distribution linear-regression mismatch is left unexplained.","tokens_in":26908,"tokens_out":5741,"duration_ms":57823,"significance":"If the central claim holds, the paper makes a useful methodological contribution: it unifies several synthetic ICL studies as k-Markov exchangeable BFTs, moves evaluation from prediction space to latent space, and offers a model-agnostic, non-circular way to probe internalized beliefs, with external analytic references as checks. The experiments are carefully specified, the appendices are thorough, code is provided, and the paper is unusually honest about its limitations. However, the significance is heavily conditional on PMC validity, which the paper does not establish. The 'agreement certifies PMC output' argument covers only regimes with known references, not the regime where the tool is most needed. The explicit statement that 'We do not verify any of these conditions for the BFTs studied here' is a central caveat, not a peripheral one, and it should gate the strength of the claims.","major_comments":[{"comment":"PMC samples are interpretable as draws from the BFT's implicit posterior only if P_phi satisfies the hypotheses of de Finetti's or Diaconis–Freedman's theorem. Appendix C states: 'We do not verify any of these conditions for the BFTs studied here.' Without verification, finite-rollout empirical frequencies, OLS estimates, and transition counts are not established as draws from an implicit posterior. The agreement on prior and in-distribution prompts is encouraging but certifies only those regimes; the OOD linear-regression regime, where no reference matches, is exactly where the interpretation is needed. Please either supply diagnostics for P_phi (e.g., predictive martingale or conditional-identically-distributed checks for the trained models) or weaken the abstract and contributions to claim recovery only in validated regimes.","section":"§4 Validity; Appendix C"},{"comment":"On an out-of-distribution prompt, the PMC-recovered posterior matches neither the memorizing nor the generalizing baseline. The paper states: 'Two explanations are consistent with this mismatch: the trained model may genuinely depart from both reference posteriors, or the sufficient conditions for PMC validity ... may fail for P_phi. We do not know which explanation is operative.' This unresolved ambiguity is load-bearing because it leaves open that PMC samples are merely rollout statistics rather than implicit beliefs. A concrete test would be to check whether the model's one-step predictives along rollouts satisfy an asymptotic-exchangeability or martingale-type condition, or to compare PMC against an independent latent-space probe on the same OOD prompts. Without such a test, the central claim is not established beyond matched regimes.","section":"§5 Linear regression; Fig. 9 / App. F"},{"comment":"The Markov-chain baselines are not the exact posterior under the generative process. Appendix H notes that 'the exact posterior under the generative process carries a stationary-probability factor that the baselines omit, one of ℓ=8 likelihood contributions.' Because both memorizing and generalizing baselines omit the P(y1|Q) factor, the latent-space distances and the reported 'looser match' may reflect baseline misspecification rather than BFT behavior. This weakens the validation of PMC in the 1-Markov family. Please quantify the effect (e.g., by also computing baselines with the stationary factor) or justify the omission specifically for the comparisons being made.","section":"Appendix H; Eq. (7)"}],"minor_comments":[{"comment":"The KL in Eq. (4) is written between P(Y1:n) and P_phi(Y1:n), but P_phi is initially defined as a law on Y^∞. Please clarify that the KL is taken between the n-dimensional marginals.","section":"§3, Eq. (4)"},{"comment":"The statement 'Throughout the experiments, P denotes the joint law (2) under Π = Π∞' overloads the symbol P, which was used generically in Section 2. Consider writing P_∞ for the evaluation distribution to avoid ambiguity in Eqs. (8) and (14).","section":"§5, notation"},{"comment":"The Autoregressive-PFN attention mask is described in words and shown in Figure 8, but it is not stated whether the first even-position token (x1,0) attends only to itself and whether its output logits are used for predicting y1. Please make this explicit.","section":"Appendix F, Fig. 8"},{"comment":"The explanation for the absent prediction-space transition in balls-and-urns says that 'in early positions in the sequence, the transformer behaves closer to the memorizing predictor.' This claim is not supported by a displayed figure; please add evidence or rephrase as a hypothesis.","section":"Appendix G"},{"comment":"The Markov-chain PMC evaluation uses only 16 prompts per source, because the memorizing baseline likelihood scales with M. This is reasonable, but the small prompt count should be acknowledged as a source of variance in the reported latent-space distances.","section":"Appendix H"}],"recommendation":"major_revision","confidential_remarks":"The paper is well-executed and thoroughly honest, but the abstract's central claim—'PMC returns an approximation to the implicit prior and posterior over the latent task'—exceeds what is established, given the explicitly unverified PMC validity conditions and the unexplained OOD mismatch. If the authors can add at least one empirical validity check (e.g., predictive martingale tests for the trained models' predictives) or sharply scope the claims, I would support publication. The inclusion of code, detailed appendices, and explicit acknowledgment of alternative explanations are strong points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should read this one if you care about in-context learning and interpretability. The paper takes predictive Monte Carlo, an established Bayesian tool, and applies it to Bayes-filtered transformers to recover the model's implicit prior and posterior over the latent task. That application is new, as is the tidy framing of balls-and-urns, linear regression, and Markov chains as k-Markov exchangeable families. The experiments are carefully run, the code is available, and the paper reproduces the task-diversity threshold and transient generalization in latent space, which is a useful confirmation that these phenomena are not just artifacts of prediction-space metrics.\n\nWhat it does well: the writing is clear, the baselines are sensible, and the authors are unusually frank about their limitations. They do not hide the weak spot. In Appendix C they state plainly that they do not verify the exchangeability or Markov-exchangeability conditions for the trained BFTs. And on out-of-distribution linear regression prompts, the recovered posterior matches neither baseline, with the authors admitting they cannot tell whether the model genuinely departs from both references or whether PMC validity fails. That ambiguity is load-bearing: if the conditions fail, the PMC samples are just rollout statistics, not draws from an internal belief. The in-distribution agreement with known references is reassuring in those regimes, but it does not certify the method in the regime that matters most for interpretability, namely where no reference is available.\n\nThe Markov chain results are also looser than the other two families; the authors propose plausible reasons (higher-dimensional latent, heavier load) but the gap remains. On novelty, the 'first application' claim is modestly strained by prior BPI work on TabPFN, though the authors cite it and the specific use here differs.\n\nAll that said, the paper is honest and technically competent. The central claim in the abstract—that PMC answers the interpretive question directly in latent space—is stronger than the evidence supports, and I would want the authors to either verify the conditions on at least one realistic case or soften the claim to 'approximate latent-space diagnostics.' But this is a paper that a good referee can work with; it is not a rejection. I would send it to review, primarily because the authors have done the hard work of setting up a clean testbed and have clearly stated what remains open.\n\nFor your reading group: maybe. It is a solid within-subfield contribution, not a paradigm shift. I would cite it if I worked on ICL or on BPI for transformers.","headline":"Worth sending to review: a clear, honest paper that moves ICL evaluation into latent space with PMC, but the central interpretability claim is conditional on unverified representation-theorem conditions that the authors explicitly flag.","tokens_in":27388,"tokens_out":1818,"would_cite":true,"duration_ms":22056,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","60G09","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Predictive Monte Carlo recovers the implicit prior and posterior a Bayes-filtered transformer has internalized, using only the model's own next-token generation.","keywords":["Bayes-filtered transformer","predictive Monte Carlo","in-context learning","latent-space interpretability","exchangeability","Markov exchangeability","task-diversity threshold","transient generalization"],"falsifier":"Roll out one very long sequence from the trained linear-regression BFT and test whether its sequence of one-step predictive distributions converges almost surely to a random measure (equivalently, whether the predictives form a martingale). If they do not converge, PMC samples lose their status as draws from an implicit posterior, and the paper's unexplained out-of-distribution mismatch is a symptom of invalidity rather than of genuinely non-Bayesian belief. A complementary check: build two priors with identical posterior means but different shapes and see whether PMC, unlike prediction-space","tokens_in":26494,"feed_emoji":"🎲","tokens_out":17358,"duration_ms":126871,"temperature":0.7,"pith_summary":"This paper is trying to establish that a Bayes-filtered transformer — a transformer trained on sequences generated by first drawing a latent task from a prior and then sampling observations conditional on that task — internalizes a definite prior and posterior over that latent task, and that predictive Monte Carlo (PMC) can recover them using nothing but the model's own next-token generation. The motivation is that the usual way of interpreting such models, by comparing their predictions against candidate reference posteriors, is ambiguous: distinct priors and posteriors can yield identical prediction means, so the interpretive question is moved into latent space, where the beliefs actually live. The method rolls out sequences from the trained model, reads the latent task off each completed rollout (empirical frequencies, a regression fit, or transition counts), and treats the empirical distribution of these readings as the model's implicit prior (from an empty prompt) or implicit posterior (from a given prompt). Applied to three task families covering both exchangeable and Markov-exchangeable processes (the paper's 0-Markov and 1-Markov cases), the paper reproduces the task-diversity threshold and transient generalization directly in latent space, with the recovered prior and posterior matching closed-form references on in-distribution prompts.","feed_headline":"Rollouts reveal what a Bayes-filtered transformer truly believes","feed_subtitle":"Predictions can look identical under two priors; sampling a model's own continuations tells them apart in latent space.","key_machinery":"The load-bearing mechanism is the pairing of a representation theorem with a rollout estimator. For the exchangeable families (balls-and-urns, linear regression), the representation theorem for exchangeable sequences makes the latent task — urn proportions, regression vector — a deterministic function of the infinite sample path, realized as empirical frequencies or an ordinary least squares fit. For the Markov-chain family, the analogous theorem for Markov-exchangeable chains makes the transition matrix such a function, given a recurrence condition. Predictive Monte Carlo replaces the infinite path with a finite rollout drawn autoregressively from the trained transformer using forward passe","core_discovery":"A trained Bayes-filtered transformer is, in the ideal limit, the Bayesian posterior predictive of its pretraining model; real training breaks that ideal. The paper's central claim: predictive Monte Carlo can read off what prior and posterior this particular model has actually internalized. Generate continuations using only the model's next-token probabilities; read the latent task off each completed continuation (empirical frequencies, OLS fit, transition counts); the empirical distribution of those readings is the implicit prior (empty prompt) or implicit posterior (given prompt). Representation theorems — for exchangeable and Markov-exchangeable sequences — make the latent task a determini","pith_inferences":["Beyond the paper's claims: the unexplained mismatch between the PMC posterior and both reference posteriors on out-of-distribution linear-regression prompts is itself testable — if the validity conditions can be verified, that mismatch would indicate the model's latent beliefs genuinely depart from both baselines when extrapolating, which would be a substantive finding about how these models gener","The paper's suggested persona-tracking experiment generalizes: any suspected belief drift during finetuning — for example misalignment induced by narrow finetuning — could be monitored by running PMC before, during, and after the intervention, turning a qualitative suspicion into a measurable shift in a latent distribution.","The weaker match in the Markov-chain family (a high-dimensional latent task) suggests a practical scaling rule: the number of rollouts needed for a faithful latent-space readout grows with the latent dimension, which practitioners can calibrate by comparing PMC samples to known references.","Because PMC produces a target distribution without needing ground truth, it offers a natural validation target for latent probes of the residual stream, connecting predictive-level beliefs to internal representations — a link the paper leaves as future work."],"forward_implications":["The task-diversity threshold — where a model shifts from memorizing to generalizing behavior as the number of pretraining tasks grows — is a real shift in the model's latent beliefs, not an artifact of the measurement space.","Transient generalization at intermediate task diversity — the model first approaching the generalizing posterior and later drifting to the memorizing one — shows up in the recovered posterior, so it is a belief-level phenomenon.","Because PMC needs only forward passes, it extends to any Bayes-filtered transformer whose next-token distribution is available, including models too large or too opaque for closed-form posterior analysis.","Where the PMC-recovered distribution matches a closed-form reference (the training prior; the memorizing posterior on in-distribution prompts), the match itself certifies the recovery without requiring verification of the abstract sufficient conditions.","Unifying the three task families as k-Markov exchangeable Bayes-filtered transformers gives a common vocabulary for studying in-context learning phenomena across different generative mechanisms."],"fun_headline_variants":["Rollouts reveal the true prior inside a Bayes-filtered transformer","Sample a transformer's own predictions to read its latent beliefs","Predictive Monte Carlo: exposing the implicit posterior of BFTs","What does a BFT truly believe? Ask its own rollouts","Sampling continuations uncovers a transformer's hidden assumptions"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire latent-space interpretation rests on the unverified premise that the trained transformer's own sequence distribution satisfies the relevant representation theorem (exchangeability for the 0-Markov families; Markov exchangeability plus recurrence for Markov chains); the paper states explicitly in its appendix that it does not verify these conditions for the models it studies.","fun_headline_variants_meta":{"raw":{"variants":["Rollouts reveal the true prior inside a Bayes-filtered transformer","Sample a transformer's own predictions to read its latent beliefs","Predictive Monte Carlo: exposing the implicit posterior of BFTs","What does a BFT truly believe? Ask its own rollouts","Sampling continuations uncovers a transformer's hidden assumptions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000511,"raw_usage":{"total_tokens":2338,"prompt_tokens":772,"completion_tokens":1566,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":1480}},"tokens_in":516,"tokens_out":1566,"duration_ms":10660,"temperature":1.0,"reasoning_tokens":1480,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T19:07:48.785342+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Roll out one very long sequence from the trained linear-regression BFT and test whether its sequence of one-step predictive distributions converges almost surely to a random measure (equivalently, whether the predictives form a martingale). If they do not converge, PMC samples lose their status as draws from an implicit posterior, and the paper's unexplained out-of-distribution mismatch is a symptom of invalidity rather than of genuinely non-Bayesian belief. A complementary check: build two priors with identical posterior means but different shapes and see whether PMC, unlike prediction-space","supporting_citations":[],"review_version":1}