{"id":"b9db8fc6-8e1f-4509-a29f-bb4a4a8bb7f5","arxiv_id":"2412.10425","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"An active inference controller selects prompts and search actions for an LLM agent, with experiments showing learned structure in observation matrices and an exploration-to-exploitation shift.","lead":"This paper wraps an LLM-based research agent in an active inference layer that selects prompts and search actions by minimizing free energy. The authors report learned structure in the agent's observation matrices and an exploration-to-exploitation shift, but provide no quantitative evaluation or baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The derivation of the expected free energy in Section 3.3 rests on a false Bayes identity (Eq. 10), so the paper's theoretical basis for the claimed exploration-exploitation behavior is unsupported.","rationale":"The reader's rationale already mentions an incorrect derivation equation, but the weakest_assumption field focuses on evaluator reliability. I agree that evaluator reliability is a serious issue: because the 'environment' is the paper's own GPT-4o-mini evaluation pipeline, the learned observation matrices may merely mirror the evaluator's scoring tendencies. However, I consider the incorrect derivation in Section 3.3 to be the more load-bearing concern because it attacks the framework itself, independent of whether the environment is objective. The central claim is that the agent exhibits 'sophisticated exploration-exploitation behavior emerging from the free energy minimization framework.' The only mechanism connecting free-energy minimization to the reported behavior is the expected free energy objective used in Section 4.5. If the derivation of that objective is invalid, then the observed action patterns are not shown to be the result of the claimed principled process. The empirical evidence alone—heatmaps and timelines, without baselines or quantitative metrics—cannot establish that link. I therefore keep the reader's REJECT verdict, with no adjustment needed. The concern is concrete and testable: replacing Eq. (10) with the correct Bayes identity should reveal whether Eqs. (13)-(14) are derivable. If the final formula happens to be correct via another route, the paper still needs to provide that derivation, since the submitted text does not. Additionally, the evaluator-reliability concern remains valid and could be addressed by comparing learned matrices against held-out evaluations or human-rated ground truth, but the mathematical issue is the primary blocker.","tokens_in":9073,"tokens_out":6079,"duration_ms":55871,"concrete_test":"Analytically re-derive Eq. (13) from Eq. (8) using the correct Bayes identity q(s|o,π)q(o|π) = q(o|s,π)q(s|π) instead of Eq. (10). If Eq. (13) cannot be obtained without additional unstated assumptions, the EFE derivation fails. As a numerical cross-check, choose a 2-state, 2-observation POMDP with q(s=1)=0.4, q(o=1|s=1)=0.8, q(o=1|s=0)=0.2, and compute both sides of Eq. (10); they will differ, demonstrating that the asserted equality is false.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that action selection reveals sophisticated exploration-exploitation behavior depends on policies being chosen by minimizing expected free energy (Section 4.5). The only mathematical justification for this objective is the derivation in Section 3.3, which relies on the asserted equality q(s|π)q(s|o,π) = q(o|π)q(o|s) (Eq. 10). This is not Bayes' theorem: the correct identity is q(s|o,π)q(o|π) = q(o|s,π)q(s|π). Substituting the correct identity into the step from Eq. (12) to Eq. (13) does not produce the paper's result; instead, the terms involve q(o|π)q(s|o,π) on the left, not q(s|π)q(s|o,π). Consequently, Eqs. (13) and (14) do not follow as derived. Because Section 4.5 explicitly uses this G as the policy selection objective, the observed action patterns are not demonstrably produced by free-energy minimization. The empirical sections provide only visual inspection of matrices and timelines, with no baseline, no error bars, and no quantitative link between the computed G values and the chosen actions. Thus the strongest form of the central claim—'sophisticated exploration-exploitation behavior emerging from the free energy minimization framework'—rests on an invalid derivation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes an active inference layer as a cognitive controller for a multi-LLM research agent. The agent maintains discrete state factors for prompt, search, and information states, receives seven observation modalities (quality scores and information-state observations), and selects actions by minimizing an expected free energy (EFE) objective. The authors report that the learned observation matrices develop 'structure,' that the EFE landscape differentiates policies over time, and that action selection shifts from search-dominated exploration to prompt-dominated exploitation. The paper includes a derivation of EFE, a description of the generative model and learning rules, and qualitative results based on visual inspection of heatmaps and timelines. A GitHub repository is cited as containing the full implementation.","tokens_in":9459,"tokens_out":2924,"duration_ms":28553,"significance":"If the central claims were established, the paper would offer a useful proof-of-concept that active inference can be applied to high-dimensional, language-driven agent control, with a principled objective balancing information gain and goal-seeking. The manuscript is honest about the fixed state-space limitation and releases code, which supports reproducibility. However, the significance is currently limited by two load-bearing problems: the EFE derivation in Section 3.3 relies on an incorrect probabilistic equality, and the empirical evaluation in Section 5 is entirely qualitative and self-referential, with no quantitative metrics, baselines, error bars, or external ground truth. The claimed 'accurate models of environment dynamics' and 'sophisticated exploration-exploitation behavior' are therefore not established by the evidence presented.","major_comments":[{"comment":"The derivation of the expected free energy is invalid. The asserted equality q(s|π)q(s|o,π) = q(o|π)q(o|s) in Eq. (10) is not Bayes' theorem; the correct identity is q(s|o,π)q(o|π) = q(o|s,π)q(s|π). Substituting the correct identity into the step from Eq. (12) to Eq. (13) yields terms involving q(o|π)q(s|o,π), not the expression the authors obtain. Consequently, Eqs. (13) and (14) do not follow, and the policy-selection objective Gπ used in Section 4.5 and Eq. (24) is not derived. Because the paper explicitly attributes the observed exploration-exploitation pattern to free-energy minimization, this invalid derivation undermines the central theoretical claim.","section":"Section 3.3, Eqs. (10)-(14)"},{"comment":"The empirical claim that the agent learns 'accurate models of environment dynamics' is not supported by any external validation. The only source of observations is the GPT-4o-mini evaluator described in Section 4.4, which is part of the agent's own pipeline. The learned observation matrices in Figure 2 may simply mirror the evaluator's scoring tendencies rather than objective environment structure. There is no ground truth, no held-out prediction test, and no quantitative metric (e.g., predictive log-likelihood, calibration, or comparison to a random learner). Without such validation, the 'emergent structure' is not evidence of accurate environment modeling.","section":"Sections 4.4 and 5.1"},{"comment":"The reported exploration-exploitation transition is not demonstrated to emerge from free-energy minimization, because the preference matrix C (Section 4.2.3) strongly encodes a preference for 'detailed info' (value 64.0) over 'no info' (-32.0) and the action precision is high (α=16.0). These hand-set preferences, together with the invalid Gπ derivation, could trivially produce the observed shift from search to prompt actions. The paper provides no ablation, no sensitivity analysis, and no comparison against a baseline that uses random action selection or a simpler heuristic (e.g., always search until information state is high, then exploit). The claim that the behavior is 'sophisticated' and 'emerges from the framework' requires such controls.","section":"Sections 4.5, 4.6, and 5.2-5.3"}],"minor_comments":[{"comment":"The notation `a_{t+1} <- a_t + η·(o_t ⊗ q(s_t)) ⊙ (A > 0)` appears to update the observation matrix A, not the action a; Eq. (22) uses `a_{t+1}^m` for Dirichlet concentration parameters. Please align the pseudocode with the mathematical notation.","section":"Algorithm 1, line 19"},{"comment":"All figures lack axis labels, colorbar scales, and units. For example, Figure 3 shows EFE values but no quantitative scale, and Figure 5 uses 'prompt IDs' and 'search IDs' without defining the mapping to the 33 prompt and 11 search states. The qualitative claims would be more interpretable with labeled axes and a legend.","section":"Figures 1-5"},{"comment":"Several references are malformed or incomplete: the 'graph websearch agent' GitHub entry contains a space in the repository name, the Chou/Champion et al. reference is incomplete, and two different works by Sun et al. (2023) share the same arXiv identifier. Please verify and correct the bibliography.","section":"References"},{"comment":"The GitHub URL ends with 'Adaptat', which appears truncated; please provide the full, working URL and ideally a DOI or versioned archive so that the code can be accessed reproducibly.","section":"Data Availability"}],"recommendation":"reject","confidential_remarks":"I did not attempt to access the GitHub repository, but the URL appears truncated. The manuscript is a prototype with a promising direction, but the invalid EFE derivation and the absence of any quantitative evaluation make the central claims unsupported. A resubmission would need to correct the derivation (or cite an existing correct derivation) and add a rigorous empirical study with baselines, ground-truth comparisons, and sensitivity analyses. As it stands, the paper does not meet the bar for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is applying active inference to select prompts and search actions for a multi-LLM research agent, and the author does ship code and a fairly detailed generative-model setup. That is real, and it distinguishes this from the usual LLM-agent papers. The related work is honest and the state-factor decomposition (prompt, search, information) is a reasonable first cut. I also think the idea of using an active-inference layer instead of yet another RL loop is worth taking seriously.\n\nThe soft spots are real, though. The stress-test note is right: Eq. 10 in Section 3.3 is not Bayes' theorem. The correct identity is q(s|o,pi)q(o|pi) = q(o|s,pi)q(s|pi), and what the paper writes instead does not follow. Since the expected-free-energy objective in Section 4.5 is the stated basis for the claimed exploration-exploitation behavior, this is a load-bearing error, not a typo. The author cites standard active-inference sources and the formula itself is familiar, so a sympathetic reader could repair the derivation, but as written the theoretical justification does not hold.\n\nThe empirical section is the bigger gap. There are no baselines, no quantitative metrics, no error bars, and no external ground truth. The \"environment\" is the paper's own GPT-4o-mini evaluator, so the claim that the agent learned \"accurate models of environment dynamics\" is largely self-referential: the learned matrices may just mirror the evaluator's scoring tendencies. Figures 2-5 show structure, but visual inspection alone does not establish that the agent learned anything useful, nor that the action pattern is driven by free-energy minimization rather than by the hand-chosen preference matrix and learning rates. The thermodynamic framing in the introduction is also mostly rhetorical; it does not do quantitative work in the experiments.\n\nWho is this for? Researchers working on adaptive LLM agents and maybe the active-inference community. It is an exploratory paper with a new application and reproducible code, but the current evidence would not justify accepting it as a strong archival result. It deserves a serious referee, though: the idea is novel enough and the code is available, and external feedback could turn this into something useful. I would send it to review rather than desk-reject, with the expectation that the author must add baselines, quantify the results, and fix the derivation. My own verdict is skeptical, but this is a paper to engage with, not dismiss.","headline":"A genuinely new application of active inference to prompt and search selection for LLM agents, with released code, but the evidence is qualitative and the expected-free-energy derivation contains a false identity, so the central claims are not yet supported.","tokens_in":9872,"tokens_out":2016,"would_cite":false,"duration_ms":19618,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An active inference layer can learn which prompts produce good LLM output.","keywords":["active inference","free energy principle","large language models","prompt adaptation","exploration-exploitation","Bayesian learning","multi-agent systems","expected free energy"],"falsifier":"Run the same active inference loop with two different evaluator LLMs (or with human judges) on the same prompt and search space; if the learned observation matrices and the ranking of preferred prompts diverge substantially across evaluators, the claim that the agent has learned accurate environment dynamics rather than evaluator idiosyncrasies is refuted.","tokens_in":8891,"feed_emoji":"🧠","tokens_out":4941,"duration_ms":41538,"temperature":0.7,"pith_summary":"This paper argues that active inference, a Bayesian scheme that minimizes variational free energy by updating beliefs and choosing actions, can act as an adaptive layer over a multi-LLM research agent. The agent models its environment with three hidden state factors—prompt, search, and information state—and receives seven quality-scored observation modalities, then learns which prompt combinations and search actions produce strong outputs. The authors report that the learned observation matrices develop clear structure and that action selection shifts from search-heavy exploration to prompt-focused exploitation over time. If this holds, LLM-based agents could adapt their own prompts and strategies without retraining, guided by a principled information-seeking objective.","feed_headline":"LLM agents learn which prompts work via active inference","feed_subtitle":"A Bayesian layer converts quality scores into prompt choices and shifts from search to exploitation.","key_machinery":"The engine of the system is the active inference generative model: observation matrices A that map hidden prompt, search, and information states to seven quality modalities, transition matrices B that govern state changes under actions, preference matrix C encoding which observations are desirable, and Dirichlet-based parameter learning that updates A and B after each observation. Policy selection minimizes expected free energy Gπ, decomposed into information gain and pragmatic value, and applies a softmax over policies. This machinery is what converts raw LLM quality scores into beliefs about which prompts work and what to try next.","core_discovery":"On its own terms, the paper claims that an active inference agent placed above an LLM-based research agent can learn an accurate generative model of the prompt/search environment from structured evaluation scores alone. The evidence offered is the emergence of non-uniform structure in the learned observation matrices, which now distinguish high-scoring prompt and search states from poor ones, and a temporal action pattern in which early search-dominant policies give way to targeted prompt testing. The paper presents this as a demonstration that expected free energy minimization generalizes active inference from low-dimensional control problems to high-dimensional, language-driven agent adaptation.","pith_inferences":["Because the 'environment' is defined by the GPT-4o-mini evaluator, the learned structure may reflect the evaluator's scoring tendencies; a fair test would compare learned prompt rankings against human or held-out ground truth.","A neighbouring question the paper leaves open is whether EFE-guided prompt selection beats simpler baselines such as random search or upper-confidence-bound bandits; the reported experiments do not include that comparison.","Scaling to larger prompt spaces would likely require factored transition tensors or hierarchical models, since the current 33×33×33 prompt transition tensor grows cubically with the number of prompt states.","The exploration-first pattern may be shaped by the strong preference prior on detailed information states; adjusting that prior could shift the exploration-exploitation balance and is a testable extension."],"forward_implications":["LLM agents can adapt their own prompts and search strategies during deployment without gradient updates or fine-tuning.","The early exploration-to-exploitation shift emerges from the free energy objective, not from a hand-coded schedule.","The learned observation matrices provide an inspectable record of which prompt and search states the agent believes produce high-quality outputs.","The framework requires only structured, machine-readable quality feedback, making it compatible with LLM evaluators that output JSON scores."],"supporting_citations":[{"why":"Supplies the variational free energy and belief-propagation foundation the agent's message passing is built on.","marker":"Friston et al., 2017"},{"why":"Gives the expected free energy expression the paper uses as the policy objective.","marker":"Smith et al."},{"why":"Supplies the derivation connecting that expected free energy to risk and ambiguity.","marker":"Champion et al."},{"why":"Motivates information-gain-driven exploration in active inference, which the search-heavy early phase exemplifies.","marker":"Schwartenbeck et al., 2019"},{"why":"Grounds the thermodynamic framing of belief updating that the paper's title and introduction invoke.","marker":"Fields et al., 2023"}],"fun_headline_variants":["LLM agents self-organize prompts via active inference","Bayesian layer lets LLM agents adapt prompts on the fly","Free energy drives LLM agents to learn prompt strategies","Active inference tunes LLM prompts from quality scores","Self-organizing LLM agents use Bayesian adaptation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim collapses if the LLM evaluator's structured quality scores are not reliable, consistent observations of a fixed environment, because the agent learns only from those scores and there is no external ground truth in the experiments.","fun_headline_variants_meta":{"raw":{"variants":["LLM agents self-organize prompts via active inference","Bayesian layer lets LLM agents adapt prompts on the fly","Free energy drives LLM agents to learn prompt strategies","Active inference tunes LLM prompts from quality scores","Self-organizing LLM agents use Bayesian adaptation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000153,"raw_usage":{"total_tokens":1155,"prompt_tokens":841,"completion_tokens":314,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":238}},"tokens_in":457,"tokens_out":314,"duration_ms":3264,"temperature":1.0,"reasoning_tokens":238,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:37:40.858132+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same active inference loop with two different evaluator LLMs (or with human judges) on the same prompt and search space; if the learned observation matrices and the ranking of preferred prompts diverge substantially across evaluators, the claim that the agent has learned accurate environment dynamics rather than evaluator idiosyncrasies is refuted.","supporting_citations":[],"review_version":1}