{"id":"f6d7c1c5-907b-4dd1-8dcd-56012ce43b17","arxiv_id":"2608.12942","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An online agent harvesting work from a correlated quantum state stream has a computable optimal policy, and its efficiency loss relative to unconstrained extraction equals the entropy increase from sequential measurements, called causal dissipation.","lead":"This paper defines a new quantity, time-ordered free energy (TOFE), the maximum work an agent can harvest from a stream of quantum states while acting online, without storing the whole stream. It gives a dynamic programming algorithm to find the best such strategy and shows that the efficiency loss relative to unconstrained work extraction equals an entropy quantity they call causal dissipation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's exact-equivalence claim is not established by Algorithm 1, which solves a finite-grid MDP; absent an error bound, the computed TOFE is a lower bound rather than Eq. (2).","rationale":"The reader's weakest assumption is the same one I find load-bearing: the gap between the continuous TOFE and the discretized DP. I agree with that assessment. I do not see a more fundamental flaw: the belief-state sufficiency argument (Appendix C) is standard, the causal dissipation formula is plausible, and the L=3,4 numerics show internal consistency. However, those numerics cannot certify exactness of the maximum. The correct remedy is to weaken Theorem 1 to a statement about the discretized MDP or to supply a certified bound; this is exactly the CONDITIONAL verdict already given. No change to the reader's verdict is needed.","tokens_in":29584,"tokens_out":7742,"duration_ms":81080,"concrete_test":"For the perturbed coin at p=0.2, r=0.1 and L=4, run Algorithm 1 with identical settings at N=M=300, then at N=M=600, then at N=M=1200, tracking the cumulative work value and the optimal policy at the initial belief mu_0=(1/2,1/2). If the value changes by more than numerical tolerance (e.g., 10^-6 k_BT), the finite grid is not computing Eq. (2). Additionally, for L=2 derive the continuous optimum analytically by maximizing V_1(mu_0)=max_{theta_1}[-D(xi||rho_{theta_1})+E_w D(xi_{eta_1(w)}||gamma)] over all theta_1 in [0,2pi) and compare with the grid value; any finite grid that does not contain the exact maximizing theta_1 gives a strictly smaller value, settling the exactness question.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing overclaim is the second sentence of Theorem 1: for a given discretized belief set K_DP and finite action set A, the DP value is declared 'mathematically equivalent' to the TOFE in Eq. (2). But Eq. (2) maximizes over all causal strategies, with actions ranging over the continuous family A={W_rho}; Algorithm 1 instead searches N grid beliefs and M actions and does not specify how post-update beliefs outside K_DP are represented. Appendix D proves only backward induction for the finite discretized MDP, and the cited convergence results [37,38] are approximation guarantees, not finite-N exactness certificates. Consequently the computed value is, in general, a lower bound on F_TO, and the reported f_TO, the hierarchy Eq. (10), and the numerical check in Fig. 5 are statements about the discretized quantity unless an explicit error bound is supplied. This affects the central claim because the advertised 'provably optimal agent strategy' and the exact causal dissipation identity both rely on F_TO being the true maximum.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies sequential work extraction from temporally correlated quantum states emitted by a hidden Markov source, under an online agent constrained by causality and lacking persistent quantum memory. It defines the time-ordered free energy (TOFE) as the maximum cumulative expected work over causal strategies, proposes a dynamic programming reduction to a belief-state MDP, and proves an identity expressing the work deficit relative to the global nonequilibrium free energy as a causal dissipation in terms of Shannon and von Neumann entropies. The framework is illustrated on a perturbed-coin model, with numerical evidence for finite horizons L=3 and L=4. The main derivations are self-contained: belief-state sufficiency, backward induction for the finite discretized MDP, an entropy-balance argument for the causal dissipation identity, and an epsilon-argument showing that belief memory updates can be implemented with arbitrarily small thermodynamic cost.","tokens_in":29753,"tokens_out":20926,"duration_ms":205894,"significance":"If the central claims hold, the paper provides an operational, causally constrained notion of free energy for temporal quantum sequences, a linear-time method (for fixed discretization) to compute optimal online extraction policies, and a quantitative information-theoretic measure of the cost of causality. The causal dissipation identity usefully extends discord-like quantities to multi-time quantum processes and connects thermodynamic resource theories with POMDP and computational-mechanics methods. The paper also contains a strong technical contribution in Appendix J, where near-reversible memory updates are shown to be implementable with arbitrarily small work penalty. The main weakness is the gap between the continuous TOFE definition and the finite-grid algorithm, which currently makes the advertised 'provably optimal' claim and the exact causal dissipation identity overstated.","major_comments":[{"comment":"The second sentence of Theorem 1 overstates what is proved. Eq. (2) maximizes over all causal strategies with a continuous belief simplex and a continuous action set, while Algorithm 1 solves a finite MDP over N grid points K_DP and M actions. Appendix D proves backward induction only for this finite discretized MDP; the cited convergence results [37,38] are asymptotic approximation guarantees, not exactness certificates for finite N. Algorithm 1 also does not specify how a Bayesian update that lands outside K_DP is represented (e.g., projection or interpolation). In general, the computed value is a lower bound on the TOFE of Eq. (2), not the maximum itself, so the reported f_TO, the hierarchy (10), and the numerical checks in Fig. 5 concern the discretized quantity until an explicit error bound is supplied. This gap also undermines the abstract's claim of a 'provably optimal agent strategy.'","section":"Policy optimization / Theorem 1 / Algorithm 1"},{"comment":"The exact causal dissipation identity inherits the same discretization problem. Eq. (13) is stated as an equality involving the continuous TOFE, but the minimization over policies Lambda is implemented in practice over the finite grid of Algorithm 1; if the grid misses the optimal belief trajectory, the entropy difference in Eq. (13) is not the true delta of Eq. (11). The numerical confirmation in Fig. 5 compares a simulated work deficit obtained with a 300-point grid and 300 actions against a causal dissipation obtained by differential evolution, so it does not resolve the exactness question. The theorem needs either a finite-grid error bound for both sides of Eq. (13) or a reformulation that makes the discretized object explicit.","section":"Causal dissipation (Theorem 2, Eq. (13)) and Fig. 5"},{"comment":"The statement that the method has time complexity O(L) is only valid for fixed discretization sizes N and M. Since the algorithm does not bound the error relative to the continuous TOFE, the sizes N and M may need to grow with L or with the desired accuracy, and no complexity estimate in terms of N, M, and L is given. The abstract's 'time complexity that scales linearly with sequence length' is therefore potentially misleading; please qualify it as the per-step cost for a fixed discretized MDP and state the dependence on grid size.","section":"Abstract and complexity claims"}],"minor_comments":[{"comment":"Please replace 'max' in Eq. (2) by 'sup' if the maximum may not be attained over the continuous strategy space, or justify attainment; otherwise the definition is formally problematic.","section":"Eq. (2)"},{"comment":"The claim that equal diagonal entries of xi_k in two different bases imply identical future statistics is insufficient, because future statistics depend on the diagonal entries of each sigma(x), not only on the mixture xi_k. Since Theorem 3 fixes a basis, this sentence is unnecessary for the proof and should be corrected or removed.","section":"Appendix G"},{"comment":"Please specify the projection or rounding rule for updated beliefs not contained in K_DP, or use the continuous belief update with interpolation, so that the transition probabilities in the finite-state DP are well defined.","section":"Algorithm 1"},{"comment":"Please report quantitative agreement (e.g., maximum absolute difference between the simulated deficit and the causal dissipation) and include a convergence check over the discretization sizes N and M, since both plotted quantities are approximate.","section":"Fig. 5 and Appendix I.3"},{"comment":"Minor typos and formatting: 'free energy(TOFE)' is missing a space, and the axis labels in Figs. 2, 3, and 5 appear garbled ('0 1 /2 1'); please correct these in revision.","section":"General formatting"}],"recommendation":"major_revision","confidential_remarks":"The discretization gap between Eq. (2) and Algorithm 1 is the main technical obstacle; if the authors can supply an explicit error bound or reformulate the claims as approximate with convergence guarantees, the paper would be a solid contribution. The numerical evidence should not be presented as exact verification of the unproved continuous statement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a real contribution with a fixable overclaim. It defines time-ordered free energy (TOFE) for online work extraction from hidden Markov quantum sources, gives a DP method that runs linear in the sequence length, and proves a causal-dissipation identity linking the work deficit to measurement entropy. The catch is Theorem 1's second sentence, which asserts exact equivalence to the continuous TOFE, but the algorithm works on a finite discretized belief set and no error bound is supplied. That is load-bearing and should be fixed before publication.\n\nWhat's new: TOFE as max over causal strategies; the L-partite causal dissipation theorem; and the answer to whether greedy local optimization is optimal—it isn't, and the DP policy shows when sacrificing immediate work for information pays. The belief-state reduction is standard POMDP machinery, but the adaptation to quantum thermodynamical agents is clean. The proof that belief states are sufficient statistics (Appendix C) is thorough. The perturbation trick in Appendix B to restore observability at O(epsilon^2) thermodynamic cost is nice.\n\nWhat's solid: the backward induction in Appendix D is correct for the discretized MDP; the numerical consistency check in Fig. 5 between simulated deficit and causal dissipation for L=3,4 is plausible; the stated limitations—non-degenerate Hamiltonian control cost unresolved, and agents knowing the HMM—are honestly reported.\n\nSoft spots, in proportion: the discretization gap is real. Theorem 1 claims 'mathematically equivalent' to Eq. (2), but Appendix D proves optimality only over the finite grid. The cited convergence results [37,38] are asymptotic value-function approximations, not finite-N exactness certificates. So the computed f_TO and the hierarchy Eq. (10) are, strictly, about the discretized quantity unless an error bound is added. That weakens the 'provably optimal' claim, though the underlying concepts survive. Minor: no code or error bars with the numerics, which makes the discretization and optimization details harder to verify. Also, the final reset protocol in Appendix J is a bit compressed, but the result seems okay.\n\nThis paper deserves a serious referee. I'd send it out, with a request that the authors either qualify Theorem 1 or provide explicit finite-discretization bounds. It's a useful paper for anyone working on quantum engines, Maxwell's demons, or thermodynamic costs of information.","headline":"A genuinely new measure and method for online quantum work extraction, with a load-bearing but fixable overclaim in Theorem 1 about discretization exactness.","tokens_in":30316,"tokens_out":4191,"would_cite":true,"duration_ms":35598,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The time-ordered free energy of a quantum sequence is attainable by a belief-based dynamic-programming agent, and the loss from causality is exactly an entropy gap the paper calls causal dissipation.","keywords":["time-ordered free energy","causal dissipation","dynamic programming","quantum work extraction","hidden Markov models","belief states","sequential measurement","thermodynamics of prediction"],"falsifier":"For a fixed perturbed-coin source, compare the dynamic-programming value on K_DP at increasing grid resolutions with a dense-grid or analytic optimum over the continuous belief simplex; if the DP value converges to a number strictly below the continuous optimum for some transition probability, state overlap, and horizon L, then the exact equivalence claimed in Theorem 1 fails.","tokens_in":29360,"feed_emoji":"⚡","tokens_out":5957,"duration_ms":57745,"temperature":0.7,"pith_summary":"This paper tries to establish that a causally constrained agent, one that can act on each quantum system as it arrives and keeps no persistent quantum memory, can still extract the maximum possible sequential work from a correlated quantum state stream. That maximum is captured by a quantity the paper introduces, the time-ordered free energy (TOFE), and the paper claims a dynamic-programming algorithm finds the optimal policy in time linear in the sequence length for any fixed action set. The central result is an exact accounting of what causality costs: the shortfall between the unconstrained non-equilibrium free energy and the TOFE equals a quantity the paper calls causal dissipation, expressed as an entropy difference. A sympathetic reader would care because it turns the intuitive trade-off between immediate energy harvest and predictive information into a provable optimality statement with a concrete computational route.","feed_headline":"Optimal sequential work policy is computable in linear time","feed_subtitle":"New proof: online agents can hit the sequential-work ceiling, losing only causal dissipation.","key_machinery":"The load-bearing object is the belief state, the posterior distribution over the hidden states of the hidden Markov model conditioned on the agent's action-observation history. Because this belief is a sufficient statistic for the past, the original history-dependent strategy can be replaced by a belief-dependent policy without loss of expected work. The paper then applies backward dynamic programming over a finite discretization of the continuous belief simplex, using the reward formula for expected work as the non-equilibrium free energy of the expected state minus a relative-entropy mismatch, and it shows the action search can be narrowed to eigenbases. The final identity that carries the thermodynamic conclusion is the entropy formula for causal dissipation, which connects the work deficit to the Shannon entropy of the work record and the von Neumann entropy of the final reduced state.","core_discovery":"The central claim is that the TOFE, defined as the maximum over all causal strategies of the expected cumulative extracted work, is not merely a formal benchmark: it is achievable by a policy that maps the agent's Bayesian belief about the hidden Markov source directly to an extraction action, and this optimal policy can be computed by backward dynamic programming in O(L) time for fixed action and belief sets. The paper further proves that the work deficit of the TOFE relative to the global non-equilibrium free energy is the causal dissipation, the minimum over policies of the accumulated Shannon entropy of observed work outcomes plus the von Neumann entropy of the final conditional state minus the entropy of the entire multi-time state. Operationally, sequential work extraction is equivalent to sequential quantum measurement, so the deficit is exactly the entropy increase produced by measuring the stream one system at a time. The paper illustrates the mechanism on a perturbed-coin qubit source, where the optimal policy sacrifices immediate work to buy predictive information when the source is predictable but not classical.","pith_inferences":["If the discretization gap in Theorem 1 can be quantified, the same dynamic-programming scheme would give certified lower bounds on the TOFE rather than merely approximate values.","The L=2 reduction of causal dissipation to quantum discord suggests that, for longer sequences, causal dissipation may be the multi-time analogue of measurement-induced disturbance; testing whether it matches a known multi-partite discord would link these results to the broader discord literature.","Because the TOFE is defined relative to a fixed action set, a natural next question is how the maximum grows when the action set is enlarged or when the agent is allowed a bounded quantum memory; the paper does not analyze that trade-off.","The non-degenerate Hamiltonian case is left with an unresolved control cost, so the clean entropy identity for causal dissipation would only survive if that control cost is accounted for elsewhere."],"forward_implications":["A causally constrained agent with no persistent quantum memory can provably attain the TOFE, making the benchmark operationally meaningful for online energy harvesters.","Maximizing long-term work generally requires deliberately choosing mismatched protocols at early steps to gain predictive information; greedy local optimization is suboptimal in predictable but non-classical regimes.","The gap between unconstrained and causal work equals causal dissipation, and for two time steps it reduces to quantum discord, so the thermodynamic cost of causality is an information-theoretic quantity.","Causal dissipation can have zero asymptotic rate even when finite-time dissipation is nonzero: in deterministic limits the rate gap vanishes.","The algorithm's runtime is O(L) for fixed action and belief sets, avoiding exponential scaling in the sequence length."],"supporting_citations":[{"why":"Supplies the individual-system work-extraction protocol whose work statistics match the Born-rule probabilties of a projective measurement.","marker":"[16]"},{"why":"Defines the predictive work-extraction engine and the greedy local-optimizing baseline that the TOFE generalizes.","marker":"[20]"},{"why":"Gives the two-term relative-entropy formula for average extracted work that underlies the reward function.","marker":"[28]"},{"why":"Provides the belief-state optimal-control framework used to replace history-dependent policies with belief-dependent policies.","marker":"[34]"},{"why":"Supplies the convergence argument that finite discretizations of the belief simplex approach the continuous optimal value function.","marker":"[37, 38]"},{"why":"Establishes quantum discord, which the paper shows reduces to causal dissipation for L=2.","marker":"[41, 42]"},{"why":"Supports the stationarity of the optimal policy used to define the asymptotic TOFE rate.","marker":"[39, 40]"}],"fun_headline_variants":["Time-ordered free energy: causal work ceiling hit in linear time","Causal agents can reach the sequential-work ceiling in O(L) steps","Introducing time-ordered free energy: measure of causally available work","Optimal causal work extraction from quantum sequences is linear-time"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that maximizing over the finite discretized belief set K_DP gives exactly the same value as maximizing over the continuous belief simplex in the definition of the TOFE, and the paper does not prove a finite-discretization error bound.","fun_headline_variants_meta":{"raw":{"variants":["Time-ordered free energy: causal work ceiling hit in linear time","Causal agents can reach the sequential-work ceiling in O(L) steps","Introducing time-ordered free energy: measure of causally available work","Optimal causal work extraction from quantum sequences is linear-time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00106,"raw_usage":{"total_tokens":4404,"prompt_tokens":862,"completion_tokens":3542,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":478,"completion_tokens_details":{"reasoning_tokens":3476}},"tokens_in":478,"tokens_out":3542,"duration_ms":22202,"temperature":1.0,"reasoning_tokens":3476,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:08:29.200963+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a fixed perturbed-coin source, compare the dynamic-programming value on K_DP at increasing grid resolutions with a dense-grid or analytic optimum over the continuous belief simplex; if the DP value converges to a number strictly below the continuous optimum for some transition probability, state overlap, and horizon L, then the exact equivalence claimed in Theorem 1 fails.","supporting_citations":[{"cited_title":"Work extraction and thermodynamics for individual quantum systems.Nature communications, 5(1):4185, 2014","cited_arxiv_id":null,"evidence_quote":"Supplies the individual-system work-extraction protocol whose work statistics match the Born-rule probabilties of a projective measurement."},{"cited_title":"Engines for predictive work extraction from memoryful quantum stochastic processes.Quan- tum, 7:1203, 2023","cited_arxiv_id":null,"evidence_quote":"Defines the predictive work-extraction engine and the greedy local-optimizing baseline that the TOFE generalizes."},{"cited_title":"Initial-state dependence of thermodynamic dissipation for any quantum process","cited_arxiv_id":null,"evidence_quote":"Gives the two-term relative-entropy formula for average extracted work that underlies the reward function."},{"cited_title":"The opti- mal control of partially observable markov processes over a finite horizon.Operations research, 21(5):1071–1088, 1973","cited_arxiv_id":null,"evidence_quote":"Provides the belief-state optimal-control framework used to replace history-dependent policies with belief-dependent policies."}],"review_version":1}