{"id":"1af8330e-8a1c-4c04-abc4-13b7446c27eb","arxiv_id":"2606.23933","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"fcTS corrects and reweights historical observations via drift models for linear, periodic, and regime-switching non-stationarities in contextual bandits, outperforming forgetting baselines in structured cases.","lead":"The paper proposes Flow-Corrected Thompson Sampling (fcTS) that transports and reweights past rewards using an explicit drift model to handle non-stationary linear contextual bandits. If effective, this could improve sample efficiency in changing real-world settings by reusing structured historical data instead of discarding it.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Accuracy of online drift model estimation for reward transport without net bias","rationale":"The reader's weakest assumption is exactly the load-bearing point. Because the original verdict was formed on the abstract alone, the full manuscript would need to contain either (a) an oracle ablation of the kind above or (b) explicit misspecification experiments showing graceful degradation; absent those, the claim remains conditional on drift-model fidelity. No other internal inconsistency is visible from the abstract description of the method.","tokens_in":1695,"tokens_out":352,"duration_ms":11952,"concrete_test":"In the five controlled case studies, replace the online drift estimator with an oracle that supplies the true drift parameters at each step; recompute the regret curves. If the oracle version shows substantially lower regret than the estimated-drift version (while both still beat baselines), the headline gains depend on drift-estimation quality rather than the correction idea itself.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that an explicit drift model (linear slope, phase, or changepoint) can be estimated online such that transported rewards reduce effective bias more than the estimation error adds it, after confidence weighting. If the model is even modestly misspecified or the online estimator (e.g., slope estimation in the linear case) has variance that the weight does not fully cancel, the posterior update can be pulled in the wrong direction, making fcTS inferior to pure forgetting. The abstract asserts largest gains precisely when structure is recurring, but this is the regime where small phase or regime errors compound across cycles. The closed-form linear-Gaussian updates do not protect against this; they only propagate whatever bias the transport step injects.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces Flow-Corrected Thompson Sampling (fcTS), a Bayesian algorithm for non-stationary linear contextual bandits that reuses historical observations by transporting past rewards to the current time via an explicit drift model (linear slope, periodic phase, or changepoint regime) and reweighting each transported sample by a transport-reliability factor. The resulting posterior remains closed-form under linear-Gaussian assumptions and is implemented via truncated sufficient statistics. The central empirical claim is that fcTS outperforms standard forgetting baselines (discounting, sliding windows, periodic restarts) across five controlled case studies and a semi-synthetic portfolio benchmark, with the largest gains occurring when non-stationarity exhibits recurring temporal structure.","tokens_in":1817,"tokens_out":462,"duration_ms":16809,"significance":"If the empirical claims and the net-bias-reduction assumption hold, the work would offer a practically useful template for structured non-stationarity that improves sample efficiency over pure forgetting. The closed-form updates and unified specialization to linear, periodic, and regime-switching cases are clear technical strengths. The absence of any reported quantitative metrics, confidence intervals, or statistical tests in the provided abstract, however, leaves the magnitude and reliability of the reported gains unassessable at present.","major_comments":[{"comment":"Abstract: the assertion that fcTS \"outperforms standard forgetting-based baselines ... with the largest gains in settings exhibiting recurring temporal structure\" supplies no numerical results, confidence intervals, statistical tests, baseline implementation details, or drift-model selection procedure, rendering the central empirical claim impossible to evaluate.","section":"Abstract"},{"comment":"Abstract (method template): the claim that transported rewards reduce effective bias more than online drift-model estimation error adds it is load-bearing for the superiority result, yet no analysis, bounds, or sensitivity experiments are referenced that would show when the online estimator (slope, phase, or changepoint) keeps net bias negative, especially under recurring structure where phase or regime errors can compound across cycles.","section":"Abstract"},{"comment":"Abstract (closed-form updates): while the linear-Gaussian posterior updates are stated to remain closed-form after transport and weighting, the manuscript provides no derivation or sufficient-statistic update rule that would allow a reader to verify that the confidence weighting exactly cancels the injected transport bias rather than merely propagating it.","section":"Abstract"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments on the abstract and supporting claims. We will revise the abstract to include quantitative results and add clarifying analysis and derivations to the main text and appendix as needed. We address each major comment below.","responses":[{"response":"We agree that the abstract should be more self-contained. In the revision we will add specific numerical results (e.g., average cumulative regret reductions with 95% confidence intervals across the five case studies and the portfolio benchmark), note the baseline implementations (discount factor grid, window sizes, restart periods), and briefly describe the drift-model selection procedure used in each experiment.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the assertion that fcTS \"outperforms standard forgetting-based baselines ... with the largest gains in settings exhibiting recurring temporal structure\" supplies no numerical results, confidence intervals, statistical tests, baseline implementation details, or drift-model selection procedure, rendering the central empirical claim impossible to evaluate."},{"response":"This point is well taken; the net-bias claim is central and currently supported only empirically. We will add a dedicated paragraph in Section 3 together with a short appendix deriving a simple bias bound for the linear-drift case and reporting sensitivity experiments that vary estimator error under periodic and regime-switching non-stationarity. These additions will clarify the operating regime in which net bias remains negative.","revision_made":"yes","referee_comment":"[Abstract] Abstract (method template): the claim that transported rewards reduce effective bias more than online drift-model estimation error adds it is load-bearing for the superiority result, yet no analysis, bounds, or sensitivity experiments are referenced that would show when the online estimator (slope, phase, or changepoint) keeps net bias negative, especially under recurring structure where phase or regime errors can compound across cycles."},{"response":"The closed-form updates and truncated sufficient-statistic recursions are derived in Section 3.2 and Appendix A. To improve accessibility we will insert a compact statement of the update equations immediately after the abstract claim and add a one-page step-by-step derivation in the appendix that explicitly shows how the reliability weight modifies the posterior covariance and mean, thereby addressing the bias-cancellation question.","revision_made":"yes","referee_comment":"[Abstract] Abstract (closed-form updates): while the linear-Gaussian posterior updates are stated to remain closed-form after transport and weighting, the manuscript provides no derivation or sufficient-statistic update rule that would allow a reader to verify that the confidence weighting exactly cancels the injected transport bias rather than merely propagating it."}],"tokens_in":1478,"tokens_out":563,"duration_ms":16742,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core idea here is a Thompson Sampling variant that corrects historical rewards for known drift patterns before feeding them into the posterior. It handles linear parameter drift with slope estimation, periodic cycles with phase alignment, and regime switches with changepoint memory, all while keeping updates closed-form under linear-Gaussian assumptions and using truncated sufficient statistics for efficiency.\n\nWhat stands out is the unified template across those three cases and the focus on reusing rather than discarding data when the non-stationarity has recurring structure. That matches a practical need in settings like portfolio selection where pure forgetting wastes information.\n\nThe main soft spot is exactly the one in the stress-test note: the method only helps if the online drift estimator (slope, phase, or changepoint) is accurate enough that transported rewards reduce bias more than they add it. Small errors in recurring regimes could compound and pull the posterior the wrong way, making fcTS worse than discounting or windows. The abstract claims the biggest gains precisely in those recurring cases, so the experiments need to show that the weighting actually protects against estimation variance.\n\nThis is for readers working on contextual bandits or adaptive control who already deal with non-stationary data and want a concrete alternative to standard forgetting. It is worth sending to peer review because the problem is real, the method is implementable, and the central assumption is testable even if the current evidence is thin on quantitative details.","headline":"fcTS reuses data via explicit drift transport in structured non-stationary bandits but its advantage hinges on whether online model estimation avoids net bias.","tokens_in":2305,"tokens_out":354,"would_cite":false,"duration_ms":16070,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Flow-Corrected Thompson Sampling reuses past rewards in non-stationary contextual bandits by transporting them with an explicit drift model and weighting each by transport reliability, outperforming methods that discard old data.","keywords":["non-stationary bandits","contextual bandits","Thompson sampling","drift correction","reward transport","linear Gaussian model","forgetting mechanisms"],"falsifier":"A controlled simulation in which the supplied drift model is deliberately misspecified by 20 percent; if fcTS then underperforms a simple sliding-window baseline, the central claim is falsified.","tokens_in":2583,"feed_emoji":"","tokens_out":693,"duration_ms":17273,"temperature":0.7,"pith_summary":"The paper introduces a Bayesian method for linear contextual bandits where the reward function changes over time. Standard algorithms become unreliable because old observations no longer match the current environment, so most approaches simply forget or down-weight history. Flow-Corrected Thompson Sampling instead estimates a drift model, adjusts past rewards to the present time, and adds each corrected observation to the posterior with a confidence weight that reflects how trustworthy the transport is. This produces closed-form updates under linear-Gaussian assumptions and specializes to several common drift patterns. Experiments across controlled cases and a portfolio benchmark show the largest gains precisely when the non-stationarity has recurring structure, indicating that structured reuse can be more sample-efficient than uniform forgetting.","feed_headline":"Corrected old rewards beat forgetting in drifting bandits","feed_subtitle":"Flow-Corrected Thompson Sampling transports past data with a drift model and weights it by reliability, outperforming discard-based baseline","key_machinery":"Flow correction: an explicit drift model that maps past rewards to the present time, combined with a per-observation weight that reflects transport uncertainty.","core_discovery":"Flow-Corrected Thompson Sampling (fcTS) maintains a posterior over the current reward parameters by transporting historical rewards forward via an explicit drift model and incorporating each transported observation with a reliability weight; the resulting updates remain closed-form and yield higher cumulative reward than discounting, sliding-window, or restart baselines, with the biggest advantage in regimes that repeat.","pith_inferences":["The same transport-and-reweight template could be applied to non-Bayesian algorithms such as LinUCB if a suitable weighting scheme is derived.","In practice the method requires a separate module to learn or specify the drift model; the overall performance therefore depends on how well that module tracks the true non-stationarity.","Recurring-structure gains suggest testing the approach on problems whose non-stationarity arises from external calendars or cycles rather than pure random walks."],"forward_implications":["When the drift is linear in time, online slope estimation plus reward correction produces a usable posterior without restarting.","When variation is periodic, phase alignment allows direct reuse of observations from previous cycles.","When the environment switches among a finite set of regimes, changepoint detection plus regime-specific memory preserves useful history.","Posterior updates stay closed-form and can be maintained with truncated sufficient statistics even after many corrections."],"fun_headline_variants":["Flow-corrected Thompson sampling reuses rewards via drift transport","Transporting rewards to present with reliability weights in fcTS","Closed-form updates from transported observations in drifting bandits","fcTS handles recurring regimes by regime-specific posterior memory"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"An explicit drift model can be estimated or specified accurately enough that the benefit of reusing corrected data exceeds any bias introduced by imperfect transport.","fun_headline_variants_meta":{"raw":{"variants":["Flow-corrected Thompson sampling reuses rewards via drift transport","Transporting rewards to present with reliability weights in fcTS","Closed-form updates from transported observations in drifting bandits","fcTS handles recurring regimes by regime-specific posterior memory"]},"model":"grok-4.3","cost_usd":0.006856,"raw_usage":{"total_tokens":3172,"prompt_tokens":644,"num_sources_used":0,"completion_tokens":61,"cost_in_usd_ticks":68562000,"prompt_tokens_details":{"text_tokens":644,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2467,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":644,"tokens_out":61,"duration_ms":17039,"temperature":1.0,"reasoning_tokens":2467,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T06:40:01.167291+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A controlled simulation in which the supplied drift model is deliberately misspecified by 20 percent; if fcTS then underperforms a simple sliding-window baseline, the central claim is falsified.","supporting_citations":[],"review_version":1}