{"id":"0e3a57f1-15b5-4582-a51a-105d1f563bf1","arxiv_id":"2506.04658","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Deep reinforcement learning agents, especially PPO with a Transformer network, outperformed buy-and-hold on most tested assets over 2019-2023 in reported Sharpe ratios.","lead":"This paper tests whether AI trading agents using deep reinforcement learning can beat simply buying and holding a market, across currencies, the S&P 500, and Bitcoin, using daily data from 2019 to 2023. It reports that a PPO agent with a Transformer network often posted higher risk-adjusted returns than buy-and-hold, and sometimes even beat a hypothetical perfect-foresight annual strategy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported PPO_T Sharpe ratios cannot support the central claim because the walk-forward protocol selects among generations and configurations with no multiple-testing control or significance testing.","rationale":"The reader's weakest assumption is exactly the load-bearing point: the walk-forward evaluation is treated as an unbiased estimate, but the generation-selection procedure plus unreported search over configs/assets creates a multiple-testing problem. I agree with that identification. I would not make the missing supervised baseline the central objection: the paper's hypothesis is about DRL's ability to trade, and a supervised baseline is one comparison that could support the claim but is not logically required for it. The decisive issue is that the reported PPO_T numbers are selected. The paper itself supplies supporting evidence for this concern: Section 4 says no search was implemented but hyperparameters were tested based on expert knowledge; Section 3.1 describes checking nearby generations to confirm validation performance, which is a heuristic robustness check, not a statistical test. There is also an internal inconsistency between Figure 2 and §3.1 about whether validation begins in 2017 or 2018; while not by itself fatal, it makes the protocol too imprecise to support the strong conclusion. If the pre-registered bootstrap test still yields a significant, corrected result, the central claim would be strengthened; if not, the current wording should be weakened. Since the reader already recommends REJECT, my stress-test leaves that verdict unchanged.","tokens_in":28518,"tokens_out":5800,"duration_ms":70449,"concrete_test":"Freeze a pre-registered protocol before seeing 2019-2023 data: for each of the five assets, train PPO_T and a random-action baseline with the same walk-forward generation rules, select the generation using only 2018 validation, score 2019-2023 once, and bootstrap the mean Sharpe difference across assets (with a multiple-testing correction for the 20 configuration-asset cells). If the adjusted p-value is not below 0.05, the headline Sharpe ratios are consistent with selection/luck and the central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that PPO_T's out-of-sample Sharpe ratios (0.847, 1.752, 2.179, 2.158, 1.093) demonstrate that DRL can trade profitably. For that inference to be valid, the walk-forward protocol must yield unbiased estimates of true out-of-sample performance. Section 3.1 says the authors 'select agent's generation that have undergone at least half of their training period' and then check two neighbouring generations, but it never states how many generation checkpoints were inspected, whether validation Sharpe was used to choose the reported generation, or how the 20 configuration-asset cells were selected for emphasis. Section 4 reports that no random/grid search was done and that hyperparameters such as gamma (0.75, 0.9, 0.95) and per-asset reward scaling were 'tested' on expert judgment; each tested setting is an extra opportunity to pick a lucky result. No formal significance test, bootstrap, or multiple-testing correction appears in Sections 3.2 or 4. Under even a modest number of such choices, the best of several Sharpe ratios is inflated by selection, so the reported ratios do not distinguish skill from luck.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper tests two deep reinforcement learning algorithms, DDQN and PPO, each paired with either a fully connected neural network or a Transformer architecture, as trading agents on five assets (EUR/USD, EUR/JPY, USD/JPY, S&P 500, Bitcoin) over 2019–2023. The authors use a walk-forward optimization scheme with a validation year preceding each out-of-sample year, and they compare the agents against buy-and-hold and a hypothetical perfect-foresight annual strategy. The central empirical claim, stated in the Conclusions, is that PPO with a Transformer network achieves strong risk-adjusted returns (Sharpe ratios of 0.847, 1.752, 2.179, 2.158, and 1.093 across the five assets), supporting the hypothesis that DRL can effectively trade financial markets.","tokens_in":28704,"tokens_out":5026,"duration_ms":56977,"significance":"If the reported results were statistically validated, the paper would offer a notable demonstration that model-free DRL agents can deliver positive risk-adjusted returns across multiple asset classes without explicit price forecasting. The study has several strengths: it uses a multi-asset walk-forward design, accounts for transaction costs, reports a range of performance and behavioral metrics, and includes an explicit discussion of methodological limitations. However, the central claim is not supported by the evidence as presented: there are no significance tests or confidence intervals for the Sharpe ratios, the best-performing configuration appears to be selected from a large set of tested settings with no multiple-testing correction, and the abstract's claim of an edge over supervised learning is never tested empirically. These shortcomings are load-bearing for the paper's main conclusion.","major_comments":[{"comment":"The headline Sharpe ratios for PPO_T (0.847, 1.752, 2.179, 2.158, 1.093) are single point estimates from one five-year backtest, with no confidence intervals, bootstrap resampling, or significance tests reported anywhere. For daily returns, the sampling uncertainty of an annualized Sharpe ratio over five years is substantial (roughly ±0.7 for a Sharpe near 1.6), so the observed differences between PPO_T and buy-and-hold (e.g., 1.093 vs. 0.89 on Bitcoin) are well within noise. The statement in the Conclusions that the findings 'robustly support' the research hypothesis is not justified without formal inference.","section":"Section 4, Tables 1–5; Conclusions"},{"comment":"The evaluation protocol involves a large implicit multiple-comparison problem. The authors state that they did not perform random or grid search but 'relied on the expert knowledge of the authors to select and test parameters,' and Section 4.6 reports testing gamma values of 0.75, 0.9, and 0.95, plus reward scaling factors of 100 or 1000. Across four model configurations and five assets, this yields roughly 20 configuration-asset cells, and the paper highlights the best-performing one (PPO_T). With no correction for best-of-N selection, the reported out-of-sample Sharpe ratios are likely inflated by selection, and the results do not distinguish skill from luck.","section":"Chapter IV, introductory paragraphs; Section 4.6"},{"comment":"The abstract claims a 'substantial edge over classical approaches, based on supervised learning in terms of risk-adjusted returns,' but no supervised learning baseline is implemented or evaluated in the paper. Section 5.2 discusses supervised learning only conceptually, arguing that it lacks a natural 'stay out of the market' mechanism. Without an empirical comparison to at least one supervised baseline (e.g., a logistic regression or a recurrent neural network with the same inputs), this claim is unsupported and should be either removed or backed by experiments.","section":"Abstract; Section 5.2"},{"comment":"The walk-forward agent generation selection is underspecified and non-reproducible. The text says 'we select agent’s generation that have undergone at least half of their training period' and that the authors 'analyse agents from two generations before and after the chosen generation,' but it does not state how many generations were trained, how a generation is defined, or whether the generation ultimately reported was chosen based on validation performance. If validation Sharpe was used to select the generation, this selection step must be accounted for in the inference; otherwise the out-of-sample claim is not credible as stated.","section":"Section 3.1"}],"minor_comments":[{"comment":"The manuscript contains many typos and grammatical errors, including 'Netowork' in table captions, 'Suprisly' in Section 4.5, 'Suppressively' in Section 4.6, 'especialy' in Section 3.1, and 'notatios' in Chapter IV.","section":"Throughout"},{"comment":"Several cross-references to figures and tables are incorrect: Section 4.2 refers to Figure 4 for EUR/JPY but the correct figure is Figure 5; Section 4.3 refers to 'Table 2' for USD/JPY but should be Table 3; Section 4.4 refers to 'Table 2' and 'Figure 5' for S&P 500 but should be Table 4 and Figure 9; Section 4.5 refers to 'Figure 5' for Bitcoin but should be Figure 11.","section":"Sections 4.2–4.5"},{"comment":"The CAGR formula is not typeset correctly; the exponent and fraction appear as garbled text. It should be rewritten in standard mathematical notation.","section":"Equation [14]"},{"comment":"There are several formatting errors in the references, such as 'Mnih, V olodymyr' with an extraneous space and the duplicated phrase 'Long Short-Term Memory. Long Short-Term Memory.' in the Hochreiter entry.","section":"Bibliography"},{"comment":"The paper does not state the specific data sources for the OHLC data (e.g., which vendor or API), nor does it indicate whether code is available. This limits reproducibility of the empirical results.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper reads as a preliminary working paper with numerous presentation issues. The core empirical claim is not established because of the absence of statistical inference, the multiple-testing problem in configuration selection, and the unsupported claim about supervised learning. These issues are substantial but potentially addressable: the authors could add bootstrap confidence intervals or significance tests for the Sharpe ratios, report results for all tested configurations rather than only the best, and implement a simple supervised baseline. If they do so, the paper could become a useful empirical contribution. In its current form, however, the central conclusion is not supported by the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper is a careful-looking DRL trading study with some impressive Sharpe ratios, but the headline claim is not backed by statistical inference. The best configuration (PPO with Transformer) is highlighted after testing multiple architectures and assets, so those numbers are probably optimistically biased.\n\nWhat's genuinely useful: the authors run DDQN and PPO with both fully-connected and Transformer networks over five assets using a rolling window, with buy-and-hold and a perfect-foresight annual benchmark. They report transaction costs, position durations, win rates, and drawdowns. They also openly discuss the RL assumption that the agent affects the environment, which is false in trading, and the problematic credit assignment in TD. That candor is rare.\n\nThe soft spots: no supervised learning baseline, despite the abstract explicitly claiming an edge over supervised approaches. No confidence intervals or significance tests for Sharpe ratios. The walk-forward selection picks agent generations on validation data, and they test 20 configuration-asset combinations; the best result is likely inflated by selection. They state they used expert judgment for gamma and reward scaling rather than grid search, but that is still multiple testing. There's also a mismatch between Figure 2 and the text about which year is validation versus test. These issues don't destroy the descriptive results, but they do undermine the conclusion that the results 'robustly support' the hypothesis.\n\nWho is this for? People working on DRL trading or trying to avoid overfitting in backtests. It's a good case study in evaluation pitfalls. My own verdict is skeptical: I wouldn't take the Sharpe ratios at face value. But the experiment is detailed enough to be worth a serious referee, with major revision likely. I'd send it to review.","headline":"A transparent DRL trading study with impressive but statistically unbacked Sharpe ratios; worth reviewing if the authors add a supervised baseline and significance tests.","tokens_in":29262,"tokens_out":2595,"would_cite":false,"duration_ms":33067,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A model-free AI trading agent that never forecasts prices beat buy-and-hold on all five markets it tested, the paper reports.","keywords":["deep reinforcement learning","algorithmic trading","Proximal Policy Optimization","Double Deep Q-Network","Transformer network","walk-forward optimization","Sharpe ratio","market timing"],"falsifier":"Run the same walk-forward protocol on shuffled return series (or on synthetic price paths with the same drift and volatility); if the validation-selection procedure still yields out-of-sample Sharpe ratios near 2.18 for USD/JPY, the headline numbers are selection artifacts. The more direct check is to construct a bootstrap or deflated-Sharpe confidence interval for each reported out-of-sample Sharpe ratio; if the interval for the EUR/USD PPO_T result (0.847) includes zero, the claim that DRL beats the market is not supported.","tokens_in":28266,"feed_emoji":"📈","tokens_out":10259,"duration_ms":106152,"temperature":0.7,"pith_summary":"This paper is trying to establish that deep reinforcement learning (DRL) can trade financial markets profitably without predicting future prices. The authors train two model-free DRL algorithms, DDQN and PPO, with two network architectures on daily data for three currency pairs, the S&P 500, and Bitcoin over 2019-2023, using walk-forward optimization. Their best configuration, PPO with a Transformer network, beat buy-and-hold on all five assets and even outperformed a perfect-foresight annual strategy on several, with Sharpe ratios of 0.847, 1.752, 2.179, 2.158, and 1.093. The claimed edge comes from learning when to be long, short, or out of the market, using strategic non-engagement to avoid unfavorable conditions. If true, this would show that trading skill can be acquired by interaction rather than forecasting, and would challenge strong forms of the efficient-market hypothesis.","feed_headline":"AI trader that never forecasts prices beats buy-and-hold on 5 markets","feed_subtitle":"PPO plus a Transformer posts Sharpe ratios up to 2.18 by choosing when to stay out of the market.","key_machinery":"The central machinery is a model-free DRL policy trained with temporal-difference learning, where the agent's action space is three discrete positions: long, short, or out of the market, and the reward is scaled profit-and-loss. For PPO, the policy is updated with a clipped surrogate objective and Generalized Advantage Estimation, while the DDQN variant uses double Q-learning with separate primary and target networks. The authors pair these algorithms with two function approximators, a fully connected neural network and a Transformer, and evaluate them through a walk-forward procedure that retrains from scratch on rolling windows, validates on one year, and tests on the next year. The load-bearing element is the learned \"stay out\" action, which the results show is used roughly 30% of the time and is presented as the key to superior risk-adjusted returns.","core_discovery":"The paper's central claim is that DRL, as an advanced form of AI, can effectively conduct financial trading by autonomously identifying and exploiting patterns within complex, high-dimensional data, without being given a model of price dynamics. Empirically, the paper reports that PPO combined with a Transformer network (PPO_T) produced annualized Sharpe ratios of 0.847 on EUR/USD, 1.752 on EUR/JPY, 2.179 on USD/JPY, 2.158 on the S&P 500, and 1.093 on Bitcoin over the 2019-2023 out-of-sample period, beating buy-and-hold in final balance on every asset and beating a hypothetical perfect-foresight annual strategy on EUR/USD, EUR/JPY, USD/JPY, and the S&P 500. The authors attribute the success to DRL's ability to manage risk by choosing not to trade during unfavorable conditions, a mechanism supervised-learning approaches lack, and to the Transformer's attention mechanism capturing sequential dependencies in market data. The paper also reports a consistent ordering across configurations: PPO_T best, followed by DDQN_T, DDQN_NN, and PPO_NN, showing that network architecture matters at least as much as the choice of RL algorithm.","pith_inferences":["The reported out-of-sample Sharpe ratios are likely optimistic because the best agent generation was chosen on one year of validation data across many configuration-asset combinations without multiple-testing correction; a bootstrap or deflated-Sharpe adjustment would give a fairer estimate.","The paper's own Section 5.1 notes that training on historical prices cannot capture the price impact an agent would have at scale, so the results apply to small traders rather than strategies large enough to move markets.","If the \"stay out\" behavior is the true source of edge, the results may be better characterized as a volatility- or tail-risk-avoidance effect than as evidence of predictable directional patterns, and that distinction is testable by decomposing returns into exposure and timing components.","A natural extension, also suggested by the authors, is continuous position sizing or hierarchical multi-agent management, which could outperform the discrete long/short/out action space in risk-adjusted terms."],"forward_implications":["PPO with a Transformer network, the best configuration, beat buy-and-hold on all five assets and beat a perfect-foresight annual strategy on four, implying the claimed edge is not just about overall market direction but about timing and risk avoidance.","Transformer backbones consistently outperformed fully connected networks for both algorithms, so architecture choice is at least as decisive as the RL algorithm itself.","DRL agents added value mainly by staying out of the market during unfavorable conditions, a behavior that has no direct analogue in supervised-learning trading rules.","The weaker Bitcoin results, blamed on the shorter training history, imply that data quantity and coverage are limiting factors for this approach.","If these results generalize, model-free DRL could serve as an adaptive trading layer that does not rely on price forecasts, with implications for the efficient-market debate."],"supporting_citations":[{"why":"Introduces the deep Q-network that the paper's DDQN value-based agent is built on.","marker":"Mnih et al. (2015)"},{"why":"Presents double Q-learning, the overestimation-reducing update the paper implements as DDQN.","marker":"Hasselt et al., 2016"},{"why":"Defines Proximal Policy Optimization, the policy-gradient algorithm behind all PPO agents in the study.","marker":"Schulman et al., 2017"},{"why":"Provides Generalized Advantage Estimation, which the PPO agents use to compute policy-update advantages.","marker":"Schulman et al., 2016"},{"why":"Introduces the Transformer and self-attention architecture that the paper's best-performing PPO_T strategy uses.","marker":"Vaswani et al., 2017"},{"why":"Supplies the temporal-difference learning framework, notation, and the stochastic-reward logic for all the agents.","marker":"Sutton and Barto (2018)"},{"why":"Defines the Sharpe ratio, the primary risk-adjusted performance metric used to compare strategies.","marker":"Sharpe (1966)"},{"why":"Reviews DRL applications in algorithmic trading, motivating the paper's choice of DDQN and PPO as representative algorithms.","marker":"Pricope (2021)"}],"fun_headline_variants":["Reinforcement learning AI beats buy-and-hold on 5 markets","AI trader sidesteps bad days to top stock market benchmark","PPO plus Transformer posts Sharpe 2.18 by knowing when to quit","Deep RL agent outscores buy-and-hold across FX, stocks, Bitcoin","AI learns when not to trade, beating classic strategies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that selecting each agent generation on one year of validation data, across 20 configuration-asset combinations and with no multiple-testing correction or significance tests, produces an unbiased estimate of true out-of-sample performance.","fun_headline_variants_meta":{"raw":{"variants":["Reinforcement learning AI beats buy-and-hold on 5 markets","AI trader sidesteps bad days to top stock market benchmark","PPO plus Transformer posts Sharpe 2.18 by knowing when to quit","Deep RL agent outscores buy-and-hold across FX, stocks, Bitcoin","AI learns when not to trade, beating classic strategies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1352,"prompt_tokens":905,"completion_tokens":447,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":365}},"tokens_in":521,"tokens_out":447,"duration_ms":5642,"temperature":1.0,"reasoning_tokens":365,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:36:39.606400+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same walk-forward protocol on shuffled return series (or on synthetic price paths with the same drift and volatility); if the validation-selection procedure still yields out-of-sample Sharpe ratios near 2.18 for USD/JPY, the headline numbers are selection artifacts. The more direct check is to construct a bootstrap or deflated-Sharpe confidence interval for each reported out-of-sample Sharpe ratio; if the interval for the EUR/USD PPO_T result (0.847) includes zero, the claim that DRL beats the market is not supported.","supporting_citations":[],"review_version":1}