Pith. sign in

REVIEW 5 major objections 7 minor 1 cited by

A New DAPO Algorithm for Stock Trading

T0 review · 5 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a GRPO-based trading agent with DAPO-style asymmetric clipping and dynamic sampling earns a 230.49% cumulative return on NASDAQ-100 from 2020 to 2023 while using far less compute than the LLM-informed CPPO baseline.

desk verdict A plausible engineering adaptation of DAPO/GRPO to trading with credible compute savings, but the headline return is selected on the test window with no seeds, so the core performance claim is unsupported. read the letter →

arxiv 2505.06408 v2 pith:E6BYIPQH submitted 2025-05-09 cs.CE

Ruijian Zha , Bojun Liu This is my paper · ORCID
classification cs.CE
keywords algorithmictradingreinforcementlearningDAPOGRPOsentimentanalysisriskmanagementlargelanguagemodelsNASDAQ-100
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether reinforcement-learning techniques developed for large language models can be moved into stock trading. It builds a trading agent on Group Relative Policy Optimization (GRPO), a critic-free variant of policy optimization, and grafts in two ideas from DAPO (Dynamic sAmpling Policy Optimization): asymmetric clipping of policy updates and dynamic sampling that skips uninformative states. The reward is the portfolio return multiplied by exponent-weighted sentiment and risk scores extracted by an LLM from financial news. On the NASDAQ-100 index from 2020 to 2023 the agent reports a 230.49% cumulative return and an information ratio of 0.37, ahead of the LLM-informed CPPO baseline it is compared with, while training time falls from roughly 8 hours to 2.5 hours for 100 epochs and RAM usage from 120 GB to 15 GB. The practical significance is that competitive RL trading agents can be trained on much smaller machines and re-run as new data arrive.

What carries the argument

The load-bearing mechanism is the reward-shaped group advantage with filtered updates. GRPO's advantage formula removes the value network entirely, which is the main source of the memory saving. The exponent-weighted sentiment-risk multiplier lets the designer tilt policy updates toward sentiment or away from risk without retraining the LLM. DAPO's decoupled clipping gives positive probability-ratio updates a wider upper bound while keeping negative updates within a tight lower bound, and dynamic sampling drops states with all-equal group rewards so gradient steps concentrate where the learning signal is non-trivial.

What would settle it

Run the identical setup on a walk-forward split: optimize $α$ and $β$ on 2019–2021 only, then evaluate on 2022–2023 against the NASDAQ-100 index and the LLM-informed CPPO baseline. If the 230.49% cumulative return is not reproducibly above both benchmarks across ten random seeds, the headline number is tied to the in-sample configuration rather than to the algorithm.

Watch

Extended reading notes

Core claim

The central claim is that the DAPO machinery transfers to finance: combining GRPO's group-relative advantage with DAPO's decoupled clipping and dynamic sampling, plus an exponent-weighted sentiment-risk reward, yields better reported returns than the LLM-informed CPPO baseline at roughly one-third the training time and one-eighth the memory. The update uses $A_G = (r'_{t,i} - μ_t)/(σ_t + ε)$ for group advantage, with adjusted reward $r'_{t,i} = r_{t,i} (S_{t,i})^α (R_{t,i})^β + 10^{-8}$, where $S_{t,i}$ and $R_{t,i}$ are portfolio-weighted LLM sentiment and risk scores mapped from discrete values 1–5 to multipliers 0.99–1.01. The loss clips the probability ratio asymmetrically with $[1 - ε_{low}, 1 + ε_{high}]$, and states whose sampled actions all have equal reward are filtered out before gradient updates. With exponents $α=3$, $β=1$ the paper reports 230.49% cumulative return; the balanced setting $α=1$, $β=1$ lands at about the same figure.

Load-bearing premise

The reported 230.49% return assumes the pre-extracted LLM sentiment and risk scores are accurate for 2020–2023 and that the reward exponents $α$ and $β$, selected on that same period, carry over to other market conditions; the paper does not report out-of-sample or multi-seed validation.

Editorial extensions

If this is right

  • Group-relative normalization without a critic network is enough to train a NASDAQ-100 trading agent to a reported 230.49% cumulative return, so a value network is not a necessary component for this setting.
  • DAPO's asymmetric clipping and dynamic sampling, originally designed for LLM preference tuning, transfer to a financial reward landscape where many states have flat rewards.
  • Sentiment and risk signals are complementary in the reward: combining them outperforms either alone, and moderate sentiment emphasis with $α=3$, $β=1$ beats risk-heavy and strong-sentiment settings.
  • Lower training cost (about 2.5 hours versus 8) and lower memory (15 GB versus 120 GB) mean the agent can be retrained more often as new data arrive.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper fixes $α$ and $β$ on the same evaluation window, so the cleanest test of the reward-shaping contribution is a walk-forward split: choose exponents on 2019–2021 data only and score on 2022–2023.
  • Because the LLM signals are pre-extracted rather than learned, the method's marginal value over a purely price-based policy is untested; comparing the same optimizer with $α=0$, $β=0$ against $α=3$, $β=1$ would isolate that value.
  • If the reported compute savings replicate, the practical bottleneck for RL trading shifts from hardware to signal quality and to the one-off choice of reward exponents.
  • Dynamic sampling gains depend on how often group rewards are flat; intraday trading, where prices move every minute, may have fewer flat states and so may see less benefit from this component.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes an RL-LLM trading agent that replaces PPO's critic with GRPO, adds DAPO-style decoupled clipping and dynamic sampling, and shapes rewards with LLM sentiment and risk scores through exponentiated factors (Eq. 3). On NASDAQ-100 data from FNSPID with pre-extracted DeepSeek signals, it reports a 230.49% cumulative return and an information ratio of 0.37 versus the CPPO-DeepSeek 10% baseline, along with reduced training time (2.5 hours vs 7-8 hours) and RAM usage (15 GB vs 120 GB). The authors conclude that the algorithm offers a scalable path for building trading agents. The paper provides a public code repository.

Significance. If the performance and efficiency results were properly validated, the paper would be a useful engineering contribution: critic-free training, decoupled clipping, and dynamic sampling are reasonable mechanisms, and the reported memory/time savings are directionally plausible. The code is provided, which supports reproducibility. However, the current evidence does not support the headline outperformance claim because the reward-shaping hyperparameters were selected on the same backtest window used for evaluation, no multiple-seed or holdout results are reported, and the baseline comparison lacks statistical detail.

major comments (5)
  1. [Section IV-C, Fig. 2, Table 1] The alpha,beta grid search is scored on the 2020-2023 window, and the same window is then used as the evaluation in Table 1. Since the paper states that alpha=3, beta=1 'yields notably higher cumulative returns' on this window, the 230.49% figure is a selected maximum over five reward-shaping specifications rather than an unbiased estimate of a trained policy's expected performance. With one seed per configuration, this is insufficient to support the claim of outperforming the baseline.
  2. [Section IV-A, Table 2] No multiple seeds, confidence intervals, or a genuine out-of-sample period is reported. The additional evaluation in Table 2 covers 2019-2023, which substantially overlaps the 2020-2023 selection window, and the paper does not state how the model was trained or whether that window was held out during any part of the tuning process. RL training is stochastic, and the reported returns may reflect seed luck rather than a real edge.
  3. [Table 1, Section I] The baseline column reports approximate values ('~215%', '~-35%', '~4.37%') with no tuning details or run count. More importantly, the proposed method is worse on max drawdown (-49.11% vs ~-35%) and CVaR (-5.64% vs -4.37%); since the paper motivates risk-sensitive trading, this contradicts the risk-management contribution and makes the 'outperforms' claim metric-dependent. The paper does not address this imbalance.
  4. [Section III-B, Eq. (3)] The f mapping from discrete LLM scores {1,...,5} to {0.99, 0.995, 1.0, 1.005, 1.01} is introduced without calibration or sensitivity analysis, and the exponents alpha and beta are selected on the evaluation window. The paper provides no evidence that these choices generalize beyond the 2020-2023 period, and the statement in Section IV-C that both balanced and sentiment-emphasis settings achieve ~230.49% return does not establish robustness across market regimes.
  5. [Table 3, Section IV-D] Table 3 reports 2.5 hours vs 7-8 hours and 15 GB vs 120 GB RAM, but no details are given on hardware configuration parity, the number of environment steps, or whether the baseline was rerun under the same conditions. The efficiency claim is plausible, but the comparison is not fully controlled.
minor comments (7)
  1. [Section III-B, Eq. (3)] The symbol m in the definitions of S_t,i and R_t,i is not defined; the paper should specify whether it is the number of stocks in the portfolio or the number of actions in the group.
  2. [Section III-B, Eq. (3)] The text says 'f map' where 'f maps' would be grammatically correct, and the domain and codomain of f should be stated explicitly rather than only by example.
  3. [Section III-C] The dynamic-sampling rule that filters out 'states with all the same rewards' is not formalized; the paper should state the exact condition and how it is implemented in the training loop.
  4. [Table 1] The header '10%' is unexplained; presumably it refers to the CPPO-DeepSeek 10% benchmark from the FinRL contest, but this should be stated explicitly in the table caption or text.
  5. [References] References [3] and [8] are duplicate entries for the same IJCAI paper; one should be removed or merged.
  6. [Figures 1 and 2] The figures do not include error bars, number of seeds, or confidence intervals; the captions should state that these are single-run backtests.
  7. [Section IV-C] The text says sentiment emphasis (alpha=3, beta=1) 'yields notably higher cumulative returns' but then states that both balanced and sentiment emphasis achieve ~230.49%; these two statements should be reconciled.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline 230.49% return is the in-sample maximum over the α,β grid used for the same evaluation window, so the performance claim reduces to reward-shaping selection; the compute-efficiency claim is independent.

  1. fitted input called prediction [Section IV-C and Table 1 (reward exponents α,β in Eq. 3)]
    "Fig. 2 illustrates that a sentiment-emphasized configuration (α = 3,β = 1) yields notably higher cumulative returns than either balanced or risk-focused settings... Both balanced (α = 1,β = 1) and Sentiment Emphasis (α = 3,β = 1) achieve ∼ 230.49% return, reflecting the reward function’s robustness within a reasonable α,β range. ... Table 1: Cumulative Return 230.49%"

    The α,β exponents in Eq. (3) are not fixed a priori; Section IV-C chooses them by ranking five settings on the 2020–2023 backtest. The winning settings' 230.49% cumulative return is then reported in Table 1 as 'Our Model' for the same 2020–2023 period. Thus the headline performance is the maximum of a grid search evaluated on the same data that serves as the final evaluation; it is an artifact of selection rather than an independent estimate. The claim of robustness ('both balanced and Sentiment Emphasis achieve ∼230.49%') is also inferred from the same in-sample window. This is a fitted input (tuned reward exponents) presented as a predictive performance result, so the outperformance claim reduces to the selection procedure.

full rationale

The paper's only mathematical content is standard GRPO/DAPO machinery (Eqs. 1 and 4) plus the sentiment-risk reward in Eq. (3); the latter introduces free exponents α,β. Section IV-C selects these exponents by comparing five settings on the 2020–2023 window and reports that both (α=1,β=1) and (α=3,β=1) 'achieve ∼230.49% return.' Table 1 then lists 230.49% as 'Our Model' over the same period. The reported performance is therefore the selected reward-shaping configuration evaluated on the very window used to choose it; the headline outperformance claim is not an independent test and is statistically forced by the selection, though not by an equation-level identity. The efficiency results (Table 3: 2.5 h vs 7–8 h; 15 GB vs 120 GB) do not depend on α,β and remain independent evidence. There is no load-bearing self-citation: DAPO [6], FinRL-DeepSeek [5], and FNSPID [7] are external prior works. The circularity is partial, confined to the performance claim.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest on three domain assumptions: GRPO group normalization is a valid credit assignment for trading, LLM sentiment/risk scores are predictive, and a single historical backtest represents future performance. No new entities are introduced. The free parameters are the reward exponents and the score-to-factor mapping, both tuned on the reported backtest window.

free parameters (3)
  • alpha, beta (sentiment/risk exponents) = alpha=3, beta=1 reported as best; alpha=1, beta=1 also ~230%
    Selected by comparing five configurations on the 2020-2023 backtest window (Section IV-C, Fig 2); this is post-hoc tuning rather than a fixed prior.
  • f mapping from LLM scores to factors = 0.99, 0.995, 1.0, 1.005, 1.01 for scores 1-5
    Hand-chosen in Eq. (3); no justification or sensitivity analysis, and it directly scales the reward.
  • epsilon_low, epsilon_high (decoupled clipping bounds)
    Introduced in Section III-C but values not given; they control the policy update and thus affect training behavior and the reported results.
assumptions (3)
  • domain assumption Group-relative advantage computed from a small sampled action group is a valid training signal for trading.
    Eq. (1) assumes that normalizing rewards within a group of actions removes the need for a value function; this is a standard GRPO assumption but not validated here for the trading MDP.
  • domain assumption LLM sentiment and risk scores aggregated in Eq. (3) are predictive of future return.
    The reward shaping multiplies returns by these scores; if the scores are not predictive, the shaped reward is noise.
  • domain assumption The 2020-2023 backtest window is representative of strategy performance.
    No out-of-sample or multi-seed evaluation; the reported cumulative return is a single path.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A New DAPO Algorithm for Stock Trading." pith.science (2026). https://pith.science/paper/E6BYIPQH

@misc{pith2026250506408,
  author       = {Pith},
  title        = {Pith review of: A New DAPO Algorithm for Stock Trading},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E6BYIPQH}},
  note         = {Machine review of arXiv:2505.06408}
}
read the original abstract

Recent advances in reinforcement learning, such as Dynamic Sampling Policy Optimization (DAPO), show strong performance when paired with large language models (LLMs). Motivated by this success, we ask whether similar gains can be realized in financial trading. We design a trading agent that combines an improved Group Relative Policy Optimization (GRPO) algorithm, augmented with ideas from DAPO, with LLM-based risk and sentiment signals extracted from financial news. On the NASDAQ-100 index (FNSPID dataset), our agent attains a cumulative return of 230.49 percent and an information ratio of 0.37, outperforming the CPPO-DeepSeek baseline. It also cuts training time from about 8 hours to 2.5 hours over 100 epochs while markedly reducing RAM usage. The proposed RL-LLM framework offers a scalable path toward data-efficient trading agents. Code: https://github.com/Ruijian-Zha/FinRL-DAPO-SR/

Figures

Figures reproduced from arXiv: 2505.06408 by the authors.

Figure 1
Figure 1. Comparison of the Baseline, Risk-Only, and Sentiment-Only reward [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of four sentiment–risk weightings vs. the NASDAQ-100, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Financial Numerical Prediction and Allocation as Token Generation

    cs.CV 2026-08 conditional novelty 6.0 of 10

    FinATOM uses a causal language model's token vocabulary to output stock forecasts and ETF allocations, and shows policy optimization improves out-of-sample Sharpe.

Reference graph

Works this paper leans on

12 extracted references · 6 canonical work pages · cited by 1 Pith paper

  1. [1]

    Ԅ; ˏ 9"j8 6 tzyy l6@ kp' 7l6 d2X Q q Gr P m6o&EQo>qD0 dY ٗ_ ڵk (Y ` ^ݻw ޽ xSSS:j|Ǐ|YUnKh0 `` 歷2 ap8 4M˲o 9r xUUj H / Q eCP4 T < _ \ Ȋ p8l0 X t:L&F oQw 7 h a&6c(ʲl27 D

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...

    work page

  2. [2]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, ``Proximal policy optimization algorithms,'' arXiv preprint arXiv:1707.06347, 2017

    arXiv 2017

  3. [3]

    X.-Y. Liu, H. Yang, Q. Chen, et al., ``FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance,'' arXiv preprint arXiv:2011.09607, 2022

    arXiv 2011

  4. [4]

    C. Y. Ying, X. Zhou, H. Su, et al., ``Towards safe reinforcement learning via constraining conditional value-at-risk,'' in IJCAI, 2022, pp. 3673--3680

    work page 2022

  5. [5]

    DeepSeek-AI et al., ``DeepSeek-V3 Technical Report,'' arXiv preprint arXiv:2412.19437, 2024

    arXiv 2024

  6. [6]

    Benhenda, ``FinRL-DeepSeek: LLM-infused risk-sensitive reinforcement learning for trading agents,'' arXiv preprint arXiv:2502.07393, 2025

    M. Benhenda, ``FinRL-DeepSeek: LLM-infused risk-sensitive reinforcement learning for trading agents,'' arXiv preprint arXiv:2502.07393, 2025

    arXiv 2025

  7. [7]

    Q. Yu, Z. Zhang, R. Zhu, et al., ``DAPO: An open-source LLM reinforcement learning system at scale,'' arXiv preprint arXiv:2503.14476, 2025

    arXiv 2025

  8. [8]

    Z. Dong, X. Fan, Z. Peng, et al., ``Fnspid: A comprehensive financial news dataset in time series,'' arXiv preprint arXiv:2402.06698, 2024

    arXiv 2024

Show all 12 references
  1. [9]

    C. Y. Ying, X. Zhou, H. Su, D. Yan, N. Chen, and J. Zhu, ``Towards safe reinforcement learning via constraining conditional value-at-risk,'' in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI-22), pp. 3673–3680, 2022, doi: 10.24...

    2022 doi

  2. [10]

    K. Wang, K. Xiao, and X.Y. Liu, ``Parallel Market Environments for FinRL Contests,'' arXiv preprint arXiv:2504.02281, 2025

    2025 arXiv

  3. [11]

    X.-Y. Liu, Z. Xia, H. Yang, J. Gao, D. Zha, M. Zhu, C. D. Wang, Z. Wang, and J. Guo,``Dynamic datasets and market environments for financial reinforcement learning, '' Machine Learning - Nature, 2024

    2024

  4. [12]

    H. H. Yang, X.-Y. Liu, H. Tong, Y. Zhang, J. Wang, and L. Deng, ``Deep reinforcement learning for automated stock trading: an ensemble strategy,'' Proceedings of the 30th ACM International Conference on Information & Knowledge Management (CIKM), pp. 2245--2252, 2020

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.