Pith. sign in

REVIEW 4 major objections 5 minor 11 references

FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning LLaMA-3.2 on market feedback yields steadier trading signals than the base model, and the approach won Task II of FinRL 2024.

desk verdict A contest write-up with a sensible idea but no quantitative evidence; the central claim of outperformance rests on a self-contradictory reading of two figures. read the letter →

arxiv 2502.01992 v1 pith:4FQM45H6 submitted 2025-02-04 q-fin.TR cs.LG

Arnav Grover This is my paper · ORCID
classification q-fin.TRcs.LG
keywords LargeLanguageModelsReinforcementLearningfromMarketFeedbackFinancialSentimentAnalysisPromptEngineeringTradingSignalsLLaMA-3.2-3B-InstructFinRL2024
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a large language model can be made to produce better trading signals by fine-tuning it with rewards derived from actual market outcomes, rather than from human sentiment labels alone. Its framework, FinRLlama, wraps news headlines, price data, and a scoring rubric into a prompt, then uses reinforcement learning from market feedback to push the LLaMA-3.2-3B-Instruct model toward scores that match forward price moves. On 2023 data, the fine-tuned model shows lower volatility in cumulative returns across tickers and a smoother mean evaluation than the base model, which the paper reads as a more conservative and consistent signal profile. The paper reports that this approach won Task II of the FinRL 2024 challenge.

What carries the argument

The engine of the method is the RLMF fine-tuning loop combined with a structured scoring prompt. The prompt (Algorithm 1) fixes a scoring scale with a signal-strength bound and a threshold, tells the model to incorporate past market responses, sentiment alignment, and historical price patterns, and gives worked examples; the RL loop then scores the model's integer sentiment signal against realized market performance, rewarding correct direction, rewarding confidence when returns confirm the signal, and penalizing errors. The three-day forward close price provides the label that connects a headline to market impact, and the Adam-optimized training minimizes the discrepancy between predicted sentiment and actual outcome. What this machinery buys is a single integer score per headline that can be mapped directly to buy, hold, or sell decisions and is repeatedly corrected by market feedback.

What would settle it

Train a control model with identical prompts but shuffled three-day forward returns as the reward, test both on 2023, and compare the cross-ticker spread of cumulative returns. If the control reproduces FinRLlama's narrower spread and smoother mean, the fine-tuning effect is not due to market feedback; if it does not, the reward signal is doing the work.

Watch

Extended reading notes

Core claim

The central claim is that fine-tuning an LLM's sentiment output with reinforcement learning from market feedback yields a signal generator that is better aligned with short-term price dynamics than the untuned model. In FinRLlama, each news headline is paired with price data and the model outputs an integer sentiment score between $-\mathit{signal\_strength}$ and $\mathit{signal\_strength}$; the reward function judges that score against actual market outcomes, including the direction and size of subsequent returns and the profit or loss of long, short, or hold positions. After training on 2020-2022 headlines and evaluating on 2023, cumulative returns across tickers are less volatile and the spread between best and worst performers narrows compared with the base LLaMA-3.2-3B-Instruct model, which the author interprets as a more conservative model response. The paper also reports being awarded winner of Task II.

Load-bearing premise

The whole training signal rests on treating a three-day forward close price as the correct measure of a news headline's market impact; if three-day returns do not capture the relevant market response, the fine-tuned sentiment scores will be misaligned with the trading task.

Editorial extensions

If this is right

  • If FinRLlama is correct, any base LLM sentiment scorer can be realigned to market behavior by adding a market-feedback reward, without changing the model architecture.
  • Trading strategies built on the fine-tuned signals should show narrower dispersion of cumulative returns across assets than strategies built on the base model's sentiments.
  • A single integer signal in $[-\mathit{signal\_strength}, \mathit{signal\_strength}]$ is sufficient to drive a long, short, or hold policy, which keeps the decision pipeline fast and simple.
  • The evaluation protocol, with 2020-2022 training and 2023 testing, demonstrates that the alignment survives a change of market conditions, at least to the extent claimed here.
  • The same RLMF recipe could be applied to other news sources and ticker sets, since the prompt and reward do not depend on the specific headlines used in training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The reduced volatility may partly reflect reward-function regularization, penalizing confident wrong calls, rather than improved directional accuracy; the paper does not report a direct correlation between sentiment scores and forward returns.
  • Editorial inference: The paper's future-work section concedes that delayed market reactions are not fully captured, which undermines confidence in the three-day forward-close label as the one true horizon; trying 1-day and 10-day horizons would show whether the signal advantage is specific to the lookahead.
  • Editorial inference: A direct information-coefficient test, the rank correlation between FinRLlama's scores and subsequent returns, would separate signal quality from the volatility-reduction effect, since the two can move independently.
  • Editorial inference: The reported comparison is against the base Llama model; re-running against dictionary-based sentiment and other contest baselines would clarify what the contest win adds beyond the conservative-spread effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FinRLlama, a prompt-based framework for fine-tuning LLaMA-3.2-3B-Instruct with reinforcement learning from market feedback (RLMF) to generate stock trading signals from news headlines. The method integrates historical price data, technical indicators, and a custom scoring prompt, with a reward function based on three-day forward returns. The authors evaluate the model on 2023 data after training on 2020–2022, comparing its cumulative returns qualitatively against a baseline LLaMA model in Figures 3 and 4. The paper claims that the RLMF-tuned model outperforms the baseline in signal consistency and achieves tighter trading outcomes, and states that it was awarded the winner of Task II at the FinRL Contest 2024.

Significance. If the claimed outperformance held with rigorous evidence, the paper would offer a useful case study in adapting small open-weight LLMs to financial sentiment via market-feedback fine-tuning, and the released code could serve as a starting point for reproducible contest-style baselines. However, the significance is currently hypothetical: the evaluation is entirely qualitative, the RL methodology is not specified in enough detail to be reproduced or verified, and the internal inconsistency in the prose undermines the central comparison. The paper provides no quantitative metrics, no confidence intervals, no statistical tests, and no official contest metric, so the reader cannot assess whether the claimed improvement is real or an artifact of figure interpretation.

major comments (4)
  1. [Section 3.3, Figures 3 and 4] The central claim that FinRLlama 'outperforms baseline methods in signal consistency and achieving tighter trading outcomes' rests entirely on a qualitative reading of two figures. No cumulative-return values, win/loss rates, Sharpe ratios, standard deviations, confidence intervals, or statistical tests are reported anywhere in the paper. Moreover, the prose is internally inconsistent: the first paragraph of Section 3.3 says that in Figure 3 'the cumulative returns of the tickers appear to be less volatile' and that 'the spread between the highest and lowest performing stocks is narrower,' while the second paragraph, also nominally about Figure 3, says 'the cumulative returns show significant variation between tickers' and that the plot 'suggests a broader divergence in performance between stocks.' These two descriptions cannot both describe the same figure, so the summary conclusion that FinRLlama 'suggests a more conservative approach with reduced volatility' is not reliably anchored to the presented data.
  2. [Section 2.2, Training Process] The paper describes the method as reinforcement learning from market feedback, but the actual training objective is never specified. The text says the reward function 'evaluates the model's predictions by comparing sentiment scores to actual market performance' and is 'dynamically adjusted,' yet no equation, pseudo-code, or hyperparameter values are given for the reward, and no RL algorithm (e.g., PPO, REINFORCE, DQN) is identified. The subsequent sentence that 'the model's fine-tuning process is guided by the Adam optimizer, minimizing the loss function based on the discrepancy between predicted sentiment signals and actual market outcomes' reads like supervised learning rather than RLMF. Because the claimed contribution is precisely this 'RLMF-tuned framework,' the missing formal specification is a load-bearing gap that prevents replication and even the verification that the method is RL at all.
  3. [Section 3.1, Experimental Setup] The experimental setup omits several essential details needed to assess the claimed winner result: the dataset is not identified (no source for news headlines or price data), the baseline configuration is not specified beyond 'Llama' in the figure captions, and the contest's official Task II evaluation metric is never stated. The paper also asserts it was 'awarded as winner of Task II' without providing a contest result table, a link to the official leaderboard, or any numerical score. Consequently, the reader cannot verify the headline claim, and the comparison between Figure 3 and Figure 4 is not placed in any reproducible experimental context.
  4. [Section 3.1, three-day forward close price] The training signal depends on the assumption that a three-day forward close price is a valid and sufficient label for the market impact of a news headline, but the paper provides no sensitivity analysis for this horizon and no justification beyond 'to facilitate forward-looking impact analysis.' If the relevant market response occurs over a shorter or longer window, or if the response is not monotone in the three-day return, the trained sentiment scores will be misaligned with the trading task. Since the reward function and the sentiment labels are both built on this single horizon, the absence of any robustness check is a load-bearing weakness in the methodology.
minor comments (5)
  1. [Section 2.1] The sentence 'Adjustable parameters enables market adaptation' has a subject–verb agreement error; it should be 'Adjustable parameters enable market adaptation.'
  2. [Section 3.3, second paragraph] The passage beginning 'In Figure 3, the cumulative returns show significant variation between tickers' appears to describe Figure 4 (the baseline model), not Figure 3; the figure labels in the text should be corrected to match the figures they describe.
  3. [Algorithm 1] The prompt template uses parameters 'signal_strength' and 'threshold' without defining their values or ranges, and the scoring examples (e.g., 'Sentiment Score: -8') are not shown to be consistent with the stated output range '[-signal_strength, signal_strength]', since signal_strength is never specified.
  4. [Figure 2] The caption 'Figure 2: FinRL Train-Test-Trade Pipeline' is never referenced in the text, and the pipeline is not described in the methodology section; adding a brief explanation would improve readability.
  5. [References] Several reference entries have formatting inconsistencies, such as trailing periods after DOI URLs and a mix of arXiv identifiers and venue names; for example, reference [1] ends with a period after the URL and reference [10] uses a full arXiv URL rather than a DOI.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training/evaluation split and out-of-sample 2023 comparison prevent the claimed outperformance from reducing to the training objective by construction.

full rationale

FinRLlama's claimed result is that an RLMF-tuned LLaMA prompt outperforms a baseline in signal consistency and trading tightness. The training loop (Section 2.2) fits sentiment scores to a reward derived from actual market outcomes over 2020-2022, while the evaluation (Section 3.1) is explicitly out-of-sample on 2023. This split means the reported comparison is not statistically forced from the fitted values. The paper contains no equations that make the evaluation metric identical to the training objective, and no fitted parameter is renamed as a prediction. The qualitative and internally inconsistent reading of Figures 3 and 4 in Section 3.3 is an evidence-quality and correctness concern, not a circularity concern under the required standard. The only citation to work by acknowledged advisors ([2]) is used for contest task requirements, not as load-bearing support for the outperformance claim. Thus no circular step can be exhibited with the specific reduction required by the analysis rules.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

All free parameters are hand-set or unspecified; the central training signal depends on a three-day forward return label that is assumed without justification. The framework's axioms are domain assumptions about the predictive content of news sentiment and the validity of the comparison, not results established in the paper.

free parameters (5)
  • signal_strength
    Bounds the output sentiment score in Algorithm 1; set by the user, not derived. Controls the scale of sentiment and thus the sensitivity of trading signals.
  • threshold
    In Algorithm 1, threshold marks the boundary between moderately positive/negative sentiment; choice affects which scores trigger long/short/hold actions.
  • prompt scoring examples = -8, 7, 5
    Three hand-assigned example scores in the prompt (Algorithm 1) anchor the model's calibration; they are not derived from data.
  • three-day forward return window = 3 days
    Used as the market outcome label for each headline (Section 3.1); chosen without sensitivity analysis.
  • reward function weights and thresholds
    Described only in prose in Section 2.2; no equation or values, making the training objective underspecified.
assumptions (4)
  • domain assumption Sentiment scores of news headlines predict short-term stock returns
    The entire framework assumes that the sentiment expressed in a headline, as scored by an LLM, carries predictive information about three-day forward returns (Sections 2.2 and 3.1).
  • domain assumption Historical market responses to similar news are informative for current scoring
    The prompt instructs the model to incorporate 'Past Market Responses' and 'Historical Price Patterns' (Algorithm 1), assuming these are available and useful features.
  • domain assumption The base Llama-3.2-3B-Instruct model is a valid baseline for comparison
    The only comparison is to the un-tuned base model; no other baseline or contest baseline is reported (Section 3.3).
  • standard math Adam optimization of the sentiment-reward discrepancy is a valid RL training procedure
    The paper invokes Adam and loss minimization (Section 2.2), treating the procedure as standard; no proof of convergence or policy-improvement guarantee is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024." pith.science (2026). https://pith.science/paper/4FQM45H6

@misc{pith2026250201992,
  author       = {Pith},
  title        = {Pith review of: FinRLlama: A Solution to LLM-Engineered Signals Challenge at FinRL Contest 2024},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4FQM45H6}},
  note         = {Machine review of arXiv:2502.01992}
}
read the original abstract

In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Market Feedback (RLMF). Our framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals. Traditional LLMs, while competent in sentiment analysis, lack contextual alignment for financial market applications. To bridge this gap, we fine-tune the LLaMA-3.2-3B-Instruct model using a custom RLMF prompt design that integrates historical market data and reward-based feedback. Our evaluation shows that this RLMF-tuned framework outperforms baseline methods in signal consistency and achieving tighter trading outcomes; awarded as winner of Task II. You can find the code for this project on GitHub.

Figures

Figures reproduced from arXiv: 2502.01992 by the authors.

Figure 1
Figure 1. FinRLlama Abstract In response to Task II of the FinRL Challenge at ACM ICAIF 2024, this study proposes a novel prompt framework for fine-tuning large language models (LLM) with Reinforcement Learning from Mar￾ket Feedback (RLMF). Our framework incorporates market-specific features and short-term price dynamics to generate more precise trading signals. Traditional LLMs, while competent in sentiment analysis, lack co… view at source ↗
Figure 2
Figure 2. FinRL Train-Test-Trade Pipeline requirements outlined in the ACM ICAIF FinRL 2024 Competition [2]. The reward function is dynamically adjusted on the basis of the strength of the model’s sentiment signal, reinforcing correct predictions, and penalizing errors. The function takes into account the confidence of the model, incorporating adjustable thresholds to assess market direction. For instance, when the sentiment … view at source ↗
Figure 4
Figure 4. Llama Cumulative Returns the highest and lowest performing stocks is narrower compared to the first graph. XOM continues to trend downward, but with a less steep decline. This indicates a model response that may be more conservative, possibly due to model fine-tuning. In [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: FinRLlama Cumulative Returns 3.3 Comparative Analysis In [PITH_FULL_IMAGE:figures/full_fig_p003_3.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 2 canonical work pages

  1. [1]

    Fahad Alhindi, S. M. Raza, and P. Gupta. 2023. Predicting Economic Trends and Stock Market Prices with Deep Learning and Advanced Machine Learn- ing Techniques. MDPI Electronics 13, 17 (2023), 3396. https://doi.org/10.3390/ electronics13173396 Explores deep learning methods for predicting stock market trends

    work page 2023

  2. [2]

    Nikolaus Holzer, Keyi Wang, Kairong Xiao, and Xiao-Yang Liu. 2024. Revisiting Ensemble Methods for Stock Trading and Crypto Trading Tasks at ACM ICAIF FinRL Contest 2023-2024. arXiv preprint (2024). https://doi.org/10.48550/arXiv. 2501.10709

    work page Pith review arXiv doi:10.48550/arxiv.2501.10709 2024

  3. [3]

    Huina Mao Johan Bollen and Xiaojun Zeng. 2011. Twitter mood predicts the stock market. Journal of Computational Science 2, 1 (2011), 1–8. https://doi.org/ 10.1016/j.jocs.2010.12.007

    doi:10.1016/j.jocs.2010.12.007 2011

  4. [4]

    Xiao-Yang Liu, Ziyi Xia, Jingyang Rui, Jiechao Gao, Hongyang Yang, Ming Zhu, Christina Dan Wang, Zhaoran Wang, and Jian Guo. 2022. FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learn- ing. NeurIPS Datasets and Benchmarks (2022), 1–36. https://doi.org/10.48550/ arXiv.2211.03107

    arXiv doi:10.48550/arxiv.2211.03107 2022

  5. [5]

    Xiao-Yang Liu, Hongyang Yang, Qian Chen, Runjia Zhang, Liuqing Yang, Bowen Xiao, and Christina Dan Wang. 2020. FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance. NeurIPS Workshop on Deep Reinforcement Learning (2020). https://doi.org/10.48550/arXiv.2011.09607

    arXiv doi:10.48550/arxiv.2011.09607 2020

  6. [6]

    Tim Loughran and Bill McDonald. 2011. When is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks. The Journal of Finance 66, 1 (Feb. 2011), 35–65. https://doi.org/10.1111/j.1540-6261.2010.01625.x Explores domain-specific dictionaries for financial text analysis

    arXiv 2011

  7. [7]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2015. Human-level control through deep reinforcement learning. Nature 518 (2015), 529–533. https: //doi.org/10.1038/nature14236

    doi:10.1038/nature14236 2015

  8. [8]

    Mulvey, H

    Yuqi Nie, Yaxuan Kong, Xiaowen Dong, John M. Mulvey, H. Vincent Poor, Qing- song Wen, and Stefan Zohren. 2024. A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges. arXiv preprint (2024). https://doi.org/10.48550/arXiv.2406.11903

    arXiv doi:10.48550/arxiv.2406.11903 2024

Show all 11 references
  1. [9]

    Shuo Sun, Rundong Wang, and Bo An. 2023. Reinforcement Learning for Quan- titative Trading. ACM Transactions on Intelligent Systems and Technology 14, 3 (2023), 1–29. https://doi.org/10.1145/3582560

    2023 doi

  2. [10]

    Shubham Vatsal and Harsh Dubey. 2023. A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks. arXiv preprint (2023). https://arxiv.org/pdf/2407.12994 Discusses various prompt engineering methods for different NLP tasks

    2023 arXiv

  3. [11]

    Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M

    Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Chain of Thought Prompt- ing Elicits Reasoning in Large Language Models. arXiv preprint arXiv:2201.11903 (2022). https://arxiv.org/abs/2201.11903 Dem...

    2022 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.