Pith. sign in

REVIEW 3 major objections 8 minor 26 references

AgentFly: Extensible and Scalable Reinforcement Learning for LM Agents

T0 review · 3 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AgentFly trains language-model agents with multi-turn RL by masking environment tokens, achieving reward gains on six tool-use environments.

desk verdict Useful RL-for-agents engineering with a genuinely clean design and code, but the empirical evidence is training reward curves only—so the 'effectiveness' claim is unproven. read the letter →

arxiv 2507.14897 v1 pith:QR556MGW submitted 2025-07-20 cs.AI

Renxi Wang , Rifo Ahmad Genadi , Bilal El Bouardi , Yongxin Wang , Fajri Koto , Zhengzhong Liu , Timothy Baldwin , Haonan Li This is my paper
classification cs.AI
keywords Agent-RLreinforcementlearninglanguagemodelagentsmulti-turntrainingtoken-levelmaskingtoolusescalableframework
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces AgentFly, a framework for training language-model agents with reinforcement learning over multi-turn interactions with tools, APIs, and environments. Its central technical move is to apply the policy-gradient loss and advantages only to the language model's own output tokens, masking out all tokens produced by the environment or tools. This converts single-turn RL algorithms like PPO, GRPO, REINFORCE++, and RLOO into multi-turn agent trainers without altering the underlying objective. Across six environments—code interpretation, web search, Wikipedia retrieval, WebShop, ALFWorld, and ScienceWorld—the authors report that both 3B and 7B instruction-tuned models show rising reward curves, and they document that learning manifests as better tool use, such as querying admissible commands rather than guessing actions. The broader point is that agent RL can be supported by a single extensible and scalable pipeline rather than bespoke per-task systems.

What carries the argument

The load-bearing mechanism is the token mask M_t, which equals 1 for tokens in the LM's own responses and 0 for environment/tool tokens, applied to both the loss and the advantage in multi-turn trajectories. This adapts the single-turn PPO objective to arbitrary turn counts without changing the algorithm's math. The second pillar is the tool abstraction: every external interface (function, API, environment) is a tool, and stateful tools are bound to isolated environment instances drawn from a central resource pool, so rollouts can run asynchronously at scale. The mask supplies the learning-theoretic fix; the asynchronous tool/environment system supplies the scalability.

What would settle it

Re-run AgentFly's WebShop or ScienceWorld training for the 7B model with the same hyperparameters and check whether the reward curve rises above the untrained instruct-model baseline; if it does not, the framework's effectiveness claim fails. To isolate the mask, run a second ablation without the token mask (applying loss over all trajectory tokens): if the unmasked run matches or beats the masked one on a short-horizon task like code-interpreter math, then the mask is not the component doing the work.

Watch

Extended reading notes

Core claim

The central claim is that with token-level masking, ordinary single-turn RL algorithms can optimize language-model agents in multi-turn settings where the model alternates between generating responses and receiving tool observations. For a trajectory (p, (r1, o1), ..., (rk, ok)), the mask M_t is 1 when the token at_t belongs to one of the model's responses r_i and 0 otherwise, and this mask is applied to both the PPO-style loss and the advantage estimate. The authors argue this prevents the model from being penalized or credited for tokens it did not generate. Empirically, they show that all four algorithms converge to similar rewards on a code-interpreter math task, that reward increases across all six environments, that larger models benefit more, and that RL changes behavior: the ALFWorld agent learns to use 'get admissible commands' and 'get task objective' tools, and hallucination (malformed tool calls) decreases over training. The discovery, on the paper's own terms, is that a masked multi-turn objective plus scalable environment management is sufficient to train capable agent policies from an instruct-tuned base.

Load-bearing premise

The framework assumes that computing the policy-gradient loss and advantages only on the language model's generated tokens, and ignoring tokens emitted by tools and environments, yields a correct and stable reinforcement-learning update for multi-turn agent trajectories.

Editorial extensions

If this is right

  • Standard on-policy RL algorithms can be applied to multi-turn agent training without a new objective; only a mask selecting the LM's own tokens is needed.
  • New environments and rewards plug in with a decorator, so a developer can go from an environment wrapper to a training run with a small amount of code.
  • Rollout throughput scales with the number of environment instances in the pool, which is the key to making online agent RL feasible at large batch sizes.
  • The four compared algorithms (PPO, GRPO, REINFORCE++, RLOO) reach similar reward levels on the code task, suggesting the framework is algorithm-agnostic and the gain comes from the masked multi-turn setup.
  • Increasing the maximum number of turns improves final reward on the tested task but makes both reward and gradient norm fluctuate more, indicating a stability/expressiveness trade-off in rollout length.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Masking out tool tokens may weaken long-horizon credit assignment because the advantage reflects only the final outcome; coupling the mask with a learned value function or process rewards on tool feedback could stabilise long trajectories, a testable modification of the paper's scheme.
  • The same mask-and-tool abstraction could be used for multi-agent training, where each agent's tokens are masked independently and shared environments are handled by the central pool.
  • The caching of search queries in the Retrieve environment changes the reward landscape over training; one could test whether the agent learns to exploit the cache by rephrasing queries, which would be an emergent, potentially unintended behavior.
  • The 3B model's failure on ALFWorld under the same framework suggests that a minimum base-model capability is needed for long-horizon tasks; scaling the same setup to larger instruction-tuned models is a natural next experiment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper presents AgentFly, a reinforcement learning (RL) framework for training language model (LM) agents, built on top of the verl training infrastructure. The framework introduces token-level masking to adapt single-turn RL losses (PPO, REINFORCE++, GRPO, RLOO) to multi-turn agent trajectories, a tool abstraction with stateful and non-stateful tools, asynchronous chain-based rollouts, and a centralized environment pool for scalable environment coordination. The authors implement decorator-based tool and reward definitions and provide prebuilt tools/environments including a code interpreter, Search, Retrieve, ScienceWorld, WebShop, and ALFWorld. Experiments train Qwen2.5-Instruct 3B and 7B models on these tasks, reporting reward curves that increase during training. The paper claims that the framework is scalable, extensible, and effective for agent training across diverse tasks.

Significance. If the empirical claims are substantiated, AgentFly would be a useful open-source infrastructure contribution to the emerging area of Agent-RL, addressing real engineering challenges: multi-turn optimization, asynchronous tool execution, environment pooling, and modular tool/reward definitions. The code release and the breadth of integrated environments are concrete strengths. However, the current evidence falls short of demonstrating that the framework 'successfully trains' agents: the only empirical support is training reward curves, with no final task-success metrics, no held-out evaluation, no baseline comparisons (e.g., prompt-based ReAct, SFT, or existing agent RL frameworks), and no variance or seed information. Because several reward definitions (e.g., a format reward of 0.1 for any tool call, and using the highest subgoal reward in ScienceWorld) can increase without improving task completion, the training curves do not by themselves establish genuine agent learning. The central claim therefore rests on a load-bearing but unverified empirical foundation.

major comments (3)
  1. [§3, §3.2, Figures 4 and 6, Appendix B] The only empirical evidence for the central claim of 'successful agent training' is training reward curves computed on training rollouts. No final task-success rates, held-out evaluations, or comparisons against untrained/initial checkpoints or baseline methods are reported. This is particularly concerning because several reward designs can increase without improving task competence: in B.1, the code-interpreter reward gives 0.1 just for calling a tool, so an agent can raise reward by learning to invoke the interpreter even if final answers remain wrong; in B.5, the ScienceWorld reward is defined as the highest subgoal reward achieved at any point, so curves reflect partial progress rather than successful task completion. Additionally, Figure 4 shows the ALFWorld reward for the 3B model 'keeps around zero', yet §3.2 states that 'both the 3B and 7B models can be effectively trained'. The abstract's claim of demonstrated effectiveness is therefore not supported by the presented metrics.
  2. [§3.1, Figure 3] The comparison of PPO, REINFORCE++, GRPO, and RLOO reports only reward curves with no variance, no number of seeds, no final reward values, and no statistical significance. The observation that 'REINFORCE++ shows slightly lower performance' is followed by a speculation that masking the advantage function is the cause, but no ablation isolates the effect of masking. Without controlling for hyperparameters or reporting run-to-run variance, the algorithm comparison is not informative, and the masking hypothesis remains untested.
  3. [§2.2, Eq. (1)] The multi-turn masking adaptation is the core algorithmic contribution, but its correctness is asserted rather than analyzed or ablated. The paper does not specify how advantages are computed over multi-turn trajectories (e.g., whether a value network sees the full context including observations, and whether GAE is computed over all tokens or only LM-generated tokens). The claim that masking 'ensures that the model learns only from its own outputs' is plausible, but the potential effect on credit assignment across turns is not examined. Given that the paper itself attributes the REINFORCE++ result to this masking, an ablation or at least a formal description of the advantage computation is needed to establish that the update optimizes the intended objective.
minor comments (8)
  1. [§2.2, Eq. (1)] The notation in Eq. (1) is ambiguous: 'Mt = 1 if at ∈ r1' uses set membership for a token in a response, but it is not defined how token indices map to turns, nor whether the context st includes prior observations. Please clarify the notation and define the multi-turn context.
  2. [§3.1] The sentence 'We evaluate four RL algorithms: [11], REINFORCE++ [6], GRPO [12], and RLOO [1]' should spell out 'PPO' instead of citing the reference number alone.
  3. [Figure 3] The legend contains a typo: 'REINFOCE++' should be 'REINFORCE++'.
  4. [Figure 5] The four subplots have no y-axis labels; the caption lists the metrics (avg_turns, hallucination, alfworld_admissible_commands, alfworld_step), but the plots themselves should be labeled for readability.
  5. [Appendix B.1] The reward description 'if the model calls at least one tool, we give it a format reward 0.1, if it further gets the answer correct, we give it a reward of 1.0' is unclear about whether the rewards are additive or whether 1.0 replaces 0.1; please specify the exact reward-scale logic.
  6. [Appendix B.3] The code snippet '*, reward_val, *, * = await env.step("")' is not valid Python; please provide a correct unpacking or illustrate the intended interface in pseudocode.
  7. [References] Several references are incomplete or garbled, notably [6] (the REINFORCE++ citation) and [15] (RL-Factory, listed as a bare GitHub URL with no author or year); these should be cleaned up before publication.
  8. [§2.4] The queueing behavior when no environment instance is available is described, but it is unclear whether requests can deadlock if all environments are occupied and a chain times out; please clarify the timeout and release policy.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AgentFly's claims rest on external benchmarks and standard RL machinery, rather than on fitted inputs or self-citation.

full rationale

AgentFly is a systems paper whose core technical contribution is an engineering adaptation: extending PPO-style training to multi-turn agent trajectories via token-level masking. The masking rule (Section 2.2) is explicitly defined as a direct generalization of the standard single-turn PPO mask, not as a quantity that is fitted from the data it later claims to predict. The empirical validation uses external, pre-existing environments and benchmarks (ALFWorld, WebShop, ScienceWorld, HotpotQA, SimpleRL-Zoo math data) and standard RL algorithms (PPO, REINFORCE++, GRPO, RLOO). Rewards are defined by the environments or by stated rule-based functions (e.g., a 0.1 format reward plus a 1.0 correctness reward for the code interpreter), and no fitted parameter is renamed as a prediction. The paper's self-citations (ScienceWorld [18], ToolGen [17]) are not load-bearing: ScienceWorld is cited as an external benchmark that the framework wraps, and ToolGen is related work. The central claims about scalability and extensibility are supported by the framework's architecture and by training curves on these external tasks. The absence of final task-success rates and baseline comparisons is a missing-evidence concern, not a circularity concern, because the reported reward improvements are not constructed to equal the evaluation metric by definition.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central claim is about software functionality and empirical training results, not a derived physical law. Listed free parameters are training hyperparameters and reward shaping constants that influence the reported curves. The framework relies on standard RL assumptions and the correctness of the benchmark environments.

free parameters (5)
  • learning_rate = 5e-7
    Chosen a priori for all runs; standard for RLHF, but results depend on it.
  • n_chains = 16
    Number of sampled sequences per query during rollout; set experimentally.
  • pool_size = 8
    Number of environment instances per tool; affects parallelism and resource consumption.
  • max_turns = 4 or 8
    Maximum agent turns; the short/long comparison uses these two values.
  • reward_shaping = format_reward=0.1, correct_reward=1.0
    Hand-specified reward constants for the code interpreter task; influence training signal.
assumptions (3)
  • domain assumption Reinforcement learning on language model outputs with advantages computed from external rewards leads to improved agent behavior.
    The entire framework is built on this premise; the paper validates it empirically but does not prove it.
  • domain assumption The benchmark environments (ALFWorld, WebShop, ScienceWorld, HotpotQA) provide reliable reward signals that reflect task success.
    Section 3 and Appendix B rely on these environments as ground truth for rewards without independent verification.
  • domain assumption The async tool execution and pooled environment resets do not introduce spurious training signals.
    The framework's scalability claims depend on correct isolation and reset behavior, which is not formally verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AgentFly: Extensible and Scalable Reinforcement Learning for LM Agents." pith.science (2026). https://pith.science/paper/QR556MGW

@misc{pith2026250714897,
  author       = {Pith},
  title        = {Pith review of: AgentFly: Extensible and Scalable Reinforcement Learning for LM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QR556MGW}},
  note         = {Machine review of arXiv:2507.14897}
}
read the original abstract

Language model (LM) agents have gained significant attention for their ability to autonomously complete tasks through interactions with environments, tools, and APIs. LM agents are primarily built with prompt engineering or supervised finetuning. At the same time, reinforcement learning (RL) has been explored to enhance LM's capabilities, such as reasoning and factuality. However, the combination of the LM agents and reinforcement learning (Agent-RL) remains underexplored and lacks systematic study. To this end, we built AgentFly, a scalable and extensible Agent-RL framework designed to empower LM agents with a variety of RL algorithms. Our framework supports multi-turn interactions by adapting traditional RL methods with token-level masking. It features a decorator-based interface for defining tools and reward functions, enabling seamless extension and ease of use. To support high-throughput training, we implement asynchronous execution of tool calls and reward computations, and design a centralized resource management system for scalable environment coordination. We also provide a suite of prebuilt tools and environments, demonstrating the framework's effectiveness through successful agent training across multiple tasks.

Figures

Figures reproduced from arXiv: 2507.14897 by the authors.

Figure 1
Figure 1. Overview of the AgentFly training framework. The left part follows the standard RL [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Examples of non-stateful and stateful tools. For both tools, their schemas (name, description, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Reward curves for Qwen2.5-Instruct 3B and 7B models. For ALFWorld, we find it is too [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (9 more)
Figure 3
Figure 3. Figure 3: Reward curves of the four algorithms trained on code interpreter to solve math tasks. We detail our experiments in this section, where we first introduce the running of different RL al￾gorithms for multi-turn agent, then we show the experiments on various tools and env…
Figure 5
Figure 5. Figure 5: Statistics of number of tool calls, hallu [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Reward and gradient norm change for training with a maximum of 4 and 8 turns respec￾tively. We compare the training with different turns to show the impact of number of turns. We use the code tool with GRPO. For short-turn setting, we allow a maximum of 4 turns, while …
Figure 7
Figure 7. Figure 7: Examples of reward function that does not require and require environments to calculate [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: An illustration of the client-server interaction for executing an action in ALFWorld. The [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: An agent trajectory for the task "Put the phone on the bed." This example highlights the [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Environment info for WebShop, appended as system prompt to the model context [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Environment info for ScienceWorld, appended as system prompt to the model context [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Agent trajectory for the task "Your task is to find a plant. First, focus on the thing. Then, [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 15 canonical work pages

  1. [1]

    Ahmadian, C

    A. Ahmadian, C. Cremer, M. Gallé, M. Fadaee, J. Kreutzer, O. Pietquin, A. Üstün, and S. Hooker. Back to basics: Revisiting REINFORCE-style optimization for learning from human feedback in LLMs. In L.-W. Ku, A. Martins, and V . Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers)...

    work page 2024

  2. [2]

    B. Chen, C. Shu, E. Shareghi, N. Collier, K. Narasimhan, and S. Yao. Fireact: Toward language agent fine-tuning, 2023

    work page 2023

  3. [3]

    L. Feng, Z. Xue, T. Liu, and B. An. Group-in-group policy optimization for llm agent training. arXiv preprint arXiv:2505.10978, 2025

    arXiv 2025

  4. [4]

    W. Fu, J. Gao, X. Shen, C. Zhu, Z. Mei, C. He, S. Xu, G. Wei, J. Mei, J. Wang, T. Yang, B. Yuan, and Y . Wu. Areal: A large-scale asynchronous reinforcement learning system for language reasoning, 2025

    work page 2025

  5. [5]

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  6. [6]

    J. Hu, J. K. Liu, and W. Shen. Reinforce++: An efficient rlhf algorithm with robustness to both prompt and reward models. URL https://arxiv. org/abs/2501, 3262:32–33, 2025

    work page 2025

  7. [7]

    B. Jin, H. Zeng, Z. Yue, J. Yoon, S. Arik, D. Wang, H. Zamani, and J. Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516, 2025

    arXiv 2025

  8. [8]

    Ouyang, J

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems , 35:27730–27744, 2022. 7

    work page 2022

Show all 26 references
  1. [9]

    Y . Qin, S. Liang, Y . Ye, K. Zhu, L. Yan, Y . Lu, Y . Lin, X. Cong, X. Tang, B. Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. In The Twelfth International Conference on Learning Representations , 2024

    2024

  2. [10]

    Qwen, :, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, ...

    2025

  3. [11]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  4. [12]

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  5. [13]

    Sheng, C

    G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y . Peng, H. Lin, and C. Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256 , 2024

    2024 arXiv

  6. [14]

    Shridhar, X

    M. Shridhar, X. Yuan, M. Côté, Y . Bisk, A. Trischler, and M. J. Hausknecht. ALFWorld: Aligning text and embodied environments for interactive learning.CoRR, abs/2010.03768, 2021. Presented at the 9th International Conference on Learning Representations (ICLR 2021)

    2010 arXiv

  7. [15]

    Rl-factory

    Simple-Efficient. Rl-factory. https://github.com/Simple-Efficient/RL-Factory , 2025

    2025

  8. [16]

    Sumers, S

    T. Sumers, S. Yao, K. Narasimhan, and T. Griffiths. Cognitive architectures for language agents. Transactions on Machine Learning Research

  9. [17]

    R. Wang, X. Han, L. Ji, S. Wang, T. Baldwin, and H. Li. Toolgen: Unified tool retrieval and calling via generation. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  10. [18]

    R. Wang, P. Jansen, M.-A. Côté, and P. Ammanabrolu. Scienceworld: Is your agent smarter than a 5th grader? arXiv preprint arXiv:2203.07540, 2022

    2022 arXiv

  11. [19]

    Z. Wang, K. Wang, Q. Wang, P. Zhang, L. Li, Z. Yang, X. Jin, K. Yu, M. N. Nguyen, L. Liu, E. Gottlieb, Y . Lu, K. Cho, J. Wu, L. Fei-Fei, L. Wang, Y . Choi, and M. Li. Ragen: Understand- ing self-evolution in llm agents via multi-turn reinforcement learning, 2025

    2025

  12. [20]

    Q. Wu, G. Bansal, J. Zhang, Y . Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. Autogen: Enabling next-gen llm applications via multi-agent conversations. In First Conference on Language Modeling, 2024

    2024

  13. [21]

    Z. Yang, P. Qi, S. Zhang, Y . Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, editors, Proceedings of the 2018 Conference on Empirical Meth...

    2018

  14. [22]

    S. Yao, H. Chen, J. Yang, and K. Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. arXiv preprint arXiv:2207.01206, 2022

    2022 arXiv

  15. [23]

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao. React: Synergizing rea- soning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

    2023

  16. [24]

    A. Zeng, M. Liu, R. Lu, B. Wang, X. Liu, Y . Dong, and J. Tang. AgentTuning: Enabling generalized agent abilities for LLMs. In L.-W. Ku, A. Martins, and V . Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024 , pages 3053–3077, Bangkok, Thail...

    2024

  17. [25]

    qa_f1_reward

    W. Zeng, Y . Huang, Q. Liu, W. Liu, K. He, Z. Ma, and J. He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild, 2025. 8 A Reward Function Example @reward(name="qa_f1_reward") def qa_f1_reward( prediction: str, answer: str, traj...

    2025

  18. [2018]

    Association for Computational Linguistics

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.