Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

RL-LLM-DT: An Automatic Decision Tree Generation Method Based on RL Evaluation and LLM Enhancement

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Repeatedly training an RL opponent to beat a decision tree and having an LLM repair the tree's losing plays can produce a top-ranked, interpretable game strategy without human intervention, as demonstrated in curling.

desk verdict A plausible RL+LLM decision-tree refinement loop, but the paper never shows the RL feedback is what drives the gains. read the letter →

arxiv 2412.11417 v2 pith:THTG5FSW submitted 2024-12-16 cs.AI cs.LG

Junjie Lin , Jian Zhao , Lin Liu , Yue Deng , Youpeng Zhao , Lanxiao Huang , Xia Lin , Wengang Zhou
show 1 more author
Houqiang Li
This is my paper · ORCID
classification cs.AIcs.LG
keywords decisiontreelargelanguagemodelsreinforcementlearninggameAIcurlingstrategyrefinementinterpretabilityPPO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a fully automatic loop can strengthen an interpretable decision-tree strategy in a two-player zero-sum game, using no human expert input after the first rough draft. The loop works by training a reinforcement-learning agent to beat the current tree, feeding the recorded losing games to a large language model, and having the model propose and code an improved tree. The authors demonstrate the loop on a curling game, where three refinement cycles produce a decision tree that ranks first among 34 submitted AIs and scores 0.93 against unseen opponents, ahead of a human-designed tree's 0.80. The result matters because it suggests that LLMs can substitute for human strategists in the iterative "find the bug, fix the strategy" part of game AI development.

What carries the argument

The load-bearing mechanism is the RL-LLM-DT loop, composed of three modules: the LLM Coder, which turns a tactic description into executable decision-tree Python code; the RL Debugger, which trains a distributed-PPO neural policy as an adversary against that code and records the tree's losing games; and the LLM Critic, which receives the game rules, the tactic, the code, and the losing trajectories and returns a diagnosed, improved tactic. The loop works because the RL agent is a targeted flaw detector -- it searches specifically for ways to beat the current tree -- and the LLM is a strategy synthesizer that converts those concrete failure traces into new conditional rules rather than just tweaking parameters. The paper's key evidence that the mechanism is doing the work is the win-rate curve in Figure 7, where later tree versions require noticeably longer RL training before a neural model can beat them.

What would settle it

Evaluate the final tree against a set of fresh RL opponents trained with different random seeds, reward functions, or architectures: if any of them beats the tree at a rate well above the near-50% level that would indicate the loop truly closed, the claim that the iterative process converged to a robust strategy is undercut. A second check is whether the RL debugger's final trained policy is actually unable to beat the tree -- if the loop terminated because the LLM stopped proposing changes rather than because the adversary failed, the stopping rule did not do what it claims.

Watch

Extended reading notes

Core claim

The central claim is that a decision tree can be iteratively refined into a top-performing, interpretable game policy by alternating two roles: a reinforcement-learning "debugger" that trains a neural policy specifically to defeat the current tree and collect the trajectories where the tree loses, and a large language model "critic" that reads those trajectories, diagnoses the tactical flaws, and writes out an improved tree as executable Python code. The paper calls this loop RL-LLM-DT, and reports that in the Jidi curling environment each cycle makes the tree harder for a freshly trained RL opponent to beat, until the third generated tree reaches rank 1 among 34 AIs and a platform evaluation score of 0.93, surpassing the human-designed tree (0.80) from the authors' earlier work. The termination condition is either that the RL debugger can no longer find a flaw or that the LLM critic can no longer propose a meaningfully different improvement; in the reported runs the loop stopped when the LLM produced only cosmetically different code.

Load-bearing premise

The whole improvement loop assumes that the RL-trained opponent is a broad enough adversary that the losing games it produces expose the decision tree's most important weaknesses; if it exploits only a narrow style of play, the LLM critic never sees the other failures and the final tree's robustness is unverified.

Editorial extensions

If this is right

  • If the loop is correct, interpretable decision-tree policies can reach competitive levels in strategy games without a human expert iterating on them manually.
  • The same loop could be applied to other two-player zero-sum games, such as chess, card games, or StarCraft, by swapping in the relevant environment rules and interface definitions.
  • The termination condition (RL finds no flaw or LLM cannot improve) gives a practical stopping rule for automatic strategy refinement, and the authors observe that it fired when the LLM Critic's suggestions became semantically empty.
  • Because the final artifact is a decision tree, the resulting AI remains inspectable and explainable, unlike the neural opponent used to probe it.
  • The approach suggests that RL evaluation and LLM code generation are complementary: RL supplies concrete counterexamples, LLM supplies the repair.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test is whether the RL Debugger's coverage determines the ceiling: if the trained opponent exploits only a narrow style of play, the LLM Critic is blind to other weaknesses, so the final tree's robustness to diverse unseen opponents is not guaranteed by the rank-1 result alone.
  • The method might transfer to non-game settings where policies are rule-based and failure traces are available, such as robot control or automated negotiation, with the LLM proposing condition changes rather than full code rewrites.
  • One testable prediction is that the loop's improvement rate should degrade if the RL opponent's exploration is restricted (fewer seeds, less entropy), because the critic would receive fewer distinct failure modes to analyze.
  • The comparison to the human-designed tree is suggestive but not controlled: the human tree came from earlier work with a different starting point, so a fair test would run the same loop seeded with the human tree and see whether it also improves.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes RL-LLM-DT, an iterative framework for refining decision-tree policies in a curling game. An RL agent is trained against the current decision tree to collect losing trajectories; an LLM critic analyzes these trajectories and proposes an improved tactic; an LLM coder implements the tactic as Python code. The loop terminates when the RL debugger cannot find further flaws or the LLM fails to propose an improvement. The authors report that the third iteration of the decision tree ranks first on the Jidi platform with a score of 0.93, outperforming a human-designed decision tree from their prior work (score 0.80).

Significance. If the causal role of the RL feedback were established, the framework would be a meaningful step toward automating decision-tree strategy improvement by combining LLM prior knowledge with targeted adversarial search. The paper is clearly structured, the code is released, and the evaluation is performed on an external platform against 34 submitted AIs, which is a strength relative to many internal-gym evaluations. The resulting final tree is more complex than the human-designed version and includes strategic distinctions such as last-throw behavior and obstacle protection. However, the current empirical evidence does not isolate the contribution of the RL feedback loop from the LLM's own prior knowledge, and the leaderboard-based claim lacks statistical support. These issues are load-bearing for the central claim that the RL-guided iterative process is what drives the improvement.

major comments (4)
  1. [V-B and Table I] The causal role of the RL feedback is not established. Section V-B explicitly states that the LLM Coder, without any RL feedback, already added a protective operation not present in the initial tactic. Table I then shows scores of 0.47, 0.90, and 0.93 for Tree I, Tree II, and Tree III, but there is no baseline in which the LLM is re-prompted without the lose trajectories. The observed improvement is compatible with the LLM simply being re-prompted with its own previous code and the game rules. To support the central claim that the RL Debugger's losing trajectories drive the refinement, the authors should run an LLM-only control (identical prompts but omitting Itrace) and compare the generated tree's structure and leaderboard score.
  2. [V-A and Table I] The leaderboard scores are reported as single point values without error bars, game counts, or a description of the opponent pool and evaluation protocol. The central claim that the final tree 'ranks first among 34 curling AIs' rests on a single online submission, which could be sensitive to the particular set of opponents, game randomness, and the leaderboard's updating schedule. The authors should report multiple independent evaluations or submissions, with at least the mean and standard deviation of the score, the number of games per evaluation, and the date or version of the leaderboard used.
  3. [IV-C and Figure 7] The RL Debugger's coverage of the decision tree's weakness space is unverified. The method assumes that the losing trajectories collected by the RL-trained neural policy expose the most important flaws of the current tree, but Figure 7 only shows unlabeled win-rate curves, with no axes labels, no game counts, and no indication of how many distinct losing states or strategies were encountered. Without a coverage or diversity metric for the trajectories, the termination condition 'RL can't find any flaw' (Section IV-A) is not verifiable, and the final tree's robustness against a broad set of opponents is not established.
  4. [V-D and Figure 10] The stopping criterion is under-specified and the final iteration is not fully reported. Section V-D states that the loop terminates when the LLM Critic fails to improve the tree, and Figure 10 shows that Tree 3 and Tree 4 are semantically identical except for comments. However, Table I does not include a score for Tree 4, and the paper does not explain why Tree III, rather than a later version, is the reported final result. The stopping rule also lacks a performance-regression check: the framework stops when the LLM cannot generate different code, but it does not verify that the new tree is at least as strong as the previous one before proceeding.
minor comments (6)
  1. [Figure 7] The winning-rate figure has no labeled axes, no legend, and no explicit indication of the x-axis unit (episodes, steps, or wall-clock time). The three curves presumably correspond to the three tree versions, but this should be stated in the caption and the figure itself.
  2. [III-B1] The heading 'Markov Decisioin Process' contains a typo; it should read 'Markov Decision Process.'
  3. [III-B1, Eq. (2)] The expectation in Eq. (2) is ambiguous because the distribution over the initial state s0 is not specified, and the notation in Eq. (3) for J(π) does not match the state-dependent definition in Eq. (2). Clarify that Eq. (2) conditions on s0 = s and that Eq. (3) averages over an initial-state distribution.
  4. [Figure 8] The prompt text contains grammatical errors, for example 'The first game end begins' and 'Please summarize why the code cause the result.' These should be corrected for clarity.
  5. [Figure 10] The code uses the threshold `significant_distance` without defining its value or explaining how it is chosen; this parameter should be documented in Section IV or an appendix for reproducibility.
  6. [V-A] The abstract and conclusion repeat 'ranks first among 34 curling AIs' without specifying the evaluation date or the leaderboard version; a timestamp or archived snapshot would make the claim reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RL-LLM-DT loop is an empirical generate-and-evaluate pipeline whose final claim is checked against an external leaderboard, not derived from its inputs by construction.

full rationale

The paper's central claim is that an RL-trained adversary's losing trajectories, when fed to an LLM critic and coder, yield iteratively stronger decision-tree code, ultimately ranked first on the Jidi platform. Nothing in the formal chain (Eqs. 1-12) defines the improved tree as the RL output or the RL loss; the tree is generated by an LLM and evaluated externally on Jidi against 34 submitted AIs. The only self-citation is to the authors' prior human-refined tree [9], used as a comparison baseline and motivation, not as an unverified premise that forces the conclusion. Concerns that the RL component might be superfluous (because the LLM Coder already adds a protective operation and no LLM-only ablation is reported) are causal-attribution or experimental-design concerns, not circularity: there is no equation or definition by which the leaderboard result reduces to the input trajectory. The paper even documents a saturation point where the LLM produces identical code and the loop terminates, which is consistent with an honest empirical stopping rule. Therefore no load-bearing circular step is exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or mathematical entities are introduced; the framework's modules are software components. The principal uncharged inputs are the LLM's latent knowledge and the Jidi environment's evaluation.

free parameters (4)
  • PPO hyperparameters (learning rate, clip epsilon, coefficients, batch size, actor count) = not reported
    Set by hand; affect the strength and diversity of the RL adversary, which determines which tree flaws are discovered.
  • Action discretization resolution = not reported
    Continuous throw angle and magnitude are discretized by uniform sampling (Section V-C); grid size changes the precision of shots and therefore which strategies are possible.
  • significant_distance threshold = not reported
    Appears in the LLM-generated decision tree code as self.significant_distance to decide whether an opponent stone is a threat; the value is chosen by the LLM and is not stated in the paper.
  • Win-rate stabilization threshold = not reported
    The RL Debugger trains until convergence and when the win rate stabilizes (Algorithm 2, Section IV-C); the threshold is not defined, making the strength of the adversary unspecified.
assumptions (4)
  • domain assumption The curling game can be modeled as a finite MDP and PPO with GAE converges to a near-optimal policy against the decision tree.
    Invoked in Sections III-B and IV-C; the RL Debugger's usefulness depends on the trained policy being strong.
  • domain assumption Template matching recovers exact stone coordinates from the 30x30 observation.
    Section III-A2 states this pre-processing is used; coordinate errors would corrupt both tree and policy inputs.
  • domain assumption Jidi platform leaderboard scores measure true playing strength against unseen opponents.
    Section V-A uses rank and score as the principal evidence; the opponent pool is uncontrolled and changes over time.
  • ad hoc to paper LLM-generated code is executable and semantically faithful to the intended tactic.
    The LLM Coder is trusted to translate tactic descriptions into Python (Section IV-B); the paper shows successful examples but does not verify every iteration systematically.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RL-LLM-DT: An Automatic Decision Tree Generation Method Based on RL Evaluation and LLM Enhancement." pith.science (2026). https://pith.science/paper/THTG5FSW

@misc{pith2026241211417,
  author       = {Pith},
  title        = {Pith review of: RL-LLM-DT: An Automatic Decision Tree Generation Method Based on RL Evaluation and LLM Enhancement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/THTG5FSW}},
  note         = {Machine review of arXiv:2412.11417}
}
read the original abstract

Traditionally, AI development for two-player zero-sum games has relied on two primary techniques: decision trees and reinforcement learning (RL). A common approach involves using a fixed decision tree as one player's strategy while training an RL agent as the opponent to identify vulnerabilities in the decision tree, thereby improving its strategic strength iteratively. However, this process often requires significant human intervention to refine the decision tree after identifying its weaknesses, resulting in inefficiencies and hindering full automation of the strategy enhancement process. Fortunately, the advent of Large Language Models (LLMs) offers a transformative opportunity to automate the process. We propose RL-LLM-DT, an automatic decision tree generation method based on RL Evaluation and LLM Enhancement. Given an initial decision tree, the method involves two important iterative steps. Response Policy Search: RL is used to discover counter-strategies targeting the decision tree. Policy Improvement: LLMs analyze failure scenarios and generate improved decision tree code. In our method, RL focuses on finding the decision tree's flaws while LLM is prompted to generate an improved version of the decision tree. The iterative refinement process terminates when RL can't find any flaw of the tree or LLM fails to improve the tree. To evaluate the effectiveness of this integrated approach, we conducted experiments in a curling game. After iterative refinements, our curling AI based on the decision tree ranks first on the Jidi platform among 34 curling AIs in total, which demonstrates that LLMs can significantly enhance the robustness and adaptability of decision trees, representing a substantial advancement in the field of Game AI. Our code is available at https://github.com/Linjunjie99/RL-LLM-DT.

Figures

Figures reproduced from arXiv: 2412.11417 by the authors.

Figure 1
Figure 1. An illustration of the virtual environment of curling [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. An illustration of the overall framework. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The final decision tree developed by human in our [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 6
Figure 6. Figure 6: Even the refined decision tree tactic is not very [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: LLM Coder output for the initial decision tree. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: The winning rate of the policy model during training [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 9
Figure 9. Figure 9: An LLM Critic output example. corresponding executable Python code implemented by the LLM Coder, and the trajectory of the lost game as input. The LLM Critic is tasked with diagnosing the reasons behind the decision tree code’s failure in the curling game against the c…
Figure 8
Figure 8. Figure 8: An illustration about the LLM Critic prompt. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 10
Figure 10. Figure 10: Decision Tree 3’s and Decision Tree 4’s Python codes [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Armed Bandits-Based Optimization of Decision Trees

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    A multi-armed bandit-based dynamic pruning method for decision trees is proposed, reporting improved generalization over greedy cost-complexity and reduced-error pruning on benchmark datasets.

Reference graph

Works this paper leans on

33 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [9]

    Mastering curling with rl-revised decision tree,

    J. Lin, Y . Gong, J. Zhao, W. Zhou, and H. Li, “Mastering curling with rl-revised decision tree,” in 2023 IEEE Conference on Games (CoG) . IEEE, 2023, pp. 1–8

    work page 2023

  2. [1]

    Emulating human play in a leading mobile card game,

    H. Baier, A. Sattaur, E. J. Powley, S. Devlin, J. Rollason, and P. I. Cowling, “Emulating human play in a leading mobile card game,” IEEE Transactions on Games (TOG) , vol. 11, no. 4, pp. 386–395, 2018

    work page 2018

  3. [2]

    Strategy generation for multiunit real-time games via voting,

    C. Silva, R. O. Moraes, L. H. Lelis, and K. Gal, “Strategy generation for multiunit real-time games via voting,” IEEE Transactions on Games (TOG), vol. 11, no. 4, pp. 426–435, 2018

    work page 2018

  4. [3]

    Mastering the game of go with deep neural networks and tree search,

    D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V . Panneershelvam, M. Lanctot et al., “Mastering the game of go with deep neural networks and tree search,” Nature, vol. 529, no. 7587, pp. 484–489, 2016

    2016

  5. [4]

    Mastering the game of go without human knowledge,

    D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton et al. , “Mastering the game of go without human knowledge,” Nature, vol. 550, no. 7676, pp. 354–359, 2017

    work page 2017

  6. [5]

    Dota 2 with large scale deep reinforcement learning,

    C. Berner, G. Brockman, B. Chan, V . Cheung, P. D˛ ebiak, C. Dennison, D. Farhi, Q. Fischer, S. Hashme, C. Hesse et al. , “Dota 2 with large scale deep reinforcement learning,” arXiv preprint arXiv:1912.06680 , 2019

    arXiv 1912

  7. [6]

    Grand- master level in starcraft ii using multi-agent reinforcement learning,

    O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev et al., “Grand- master level in starcraft ii using multi-agent reinforcement learning,” Nature, vol. 575, no. 7782, pp. 350–354, 2019

    2019

  8. [7]

    Full douzero+: Improving doudizhu ai by opponent modeling, coach-guided training and bidding learning,

    Y . Zhao, J. Zhao, X. Hu, W. Zhou, and H. Li, “Full douzero+: Improving doudizhu ai by opponent modeling, coach-guided training and bidding learning,” IEEE Transactions on Games (ToG) , 2023

    work page 2023

Show all 33 references
  1. [8]

    Danzero+: Dominating the guandan game through reinforcement learning,

    Y . Zhao, Y . Lu, J. Zhao, W. Zhou, and H. Li, “Danzero+: Dominating the guandan game through reinforcement learning,” IEEE Transactions on Games (ToG), 2024

    2024

  2. [10]

    Emergent abilities of large language models,

    J. Wei, Y . Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler et al., “Emergent abilities of large language models,” arXiv preprint arXiv:2206.07682 , 2022

    2022 arXiv

  3. [11]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022

    2022 arXiv

  4. [12]

    Do as i can, not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman et al. , “Do as i can, not as i say: Grounding language in robotic affordances,” arXiv preprint arXiv:2204.01691, 2022

    2022 arXiv

  5. [13]

    Toolformer: Language models can teach themselves to use tools,

    T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,” Advances in Neural Infor- mation Processing Systems (NeurIPS) , vol. 36, 2024

    2024

  6. [14]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,

    Y . Shen, K. Song, X. Tan, D. Li, W. Lu, and Y . Zhuang, “Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,” Advances in Neural Information Processing Systems (NeurIPS) , vol. 36, 2024

    2024

  7. [15]

    Webgpt: Browser- assisted question-answering with human feedback,

    R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V . Kosaraju, W. Saunders et al., “Webgpt: Browser- assisted question-answering with human feedback,” arXiv preprint arXiv:2112.09332, 2021

    2021 arXiv

  8. [16]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence,

    D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y . Wu, Y . Li et al., “Deepseek-coder: When the large language model meets programming–the rise of code intelligence,” arXiv preprint arXiv:2401.14196, 2024

    2024 arXiv

  9. [17]

    Qwen2. 5-coder technical report,

    B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu et al. , “Qwen2. 5-coder technical report,” arXiv preprint arXiv:2409.12186, 2024

    2024 arXiv

  10. [18]

    Google research football: A novel reinforcement learning environment,

    K. Kurach, A. Raichuk, P. Sta ´nczyk, M. Zaj ˛ ac, O. Bachem, L. Espeholt, C. Riquelme, D. Vincent, M. Michalski, O. Bousquet et al. , “Google research football: A novel reinforcement learning environment,” in AAAI Conference on Artificial Intelligence (AAAI) , vol. 34, no. 04...

    2020

  11. [19]

    Fever basketball: A complex, flexible, and asynchronized sports game environment for multi-agent reinforcement learning,

    H. Jia, Y . Hu, Y . Chen, C. Ren, T. Lv, C. Fan, and C. Zhang, “Fever basketball: A complex, flexible, and asynchronized sports game environment for multi-agent reinforcement learning,” arXiv preprint arXiv:2012.03204, 2020

    2012 arXiv

  12. [20]

    Suphx: Mastering mahjong with deep reinforcement learning,

    J. Li, S. Koyamada, Q. Ye, G. Liu, C. Wang, R. Yang, L. Zhao, T. Qin, T.-Y . Liu, and H.-W. Hon, “Suphx: Mastering mahjong with deep reinforcement learning,” arXiv preprint arXiv:2003.13590 , 2020

    2003 arXiv

  13. [21]

    Douzero: mastering doudizhu with self-play deep reinforcement learning,

    D. Zha, J. Xie, W. Ma, S. Zhang, X. Lian, X. Hu, and J. Liu, “Douzero: mastering doudizhu with self-play deep reinforcement learning,” in International Conference on Machine Learning (ICML) . PMLR, 2021, pp. 12 333–12 344

    2021

  14. [22]

    Chessgpt: Bridging policy learning and language mod- eling,

    X. Feng, Y . Luo, Z. Wang, H. Tang, M. Yang, K. Shao, D. Mguni, Y . Du, and J. Wang, “Chessgpt: Bridging policy learning and language mod- eling,” Advances in Neural Information Processing Systems (NeurIPS) , vol. 36, 2024

    2024

  15. [23]

    V oyager: An open-ended embodied agent with large language models,

    G. Wang, Y . Xie, Y . Jiang, A. Mandlekar, C. Xiao, Y . Zhu, L. Fan, and A. Anandkumar, “V oyager: An open-ended embodied agent with large language models,” arXiv preprint arXiv:2305.16291 , 2023

    2023 arXiv

  16. [24]

    Minedojo: Building open- ended embodied agents with internet-scale knowledge,

    L. Fan, G. Wang, Y . Jiang, A. Mandlekar, Y . Yang, H. Zhu, A. Tang, D.-A. Huang, Y . Zhu, and A. Anandkumar, “Minedojo: Building open- ended embodied agents with internet-scale knowledge,” Advances in Neural Information Processing Systems (NeurIPS) , vol. 35, pp. 18 343– 18 362, 2022

    2022

  17. [25]

    Large language models play starcraft ii: Benchmarks and a chain of summarization approach,

    W. Ma, Q. Mi, Y . Zeng, X. Yan, Y . Wu, R. Lin, H. Zhang, and J. Wang, “Large language models play starcraft ii: Benchmarks and a chain of summarization approach,” arXiv preprint arXiv:2312.11865 , 2023

    2023 arXiv

  18. [26]

    Pok \’ellmon: A human-parity agent for pok \’emon battles with large language models,

    S. Hu, T. Huang, and L. Liu, “Pok \’ellmon: A human-parity agent for pok \’emon battles with large language models,” arXiv preprint arXiv:2402.01118, 2024

    2024 arXiv

  19. [27]

    Towards general computer control: A multi- modal agent for red dead redemption ii as a case study,

    W. Tan, Z. Ding, W. Zhang, B. Li, B. Zhou, J. Yue, H. Xia, J. Jiang, L. Zheng, X. Xu et al. , “Towards general computer control: A multi- modal agent for red dead redemption ii as a case study,” in ICLR 2024 Workshop on Large Language Model (LLM) Agents , 2024

    2024

  20. [28]

    Executable code actions elicit better llm agents,

    X. Wang, Y . Chen, L. Yuan, Y . Zhang, Y . Li, H. Peng, and H. Ji, “Executable code actions elicit better llm agents,” arXiv preprint arXiv:2402.01030, 2024

    2024 arXiv

  21. [29]

    Eureka: Human- level reward design via coding large language models,

    Y . J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Ja- yaraman, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human- level reward design via coding large language models,” arXiv preprint arXiv:2310.12931, 2023

    2023 arXiv

  22. [30]

    R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction . MIT press, 2018

    2018

  23. [31]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  24. [32]

    Trust region policy optimization,

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International Conference on Machine Learning (ICML). PMLR, 2015, pp. 1889–1897

    2015

  25. [33]

    High- dimensional continuous control using generalized advantage estimation,

    J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, “High- dimensional continuous control using generalized advantage estimation,” in International Conference on Learning Representations (ICLR) , 2016

    2016

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.