Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

lmgame-Bench: How Good are LLMs at Playing Games?

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read lmgame-Bench claims that off-the-shelf video games, wrapped in perception and memory scaffolds, can reliably rank LLMs, and that game-based reinforcement learning transfers to planning tasks.

desk verdict A useful game-harness benchmark and an interesting RL transfer study, but the o3-generated memory reflections leak policy advice into the harness, so the reliable-ranking claim needs an independent-scaffold check. read the letter →

arxiv 2505.15146 v2 pith:OI6QZSLI submitted 2025-05-21 cs.AI

Lanxiang Hu , Mingjia Huo , Yuxuan Zhang , Haoyang Yu , Eric P. Xing , Ion Stoica , Tajana Rosing , Haojian Jin
show 1 more author
Hao Zhang
This is my paper
classification cs.AI
keywords LLMevaluationvideogamebenchmarksreinforcementlearningtransferplanningdatacontaminationpromptoptimizationagenticAIbenchmarkscaffolding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

lmgame-Bench asks whether off-the-shelf video games can be turned into a dependable way to evaluate large language models, and answers yes, provided the games come with carefully built scaffolding. Directly dropping models into games leaves most runs near random play, so the benchmark adds perception modules that convert screens into text and memory modules that write reflections, plus contamination checks and prompt standardization. With those supports, 86.7% of model runs beat a random baseline and the 13-model leaderboard separates reasoning from non-reasoning models. The paper also reports that reinforcement learning on a single game transfers to unseen games and to planning and agentic tasks such as Blocksworld and WebShop. A reader would care because games stress perception, memory, and planning together, and the results suggest games can serve both as a testing ground and as a training ground for those abilities.

What carries the argument

The load-bearing mechanism is the gaming harness, a set of modular scaffolds wrapped around a Gym-style API. Perception modules read game backends and convert grids or UI into textual state descriptions, with o3 used for complex frames; memory modules store recent state-action-reward tuples and generate reflections that compress lessons; reasoning modules allow long chain-of-thought. Prompt standardization is carried out by a two-stage procedure that starts from empirical agent prompts and then uses a bootstrapping optimizer to refine prompts against game rewards across all target models. Contamination checks use frame-reordering for Super Mario Bros. and Sentence-BERT similarity against public transcripts for Ace Attorney. These components together lift scores above random, reduce variance, and make model rankings interpretable.

What would settle it

Run the harness on the same six games with the o3 scaffold replaced by an open-weight descriptor that never sees solutions, and compare leaderboards. If model rankings shift substantially, or if the o3 reflections for failed runs already contain the correct next move, the central claim that lmgame-Bench measures target-model gaming ability is falsified. A second check: after Ace Attorney mitigation, verify whether the o3 judge's quality scores still predict rankings when the judge is given only entity-masked testimony and no access to the original script.

Watch

Extended reading notes

Core claim

The paper claims that classic games—Super Mario Bros., Tetris, Sokoban, Candy Crush, 2048, and Ace Attorney—can be made into an effective LLM benchmark if the raw game is wrapped in a harness that fixes three failure modes: brittle vision, prompt sensitivity, and contamination. The harness converts game states into symbolic text or o3-generated descriptions, keeps a short memory of past states and reflections, and lets reasoning models emit long chain-of-thought traces. Contamination is measured directly: Ace Attorney performance initially tracks similarity to public transcripts, and the correlation disappears after entity masking, paraphrasing, and enforced reasoning. After these fixes, harnessed runs separate models far from random play, and rank correlation plus low-rank decomposition show each game draws on a distinct blend of math, code, language, physics, and puzzle abilities. The paper's strongest empirical claim is that fine-tuning a 7B model with reinforcement learning on simplified Sokoban or Tetris improves performance on larger boards, on the other game, on Blocksworld, and on WebShop, while leaving math and coding largely unchanged.

Load-bearing premise

The benchmark's discriminative power rests on the assumption that the o3-generated text scaffolds are neutral: if o3's perception descriptions or memory reflections leak solution information, rankings measure scaffold leakage rather than the target model's ability.

Editorial extensions

If this is right

  • With the harness, 86.7% of model runs beat random play, so the benchmark can separate models that raw games cannot.
  • Rankings on lmgame-Bench separate reasoning models from non-reasoning ones, giving a scalable signal of agentic competence.
  • RL on simplified Sokoban lifts Blocksworld 2D from 9.0 to 29.5 and WebShop from 7.0 to 19.1, suggesting game-based training transfers to planning and agentic tasks.
  • Ace Attorney scores no longer track transcript similarity after mitigation, so post-mitigation rankings plausibly reflect reasoning rather than memorization.
  • Prompt standardization cuts variance between candidate prompts by 33.8% to 63.5% on 2048, making comparisons across models more stable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If scaffold neutrality holds, the leaderboard measures planning over textual state representations, not raw visual perception; the benchmark is thus a proxy for agentic reasoning with a text interface rather than for human-like game vision.
  • The transfer results suggest games could serve as a cheap, verifiable-reward training source for planning skills; a natural extension is to test whether Sokoban-style training transfers to other long-horizon agent benchmarks beyond Blocksworld and WebShop.
  • Because the scaffolds are partly generated by o3, the benchmark is coupled to one model's perception; an open-weight scaffold or a purely backend-derived state representation would make the benchmark self-contained and reusable as models evolve.
  • The contamination findings imply other narrative or visual benchmarks built from public assets should run the same similarity-against-performance check before trusting leaderboards.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces lmgame-Bench, a benchmark that wraps six off-the-shelf video games in a Gym-style API with optional perception, memory, and reasoning scaffolds. Across 13 models, the authors report that the harness raises most runs above random play and separates reasoning models from non-reasoning models. They also analyze correlations between game performance and 20 established benchmarks via Spearman correlation, low-rank factorization, and linear modeling, and report that RL fine-tuning on simplified Sokoban/Tetris transfers to planning and agentic tasks. Contamination tests for Super Mario Bros. and Ace Attorney, a DSPy-based prompt standardization procedure, and detailed appendices support the main text.

Significance. If the harness were a neutral mediator, lmgame-Bench would be a valuable contribution: video games are a scalable source of long-horizon, partially observable interactive tasks, and the RL transfer results to Blocksworld and WebShop are interesting. The paper's strengths include a released codebase, a clean MDP formalism, toggleable harness modules, explicit contamination checks, and a quantitative attempt to connect game performance to capabilities measured by established benchmarks. However, the core reliability claim is compromised by the finding that the memory reflection module is generated by o3 and contains explicit action-level policy advice. Because o3 is itself the top-ranked model, harnessed scores may partly reflect how well target models follow o3's coaching rather than their own game-playing ability. The single-run results reported with zero variance and the selective exclusion of Super Mario Bros. from capability analyses further weaken the quantitative claims. The benchmark artifact is plausible, but the measurement validity needs to be established with additional ablations and reporting changes.

major comments (4)
  1. [§2.2.1, Appendix F] The memory module is not a neutral scaffold. Section 3.1 states that "we employ o3 to generate reflections" and the Appendix F prompt asks for "strategic insights" and "actionable" advice; the example reflections contain explicit policy directives such as "Next: keep sliding Right/Down to compress toward the bottom-right; avoid Up unless desperate" and "Prefer more up/left sequences to clear column 2." For every evaluated model that uses the harness's memory module, these o3-generated reflections enter the context before the model chooses an action. The paper provides no ablation showing that rankings are stable when this o3 policy content is removed or replaced by neutral state summaries. Since o3 is itself the top-ranked model, the harnessed leaderboard scores may measure how well weaker models follow o3's coaching rather than their own game-playing ability. The same concern applies to the o3-based perception module used for Super Mario Bros. and Ace Attorney. This is a load-bearing confound for the central claim that lmgame-Bench gives reliable model evaluations.
  2. [Table 1] Several reported zero standard deviations come from single runs. Table 1 marks o1-2024-12-17 and o3-2025-04-16 as single-run models, yet reports values such as 3445.0±0.0, and the Ace Attorney column is marked with a star throughout, implying a single run for every model in that game while still printing ±0.0. A single observation has no sample variance; reporting ±0.0 is misleading and makes the "harness reduces variance" and "models separate well" claims appear stronger than the data support. Since Ace Attorney and the o3 row are used directly in the leaderboard discussion (o3 and o1 both score 16), the affected results need to be either re-run or reported without error bars and with appropriate caveats.
  3. [§3.2, Table 3] The capability attribution claims are not supported for two of the six games. Super Mario Bros. is excluded from the main correlation and factorization analyses after observing its high variance, and the supplementary inclusion (Appendix D.2) shows weak, diffuse loadings. Table 3 shows the linear model cannot predict SMB (r=0.295) or 2048 (r=0.248), yet the text states that "long-horizon games like Sokoban, Tetris and 2048's rankings are driven primarily by math and coding performance." This is contradicted by the 2048 row, whose offset weight (3.130) dominates and whose coding weight is 0. The claim that "every game probes a unique blend of capabilities" therefore holds only for the subset of games on which the model fits; the paper should either restrict the claim or present a model that fits 2048.
  4. [§2.2.2, Appendix B.2] It is unclear whether the main benchmark scores for Ace Attorney are collected with the contamination-mitigation prompts. Appendix B.2 detects a strong pre-mitigation correlation between script similarity and performance, then shows this correlation disappears after name-masking, paraphrasing, and enforced reasoning. But Table 1 presents Ace Attorney scores without stating whether these are raw or mitigated. If the leaderboard uses unmitigated prompts, the contamination claim in the abstract ("designed to ... remove contamination") is not satisfied for the reported rankings. If it uses mitigated prompts, the paper should say so explicitly and state which prompts were used for Table 1. The o3-as-judge validation in Figure 10b does not resolve this, because the judge is generated by the same model family that supplies the scaffold.
minor comments (5)
  1. [Appendix B.2, Figure 9] The sign of the reported correlations is inconsistent: the text says similarity strongly correlates with leaderboard rank (r=-0.773, p=0.005), while Figure 9a reports r=0.773 for similarity vs. score and r=-0.911 for similarity vs. rank; align the signs and captions with the actual variables.
  2. [§1] The claim of being "the first benchmark that uses video games to evaluate state-of-the-art LLMs with or without scaffolds" is too strong given the BALROG, GameBench, and SmartPlay systems discussed in Related Work; soften to a comparative claim.
  3. [Appendix D.3, Figure 13] The text says the first version (Figure 13b) excludes Super Mario Bros. and the second version (Figure 13a) includes it, but the figure captions state the opposite; the labels and text should be reconciled.
  4. [Table 9 caption] There are typos in the caption: "empirically deisgn prompt pairs" and "DSPy optimzied prompt pairs" should be corrected, and the variables Δe and Δp should be defined in the caption.
  5. [§2.1.3] The word "narative" should be "narrative," and the sentence on long-horizon rewards could clarify why Sokoban is classified as long-horizon rather than progression-based.

Circularity Check

3 steps flagged · score 6.0 of 10

o3-generated scaffolds leak next-move hints and the target contradiction, while o3-as-judge validates the same pipeline; harnessed leaderboard scores are partly o3's own output.

  1. other [Section 2.2.1 / Figure 2 (Ace Attorney perception module)]
    "o3 Generated Text: Frank Sahwit, Round 1 - Statement: "It was exactly 1:00 PM when I saw Larry Butz run out of the apartment." Present: Cindy’s Autopsy Report -Contradiction Exposed: Autopsy shows death occurred between 4:00 and 5:00 PM, making a 1:00 PM murder impossible."

    Section 3.1 states that for Ace Attorney, "lmgame-Bench uses o3 in perception module to extract key iterative visual elements and status indicators as textual descriptions." The o3 output shown in Figure 2 does not neutrally describe the scene; it restates the exact contradiction that the game task asks the model to discover, matching the ground-truth "Contradiction Exposed" line. Every evaluated model receives this o3-generated text as its observation stream, so Ace Attorney scores measure o3's detective work plus the target model's ability to act on the leaked hint, not the target model's own deductive reasoning. The harnessed Ace Attorney leaderboard is therefore partially o3's answer by construction.

  2. other [Appendix F, o3 Game 2048 Memory Module Reflections, Sample 1]
    "Next: keep sliding Right/Down to compress toward the bottom-right; avoid Up unless desperate to prevent dislodging the 256."

    Section 2.2.1 defines the reflection module as encoding "explicit lessons learned to avoid failure, inducing actions in specific game states," and Appendix F's memory prompt asks for "strategic insights and patterns that would help the agent make better decisions. Keep your reflections short, precise, and actionable." The sample o3 reflection is not a neutral state summary; it is an explicit next-move policy. Because o3 generates the reflections that are fed to all target models, harnessed 2048 (and other memory-module games) scores reflect how well each model follows o3's coaching rather than purely the target model's own game-playing ability. Since o3 is itself the top-scoring model, the harnessed separation between models is partly o3's policy executed by proxy.

1 more flagged steps
  1. other [Appendix B.2, Conclusion (Ace Attorney contamination mitigation)]
    "Post-intervention rankings instead align with reasoning quality, as verified by an independent evaluator (o3)."

    The o3 evaluator is not independent of the pipeline being validated: o3 is the same model that generates the perception and reflection scaffolds (Section 3.1), and it is also the top-ranked model on the lmgame-Bench leaderboard. The conclusion that post-mitigation Ace Attorney rankings reflect "reasoning quality" rather than memorization is therefore confirmed by the very model whose outputs are embedded in the evaluated harness. This is a self-referential validation: o3's judgments confirm rankings of a benchmark in which o3's own solution-relevant text is part of the input stream. The disappearance of the similarity correlation is empirical, but the stronger claim that rankings measure target-model reasoning rests on a non-independent judge.

full rationale

The main circularity is operational rather than equation-based: lmgame-Bench's harness is not a neutral mediator. For Ace Attorney, o3's perception output restates the target contradiction verbatim, so the harnessed score is partly o3's solution. For 2048 and other memory-module games, o3-generated reflections contain explicit next-move suggestions, so the harness injects o3's policy into every model's context. The o3-as-judge step then validates the post-mitigation rankings using the same model that powers the scaffold, making the "independent evaluator" claim self-referential. These issues directly affect the central claim that the harnessed leaderboard reliably separates model abilities, so a moderate-to-high circularity score is warranted. However, the paper is not wholly circular: the unharnessed results in Table 1, the RL transfer experiments in Section 3.3 (which train Qwen on simplified Sokoban/Tetris without o3 scaffolds), and the correlation/decomposition analyses provide independent content and are not forced by construction. There is no load-bearing self-citation chain or imported uniqueness theorem. The score is therefore 6 rather than higher.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claims rest on statistical modeling choices, scaffold neutrality, and contamination-metric assumptions. No physical or invented entities are introduced. The benchmark and transfer results are empirical, so the ledger mainly records the fitted regression and factorization weights and the assumption that the o3-based harness and judge do not leak information.

free parameters (2)
  • Linear model category weights = Table 3, e.g., Sokoban math 2.160, coding 2.206; Ace Attorney language 3.392
    Fitted by non-negative least squares to 8 model rankings to predict game rankings. With 5 or more predictors and 8 observations, the weights are likely overfit and are used as evidence for capability attribution.
  • Low-rank factorization latent loadings = Figure 3b top-weight benchmarks per feature
    Non-unique factorization of an 8-model by 20-benchmark matrix; used to claim each game loads on distinct latent abilities, but components are not identifiable and depend on factorization algorithm choices.
assumptions (6)
  • domain assumption Video game performance is a valid proxy for the perception, memory, and planning abilities expected of LLM agents.
    The introduction frames games as a natural testbed and the whole benchmark depends on this mapping being meaningful for LLM agents, not just for RL agents.
  • domain assumption Three-run averages, 30-run random baselines, and single-run cells are sufficient to estimate model ability and game randomness.
    Section 3.1 and Appendix E rely on roughly 3 runs per model and 30 random runs; several cells are single-run, so the statistical basis is thin.
  • domain assumption Sentence-BERT cosine similarity between generated Ace Attorney text and public scripts measures memorization contamination.
    Appendix B.2 uses this similarity as the contamination metric; the metric may also capture stylistic or template reuse rather than true memorization.
  • ad hoc to paper Prompt-based mitigation, including entity masking, paraphrasing, and enforced reasoning, removes memorization without changing task difficulty or what is being measured.
    Appendix B.2 concludes contamination disappears after these interventions, but this assumes the interventions do not simply make the task harder or shift the measured skill.
  • ad hoc to paper The o3-based perception, memory, and judge modules are neutral and do not leak solution-relevant information to evaluated models.
    Section 2.2.1 and Appendix F use o3 to generate perception text and reflections for all models; Appendix B.2 uses o3 as the reasoning-quality judge. If o3 outputs encode answers, rankings are confounded.
  • standard math The closed-form non-negative least squares solution and low-rank factorization are valid for the 8-model by 20-benchmark matrix.
    Appendix D.4 uses the closed form; low-rank factors are non-unique and the paper does not establish identifiability, so interpretability is limited.

how reviews work

0 comments
Cite this review

Pith. "Pith review of lmgame-Bench: How Good are LLMs at Playing Games?." pith.science (2026). https://pith.science/paper/OI6QZSLI

@misc{pith2026250515146,
  author       = {Pith},
  title        = {Pith review of: lmgame-Bench: How Good are LLMs at Playing Games?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OI6QZSLI}},
  note         = {Machine review of arXiv:2505.15146}
}
read the original abstract

Playing video games requires perception, memory, and planning, exactly the faculties modern large language model (LLM) agents are expected to master. We study the major challenges in using popular video games to evaluate modern LLMs and find that directly dropping LLMs into games cannot make an effective evaluation, for three reasons -- brittle vision perception, prompt sensitivity, and potential data contamination. We introduce lmgame-Bench to turn games into reliable evaluations. lmgame-Bench features a suite of platformer, puzzle, and narrative games delivered through a unified Gym-style API and paired with lightweight perception and memory scaffolds, and is designed to stabilize prompt variance and remove contamination. Across 13 leading models, we show lmgame-Bench is challenging while still separating models well. Correlation analysis shows that every game probes a unique blend of capabilities often tested in isolation elsewhere. More interestingly, performing reinforcement learning on a single game from lmgame-Bench transfers both to unseen games and to external planning tasks. Our evaluation code is available at https://github.com/lmgame-org/GamingAgent/lmgame-bench.

Figures

Figures reproduced from arXiv: 2505.15146 by the authors.

Figure 1
Figure 1. lmgame-Bench uses modular harnesses—such as perception, memory, and reasoning modules—to systematically extend a model’s game-playing capabilities, allowing the model to engage with a simulated game environment through iterative interaction loops. game settings to mitigate data contamination once it is detected. Finally, lmgame-Bench also employs a standardized prompt optimization technique to reduce prompt sensitiv… view at source ↗
Figure 2
Figure 2. (Left) Example from Ace Attorney showing contradictions in O3-generated text vs. ground [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Correlation Analysis and Latent Feature Decomposition Among Benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Success rates during training across different evaluation tasks, with models trained on [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Position-wise reconstruction accuracy for the shuffled Super Mario Bros level 1-1 frames. Lighter cells denote higher accuracy; only the first and last positions remain relatively high accuracy 23 [PITH_FULL_IMAGE:figures/full_fig_p023_5.png]
Figure 6
Figure 6. Figure 6: Kendall’s τ and RBO scores for each model on the frame ordering task. Higher values indicate stronger alignment between predicted and ground-truth frame sequences. Only Gemini-2.5- pro-preview, o3, and o4-mini achieve moderate agreement [PITH_FULL_IMAGE:figures/full_f…
Figure 7
Figure 7. Figure 7: Pearson and spearman correlation between model performance ranks and their alignment [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: Global Embedding Correlation (Generated + Ground Truth) between model-generated texts [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 9
Figure 9. Figure 9: Linear correlations between script similarity and model performance across the full model [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]
Figure 10
Figure 10. Figure 10: Post-mitigation analysis. Left: model similarity no longer predicts rank or score. Right: [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 11
Figure 11. Figure 11: A representative side-by-side sample comparison between o3 and ground truth. [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]
Figure 12
Figure 12. Figure 12: Benchmark relationships overview. vector based on either model performance scores (ranging from 0–100) or model rankings, and project them into 2D using t-SNE. Benchmarks with NaN values (e.g., missing model scores) were excluded to ensure reliable embeddings. We show…
Figure 13
Figure 13. Figure 13: Benchmark relationships overview [PITH_FULL_IMAGE:figures/full_fig_p035_13.png]
Figure 14
Figure 14. Figure 14: Distribution of paired score improvements (Harness – No Harness) across ten models for [PITH_FULL_IMAGE:figures/full_fig_p038_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CAST: Game Solvers as Turn-Level Teachers for LLM Agents

    cs.CL 2026-07 conditional novelty 6.0 of 10

    CAST converts a game solver's per-action cost-to-go changes into turn-level RL credits for LLM agents and reports gains over outcome-only RLVR on three games plus zero-shot transfer to ALFWorld and WebShop.

  2. Game Reasoning Arena: A Framework and Benchmark for Assessing Reasoning Capabilities of Large Language Models via Game Play

    cs.AI 2025-08 conditional novelty 4.0 of 10

    Game Reasoning Arena is a modular OpenSpiel-based framework for benchmarking LLM decision making in games, with exploratory analyses suggesting models adapt their verbalized reasoning to game structure and model size.

Reference graph

Works this paper leans on

115 extracted references · 24 canonical work pages · cited by 2 Pith papers

  1. [1]

    arXiv preprint arXiv:1606.01540 (2016)

    Brockman, G., Cheung, V ., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym. arXiv preprint arXiv:1606.01540 (2016)

    arXiv 2016

  2. [2]

    arXiv preprint arXiv:2407.17032 (2024)

    Towers, M., Kwiatkowski, A., Terry, J., Balis, J.U., De Cola, G., Deleu, T., Goulão, M., Kallinteris, A., Krimmel, M., KG, A., et al.: Gymnasium: A standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032 (2024)

    arXiv 2024

  3. [3]

    arXiv preprint arXiv:2504.20073 (2025)

    Wang, Z., Wang, K., Wang, Q., Zhang, P., Li, L., Yang, Z., Yu, K., Nguyen, M.N., Liu, L., Gottlieb, E., et al.: Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning. arXiv preprint arXiv:2504.20073 (2025)

    arXiv 2025

  4. [4]

    arXiv preprint arXiv:2503.15478 (2025)

    Zhou, Y ., Jiang, S., Tian, Y ., Weston, J., Levine, S., Sukhbaatar, S., Li, X.: Sweet-rl: Training multi-turn llm agents on collaborative reasoning tasks. arXiv preprint arXiv:2503.15478 (2025)

    arXiv 2025

  5. [5]

    In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C., eds.: Advances in Neural Information Processing Systems

    Zhai, Y ., Bai, H., Lin, Z., Pan, J., Tong, S., Zhou, Y ., Suhr, A., Xie, S., LeCun, Y ., Ma, Y ., Levine, S.: Fine-tuning large vision-language models as decision-making agents via reinforcement learning. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J., Zhang, C., eds.: Advances in Neural Information Processing Systems. V olu...

    2024

  6. [6]

    Shi, J., Yang, J., Liu, J., Bu, X., Chen, J., Zhou, J., Ma, K., Wen, Z., Wang, B., He, Y ., Song, L., Zhu, H., Li, S., Wang, X., Zhang, W., Yuan, R., Yao, Y ., Yang, W., Wang, Y ., Fang, S., Yuan, S., He, Q., Tang, X., Tan, Y ., Zhou, W., Zhang, Z., Li, Z., Huang, W., Zhang, G.: Korgym: A dynamic game platform for llm reasoning evaluation (2025)

    2025

  7. [7]

    Ruoss, A., Pardo, F., Chan, H., Li, B., Mnih, V ., Genewein, T.: Lmact: A benchmark for in-context imitation learning with long multimodal demonstrations (2025)

    2025

  8. [8]

    arXiv preprint arXiv:2502.08859 (2025)

    Wang, C.J., Lee, D., Menghini, C., Mols, J., Doughty, J., Khoja, A., Lynch, J., Hendryx, S., Yue, S., Hendrycks, D.: Enigmaeval: A benchmark of long multimodal reasoning challenges. arXiv preprint arXiv:2502.08859 (2025)

    arXiv 2025

Show all 115 references
  1. [9]

    Anthropic: Claude 3.7 sonnet: Frontier reasoning made practical (February 2025) Accessed: 2025-05-02

    2025

  2. [10]

    arXiv preprint arXiv:2411.13543 (2024)

    Paglieri, D., Cupiał, B., Coward, S., Piterbarg, U., Wolczyk, M., Khan, A., Pignatelli, E., Kuci´nski, Ł., Pinto, L., Fergus, R., et al.: Balrog: Benchmarking agentic llm and vlm reasoning on games. arXiv preprint arXiv:2411.13543 (2024)

    2024 arXiv

  3. [11]

    arXiv preprint arXiv:2406.06613 (2024)

    Costarelli, A., Allen, M., Hauksson, R., Sodunke, G., Hariharan, S., Cheng, C., Li, W., Clymer, J., Yadav, A.: Gamebench: Evaluating strategic reasoning abilities of llm agents. arXiv preprint arXiv:2406.06613 (2024)

    2024 arXiv

  4. [12]

    arXiv preprint arXiv:2310.01557 (2023)

    Wu, Y ., Tang, X., Mitchell, T.M., Li, Y .: Smartplay: A benchmark for llms as intelligent agents. arXiv preprint arXiv:2310.01557 (2023)

    2023 arXiv

  5. [13]

    IEEE Transactions on Games11(3) (2019) 195–202

    Perez-Liebana, D., Liu, J., Khalifa, A., Gaina, R.D., Togelius, J., Lucas, S.M.: General video game ai: A multi-track framework for evaluating agents, games and content generation algorithms. IEEE Transactions on Games11(3) (2019) 195–202

    2019

  6. [14]

    AI Magazine22(2) (2001) 15–25

    Laird, J.E., van Lent, M.: Human-level ai’s killer application: Interactive computer games. AI Magazine22(2) (2001) 15–25

    2001

  7. [15]

    arXiv preprint arXiv:2412.14171 (2024)

    Yang, J., Yang, S., Gupta, A.W., Han, R., Fei-Fei, L., Xie, S.: Thinking in space: How multi- modal large language models see, remember, and recall spaces. arXiv preprint arXiv:2412.14171 (2024)

    2024 arXiv

  8. [16]

    arXiv preprint arXiv:2408.15950 (2024) 11

    Waytowich, N.R., White, D., Sunbeam, M., Goecks, V .G.: Atari-gpt: Investigating the capabilities of multimodal large language models as low-level policies for atari games. arXiv preprint arXiv:2408.15950 (2024) 11

    2024 arXiv

  9. [17]

    arXiv preprint arXiv:2403.11381 (2024)

    Mosquera, M., Pinzon, J.S., Rios, M., Fonseca, Y ., Giraldo, L.F., Quijano, N., Manrique, R.: Can llm-augmented autonomous agents cooperate?, an evaluation of their cooperative capabilities through melting pot. arXiv preprint arXiv:2403.11381 (2024)

    2024 arXiv

  10. [18]

    OpenAI: Openai o3 and o4-mini system card (April 2025) Accessed: 2025-05-10

    2025

  11. [19]

    OpenAI: Openai o1 system card (December 2024) arXiv preprint arXiv:2412.16720

    2024 arXiv

  12. [20]

    DeepMind, G.: Gemini 2.5: Our most intelligent ai model (March 2025) Accessed: 2025-05-10

    2025

  13. [21]

    In ICAPS

    Rintanen, J.: Complexity of planning with partial observability. In ICAPS. V olume 4. (2004) 345–354

    2004

  14. [22]

    Applied cognitive psychology31(4) (2017) 438–445

    Lau-Zhu, A., Holmes, E.A., Butterfield, S., Holmes, J.: Selective association between tetris game play and visuospatial working memory: A preliminary investigation. Applied cognitive psychology31(4) (2017) 438–445

    2017

  15. [23]

    In International Computing and Combinatorics Conference (COCOON)

    Demaine, E.D., Hohenberger, S., Liben-Nowell, D.: Tetris is hard, even to approximate. In International Computing and Combinatorics Conference (COCOON). V olume 2697 of Lecture Notes in Computer Science., Springer (2003) 351–363

    2003

  16. [24]

    Culberson, J.: Sokoban is pspace-complete. (1997)

    1997

  17. [25]

    Zaky, A.: Minimax and expectimax algorithm to solve 2048. (2014)

    2014

  18. [26]

    arXiv (2024)

    Group, A.R.: Ufo: A unified framework for gui interaction in windows applications. arXiv (2024)

    2024

  19. [27]

    arXiv (2025)

    Group, A.R.: Infiguiagent: A multimodal agent for gui interaction and reasoning. arXiv (2025)

    2025

  20. [28]

    arXiv preprint arXiv:2501.12948 (2025)

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al.: Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 (2025)

    2025 arXiv

  21. [29]

    Computational Geometry 13(4) (1999) 215–228

    Dor, D., Zwick, U.: SOKOBAN and other motion planning problems. Computational Geometry 13(4) (1999) 215–228

    1999

  22. [30]

    arXiv preprint arXiv:1403.5484 (2014)https://arxiv.org/abs/1403.5484

    Gualà, S., Leucci, S., Natale, E.: Bejeweled, candy crush and other match -three games are (np-)hard. arXiv preprint arXiv:1403.5484 (2014)https://arxiv.org/abs/1403.5484

    2014 arXiv

  23. [31]

    arXiv preprint arXiv:2402.15938 (2024)

    Dong, Y ., Jiang, X., Liu, H., Jin, Z., Gu, B., Yang, M., Li, G.: Generalization or memorization: Data contamination and trustworthy evaluation for large language models. arXiv preprint arXiv:2402.15938 (2024)

    2024 arXiv

  24. [32]

    arXiv preprint arXiv:1802.08232 (2018)

    Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., Song, D.: The secret sharer: Evaluating and testing unintended memorization in neural networks. arXiv preprint arXiv:1802.08232 (2018)

    2018 arXiv

  25. [33]

    Wang, X., Zhuang, B., Wu, Q.: Are large vision language models good game players? In The Thirteenth International Conference on Learning Representations

  26. [34]

    Agashe, S., Han, J., Gan, S., Yang, J., Li, A., Wang, X.E.: Agent s: An open agentic framework that uses computers like a human (2024)

    2024

  27. [35]

    arXiv preprint arXiv:2403.03186 (2024)

    Tan, W., Zhang, W., Xu, X., Xia, H., Ding, Z., Li, B., Zhou, B., Yue, J., Jiang, J., Li, Y ., An, R., Qin, M., Zong, C., Zheng, L., Wu, Y ., Chai, X., Bi, Y ., Xie, T., Gu, P., Li, X., Zhang, C., Tian, L., Wang, C., Wang, X., Karlsson, B.F., An, B., Yan, S., Lu, Z.: Cradle: Em...

    2024 arXiv

  28. [36]

    In The Twelfth International Conference on Learning Representations

    Khattab, O., Singhvi, A., Maheshwari, P., Zhang, Z., Santhanam, K., Haq, S., Sharma, A., Joshi, T.T., Moazam, H., Miller, H., et al.: Dspy: Compiling declarative language model calls into state-of-the-art pipelines. In The Twelfth International Conference on Learning Represent...

    2024

  29. [37]

    V ALS AI: Mmlu-pro benchmark leaderboard.https://www.vals.ai/benchmarks/mmlu_ pro-05-09-2025(2025) Accessed: 2025-05-13. 12

    2025

  30. [38]

    arXiv preprint arXiv:2009.03300 (2021)

    Hendrycks, D., Burns, C., Kadavath, S., Arora, P., Basart, S., Tang, D.S., et al.: Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2021)

    2021 arXiv

  31. [39]

    arXiv preprint arXiv:2501.14249 (2024)

    Phan, L., Gatti, A., Han, Z., Li, N., Hu, J., Zhang, H., Zhang, C.B.C., Shaaban, M., Ling, J., Shi, S., Choi, M., Agrawal, A., Chopra, A., Khoja, A., Kim, R., Ren, R., Hausenloy, J., Zhang, O., Mazeika, M., Yue, S., Wang, A., Hendrycks, D.: Humanity’s last exam: Benchmarking a...

    2024 arXiv

  32. [40]

    https://scale.com/leaderboard/ humanitys_last_examAccessed: 2025-05-14

    Scale AI: Humanity’s last exam leaderboard. https://scale.com/leaderboard/ humanitys_last_examAccessed: 2025-05-14

    2025

  33. [41]

    Scale AI: Humanity’s last exam leaderboard (text only).https://scale.com/leaderboard/ humanitys_last_exam_text_onlyAccessed: 2025-05-14

    2025

  34. [42]

    arXiv preprint arXiv:2311.12022 (2023)

    Rein, D., Hou, B.L., Stickland, A.C., Petty, J., Pang, R.Y ., Dirani, J., Michael, J., Bowman, S.R.: Gpqa: Graded physics question answering benchmark for large language models. arXiv preprint arXiv:2311.12022 (2023)

    2023 arXiv

  35. [43]

    https://www.vals.ai/benchmarks/ gpqa-05-09-2025Accessed: 2025-05-14

    V ALS AI: Gpqa benchmark leaderboard. https://www.vals.ai/benchmarks/ gpqa-05-09-2025Accessed: 2025-05-14

    2025

  36. [44]

    arXiv preprint arXiv:2504.16074 (2025)

    Qiu, S., Guo, S., Zhuo, Y ., Wang, Y ., Li, Z., Zhang, Y ., Wang, Y ., Li, Z., Zhang, Y ., Wang, Y ., Li, Z., Zhang, Y .: Phybench: Holistic evaluation of physical perception and reasoning in large language models. arXiv preprint arXiv:2504.16074 (2025)

    2025 arXiv

  37. [45]

    arXiv preprint arXiv:2501.05444 (2025)

    Hao, Y ., Gu, J., Wang, H.W., Li, L., Yang, Z., Wang, L., Cheng, Y .: Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501.05444 (2025)

    2025 arXiv

  38. [46]

    https://www.vals.ai/benchmarks/ math500-05-09-2025Accessed: 2025-05-14

    Vals AI: Math 500 benchmark leaderboard. https://www.vals.ai/benchmarks/ math500-05-09-2025Accessed: 2025-05-14

    2025

  39. [47]

    arXiv preprint arXiv:2410.03131 (2024)

    Patel, B., Chakraborty, S., Suttle, W.A., Wang, M., Bedi, A.S., Manocha, D.: Aime: Ai system optimization via multiple llm evaluators. arXiv preprint arXiv:2410.03131 (2024)

    2024

  40. [48]

    https://www.vals.ai/benchmarks/ aime-2025-05-09Accessed: 2025-05-14

    Vals AI: Aime benchmark leaderboard. https://www.vals.ai/benchmarks/ aime-2025-05-09Accessed: 2025-05-14

    2025

  41. [49]

    arXiv preprint arXiv:2406.19314 (2024)

    White, C., Dooley, S., Roberts, M., Pal, A., Feuer, B., Jain, S., Shwartz-Ziv, R., Jain, N., Saifullah, K., Naidu, S., Hegde, C., LeCun, Y ., Goldstein, T., Neiswanger, W., Goldblum, M.: Livebench: A challenging, contamination-free llm benchmark. arXiv preprint arXiv:2406.19314 (2024)

    2024 arXiv

  42. [50]

    https://livebench.ai/#/?Coding=a& Mathematics=a&Data+Analysis=a&Language=a&IF=aAccessed: 2025-05-14

    LiveBench Team: Livebench leaderboard. https://livebench.ai/#/?Coding=a& Mathematics=a&Data+Analysis=a&Language=a&IF=aAccessed: 2025-05-14

    2025

  43. [51]

    arXiv preprint arXiv:2406.15877 (2024)

    Zhuo, T.Y ., Vu, M.C., Chim, J., Hu, H., Yu, W., Widyasari, R., Yusuf, I.N.B., Zhan, H., He, J., Paul, I., Brunner, S., Gong, C., Hoang, T., Zebaze, A.R., Hong, X., Li, W.D., Kaddour, J., Xu, M., Zhang, Z., Yadav, P., Jain, N., Gu, A., Cheng, Z., Liu, J., Liu, Q., Wang, Z., Lo...

    2024 arXiv

  44. [52]

    https://aider.chat/docs/leaderboards/ Ac- cessed: 2025-05-14

    Aider Team: Aider llm leaderboards. https://aider.chat/docs/leaderboards/ Ac- cessed: 2025-05-14

    2025

  45. [53]

    https://bigcode-bench.github.io/ Accessed: 2025-05-14

    BigCodeBench Team: Bigcodebench leaderboard. https://bigcode-bench.github.io/ Accessed: 2025-05-14

    2025

  46. [54]

    https://scale

    Scale AI: Vista: Visual language understanding benchmark leaderboard. https://scale. com/leaderboard/visual_language_understandingAccessed: 2025-05-14

    2025

  47. [55]

    https://lmarena.ai/leaderboard Accessed: 2025-05-14

    LMSYS Org: Chatbot arena leaderboard. https://lmarena.ai/leaderboard Accessed: 2025-05-14. 13

    2025

  48. [56]

    arXiv preprint arXiv:2311.16502 (2023)

    Yue, X., Ni, Y ., Zhang, K., Zheng, T., Liu, R., Zhang, G., Stevens, S., Jiang, D., Ren, W., Sun, Y ., et al.: Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. arXiv preprint arXiv:2311.16502 (2023)

    2023 arXiv

  49. [57]

    https://www.vals.ai/benchmarks/ mmmu-05-09-2025Accessed: 2025-05-14

    Vals AI: Mmmu benchmark leaderboard. https://www.vals.ai/benchmarks/ mmmu-05-09-2025Accessed: 2025-05-14

    2025

  50. [58]

    arXiv preprint arXiv:2501.17399 (2025)

    Sirdeshmukh, V ., Deshpande, K., Mols, J., Jin, L., Cardona, E.Y ., Lee, D., Kritz, J., Primack, W., Yue, S., Xing, C.: Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms. arXiv preprint arXiv:2501.17399 (2025)

    2025 arXiv

  51. [59]

    https://scale.com/leaderboard/ multichallengeAccessed: 2025-05-14

    Scale AI: Multichallenge leaderboard. https://scale.com/leaderboard/ multichallengeAccessed: 2025-05-14

    2025

  52. [60]

    https://scale.com/leaderboard/ enigma_evalAccessed: 2025-05-14

    Scale AI: Enigmaeval benchmark leaderboard. https://scale.com/leaderboard/ enigma_evalAccessed: 2025-05-14

    2025

  53. [61]

    Mazur, L.: Nyt connections benchmark: Evaluating llms with extended word association puzzles.https://github.com/lechmazur/nyt-connectionsAccessed: 2025-05-14

    2025

  54. [62]

    https://github.com/mpSchrader/gym-sokoban (2018)

    Schrader, M.P.B.: gym-sokoban. https://github.com/mpSchrader/gym-sokoban (2018)

    2018

  55. [63]

    https://github.com/jaybutera/ tetrisRL(2023) GitHub repository

    TetrisRL: Tetrisrl: Reinforcement learning for tetris. https://github.com/jaybutera/ tetrisRL(2023) GitHub repository

    2023

  56. [64]

    5 technical report

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al.: Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  57. [65]

    Communications of the ACM 38(3) (1995) 58–68

    Tesauro, G., et al.: Temporal difference learning and td-gammon. Communications of the ACM 38(3) (1995) 58–68

    1995

  58. [66]

    nature550(7676) (2017) 354–359

    Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al.: Mastering the game of go without human knowledge. nature550(7676) (2017) 354–359

    2017

  59. [67]

    In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    Nasir, M.U., James, S., Togelius, J.: Gametraversalbenchmark: Evaluating planning abilities of large language models through traversing 2d game maps. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. (2024)

    2024

  60. [68]

    arXiv preprint arXiv:2503.09617 (2025)

    Hopkins, J., Bakler, M., Khan, A.: Factorio learning environment. arXiv preprint arXiv:2503.09617 (2025)

    2025 arXiv

  61. [69]

    In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    Xie, J., Zhang, R., Chen, Z., Wan, X., Li, G.: Whodunitbench: Evaluating large multimodal agents via murder mystery games. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. (2024)

    2024

  62. [70]

    arXiv preprint arXiv:2502.18431 (2025)

    Hudi, F., Winata, G.I., Zhang, R., Aji, A.F.: Textgames: Learning to self-play text-based puzzle games via language model reasoning. arXiv preprint arXiv:2502.18431 (2025)

    2025 arXiv

  63. [71]

    arXiv preprint arXiv:2308.10032 (2023)

    Qiao, D., Wu, C., Liang, Y ., Li, J., Duan, N.: Gameeval: Evaluating llms on conversational games. arXiv preprint arXiv:2308.10032 (2023)

    2023 arXiv

  64. [72]

    arXiv preprint arXiv:2412.06394 (2024)

    Hu, L., Li, Q., Xie, A., Jiang, N., Stoica, I., Jin, H., Zhang, H.: Gamearena: Evaluating llm reasoning through live computer games. arXiv preprint arXiv:2412.06394 (2024)

    2024 arXiv

  65. [73]

    Jimenez, C.E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., Narasimhan, K.: Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770 (2023)

    2023 arXiv

  66. [74]

    arXiv preprint arXiv:2307.13854 (2023)

    Zhou, S., Xu, F.F., Zhu, H., Zhou, X., Lo, R., Sridhar, A., Cheng, X., Ou, T., Bisk, Y ., Fried, D., et al.: Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854 (2023)

    2023 arXiv

  67. [75]

    arXiv preprint arXiv:2401.13919 (2024) 14

    He, H., Yao, W., Ma, K., Yu, W., Dai, Y ., Zhang, H., Lan, Z., Yu, D.: Webvoyager: Building an end-to-end web agent with large multimodal models. arXiv preprint arXiv:2401.13919 (2024) 14

    2024 arXiv

  68. [76]

    arXiv preprint arXiv:2407.18901 (2024)

    Trivedi, H., Khot, T., Hartmann, M., Manku, R., Dong, V ., Li, E., Gupta, S., Sabharwal, A., Balasubramanian, N.: Appworld: A controllable world of apps and people for benchmarking interactive coding agents. arXiv preprint arXiv:2407.18901 (2024)

    2024 arXiv

  69. [77]

    Advances in Neural Information Processing Systems37(2024) 52040–52094

    Xie, T., Zhang, D., Chen, J., Li, X., Zhao, S., Cao, R., Hua, T.J., Cheng, Z., Shin, D., Lei, F., et al.: Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems37(2024) 52040–52094

    2024

  70. [78]

    Liu, X., Yu, H., Zhang, H., Xu, Y ., Lei, X., Lai, H., Gu, Y ., Ding, H., Men, K., Yang, K., et al.: Agentbench: Evaluating llms as agents. In ICLR. (2024)

    2024

  71. [79]

    In The Twelfth International Conference on Learning Representations

    Mialon, G., Fourrier, C., Wolf, T., LeCun, Y ., Scialom, T.: Gaia: a benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations. (2023)

    2023

  72. [80]

    Advances in neural information processing systems37(2024) 110935–110971

    Zhai, S., Bai, H., Lin, Z., Pan, J., Tong, P., Zhou, Y ., Suhr, A., Xie, S., LeCun, Y ., Ma, Y ., et al.: Fine-tuning large vision-language models as decision-making agents via reinforcement learning. Advances in neural information processing systems37(2024) 110935–110971

    2024

  73. [81]

    arXiv preprint arXiv:1707.06347 (2017)

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  74. [82]

    Science10(3) (1995) 237–304

    Settles, B.: Active learning literature survey. Science10(3) (1995) 237–304

    1995

  75. [83]

    arXiv preprint arXiv:2503.14476 (2025)

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y ., Zuo, X., Yue, Y ., Fan, T., Liu, G., Liu, L., Liu, X., et al.: Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476 (2025)

    2025 arXiv

  76. [84]

    Advances in Neural Information Processing Systems36(2023) 38975–38987

    Valmeekam, K., Marquez, M., Olmo, A., Sreedharan, S., Kambhampati, S.: Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Advances in Neural Information Processing Systems36(2023) 38975–38987

    2023

  77. [85]

    arXiv preprint arXiv:2110.14168 (2021)

    Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., Schulman, J.: Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)

    2021 arXiv

  78. [86]

    Advances in Neural Information Processing Systems36(2024)

    Li, J., Hui, B., Qu, G., Yang, J., Li, B., Li, B., Wang, B., Qin, B., Geng, R., Huo, N., et al.: Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems36(2024)

    2024

  79. [87]

    Advances in Neural Information Processing Systems 35(2022) 20744–20757

    Yao, S., Chen, H., Yang, J., Narasimhan, K.: Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems 35(2022) 20744–20757

    2022

  80. [88]

    Biometrika30(1/2) (1938) 81–93

    Kendall, M.G.: A new measure of rank correlation. Biometrika30(1/2) (1938) 81–93

    1938

  81. [89]

    In Proceedings of the 19th international conference on World wide web, ACM (2010) 577–586

    Webber, W., Moffat, A., Zobel, J.: A similarity measure for indefinite rankings. In Proceedings of the 19th international conference on World wide web, ACM (2010) 577–586

    2010

  82. [90]

    Educational Researcher5(10) (1976) 3–8

    Glass, G.V .: Primary, secondary, and meta-analysis of research. Educational Researcher5(10) (1976) 3–8

    1976

  83. [91]

    Block 1 is on top of block 3, block 3 is on top of block 2, and block 2 is on the table

    Gosset, W.S.: The probable error of a mean. Biometrika6(1) (1908) 1–25 15 A Training Details A.1 Training Framework Sokoban and Tetris training is based on the StarPO-S (State-Thinking-Actions-Reward Policy Opti- mization - Stable) framework introduced in the RAGEN system [3],...

    1908

  84. [92]

    Build a stable structure - Keep your highest value tiles in a corner

  85. [93]

    Maintain a clear path - Always have a direction where you can combine tiles

  86. [94]

    Chain reactions - Set up sequences of merges that can happen in a single move

  87. [95]

    Look ahead - Think about the consequences of your moves 2-3 steps ahead

  88. [96]

    up", "down

    Building patterns - Common patterns include: (1) Snake/Zig-zag pattern: Arrange tiles in decreasing order in a zigzag; (2) Corner anchoring: Keep the highest tile in a corner and build around it. Avoid: - Getting high-value tiles stuck in the middle of the board - Creating sca...

  89. [97]

    Establish your highest-value tile in one corner and build a descending value structure around it

  90. [98]

    Maintain consistent movement patterns that preserve your high-value corner configuration while allowing for regular merges

  91. [99]

    Anticipate how each potential move affects not just the immediate board state but your options 2-3 moves ahead

  92. [100]

    Create opportunities for chain reactions where multiple merges can occur in a single directional move

  93. [101]

    up", "down

    Implement proven arrangements such as: - Decreasing value snakes that zigzag across the board. - Corner-anchored structures with decreasing values along the edges. ### Pitfalls to Avoid Certain decisions consistently lead to board deterioration: - Allowing high-value tiles to ...

  94. [102]

    **Corner Anchoring**: Position your highest-value tile in a corner and build around it

  95. [103]

    **Structural Stability**: Arrange surrounding tiles in descending order to create a stable formation

  96. [104]

    **Maintaining Merge Paths**: Always keep at least one direction available for safe combinations

  97. [105]

    **Creating Chain Reactions**: Set up moves that trigger multiple merges in a single action

  98. [106]

    up”, “down

    **Forward Planning**: Think 2-3 moves ahead to avoid grid-lock and maintain empty spaces ## Pitfalls to Avoid - Allowing high-value tiles to drift into central positions - Scattering small-value tiles that obstruct potential merges - Making moves that leave the board with no f...

  99. [107]

    **Corner anchor** - Park your highest tile in one corner and build around it

  100. [108]

    **Stable structure** - Arrange surrounding tiles in descending order to protect the corner

  101. [109]

    **Clear merging path** - Keep at least one direction available for safe combinations

  102. [110]

    **Chain reactions** - Set up moves that trigger multiple merges in one swipe

  103. [111]

    up increased empty spaces but produced no merges

    **Look ahead** - Plan 2–3 moves in advance to avoid grid-lock. Preserve empty spaces for new tiles. — ## Avoid - Letting high-value tiles drift into the center. - Scattering small tiles that block merges. - Moves that leave the board with no follow-up merges. — ## Previous Gam...

    2025

  104. [112]

    How the game state changed after the last action

  105. [113]

    Whether the action was effective for the situation

  106. [114]

    Patterns or issues to be aware of

  107. [115]

    G Details of Evaluation Metrics Because games are well-designed, we utilize their built-in metrics to quantify models’ proficiency

    Any strategic insights for future actions Keep your reflection under 100 words and focus only on the most important insights. G Details of Evaluation Metrics Because games are well-designed, we utilize their built-in metrics to quantify models’ proficiency. For each game, we c...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.