Pith. sign in

REVIEW 4 major objections 5 minor 91 references

Cardiverse: Harnessing LLMs for Novel Card Game Prototyping

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Cardiverse claims that a single LLM-based pipeline can turn a text description of a card game into novel, rule-consistent, playable variants with trained AI opponents, cutting the human labor of prototyping to a fraction.

desk verdict A solid, well-documented system paper for LLM-based card game prototyping; the consistency-validation claim is weaker than it looks and the baseline comparison is not model-matched, but the integration is novel and deserves a serious referee. read the letter →

arxiv 2502.07128 v2 pith:5ELKYYHJ submitted 2025-02-10 cs.CL cs.AIcs.MM

Danrui Li , Sen Zhang , Sam S. Sohn , Kaidong Hu , Muhammad Usman , Mubbasir Kapadia This is my paper · ORCID
classification cs.CLcs.AIcs.MM
keywords cardgameprototypinglargelanguagemodelsmechanicsdesigncodegenerationgameplayAIself-playoptimizationheuristicfunctionensembleprogramsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a single LLM-based pipeline, Cardiverse, can take the text description of a card game and deliver what normally requires a design and engineering team: novel game variants, runnable code that implements them, and AI opponents strong enough to evaluate them. Three chained stages carry the claim: mechanics extracted from a database of 106 card games are clustered and re-bred into new ones, generated code is refined until a second LLM judging gameplay logs finds no rule violations, and an ensemble of LLM-written action-value functions is pruned by self-play win rates to become the game AI. The reported payoff is that a designer receives interactable, AI-playable variants without writing code or training agents, and that novelty, consistency, and AI quality all beat the paper's chosen baselines: variations score as more distant from the source database and more strategically engaging, validated code scores higher on LLM-judged play-record consistency, and the AI's win-rate advantage over random play ($16.3\pm22.6$ across the 19 games that survived manual filtering) exceeds all four LLM-agent comparisons.

What carries the argument

Three objects carry the argument. The game mechanic graph is a directed graph whose nodes are short mechanic phrases and whose edges encode causal contribution to the game ending, anchored at a shared root node "the game ends"; it is extracted by recursive, breadth-first LLM expansion, clustered with UPGMA over text embeddings, and used to breed new mechanics that replace frequent database mechanics in a target game. The gameplay-record validation loop is the consistency mechanism: five logs sampling the last six rounds and the ending are judged by a second LLM against the structured game description, violations are quoted and matched to a code-snippet database, and the retrieved implementations are patched into the code until the judge passes all sampled logs. The heuristic ensemble is the gameplay AI: LLM-proposed strategies and reward metrics become Python action-value functions, each candidate action receives the average score across the ensemble, and a two-phase stepwise inclusion — adding the function that most improves win rate over 400 games against random opponents, then against a stronger pool — selects the final agent.

What would settle it

Give two card-game experts the code and descriptions for the 55 games that passed the validation loop, have them enumerate every rule the code fails to implement (candidates the paper itself names: omitted poker hand ranks, private information leaked into public state), and compare violation counts with PCon scores — if high-PCon games routinely contain unenforced rules, the oracle assumption is refuted. A sharper variant: delete a single rule from a game's code, rerun the five-log validation loop, and measure how often the judge fails to flag the deletion.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that card-game prototyping decomposes into three subproblems an LLM orchestration can solve end-to-end: inventing mechanics that are genuinely new relative to a database, compiling text rules into code that demonstrably follows them, and producing playable AI without reinforcement learning or hand-built features. The mechanic graph makes novelty a structured edit: mechanics are extracted as directed graphs from game descriptions, clustered by embedding similarity, re-synthesized by the LLM into alternative instances, and swapped into a target game wherever database-frequent mechanics sit. Consistency is treated as an LLM judgment problem: a second LLM reads sampled gameplay logs, quotes violated rules, and the pipeline retrieves correct implementation snippets for those rules and patches the code until the logs pass. The gameplay AI is a code-level policy: LLM-proposed strategies and reward metrics become Python action-scoring functions, their negations double the pool, and stepwise win-rate selection over 400-game matches builds the final ensemble, which never calls an LLM during play. The paper reports that this ensemble's win-rate advantage over random opponents exceeds all four LLM baselines on the combined set ($16.3\pm22.6$, statistically significant), with the only positive average on the varied-game subset.

Load-bearing premise

The load-bearing premise is that an LLM reading a handful of gameplay logs can reliably tell whether generated code obeys the written rules; the paper's consistency claims rest on that judge's verdicts, and the reported metric-effectiveness study compares the judge against other LLM-scored groups rather than against human rule-checking on the 55 reported games.

Editorial extensions

If this is right

  • From a single text description, a designer receives several playable variants whose rules are written into executable code, cutting the manual implementation step out of the prototyping loop.
  • Novelty becomes a controlled edit rather than a gamble: because frequent mechanics are explicitly replaced with cluster-derived alternatives, variations land further from the source database (at $p=0.75$, similarity 87.4 versus PromptBreeder's 88.3) while remaining coherent.
  • Gameplay AI can be built for a new variant in minutes of LLM time, with no feature engineering, no RL training, and no per-decision LLM calls, reaching positive win-rate advantage over random play on 13 common and 6 varied games.
  • Large-scale automated evaluation becomes affordable: the one-time token cost of building the ensemble crosses below per-decision LLM agents after roughly 50 game runs, and the gap widens with every additional game played.
  • Validation with gameplay records lifts the code consistency of a weaker backbone model to the level of a stronger one (PCon 8.99 versus 8.90 for an o1-preview-based generator on common games), so the pipeline does not depend on the strongest model being available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves open whether the mechanic graph supports an explainable distance metric over games; a direct test is whether graph edit distance predicts human similarity judgments better than embedding cosine similarity, which would give designers a legible map of game space.
  • The consistency and win-rate numbers describe the pipeline at its most favorable operating point: only 19 of the 57 generated games carried the AI evaluation because the rest were luck-only or rule-inconsistent, so a stress test is whether tightening the validation loop raises the fraction of strategy-bearing, consistent games.
  • The validation oracle is a single point of failure, and the missing experiment is calibrating the second LLM against human rule-checkers on the same logs; until then "consistent" means "passed an LLM judge," and the pipeline's guarantee is self-referential for the 55 reported games.
  • The engine template, the log-judging oracle, and the heuristic ensemble depend only on state-action dictionaries, so the framework's suggested extensions to branching narratives and puzzle dependencies are directly testable with the same loop.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Cardiverse, an LLM-based pipeline for prototyping card games, with three components: a graph-based mechanic extraction/indexing method that generates novel game variations; a game-code generation system that validates code-rule consistency by having an LLM judge gameplay records and iteratively refine the code; and a gameplay-AI construction method that builds an ensemble of LLM-generated heuristic functions selected by win rate in self-play. The evaluation covers novelty of generated variations (QMS and a small user study), code-generation success/executability/consistency (Succ, Exec, ECon, PCon), and gameplay-AI win-rate advantage against LLM-agent baselines and random agents, plus token-cost analysis. The paper claims the pipeline reduces human labor in prototyping and enables scalable automated evaluation of new game variants.

Significance. If the results hold, the pipeline would be a useful integration of three hard subproblems in game prototyping: guided novelty, consistent code generation, and scalable AI evaluation. The paper has concrete strengths: it ships a code repository, gives detailed prompts and intermediate results in the appendix, and honestly acknowledges residual inconsistencies in generated code and the use of weaker LLM backbones for baselines. The win-rate-based stepwise ensemble selection and the token-cost comparison are sensible engineering contributions. However, the two load-bearing evaluation tools—LLM-based consistency scoring and the win-rate evaluation against random agents—are only weakly grounded, and the main comparisons are partially confounded by model choice and manual test-set filtering. The central claim of consistent code generation is therefore not yet established beyond the paper's own caveats.

major comments (4)
  1. [§3.2, Appendix B.4, Table 2] The consistency-validation oracle is the only filter enforcing code-rule alignment, but it is not validated against human ground truth for subtle errors. The PCon validation in Appendix B.4 only shows that LLM scores drop when the compared description has 70% cosine similarity; it does not show that the LLM reliably catches rule violations in an otherwise correctly described game. The paper's own manual review (B.4) finds that code passing the LLM validation loop still contains unimplemented poker hand rankings and private information leaking into public state, so the high PCon values in Table 2 do not substantiate the claim of consistent game code generation. Please report a human-verified error count on a sample of generated games, or at least an explicit inventory of residual inconsistency types and their frequencies across the test set.
  2. [§4.3, Table 3, Appendix C.4] The gameplay-AI comparison is confounded by model choice. The paper states that the baseline LLM agents use gpt-4o-mini while the proposed method uses gpt-4o, justified by token budget. Since gpt-4o is generally a stronger model, the reported win-rate advantage (e.g., 14.6 vs. 4.2 for the best baseline in common games) cannot be attributed to the ensemble method alone. A same-backbone comparison, or at least a gpt-4o baseline on a subset of games, is needed to support the claim that the method, not the model, drives the performance gain.
  3. [§4.3, Table 5, Appendix C.1] The gameplay-AI evaluation is performed on a manually screened subset: games that are 'not completely consistent' or purely luck-oriented are excluded before evaluation. This means the reported advantage is not an end-to-end property of the full pipeline; human filtering does part of the work. In addition, the ensemble is selected by win rate against random agents and then evaluated against random agents, so the headline result may reflect specialization to random play. Evaluating against stronger fixed opponents across the full test set—rather than only Leduc Hold'em and Uno in Appendix C.1—would make the generalization claim credible.
  4. [§4.1, Table 1, Appendix A.3] The novelty claim rests primarily on cosine similarity of text embeddings, with no evidence that the embedding distance corresponds to mechanical novelty, and the subjective evaluation uses only 12 participants reading text summaries of Go Fish variants rather than playing them. The 'strategically more engaging' conclusion (PromptBreeder coefficient -0.722 in Strategic Engagement) comes from this small, non-playtest study and should be labeled as preliminary. At minimum, report inter-rater reliability and show examples of embeddings-dissimilar but mechanically redundant variations to demonstrate that QMS captures design-level novelty.
minor comments (5)
  1. [Equation (3)] The definition A(p1,p2)=ω(p2,p1)-ω(p2,p2) is only the standard advantage over random when p2 is random; for general policies the second term is a self-play baseline against identical defenders rather than p2's win rate against p1 defenders. Please clarify the intended baseline or restrict the definition to the random-opponent setting actually used in Table 3.
  2. [§3.1.2] The clustering similarity threshold of 0.4 is a free parameter with no sensitivity analysis; since downstream variation and code generation depend on the resulting clusters, a short sensitivity check would strengthen the method.
  3. [Table 7] The footnote is confusing: the text says all differences from our method are significant, while '†' marks pairs that are unlikely to differ (p≥0.7). Please clarify whether '†' refers to differences between the two ablations rather than to comparisons with 'Ours'.
  4. [Appendix A.3] The user study should report how the 12 participants were recruited and whether they were domain experts; with only 12 participants, the mixed-effects model results should be interpreted cautiously.
  5. [Figure 6] The statement that cost after the 50th run is estimated by linear extrapolation is not described in the main text; indicate the extrapolation model and, if possible, show error bars or confidence intervals.

Circularity Check

2 steps flagged · score 5.0 of 10

Partial circularity: code consistency is measured by the same kind of LLM judge used to refine the code, and the headline random-agent win-rate advantage is the exact objective optimized by stepwise selection; independent baselines, user study, and manual checks keep the core pipeline from being fully self-derived.

  1. fitted input called prediction [Sections 3.2 and 4.2, Appendix B.4]
    "For each record, we send it to LLM with the structured game description, asking if the game play record violates any rule in the game description. ... This refinement loop continues until the maximum iteration is reached or the game code passes all validations from the records. ... Each record is evaluated by an LLM to score the consistency (scaled 1 to 10) between actual game play record and game description, which applies a similar implementation as 3.2. ... LLM-based validation process improves the code alignment, rather than ensuring 100% correct code."

    The code is iteratively edited until an LLM judge reports no rule violations, and then consistency is scored by another LLM using 'a similar implementation as 3.2'. Thus the reported PCon improvement over the ablated version is largely the value of the objective that drove the edits, not an independent measure of code-rule alignment. The paper's own manual check confirms the limit: code that passed LLM-based validation still contained unimplemented evaluation standards and private-information leaks, so high PCon scores do not by themselves establish the 'consistent game code generation' claimed in Section 3.2.

  2. fitted input called prediction [Section 3.3.3 and Section 4.3, Table 3]
    "The function with the highest win rate over 400 games against randomly behaving agents is selected and added to the current subset. In each subsequent iteration, we evaluate every remaining candidate function ... and its win rate is again measured over 400 games against the same random players. ... The final subset from the second phase serves as the gameplay policy. ... The win rate performance of a method is assessed by its advantage over competing policies."

    The stepwise inclusion procedure greedily maximizes win rate against random agents, and the headline result in Table 3 is exactly that quantity: 'win rate advantages over random agents'. The reported 'Ours' advantage is therefore the optimized objective value, not a prediction that could independently fail; the LLM-agent baselines were not optimized on this objective, so part of the comparison is an optimizer-versus-non-optimizer artifact. The paper does provide independent evidence, including comparisons to human-designed heuristics and ablations showing both ensembling and optimization matter, so this is partial rather than total circularity.

full rationale

The derivation chain is not wholly circular: the graph-based variation method is evaluated with a user study and with QMS against external baselines; code generation is at least executable and partly manually verified; gameplay AI is compared to LLM baselines and human-designed heuristics. However, two load-bearing measurements are self-referential. First, code consistency is enforced by an LLM judge and then measured by an LLM judge using 'a similar implementation', making the PCon gain over the ablated pipeline a partially forced outcome; the paper's own Appendix B.4 shows the LLM oracle misses real inconsistencies. Second, the gameplay AI is selected by stepwise optimization of win rate against random agents and then primarily evaluated by win rate advantage against random agents, so the main numerical claim is the optimized objective rather than an independent test. These issues are real but not total: independent baselines and manual screening provide some external grounding. A score of 5 reflects this partial circularity, below the 6+ reserved for central claims that reduce entirely to a fit or to a self-citation chain, and above a clean 0-2 because two of the three headline contributions use evaluation metrics that substantially overlap their optimization targets.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest on three unverified premises: that LLM extraction and reflection are trustworthy, that embedding cosine similarity captures novelty and consistency, and that random-agent win rate is an adequate optimization signal. The hand-set thresholds (0.4 clustering threshold, n=4 policies, 6-round horizon, 5-game example database) are free parameters not rigorously justified. No new physical or mathematical entities are introduced.

free parameters (5)
  • Mechanic clustering similarity threshold = 0.4
    Manually set in Section 3.1.2 for UPGMA clustering. It determines which mechanics are grouped, and hence which clusters receive newly generated mechanics. No sensitivity analysis is reported for this value.
  • Base policy item count n = 4
    Section 3.3.1 sets n=4. Appendix C.3 shows n=2 degrades performance and n=6 gives no significant gain, so this hand-chosen value affects results.
  • Gameplay record horizon = last 6 rounds
    Section 3.2 samples five gameplay records containing the last 6 rounds; the horizon length is not ablated.
  • Mechanic extraction depth limit = not reported
    Section 3.1.1 stops expansion by LLM judgment and a manually set depth limit. The value is not reported or varied.
  • Example database size = 5 games
    Appendix B.2 selects 5 games for in-context code drafting. The manual selection is not analyzed for sensitivity.
assumptions (5)
  • domain assumption LLM-generated gameplay records and LLM judgments reliably indicate whether generated code violates the rule description.
    Central to Section 3.2 and the PCon metric; if LLM evaluators are biased, the consistency numbers are not meaningful.
  • domain assumption LLMs extract complete and accurate game mechanic graphs from short text descriptions.
    Section 3.1.1 uses breadth-first LLM expansion to build graphs; errors propagate to clustering, variation, and code generation.
  • domain assumption Text-embedding cosine similarity is a meaningful measure of game novelty and code-description consistency.
    Used in QMS (Equation 2) and ECon/PCon; only a small user study partially validates this for novelty.
  • domain assumption The six-function card game engine abstraction can represent the rules of arbitrary card games.
    Appendix B.1 defines the engine; if some games do not fit the fixed functional structure, generated code cannot be correct.
  • domain assumption Win rate against random agents is a sufficient signal for selecting useful heuristic functions.
    Section 3.3.3 optimizes against random agents and the main evaluation uses the same opponent type; generalization to diverse opponents is not established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cardiverse: Harnessing LLMs for Novel Card Game Prototyping." pith.science (2026). https://pith.science/paper/5ELKYYHJ

@misc{pith2026250207128,
  author       = {Pith},
  title        = {Pith review of: Cardiverse: Harnessing LLMs for Novel Card Game Prototyping},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5ELKYYHJ}},
  note         = {Machine review of arXiv:2502.07128}
}
read the original abstract

The prototyping of computer games, particularly card games, requires extensive human effort in creative ideation and gameplay evaluation. Recent advances in Large Language Models (LLMs) offer opportunities to automate and streamline these processes. However, it remains challenging for LLMs to design novel game mechanics beyond existing databases, generate consistent gameplay environments, and develop scalable gameplay AI for large-scale evaluations. This paper addresses these challenges by introducing a comprehensive automated card game prototyping framework. The approach highlights a graph-based indexing method for generating novel game variations, an LLM-driven system for consistent game code generation validated by gameplay records, and a gameplay AI constructing method that uses an ensemble of LLM-generated heuristic functions optimized through self-play. These contributions aim to accelerate card game prototyping, reduce human labor, and lower barriers to entry for game developers. For code repo visit this http URL https://github.com/danruili/Cardiverse

Figures

Figures reproduced from arXiv: 2502.07128 by the authors.

Figure 1
Figure 1. Our work integrates game mechanic design, code generation, and gameplay AI creation into an LLM-based [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Game mechanic design framework. (Left) Overall pipeline consists of an indexing process (blue and black arrows) and a variation process (orange arrows). (Middle) Mechanic graphs shown in a text embedding space. The lower right corner showcase a new mechanics is created within a cluster (transparent green) and proposed as a replacement to a mechanic unit in the input game (light orange). (Right) Breath-first-search-l… view at source ↗
Figure 3
Figure 3. Use gameplay records to validate code. Gameplay records, generated by game code, are used to validate and refine the consistency of game code. We run the game code in a card game engine designed by us (detailed in § B.1), sampling five gameplay records that are generated by the engine. The record includes the information from the last 6 game rounds and the game ending. In each round, the information includes the obs… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Gameplay AI generation pipeline [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Extracted game mechanics from two card games. The differences are colored in green (modification) and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Gameplay AI Token Cost. Total LLM cost as a function of number of game runs, averaged across 19 games. The cost after 50th run is estimated by linear extrapolations. 5 Conclusion This work introduces an LLM-based pipeline for card game prototyping. It integrates three …
Figure 7
Figure 7. Figure 7: Extracted mechanics in an embedding space. Mechanics are distributed in a 2D UMAP projection of their text embedding space, colored by its game ID. Selected card games shown as mechanic graphs, where the shared root node is displayed in black star mark [PITH_FULL_IMAG…
Figure 8
Figure 8. Figure 8: Extracted mechanic dependencies. Mechanics are distributed in a 2D UMAP projection of a text embedding space, colored by its cluster ID. Starting from selected mechanic clusters (in bold text), their dependen￾cies on supporting mechanics (in normal text) are drawn in l…
Figure 9
Figure 9. Figure 9: More extracted game mechanics from game descriptions. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Game Engine Framework Design. The game engine, composed of 6 core functions (in light blue) in a fixed flow, interacts with external gameplay agent (in light green) through game observation and game action in dictionary forms. The gameplay AI is integrated into this s…
Figure 11
Figure 11. Figure 11: LLM agent workflow of game code generation. [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]
Figure 12
Figure 12. Figure 12: Effectiveness of PCon metric. Average PCon metrics in three game sets, showing PCon is aligned with the actual consistency between the game play record and game description. Gray shaded areas represent the standard errors of average PCon. CoT ReAct Re exion Belief Our…
Figure 13
Figure 13. Figure 13: Gameplay AI win rate advantage by games. Win rate advantage to random agents when different methods are applied. The performance of random agents (A(p, prandom) = 0) is shown as black dotted circles. (left) Common games (right) varied games, where the names of base ga…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

91 extracted references · 64 canonical work pages

  1. [1]

    CRC Press, 3rd edition, 2014

    Tracy Fullerton.Game Design Workshop: A Playcentric Approach to Creating Innovative Games. CRC Press, 3rd edition, 2014

    2014

  2. [2]

    Yannakakis

    Roberto Gallotta, Graham Todd, Marvin Zammit, Sam Earle, Antonios Liapis, Julian Togelius, and Georgios N. Yannakakis. Large language models and games: A survey and roadmap.IEEE Transactions on Games, page 1–18, 2024. ISSN 2475-1510. doi: 10.1109/tg.2024.3461510. URLhttp://dx.doi.org/10.1109/TG.2024. 3461510

    work page arXiv 2024

  3. [3]

    A preliminary study on a conceptual game feature generation and recommendation system

    M Charity, Yash Bhartia, Daniel Zhang, Ahmed Khalifa, and Julian Togelius. A preliminary study on a conceptual game feature generation and recommendation system. In2023 IEEE Conference on Games (CoG), pages 1–5, 2023. doi: 10.1109/CoG57401.2023.10333164. 11

    work page arXiv 2023

  4. [4]

    Instruction-driven game engine: A poker case study

    Hongqiu Wu, Xingyuan Liu, Yan Wang, and Hai Zhao. Instruction-driven game engine: A poker case study. In Delia Irazu Hernandez Farias, Tom Hope, and Manling Li, editors,Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 507–519, Miami, Florida, USA, November 2024. Association for Computatio...

    doi:10.18653/v1/2024.emnlp-demo.51 2024

  5. [5]

    ChatDev: Communicative agents for software development

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. ChatDev: Communicative agents for software development. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Li...

    2024

  6. [6]

    ReAct: Synergizing Reasoning and Acting in Language Models, March 2023

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing Reasoning and Acting in Language Models, March 2023. URLhttp://arxiv.org/abs/2210. 03629. arXiv:2210.03629 [cs]

    arXiv 2023

  7. [7]

    Mda: A formal approach to game design and game research

    Robin Hunicke, Marc Leblanc, and Robert Zubek. Mda: A formal approach to game design and game research. AAAI Workshop - Technical Report, 1, 01 2004

    2004

  8. [8]

    Matthew Guzdial and Mark O. Riedl. Conceptual game expansion.IEEE Transactions on Games, 14(1): 93–106, March 2022. ISSN 2475-1510. doi: 10.1109/tg.2021.3060005. URLhttp://dx.doi.org/10.1109/TG. 2021.3060005

    work page arXiv 2022

Show all 91 references
  1. [9]

    Pitako - recommending game design elements in cicero

    Tiago Machado, Dan Gopstein, Andy Nealen, and Julian Togelius. Pitako - recommending game design elements in cicero. In2019 IEEE Conference on Games (CoG), pages 1–8, 2019. doi: 10.1109/CIG.2019. 8848081

    2019 doi

  2. [10]

    Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and LINGMING ZHANG. Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation. InThirty-seventh Conference on Neural Information Processing Systems, 2023. URLhttps://openreview...

    2023

  3. [11]

    Exploring game space using survival analysis

    Aaron Isaksen, Dan Gopstein, and Andy Nealen. Exploring game space using survival analysis. InInternational Conference on Foundations of Digital Games, 2015. URLhttps://api.semanticscholar.org/CorpusID: 8611109

    2015

  4. [12]

    DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning

    Daochen Zha, Jingru Xie, Wenye Ma, Sheng Zhang, Xiangru Lian, Xia Hu, and Ji Liu. DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning. InProceedings of the 38th International Conference on Machine Learning, pages 12333–12344. PMLR, July 2021. URLhttps://proc...

    2021

  5. [13]

    Reflexion: lan- guage agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. Reflexion: lan- guage agents with verbal reinforcement learning. InThirty-seventh Conference on Neural Information Pro- cessing Systems, 2023. URLhttps://openreview.net/forum?id=vAElhFcKW6

    2023

  6. [14]

    Boosting LLM agents with recursive contemplation for effective deception handling

    Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, and Gao Huang. Boosting LLM agents with recursive contemplation for effective deception handling. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Findings ...

    2024 doi

  7. [15]

    Inner monologue: Embodied reasoning through planning with language models

    Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, Pierre Sermanet, Tomas Jackson, Noah Brown, Linda Luu, Sergey Levine, Karol Hausman, and brian ichter. Inner monologue: Embodied reasoning th...

    2023

  8. [16]

    Zhaoyang Xia, Somdeb Sarkhel, Mehrab Tanjim, Stefano Petrangeli, Ishita Dasgupta, Yuxiao Chen, Jinxuan Xu, Di Liu, Saayan Mitra, and Dimitris N. Metaxas. VISIAR: Empower MLLM for visual story ideation. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pileh...

    2025 doi

  9. [17]

    Sohn, Sen Zhang, Che-Jui Chang, and Mubbasir Kapadia

    Danrui Li, Samuel S. Sohn, Sen Zhang, Che-Jui Chang, and Mubbasir Kapadia. From words to worlds: Transforming one-line prompts into multi-modal digital stories with llm agents. InProceedings of the 17th ACM SIGGRAPH Conference on Motion, Interaction, and Games, MIG ’24, New Yo...

    2024

  10. [18]

    Archseek: Retrieving architectural case studies using vision-language models, 2025

    Danrui Li, Yichao Shi, Yaluo Wang, Ziying Shi, and Mubbasir Kapadia. Archseek: Retrieving architectural case studies using vision-language models, 2025. URLhttps://arxiv.org/abs/2503.18680

    2025 arXiv

  11. [19]

    Level generation through large language models

    Graham Todd, Sam Earle, Muhammad Umair Nasir, Michael Cerny Green, and Julian Togelius. Level generation through large language models. InProceedings of the 18th International Conference on the Foundations of Digital Games, FDG 2023. ACM, April 2023. doi: 10.1145/3582437.35872...

    2023

  12. [20]

    A framework for exploring player perceptions of LLM- generated dialogue in commercial video games

    Nader Akoury, Qian Yang, and Mohit Iyyer. A framework for exploring player perceptions of LLM- generated dialogue in commercial video games. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2295...

    2023 doi

  13. [21]

    Scenecraft: an llm agent for synthesizing 3d scenes as blender code

    Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi. Scenecraft: an llm agent for synthesizing 3d scenes as blender code. InProceedings of the 41st Interna- tional Conference on Machine Learning, ICML’24. JMLR.org, 2024

    2024

  14. [22]

    Vikram Kumaran, Jonathan Rowe, Bradford Mott, and James Lester. Scenecraft: Automating interactive narrative scene generation in digital games with large language models.Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 19(1):...

    2023 doi

  15. [23]

    The angelina videogame design system—part i.IEEE Transac- tions on Computational Intelligence and AI in Games, 9(2):192–203, 2017

    Michael Cook, Simon Colton, and Jeremy Gow. The angelina videogame design system—part i.IEEE Transac- tions on Computational Intelligence and AI in Games, 9(2):192–203, 2017. doi: 10.1109/TCIAIG.2016.2520256

    2017

  16. [24]

    An experiment in automatic game design

    Julian Togelius and Jurgen Schmidhuber. An experiment in automatic game design. In2008 IEEE Symposium On Computational Intelligence and Games, pages 111–118, 2008. doi: 10.1109/CIG.2008.5035629

    2008

  17. [25]

    Margaret A. Boden. Creativity and artificial intelligence.Artificial Intelligence, 103(1):347–356, 1998. ISSN 0004-3702. doi: https://doi.org/10.1016/S0004-3702(98)00055-1. URLhttps://www.sciencedirect.com/ science/article/pii/S0004370298000551. Artificial Intelligence 40 years later

    1998 doi

  18. [26]

    Matthew Guzdial, Nicholas Liao, Vishwa Shah, and Mark O. Riedl. Creative invention benchmark, 2018. URL https://arxiv.org/abs/1805.03720

    2018 arXiv

  19. [27]

    Tiannan Chen and Stephen Guy. Chaos cards: Creating novel digital card games through grammatical content generation and meta-based card evaluation.Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 16(1):196–202, Oct. 2020. doi...

    2020 doi

  20. [28]

    Megan Sumner, Vardan Saini, and Matthew Guzdial. Mechanic Maker: Accessible Game Development via Symbolic Learning Program Synthesis.Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 20(1):235–244, November 2024. doi: 10.1609/...

    2024 doi

  21. [29]

    From local to global: A graph rag approach to query-focused summarization, 2024

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. From local to global: A graph rag approach to query-focused summarization, 2024. URL https://arxiv.org/abs/2404.16130

    2024 arXiv

  22. [30]

    Tf-coder: Program synthesis for tensor manipulations.ACM Trans

    Kensen Shi, David Bieber, and Rishabh Singh. Tf-coder: Program synthesis for tensor manipulations.ACM Trans. Program. Lang. Syst., 44(2), May 2022. ISSN 0164-0925. doi: 10.1145/3517034. URLhttps://doi. org/10.1145/3517034

    2022 doi

  23. [31]

    Tenenbaum

    Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sabl´ e-Meyer, Lucas Morales, Luke Hewitt, Luc Cary, Armando Solar-Lezama, and Joshua B. Tenenbaum. DreamCoder: bootstrapping inductive program synthesis with wake-sleep library learning. InProceedings of the 42nd ACM SIGPLAN I...

    2021

  24. [32]

    Shuzheng Gao, Xin-Cheng Wen, Cuiyun Gao, Wenxuan Wang, Hongyu Zhang, and Michael R. Lyu. What makes good in-context demonstrations for code intelligence tasks with llms? In2023 38th IEEE/ACM Inter- national Conference on Automated Software Engineering (ASE), page 761–773. IEEE...

    2023

  25. [33]

    Demystifying llm-based software engineering agents.Proc

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Demystifying llm-based software engineering agents.Proc. ACM Softw. Eng., 2(FSE), June 2025. doi: 10.1145/3715754. URLhttps: //doi.org/10.1145/3715754

    2025 doi

  26. [34]

    SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? InThe Twelfth International Conference on Learning Representations, 2024. URLhttps://openreview.net/foru...

    2024

  27. [35]

    Sturtevant and Adam M

    Nathan R. Sturtevant and Adam M. White. Feature Construction for Reinforcement Learning in Hearts. In H. Jaap van den Herik, Paolo Ciancarini, and H. H. L. M. (Jeroen) Donkers, editors,Computers and Games, Lecture Notes in Computer Science, pages 122–134, Berlin, Heidelberg, 2...

    2007

  28. [36]

    Zacharias Holland, Elnaz Davoodi, Alden Christianson, and Michael Bowling

    Martin Schmid, Matej Moravˇ c ´ ık, Neil Burch, Rudolf Kadlec, Josh Davidson, Kevin Waugh, Nolan Bard, Finbarr Timbers, Marc Lanctot, G. Zacharias Holland, Elnaz Davoodi, Alden Christianson, and Michael Bowling. Student of Games: A unified learning algorithm for both perfect a...

    2023 doi

  29. [37]

    doi: 10.1007/978-3-540-75538-8 11

    doi

  30. [38]

    Suspicion agent: Playing imperfect information games with theory of mind aware gpt-4

    Jiaxian Guo, Bo Yang, Paul Yoo, Bill Yuchen Lin, Yusuke Iwasawa, and Yutaka Matsuo. Suspicion agent: Playing imperfect information games with theory of mind aware gpt-4. InProceedings of the First Conference on Language Modeling (COLM 2024), 2024. URLhttp://arxiv.org/abs/2309.17277

    2024 arXiv

  31. [39]

    Superhuman AI for multiplayer poker.Science, 365(6456):885–890, Au- gust 2019

    Noam Brown and Tuomas Sandholm. Superhuman AI for multiplayer poker.Science, 365(6456):885–890, Au- gust 2019. doi: 10.1126/science.aay2400. URLhttps://www.science.org/doi/10.1126/science.aay2400. Publisher: American Association for the Advancement of Science

    2019 doi

  32. [40]

    Eureka: Human-level reward design via coding large language models

    Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. InThe Twelfth International Conference on Learning Representations, 2024...

    2024

  33. [41]

    Agent-pro: Learning to evolve via policy-level reflection and optimization

    Wenqi Zhang, Ke Tang, Hai Wu, Mengna Wang, Yongliang Shen, Guiyang Hou, Zeqi Tan, Peng Li, Yueting Zhuang, and Weiming Lu. Agent-pro: Learning to evolve via policy-level reflection and optimization. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 6...

    2024 doi

  34. [42]

    Evaluating and Enhancing LLMs Agent Based on Theory of Mind in Guandan: A Multi-Player Cooperative Game Under Imperfect Information

    Yauwai Yim, Chunkit Chan, Tianyu Shi, Zheye Deng, Wei Fan, Tianshi Zheng, and Yangqiu Song. Evaluating and Enhancing LLMs Agent Based on Theory of Mind in Guandan: A Multi-Player Cooperative Game Under Imperfect Information . In2024 IEEE/WIC International Conference on Web Int...

    2024

  35. [43]

    Chatpcg: Large language model-driven reward design for procedural content generation, 2024

    In-Chang Baek, Tae-Hwa Park, Jin-Ha Noh, Cheong-Mok Bae, and Kyung-Joong Kim. Chatpcg: Large language model-driven reward design for procedural content generation, 2024. URLhttps://arxiv.org/abs/ 2406.11875

    2024 arXiv

  36. [44]

    Code as Policies: Language Model Programs for Embodied Control

    Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as Policies: Language Model Programs for Embodied Control. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 9493–9500, May 2023. doi: 10.1...

    2023

  37. [45]

    Mastering board games by external and internal planning with language models,

    John Schultz, Jakub Adamek, Matej Jusup, Marc Lanctot, Michael Kaisers, Sarah Perrin, Daniel Hennes, Jeremy Shar, Cannada Lewis, Anian Ruoss, Tom Zahavy, Petar Veliˇ ckovi´ c, Laurel Prince, Satinder Singh, Eric Malmi, and Nenad Tomaˇ sev. Mastering board games by external and...

  38. [46]

    Sokal, C.D

    R.R. Sokal, C.D. Michener, and University of Kansas.A Statistical Method for Evaluating Systematic Rela- tionships. University of Kansas science bulletin. University of Kansas, 1958

    1958

  39. [47]

    Prompt- breeder: self-referential self-improvement via prompt evolution

    Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt¨ aschel. Prompt- breeder: self-referential self-improvement via prompt evolution. InProceedings of the 41st International Con- ference on Machine Learning, ICML’24. JMLR.org, 2024

    2024

  40. [48]

    Strategist: Self-improvement of LLM decision making via bi-level tree search

    Jonathan Light, Min Cai, Weiqin Chen, Guanzhi Wang, Xiusi Chen, Wei Cheng, Yisong Yue, and Ziniu Hu. Strategist: Self-improvement of LLM decision making via bi-level tree search. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview...

    2025

  41. [49]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. InProceedings of the 36th International Conference on Neural Information Processing Syst...

  42. [50]

    RLCard: A Toolkit for Reinforcement Learning in Card Games, February 2020

    Daochen Zha, Kwei-Herng Lai, Yuanpu Cao, Songyi Huang, Ruzhe Wei, Junyu Guo, and Xia Hu. RLCard: A Toolkit for Reinforcement Learning in Card Games, February 2020. URLhttp://arxiv.org/abs/1910.04376. arXiv:1910.04376 [cs]

    2020 arXiv

  43. [51]

    Learn how to play card games, 2025

    Bicycle Playing Cards. Learn how to play card games, 2025. URLhttps://bicyclecards.com/how-to-play. Accessed: 2025-05-17

    2025

  44. [52]

    Cardiverse: Harnessing LLMs for Novel Card Game Prototyping

    Anthony Costarelli, Mat Allen, Roman Hauksson, Grace Sodunke, Suhas Hariharan, Carlson Cheng, Wenjie Li, Joshua Clymer, and Arjun Yadav. Gamebench: Evaluating strategic reasoning abilities of llm agents, 2024. URLhttps://arxiv.org/abs/2406.06613. 15 Appendix for “Cardiverse: H...

    2024 arXiv

  45. [53]

    ISBN 9781713871088

    Curran Associates Inc. ISBN 9781713871088

  46. [55]

    Joshua D. A. Jung and Jesse Hoey. Distance-based mapping for general game playing. In2021 IEEE Conference on Games (CoG), pages 1–8, 2021. doi: 10.1109/CoG52621.2021.9619036

    2021

  47. [57]

    - Pre - r e q u i s i t e : Must follow suit , unless void in that suit

    Lead a Trick : - Play any card from their hand . - Pre - r e q u i s i t e : Must follow suit , unless void in that suit

  48. [58]

    - Else , play any card

    Follow a Trick : - Play a card ma tc hin g the led suit , if p oss ib le . - Else , play any card

  49. [59]

    This can be de cla re d before playing any card during a player ’ s turn

    Declare Tem po ra l Suit Shift : - A nn oun ce the shift of rule for a trick to e s t a b l i s h a t e m p o r a r y trump suit . This can be de cla re d before playing any card during a player ’ s turn . - Pre - r e q u i s i t e : Can only be used once per game per player ....

  50. [60]

    The player with the 2 of Clubs starts the first trick

  51. [61]

    Players take turns in c l o c k w i s e order , leading with the suit f o l l o w i n g rules

  52. [62]

    - Using Te mp ora l Suit Shift if s t r a t e g i c a l l y a d v a n t a g e o u s

    Each trick con si sts of : - Playing cards in the order of player turns . - Using Te mp ora l Suit Shift if s t r a t e g i c a l l y a d v a n t a g e o u s . When declared , the player ’ s chosen suit acts as trump for that entire trick

  53. [63]

    The winner of a trick leads the next trick

  54. [64]

    Hearts ␣ of ␣ Time

    Play c o n t i n u e s until all cards have been played . #### Winning C o n d i t i o n s : - Game ends when a player reaches a preset d i s t u r b a n c e t h r e s h o l d ( c om mon ly 100 points ) . - Player with the lowest d i s t u r b a n c e score wins . --- ### 6. O...

  55. [65]

    - P r e r e q u i s i t e : The card must match the top card by rank , suit , or symbol

    Play a Card : - Place one card from their hand on the starter pile . - P r e r e q u i s i t e : The card must match the top card by rank , suit , or symbol

  56. [66]

    - R eq uir ed : Se qu enc e of three or more cards sharing a common symbol

    Merge S equ en ce : - Create a se qu enc e with the top card of the starter pile co mb ine d with cards from the player ’ s hand . - R eq uir ed : Se qu enc e of three or more cards sharing a common symbol . - A d d i t i o n a l Rule : Only one merged s equ en ce can be creat...

  57. [67]

    Use Wild Card ( Eight ) : - Play an eight to change suit and symbol t r a j e c t o r y

  58. [68]

    - P r e r e q u i s i t e : No mat ch in g card in hand , or by s t r a t e g i c choice

    Draw a Card : - Draw from the stock if they can ’ t play , OR opt to s t r a t e g i c a l l y draw despite a l t e r n a t i v e s . - P r e r e q u i s i t e : No mat ch in g card in hand , or by s t r a t e g i c choice . --- ### 5. Round #### Se que nc e of Play :

  59. [69]

    Player left of the dealer starts

  60. [70]

    Turns proceed in c l o c k w i s e order

  61. [71]

    - Draws a card ( if needed )

    During a turn , a player : - Plays a card or merges . - Draws a card ( if needed ) . - The turn ends and passes to the next player

  62. [72]

    crazy- eights

    Play c o n t i n u e s until : - A player reaches 150 points through merges and regular g ame pl ay . #### Winning C o n d i t i o n s : - A player wins upon re ach in g 150 points through s u c c e s s f u l merges and card play . --- ### 6. Other Game M e c h a n i c s & Rul...

  63. [73]

    Go ␣ Fish ,

    Turn 0 ( Player 0) : - Player 0 has cards K , 10 , 10 , K , K and chooses to request from Player 1 the rank 10. This is legal because Player 0 has cards of rank 10. - Player 1 has 1 card ; no i n f o r m a t i o n if it was 10. Ass um in g it wasn ’t , Player 0 should " Go ␣ F...

  64. [74]

    - Since Player 0 does not have this card ( as per obs er ve d hand K , 10 , 10 , K , K ) , the request fails , but again no stock to draw from

    Turn 1 ( Player 1) : - Player 1 has only card 8 and re que st s rank 8 from Player 0. - Since Player 0 does not have this card ( as per obs er ve d hand K , 10 , 10 , K , K ) , the request fails , but again no stock to draw from . The turn ends

  65. [75]

    This action is valid since they have rank 10 cards

    Turn 2 ( Player 2) : - Player 2 has cards 10 , 10 and re qu est s rank 10 from Player 0. This action is valid since they have rank 10 cards . - Player 0 has two 10 s to pass over - i n d i c a t i n g a s u c c e s s f u l request . Player 2 should take another turn but this i...

  66. [76]

    Go ␣ Fish

    Turn 3 ( Player 3) and S u b s e q u e n t O b s e r v a t i o n s : - Player 3 re qu est s rank 8 from Player 1. Since Player 1 does not have the rank ( hand size is 1 , card 8 not e x p l i c i t l y c o n f i r m e d ) , the turn would t y p i c a l l y result in " Go ␣ Fis...

  67. [77]

    "" Process the action and update the game state

    Game End : - Legal actions r e m a i n i n g for the player with hand K , K , K show r e m a i n i n g p o s s i b i l i t i e s a c c u r a t e l y . No correct h an dli ng of p o t e n t i a l book f o r m a t i o n or r e a t t e m p t s cap tu red . * Ana ly sis Summary * ...

  68. [78]

    These con ce pt s are often ce nte re d around a theme or have similar m e c h a n i c s

    ** Input **: You will receive a list of ga me pla y concept i n s t a n c e s . These con ce pt s are often ce nte re d around a theme or have similar m e c h a n i c s

  69. [79]

    name ":

    ** Output **: Your re sp ons e should include : ‘‘‘ json { " name ": " < The name of the game concept >" , " d e s c r i p t i o n ":{ " common ": " < A concise s t a t e m e n t that g e n e r a l i z e s the core concept or theme common across the listed ga mep la y concept ...

  70. [80]

    ** Game State ** - Define the game state , c a t e g o r i z e d into common i n f o r m a t i o n and player - s pe ci fic i n f o r m a t i o n ( grouped into public and private )

  71. [81]

    ** Card ** - Specify card a t t r i b u t e s such as rank , suit , and any special a b i l i t i e s or values

  72. [82]

    ** Deck and Initial Dealing ** - D es cri be the deck composition , dealing process , and setup at the b e g i n n i n g of the game

  73. [83]

    ** Legal Action Space ** - List all po ss ibl e actions players can perform during their turn , s p e c i f y i n g the p r e r e q u i s i t e s of each action

  74. [84]

    - E l a b o r a t e in each players ’ turn , the order of actions they can take , and the o ut com es of each action

    ** Round ** - D es cri be the se que nc e of play and how the game p r o g r e s s e s from one player to the next . - E l a b o r a t e in each players ’ turn , the order of actions they can take , and the o ut com es of each action . - Explain how the game ends and the winni...

  75. [85]

    ** Other Game M e c h a n i c s & Rules ** - Detail any a d d i t i o n a l game mechanics , rules , or special actions that players can take during the game

  76. [86]

    ** Player O b s e r v a t i o n I n f o r m a t i o n ** - Specify what i n f o r m a t i o n players can observe during the game , such as their hand , the starter pile , de cl are d suits , and o pp one nt actions

  77. [87]

    s t r a t e g i c a l l y use

    ** Payoffs ** - Explain when game ends , how scoring works , i n c l u d i n g point values for cards . Ensure clarity and p r e c i s i o n to f a c i l i t a t e i m p l e m e n t a t i o n or usage as a r e f e r e n c e for game rules . # Example { example } System Prompts...

  78. [90]

    thought

    Respond in the f o l l o w i n g JSON format : Re me mbe r : - Be t ho rou gh in your r e a s o n i n g . - Choose actions when you need more i n f o r m a t i o n . - Always base your r e a s o n i n g on the actual o b s e r v a t i o n s from chosen action . If you have eno...

  79. [91]

    Analyze the query , pre vi ou s r e a s o n i n g steps , and o b s e r v a t i o n s

  80. [92]

    Decide on the next action : choose an action or provide a final answer

  81. [93]

    thought

    Respond in the f o l l o w i n g JSON format : Re me mbe r : - Be t ho rou gh in your r e a s o n i n g . - Choose actions when you need more i n f o r m a t i o n . - Always base your r e a s o n i n g on the actual o b s e r v a t i o n s from chosen action . If you have eno...

  82. [2021]

    ISBN 978-1-4503-8391-2

    Association for Computing Machinery. ISBN 978-1-4503-8391-2. doi: 10.1145/3453483.3454080. URL https://dl.acm.org/doi/10.1145/3453483.3454080

  83. [2024]

    URLhttps://arxiv.org/abs/2412.12119

    arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.