Pith. sign in

REVIEW 3 major objections 5 minor 21 references

The Jumping Reasoning Curve? Tracking the Evolution of Reasoning Performance in GPT-[n] and o-[n] Models on Multimodal Puzzles

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that o-series reasoning models, especially o3 and o4-mini, outperform GPT-series models by a wide margin on multimodal puzzle benchmarks, with accuracy rising as reasoning effort increases, while fine-grained perception…

desk verdict A useful capability snapshot with a real contamination blind spot; the o-series 'huge margin' claim is conditional on unseen test items. read the letter →

arxiv 2502.01081 v2 pith:UDY2O2BH submitted 2025-02-03 cs.CV cs.AIcs.CL

Vernon Y.H. Toh , Yew Ken Chia , Deepanway Ghosal , Soujanya Poria This is my paper · ORCID
classification cs.CVcs.AIcs.CL
keywords multimodalreasoningpuzzlebenchmarkso-seriesmodelsGPT-serieseffortscalingvisualperceptionbottlenecksalgorithmicopen-endedevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the reasoning-focused o-series models, especially o3 and o4-mini, beat the GPT-series models by a large margin on two multimodal puzzle benchmarks: PuzzleVQA, which tests visual pattern reasoning, and AlgoPuzzleVQA, which tests algorithmic and spatial reasoning. The gains are not a single jump: within the o-series, accuracy rises steadily as the model is allowed to reason longer (Low, Medium, High). The paper also claims that the advantage is uneven, with fine-grained perception (particularly size) and combinatorial puzzles such as Map Colour and Chain Link still largely unsolved. Because puzzles require no specialized knowledge, the results are read as a progress indicator for general multimodal reasoning and as a diagnostic for which bottlenecks remain.

What carries the argument

The machinery that carries the argument is the pair of benchmark datasets with predefined ontologies: PuzzleVQA's ten puzzle categories (single-concept and dual-concept) and AlgoPuzzleVQA's eighteen puzzle classes allow failures to be attributed to specific reasoning components. The active mechanism is the progressive bottleneck analysis: starting from a bare image-and-question prompt, the authors add ground-truth visual perception, then ground-truth inductive reasoning, and measure the accuracy gain at each step. That decomposition turns a single benchmark score into separate measurements of perception-limited and induction-limited performance. Evaluation is standardized by using a GPT-4o model to match free-form candidate answers, with a manual check of 200 outputs showing 99% agreement.

What would settle it

Run o4-mini on newly generated PuzzleVQA and AlgoPuzzleVQA instances that follow the same puzzle ontologies but were published after the model's training cutoff; if its accuracy drops to roughly GPT-4.5 levels, the reported o-series advantage is largely memorization, whereas if the gains persist, the paper's scaling claim is supported.

Watch

Extended reading notes

Core claim

The central discovery, as the authors state it, is that o-series models consistently outperform GPT-series models by a huge margin on both datasets, and that within the o-series performance scales reliably with increased reasoning modes. The paper shows that o3 and especially o4-mini set new accuracy records on both PuzzleVQA and AlgoPuzzleVQA in multiple-choice and open-ended formats, while the newer GPT models (GPT-4.5, GPT-4.1) improve only marginally over GPT-4o. The same results show persistent ceilings: all models lose substantial accuracy on open-ended questions compared with multiple-choice, and the hardest algorithmic puzzles (Map Colour, Chain Link, Wood Slide) remain near zero for even the best model. A bottleneck experiment injects ground-truth visual descriptions and then ground-truth inductive explanations into the prompt; both injections raise accuracy substantially, leading the paper to conclude that visual perception and inductive reasoning are separate limiting factors, not a single reasoning deficit.

Load-bearing premise

The comparison assumes o3 and o4-mini have not been trained on the public PuzzleVQA and AlgoPuzzleVQA instances, yet the paper reports no contamination check, so part of the reported o-series gains could be memorization rather than reasoning.

Editorial extensions

If this is right

  • If the o-series advantage is real, later reasoning models will continue to convert test-time reasoning effort into higher accuracy on multimodal puzzles, so benchmark scores should keep climbing as long as models can spend more time thinking.
  • Because open-ended accuracy trails multiple-choice accuracy by roughly 15 to 20 points across models, reported progress depends heavily on evaluation format; multiple-choice scores overstate models' generative reasoning ability.
  • The bottleneck analysis implies that improving perception, not just reasoning, is a pre-condition for further progress, since providing perfect visual descriptions lifts even the best model by about ten points on PuzzleVQA.
  • Combinatorial and graph-style puzzles (Map Colour, Chain Link, Wood Slide) remain near zero even with higher reasoning effort, marking a class of problems that this scaling approach does not yet solve.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report a contamination check, so an immediate extension would be to generate fresh puzzle instances from the same ontologies and rerun o4-mini; the paper's claim predicts the ordering survives, while a memorization explanation predicts a drop.
  • The ordering of bottlenecks suggests that making visual perception reliable may produce larger gains than additional test-time compute, a hypothesis the paper itself does not test directly.
  • The near-total failures on Map Colour and Chain Link resemble classic combinatorial search problems, implying these benchmarks are probing planning capability rather than reasoning style; that may explain why reasoning effort alone does not move them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper evaluates seven OpenAI models from the GPT and o series (GPT-4-Turbo, GPT-4o, GPT-4.5, GPT-4.1, o1, o3, and o4-mini) on two multimodal puzzle benchmarks, PuzzleVQA and AlgoPuzzleVQA, in both multiple-choice and open-ended settings. The authors report that o-series models, especially o4-mini, outperform GPT-series models by large margins, that performance improves with increased reasoning effort, and that open-ended evaluation is harder than multiple-choice. They also conduct a bottleneck analysis on PuzzleVQA by injecting ground-truth visual perception and induction explanations, concluding that visual perception and inductive reasoning are the main limiting factors. The datasets and evaluation resources are publicly released on GitHub.

Significance. If the main claims hold, the paper provides a useful measurement of how reasoning-oriented models transfer to multimodal puzzles and where they still fail, with implications for the current debate on reasoning scaling. The paper has several concrete strengths: the comparison spans seven closed models across many task categories, the benchmark is extended to an open-ended format, the GPT-4o-based answer matching was manually validated on 200 samples with 99% agreement, and the data and resources are openly released. However, the headline conclusions are currently conditional on three issues: the scalability claim is contradicted by several rows of the paper's own table, no contamination check is reported for the public benchmark instances, and the bottleneck-injection protocol may leak the answer via the ground-truth explanations. These issues are load-bearing for the paper's central claims, so the contribution cannot be accepted in its present form.

major comments (3)
  1. [Section 5, Table 1] The claim that 'performance scales reliably with increased reasoning modes' is contradicted by the paper's own data. For example, in multiple-choice PuzzleVQA Shapes, o3 scores are 78.5 (Low), 78.0 (Medium), and 74.0 (High); in AlgoPuzzleVQA Checker Move, o3 scores decrease from 61 to 59 to 56; and in AlgoPuzzleVQA Number Slide, o1 scores are 26 (Low), 88 (Medium), and 27 (High). Many other rows are also non-monotonic. Because the table reports a single run with no confidence intervals or repeated seeds, the statement that 'High variants outperform Medium, which in turn surpass Low' is not supported as written. The authors should either report repeated runs with variance/confidence intervals and a statistical comparison, or substantially qualify the scalability claim in light of these exceptions.
  2. [Section 3 and Section 5, Table 1] The central comparison assumes that o3 and o4-mini have not seen the public PuzzleVQA and AlgoPuzzleVQA instances during training. Both datasets are fully public in a GitHub repository, and the paper reports no contamination check such as n-gram overlap, membership inference, or evaluation on newly generated held-out variants. Since OpenAI models are trained on web-scale data, memorization could inflate the reported o-series margins, including the headline AlgoPuzzleVQA multiple-choice averages (GPT-4.1: 37.1 vs o4-mini High: 65.4). The Limitations section also does not mention this risk. The performance trend may still survive a contamination check, but the magnitude of the reported gains is not interpretable without one.
  3. [Section 8, Table 2 and Figure 6] The bottleneck analysis claims that injecting ground-truth visual perception and induction explanations isolates the contribution of each component. However, the induction prompt includes 'an explanation of the underlying pattern within it,' which, together with the ground-truth perception, can be nearly answer-revealing. The observed gains therefore do not cleanly measure inductive reasoning as a separate bottleneck. The conclusion that 'inductive reasoning [is] a separate and significant limiting factor' needs additional controls, such as providing partial or perturbed explanations, withholding the final answer explicitly, or requiring the model to produce the answer from the explanation rather than receiving it implicitly.
minor comments (5)
  1. [Section 4.1] GPT-4o is used both as an answer grader and as one of the evaluated models; the manual 99% agreement on 200 samples is reassuring, but the paper should state the grader prompt temperature and whether the same samples were checked by multiple annotators.
  2. [Table 1] The table is very dense and the repeated column headers for reasoning modes make it difficult to read; splitting the table by dataset or adding explicit sub-column rules would improve clarity.
  3. [Section 7] There is a typo in the phrase 'fundamental difficulty in undestanding the underlying principles'; it should be 'understanding'.
  4. [Section 2.1] The notation 'O-[n]' is used inconsistently with the rest of the paper, which uses lowercase 'o-[n]'.
  5. [Section 10] The limitations section is generic and does not mention the two most important threats to the reported conclusions: the lack of a contamination check and the single-run evaluation without variance estimates.

Circularity Check

0 steps flagged · score 2.0 of 10

No material circularity: the paper's claims are external measurements on public benchmarks, with a minor self-citation of the authors' own datasets that is not load-bearing.

full rationale

The paper's central claims are empirical accuracy measurements on two public benchmarks, PuzzleVQA and AlgoPuzzleVQA, rather than a derivation from first principles or from fitted parameters. The comparison between GPT-[n] and o-[n] models is obtained by running closed models on fixed, publicly released test instances and comparing outputs against ground-truth answers. There is no equation that defines a predicted quantity in terms of an input quantity, no parameter fitted to a subset of data and then reported as a prediction, and no formal theorem invoked as a load-bearing premise. The authors do cite their own prior benchmark papers, Chia et al. (2024) for PuzzleVQA and Ghosal et al. (2024) for AlgoPuzzleVQA, and they describe these datasets as 'ideal benchmarks' for the study. This is a self-citation, but it is not circular in the sense used in this analysis: the benchmarks are fixed public artifacts, and the paper's headline result is a measurement on those artifacts, not a claim derived from the benchmarks' construction or from the authors' prior results. The GPT-4o-based answer matching step is also not circular because it is a scoring mechanism for answer extraction, not the target prediction itself, and the authors manually validated it on 200 random samples with 99% agreement. The bottleneck analysis, which injects ground-truth visual perception and inductive reasoning explanations into the prompt, is an ablation rather than a prediction derived from the injected information; it quantifies how performance changes when those components are supplied, and it does not equate the final accuracy with the injected ground truth by construction. The reader's concern about potential training-data contamination of public instances in closed models is a validity and correctness risk affecting the interpretability of the reported gains, but it is not circularity under the definitions adopted here: the paper could be wrong about generalization while still performing a genuine measurement. Overall, the derivation chain is self-contained as an empirical evaluation, with only a minor self-citation of the authors' own benchmark datasets that does not carry the central claim.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper performs no mathematical derivation and fits no parameters. Its claims rest on the validity of the two benchmarks, the accuracy of GPT-4o as an answer judge, the absence of training contamination, and the assumption that injecting ground-truth explanations isolates perception and induction without revealing answers.

assumptions (4)
  • domain assumption PuzzleVQA and AlgoPuzzleVQA ground-truth labels are correct and the benchmarks are valid proxies for multimodal reasoning ability.
    The entire evaluation treats accuracy on these datasets as measuring visual perception and reasoning; this is adopted from the benchmark authors, including the current paper's authors.
  • domain assumption GPT-4o answer matching correctly classifies open-ended responses as correct or incorrect across all models.
    The paper relies on GPT-4o for answer extraction and matching, validated only on 200 random outputs in Section 4.1.
  • domain assumption The evaluated closed-source models have not memorized the public benchmark instances.
    No contamination check is reported, and the datasets are openly available at a public GitHub repository.
  • ad hoc to paper Injecting ground-truth visual perception and induction explanations isolates those components without revealing the final answer.
    The bottleneck analysis in Section 8 assumes the injected explanations provide no answer cues; the paper does not test this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Jumping Reasoning Curve? Tracking the Evolution of Reasoning Performance in GPT-[n] and o-[n] Models on Multimodal Puzzles." pith.science (2026). https://pith.science/paper/UDY2O2BH

@misc{pith2026250201081,
  author       = {Pith},
  title        = {Pith review of: The Jumping Reasoning Curve? Tracking the Evolution of Reasoning Performance in GPT-[n] and o-[n] Models on Multimodal Puzzles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UDY2O2BH}},
  note         = {Machine review of arXiv:2502.01081}
}
read the original abstract

The releases of OpenAI's o-[n] series, such as o1, o3, and o4-mini, mark a significant paradigm shift in Large Language Models towards advanced reasoning capabilities. Notably, models like o3 have demonstrated strong performance on benchmarks like the Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI). However, this benchmark is limited to symbolic patterns, whereas humans often perceive and reason about multimodal scenarios involving both vision and language data. Thus, there is an urgent need to investigate advanced reasoning capabilities in multimodal tasks. To this end, we track the evolution of the GPT-[n] and o-[n] series models (including o1, o3, and o4-mini) on challenging multimodal puzzles from PuzzleVQA and AlgoPuzzleVQA, which demand fine-grained visual perception. Our results reveal that o-[n] series, particularly later iterations like o3 and o4-mini, significantly outperform the GPT-[n] series and show strong scalability in multimodal reasoning. Nonetheless, despite these substantial advancements and the superior capabilities demonstrated by the o-[n] series, our findings highlight that even these leading models face persistent challenges. Difficulties are particularly evident in tasks requiring precise visual perception, robust compositional reasoning across multiple visual attributes, and solving complex algorithmic or highly combinatorial puzzles, indicating critical areas for future AGI development. We plan to continuously track new models in the series and update our results in this paper accordingly. All resources used in this evaluation are openly available at https://github.com/declare-lab/LLM-PuzzleTest.

Figures

Figures reproduced from arXiv: 2502.01081 by the authors.

Figure 1
Figure 1. Puzzle from PUZZLEVQA. GPT-4-Turbo, GPT-4o, and o1 all got the puzzle incorrect. OpenAI’s GPT-[n] series of models (Brown et al., 2020). Yet, true artificial general intelligence (AGI) requires robust reasoning abilities across different modalities (Fei et al., 2021). Simply excelling in text-based tasks is insufficient for agents intended to understand and interact with the rich, multi￾faceted physical world humans… view at source ↗
Figure 2
Figure 2. The performance of GPT-[n] and o-[n] series models on [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Examples of puzzles from PUZZLEVQA and ALGOPUZZLEVQA. 3 PUZZLEVQA & ALGOPUZZLEVQA Understanding the capabilities and limitations of large multimodal models in reasoning tasks re￾quires datasets that challenge their cognitive capa￾bilities in nuanced ways. Multimodal puzzles are essential benchmarks for evaluating these models, as they require a unique combination of percep￾tion, reasoning, and abstraction. In this s… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: A case study on the Size & Shape puzzle in PUZZLEVQA reveals that GPT-[n] models exhibit weaker visual perception capabilities compared to o-[n] models. o3 and especially o4-mini, set new benchmarks, underscoring continued progress. GPT-[n] Models Progression and Limit…
Figure 5
Figure 5. Figure 5: A case study on the Clock puzzle in ALGOPUZZLEVQA, evaluated in both multiple-choice and open￾ended settings, shows that both GPT-[n] and o-[n] models still lack fine-grained visual perception capabilities. comparison, the o4-mini (High) model performed significantly b…
Figure 6
Figure 6. Figure 6: An example of prompts used in the bottleneck [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: A case study on the Colors & Size puzzle in PUZZLEVQA, evaluated in multiple-choice setting, shows the limitations of GPT-[n] models as compared to o-[n] models [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: A case study on the Size & Shapes puzzle in PUZZLEVQA, evaluated in multiple-choice setting, shows the limitations of GPT-[n] models as compared to o-[n] models [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: A case study on the Numbers & Size puzzle in PUZZLEVQA, evaluated in both multiple-choice and open-ended settings on GPT-4.1 [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: A case study on the Numbers & Size puzzle in PUZZLEVQA, evaluated in both multiple-choice and open-ended settings on o4-mini (High) [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: A case study on the Rotten Fruits puzzle in ALGOPUZZLEVQA, evaluated in multiple-choice setting on GPT-4.1 and GPT-4o where both models got it wrong [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: A case study on the Rotten Fruits puzzle in ALGOPUZZLEVQA, evaluated in a multiple-choice setting using the o1 (High) and o4-mini (High) models, both of which answered correctly [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: A case study on the Checker Move puzzle in ALGOPUZZLEVQA, evaluated in a multiple-choice format using the o1 (High) and o4-mini (High) models, both of which produced correct answers [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: A case study on the Chain Link puzzle in ALGOPUZZLEVQA, evaluated in both multiple-choice setting on o4-mini (High) where in the multiple-choice setting the model got it correct and in the open-ended setting, the model got it incorrect [PITH_FULL_IMAGE:figures/full_f…
Figure 15
Figure 15. Figure 15: A bottleneck analysis case study on the Numbers & Size puzzle in PUZZLEVQA using o1 (High) reveals that the model failed across all three settings [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: A bottleneck analysis case study on the Numbers & Size puzzle in PUZZLEVQA using o4-mini (High) demonstrates that the model successfully solves the task when provided with visual perception ground truth [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 8 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Carolyn Jane Anderson, Joydeep Biswas, Aleksander Boruch-Gruszecki, Federico Cassano, Molly Q Feldman, Arjun Guha, Francesca Lucchetti, and Zixuan Wu. 2025. https://arxiv.org/abs/2502.01584 Phd knowledge not required: A reasoning challenge for large language models . Preprint, arXiv:2502.01584

    arXiv 2025

  4. [4]

    Tom Brown, Benjamin Mann, Nick Ryder, et al. 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf Language models are few-shot learners . In Advances in Neural Information Processing Systems, volume 33, pages 1877--1901. Curran Associates, Inc

    work page 2020

  5. [5]

    Yew Ken Chia, Vernon Toh Yan Han, Deepanway Ghosal, Lidong Bing, and Soujanya Poria. 2024. https://arxiv.org/abs/2403.13315 Puzzlevqa: Diagnosing multimodal reasoning challenges of language models with abstract visual patterns . Preprint, arXiv:2403.13315

    arXiv 2024

  6. [6]

    François Chollet. 2019. https://arxiv.org/abs/1911.01547 On the measure of intelligence . Preprint, arXiv:1911.01547

    arXiv 2019

  7. [7]

    Maksym Del and Mark Fishel. 2023. https://arxiv.org/abs/2212.10114 True detective: A deep abductive reasoning benchmark undoable for gpt-3 and challenging for gpt-4 . Preprint, arXiv:2212.10114

    work page Pith review arXiv 2023

  8. [8]

    Lanzendörfer, Yannick Niedermayr, and Roger Wattenhofer

    Benjamin Estermann, Luca A. Lanzendörfer, Yannick Niedermayr, and Roger Wattenhofer. 2024. https://arxiv.org/abs/2407.00401 Puzzles: A benchmark for neural algorithmic reasoning . Preprint, arXiv:2407.00401

    arXiv 2024

Show all 21 references
  1. [9]

    Nanyi Fei, Zhiwu Lu, Yizhao Gao, Guoxing Yang, Yuqi Huo, Jing Wen, Haoyu Lu, Ruihua Song, Xin Gao, Tao Xiang, Haoran Sun, and Jiling Wen. 2021. https://api.semanticscholar.org/CorpusID:249314857 Towards artificial general intelligence via a multimodal foundation model . Nature...

    2021

  2. [10]

    Deepanway Ghosal, Vernon Toh Yan Han, Chia Yew Ken, and Soujanya Poria. 2024. https://arxiv.org/abs/2403.03864 Are language models puzzle prodigies? algorithmic puzzles unveil serious challenges in multimodal reasoning . Preprint, arXiv:2403.03864

    2024 arXiv

  3. [11]

    Panagiotis Giadikiaroglou, Maria Lymperaiou, Giorgos Filandrianos, and Giorgos Stamou. 2024. https://arxiv.org/abs/2402.11291 Puzzle solving using reasoning of large language models: A survey . Preprint, arXiv:2402.11291

    2024 arXiv

  4. [12]

    Andrew Gritsevskiy, Arjun Panickssery, Aaron Kirtland, Derik Kauffman, Hans Gundlach, Irina Gritsevskaya, Joe Cavanagh, Jonathan Chiang, Lydia La Roux, and Michelle Hung. 2024. https://arxiv.org/abs/2401.05604 Rebus: A robust evaluation benchmark of understanding symbols . Pre...

    2024 arXiv

  5. [13]

    Robert Johansson. 2024. https://arxiv.org/abs/2405.19498 Machine psychology: Integrating operant conditioning with the non-axiomatic reasoning system for advancing artificial general intelligence research . Preprint, arXiv:2405.19498

    2024 arXiv

  6. [14]

    Ehsan Latif, Gengchen Mai, Matthew Nyaaba, Xuansheng Wu, Ninghao Liu, Guoyu Lu, Sheng Li, Tianming Liu, and Xiaoming Zhai. 2024. https://arxiv.org/abs/2304.12479 Agi: Artificial general intelligence for education . Preprint, arXiv:2304.12479

    2024 arXiv

  7. [15]

    Alhassan Mumuni and Fuseini Mumuni. 2025. https://arxiv.org/abs/2501.03151 Large language models for artificial general intelligence (agi): A survey of foundational principles and approaches . Preprint, arXiv:2501.03151

    2025 arXiv

  8. [16]

    Jaehyun Park, Jaegyun Im, Sanha Hwang, Mintaek Lim, Sabina Ualibekova, Sejin Kim, and Sundong Kim. 2023. https://arxiv.org/abs/2306.08204 Unraveling the arc puzzle: Mimicking human solutions with object-centric decision transformer . Preprint, arXiv:2306.08204

    2023 arXiv

  9. [17]

    John Raven. 1989. http://www.jstor.org/stable/1434619 The raven progressive matrices: A review of national norming studies and ethnic and socioeconomic variation within the united states . Journal of Educational Measurement, 26(1):1--16

    1989

  10. [18]

    Hadfield, Richard Ngo, Konstantin Pilz, George Gor, Emma Bluemke, Sarah Shoker, Janet Egan, Robert F

    Girish Sastry, Lennart Heim, Haydn Belfield, Markus Anderljung, Miles Brundage, Julian Hazell, Cullen O'Keefe, Gillian K. Hadfield, Richard Ngo, Konstantin Pilz, George Gor, Emma Bluemke, Sarah Shoker, Janet Egan, Robert F. Trager, Shahar Avin, Adrian Weller, Yoshua Bengio, an...

    2024 arXiv

  11. [19]

    Wang, Dean Lee, Cristina Menghini, Johannes Mols, Jack Doughty, Adam Khoja, Jayson Lynch, Sean Hendryx, Summer Yue, and Dan Hendrycks

    Clinton J. Wang, Dean Lee, Cristina Menghini, Johannes Mols, Jack Doughty, Adam Khoja, Jayson Lynch, Sean Hendryx, Summer Yue, and Dan Hendrycks. 2025. https://arxiv.org/abs/2502.08859 Enigmaeval: A benchmark of long multimodal reasoning challenges . Preprint, arXiv:2502.08859

    2025 arXiv

  12. [20]

    Dhami, Constantin A

    Antonia Wüst, Tim Tobiasch, Lukas Helff, Devendra S. Dhami, Constantin A. Rothkopf, and Kristian Kersting. 2024. https://arxiv.org/abs/2410.19546 Bongard in wonderland: Visual puzzles that still make ai go mad? Preprint, arXiv:2410.19546

    2024 arXiv

  13. [21]

    Bowen Xu. 2024. https://arxiv.org/abs/2404.10731 What is meant by agi? on the definition of artificial general intelligence . Preprint, arXiv:2404.10731

    2024 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.