Pith. sign in

REVIEW 3 major objections 6 minor 4 cited by

GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GeoX claims that geometry problem solving by multimodal LLMs is best advanced by pre-training a vision encoder and language decoder separately on geometry data, then aligning them through formalized diagram captions before instruction…

desk verdict A solid three-stage geometry MLLM pipeline with genuinely new alignment components, but the headline SOTA claim is not yet trustworthy until the cross-dataset leakage risk and asymmetric baselines are addressed. read the letter →

arxiv 2412.11863 v2 pith:MDQ6XBZT submitted 2024-12-16 cs.CV cs.CL

Renqiu Xia , Mingsheng Li , Hancheng Ye , Wenjie Wu , Hongbin Zhou , Jiakang Yuan , Tianshuo Peng , Xinyu Cai
show 7 more authors
Xiangchao Yan Bin Wang Conghui He Botian Shi Tao Chen Junchi Yan Bo Zhang
This is my paper · ORCID
classification cs.CVcs.CL
keywords geometryproblemsolvingmultimodallargelanguagemodelsformalizedpre-trainingvision-languagealignmentGS-FormerdiagramunderstandingsymbolicsolverGeoX
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GeoX is a proposal for teaching a multimodal large model to solve plane-geometry problems by generating formal program sequences that a symbolic solver can execute and verify. The paper's central claim is that the decisive ingredient is not the base language model or the solver, but a three-stage pre-training scheme: geometry-only masked auto-encoding for the vision encoder, auto-regressive fine-tuning on a 100M-token geometry corpus for the language decoder, and a Generator-and-Sampler Transformer that aligns diagrams with formalized captions before instruction tuning. On four public benchmarks the model reports top-1 accuracy of 54.9% on GeoQA, 54.4% on UniGeo calculation, 58.6% Completion on Geometry3K, 52.7% Completion on PGPS9K, and 72.6% on MathVista-GEO, exceeding both generalist MLLMs and geometry specialists. The broader interest is that the outputs are verifiable: because the model emits formal programs rather than free-form explanations, claim and reasoning can be checked automatically.

What carries the argument

The mechanism that carries the argument is the Generator-and-Sampler Transformer (GS-Former), a cross-modal alignment module that converts a geometric diagram into a small set of discriminative queries. Its Geo-aware Query Generator builds sample-specific queries by pooling and projecting the diagram's visual features, and its Semantics-guided Geometry Sampler learns a binary mask that progressively drops uninformative patches, regularized by a sparsity term so the mask does not collapse to all-ones. GS-Former is trained on a 6,232-image dataset of formalized captions—only collinear (Line ...) and concyclic (\odot ... lieson ...) point relations—with contrastive, matching, and caption-generation losses. The updated queries are projected into the token space of Geo-LLM-7B, which decodes a formal program that a symbolic solver executes to yield the answer.

What would settle it

Audit the overlap: extract the question text and program annotations from the GeoQA and UniGeo test splits and search for exact or near-exact matches in the 100M-token geometric corpus and in the instruction-tuning splits described in Appendix E. If any test item appears in training data, comparisons are confounded; retraining on a deduplicated corpus and re-running GeoQA, UniGeo, Geometry3K, and PGPS9K would settle whether the formalized pre-training gains survive.

Watch

Extended reading notes

Core claim

The central claim is that automatic geometry problem solving is not primarily limited by the base language model's reasoning ability but by the mismatch between natural-image and natural-text pre-training and the sparse, formal nature of geometric diagrams and symbols. GeoX therefore stages training: masked auto-encoding teaches Geo-ViT to represent diagrams; auto-regressive fine-tuning on a 100M-token geometry corpus teaches Geo-LLM to speak the formal language; GS-Former aligns the two modalities using formalized captions of collinear and concyclic point relations; and visual instruction tuning adapts the whole system to emit solver-executable program sequences. The paper reports that this sequence of choices outperforms generalists and specialists on GeoQA, UniGeo, Geometry3K, and PGPS9K, and it attributes the gains to the formalized alignment via the GS-Former rather than to the LLEMMA base or the external solver, citing ablations that drop from 58.6% to 33.1% Completion on Geometry3K when alignment is removed and from 58.6% to 55.7% when formal captions are replaced by natural-language captions.

Load-bearing premise

The load-bearing premise is that no benchmark test question ever appears in the model's own training data through dataset overlap; the paper states that the 100M-token corpus was built only from training splits, but it does not describe cross-dataset deduplication, and because UniGeo's calculation split is the same 4,998 problems as GeoQA with both used in pre-training, instruction tuning, and evaluation, a leaked test question would make the reported gains—including near-perfect UniGeo proving scores—memorization rather than geometric reasoning.

Editorial extensions

If this is right

  • Unified generalists become viable: one GeoX model reports state-of-the-art numbers on GeoQA, UniGeo, Geometry3K, and PGPS9K, so per-dataset specialist architectures are no longer the default choice for geometry problem solving.
  • Answers become auditable: because the model emits formal program sequences executed by a symbolic solver, any output can be checked step-by-step, and the paper's evaluation treats process errors as incorrect rather than crediting lucky final digits.
  • Formal language is the better alignment target: the ablation showing formal captions beat natural-language captions by 2.9 Completion points on Geometry3K implies that future diagram-caption datasets should encode structural relations like collinearity and concyclicity.
  • The near-perfect UniGeo proving scores (97.8–100%) suggest that proof generation for these datasets is largely solvable once the model is geometry-pretrained, conditional on the training and evaluation protocol being leak-free.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the GS-Former sparsification idea should transfer to other diagram-heavy domains where information is sparse, such as circuit diagrams or molecular drawings, but GeoX does not test this.
  • The Appendix D 'random-guess correction' is a paper-specific metric; if the same correction were applied to G-LLaVA and MAVIS under exact-process scoring, the ranking in Table 8 could change.
  • The cleanest test the paper leaves open is cross-dataset deduplication: since UniGeo's calculation split is identical to GeoQA, an overlap audit between the pre-training corpus and test splits would separate memorization from generalization.
  • The recipe's dependence on the base LLM is not isolated; a controlled experiment swapping LLEMMA for another math-pretrained 7B model would show how much of the gain is attributable to the formalized pre-training alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. GeoX is a multimodal geometry problem-solving model trained in three stages: unimodal pre-training (MAE-based Geo-ViT on 120K diagrams, auto-regressive fine-tuning of LLEMMA-7B on a 100M-token geometric corpus), geometry-language alignment with a new GS-Former (Geo-aware Query Generator and Semantics-guided Geometry Sampler), and end-to-end visual instruction tuning. The model outputs formal program sequences that are executed by a symbolic solver. Experiments on GeoQA, UniGeo, Geometry3K, PGPS9K, and MathVista-GEO report state-of-the-art results, and ablations attribute the gains to the proposed formalized alignment and to the GS-Former components.

Significance. If the results are robust, the paper makes a useful contribution by demonstrating that formalized (program-sequence) vision-language pre-training can outperform both generalist MLLMs and task-specific geometry solvers, and by introducing an architecture (GS-Former) with content-aware query generation and sparsification. The paper includes helpful ablations (Tables 5 and 6) that isolate the contributions of geometry-language alignment and of the GQG/SGS modules, and it releases code and data, which supports reproducibility. The headline SOTA numbers, however, are credible only if the training/evaluation boundary is contamination-free; the paper does not yet establish this, and the comparison against generalist models is confounded by fine-tuning status.

major comments (3)
  1. [Appendix E / Sec. 4.2 / Table 2] The 100M-token geometric corpus is built from the training portions of GeoQA, UniGeo, Geometry3K, PGPS9K, PGDP5K, and G-LLaVA. Since UniGeo's calculation problems are exactly the 4,998 GeoQA problems, and since G-LLaVA was generated from GeoQA problems via equation solving, value scaling, and paraphrasing, test items from GeoQA and UniGeo may appear verbatim or near-verbatim in the pre-training corpus. The paper reports no overlap statistics, no cross-dataset deduplication procedure, and no release of the filtered corpus. The near-saturated UniGeo proving results in Table 2 (97.8%-100% top-1) are exactly the pattern one would expect from memorization of short formal proof programs. To support the central claim, the authors must quantify the overlap between the corpus and each test split (exact and near-duplicate matching) and re-run the experiments on a deduplicated corpus, reporting which numbers change.
  2. [Sec. 4.2 / Tables 1-3 / Appendix F] The headline 'significantly outperforms' claim compares GeoX, which is fine-tuned on each benchmark's training split (hyperparameters in Appendix F, Table 11), against generalist MLLMs (GPT-4V, Qwen-VL, LLaVA-v1.5, mPLUG-Owl2) that are evaluated zero-shot. This confounds model quality with fine-tuning advantage, and the paper reports no error bars or significance tests across random seeds. A fair evaluation would either fine-tune the generalist baselines on the same instruction data or evaluate GeoX in a zero-shot/few-shot setting. Without this, the magnitude of the reported gains (e.g., 54.9% vs 43.4% on GeoQA, 58.6% vs 34.8% Completion on Geometry3K) cannot be attributed to the proposed pre-training paradigm.
  3. [Sec. 4.2 / Table 3] The paper states that to make a fair comparison with specialists, it removes Diagram GT annotations and 'replicate[s] these methods under the original conditions,' but it provides no details of this replication (training data, hyperparameters, number of runs). Since the reproduced numbers for PGPSNet and Geoformer in Table 3 underpin the SOTA claim, the authors should either describe the replication protocol or report the original published numbers with a clear statement of differences.
minor comments (6)
  1. [Abstract / Secs. 1, 4] The benchmark name is written as 'PGPS9k' in the abstract and Section 1 but as 'PGPS9K' elsewhere; please unify.
  2. [Abstract / Appendix G] Two different code URLs are given: github.com/Alpha-Innovator/GeoX in the abstract and github.com/UniModal4Reasoning/GeoX in the appendix; please ensure the final version points to the correct repository.
  3. [Eq. (1)] The Gumbel-Softmax reparameterization is applied to a binary mask M, but the paper does not specify whether hard or soft samples are used during training and inference; this should be clarified because it affects the optimization and the interpretation of the sparsification term.
  4. [Sec. 3.3.2] GS-Former is initialized with BERT weights, but BERT is an encoder-only model while GS-Former includes cross-attention and is later used in a generation-oriented pipeline; a short explanation of how the initialization is applied and why it is appropriate would improve clarity.
  5. [Sec. 4.4 / Fig. 3] Figure 3 reports performance values but does not show numerical labels, error bars, or the definition of the y-axis; adding these would make the unimodal pre-training comparison more interpretable.
  6. [Table 5] The second row labels the Alignment column as '×' while the Language column is also '×'; this is ambiguous. Use explicit entries such as 'None' or 'No alignment' for readability.

Circularity Check

1 steps flagged · score 2.0 of 10

Empirical results are external benchmark numbers; only the Appendix A theoretical justification reduces to its own assumptions, so the paper is essentially non-circular (score 2).

  1. self definitional [Appendix A, Lemma 1 proof (arXiv v2 pp. 15-16)]
    "However, since Tinf only contains the semantic information without structural context that is required by the downstream tasks. Therefore, it holds that I(zinf , T) ≤ I(zsuf, T), ∀zf that is sufficient. That is, zinf is a minimal sufficient representation. As for zf , it learns information from the formalized description and thus is more related to the downstream tasks. Consequently, we have the relationship between zinf and zf as follows, I(zf , T) = I(zinf , T) + [I(Tf , T|zinf ) − I(Tf , T|zf )] ≥ I(zinf , T)."

    The lemma to be proved is I(zf,T) ≥ I(zinf,T), and the proof reaches that inequality by stipulating the same ordering. The statement that Tinf 'only contains the semantic information without structural context that is required by the downstream tasks' already asserts that informal representations carry less downstream-task information, and 'zf ... is more related to the downstream tasks' states the conclusion in different words. The mutual-information decomposition is then algebraic bookkeeping around an assumed information ordering; no independent bound or empirical input fixes the comparison. This is a post hoc theoretical gloss rather than a derivation used to produce the reported accuracies, so it is minor circularity, not a defect of the main experimental claim.

full rationale

The paper's central empirical assertion—that GeoX outperforms generalists and geometry specialists on GeoQA, UniGeo, Geometry3K, PGPS9K, and MathVista-GEO—rests on externally defined benchmark accuracies and a symbolic solver. Those numbers are not defined in terms of the model's own fitted parameters or of the method's equations, so they cannot be circular by construction. Tables 5 and 6 ablate the alignment objective and GS-Former modules against held-out benchmark splits, providing independent evidence for the attribution of the gains. Self-citations in the paper (Xia et al. 2023/2024, Li et al. 2023c, Peng et al. 2024, Chen et al. 2024a) are background or related-work citations and are not load-bearing for the method's correctness; they do not carry the derivation of the reported results. The only genuinely circular passage is Appendix A's Lemma 1, which purports to prove that formalized pre-training is informationally superior but effectively assumes the conclusion by declaring that informal targets lack structural context and that formalized representations are 'more related to the downstream tasks.' That theorem is presented as post hoc conceptual support and does not generate any experimental number, so it warrants a low score rather than a charge affecting the main results. The cross-dataset corpus overlap noted in Appendix E—where GeoQA, UniGeo, and G-LLaVA serve both as pre-training corpus sources and as evaluation benchmarks—is a possible contamination risk rather than a circularity: the paper states it extracted 'only the training portions' to prevent label leakage, and no equation reduces the reported accuracy to a fitted input. That concern is a correctness and reproducibility caveat, not definitional circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The system rests on several domain assumptions shared by many geometry solvers; the most fragile is dataset separation. No new physical or mathematical entities are introduced.

free parameters (3)
  • GS-Former query count K = 8
    Set in Section 4.1 without sensitivity analysis; affects alignment capacity.
  • Sparsification weight lambda = not reported
    In Eq. 2, lambda balances Lalign and Lspr; no value is given, so the exact objective is not reproducible.
  • Beam search size = 10
    Used at inference for all datasets, consistent with prior work, but it is a hand-chosen decoding parameter.
assumptions (5)
  • domain assumption Formalized descriptions are more effective than natural language for geometric pre-training
    Central to the method; argued in Appendix A with a non-rigorous mutual information proof, and supported only by the authors' internal ablations in Table 5.
  • domain assumption The 120K web and textbook diagrams used for Geo-ViT pre-training are representative of benchmark diagrams
    Section 3.2; no distribution analysis or domain shift measurement is provided.
  • domain assumption The 6232-image formalized caption dataset contains sufficient supervision to align geometry diagrams with formal language
    Section 3.3.1; the dataset is small and annotated by the authors' team, with no external validation of annotation quality.
  • domain assumption Benchmark test sets are uncontaminated by the pre-training corpus
    Appendix E states only that training portions were used; no deduplication across GeoQA, UniGeo, Geometry3K, and PGPS9K is reported, and GeoQA calculation questions overlap with UniGeo.
  • domain assumption Symbolic solver execution is a reliable verifier for generated programs
    Sections 3.1 and 4.5 assume the solver's success implies correct reasoning and its failure implies model error, which underlies the Appendix D adjusted metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training." pith.science (2026). https://pith.science/paper/MDQ6XBZT

@misc{pith2026241211863,
  author       = {Pith},
  title        = {Pith review of: GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MDQ6XBZT}},
  note         = {Machine review of arXiv:2412.11863}
}
read the original abstract

Despite their proficiency in general tasks, Multi-modal Large Language Models (MLLMs) struggle with automatic Geometry Problem Solving (GPS), which demands understanding diagrams, interpreting symbols, and performing complex reasoning. This limitation arises from their pre-training on natural images and texts, along with the lack of automated verification in the problem-solving process. Besides, current geometric specialists are limited by their task-specific designs, making them less effective for broader geometric problems. To this end, we present GeoX, a multi-modal large model focusing on geometric understanding and reasoning tasks. Given the significant differences between geometric diagram-symbol and natural image-text, we introduce unimodal pre-training to develop a diagram encoder and symbol decoder, enhancing the understanding of geometric images and corpora. Furthermore, we introduce geometry-language alignment, an effective pre-training paradigm that bridges the modality gap between unimodal geometric experts. We propose a Generator-And-Sampler Transformer (GS-Former) to generate discriminative queries and eliminate uninformative representations from unevenly distributed geometric signals. Finally, GeoX benefits from visual instruction tuning, empowering it to take geometric images and questions as input and generate verifiable solutions. Experiments show that GeoX outperforms both generalists and geometric specialists on publicly recognized benchmarks, such as GeoQA, UniGeo, Geometry3K, and PGPS9k.

Figures

Figures reproduced from arXiv: 2412.11863 by the authors.

Figure 1
Figure 1. Highlights of GeoX: 1) Comparison between GPT-4V ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of GeoX for training. We present a versatile method for automatic geometric [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Effectiveness of Uni-modal Pre-training. We compare the widely used CLIP-ViT-B and our [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Visualization results on four datasets by our GeoX. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Four visualized examples of geometric problem in natural images solved by our GeoX. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Attention map of GS-Former on different types of geometric diagrams. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Comparison of GPT-4o and GeoX in predicting formalized programs for solving complex [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A new 1,188-question multimodal benchmark covering deductive, inductive, and abductive reasoning shows that leading MLLMs score around 60% and are especially weak at abductive reasoning.

  2. Chimera: Improving Generalist Model with Domain-Specific Experts

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Chimera fuses frozen domain-expert encoders into a generalist multimodal LLM via routing and a 30% masking of general tokens, lifting InternVL2-8B from 61.6 to 64.9 on MathVista and from 31.3 to 32.4 on MathVerse.

  3. RedStar: Does Scaling Long-CoT Data Unlock Better Slow-Reasoning Systems?

    cs.LG 2025-01 reject novelty 5.0 of 10

    Long-CoT fine-tuning with QwQ-generated traces improves math and geometry benchmarks, and RedStar-DPO surpasses QwQ on AIME24, but several reported numbers conflict with the paper's own tables.

  4. Towards Geometry Problem Solving in the Large Model Era: A Survey

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A survey that organizes geometry problem-solving research into benchmark construction, parsing, and reasoning, and proposes a unified parse-then-reason paradigm for the large-model era.

Reference graph

Works this paper leans on

35 extracted references · 14 canonical work pages · cited by 4 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

  2. [3]

    Improving image captioning descriptiveness by ranking and llm-based fusion

    Simone Bianco, Luigi Celona, Marco Donzella, and Paolo Napoletano. Improving image captioning descriptiveness by ranking and llm-based fusion. arXiv preprint arXiv:2306.11593,

    arXiv

  3. [4]

    Language models are few-shot learners

    10 preprint Tom B Brown. Language models are few-shot learners. arXiv preprint ArXiv:2005.14165,

    arXiv 2005

  4. [7]

    Ll3da: Visual interactive instruction tuning for omni-3d understanding reasoning and planning

    Sijin Chen, Xin Chen, Chi Zhang, Mingsheng Li, Gang Yu, Hao Fei, Hongyuan Zhu, Jiayuan Fan, and Tao Chen. Ll3da: Visual interactive instruction tuning for omni-3d understanding reasoning and planning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26428–26438, 2024a. Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye,...

    arXiv

  5. [10]

    Mathprompter: Mathematical reasoning using large language models

    Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303.05398,

    arXiv

  6. [11]

    Categorical reparameterization with gumbel-softmax

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144,

    arXiv

  7. [13]

    Unimath: A foundational and multimodal mathematical reasoner

    Zhenwen Liang, Tianyu Yang, Jipeng Zhang, and Xiangliang Zhang. Unimath: A foundational and multimodal mathematical reasoner. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7126–7133,

    work page 2023

  8. [14]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In The Twelfth International Conference on Learning Representations. Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiao...

    arXiv

Show all 35 references
  1. [15]

    Geodrl: A self-learning framework for geometry problem solving using reinforcement learning in deductive reasoning

    Shuai Peng, Di Fu, Yijun Liang, Liangcai Gao, and Zhi Tang. Geodrl: A self-learning framework for geometry problem solving using reinforcement learning in deductive reasoning. In Findings of the Association for Computational Linguistics: ACL 2023, pp. 13468–13480,

    2023

  2. [16]

    Chimera: Improving generalist model with domain-specific experts

    Tianshuo Peng, Mingsheng Li, Hongbin Zhou, Renqiu Xia, Renrui Zhang, Lei Bai, Song Mao, Bin Wang, Conghui He, Aojun Zhou, et al. Chimera: Improving generalist model with domain-specific experts. arXiv preprint arXiv:2412.05983,

    arXiv

  3. [17]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    12 preprint Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contex...

    arXiv

  4. [18]

    Fusecap: Lever- aging large language models to fuse visual data into enriched image captions

    Noam Rotstein, David Bensaid, Shaked Brody, Roy Ganz, and Ron Kimmel. Fusecap: Lever- aging large language models to fuse visual data into enriched image captions. arXiv preprint arXiv:2305.17718,

    arXiv

  5. [19]

    Learning to solve geometry problems from natural language demon- strations in textbooks

    Mrinmaya Sachan and Eric Xing. Learning to solve geometry problems from natural language demon- strations in textbooks. In Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (* SEM 2017), pp. 251–261,

    2017

  6. [22]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805,

    arXiv

  7. [23]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a. Hugo Touvron, Lo...

    arXiv

  8. [24]

    Unimernet: A univer- sal network for real-world mathematical expression recognition

    Bin Wang, Zhuangcheng Gu, Chao Xu, Bo Zhang, Botian Shi, and Conghui He. Unimernet: A univer- sal network for real-world mathematical expression recognition. arXiv preprint arXiv:2404.15254, 2024a. Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, K...

    arXiv

  9. [25]

    Structchart: Perception, structuring, reasoning for visual chart understanding

    Renqiu Xia, Bo Zhang, Haoyang Peng, Ning Liao, Peng Ye, Botian Shi, Junchi Yan, and Yu Qiao. Structchart: Perception, structuring, reasoning for visual chart understanding. arXiv preprint arXiv:2309.11268,

    arXiv

  10. [26]

    Docgenome: An open large-scale scientific document benchmark for training and testing multi-modal large language models

    Renqiu Xia, Song Mao, Xiangchao Yan, Hongbin Zhou, Bo Zhang, Haoyang Peng, Jiahao Pi, Daocheng Fu, Wenjie Wu, Hancheng Ye, et al. Docgenome: An open large-scale scientific document benchmark for training and testing multi-modal large language models. arXiv preprint arXiv:2406....

  11. [27]

    Uni3d: A unified baseline for multi-dataset 3d object detection

    Bo Zhang, Jiakang Yuan, Botian Shi, Tao Chen, Yikang Li, and Yu Qiao. Uni3d: A unified baseline for multi-dataset 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9253–9262, 2023a. Ming-Liang Zhang, Zhong-Zhi Li, Fe...

    arXiv

  12. [28]

    A: Theoretical analysis on the proposed formalized vision-language pre-training

    14 preprint APPENDIX The appendix mainly includes the following aspects: • Sec. A: Theoretical analysis on the proposed formalized vision-language pre-training. • Sec. B: More visualization results. • Sec. C: Examples of formalized diagram-caption pairs. • Sec. D: Additional q...

    2022

  13. [29]

    to estimate the lowest achievable error of the classifier. According to the paper (Wang et al., 2022), for arbitrary representations z, its Bayes error rate Pe satisfies that, Pe ≤ 1 − exp[−H(T ) +I(z, T)], (5) where H(T ) represents the entropy of variable T . Since I(zf , T)...

    2023

  14. [30]

    Vicuna-1.5-7B 67.1 GeoX (Ours) Geo-LLM-7B 67.4 E D ATA ACQUISITION FOR GEOMETRIC CORPUS Data Sources. We detail the geometric corpus collections used to train Geo-LLM, sourced from a variety of publicly available geometric datasets, including GeoQA (Chen et al., 2021), GeoQA+(...

    2021

  15. [31]

    The proof problems are categorized into five sub-tasks (parallel, triangle, quadrangle, congruent, and similarity) with detailed reasoning and expressions

    introduces a comprehensive geometry dataset encompassing both calculation and proof problems, including 9,543 proving problems sourced from educational websites and 4,998 calculation problems from GeoQA (Chen et al., 2021). The proof problems are categorized into five sub-task...

    2021

  16. [32]

    Data Collection and Filtering

    using various strategies, including equation solving, value scaling, and sentence paraphrasing. Data Collection and Filtering. To meet the demands of pre-training for Geo-LLM, we build up a specialized filtering and pre-processing pipeline to construct the geometric corpus. In...

    2021

  17. [33]

    Implementation Details

    These include MLLMs such as mPLUG-Owl2 (Ye et al., 2023), Qwen-VL (Bai et al., 2023), LLaV A-v1.5 (Liu et al., 2024), GPT-4V (OpenAI, 2023), and GPT-4o (OpenAI, 2024). Implementation Details. After unified formal vision-language pre-training, we fine-tuned GeoX on each dataset...

    2023

  18. [34]

    gpt-4o-2024-05-13 Table 10: Model / API versions used for evaluation across different MLLMs. Instruction Tuning GeoQA UniGeo PGPS9K Geometry3K Training Batch Size 64 Scheduler Cosine Annealing Optimizer AdamW Warmup Ratio 0.05 0.05 0.05 0.03 Epochs 100 80 45 30 Learning Rate 3...

    2024

  19. [35]

    demonstrates the highest accuracy on MathVista-GEO. In this section, we delve deeper into the few-shot learning ability of GPT-4o’s in generating formalized program sequences, which are then sent to the GPS solver for solving (Chen et al., 2022). Specifically, we apply 2-shot ...

    2022

  20. [2013]

    G-llava: Solving geometric problem with multi-modal large language model

    Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, et al. G-llava: Solving geometric problem with multi-modal large language model. arXiv preprint arXiv:2312.11370,

    arXiv

  21. [2015]

    Math-llava: Bootstrapping mathematical reasoning for multimodal large language models

    Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See-Kiong Ng, Lidong Bing, and Roy Ka-Wei Lee. Math-llava: Bootstrapping mathematical reasoning for multimodal large language models. arXiv preprint arXiv:2406.17294,

    arXiv

  22. [2017]

    Solving geometry problems: Combining text and diagram interpretation

    Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. Solving geometry problems: Combining text and diagram interpretation. In Proceedings of the 2015 conference on empirical methods in natural language processing, pp. 1466–1476,

    2015

  23. [2019]

    Mimic-it: Multi-modal in-context instruction tuning

    11 preprint Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Mimic-it: Multi-modal in-context instruction tuning. arXiv preprint arXiv:2306.05425, 2023a. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootst...

    arXiv

  24. [2021]

    Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression

    Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen, and Xiaodan Liang. Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3313–3323,

    2022

  25. [2022]

    Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning

    Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 513–523,

    2021

  26. [2023]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966,

    arXiv

  27. [2024]

    Pgdp5k: A diagram parsing dataset for plane geometry problems

    Yihan Hao, Mingliang Zhang, Fei Yin, and Lin-Lin Huang. Pgdp5k: A diagram parsing dataset for plane geometry problems. In 2022 26th International Conference on Pattern Recognition (ICPR), pp. 1763–1769. IEEE,

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.