Pith. sign in

REVIEW 4 major objections 5 minor 59 references

E-Gen: Leveraging E-Graphs to Improve Continuous Representations of Symbolic Expressions

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper establishes that training on large e-graph-generated clusters of equivalent expressions yields embeddings that outperform a state-of-the-art LLM on clustering, mistake detection, and mathematical analogies.

desk verdict Solid incremental contribution; the equivalence-cluster generation is new and useful, but the unverified rewrite rules and missing error bars mean the headline numbers are upper bounds until validated. read the letter →

arxiv 2501.14951 v2 pith:R42P24MU submitted 2025-01-24 cs.LG cs.CLcs.SC

Hongbo Zheng , Suyuan Wang , Neeraj Gangwar , Nickvash Kani This is my paper · ORCID
classification cs.LGcs.CLcs.SC
keywords e-graphequalitysaturationmathematicallanguageprocessingexpressionembeddingscontrastivelearningsequence-to-sequenceequivalentexpressionssymbolicmathematics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

E-Gen is built on a simple idea: instead of simplifying a mathematical expression into one canonical form, an e-graph data structure rewrites it repeatedly until no new equivalent form appears, and the resulting equivalence classes are turned into a large synthetic corpus. On roughly 5,000 seed expressions, about 800 hand-written rewrite rules produce clusters that average 102 equivalent expressions each, yielding over 50 million training samples. Two embedding models—a sequence-to-sequence model that translates between equivalent expressions and a contrastive learner that pulls equivalent expressions together in vector space—are trained on these clusters. The paper reports that both outperform a prior math-specific embedding model and a state-of-the-art general LLM on clustering, semantic-understanding, mistake-detection, and analogy tasks. The authors conclude that the data-generation scheme, not the model architecture, is the main lever for learning mathematical semantics.

What carries the argument

The load-bearing object is the e-graph: a data structure that stores expressions as nodes grouped into e-classes of provably equivalent subterms, and that is repeatedly expanded by applying rewrite rules until no rule changes it (saturation). From the saturated e-graph the authors extract a context-free grammar whose productions enumerate many equivalent surface forms of each seed expression, and they sample from that grammar with a 25-token length limit. The e-graph replaces a computer-algebra simplifier as the source of training data: instead of one simplified output per expression, it produces a large cluster of equivalent rewrites, and those clusters are then used both as seq2seq input-output pairs and as positive and negative triplets for contrastive learning.

What would settle it

Independently verify a random sample of E-Gen clusters with a certified computer-algebra system, and re-run the k-means and mistake-detection evaluations on only the verifiably equivalent members; if accuracy falls substantially, or if a frontier LLM matches the embeddings once the rule set is corrected, the central claim fails. A simpler check: remove the token cap and see whether many additional valid rewrites appear—if they do, the current clusters are a biased sample of the equivalence closure.

Watch

Extended reading notes

Core claim

The central claim is that the expressiveness of continuous representations for symbolic mathematics is limited by the diversity of equivalent-expression training data, and that e-graph-based saturation removes that limit. E-Gen starts from template-instantiated prefix expressions, applies around 800 hand-written mathematical rewrite rules via an e-graph until saturation, then converts the saturated graph into a context-free grammar and enumerates equivalent rewrites subject to a 25-token cap. This produces an average cluster size of 102 versus 2 for the earlier pair-based dataset, and training sets of 55 million pairs and 50 million triplets. On held-out clusters, the trained embeddings group equivalent expressions with 96–97% k-means accuracy; on an out-of-distribution derivation-mistake task they reach about 78% F1 for mistake steps, beating the prior embedding model and a frontier LLM; and on 584 manually built mathematical analogies the best model reaches 70.38%. The authors take this as evidence that embeddings trained on a diverse equivalence corpus can outperform general LLMs on these mathematical language processing tasks.

Load-bearing premise

The entire pipeline assumes the roughly 800 hand-selected rewrite rules are mathematically valid and that e-graph saturation, together with the 25-token extraction cap, yields a representative sample of each equivalence class; if any rule is unsound or the cap omits deep rewrites, the clusters that train and test the embeddings are contaminated, and the reported gains would be inflated.

Editorial extensions

If this is right

  • Mathematical embeddings can be produced from synthetic equivalence data alone, so semantic representations are available for math-only documents without natural-language context.
  • The same generation pipeline can be specialized by changing the rule set or seed templates, producing embeddings tuned for specific branches of mathematics.
  • The vector-space algebra result implies that transformations compose as vector operations, which could be used to propose or check derivation steps automatically.
  • A similarity-threshold mistake detector gives a lightweight, model-based complement to symbolic verification for step-by-step derivations.
  • Because these vectors live in a common embedding space, they can be combined with text or vision embeddings for retrieval and reasoning over mixed documents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test whether the gains come from raw cluster size, operator diversity, or the specific rule set; a controlled ablation that varies only cluster size while holding rules fixed would identify the active ingredient.
  • The contrastive models lag the seq2seq model on embedding algebra, suggesting the InfoNCE objective optimizes equivalence discrimination but not compositionality; combining contrastive learning with a generative or algebraic-consistency loss may recover both.
  • The 25-token extraction cap and the timeout mean the grammar is sampled rather than fully enumerated, so the corpus likely under-represents long rewrites; replacing the recursive enumerator with a more efficient parser, as the authors note, is a concrete way to test whether deeper rewrites further improve semantics.
  • A natural stress test is real-world mathematical text: the tasks here are built from generated expressions or derivation steps produced by a symbolic algebra tool, so applying the embeddings to published derivations or math-information-retrieval benchmarks would reveal how well the synthetic rewrite distribution transfers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces E-Gen, an e-graph-based generator that produces clusters of mathematically equivalent symbolic expressions, together with a corpus containing about 5,000 seed expressions, 34 operators, and an average cluster size of 102. The authors train two families of transformer embeddings, a seq2seq model that generates equivalent rewrites and contrastive-learning models with mean or max pooling, and evaluate them on K-means clustering, semantic discrimination beyond syntax, mistake detection, formula retrieval, and analogical embedding algebra. The reported results show substantial gains over the prior SEMEMB model and over GPT-4o on several tasks, and the authors argue that the cluster-based E-Gen corpus is the main source of these gains.

Significance. If the reported results hold, the paper makes a useful and timely contribution: it addresses a known bottleneck in semantic embeddings for symbolic mathematics, namely the limited diversity of SymPy-generated equivalent-expression pairs, and it demonstrates that e-graph saturation can generate large equivalence clusters efficiently. The release of code and datasets is a concrete asset for the mathematical-language-processing community, and the comparison with a general-purpose LLM is informative. The significance is tempered, however, by three load-bearing validation issues: the soundness of the roughly 800 hand-written rewrite rules is not independently checked, the LLM comparison protocol uses iterative feedback that reveals ground-truth labels, and the quantitative comparisons are reported without error bars or significance tests. These issues affect the strength of the central comparative claims, but they appear addressable within the scope of the manuscript.

major comments (4)
  1. [Section 3.1, Figure 1, Tables 2 and 12] The correctness of the entire corpus, and therefore of the training and in-distribution test labels, depends on the soundness of the approximately 800 manually designed rewrite rules. The rules are not enumerated, and no independent verification is reported. Because e-graph saturation merges e-classes whenever a rule fires, a single unsound, conditional, or domain-restricted rule (for example, an identity requiring x >= 0, or a rule mishandling branch cuts or the d/dx operator) would silently group non-equivalent expressions. Those clusters supply the positive pairs and triplets in Section 3.2 and the ground-truth labels for K-means, semantic understanding, and formula retrieval in Tables 4, 6, and 15, so errors would be learned and then counted as successes. This is the load-bearing condition for the main advantage over SEMEMB. I ask the authors to publish the full rule set in a supplement and to verify each rule against an independent computer algebra system on random instances, reporting any counterexamples and their effect on cluster purity.
  2. [Section 5.2, Tables 10 and 11] The GPT-4o comparison is not symmetric with the fixed embedding models. The text states that GPT-4o is first given an example derivation and that, if it fails to detect the mistake, explicit feedback indicating the erroneous step is provided until it correctly understands the task. Because that feedback reveals ground-truth error labels, the protocol amounts to supervised adaptation of GPT-4o on the test task, while the embedding models are frozen. This biases the comparison and undermines the abstract claim that the embedding approach outperforms state-of-the-art LLMs. Please either use a fixed prompt with no label feedback, or report the number of feedback rounds and ablate the comparison without them.
  3. [Section 5.1, Tables 4, 6, 8, 10, and 15] All quantitative results are single-run point estimates with no error bars, confidence intervals, or significance tests. Since the seq2seq and contrastive models share the same data-generation pipeline, a single favorable or unfavorable seed could change the ranking of the three proposed models on the semantic-understanding task, where the reported gap between seq2seq and the contrastive models is large (76.41% versus 50.36% and 48.19%). Moreover, SEMEMB is evaluated only on distributions produced by E-Gen, which is the training distribution of the proposed models; the comparison would be stronger if both approaches were also evaluated on a common held-out set of manually verified equivalences, or if the limitation were stated explicitly. Please add repeated runs with standard errors, and ideally a cross-generator evaluation.
  4. [Section 5.1, Algorithm 1 and Table 8] The mistake-detection evaluation has a potential independence problem. The threshold in Algorithm 1 is calibrated on derivations generated from expressions in the E-Gen training set, while the test set is described as derivations generated for each expression in the E-Gen corpus. If the same expressions or derivation strings appear in both the training clusters and the test derivations, then the threshold and the test labels are not independent, and the reported out-of-distribution generalization is weakened. Please report the overlap between the training and test derivation sets, or construct the test set from expression families that never appear during training.
minor comments (5)
  1. [Table 8] The reported F1 of 89.14 for the CL Max model in the 'no mistake' row is inconsistent with its precision of 97.78 and recall of 91.93, which would give an F1 around 94.77; the value 89.14 appears to be copied from the mistake-class recall. Please correct the table.
  2. [Equation (1)] The InfoNCE-style loss is not typeset cleanly: the numerator and denominator are hard to read, and the expectation is written without specifying the distribution over x, x+, and x-. Please rewrite the equation and state the sampling distribution explicitly.
  3. [Appendix C.1, Equations (3) and (4)] The K-means accuracy calculation mentions mapping predicted cluster labels to ground-truth labels, but the formula uses 1{g(x_j)=c_i} directly, which assumes K-means outputs the ground-truth cluster identities. Please clarify how the label mapping is performed, for example with a Hungarian matching step.
  4. [Section 3.1] The counts '5,000 initial expressions' and 'around 800 mathematical rules' are not tied to the released repository. Please provide exact counts and point to the rule file and template file in the GitHub repository so readers can reproduce the corpus generation.
  5. [Table 3] The table would be easier to interpret if it reported the number of distinct initial expressions, the total number of expressions after extraction, and the distribution of cluster sizes, not just the average cluster size, since the average of 102 may hide a heavy-tailed distribution that affects the training pairs and triplets.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the empirical claims do not reduce to their inputs, though the in-distribution evaluations are self-referential to the E-Gen generator.

full rationale

The paper's central claims are empirical. E-Gen builds clusters by applying ~800 hand-written rewrite rules; training pairs/triplets and in-distribution test labels both come from those clusters. This is a self-referential benchmark setup, but it is not a circular derivation: the reported accuracies (e.g., 96.72% seq2seq vs. 37.70% SEMEMB in Table 4; F1 77.99% for mistake detection in Table 8) show the models can fail and are not forced by construction. Algorithm 1 calibrates the mistake-detection threshold on training derivations and then applies it to a test set; this is standard threshold fitting, not a fitted parameter renamed as a prediction, and the nonzero error rates confirm the classification is not statistically forced. The paper's only notable self-citation, "Our empirical analysis and prior work (Gangwar and Kani, 2023) indicate that max pooling consistently outperforms average pooling for the seq2seq model," is a minor architectural choice, not load-bearing for the main comparison, which also includes the re-trained SEMEMB baseline and GPT-4o. The skeptic's concern that the ~800 rewrite rules may be unsound is a correctness/validity risk about the labels, not a circularity of the derivation; even an unsound rule would still be learned and predicted empirically. Accordingly, no step of the claimed derivation is equivalent to its input by definition, and the score reflects only the mild self-referential evaluation design.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests on assumptions about the soundness and representativeness of the manually constructed rule set and templates, and on the equivalence semantics used for labels. No numeric constants are fitted into the model; the only hand-chosen numeric elements are standard hyperparameters (e.g., InfoNCE temperature, token length limit 25, 600s timeout), which do not constitute fitted parameters of a derivation.

assumptions (5)
  • domain assumption The 800 mathematical rewrite rules preserve expression equivalence.
    Section 3.1 says E-Gen applies around 800 mathematical rules; unsound rules would corrupt cluster labels and all downstream training and evaluation.
  • domain assumption E-graph saturation and grammar extraction yield a representative sample of each equivalence class.
    Section 3.1 imposes a 25-token length limit and 600s timeout; the sampled rewrites define the training distribution and the in-distribution test set.
  • domain assumption Hand-designed templates with 5,000 initial expressions cover the operator diversity needed for generalizable embeddings.
    Sections 3.1 and 3.2; no formal coverage guarantee, and the paper's limitations note the operator set is narrower than ARXMLiv and ARQMath.
  • domain assumption Algebraic equivalence under the rule set is the correct notion of semantic similarity for mathematical expressions.
    Section 1 treats equivalent expressions as semantically identical; branch cuts, domains, and context are ignored.
  • domain assumption SymPy-generated derivations are valid ground truth for the out-of-distribution mistake-detection test.
    Section 5.1 says the test set is generated using SymPy; this assumes the external tool's simplifications are correct.

how reviews work

0 comments
Cite this review

Pith. "Pith review of E-Gen: Leveraging E-Graphs to Improve Continuous Representations of Symbolic Expressions." pith.science (2026). https://pith.science/paper/R42P24MU

@misc{pith2026250114951,
  author       = {Pith},
  title        = {Pith review of: E-Gen: Leveraging E-Graphs to Improve Continuous Representations of Symbolic Expressions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R42P24MU}},
  note         = {Machine review of arXiv:2501.14951}
}
read the original abstract

Vector representations have been pivotal in advancing natural language processing (NLP), with prior research focusing on embedding techniques for mathematical expressions using mathematically equivalent formulations. While effective, these approaches are constrained by the size and diversity of training data. In this work, we address these limitations by introducing E-Gen, a novel e-graph-based dataset generation scheme that synthesizes large and diverse mathematical expression datasets, surpassing prior methods in size and operator variety. Leveraging this dataset, we train embedding models using two strategies: (1) generating mathematically equivalent expressions, and (2) contrastive learning to explicitly group equivalent expressions. We evaluate these embeddings on both in-distribution and out-of-distribution mathematical language processing tasks, comparing them against prior methods. Finally, we demonstrate that our embedding-based approach outperforms state-of-the-art large language models (LLMs) on several tasks, underscoring the necessity of optimizing embedding methods for the mathematical data modality. The source code and datasets are available at https://github.com/MLPgroup/E-Gen.

Figures

Figures reproduced from arXiv: 2501.14951 by the authors.

Figure 1
Figure 1. Illustration of e-graph saturation (1a to 1d) and grammar creation (1e). An e-graph consists of e-classes (dashed boxes) containing equivalent e-nodes (solid boxes). Edges connect e-nodes to their child e-classes. Applying mathematical rules to an e-graph adds new e-nodes and edges (1b and 1c), or merges e-classes (1d). Additions and modifications are emphasized in black. In 1e, the saturated e-graph is converted in… view at source ↗
Figure 2
Figure 2. Overview of E-Gen and the cluster-based training framework. Equivalent expressions of each initial [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of representation vectors for 17 different single-operator mathematical expressions and their [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Example of the formula retrieval with seq2seq [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Two examples of mistake detection conversations in the GPT-4o prompt. Errors in the derivation are [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 27 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Miltiadis Allamanis, Pankajan Chanthirasegaran, Pushmeet Kohli, and Charles Sutton. 2017. Learning continuous semantic representations of symbolic expressions. In International Conference on Machine Learning, pages 80--88. PMLR

    work page 2017

  5. [5]

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597--1607. PMLR

    2020

  6. [6]

    Kyunghyun Cho. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078

    arXiv 2014

  7. [7]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  8. [8]

    Jay Earley. 1970. An efficient context-free parsing algorithm. Communications of the ACM, 13(2):94--102

    work page 1970

Show all 59 references
  1. [9]

    Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Petersen, and Julius Berner. 2024. Mathematical capabilities of ChatGPT . Advances in neural information processing systems, 36

    2024

  2. [10]

    Neeraj Gangwar and Nickvash Kani. 2023. Semantic representations of mathematical expressions in a continuous vector space. Transactions on Machine Learning Research

    2023

  3. [11]

    Liangcai Gao, Zhuoren Jiang, Yue Yin, Ke Yuan, Zuoyu Yan, and Zhi Tang. 2017. Preliminary exploration of formula embedding for mathematical information retrieval: can mathematical formulae be embedded like a natural language? arXiv preprint arXiv:1707.05154

    2017 arXiv

  4. [12]

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE : Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821

    2021 arXiv

  5. [13]

    Mor Geva, Ankit Gupta, and Jonathan Berant. 2020. Injecting numerical reasoning skills into language models. arXiv preprint arXiv:2004.04487

    2020 arXiv

  6. [14]

    Emma Hamel, Hongbo Zheng, and Nickvash Kani. 2022. An evaluation of NLP methods to extract mathematical token descriptors. In International Conference on Intelligent Computer Mathematics, pages 329--343. Springer, Springer

    2022

  7. [15]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. GPT-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  8. [16]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. OpenAI o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  9. [17]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  10. [18]

    Hwiyeol Jo, Dongyeop Kang, Andrew Head, and Marti A Hearst. 2021. Modeling mathematical notation semantics in academic papers. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3102--3115

    2021

  11. [19]

    Michael Kohlhase et al. 2024. arxmliv project. https://kwarc.info/projects/arXMLiv/. Accessed: 2024-09-17

    2024

  12. [20]

    Giovanni Yoko Kristianto, Akiko Aizawa, et al. 2014. Extracting textual descriptions of mathematical expressions in scientific papers. D-Lib Magazine, 20(11):9

    2014

  13. [21]

    Giovanni Yoko Kristianto, Goran Topic, and Akiko Aizawa. 2016. MCAT math retrieval system for NTCIR-12 MathIR task. In NTCIR

    2016

  14. [22]

    Kriste Krstovski and David M Blei. 2018. Equation embeddings. arXiv preprint arXiv:1803.09123

    2018 arXiv

  15. [23]

    Guillaume Lample and Fran c ois Charton. 2019. Deep learning for symbolic mathematics. arXiv preprint arXiv:1912.01412

    2019 arXiv

  16. [24]

    Guillaume Lample, Timothee Lacroix, Marie-Anne Lachaux, Aurelien Rodriguez, Amaury Hayat, Thibaut Lavril, Gabriel Ebner, and Xavier Martinet. 2022. Hypertree proof search for neural theorem proving. Advances in neural information processing systems, 35:26337--26349

    2022

  17. [25]

    I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  18. [26]

    Ilya Loshchilov and Frank Hutter. 2016. SGDR : Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983

    2016 arXiv

  19. [27]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. MathVista : Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255

    2023 arXiv

  20. [28]

    J Macqueen. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability/University of California Press

    1967

  21. [29]

    Behrooz Mansouri, Anurag Agarwal, Douglas W Oard, and Richard Zanibbi. 2022 a . Advancing math-aware search: the ARQMath-3 lab at CLEF 2022. In European Conference on Information Retrieval, pages 408--415. Springer

    2022

  22. [30]

    Behrooz Mansouri, V \' t Novotn \`y , Anurag Agarwal, Douglas W Oard, and Richard Zanibbi. 2022 b . Overview of ARQMath-3 (2022): Third CLEF lab on answer retrieval for questions on math. In International Conference of the Cross-Language Evaluation Forum for European Languages...

    2022

  23. [31]

    Behrooz Mansouri, V \' t Novotn \`y , Anurag Agarwal, Douglas W Oard, and Richard Zanibbi. 2022 c . Third CLEF lab on answer retrieval for questions on math (working notes version. Proc. CLEF 2022 (CEUR Working Notes)

    2022

  24. [32]

    Behrooz Mansouri, Shaurya Rohatgi, Douglas W Oard, Jian Wu, C Lee Giles, and Richard Zanibbi. 2019. Tangent-CFT : An embedding model for mathematical formulas. In Proceedings of the 2019 ACM SIGIR international conference on theory of information retrieval, pages 11--18

    2019

  25. [33]

    Jordan Meadows and Andre Freitas. 2022. A survey in mathematical language processing. arXiv preprint arXiv:2205.15231

    2022 arXiv

  26. [34]

    Kazem Meidani, Parshin Shojaee, Chandan K Reddy, and Amir Barati Farimani. 2023. SNIP : Bridging mathematical symbolic and numeric realms with unified pre-training. arXiv preprint arXiv:2310.02227

    2023 arXiv

  27. [35]

    Smith, Mateusz Paprocki, Ond r ej C ert\' i k, Sergey B

    Aaron Meurer, Christopher P. Smith, Mateusz Paprocki, Ond r ej C ert\' i k, Sergey B. Kirpichev, Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K. Moore, Sartaj Singh, Thilina Rathnayake, Sean Vig, Brian E. Granger, Richard P. Muller, Francesco Bonazzi, Harsh Gupta, Shivam ...

    2017 doi

  28. [36]

    Tomas Mikolov. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781

    2013 arXiv

  29. [37]

    Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. 2021. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114

    2021 arXiv

  30. [38]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

    2018 arXiv

  31. [39]

    Robert Pagael and Moritz Schubotz. 2014. Mathematical language processing project. arXiv preprint arXiv:1407.0167

    2014 arXiv

  32. [40]

    Shuai Peng, Ke Yuan, Liangcai Gao, and Zhi Tang. 2021. MathBERT : A pre-trained model for mathematical formula understanding. arXiv preprint arXiv:2105.00377

    2021 arXiv

  33. [41]

    Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543

    2014

  34. [42]

    Nicholas Popovic, Walter Laurito, and Michael F \"a rber. 2022. AIFB-WebScience at SemEval -2022 task 12: Relation extraction first--using relation extraction to identify entities. arXiv preprint arXiv:2203.05325

    2022 arXiv

  35. [43]

    Shaurya Rohatgi, Wei Zhong, Richard Zanibbi, Jian Wu, and C Lee Giles. 2019. Query auto completion for math formula search. arXiv preprint arXiv:1912.04115

    2019 arXiv

  36. [44]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

    2013

  37. [45]

    I Sutskever. 2014. Sequence to sequence learning with neural networks. arXiv preprint arXiv:1409.3215

    2014 arXiv

  38. [46]

    Goran Topi \'c , Giovanni Yoko Kristianto, Minh-Quoc Nghiem, and Akiko Aizawa. 2013. The MCAT math retrieval system for NTCIR-10 math track. In Proceedings of 10th NTCIR Conference, Tokyo, Japan, pages 680--685

    2013

  39. [47]

    Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE . Journal of machine learning research, 9(11)

    2008

  40. [48]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  41. [49]

    Qingxiang Wang, Chad Brown, Cezary Kaliszyk, and Josef Urban. 2020. Exploration of neural machine translation in autoformalization of mathematics in mizar. In Proceedings of the 9th ACM SIGPLAN International Conference on Certified Programs and Proofs, pages 85--98

    2020

  42. [50]

    Sean Welleck, Jiacheng Liu, Ronan Le Bras, Hannaneh Hajishirzi, Yejin Choi, and Kyunghyun Cho. 2021 a . NaturalProofs : Mathematical theorem proving in natural language. arXiv preprint arXiv:2104.01112

    2021 arXiv

  43. [51]

    Sean Welleck, Jiacheng Liu, Jesse Michael Han, and Yejin Choi. 2021 b . Towards grounded natural language proof generation. In MathAI4Ed Workshop at NeurIPS

    2021

  44. [52]

    Max Willsey, Chandrakana Nandi, Yisu Remy Wang, Oliver Flatt, Zachary Tatlock, and Pavel Panchekha. 2021. Egg: Fast and extensible equality saturation. Proceedings of the ACM on Programming Languages, 5(POPL):1--29

    2021

  45. [53]

    Yuhuai Wu, Albert Qiaochu Jiang, Wenda Li, Markus Rabe, Charles Staats, Mateja Jamnik, and Christian Szegedy. 2022. Autoformalization with large language models. Advances in Neural Information Processing Systems, 35:32353--32368

    2022

  46. [54]

    Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. 2018. Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3733--3742

    2018

  47. [55]

    Ke Yuan, Dafang He, Zhuoren Jiang, Liangcai Gao, Zhi Tang, and C Lee Giles. 2020. Automatic generation of headlines for online math questions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9490--9497

    2020

  48. [56]

    Richard Zanibbi, Akiko Aizawa, Michael Kohlhase, Iadh Ounis, Goran Topic, and Kenny Davila. 2016 a . NTCIR-12 MathIR task overview. In NTCIR

    2016

  49. [57]

    Richard Zanibbi, Kenny Davila, Andrew Kane, and Frank Wm Tompa. 2016 b . Multi-stage math formula search: Using appearance-based similarity metrics at scale. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pag...

    2016

  50. [58]

    Richard Zanibbi, Behrooz Mansouri, and Anurag Agarwal. 2024. Mathematical information retrieval: Search and question answering. arXiv preprint arXiv:2408.11646

    2024 arXiv

  51. [59]

    Jiaru Zou, Qing Wang, Pratyush Thakur, and Nickvash Kani. 2024. STEM-PoM : Evaluating language models math-symbol reasoning in document parsing. In The 4th Workshop on Mathematical Reasoning and AI at NeurIPS'24

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.