Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Granite Embedding R2 Models

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper reports that a 149M-parameter encoder retriever achieves the top average retrieval score among similar-sized open models across six benchmark groups, while supporting 8,192-token contexts.

desk verdict Useful model release, but the headline SOTA-average claim is inflated by training on the evaluation benchmarks. read the letter →

arxiv 2508.21085 v1 pith:OKMICM2Z submitted 2025-08-26 cs.CL cs.IR

Parul Awasthy , Aashka Trivedi , Yulong Li , Meet Doshi , Riyaz Bhat , Vignesh P , Vishwajeet Kumar , Yushu Yang
show 12 more authors
Bhavani Iyer Abraham Daniels Rudra Murthy Ken Barker Martin Franz Madison Lee Todd Ward Salim Roukos David Cox Luis Lastras Jaydeep Sen Radu Florian
This is my paper
classification cs.CLcs.IR
keywords denseretrievaltextembeddingmodelsbi-encoderretrievercross-encoderrerankercontrastivelearningknowledgedistillationlong-contexttable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The Granite R2 report claims that a family of three English encoder models — two bi-encoder retrievers (149M and 47M parameters) and a 149M cross-encoder reranker — sets a new high-water mark for open, commercially usable retrieval models. The central claim, backed by a six-benchmark-group table, is that the 149M retriever scores the best average retrieval performance of any comparable open model while encoding documents 19-44 percent faster than leading competitors. The models extend context length sixteenfold over the prior Granite release, to 8,192 tokens, and are trained entirely on permissively licensed, governance-screened data. If the benchmark tables hold up, the paper establishes that a base-sized encoder can match or beat larger and more specialized models across text, code, tables, long documents, and multi-turn conversation — and that a 47M small model can nearly match it on long-context tasks.

What carries the argument

The load-bearing machinery is the encoder architecture plus the five-stage retriever training pipeline. The encoder follows the ModernBERT recipe — alternating global and local attention, rotary position embeddings, sequence packing, unpadded forward passes, and Flash Attention — which is what keeps the larger R2 models running at the same speed as their R1 predecessors. The central carrying mechanism is the tabular RetroMAE variant: a table is encoded under an M1 attention mask and a shallow decoder must predict masked tokens of the paired natural-language summary, forcing the encoder to map table structure and content into the same space as text. A global RoPE theta of 80,000, chosen by ab

What would settle it

Hold out the Table-IR and MT-RAG evaluation splits so that no training pair touches them, then recompute the six-group average: if the Table-IR score (78.5) or MT-RAG score (57.6) drops enough that granite-embedding-english-r2 no longer leads Table 2, the state-of-the-art claim is unsupported. A simpler check is to rerun Table 2 using only datasets the paper did not train on (LongEmbed, COIR, and the MTEB-v2 retrieval subset) and see whether the model ordering changes.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is a training recipe that turns a ModernBERT-style encoder into a generalist enterprise retriever without sacrificing speed or license permissiveness. The recipe has five stages: masked-autoencoder retrieval pretraining; a tabular step in which a shallow decoder reconstructs a table's natural-language summary rather than its cells, aligning structure with meaning; contrastive finetuning with in-batch negatives; score-distribution distillation from a Mistral-7B teacher; and a final conversational domain-adaptation stage. The paper reports this recipe yields the top average retrieval score among comparable open models in its six-benchmark table (59.5), l

Load-bearing premise

The headline accuracy claim assumes the benchmark comparison is fair, but the Table-IR and MT-RAG evaluations overlap with the model's own training data, so the overall average may overstate how well the model generalizes to unseen retrieval domains.

Editorial extensions

If this is right

  • The 149M retriever becomes a default for open, commercially licensed dense retrieval: table, code, and long-document scores all improve over the prior release without any loss in encoding speed.
  • The reranker raises BEIR NDCG@10 from 50.9 to 54.4 when applied to the small retriever's top-20, making retrieve-and-rerank pipelines practical at 8,192-token context.
  • The 47M small model reaches a 61.9 average on LongEmbed, within reach of models several times its size, so long-context workloads no longer require base-sized encoders.
  • Because the models are Apache-2.0 licensed and trained on cleared data, the reported accuracy gains transfer to commercial products without license-audit risk.
  • The 80K global RoPE theta outperforms the 160K default on both short and long retrieval tasks, a concrete tuning choice other long-context retrievers could adopt.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 80K RoPE-theta result suggests retrieval distillation favors a different position-frequency regime than the 160K default used for language modeling; teams adapting ModernBERT for retrieval might profitably re-ablating their own theta instead of inheriting the default.
  • The table-summary prediction trick is transferable: the same RetroMAE-style decoder-rewrites-the-summary formulation could apply to other structured artifacts — JSON documents, knowledge-graph triples, database rows — whenever a natural-language summary can be generated for them.
  • The small model's near-parity on long-context benchmarks invites a testable scaling hypothesis: the performance gap between the 47M and 149M retrievers may shrink as context grows, in which case long-context RAG pipelines could prefer the small encoder for cost reasons.
  • The reranker comparison itself concedes that the closest competitor was trained on part of the evaluation data while the Granite reranker was not; a strictly held-out benchmark suite, with every model barred from its own training data, would likely reshuffle the reranker ordering and remains an open test for the field.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Granite Embedding R2, a family of English encoder-based retrieval models: a 149M bi-encoder, a 47M small bi-encoder, and a 149M cross-encoder reranker, all built on ModernBERT-style architectures with 8192-token context. The training pipeline includes RetroMAE pretraining, a novel tabular RetroMAE extension, contrastive finetuning, and knowledge distillation from a Mistral teacher. The central empirical claim is that the R2 retriever achieves state-of-the-art average retrieval performance among similarly sized open models across text, code, long-document, tabular, and multi-turn conversational benchmarks, with sizeable speed advantages. Evaluation is reported on MTEB-v2 retrieval, BEIR, COIR, MLDR, LongEmbed, Table-IR, MT-RAG, and reranking on BEIR/MLDR/Miracl.

Significance. If the performance claims held, this would be a useful contribution: the models are released under Apache 2.0, the training recipes and hyperparameters are unusually detailed, and the small-model architecture ablations and RoPE-theta scaling study are valuable to practitioners. The paper also foregrounds data governance, which is appropriate for enterprise deployment. However, the headline claims are weakened by a documented overlap between training data and two of the six benchmark groups that drive the average in Table 2, and the reranker is not state of the art by the paper's own Table 5. The underlying model release and many of the uncontaminated results (e.g., LongEmbed) remain valuable, but the empirical claims need to be recalibrated.

major comments (3)
  1. [§3.1 vs §5.1, Table 2] The aggregate state-of-the-art claim rests on benchmarks whose evaluation corpora were used in training. Section 3.1 lists OpenWikiTables, NQTables, OTT-QA, and MultiHierTT as training data for Table-IR, and states that synthetic multi-turn conversations were generated for the ClapNQ and IBM Cloud corpora of MT-RAG. Section 5.1 nonetheless describes all tasks as zero-shot except NQ, Hotpot, FEVER. Excluding Table-IR and MT-RAG from the Table 2 average gives 55.15 for granite-embedding-english-r2 versus 57.93 for gte-modernbert-base; r2 then leads only LongEmbed. The paper must either exclude contaminated benchmark groups from the headline average or demonstrate that the evaluation splits are disjoint from the training/synthetic-generation corpora.
  2. [Abstract and Table 5] The abstract claims state-of-the-art performance 'across diverse retrieval domains' and 'new performance standards,' but the reranker comparison in Table 5 shows granite-embedding-reranker-english-r2 below gte-reranker-modernbert-base on BEIR (55.4 vs 56.1), MLDR (44.4 vs 50.4), and Miracl (54.5 vs 54.8) when reranking the same top-20 retrievals. Section 5.3 concedes this in text, so the abstract is misleading as written. The abstract should be qualified to name the baselines it actually outperforms.
  3. [§3.2, Appendix E, Table 14] The global RoPE theta was selected using downstream retrieval performance on MTEB-v1, COIR, and MLDR, two of which (COIR and MLDR) appear in Table 2. This is not training-data contamination, but it means those benchmark numbers are not fully zero-shot with respect to model configuration. The claim that tasks are zero-shot should be scoped, or the theta ablation should be repeated on a held-out benchmark that does not inform the decision.
minor comments (5)
  1. [§5.1] Typo: 'granite-embedding-enlish-r2' should be 'granite-embedding-english-r2'.
  2. [§3.1] Typo: 'conversationdal' should be 'conversational'.
  3. [Table 4] The 'Rel to Granite' column for gte-modernbert-base appears inconsistent with the reported docs/s values (88 vs 144); the relative difference should be approximately -38.9%, not -1.4%. Please verify the computation or the reported speed.
  4. [Abstract and Table 4] The abstract's '19-44%' speed advantage is not straightforwardly derivable from Table 4. For example, granite-embedding-english-r2 is about 64% faster than gte-modernbert-base and about 24% faster than bge-base-en-v1.5. Specify which competitor pairs the range refers to.
  5. [Table 2] The 'Avg. Retrieval' column averages heterogeneous metrics (NDCG@10, Recall@5, Match@5, Accuracy@1) across benchmark groups. Reporting per-metric averages and confidence intervals would make the comparison more interpretable.

Circularity Check

1 steps flagged · score 6.0 of 10

SOTA average is partly an in-domain fit: Table-IR and MT-RAG evaluation data appear in the training recipe, contradicting the §5.1 zero-shot claim.

  1. fitted input called prediction [§3.1 (Table-IR Data; Multi-Turn Conversational IR Data) vs §5.1 zero-shot claim and Table 2]
    "We use several publicly available training datasets, including OpenWikiTables, NQTables, OTT-QA (Chen et al., 2021a), FinQA (Chen et al., 2021b), and MultiHierTT (Zhao et al., 2022) for training granite-embedding-english-r2 for Table-IR tasks. ... We also synthetically generate about 2000 multi-turn conversations using Mixtral8x22B for the ClapNQ and IBM Cloud corpora of the MT-RAG dataset (Katsis et al., 2025b). ... As shown in Table 2, Granite embedding R2 models show a strong performance across diverse tasks despite all tasks being zero-shot except for NQ, Hotpot, FEVER."

    The Table 2 average that supports the SOTA claim includes Table-IR (78.5) and MT-RAG (57.6), but §3.1 names the Table-IR evaluation datasets (OpenWikiTables, NQTables, OTT-QA, MultiHierTT) as training data and uses the MT-RAG ClapNQ/IBM Cloud corpora to generate synthetic training conversations. The §5.1 statement that 'all tasks are zero-shot except for NQ, Hotpot, FEVER' is false for these two groups; the scores are in-domain, not out-of-sample predictions. The aggregate lead is carried by this overlap: dropping Table-IR and MT-RAG from Table 2 gives r2 (56.4+54.8+41.6+67.8)/4=55.15 versus gte-modernbert-base (57.0+71.5+46.2+57.0)/4=57.93, so the claimed state-of-the-art average disappears when the trained-on benchmarks are excluded.

full rationale

The main load-bearing circularity is the evaluation/training overlap for Table-IR and MT-RAG. The paper is otherwise a straightforward empirical model release: architecture follows ModernBERT, benchmarks are external apart from the noted exceptions, and no mathematical derivation or uniqueness theorem is invoked. The RoPE theta selection in Appendix E is a secondary form of model-selection leakage (theta 80K chosen on MLDR/CoIR and then reported in Table 2), which further reduces independence of those specific numbers but is not the main driver of the average claim. The central SOTA-average claim therefore merits a 6: it is partially circular because two of the six benchmark groups contributing to the headline number were used, by the paper's own account, to construct the model.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. It relies on standard machine-learning assumptions about pretraining, distillation, and benchmark validity. The main free parameters are training hyperparameters, several of which are not reported (loss weights, distillation temperature) or are selected on the evaluation benchmarks themselves (RoPE theta).

free parameters (5)
  • Global RoPE theta = 80,000
    Chosen via ablation in Appendix E on MTEB-v1, CoIR, and MLDR. The 80k value is selected partly by MLDR performance, which is also an evaluation benchmark in Table 2.
  • RetroMAE masking ratios = 20% encoder, 60% decoder
    Set for the tabular pretraining stage in Section 3.2; no ablation is reported, chosen by hand.
  • Contrastive loss coefficients alpha, beta, gamma = not stated
    Defined in Section 3.2 Stage 3 but values are not reported. These weights directly shape the contrastive objective.
  • Distillation temperature tau_KD = not stated
    Used in the knowledge distillation loss in Section 3.2 Stage 4 and Section 3.3; value not reported.
  • Number of hard negatives = 3
    Used in contrastive distillation (Section 3.2 Stage 4); chosen without a reported ablation.
assumptions (4)
  • domain assumption Masked language modeling followed by contrastive finetuning produces effective retrieval embeddings
    Standard recipe in the field, inherited from prior work (RetroMAE, Li et al., 2023); invoked throughout Sections 2.3 and 3.2.
  • domain assumption Knowledge distillation from a Mistral-7B teacher improves student embedding quality
    Invoked in Section 3.2 Stage 4 and Section 3.3. Depends on the teacher being well-calibrated; no calibration analysis is provided.
  • domain assumption The evaluation benchmarks are valid proxies for enterprise retrieval quality
    The paper uses MTEB, BEIR, COIR, MLDR, LongEmbed, Table-IR, and MT-RAG as evidence for enterprise value. This is an external validity assumption.
  • domain assumption The M1 attention mask (Mouravieff et al., 2025) is appropriate for table encoding
    Used in the tabular pretraining objective in Section 3.2 Stage 2, with no comparison to other masking schemes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Granite Embedding R2 Models." pith.science (2026). https://pith.science/paper/OKMICM2Z

@misc{pith2026250821085,
  author       = {Pith},
  title        = {Pith review of: Granite Embedding R2 Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OKMICM2Z}},
  note         = {Machine review of arXiv:2508.21085}
}
read the original abstract

We introduce the Granite Embedding R2 models, a comprehensive family of high-performance English encoder-based embedding models engineered for enterprise-scale dense retrieval applications. Building upon our first-generation release, these models deliver substantial improvements, including 16x expanded context length (8,192 tokens), state-of-the-art performance across diverse retrieval domains - text, code, long-document search, multi-turn conversational, and tabular data - and measurable speed advantages of 19-44\% over leading competitors while maintaining superior accuracy. Our release encompasses both bi-encoder and cross-encoder architectures, featuring a highly effective 22-layer retriever model and its efficient 12-layer counterpart, alongside a high-quality reranker model, all trained exclusively on enterprise-appropriate data with comprehensive governance oversight. The models demonstrate exceptional versatility across standard benchmarks, IBM-developed evaluation suites, and real-world enterprise use cases, establishing new performance standards for open-source embedding models. In an era where retrieval speed and accuracy are paramount for competitive advantage, the Granite R2 models deliver a compelling combination of cutting-edge performance, enterprise-ready licensing, and transparent data provenance that organizations require for mission-critical deployments. All models are publicly available under the Apache 2.0 license at https://huggingface.co/collections/ibm-granite, enabling unrestricted research and commercial use.

Figures

Figures reproduced from arXiv: 2508.21085 by the authors.

Figure 1
Figure 1. Performance of Granite R2 embedding models and comparable size open-source models, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders

    cs.IR 2026-07 conditional novelty 7.0 of 10

    Bekko a8m, with 7.7M active parameters, scores 56.2 on MMTEB Multilingual v2 Retrieval, beating mE5 models and BGE-M3, while a25m reaches 57.5, on par with gte-multilingual-base.

  2. TEmBed-T: A Multi-Dimensional Benchmark for Table-Level Embeddings

    cs.DB 2026-07 conditional novelty 5.0 of 10

    No single table-level embedding model leads across retrieval, structural shuffling, and header-free type detection; quality is multi-dimensional.

Reference graph

Works this paper leans on

71 extracted references · 26 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Top2vec: Distributed representations of topics

    Dimo Angelov. Top2vec: Distributed representations of topics. CoRR, abs/2008.09470, 2020. URL https://arxiv.org/abs/2008.09470

    arXiv 2008

  3. [3]

    Granite embedding models, 2025

    Parul Awasthy, Aashka Trivedi, Yulong Li, Mihaela Bornea, David Cox, Abraham Daniels, Martin Franz, Gabe Goodhart, Bhavani Iyer, Vishwajeet Kumar, Luis Lastras, Scott McCarley, Rudra Murthy, Vignesh P, Sara Rosenthal, Salim Roukos, Jaydeep Sen, Sukriti Sharma, Avirup Sil, Kate Soule, Arafat Sultan, and Radu Florian. Granite embedding models, 2025. URL htt...

    arXiv 2025

  4. [4]

    Ms marco: A human generated machine reading comprehension dataset, 2018

    Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. Ms marco: A human generated machine reading comprehension dataset, 2018. URL https://arxiv.org/abs/1611.09268

    arXiv 2018

  5. [5]

    Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2024

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2024

    2024

  6. [6]

    Extending context window of large language models via positional interpolation

    Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. CoRR, abs/2306.15595, 2023. doi:10.48550/ARXIV.2306.15595. URL https://doi.org/10.48550/arXiv.2306.15595

    arXiv doi:10.48550/arxiv.2306.15595 2023

  7. [7]

    Wenhu Chen, Ming - Wei Chang, Eva Schlinger, William Yang Wang, and William W. Cohen. Open question answering over tables and text. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021 a . URL https://openreview.net/forum?id=MmCRswl1UYl

    work page 2021

  8. [8]

    Routledge, and William Yang Wang

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting - Hao Kenneth Huang, Bryan R. Routledge, and William Yang Wang. Finqa: A dataset of numerical reasoning over financial data. In Marie - Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen - tau Yih (eds.), Proceedings of the 2021 Confer...

    2021

Show all 71 references
  1. [9]

    Flashattention-2: Faster attention with better parallelism and work partitioning, 2023

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning, 2023

    2023

  2. [10]

    BERT : Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT : Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North. Association for Computational Linguistics, 2019. doi:10.18653/v1/n19-1423. U...

    2019 doi

  3. [11]

    Ugur Guney, Volkan Cirik, and Kyunghyun Cho

    Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Guney, Volkan Cirik, and Kyunghyun Cho. Searchqa: A new q&a dataset augmented with context from a search engine, 2017

    2017

  4. [12]

    Mmteb: Massive multilingual text embedding benchmark

    Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonatha...

    2025 doi

  5. [13]

    M ulti D oc2 D ial: Modeling dialogues grounded in multiple documents

    Song Feng, Siva Sankalp Patel, Hui Wan, and Sachindra Joshi. M ulti D oc2 D ial: Modeling dialogues grounded in multiple documents. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in N...

    2021 doi

  6. [14]

    S im CSE : Simple contrastive learning of sentence embeddings

    Tianyu Gao, Xingcheng Yao, and Danqi Chen. S im CSE : Simple contrastive learning of sentence embeddings. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing...

    2021 doi

  7. [15]

    Gneissweb: Preparing high quality data for llms at scale, 2025

    Hajar Emami Gohari, Swanand Ravindra Kadhe, Syed Yousaf Shah, Constantin Adam, Abdulhamid Adebayo, Praneet Adusumilli, Farhan Ahmed, Nathalie Baracaldo Angel, Santosh Subhashrao Borse, Yuan-Chi Chang, Xuan-Hong Dang, Nirmit Desai, Revital Eres, Ran Iwamoto, Alexei Karve, Yan K...

    2025

  8. [16]

    Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack...

    2024

  9. [17]

    Open domain question answering over tables via dense retrieval

    Jonathan Herzig, Thomas M \" u ller, Syrine Krichene, and Julian Martin Eisenschlos. Open domain question answering over tables via dense retrieval. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani - T \" u r, Iz Beltagy, Steven Bethard, Ryan Cotterell, T...

    2021

  10. [18]

    Distilling the Knowledge in a Neural Network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the Knowledge in a Neural Network . In NeurIPS Deep Learning Worksop, 2014

    2014

  11. [19]

    Mini CPM : Unveiling the potential of small language models with scalable training strategies

    Shengding Hu, Yuge Tu, Xu Han, Ganqu Cui, Chaoqun He, Weilin Zhao, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Xinrong Zhang, Zhen Leng Thai, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, dahai li, Zhiyuan Liu,...

    2024

  12. [20]

    Gittables: A large-scale corpus of relational tables

    Madelon Hulsebos, C agatay Demiralp, and Paul Groth. Gittables: A large-scale corpus of relational tables. Proc. ACM Manag. Data , 1 0 (1): 0 30:1--30:17, 2023. doi:10.1145/3588710. URL https://doi.org/10.1145/3588710

    2023 doi

  13. [21]

    Codesearchnet challenge: Evaluating the state of semantic code search, 2019

    Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. Codesearchnet challenge: Evaluating the state of semantic code search, 2019

    2019

  14. [22]

    TARGET : Benchmarking table retrieval for generative tasks

    Xingyu Ji, Aditya Parameswaran, and Madelon Hulsebos. TARGET : Benchmarking table retrieval for generative tasks. In NeurIPS 2024 Third Table Representation Learning Workshop, 2024. URL https://openreview.net/forum?id=gGGvnjFUfL

    2024

  15. [23]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L \' e lio Renard Lavaud, Marie - Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, ...

    2023 doi arXiv

  16. [24]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  17. [25]

    AIT-QA : Q uestion answering dataset over complex tables in the airline industry

    Yannis Katsis, Saneem Chemmengath, Vishwajeet Kumar, Samarth Bharadwaj, Mustafa Canim, Michael Glass, Alfio Gliozzo, Feifei Pan, Jaydeep Sen, Karthik Sankaranarayanan, and Soumen Chakrabarti. AIT-QA : Q uestion answering dataset over complex tables in the airline industry. In ...

    2022

  18. [26]

    Mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems, 2025 a

    Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. Mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems, 2025 a

    2025

  19. [27]

    Chulaka Gunasekara, Young - Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky

    Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, R. Chulaka Gunasekara, Young - Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. MTRAG: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. CoRR, abs/25...

    2025 doi arXiv

  20. [28]

    Open- W iki T able : Dataset for open domain question answering with complex reasoning over table

    Sunjun Kweon, Yeonsu Kwon, Seonhee Cho, Yohan Jo, and Edward Choi. Open- W iki T able : Dataset for open domain question answering with complex reasoning over table. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Li...

    2023 doi

  21. [29]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019

  22. [30]

    Position-aware listmle: a sequential learning process for ranking

    Yanyan Lan, Yadong Zhu, Jiafeng Guo, Shuzi Niu, and Xueqi Cheng. Position-aware listmle: a sequential learning process for ranking. In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, UAI'14, pp.\ 449–458, Arlington, Virginia, USA, 2014. AUAI ...

    2014

  23. [31]

    Nv-embed: Improved techniques for training llms as generalist embedding models, 2024

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Nv-embed: Improved techniques for training llms as generalist embedding models, 2024

    2024

  24. [32]

    Latent retrieval for weakly supervised open domain question answering

    Kenton Lee, Ming - Wei Chang, and Kristina Toutanova. Latent retrieval for weakly supervised open domain question answering. In Anna Korhonen, David R. Traum, and Llu \' s M \` a rquez (eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics,...

    2019 doi

  25. [33]

    Coir: A comprehensive benchmark for code information retrieval models, 2024

    Xiangyang Li, Kuicai Dong, Yi Quan Lee, Wei Xia, Yichun Yin, Hao Zhang, Yong Liu, Yasheng Wang, and Ruiming Tang. Coir: A comprehensive benchmark for code information retrieval models, 2024. URL https://arxiv.org/abs/2407.02883

    2024 arXiv

  26. [34]

    Towards general text embeddings with multi-stage contrastive learning, 2023

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning, 2023. URL https://arxiv.org/abs/2308.03281

    2023 arXiv

  27. [35]

    Roberta: A robustly optimized bert pretraining approach

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. URL https://arxiv.org/abs/1907.11692

    1907 arXiv

  28. [36]

    Arctic-embed: Scalable, efficient, and accurate text embedding models, 2024

    Luke Merrick, Danmei Xu, Gaurav Nuti, and Daniel Campos. Arctic-embed: Scalable, efficient, and accurate text embedding models, 2024

    2024

  29. [37]

    Cox, Ruchir Puri, and Rameswar Panda

    Mayank Mishra, Matt Stallone, Gaoyuan Zhang, Yikang Shen, Aditya Prasad, Adriana Meza Soria, Michele Merler, Parameswaran Selvam, Saptha Surendran, Shivdeep Singh, Manish Sethi, Xuan-Hong Dang, Pengyuan Li, Kun-Lung Wu, Syed Zawad, Andrew Coleman, Matthew White, Mark Lewis, Ra...

    2024

  30. [38]

    Structural deep encoding for table question answering

    Rapha \" e l Mouravieff, Benjamin Piwowarski, and Sylvain Lamprier. Structural deep encoding for table question answering. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics, ACL 2025...

    2025

  31. [39]

    Structural deep encoding for table question answering

    Rapha \"e l Mouravieff, Benjamin Piwowarski, and Sylvain Lamprier. Structural deep encoding for table question answering. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025,...

    2025 doi

  32. [40]

    Mteb: Massive text embedding benchmark

    Niklas Muennighoff, Nouamane Tazi, Lo \" c Magne, and Nils Reimers. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316, 2022. doi:10.48550/ARXIV.2210.07316. URL https://arxiv.org/abs/2210.07316

    2022 doi arXiv

  33. [41]

    Text and code embeddings by contrastive pre-training, 2022

    Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski S...

    2022 arXiv

  34. [42]

    Morris, Brandon Duderstadt, and Andriy Mulyar

    Zach Nussbaum, John X. Morris, Brandon Duderstadt, and Andriy Mulyar. Nomic embed: Training a reproducible long context text embedder, 2024

    2024

  35. [43]

    Sentence- BERT : Sentence embeddings using S iamese BERT -networks

    Nils Reimers and Iryna Gurevych. Sentence- BERT : Sentence embeddings using S iamese BERT -networks. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Internationa...

    2019 doi

  36. [44]

    Pubtables-1m: Towards comprehensive table extraction from unstructured documents

    Brandon Smock, Rohith Pesala, and Robin Abraham. Pubtables-1m: Towards comprehensive table extraction from unstructured documents. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022 , pp.\ 4624--4632. IEEE , 202...

    2022

  37. [45]

    Arxiv-tables: Tables extracted from arxiv scientific papers

    staghado . Arxiv-tables: Tables extracted from arxiv scientific papers. Hugging Face Dataset, ://huggingface.co/datasets/staghado/ArXiv-tables, August 2025. Includes LaTeX source and rendered images of tables; License: CC-BY-4.0; Size: 1K–10K instances

    2025

  38. [46]

    How to fine-tune bert for text classification? In China national conference on Chinese computational linguistics, pp.\ 194--206

    Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. How to fine-tune bert for text classification? In China national conference on Chinese computational linguistics, pp.\ 194--206. Springer, 2019

    2019

  39. [47]

    BEIR : A heterogeneous benchmark for zero-shot evaluation of information retrieval models

    Nandan Thakur, Nils Reimers, Andreas R \"u ckl \'e , Abhishek Srivastava, and Iryna Gurevych. BEIR : A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks ...

    2021

  40. [48]

    Davison, and Jeff Heflin

    Mohamed Trabelsi, Zhiyu Chen, Shuo Zhang, Brian D. Davison, and Jeff Heflin. Strubert: Structure-aware bert for table search and matching. In Proceedings of the ACM Web Conference 2022, WWW '22, pp.\ 442–451, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9...

    2022

  41. [49]

    GLUE : A multi-task benchmark and analysis platform for natural language understanding

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE : A multi-task benchmark and analysis platform for natural language understanding. In Tal Linzen, Grzegorz Chrupa a, and Afra Alishahi (eds.), Proceedings of the 2018 EMNLP Workshop B la...

    2018 doi

  42. [50]

    Text embeddings by weakly-supervised contrastive pre-training

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533, 2022

    2022 arXiv

  43. [51]

    Improving text embeddings with large language models, 2023

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Improving text embeddings with large language models, 2023

    2023

  44. [52]

    Improving text embeddings with large language models, 2024

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Improving text embeddings with large language models, 2024. URL https://arxiv.org/abs/2401.00368

    2024 arXiv

  45. [53]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers, 2020

    Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers, 2020

    2020

  46. [54]

    Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference, 2024

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. Smarter, better, faster, longer: A modern bidirectional e...

    2024

  47. [55]

    Stable and low-precision training for large-scale vision-language models, 2023

    Mitchell Wortsman, Tim Dettmers, Luke Zettlemoyer, Ari Morcos, Ali Farhadi, and Ludwig Schmidt. Stable and low-precision training for large-scale vision-language models, 2023

    2023

  48. [56]

    Listwise approach to learning to rank: theory and algorithm

    Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. Listwise approach to learning to rank: theory and algorithm. In Proceedings of the 25th International Conference on Machine Learning, ICML '08, pp.\ 1192–1199, New York, NY, USA, 2008. Association for Computing Machi...

    2008

  49. [57]

    R etro MAE : Pre-training retrieval-oriented language models via masked auto-encoder

    Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. R etro MAE : Pre-training retrieval-oriented language models via masked auto-encoder. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Proc...

    2022 doi

  50. [58]

    C-pack: Packaged resources to advance general chinese embedding, 2023

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. C-pack: Packaged resources to advance general chinese embedding, 2023

    2023

  51. [59]

    Approximate nearest neighbor negative contrastive learning for dense text retrieval, 2020

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval, 2020. URL https://arxiv.org/abs/2007.00808

    2020 arXiv

  52. [60]

    Learning to mine aligned code and natural language pairs from stack overflow

    Pengcheng Yin, Bowen Deng, Edgar Chen, Bogdan Vasilescu, and Graham Neubig. Learning to mine aligned code and natural language pairs from stack overflow. In 2018 IEEE/ACM 15th international conference on mining software repositories (MSR), pp.\ 476--486. IEEE, 2018

    2018

  53. [61]

    Tabert: Pretraining for joint understanding of textual and tabular data

    Pengcheng Yin, Graham Neubig, Wen - tau Yih, and Sebastian Riedel. Tabert: Pretraining for joint understanding of textual and tabular data. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association fo...

    2020 doi

  54. [62]

    Arctic-embed 2.0: Multilingual retrieval without compromise, 2024

    Puxuan Yu, Luke Merrick, Gaurav Nuti, and Daniel Campos. Arctic-embed 2.0: Multilingual retrieval without compromise, 2024

    2024

  55. [63]

    Bruce Croft, Erik Learned-Miller, and Jaap Kamps

    Hamed Zamani, Mostafa Dehghani, W. Bruce Croft, Erik Learned-Miller, and Jaap Kamps. From neural re-ranking to neural ranking: Learning a sparse representation for inverted indexing. In Proceedings of the 27th ACM International Conference on Information and Knowledge Managemen...

    2018

  56. [64]

    mgte: Generalized long-context text representation and reranking models for multilingual text retrieval, 2024

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, and Min Zhang. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval, 2024. URL ...

    2024 arXiv

  57. [65]

    Sparta: Efficient open-domain question answering via sparse transformer matching retrieval, 2020

    Tiancheng Zhao, Xiaopeng Lu, and Kyusong Lee. Sparta: Efficient open-domain question answering via sparse transformer matching retrieval, 2020. URL https://arxiv.org/abs/2009.13013

    2020 arXiv

  58. [66]

    M ulti H iertt: Numerical reasoning over multi hierarchical tabular and textual data

    Yilun Zhao, Yunxiang Li, Chenying Li, and Rui Zhang. M ulti H iertt: Numerical reasoning over multi hierarchical tabular and textual data. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computat...

    2022 doi

  59. [67]

    Global table extractor (GTE): A framework for joint table identification and cell structure recognition using visual context

    Xinyi Zheng, Douglas Burdick, Lucian Popa, Xu Zhong, and Nancy Xin Ru Wang. Global table extractor (GTE): A framework for joint table identification and cell structure recognition using visual context. In IEEE Winter Conference on Applications of Computer Vision, WACV 2021, Wa...

    2021

  60. [68]

    Longembed: Extending embedding models for long context retrieval, 2024

    Dawei Zhu, Liang Wang, Nan Yang, Yifan Song, Wenhao Wu, Furu Wei, and Sujian Li. Longembed: Extending embedding models for long context retrieval, 2024

    2024

  61. [69]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  62. [70]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  63. [71]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.