Pith. sign in

REVIEW 27 cited by

Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documents

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.19923 v4 pith:WJ22NHO5 submitted 2023-10-30 cs.CL cs.AIcs.LG

Michael Günther , Jackmin Ong , Isabelle Mohr , Alaeddine Abdessalem , Tanguy Abel , Mohammad Kalim Akram , Susana Guzman , Georgios Mastrapas
show 5 more authors
Saba Sturua Bo Wang Maximilian Werk Nan Wang Han Xiao
This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords documentsembeddingstextembeddingjinamodelmodelsperformance
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Text embedding models have emerged as powerful tools for transforming sentences into fixed-sized feature vectors that encapsulate semantic information. While these models are essential for tasks like information retrieval, semantic clustering, and text re-ranking, most existing open-source models, especially those built on architectures like BERT, struggle to represent lengthy documents and often resort to truncation. One common approach to mitigate this challenge involves splitting documents into smaller paragraphs for embedding. However, this strategy results in a much larger set of vectors, consequently leading to increased memory consumption and computationally intensive vector searches with elevated latency. To address these challenges, we introduce Jina Embeddings 2, an open-source text embedding model capable of accommodating up to 8192 tokens. This model is designed to transcend the conventional 512-token limit and adeptly process long documents. Jina Embeddings 2 not only achieves state-of-the-art performance on a range of embedding-related tasks in the MTEB benchmark but also matches the performance of OpenAI's proprietary ada-002 model. Additionally, our experiments indicate that an extended context can enhance performance in tasks such as NarrativeQA.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 27 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Attention Goes Blind: Numerical Failure in ALiBi Positional Encodings

    cs.CL 2026-08 conditional novelty 6.0 of 10

    ALiBi's linearly growing positional bias underflows floating-point attention in long contexts, zeroing out distant attention weights, with measurable but task-dependent effects on retrieval.

  2. Unsupervised Cross-Protocol Anomaly Analysis in Mobile Core Networks via Multi-Embedding Models Consensus

    cs.CR 2026-03 conditional novelty 6.0 of 10

    Multi-embedding consensus on fused SS7/Diameter/GTP records ranks synthetic cross-protocol inconsistencies far above original traffic, reducing the inspection set from 44% to under 1% at full agreement.

  3. Exploratory Retrieval-Augmented Planning For Continual Embodied Instruction Following

    cs.AI 2025-09 conditional novelty 6.0 of 10

    ExRAP couples LLM planning with a temporal knowledge-graph memory and information-based exploration, improving success and efficiency for continual embodied instruction following.

  4. GasAgent: A Multi-Agent Framework for Automated Gas Optimization in Smart Contracts

    cs.AI 2025-07 conditional novelty 6.0 of 10

    A four-agent LLM pipeline retrieves known Solidity gas-waste patterns, proposes new ones, and automatically verifies and applies the fixes, saving about 10% deployment gas on 82% of real contracts.

  5. SMCLM: Semantically Meaningful Causal Language Modeling for Autoregressive Paraphrase Generation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    SMCLM prepends a frozen sentence embedding to GPT-2 and trains with causal language modeling, producing paraphrases that the authors find competitive with supervised methods and best among the unsupervised baselines tested.

  6. The Missing Link: Joint Legal Citation Prediction using Heterogeneous Graph Enrichment

    cs.SI 2025-06 accept novelty 6.0 of 10

    A graph neural network that enriches legal citation graphs with categorical metadata nodes predicts case and law citations more accurately than prior GNN baselines, and joint training boosts case citation prediction.

  7. jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval

    cs.AI 2025-06 conditional novelty 6.0 of 10

    jina-embeddings-v4 is a multimodal embedding model built on Qwen2.5-VL with LoRA adapters and dual single/multi-vector outputs, and it introduces the Jina-VDR benchmark for visually rich retrieval.

  8. Towards A Generalist Code Embedding Model Based On Massive Data Synthesis

    cs.IR 2025-05 conditional novelty 6.0 of 10

    CodeR, a 1.5B code embedding model trained on 2.9M synthetic retrieval triples through a three-stage curriculum, reports average NDCG@10 of 81.77 on CoIR and 72.8 on CodeRAG, above every baseline it lists.

  9. FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents

    cs.IR 2025-04 conditional novelty 6.0 of 10

    A framework that builds realistic, refreshable IR and RAG benchmarks on niche technical topics, with five datasets and large measured headroom for retrieval models.

  10. Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoning

    cs.CR 2025-02 reject novelty 6.0 of 10

    RAGC⃝ watermarks RAG knowledge bases by injecting benign chain-of-thought passages that only watermarked queries retrieve, enabling ownership verification via a Wilcoxon test on whether the generated reasoning contain...

  11. Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Universal adversarial suffixes can shift any text's embedding toward a model's biased mean direction, breaking embedding-based LLM safety classifiers.

  12. Momentum Posterior Regularization for Multi-hop Dense Retrieval

    cs.CL 2024-12 conditional novelty 6.0 of 10

    MoPo regularizes multi-hop dense retrieval with per-hop query-focused gold summaries and a momentum-averaged teacher, improving retrieval and QA on HotpotQA and StrategyQA.

  13. A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection

    cs.CL 2024-11 conditional novelty 6.0 of 10

    A synthetic-data-driven method trains lightweight classifiers to flag user prompts as off-topic relative to a system prompt, outperforming heuristics and partially transferring to jailbreak and harmful content.

  14. Efficient Code Embeddings from Code Generation Models

    cs.CL 2025-08 conditional novelty 5.0 of 10

    jina-code-embeddings, 0.5B and 1.5B models built from Qwen2.5-Coder with last-token pooling, achieve strong code retrieval scores competitive with far larger embedding models.

  15. SemCSE: Semantic Contrastive Sentence Embeddings Using LLM-Generated Summaries For Scientific Abstracts

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Using multiple LLM-generated summaries of the same abstract as positive pairs trains scientific text embeddings that beat citation-trained baselines on retrieval and clustering, while the new benchmark shares its trai...

  16. KinyaColBERT: A Lexically Grounded Retrieval Model for Low-Resource Retrieval-Augmented Generation

    cs.CL 2025-07 conditional novelty 5.0 of 10

    KinyaColBERT, a morphology-aware two-tier ColBERT retriever, reports large MRR gains over multilingual baselines and commercial APIs on a new Kinyarwanda agricultural retrieval benchmark.

  17. Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning

    cs.CL 2025-06 conditional novelty 5.0 of 10

    State-of-the-art text embeddings lag far behind on tasks requiring pragmatic inference, stance detection, and social meaning, relative to their strong performance on surface semantic benchmarks.

  18. CRAWLDoc: A Dataset for Robust Ranking of Bibliographic Documents

    cs.CL 2025-06 conditional novelty 5.0 of 10

    CRAWLDoc ranks linked web documents by embedding similarity to a paper's landing page, evaluated on a new manually labeled dataset of 600 publications from six publishers.

  19. Reconstructing Context: Evaluating Advanced Chunking Strategies for Retrieval-Augmented Generation

    cs.IR 2025-04 conditional novelty 5.0 of 10

    A comparative evaluation on NFCorpus and MSMarco shows contextual retrieval beats late chunking on retrieval metrics at higher computational cost, and neither consistently beats standard early chunking.

  20. WindVE: Collaborative CPU-NPU Vector Embedding

    cs.DC 2025-04 conditional novelty 5.0 of 10

    WindVE offloads peak vector-embedding queries to idle CPUs through a regression-calibrated queue manager, achieving up to 22.3% higher concurrency than a GPU/NPU-only baseline under the same SLOs.

  21. AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuning

    cs.CL 2024-12 conditional novelty 5.0 of 10

    AIDE generates task-relevant synthetic fine-tuning data from 10 seeds via attribute-guided multi-hop synthesis and residual connections, and reports gains over gold-data fine-tuning on several benchmarks.

  22. Arctic-Embed 2.0: Multilingual Retrieval Without Compromise

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Arctic-Embed 2.0 delivers open multilingual embedding models with competitive MTEB-R and CLEF retrieval scores and strong 256-dimension MRL compression, plus new evidence on cross-lingual transfer.

  23. Advancing Large Language Models for Spatiotemporal and Semantic Association Mining of Similar Environmental Events

    cs.IR 2024-11 reject novelty 5.0 of 10

    A language-model retrieval plus Geo-Time Re-ranking pipeline finds similar environmental events more accurately than several dense retrieval and reranking baselines on the LEO Network corpus.

  24. Cohort Retrieval using Dense Passage Retrieval

    cs.IR 2025-06 conditional novelty 4.0 of 10

    A DPR-based retrieval model trained on echocardiography reports outperforms BM25 and off-the-shelf embeddings on common cohort queries, but underperforms on rare-condition (out-of-distribution) searches.

  25. SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval

    cs.IR 2024-12 conditional novelty 4.0 of 10

    LLM-generated synthetic hard negatives, combined with retrieved negatives in a hybrid mix, improve dense retrieval accuracy on BEIR benchmarks.

  26. Zero-Shot Prompting and Few-Shot Fine-Tuning: Revisiting Document Image Classification Using Large Language Models

    cs.CV 2024-12 conditional novelty 4.0 of 10

    With zero labeled examples, GPT-4-Vision reaches 69.9% accuracy on RVL-CDIP document classification, and generative LoRA fine-tuning of Mistral-7B reaches 72.5% with just 10 labels per class.

  27. LLMs are Also Effective Embedding Models: An In-depth Overview

    cs.CL 2024-12 conditional novelty 2.0 of 10

    A structured survey of using decoder-only LLMs as text embedding models, covering prompting, fine-tuning, data construction, benchmarks, and open problems.

Pith tools