Pith. sign in

REVIEW 10 cited by

AnglE-optimized Text Embeddings

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.12871 v9 pith:4RFRWGDT submitted 2023-09-22 cs.CL cs.AIcs.LG

Xianming Li , Jing Li This is my paper
classification cs.CLcs.AIcs.LG
keywords angletextoptimizationcosineembeddingsaturationtasksangle-optimized
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

High-quality text embedding is pivotal in improving semantic textual similarity (STS) tasks, which are crucial components in Large Language Model (LLM) applications. However, a common challenge existing text embedding models face is the problem of vanishing gradients, primarily due to their reliance on the cosine function in the optimization objective, which has saturation zones. To address this issue, this paper proposes a novel angle-optimized text embedding model called AnglE. The core idea of AnglE is to introduce angle optimization in a complex space. This novel approach effectively mitigates the adverse effects of the saturation zone in the cosine function, which can impede gradient and hinder optimization processes. To set up a comprehensive STS evaluation, we experimented on existing short-text STS datasets and a newly collected long-text STS dataset from GitHub Issues. Furthermore, we examine domain-specific STS scenarios with limited labeled data and explore how AnglE works with LLM-annotated data. Extensive experiments were conducted on various tasks including short-text STS, long-text STS, and domain-specific STS tasks. The results show that AnglE outperforms the state-of-the-art (SOTA) STS models that ignore the cosine saturation zone. These findings demonstrate the ability of AnglE to generate high-quality text embeddings and the usefulness of angle optimization in STS.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 21 citations worldwide. Full citation record

  1. External Data Extraction Attacks against Retrieval-Augmented Large Language Models

    cs.CR 2025-10 conditional novelty 7.0 of 10

    SECRET, an adaptive RAG extraction attack that fuses LLM-optimized jailbreak prompts with cluster-guided queries, extracts large portions of private databases from commercial and open LLMs, including ~35% of a sampled...

  2. Beyond Satisfaction: Learning Associations Between Content, Reviews, and Well-Being

    cs.SI 2026-06 conditional novelty 6.0 of 10

    Ratings and sentiment loosely track PERMA well-being facets in Goodreads reviews and align more with immediate hedonic than enduring eudaimonic expressions, while book themes like religion and drives correlate with hi...

  3. VisCon-100K: Leveraging Contextual Web Data for Fine-tuning Vision Language Models

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A web-context-derived dataset and a 'leaky modality mix' of captions with Q&A pairs improve vision-language model fine-tuning on several benchmarks.

  4. Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages

    cs.CL 2025-02 conditional novelty 6.0 of 10

    Users who first used a Spanish AI writing assistant subsequently used the English AI writing assistant less, suggesting a spillover that violates choice independence.

  5. SHIFT: Self-reconstruction Harnesses Implicit Fine-grained Thinking for Retrieval

    cs.IR 2026-07 conditional novelty 5.0 of 10

    SHIFT trains LLM retrievers to reason through latent continuous tokens and reconstruct explicit reasoning traces, improving reasoning-intensive retrieval.

  6. ExploreGS: Explorable 3D Scene Reconstruction with Virtual Camera Samplings and Diffusion Priors

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    Adding information-gain-selected virtual views refined by video diffusion priors to 3D Gaussian Splatting improves arbitrary-view rendering quality.

  7. Towards Applying Large Language Models to Complement Single-Cell Foundation Models

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A fusion model called scMPT, combining scGPT with an LLM text encoder, improves single-cell cell type classification on most tested datasets, and the paper shows the LLM relies on marker genes and simple expression patterns.

  8. A Framework for Deductive Semantic Content Analysis at Scale in Science Education Using Text Embeddings

    cs.CL 2025-08 conditional novelty 4.0 of 10

    A few-shot text embedding classification framework achieves high agreement with human coders (Cohen's Kappa 0.74-0.83) on a simulated exhaustive coding task over 2,899 physics education survey responses.

  9. Whispers of Many Shores: Cultural Alignment through Collaborative Cultural Expertise

    cs.AI 2025-05 reject novelty 4.0 of 10

    A multi-agent router that selects culturally specialized LLM personas reports a jump in self-scored cultural alignment from 0.208 to 0.820, but the metric and the claimed method are not independently validated.

  10. Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems

    cs.IR 2025-05 conditional novelty 4.0 of 10

    A reranker fine-tuned on hard negatives selected by two cosine-distance criteria outperforms older negative sampling methods on enterprise and domain-specific retrieval benchmarks.

Pith tools