Pith. sign in

REVIEW 9 cited by

Why Does the Effective Context Length of LLMs Fall Short?

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.18745 v1 pith:3Y7RFGYF submitted 2024-10-24 cs.CL

Chenxin An , Jun Zhang , Ming Zhong , Lei Li , Shansan Gong , Yao Luo , Jingjing Xu , Lingpeng Kong This is my paper · ORCID
classification cs.CL
keywords llmstrainingcontextlengthsmodelsperformancepositionsstring
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Advancements in distributed training and efficient attention mechanisms have significantly expanded the context window sizes of large language models (LLMs). However, recent work reveals that the effective context lengths of open-source LLMs often fall short, typically not exceeding half of their training lengths. In this work, we attribute this limitation to the left-skewed frequency distribution of relative positions formed in LLMs pretraining and post-training stages, which impedes their ability to effectively gather distant information. To address this challenge, we introduce ShifTed Rotray position embeddING (STRING). STRING shifts well-trained positions to overwrite the original ineffective positions during inference, enhancing performance within their existing training lengths. Experimental results show that without additional training, STRING dramatically improves the performance of the latest large-scale models, such as Llama3.1 70B and Qwen2 72B, by over 10 points on popular long-context benchmarks RULER and InfiniteBench, establishing new state-of-the-art results for open-source LLMs. Compared to commercial models, Llama 3.1 70B with \method even achieves better performance than GPT-4-128K and clearly surpasses Claude 2 and Kimi-chat.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Measuring How (Not Just Whether) VLMs Build Common Ground

    cs.CL 2025-09 conditional novelty 7.0 of 10

    A four-metric suite shows VLM self-play in referential games diverges from human grounding patterns, with GPT4o-mini closest and task success not implying common ground.

  2. UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseComp

    cs.CL 2026-07 conditional novelty 6.0 of 10

    A unified KG-plus-live-web data pipeline covering all three multimodal BrowseComp information-flow patterns, plus an exploration-degree filter, yields a 35B agent at 54.4 avg accuracy.

  3. Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE

    cs.LG 2026-07 accept novelty 6.0 of 10

    Jet-Long is a tuning-free bifocal RoPE method that dynamically sets remote group size from sequence length, recovering the base model within the pretrained window and beating prior zero-shot extenders on RULER, HELMET...

  4. NeedleChain: Measuring Intact Context Comprehension Capability of Large Language Models

    cs.CL 2025-07 conditional novelty 6.0 of 10

    LLMs do well at classic needle-in-a-haystack retrieval but fail to integrate all-relevant chains of evidence, especially when the required reasoning order is right-to-left.

  5. The Future is Agentic: Definitions, Perspectives, and Open Challenges of Multi-Agent Recommender Systems

    cs.IR 2025-07 conditional novelty 6.0 of 10

    A framework for agentic recommender systems plus a pilot study showing multi-agent pipelines beat a single-shot LLM only on high-diversity user histories.

  6. Respecting Temporal-Causal Consistency: Entity-Event Knowledge Graphs for Retrieval-Augmented Generation

    cs.IR 2025-06 conditional novelty 6.0 of 10

    A mention-level entity-event knowledge graph for RAG modestly improves temporal-causal question answering on a new narrative benchmark, with gains mostly coming from adding HyDE-style hypothetical answers.

  7. Mono: Is Your "Clean" Vulnerability Dataset Really Solvable? Exposing and Trapping Undecidable Patches and Beyond

    cs.CR 2025-06 conditional novelty 6.0 of 10

    Mono reports that 31% of MegaVul patches are non-security and about 16.7% of CVEs are 'undecidable', while its added context raises LLM vulnerability detection F1 by up to 15%.

  8. The Prompt is Mightier than the Example

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Injecting domain knowledge into prompts can substitute for many in-context examples in LLM-based synthetic tabular data generation, cutting required example counts by 40-90%.

  9. Hierarchical Reranking for Scalable Financial RAG System

    cs.IR 2026-07 reject novelty 4.0 of 10

    A finance-specific RAG pipeline combining table-to-JSON conversion, two-stage reranking, and long-context split-fusion reports NDCG@20=0.7918 and second place in the ICAIF '24 FinanceRAG challenge.

Pith tools