Pith. sign in

REVIEW 4 cited by

LongCoder: A Long-Range Pre-trained Language Model for Code Completion

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2306.14893 v1 pith:VPPO6YWL submitted 2023-06-26 cs.SE cs.AIcs.CLcs.LG

Daya Guo , Canwen Xu , Nan Duan , Jian Yin , Julian McAuley This is my paper · ORCID
classification cs.SEcs.AIcs.CLcs.LG
keywords codetokenslongcodercompletionavailablebridgeefficiencyinput
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In this paper, we introduce a new task for code completion that focuses on handling long code input and propose a sparse Transformer model, called LongCoder, to address this task. LongCoder employs a sliding window mechanism for self-attention and introduces two types of globally accessible tokens - bridge tokens and memory tokens - to improve performance and efficiency. Bridge tokens are inserted throughout the input sequence to aggregate local information and facilitate global interaction, while memory tokens are included to highlight important statements that may be invoked later and need to be memorized, such as package imports and definitions of classes, functions, or structures. We conduct experiments on a newly constructed dataset that contains longer code context and the publicly available CodeXGLUE benchmark. Experimental results demonstrate that LongCoder achieves superior performance on code completion tasks compared to previous models while maintaining comparable efficiency in terms of computational resources during inference. All the codes and data are available at https://github.com/microsoft/CodeBERT.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 7 citations worldwide. Full citation record

  1. EvolKV: Evolutionary KV Cache Compression for LLM Inference

    cs.LG 2025-09 conditional novelty 6.0 of 10

    CMA-ES search over per-layer KV cache budgets beats uniform and pyramidal compression heuristics on LongBench, NIAH, RULER, and GSM8K, and edges past the full cache on one code dataset at 1.5% of the budget.

  2. SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding

    cs.DC 2025-06 conditional novelty 6.0 of 10

    SwiftSpec uses asynchronous, disaggregated speculative decoding with parallel tree generation and fused kernels to speed up LLM decoding by 1.75x on average over baselines, reaching 348 tokens/s for Llama3-70B on 8 H800 GPUs.

  3. CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective

    cs.CL 2025-02 conditional novelty 6.0 of 10

    A new selection metric that combines attention weights with projected value-state norms reduces output perturbation in LLM key-value cache eviction.

  4. In-Context Learning as an Effective Estimator of Functional Correctness of LLM-Generated Code

    cs.SE 2025-07 conditional novelty 4.0 of 10

    Few-shot in-context examples improve LLM-based functional correctness estimation for generated code relative to zero-shot judgment, but the gains are modest and uneven.

Pith tools