Pith. sign in

REVIEW 4 major objections 4 minor 19 references

An agentic, cache-first architecture lets LLM-generated topic channels reach connected-TV home screens at production latency, with generative retrieval sped up 100x and niche-topic relevance more than tripled.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 07:25 UTC pith:ET4ROBTA

load-bearing objection A concrete, honest industry engineering paper whose caching and throughput playbook is the genuine contribution; the quality evidence is self-referential and cold-cache handling is unspecified, so it deserves a serious referee but major revision. the 4 major comments →

arxiv 2607.09988 v3 pith:ET4ROBTA submitted 2026-07-10 cs.IR cs.AI

An LLM-powered Agentic Recommendation System for Connected TV Content Discovery

Lei Shi , Di Wang , Harry Tran , Helsing Xu , Yuchen Lu , Dhara Ghodasara , Wilson Chaney , Xueting Liao
show 17 more authors
Jerry Yu Huayu Ding Reza Mirghaderi David Fan Qi Guo Chongguang He Warren Wang Warren Deng Mingze Gao Shike Mei Shuo Tang Zhe Zhang Jianming He Abhishek Kumar Haotian Wu Hamed Firooz Li Li
This is my paper
classification cs.IR cs.AI
keywords LLM agentsrecommender systemsconnected TVgenerative retrievalsemantic IDsinference latencycache-based servingcontextual signals
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that large language models can make connected-TV recommendations more contextually aware—aware of trending events, breaking news, and cross-surface activity—without paying the usual latency or feature-engineering costs. Its bet is a hybrid agentic architecture: LLM agents read a natural-language context snapshot and generate topical channel names, while traditional machine-learning models handle the latency-sensitive media ranking. The load-bearing engineering claim is that LLM inference can be removed from the real-time request path through asynchronous cache refills, with per-component caching and chained retrieval memoization buying back the remaining latency. As evidence, the paper reports a cumulative 100x throughput improvement for its generative retrieval model and offline niche-topic relevance rising from 0.15 to 0.50 across training iterations. A sympathetic reader would care because, if true, this gives recommendation systems a way to absorb new contextual signals with prompt changes alone, instead of bespoke feature pipelines.

Core claim

The paper's central claim is that an agentic division of labor makes LLMs practical for connected-TV recommendation. Three agents—an LLM topic retrieval agent, a hybrid media retrieval and ranking agent built around MediaBrain (an LLM-native generative retrieval model), and an LLM topic ranking agent—are coordinated by an orchestration layer that assembles all contextual signals into natural-language prompts. Removing real-time LLM inference from the user-facing path through asynchronous, cache-based refills, the design reports a 100x throughput improvement (2 to 200 QPS per GPU) for MediaBrain serving, and offline niche-entity relevance rising from 0.15 to 0.50 via training-data iteration.

What carries the argument

The agentic pipeline's cache-first execution model. An orchestration layer builds a shared natural-language context object; a topic-retrieval LLM generates channel names; MediaBrain—generative retrieval over residual-quantized semantic IDs (SIDs)—maps each name to media; and a topic-ranking LLM orders the channels. The load-bearing mechanism is removing LLM inference from the user-facing critical path: the pipeline runs asynchronously after the response is delivered, results are read from a multi-layer per-component cache, and generated tokens are carried forward in the chaining response so later requests skip regeneration.

Load-bearing premise

The load-bearing premise is that the LLM's open-ended topic names will consistently find enough high-quality media inventory to fill a channel, a premise left untested by any end-to-end evaluation and unprotected by any feedback loop from downstream retrieval failures.

What would settle it

Serve the full agentic pipeline on a production connected-TV population and measure the home-screen p95 latency under cache-miss conditions and the share of LLM-generated topics that return empty or poor-quality media slates; if the latency budget is missed or a material share of topics yield insufficient inventory, the central production-viability claim is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • New contextual signal types—trending events, live competitions, cross-surface activity—can be absorbed through prompt changes alone, without retraining ranking models or building feature pipelines.
  • LLM-generated topic channels can be served at home-screen latency because the LLM runs asynchronously after each response; live reads hit a per-component cache, so the user-facing path never invokes the LLM.
  • MediaBrain's retrieval quality is dominated by training-data quality: iterating annotation pipelines and task diversity lifted offline relevance on niche-entity topics from 0.15 to 0.50, and prompt-only context injection lets the same model adapt to different aesthetics or platforms without retraining.
  • SID-topic dispersion, quantified by a Herfindahl-Hirschman Index, predicts per-topic relevance, offering a cheap way to identify hard, entity-driven topics before investing in training data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The cache-and-async-refill pattern is surface-agnostic: any recommendation surface with a periodically refreshed home or feed could run LLM agents off the request path, trading a bounded freshness delay for richer contextual reasoning.
  • Because the session signals the paper already logs—watched, dismissed, and skipped topics—map directly onto the shared context object, a minimal version of its proposed closed-loop feedback is implementable immediately: feed those outcomes back into the topic-retrieval prompt on the next refill cycle.
  • The HHI finding implies a targeted data-engineering strategy the paper does not quite state: rather than adding generic training data for niche entities, collect samples that specifically reduce SID dispersion for dispersed brands and cities, addressing the bottleneck rather than data volume.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper describes an LLM-powered agentic recommendation system for Connected TV (CTV) content discovery. An orchestration layer constructs a natural-language context object; three agents—topic retrieval (Llama-3 70B), media retrieval/ranking (including the MediaBrain generative retrieval model), and topic ranking—are coordinated. To meet CTV latency constraints, the system runs the agentic pipeline asynchronously after a response and serves subsequent requests from a precomputed cache. The authors report engineering optimizations (per-component caching, cursor-carried token reuse, vLLM throughput fixes) yielding a cumulative 100× throughput improvement for MediaBrain, and offline relevance evaluations on CTV100/1000 and Niche Entities showing improved topical relevance. They also discuss limitations, including the absence of closed-loop feedback between agents and a potential SID bottleneck for niche entities.

Significance. If correct, the main contribution is practical: a hybrid architecture that uses LLM agents for open-ended topic generation and ranking while keeping them off the user-facing critical path, plus a detailed account of the 100× throughput optimizations. The paper is transparent about some limitations (§5.1) and provides concrete, stage-level engineering numbers. However, the significance is reduced by three issues: the cache-based latency claim is unqualified with respect to cold starts; the quality evaluation is self-referential with no human anchor; and no end-to-end or online evaluation is presented. The paper does not provide code or independent benchmarks, so the strengths are in the engineering narrative rather than in reproducible artifacts.

major comments (4)
  1. [§3.1/§3.7] The central latency claim—'our design removes real-time LLM inference from the critical path'—is not specified for cache misses. The pipeline runs asynchronously after a response and writes back for subsequent requests, so the first request in a session, a request after TTL expiry, or one with materially changed session context has no warm cache. If the server falls back to synchronous LLM inference, the 70B topic-retrieval call or MediaBrain's ~500ms generation violates the hundreds-of-milliseconds budget; if it falls back to conventional retrieval, the first request is not LLM-personalized. Cursor-carried token reuse (§4.1.2) only helps chaining pages after the first. No cache hit rate, cold-start frequency, or fallback latency is reported. Thus the latency claim is established only for cache-warm requests.
  2. [§4.2] The relevance evaluation is self-referential. The LLM judge that scores retrieved media is an LLM; the topic prompts come from an LLM topic retrieval agent; the benchmarks CTV100/1000 are built from the authors' impression data; Niche Entities is hand-curated by the authors. There is no human relevance judgment or external benchmark anywhere in the loop. Consequently, the improvements in Figure 2 (e.g., Niche Entities 0.15→0.50) may partly reflect alignment between the judge and the retrieval model rather than externally valid topical relevance. Without a human-annotated or third-party anchor, the claim that MediaBrain 'retrieves relevant media' is not established. This is load-bearing for the system's value proposition.
  3. [§4 / §5.1] There is no end-to-end or online evaluation of the agentic pipeline. §4 evaluates only the MediaBrain retrieval component. The central claim that the architecture enables LLM-driven CTV discovery at production latency and quality depends on the topic retrieval agent's generated topics mapping to sufficient inventory, and §5.1 explicitly concedes that no closed-loop feedback exists: 'if the topic retrieval agent consistently generates topics for which the media retrieval and ranking agent finds poor-quality or insufficient inventory, the topic agent has no mechanism to learn from this outcome.' Without any measurement of topic-to-inventory coverage or user-facing outcomes, the system-level claim is unvalidated. At minimum, the paper should report inventory coverage statistics and an online or simulated comparison.
  4. [§4.1.3] The throughput accounting is not internally consistent. The text reports a 20× increase from concurrent handling, then 'raising performance to 80 QPS' after CPU fixes, then a '4 times' GPU utilization increase to reach 200 QPS. From the stated base of 2 QPS, 20× gives 40 QPS; reaching 80 QPS is another 2×; reaching 200 QPS is 2.5×, not 4×. The three factors do not multiply to 100× as written. Also no hardware details, latency percentiles, or cost per request are given. The 100× speedup claim should be restated with a consistent decomposition.
minor comments (4)
  1. [Global] There are minor typographical issues ('Tobridgethisgap' in Background, 'isend-to-end' in §3.7).
  2. [Figure 2] Figure 2 lacks error bars and does not specify the LLM-judge prompt or sampling parameters; please include evaluation protocol details.
  3. [§5.2] The HHI correlation reports Spearman-ρ = 0.25 with no significance test or confidence interval; the claim that this supports the SID-bottleneck hypothesis is underpowered.
  4. [References] Several 2026 arXiv references may be difficult to verify at time of review; please ensure URLs/DOIs are stable and accessible.

Circularity Check

1 steps flagged

One quality result is partly manufactured by eval-aligned prompts; the architecture, latency, and throughput claims are independent engineering claims, not circular derivations.

specific steps
  1. fitted input called prediction [Section 4.2.1, Figure 2 (panel b: 'Key Changes: Eval-aligned prompts')]
    "Key Changes: Eval-aligned prompts; Multi-task expansion ... Notably the difficult niche entity topics improve from 0.15 to 0.50."

    The headline Niche Entities improvement from 0.15 to 0.35 occurs in the exact iteration whose key changes include 'Eval-aligned prompts'. This means the evaluation inputs were rewritten to match the training distribution before the score was reported. The plotted relevance gain is therefore not a pure model improvement: part of the measured 'prediction' is produced by changing the test instrument to be more aligned with the training data, rather than by the retrieval model's learned capability alone. The later gain to 0.50 continues from this recalibrated baseline, so the paper's emphasized niche-entity result is partly fitted into the evaluation protocol by construction.

full rationale

The paper is primarily an engineering/architecture report, not a mathematical derivation. The cache-based removal of LLM inference from the hot path is true by design for warm-cache reads, and the 100x throughput speedup is a measured systems optimization; neither reduces to its own inputs. The retrieval-quality evaluation, however, has a self-referential element: relevance is scored by an LLM judge using in-house benchmarks, and Figure 2 explicitly includes 'Eval-aligned prompts' as a key change, so part of the headline Niche Entities improvement is an artifact of aligning the evaluation to the training distribution. This warrants a partial-circularity flag for that specific result. The paper's central latency claim is separate, though it has an acknowledged evidence gap: cold-start behavior is unspecified (requests before the async cache is warm), and Section 5.1 concedes the pipeline has no closed-loop feedback and no end-to-end/online evaluation. Those are validity limitations, not circularity. No load-bearing self-citation chain or imported uniqueness theorem is used, so the score is 4 rather than higher.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

No new physical or conceptual entities are postulated (no new particles, forces, or conserved quantities). MediaBrain is an engineered artifact, not a postulate; semantic IDs are adopted from Rajput et al. (2023) as cited. The new SID-topic dispersion/HHI metric (§5.2) is a diagnostic construct, not an entity with independent falsifiable handles. The quality narrative rests on a chain of unvalidated proxy assumptions (LLM topics are better; an LLM judge measures relevance; topical relevance drives engagement), while the serving story rests on the argued-but-unmeasured near-determinism of beam search. No external benchmark anchors any quality number.

free parameters (4)
  • LLM-judge relevance criterion = unreported threshold/prompt for 'topically relevant'
    Every headline quality number (CTV100/1000, Niche Entities 0.15→0.50) is defined by an uncalibrated LLM judge; no human agreement and no threshold specification are given (§4.2.1, Figure 2).
  • SID codebook hyperparameters = 4 layers × 1024 codes; 'small beam'
    Chosen to balance granularity and latency; they define what retrieval can and cannot represent, and the paper attributes the niche-entity ceiling to this 'SID bottleneck' (§3.6, §4.1.3, §5.2).
  • Per-component cache TTLs = not specified per component
    Cache lifetimes per stage are engineered choices trading freshness against latency; their effect on result quality is asserted but never measured (§3.7, §4.1.1).
  • Per-iteration training data mixtures = 1.7B → 58M → 103M → 35M → 118M pairs
    Each iteration changed tasks, data size, and prompts simultaneously, so the 0.15→0.50 improvement is not attributable to any single ingredient (Figure 2).
axioms (5)
  • domain assumption LLM-generated open-ended topics improve CTV discovery over heuristic or curated topic sources.
    Core motivation of the architecture (Introduction §1); never tested against a non-LLM topic-sourcing baseline anywhere in the paper.
  • domain assumption An LLM judge's topical-relevance scores proxy user satisfaction.
    All offline quality results rest on this premise; no human-label agreement and no offline-online correlation are reported (§4.2.1).
  • domain assumption Beam-search outputs are near-deterministic, making cached tokens safe to reuse across pages and sessions without quality drift.
    Load-bearing for the entire cache-based latency story; argued from cumulative log-probability in §4.1.1 but never empirically checked for drift across chaining pages.
  • domain assumption Asynchronously refilled caches preserve request-time quality.
    The system serves from a cache refreshed after the response is sent (§3.7); the staleness-vs-latency trade-off is managed by per-component TTLs with no quality measurement of stale results.
  • domain assumption Topical relevance drives CTV engagement.
    Standard recsys premise invoked by the evaluation setup but not demonstrated for CTV surfaces in this paper.

pith-pipeline@v1.3.0-alltime-deepseek · 10247 in / 18650 out tokens · 196983 ms · 2026-08-02T07:25:39.762315+00:00 · methodology

0 comments
read the original abstract

Recommendation systems, from traditional multi-stage to recent unified generative architectures, face challenges in incorporating diverse contextual signals, such as trending topics, breaking news, cultural events, and cross-surface user activities, into their ranking pipelines. These systems are designed to consume structured behavioral signals with consistent schemas, and lack the reasoning capability to naturally process unstructured or heterogeneously formatted contextual information. Incorporating such signals typically requires feature engineering, bespoke data pipelines, and carefully tuned heuristics. In this paper, we present an LLM-powered agentic recommendation system designed for Connected TV (CTV) content discovery that addresses these limitations. Our system leverages the reasoning capabilities of large language models to naturally process and synthesize diverse signals across varying schemas and structures, eliminating much of the manual integration inherent in traditional ranking and retrieval systems. Recognizing that current LLM-based solutions still fall short of traditional machine learning models in several recommendation tasks, including retrieval efficiency, personalization precision, and scalability, we adopt an agentic architecture that orchestrates specialized components, allowing each sub-task to be handled by the most suitable method, whether LLM-based or traditional ML. The main contribution of this work is our engineering approach to successfully overcoming the practical limitations of enabling LLM for recommendation, particularly inference latency. We share insights from our work and discuss the trade-offs and lessons learned in building a hybrid system that combines the flexibility of LLMs with the performance of established recommendation techniques.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 8 linked inside Pith

  1. [1]

    ICML 2024 Tutorial: Physics of Language Models, July

    Zeyuan Allen-Zhu. ICML 2024 Tutorial: Physics of Language Models, July

    2024

  2. [3]

    Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou

    URLhttps: //arxiv.org/abs/2601.08816. Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou. Onerec: Unifying retrieve and rank with generative recommender and iterative preference alignment,

    Pith/arXiv arXiv

  3. [6]

    Xu Huang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie

    URL https://arxiv.org/abs/2510.07784. Xu Huang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie. Recommender AI agent: Integrating large language models for interactive recommendations. InProceedings of the 17th ACM International Conference on Web Search and Data Mining (WSDM 2024), pages 924–927,

    arXiv 2024

  4. [9]

    URLhttps://arxiv.org/abs/2509.10397. Zhanyu Liu, Shiyao Wang, Xingmei Wang, Rongzhou Zhang, Jiaxin Deng, Honghui Bao, Jinghao Zhang, Wuchao Li, Pengfei Zheng, Xiangyu Wu, Yifei Hu, Qigen Hu, Xinchen Luo, Lejian Ren, Zixing Zhang, Qianqian Wang, Kuo Cai, YunfanWu, HongtaoCheng, ZexuanCheng, LuRen, HuanjieWang, YiSu, RuimingTang, KunGai, andGuoruiZhou. OneR...

    Pith/arXiv arXiv

  5. [10]

    Qiyao Peng, Hongtao Liu, Hua Huang, Jian Yang, Qing Yang, and Minglai Shao

    URLhttps://arxiv.org/abs/2510.11639. Qiyao Peng, Hongtao Liu, Hua Huang, Jian Yang, Qing Yang, and Minglai Shao. A survey on LLM-powered agents for recommender systems. InFindings of the Association for Computational Linguistics: EMNLP 2025, pages 11574–11583. Association for Computational Linguistics,

    arXiv 2025

  6. [11]

    Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q

    Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan H. Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Maheswaran Sathiamoorthy. Recommender systems with generative retrieval. InAdvances in Neural Information Processing Systems (NeurIPS 2023),

    2023

  7. [12]

    doi: 10.48550/arxiv.2305.05065. Sudarshan Srinivasa Ramanujam, Antonio Alonso, Saurabh Kataria, Siddharth Dangi, Akhilesh Gupta, Birjodh Singh Tiwana, Manas Haribhai Somaiya, Luke Simon, David Byrne, Sojeong Ha, et al. Large scale retrieval for the linkedin feed using causal language models. InProceedings of the AAAI Conference on Artificial Intelligence,...

    Pith/arXiv arXiv doi:10.48550/arxiv.2305.05065

  8. [13]

    Multi-agent video recom- menders: Evolution, patterns, and open challenges

    12 Srivaths Ranganathan, Abhishek Dharmaratnakar, Anushree Sinha, and Debanshu Das. Multi-agent video recom- menders: Evolution, patterns, and open challenges. InCompanion Proceedings of the 19th ACM International Conference on Web Search and Data Mining (WSDM 2026),

    2026

  9. [14]

    Yi Tay, Vinh Q

    doi: 10.1145/3779211.3795739. Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. Transformer memory as a differentiable search index. InAdvances in Neural Information Processing Systems (NeurIPS 2022),

    arXiv 2022

  10. [15]

    Haochen Wang, Yi Wu, Daryl Chang, Li Wei, and Lukasz Heldt

    doi: 10.48550/arxiv.2202.06991. Haochen Wang, Yi Wu, Daryl Chang, Li Wei, and Lukasz Heldt. Self-evolving recommendation system: End-to-end autonomous model optimization with LLM agents,

    Pith/arXiv arXiv doi:10.48550/arxiv.2202.06991

  11. [16]

    Yang Wu, Haoze Wang, Qian Li, Jun Zhang, Huan Yu, and Jie Jiang

    URLhttps://arxiv.org/abs/2602.10226. Yang Wu, Haoze Wang, Qian Li, Jun Zhang, Huan Yu, and Jie Jiang. Internalizing multi-agent reasoning for accurate and efficient llm-based recommendation,

    Pith/arXiv arXiv

  12. [17]

    Yue Xu, Qian Chen, Zizhan Ma, Dongrui Liu, Wenxuan Wang, Xiting Wang, Li Xiong, and Wenjie Wang

    URLhttps://arxiv.org/abs/2602.09829. Yue Xu, Qian Chen, Zizhan Ma, Dongrui Liu, Wenxuan Wang, Xiting Wang, Li Xiong, and Wenjie Wang. Toward personalized llm-powered agents: Foundations, evaluation, and future directions,

    arXiv

  13. [18]

    Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Michael He, Yinghai Lu, and Yu Shi

    URLhttps://arxiv.org/abs/ 2602.22680. Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Michael He, Yinghai Lu, and Yu Shi. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations. InProceedings of the 41st International Conference on Machine Learning (ICML),

    arXiv

  14. [19]

    Guilin Zhang, Kai Zhao, Jeffrey Friedman, and Xu Chu

    URL https://arxiv.org/abs/2402.17152. Guilin Zhang, Kai Zhao, Jeffrey Friedman, and Xu Chu. LLMs as orchestrators: Constraint-compliant multi-agent optimization for recommendation systems. InProceedings of the ACM Web Conference 2026 (WWW ’26), pages 412–422,

    Pith/arXiv arXiv 2026

  15. [2022]

    URLhttps://arxiv.org/abs/2203.13366. Ruining He, Lukasz Heldt, Lichan Hong, Raghunandan Keshavan, Shifan Mao, Nikhil Mehta, Zhengyang Su, Alicia Tsai, Yueqi Wang, Shao-Chuan Wang, Xinyang Yi, Lexi Baugher, Baykal Cakici, Ed Chi, Cristos Goodrow, Ningren Han, He Ma, Romer Rosales, Abby Van Soest, Devansh Tandon, Su-Lin Wu, Weilong Yang, and Yilin Zheng. PL...

    Pith/arXiv arXiv

  16. [2023]

    Mingfu Liang, Yufei Li, Jay Xu, Kavosh Asadi, Xi Liu, Shuo Gu, Kaushik Rangadurai, Frank Shyu, Shuaiwen Wang, Song Yang, et al

    doi: 10.1145/3600006.3613165. Mingfu Liang, Yufei Li, Jay Xu, Kavosh Asadi, Xi Liu, Shuo Gu, Kaushik Rangadurai, Frank Shyu, Shuaiwen Wang, Song Yang, et al. Generative reasoning re-ranker.arXiv preprint arXiv:2602.07774,

    arXiv

  17. [2024]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E

    doi: 10.1145/3616855.3635749. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. InProceedings of the 29th Symposium on Operating Systems Principles (SOSP 2023), pages 611–626,

    arXiv 2023

  18. [2025]

    Abhimanyu Dubey, Akhil Jauhri, Abhinav Pandey, Abhishek Raghavan, and et al

    URLhttps: //arxiv.org/abs/2502.18965. Abhimanyu Dubey, Akhil Jauhri, Abhinav Pandey, Abhishek Raghavan, and et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  19. [2026]

    Weixin Chen, Yuhan Zhao, Jingyuan Huang, Zihe Ye, Clark Mingxuan Ju, Tong Zhao, Neil Shah, Li Chen, and Yongfeng Zhang

    URLhttps://arxiv.org/abs/2601.20316. Weixin Chen, Yuhan Zhao, Jingyuan Huang, Zihe Ye, Clark Mingxuan Ju, Tong Zhao, Neil Shah, Li Chen, and Yongfeng Zhang. MemRec: Collaborative memory-augmented agentic recommender system,

    arXiv