Pith. sign in

REVIEW 3 major objections 5 minor 7 cited by

The paper claims that recommendation systems can get predictable power-law scaling laws once scaling efficiency is fixed, and that the Kunlun architecture doubles the scaling slope by fusing personalized transforms into attention kernels.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 02:35 UTC pith:GM3CDGJU

load-bearing objection Solid engineering contributions, but the scaling-law claim is built on points the paper itself says are incomparable; major revision needed. the 3 major comments →

arxiv 2602.10016 v3 pith:GM3CDGJU submitted 2026-02-10 cs.IR cs.AI

Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design

Bojian Hou , Xiaolong Liu , Xiaoyi Liu , Jiaqi Xu , Yasmine Badr , Mengyue Hang , Sudhanshu Chanpuriya , Junqing Zhou
show 21 more authors
Yuhang Yang Han Xu Qiuling Suo Laming Chen Yuxi Hu Jiasheng Zhang Huaqing Xiong Yuzhen Huang Chao Chen Yue Dong Yi Yang Shuo Chang Xiaorui Gan Wenlin Chen Santanu Kolay Darren Liu Jade Nie Chunzhi Yang Ellie Wen Jiyan Yang Huayu Li
This is my paper
classification cs.IR cs.AI
keywords recommendation systemsscaling lawsCTR predictionmodel efficiencyModel FLOPs Utilizationattentionsequence modelingfeature interaction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the reason recommendation systems have resisted the power-law scaling laws established for language models is not missing compute but wasted compute. Models that jointly process user behavior sequences and non-sequential context features run at 3–15% Model FLOPs Utilization, and they allocate compute uniformly across layers and event types, so extra FLOPs buy little quality. Kunlun attacks both bottlenecks in one architecture: low-level modules (an attention-style reformulation of the personalized feed-forward network, hierarchical seed pooling, sliding-window attention) and high-level resource reallocation (per-layer compute skipping and per-event-type capacity allocation). On production-scale data the design lifts MFU from 17% to 37%, doubles scaling efficiency relative to the prior state of the art, and yields a predictable power-law relationship between normalized entropy and compute. If the claim holds, click-through-rate systems can extrapolate performance from small compute budgets to massive ones and allocate resources accordingly.

Core claim

The central claim is that scaling laws for joint sequence-and-context recommendation models become predictable once scaling efficiency is fixed. Scaling efficiency is the slope η in NE(C)=NE0−η log(C/C0), normalized by a baseline; it factorizes into algorithmic effectiveness (NE gain per FLOP) and computational efficiency (MFU). Kunlun raises MFU to 37% and η to roughly twice the value of the state-of-the-art interleaving architecture, so the same added log-compute buys twice the NE improvement. The paper presents this as the first predictable power-law scaling for the joint modeling setting, and reports that deployment in major ad-ranking models improves topline metrics by 1.2%.

What carries the argument

The load-bearing identity is the reformulation of the Personalized Feed-Forward Network as Generalized Dot-Product Attention (GDPA): the sequence acts as queries and projections of the non-sequence summary act as keys and values, so the whole block becomes a single fused attention kernel. Around that core, Hierarchical Seed Pooling (HSP) compresses long sequences via learnable seeds and a Kronecker-structured SumKronLinear compression; Sliding Window Attention cuts sequence cost to O(Tw); Computation Skip (CompSkip) alternates modules between even and odd layers; event-level personalization allocates dimension, heads, tokens, layers, and window size per event type. The target object is the p

Load-bearing premise

The fitted power law assumes the NE gains at the three compute scales sit on one comparable curve, even though the paper's own figure caption says those scales use different feature configurations and are not cross-comparable.

What would settle it

Train Kunlun at the three reported compute scales with an identical feature vocabulary and preprocessing, and fit the power law. If the three points do not lie on a single curve, or if the slope relative to the state-of-the-art interleaving model is not approximately 2x, the scaling-law claim is falsified. A second check: re-implement the fused GDPA kernel and measure MFU on the same GPU; a value well below 37% would falsify the efficiency mechanism.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Compute budgeting becomes extrapolative: knowing NE at one log-compute scale predicts NE at larger scales via the fitted power law, allowing resource planning before training.
  • Efficiency is not a side concern but the enabling condition for scaling laws: models with low MFU will not exhibit predictable gains even if capacity is scaled.
  • Joint sequence-context modeling, previously treated as inherently hard to scale, can be made to follow power-law behavior, so production systems do not have to choose between rich features and scaling.
  • The efficiency components transfer independently: sliding-window attention alone gives a 31.1% throughput gain and CompSkip alone cuts FLOPs by 43.1%, so teams can adopt pieces without the full architecture.
  • The deployment result implies the gains survive real traffic: a 1.2% topline improvement is what makes the architecture usable at scale.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An immediate testable extension is to re-run the three compute scales under one fixed feature configuration; the paper's own figure caption notes features differ across scales, so a single-configuration curve would either confirm or refute the fitted 2x slope.
  • Hold the architecture fixed and vary only kernel fusion and MFU: if the NE-vs-compute slope does not move, then MFU is not the causal driver of the scaling improvement, only a correlated change.
  • The 'attention-ify the personalized transform' pattern could generalize beyond recommendations to other memory-bound blocks in ranking or multi-task models, suggesting a recipe: find small irregular matrix multiplications and fuse them into attention-style kernels.
  • If the scaling law is real, it enables a new style of recommender experiment: choose between deeper stacks and wider experts at fixed compute using the fitted slope instead of grid search.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that poor scaling efficiency is the main obstacle to predictable power-law scaling laws for recommender systems that jointly model sequential user behavior and non-sequential context features. It introduces Kunlun, an architecture combining low-level module optimizations (GDPA, HSP, Sliding Window Attention) and high-level compute reallocation (CompSkip, Event-level Personalization, expert parallelism). The authors report that Kunlun raises MFU from 17% to 37% on NVIDIA B200 GPUs, doubles scaling efficiency relative to InterFormer, exhibits power-law scaling consistent with Eq. (2), and is deployed in major Meta Ads models with 1.2% topline improvement. The paper also contains ablations, complexity analysis, and kernel implementation details.

Significance. If the scaling-law claim were supported, this would be a substantial contribution: it would provide a practical architecture and a predictive relationship for allocating compute in large-scale recommender systems. The engineering contributions — in particular the GDPA kernel design, the HSP/SumKronLinear compression, and the CompSkip ablations — are plausible and internally coherent, and the within-scale ablation results are informative. However, the central scientific claim is not established. The one sentence in the Figure 3 caption that NEs are not comparable across scales invalidates the cross-scale curve fitting on which the power-law and the 2× scaling-efficiency statements rest. The paper provides no independent held-out extrapolation, no error bars, and no reconciled numerical support for the 2× factor; Table 3 itself shows a 1.58× advantage at the largest scale. The architecture paper may be salvageable as an engineering study, but the scaling-law contribution as presented is unsupported.

major comments (3)
  1. [§5.2, Figure 3 caption] The caption explicitly states: 'NEs are not comparable across scales due to different feature configurations; within-scale results use identical setups.' Section 5.4 and Figure 4 nevertheless fit Eq. (2) to NE gains at 6, 60, and 180 GFLOPs as a single scaling curve. If the y-axis variable changes with scale because feature configurations differ, then the fitted slope η conflates architecture scaling with configuration changes. This is not a missing error bar or an extrapolation concern; it is the absence of a well-defined dependent variable for the central claim. The authors need either (i) a within-scale compute sweep that holds feature configuration fixed across all three compute budgets, or (ii) a direct, monotone transformation showing that the reported NE gains are comparable despite different feature sets. No such evidence is provided.
  2. [§3.3, Eq. (3); §5.4] Scaling efficiency is defined as η/η_baseline, where both η and η_baseline are estimated from fits to the same NE-versus-compute curves used to assert Eq. (2). There is no held-out prediction at an unobserved compute budget and no confidence interval on either slope. Moreover, the stated 2× claim is not supported by Table 3: at the 180 GFLOPs scale Kunlun attains 0.79% NE gain versus InterFormer's 0.50%, a 1.58× advantage, not 2×. Figure 4(left) uses a different set of points (e.g., 50/100/150/200 GFLOPs with gains +0.30/+0.45/+0.61/+0.80 and +0.16/+0.25/+0.33/+0.44) that do not match Table 3's 6/60/180 GFLOPs values. The paper must reconcile these numbers, disclose all points and fitted parameters, and provide an extrapolation test before the scaling-efficiency comparison can be evaluated.
  3. [§4.5 and §D; Figure 4 (right)] The paper asserts in multiple places that NE improvement from layer l to l+1 follows ΔNE_l ≈ c/log(l+1), and that horizontal expert scaling follows a power law. No per-layer NE values, fits, confidence intervals, or held-out layer-count predictions are reported. The right panel of Figure 4 shows four points with attached '0%', '0.10%', '0.16%', '0.24%', '0.31%', '0.37%' labels that are not clearly matched to a data table. Since the predictable-scaling contribution depends on this logarithmic depth law as much as on the compute law, the evidence is insufficient.
minor comments (5)
  1. [§3.3, Eq. (2)] Equation (2) writes NE(C) = NE_0 − η·log(C/C_0), but the paper never states whether C_0 and NE_0 are fixed constants, fitted parameters, or the Wukong baseline. This should be clarified, since Figure 3 reports 'NE gains' while Eq. (2) is in absolute NE.
  2. [Table 3] The Wukong baseline GFLOPs values (7.3, 57, 174.5) differ substantially from the 'approximately 6, 60, 180 GFLOPs' nominal budgets, and Kunlun's 154.9 GFLOPs at the 180 GFLOPs 'scale' is below the baseline. The paper should explain how compute scales were matched and whether the comparison is at fixed FLOPs or at nominally labeled scale.
  3. [§5.4 / Figure 4 (left)] Figure 4(left) appears to show additional data points not present in Table 3. Please state the exact source of those points, the fitted form, and the fitted η values for Kunlun and InterFormer.
  4. [Appendix A.2] MFU is defined with throughput and number of GPUs, but the text says MFU is measured on a single GPU and reflects only the architecture. Clarify whether the reported MFU numbers include distributed-system overheads or only the kernel-level utilization.
  5. [§5.5] The production deployment claim ('1.2% improvement in topline metrics') is stated without any measurement details or confidence intervals. If this claim is retained, it should specify the metric, the duration, and the statistical methodology.

Circularity Check

2 steps flagged

Scaling-law claim is a self-referential fit: 2× efficiency is the fitted slope ratio over cross-scale NE points the paper itself calls incomparable.

specific steps
  1. fitted input called prediction [Section 3.3 (Eq. 3) and Section 5.4 (Scaling Efficiency Comparison)]
    "Scaling Efficiency= η / η_baseline = ∆NE/log(C/C0) / η_baseline (3) ... For example, 2X scaling efficiency means the model achieves 2× the NE improvement per unit of log-scaled compute compared to baseline—visualized as a 2× steeper slope in the NE vs. log(Compute) curve. ... Kunlun achieves 2× scaling efficiency over state-of-the-art approaches—doubling the rate at which performance improves with additional compute compared to InterFormer."

    The reported 2× is not a held-out prediction or an independent measurement. Equation 3 defines scaling efficiency as the ratio of slopes η/η_baseline, and the slopes η are fitted to the same NE-vs-GFLOPs points shown in Figures 3 and 4 (left). Saying Kunlun has 2× scaling efficiency is therefore an algebraic restatement of the fitted slopes, not a test of predictive power. Section 5.4's claim that Kunlun 'exhibits predictable power-law scaling behavior consistent with Equation 2' is also self-referential: the data were fit to Equation 2, and no extrapolation to an unseen compute budget, confidence interval, or independent check is provided.

  2. other [Figure 3 caption and Section 5.4 (Scaling Law Curves)]
    "Note: NEs are not comparable across scales due to different feature configurations; within-scale results use identical setups. ... Kunlun exhibits predictable power-law scaling behavior consistent with Equation 2, with the scaling coefficient approximately 2× larger than InterFormer."

    The scaling coefficient η is estimated from the NE gains at 6, 60, and 180 GFLOPs. The caption explicitly admits that the NE values at those scales are not comparable because the feature configurations differ. Thus the cross-scale 'slope' in Figure 4 (left) confounds compute-driven improvements with configuration changes, so the claimed 2× coefficient is not a well-defined measure of scaling efficiency. The paper provides no evidence that the comparability required for the fit holds, making the power-law claim unmeasured by these points.

full rationale

The engineering contributions—MFU, QPS, ablations, and within-scale architecture comparisons—are measured with fixed setups and are not circular. The circularity is concentrated in the scaling-law claim. Equation 3 defines scaling efficiency as η/η_baseline, and the 2× value reported in the abstract, Section 1, and Section 5.4 is the ratio of slopes fitted to the same NE-vs-GFLOPs points. This is a fitted parameter reported as a finding, not a prediction. Moreover, Figure 3's caption states that NE values at different scales are not comparable due to different feature configurations, so the cross-scale slope is not a well-defined performance-per-compute measure. No held-out extrapolation or independent check supports the 'predictable power-law' claim. Thus the flagship scaling-law and double-efficiency claims reduce to a self-referential fit on incomparable points, while the within-scale results remain independent evidence for the architecture's efficiency. Score 6 reflects partial circularity in the central claim, not the whole paper.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

No new physical or ontological entities are postulated; all introduced components are parameterized neural modules. The central claim instead relies on several domain assumptions: power-law transfer from LLMs, locality bias, and especially cross-scale comparability of NE, which the paper itself contradicts.

free parameters (6)
  • scaling coefficient η (Kunlun and baselines) = not reported; derived from 3-point NE-vs-GFLOPs fit
    Equation 2; the 2× scaling-efficiency claim is the ratio of fitted η values. No confidence interval or extrapolation check is provided.
  • baseline scaling coefficient η_baseline = not reported
    Equation 3 defines scaling efficiency as η/η_baseline; the baseline slope is itself fitted from the same comparison runs.
  • GDPA temperature τ = maxlen(seq)
    Section 4.3.1: chosen because maxlen(seq) 'works better than embedding dimension d or sqrt(d) empirically'.
  • per-event config (d_model, heads, tokens, L, w) = example: click d_model=256, heads=8; impression d_model=128
    Section 4.4.2 hand-assigned by event importance; directly changes compute allocation and NE.
  • HSP/SumKronLinear hyperparameters (n_seeds, n_tokens, k) = k=8 in one example; n_seeds/n_tokens not fully specified
    Affect compression capacity and parameter count; chosen by design, not learned from data.
  • constant c in ΔNE_l ≈ c/log(l+1) = not reported
    Section 4.5 / Appendix D; used to assert predictable depth scaling, but c is fitted to the layer curve.
axioms (5)
  • ad hoc to paper NE gains vs total compute follow a single power law across different feature configurations
    Required for Equation 2; Figure 3 caption states NEs are not comparable across scales, so this assumption is contradicted by the paper's own text.
  • domain assumption Locality bias: recent interactions are more predictive, so sliding-window attention preserves quality
    Section 4.3.3 cites prior observation; used to justify O(Tw) complexity reduction. If false, NE-neutrality of windowing fails.
  • domain assumption MFU is a valid proxy for the computational factor in scaling efficiency and is measured independently of system overhead
    Appendix A.2; the 17%→37% MFU claim is central to the barrier diagnosis but is hardware/implementation-specific.
  • domain assumption Power-law functional form from LLM scaling laws transfers to CTR prediction with heterogeneous features
    Section 3.3 adopts Equation 2 from Kaplan/Hoffmann without derivation for joint sequence-context recommendation.
  • domain assumption Every-other-layer skipping preserves expressiveness (layer redundancy)
    Section 4.4.1 cites LLM layer-redundancy findings (Men et al.; Gemma) as justification for CompSkip in recommender models.

pith-pipeline@v1.3.0-alltime-deepseek · 18567 in / 13710 out tokens · 133300 ms · 2026-08-03T02:35:55.471034+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design." pith.science (2026). https://pith.science/paper/GM3CDGJU

@misc{pith2026260210016,
  author       = {Pith},
  title        = {Pith review of: Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GM3CDGJU}},
  note         = {Machine review of arXiv:2602.10016}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Deriving predictable scaling laws that govern the relationship between model performance and computational investment is crucial for designing and allocating resources in massive-scale recommendation systems. While such laws are established for large language models, they remain challenging for recommendation systems, especially those processing both user history and context features. We identify poor scaling efficiency as the main barrier to predictable power-law scaling, stemming from inefficient modules with low Model FLOPs Utilization (MFU) and suboptimal resource allocation. We introduce Kunlun, a scalable architecture that systematically improves model efficiency and resource allocation. Our low-level optimizations include Generalized Dot-Product Attention (GDPA), Hierarchical Seed Pooling (HSP), and Sliding Window Attention. Our high-level innovations feature Computation Skip (CompSkip) and Event-level Personalization. These advances increase MFU from 17% to 37% on NVIDIA B200 GPUs and double scaling efficiency over state-of-the-art methods. Kunlun is now deployed in major Meta Ads models, delivering significant production impact.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation

    cs.LG 2026-07 conditional novelty 7.0

    ROCS restructures recommendation models so user-side computation is shared across all candidate items, yielding up to 3x serving throughput at equal or better prediction quality.

  2. TokenFormer: Unify the Multi-Field and Sequential Recommendation Worlds

    cs.IR 2026-04 unverdicted novelty 7.0

    TokenFormer unifies multi-field and sequential recommendation modeling via bottom-full-top-sliding attention and non-linear interaction representations to avoid sequential collapse and deliver state-of-the-art performance.

  3. ClockRoPE: Random Fourier Rotations for Temporal Routine Modeling

    cs.LG 2026-07 conditional novelty 6.0

    Random Fourier Rotations let transformer position encodings approximate any positive-definite attention kernel; ClockRoPE applies this to model daily/weekly routines in sequential recommendation.

  4. Bumblebee: Interleaved Mixed-Layer Building Blocks for Large-Scale Recommendation Systems

    cs.IR 2026-07 conditional novelty 6.0

    Interleaving sequence modeling with feature interaction in repeated blocks improves recommendation accuracy by about 0.2-1.4% NE over sequential baselines at matched parameter counts.

  5. Dual-Stream MLP is All You Need for CTR Prediction

    cs.IR 2026-06 unverdicted novelty 6.0

    DS-MLP achieves state-of-the-art CTR prediction on three benchmarks using a final vanilla MLP structure trained via knowledge distillation and two alignment strategies.

  6. WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture

    cs.IR 2026-07 conditional novelty 5.0

    Combining Wukong feature-interaction layers with HSTU sequence layers in a per-layer attention-fused architecture improves large-scale recommendation quality over either backbone alone and is deployed online.

  7. WHALE: A Scalable Unified Model for Recommendation with Wukong-HSTU Architecture

    cs.IR 2026-07 conditional novelty 5.0

    Layer-wise attention fusion between Wukong-style feature crosses and HSTU-style behavior history improves recommendation quality over each backbone alone at matched FLOPs.

Reference graph

Works this paper leans on

12 extracted references · 7 linked inside Pith · cited by 6 Pith papers

  1. [1]

    Lan- guage models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877–1901,

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners.Advances in Neural Information Processing Systems, 33:1877–1901,

    1901

  2. [5]

    Scaling laws for neural language models.arXiv preprint arXiv:2001.08361,

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361,

    Pith/arXiv arXiv 2001

  3. [7]

    Factorization machines.2010 IEEE In- ternational Conference on Data Mining, pages 995–1000,

    Steffen Rendle. Factorization machines.2010 IEEE In- ternational Conference on Data Mining, pages 995–1000,

    2010

  4. [11]

    One- trans: Unified feature interaction and sequence modeling with one transformer in industrial recommender.arXiv preprint arXiv:2510.26104,

    Zhaoqi Zhang, Haolei Pei, Jun Guo, Tianyu Wang, Yufei Feng, Hui Sun, Shaowei Liu, and Aixin Sun. One- trans: Unified feature interaction and sequence modeling with one transformer in industrial recommender.arXiv preprint arXiv:2510.26104,

    arXiv

  5. [12]

    to Rotary Temporal Embed- dings (ROTE) for recommendation sequences. Unlike LLMs where token position is the primary ordering signal, recommendation sequences exhibit temporal pat- terns where thetime gapbetween events carries crucial information—a click from yesterday is fundamentally different from a click from last month, even if they occupy adjacent p...

    2024

  6. [2014]

    Training compute-optimal large lan- guage models.arXiv preprint arXiv:2203.15556,

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large lan- guage models.arXiv preprint arXiv:2203.15556,

    Pith/arXiv arXiv

  7. [2016]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebas- tian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, YiTay, NoamShazeer, VinodkumarPrabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbu...

    Pith/arXiv arXiv

  8. [2017]

    Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems.Proceedings of the Web Confer- ence 2021, pages 1785–1797,

    Ruoxi Wang, Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems.Proceedings of the Web Confer- ence 2021, pages 1785–1797,

    2021

  9. [2018]

    Shortgpt: Layers in large language models are more redun- dant than you expect.arXiv preprint arXiv:2406.15786,

    Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. Shortgpt: Layers in large language models are more redun- dant than you expect.arXiv preprint arXiv:2406.15786,

    Pith/arXiv arXiv

  10. [2022]

    https://doi.org/10.48550/arXiv.2204.02311

    doi: 10.48550/ARXIV.2204.02311. https://doi.org/10.48550/arXiv.2204.02311. Tri Dao, Daniel Y Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. InAdvances in Neural Information Processing Systems, volume 35, pages 16344–16359,

    Pith/arXiv arXiv doi:10.48550/arxiv.2204.02311

  11. [2024]

    Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sber, James Bber, Alexandre Cha- bal, Diego Baldassarre, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118,

    Pith/arXiv arXiv

  12. [2025]

    Interformer: To- wardseffectiveheterogeneousinteractionlearningforclick- through rate prediction.arXiv preprint arXiv:2411.09852,

    Zhichen Zeng, Xiaolong Liu, Mengyue Hang, Xiaoyi Liu, Qinghai Zhou, Chaofei Yang, Yiqun Liu, Yichen Ruan, Laming Chen, Yuxin Chen, et al. Interformer: To- wardseffectiveheterogeneousinteractionlearningforclick- through rate prediction.arXiv preprint arXiv:2411.09852,

    Pith/arXiv arXiv