Pith. sign in

REVIEW 3 major objections 5 minor 32 references

COMET: A Framework for Modeling Compound Operation Dataflows with Explicit Collectives

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read COMET claims that ignoring collective communication in compound-operation dataflow search leaves large speedups on the table, and it supplies a cost model that makes those collectives explicit.

desk verdict A useful extension of TileFlow with explicit collectives, but Eq. 2's stall term is under-specified and the headline speedups are unvalidated simulator predictions. read the letter →

arxiv 2509.00599 v1 pith:LACTIHMN submitted 2025-08-30 cs.AR cs.DC

Shubham Negi , Manik Singhal , Aayush Ankit , Sudeep Bhoja , Kaushik Roy This is my paper · ORCID
classification cs.ARcs.DC
keywords compoundoperationscollectivecommunicationdataflowoptimizationMLacceleratorscostmodeloperatorfusionAll-Reduceself-attention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

COMET claims that existing dataflow optimizers for ML accelerators miss real performance because they treat fusion and collective communication (All-Reduce, All-Gather) separately or ignore collectives entirely. The paper introduces a representation in which every tensor has its own loop nest at each memory level and collective operations appear as explicit nodes, so a mapping can trade off where and when communication happens against fusion. Its stall-aware cost model adds ramp-up/ramp-down and resource-conflict latencies that earlier single-operator models ignore. On GEMM-Softmax, GEMM-LayerNorm, and self-attention workloads across edge and cloud configurations, the model finds fused mappings with geometric-mean speedups of 1.42x, 3.46x, and 1.82x over unfused baselines. The point is that whether fusion helps depends on the operation's shape and the platform, and the framework exists to make that tradeoff visible.

What carries the argument

The central object is the explicit collective representation: a hierarchical mapping tree in which each tensor at each memory level has its own loop nest (tile node), and peer-to-peer synchronization is represented by collective operation nodes annotated with collective type, tensor, reduction operator, and source/destination memories. It is paired with an iteration-level latency model, Lat(Tn) = N*MW + CS + OS, where MW is the memory window (compute time or child-node time), CS is the compulsory ramp-up/ramp-down stall, and OS is the optional stall when transfer time exceeds the window; collective latency adds a NoC term (router hops plus queueing and serialization) computed with recursive

What would settle it

Measure the same GEMM-Softmax, GEMM-LayerNorm, and self-attention mappings on the described edge and cloud accelerators, or on a cycle-accurate simulator with hardware counters. If the unfused-versus-fused latency ratios do not reproduce the predicted 1.42x, 3.46x, and 1.82x geometric means, or if the ranking of distributed versus non-distributed softmax flips, the cost model's central claim is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that collective communication cost can and should be modeled explicitly inside compound-operation dataflow search. COMET represents a mapping as a tree of tile nodes (data movement per tensor per memory level) and collective operation nodes annotated with type, tensor, reduction operator, and source/destination memory levels. The cost model accounts for per-iteration memory windows, compulsory stalls from initial fill and final drain, optional stalls when transfer time exceeds the window, and conflict stalls when fused children share memory. Using this model, the paper shows that for GEMM-Softmax and GEMM-LayerNorm, the best fusion strategy depends on tensor shap

Load-bearing premise

The whole evaluation rests on a timing model rather than measured hardware: if the analytic equations for memory stalls and NoC latency do not match a real accelerator's cycles, the claimed speedups could be wrong.

Editorial extensions

If this is right

  • Dataflow search for compound operations should include collective operations as first-class choices; restricting collectives or ignoring them misses latency differences of roughly 1.4x to 3.5x versus unfused execution.
  • The optimal fusion strategy is workload- and platform-dependent: distributed softmax wins when SIMD latency dominates on edge, while standard gather-based softmax wins on cloud where collective overhead dominates.
  • Fusing all elementary operations is not always best; the cost model identifies cases where keeping softmax or layernorm local beats full distribution.
  • The latency breakdown attributes time to collectives, compulsory stalls, optional stalls, GEMM, and SIMD units, giving accelerator designers a concrete target: improving non-GEMM throughput matters as much as GEMM throughput for fused compound operations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the representation is parameterized by memory-level source and destination lists, it could also describe collectives on other topologies, such as rings or torus networks, without changing the cost formulation.
  • The same machinery could be applied to training workloads and data-parallel pipelines where All-Reduce frequency scales with batch and sequence length; the reported speedups are inference-only.
  • A natural extension is to couple COMET's cost model with an automated or learned search strategy rather than the iterative search used here; the paper notes this as future work.
  • The distributed-versus-local softmax comparison suggests a testable design rule: choose collective placement by weighing SIMD latency against NoC bandwidth times data volume, which could become a roofline-style criterion for fused compound operations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes COMET, a framework for modeling and optimizing the dataflow of compound operations (e.g., GEMM-Softmax, GEMM-LayerNorm, self-attention) on multi-cluster ML accelerators. COMET contributes an intermediate representation that assigns per-tensor loop nests and explicitly represents collective communication nodes, together with a latency/energy cost model that augments prior analytical models (Timeloop, TileFlow) with data-staging stalls and NoC-based collective latencies. The evaluation uses an in-house C++ simulator on 12 representative workloads across edge and cloud configurations, reporting geometric mean speedups of 1.42x, 3.46x, and 1.82x versus unfused baselines, and comparing latency/energy estimates against Timeloop and TileFlow. The central claim is that the explicit collective representation plus stall-aware costing exposes a broader mapping space and yields better fusion/communication decisions.

Significance. If the cost model is accurate, COMET fills a genuine gap: prior dataflow frameworks focus on single operations or on fusion without explicitly modeling spatial collective communication, and they typically assume a single compute-unit type. The per-tensor loop nests and the explicit collective nodes are a useful representational extension, and the case studies do show qualitatively different mapping tradeoffs (e.g., distributed vs. standard softmax) that are not visible in Timeloop/TileFlow-style representations. The main strength is the clear articulation of the modeling equations and the comparison against two established analytical frameworks. However, the paper's quantitative claims rest entirely on an unvalidated analytical cost model; the simulator is not released, and the comparisons with Timeloop/TileFlow share core assumptions. Thus the contribution is significant only conditionally on the cost model being credible, which the current evidence only partially supports.

major comments (3)
  1. [Section IV-B, Eq. (2) and Fig. 5] The optional-stall term OS is under-specified. If MemLat (Eq. 1) exceeds the memory window MW, then with double-buffered memories the child memory cannot be refilled during one compute window, so the stall recurs on every subsequent iteration, not just once. The formula Lat(Tn)=N*MW+CS+OS adds OS only once, and the text never states whether OS is per-iteration or total. Figure 5 shows stalls appearing inside steady-state windows, implying repeated stalls. The corrected steady-state form should be approximately N*max(MW, MemLat) plus boundary effects, i.e., N*MW + CS + N*OS when MemLat > MW. This ambiguity is load-bearing because unfused baselines perform many more DRAM-to-buffer transfers than fused mappings, so the reported 1.42x/3.46x/1.82x speedups depend on how OS is accumulated. Please clarify the definition and recompute the results if the implementation used a single OS.
  2. [Section V-B, Fig. 6] The model-to-model comparisons are presented only as scatterplots without correlation coefficients, absolute errors, or error bars. More importantly, Timeloop and TileFlow share the same double-buffered dual-port and possibly the same memory-window assumptions as COMET, so agreement with them is not an independent validation of the core timing model. There is no cycle-accurate simulation or hardware measurement in Section V. I request: (a) quantitative fit metrics (e.g., mean absolute percentage error, R^2) for Fig. 6; and (b) at least a small cycle-accurate cross-check, or a sensitivity analysis that varies the OS and MW modeling assumptions and shows the headline speedups are robust.
  3. [Section V-A, reproducibility] The simulator is described but no artifact URL, code, or configuration files are provided. Since every central result is the output of this simulator, the absence of released code and data makes the quantitative claims non-reproducible. Please include an artifact availability statement and release the simulator, mapping instances, and architecture YAML files, or at minimum a sufficient subset to reproduce the three speedup numbers.
minor comments (5)
  1. [Abstract] The abstract reads 'we propose, a framework' — the name 'COMET' is missing after 'propose'.
  2. [Section IV-B, Eq. (3)] In NoCLat = trouter*hops + tenq*(DV/W), the second term appears to multiply a queuing delay (time) by a transfer time (time), which is dimensionally inconsistent unless tenq has unusual units. Please define tenq's units and clarify the expression.
  3. [Section IV-B] Typo: 'pipelied' should be 'pipelined'.
  4. [Figures 8 and 9] The phrase 'In the paper' appears inside the plot panels; this looks like a leftover annotation artifact and should be removed or explained.
  5. [Section V-A] The search uses up to 10,000 iterations with no convergence plots or optimality gap analysis. Since the mapper is not the focus, a short statement about search stability across seeds/initializations would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: COMET's speedups are produced by an independent analytical cost model; no fitted parameters are renamed as predictions, and the sole self-citation is illustrative.

full rationale

COMET's central claim is that its explicit collective representation plus stall-aware cost model finds better fusion/communication mappings. The cost model (Eqs. 1-7) is built from stated hardware parameters (bandwidth, NoC width, hop counts, memory windows) and external component models (CACTI, DRAMPower, SCALE-Sim, HISIM). No parameter is fitted to the reported 1.42x/3.46x/1.82x speedups; those speedups are outputs of the model when comparing fused vs unfused mappings. The validation against Timeloop/TileFlow is a model-to-model correlation check and shares some assumptions, but that is a weak-external-benchmark issue, not circularity by construction. The only self-citation is [19] in Section II, where compute-in-memory is mentioned as one possible GEMM unit implementation; this is illustrative and not load-bearing for the derivation. The potential under-specification of the optional-stall term OS in Eq. 2 (whether OS is per-window or total) is a modeling accuracy/correctness risk, not a circular reduction: it does not make the predicted speedups equal to the model inputs by construction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The model depends on several domain assumptions inherited from prior tools and one original scheduling formula. No parameters are fitted to the reported speedups, but the central numbers inherit the accuracy of these assumptions. No new physical entities are introduced; the collective operation node is a software representation, not a postulated object with independent falsifiable evidence.

assumptions (4)
  • domain assumption All on-chip memories are double buffered and dual port.
    Stated in Section IV-B; this enables overlap of memory transactions with compute and directly shapes the compulsory and optional stall terms. If the hardware is single-buffered, latencies could be significantly higher.
  • domain assumption The HISIM NoC latency model (Eq. 3) and the recursive doubling/halving collective algorithm correctly describe collective latency and data volume.
    Adopted from references [29] and [30] in Section IV-B; the paper provides no independent validation of this model on the target architectures.
  • domain assumption Component energies and latencies from CACTI, DRAMPower, SCALE-Sim, and Orion are accurate for the modeled edge and cloud systems.
    Used in Section V-A without sensitivity analysis; energy comparisons in Figures 9, 11, and 14 depend on these external calibrations.
  • ad hoc to paper The scheduling latency of fused children is modeled by Eqs. 5-7 (sequential or max-plus-conflict-stall).
    This is COMET's own model choice; it is plausible but not derived from first principles or verified against hardware.

how reviews work

0 comments
Cite this review

Pith. "Pith review of COMET: A Framework for Modeling Compound Operation Dataflows with Explicit Collectives." pith.science (2026). https://pith.science/paper/LACTIHMN

@misc{pith2026250900599,
  author       = {Pith},
  title        = {Pith review of: COMET: A Framework for Modeling Compound Operation Dataflows with Explicit Collectives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LACTIHMN}},
  note         = {Machine review of arXiv:2509.00599}
}
abstract

Modern machine learning accelerators are designed to efficiently execute deep neural networks (DNNs) by optimizing data movement, memory hierarchy, and compute throughput. However, emerging DNN models such as large language models, state space models increasingly rely on compound operations-structured compositions of multiple basic operations-which introduce new challenges for dataflow optimization and minimizing off-chip memory traffic. Moreover, as model size continues to grow, deployment across spatially distributed compute clusters becomes essential, requiring frequent and complex collective communication. Existing dataflow optimization frameworks and performance models either focus on single operations or lack explicit modeling of collective communication cost, limiting their applicability to modern workloads. To address these limitations, we propose, a framework for modeling and optimizing dataflow for compound operations on machine learning accelerators. COMET introduces a novel representation that explicitly models collective communication across spatial clusters, along with latency and energy cost models that account for both GEMM and non-GEMM operation level dependencies within compound operations. We demonstrate COMET's capabilities to analyze and optimize dataflows for compound operations such as GEMM--Softmax, GEMM--LayerNorm, and self-attention, across both edge and cloud accelerator configurations. Our collective-aware modeling enables exploration of a broader mapping space, leading to improved performance and energy efficiency. Specifically, our optimized dataflows achieve up to 1.42$\times$ speedup for GEMM-Softmax, 3.46$\times$ for GEMM-LayerNorm and 1.82$\times$ for self-attention compared to unfused baselines.

Figures

Figures reproduced from arXiv: 2509.00599 by the authors.

Figure 1
Figure 1. (a) Dataflow design alternatives. (b) Types of collective operations. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Illustration of operations in FlashAttention [6]. (b) DNN Accelerator. (c) LoopNest representation for a GEMM operation showing implicit [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. COMET’s workflow overview layers in Transformer models, proposing to retain a block of data in on-chip memory to efficiently support the softmax operator. TileFlow [32] presents a tree-based analysis to model and search for efficient dataflows for compound operations. LoopTree [9] explores the tradeoff between recomputation and retention of intermediate tensors to further reduce on-chip buffer requirements when exec… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: (a) A typical compound operation consisting of a GEMM followed by a Softmax, where the Softmax is decomposed into smaller elementary operations [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Timing diagram illustrating data movement from a parent memory to [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Comparison of COMET’s energy and latency with prior frameworks. (a) and (b) compare energy and latency with Timeloop, respectively; (c) and (d) [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Latency and energy variation for different dataflows: (a) Latency and (b) Energy for GEMM-Softmax, (c) Latency and (d) Energy for GEMM [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: (a) Normalized latency breakdown for GEMM-Softmax compound operation under distSM and SM mapping, (b) Normalized latency breakdown for [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: (a) Normalized energy breakdown for GEMM-Softmax compound operation under distSM and SM mapping, (b) Normalized energy breakdown for [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Normalized Latency (lower is better) of different fusion mapping compared to the unfused baseline for (a) GEMM-Softmax and (b) GEMM [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Normalized Energy (lower is better) of different fusion mapping compared to the unfused baseline for (a) GEMM-Softmax and (b) GEMM-LayerNorm Latency and energy impr [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]
Figure 12
Figure 12. Figure 12: Comparison of normalized (a) latency and (b) energy for different attention variants, including Unfused, Partially Fused, and Flash Attention. Lower [PITH_FULL_IMAGE:figures/full_fig_p009_12.png]
Figure 13
Figure 13. Figure 13: Normalized latency breakdown for different attention variants. Attn1–6 are deployed on the edge platform, while Attn7–12 are deployed on the cloud [PITH_FULL_IMAGE:figures/full_fig_p010_13.png]
Figure 14
Figure 14. Figure 14: Normalized energy breakdown for different attention variants. Attn1–6 are deployed on the edge platform, while Attn7–12 are deployed on the cloud [PITH_FULL_IMAGE:figures/full_fig_p010_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 24 canonical work pages

  1. [1]

    Fused-layer cnn accelerators,

    M. Alwani, H. Chen, M. Ferdman, and P. Milder, “Fused-layer cnn accelerators,” in 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 2016, pp. 1–12

    work page 2016

  2. [2]

    Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale,

    R. Y . Aminabadi, S. Rajbhandari, A. A. Awan, C. Li, D. Li, E. Zheng, O. Ruwase, S. Smith, M. Zhang, J. Rasley et al., “Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale,” in SC22: International Conference for High Performance Com- puting, Networking, Storage and Analysis . IEEE, 2022, pp. 1–15

    work page 2022

  3. [3]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  4. [4]

    Improved power modeling of ddr sdrams,

    K. Chandrasekar, B. Akesson, and K. Goossens, “Improved power modeling of ddr sdrams,” in 2011 14th Euromicro Conference on Digital System Design. IEEE, 2011, pp. 99–108

    work page 2011

  5. [5]

    Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices,

    Y .-H. Chen, T.-J. Yang, J. Emer, and V . Sze, “Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 2, pp. 292–308, 2019

    work page 2019

  6. [6]

    Flashattention: Fast and memory-efficient exact attention with io-awareness,

    T. Dao, D. Fu, S. Ermon, A. Rudra, and C. R ´e, “Flashattention: Fast and memory-efficient exact attention with io-awareness,” Advances in neural information processing systems , vol. 35, pp. 16 344–16 359, 2022

    work page 2022

  7. [7]

    Tandem processor: Grappling with emerging operators in neural networks,

    S. Ghodrati, S. Kinzer, H. Xu, R. Mahapatra, Y . Kim, B. H. Ahn, D. K. Wang, L. Karthikeyan, A. Yazdanbakhsh, J. Park et al., “Tandem processor: Grappling with emerging operators in neural networks,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, 2024, pp. 1165–1182

    work page 2024

  8. [8]

    Ai and memory wall,

    A. Gholami, Z. Yao, S. Kim, C. Hooper, M. W. Mahoney, and K. Keutzer, “Ai and memory wall,” IEEE Micro, 2024

    work page 2024

Show all 32 references
  1. [9]

    Looptree: Enabling exploration of fused-layer dataflow accelerators,

    M. Gilbert, Y . N. Wu, A. Parashar, V . Sze, and J. S. Emer, “Looptree: Enabling exploration of fused-layer dataflow accelerators,” in 2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 2023, pp. 316–318

    2023

  2. [10]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

    2023 arXiv

  3. [11]

    Data movement is all you need: A case study on optimizing transformers,

    A. Ivanov, N. Dryden, T. Ben-Nun, S. Li, and T. Hoefler, “Data movement is all you need: A case study on optimizing transformers,” Proceedings of Machine Learning and Systems , vol. 3, pp. 711–732, 2021

    2021

  4. [12]

    In-datacenter performance analysis of a tensor processing unit,

    N. P. Jouppi, C. Young, N. Patil, D. Patterson, G. Agrawal, R. Bajwa, S. Bates, S. Bhatia, N. Boden, A. Borchers et al. , “In-datacenter performance analysis of a tensor processing unit,” in Proceedings of the 44th annual international symposium on computer architecture , 2017...

    2017

  5. [13]

    Orion 2.0: A fast and accurate noc power and area model for early-stage design space exploration,

    A. B. Kahng, B. Li, L.-S. Peh, and K. Samadi, “Orion 2.0: A fast and accurate noc power and area model for early-stage design space exploration,” in 2009 Design, Automation & Test in Europe Conference & Exhibition. IEEE, 2009, pp. 423–428

    2009

  6. [14]

    Flat: An optimized dataflow for mitigating attention bottlenecks,

    S.-C. Kao, S. Subramanian, G. Agrawal, A. Yazdanbakhsh, and T. Kr- ishna, “Flat: An optimized dataflow for mitigating attention bottlenecks,” in Proceedings of the 28th ACM International Conference on Archi- tectural Support for Programming Languages and Operating Systems, Vol...

    2023

  7. [15]

    Nongemm bench: Understanding the performance horizon of the latest ml workloads with nongemm workloads,

    R. Karami, S.-C. Kao, and H. Kwon, “Nongemm bench: Understanding the performance horizon of the latest ml workloads with nongemm workloads,” arXiv preprint arXiv:2404.11788 , 2024

    2024 arXiv

  8. [16]

    Full stack optimization of transformer inference: a survey,

    S. Kim, C. Hooper, T. Wattanawong, M. Kang, R. Yan, H. Genc, G. Dinh, Q. Huang, K. Keutzer, M. W. Mahoney et al. , “Full stack optimization of transformer inference: a survey,” arXiv preprint arXiv:2302.14017, 2023

    2023 arXiv

  9. [17]

    Maestro: A data-centric approach to understand reuse, performance, and hardware cost of dnn mappings,

    H. Kwon, P. Chatarasi, V . Sarkar, T. Krishna, M. Pellauer, and A. Parashar, “Maestro: A data-centric approach to understand reuse, performance, and hardware cost of dnn mappings,” IEEE micro, vol. 40, no. 3, pp. 20–29, 2020

    2020

  10. [18]

    Cacti 6.0: A tool to understand large caches,

    N. Muralimanohar, R. Balasubramonian, and N. P. Jouppi, “Cacti 6.0: A tool to understand large caches,” University of Utah and Hewlett Packard Laboratories, Tech. Rep, vol. 147, 2009

    2009

  11. [19]

    Hcim: Adc-less hy- brid analog-digital compute in memory accelerator for deep learning workloads,

    S. Negi, U. Saxena, D. Sharma, and K. Roy, “Hcim: Adc-less hy- brid analog-digital compute in memory accelerator for deep learning workloads,” in Proceedings of the 30th Asia and South Pacific Design Automation Conference, 2025, pp. 648–655

    2025

  12. [20]

    Timeloop: A systematic approach to dnn accelerator evaluation,

    A. Parashar, P. Raina, Y . S. Shao, Y .-H. Chen, V . A. Ying, A. Mukkara, R. Venkatesan, B. Khailany, S. W. Keckler, and J. Emer, “Timeloop: A systematic approach to dnn accelerator evaluation,” in 2019 IEEE inter- national symposium on performance analysis of systems and soft...

    2019

  13. [21]

    Optimiz- ing distributed ml communication with fused computation-collective operations,

    K. Punniyamurthy, K. Hamidouche, and B. M. Beckmann, “Optimiz- ing distributed ml communication with fused computation-collective operations,” in SC24: International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 2024, pp. 1–17

    2024

  14. [22]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  15. [23]

    Scale-sim: Systolic cnn accelerator simulator,

    A. Samajdar, Y . Zhu, P. Whatmough, M. Mattina, and T. Kr- ishna, “Scale-sim: Systolic cnn accelerator simulator,” arXiv preprint arXiv:1811.02883, 2018

    2018 arXiv

  16. [24]

    Benchmarking in-memory computing architectures,

    N. R. Shanbhag and S. K. Roy, “Benchmarking in-memory computing architectures,” IEEE Open Journal of the Solid-State Circuits Society , vol. 2, pp. 288–300, 2022

    2022

  17. [25]

    Scaling equations for the accurate prediction of cmos device performance from 180 nm to 7 nm,

    A. Stillmaker and B. Baas, “Scaling equations for the accurate prediction of cmos device performance from 180 nm to 7 nm,” Integration, vol. 58, pp. 74–81, 2017

    2017

  18. [26]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  19. [27]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  20. [28]

    How to keep pushing ml accelerator performance? know your rooflines!

    M. Verhelst, L. Benini, and N. Verma, “How to keep pushing ml accelerator performance? know your rooflines!” IEEE Journal of Solid- State Circuits, 2025

    2025

  21. [29]

    Hisim: Analytical performance modeling and design space exploration of 2.5 d/3d integration for ai computing,

    Z. Wang, P. S. Nalla, J. Sun, A. A. Goksoy, S. K. Mandal, J.-s. Seo, V . A. Chhabria, J. Zhang, C. Chakrabarti, U. Y . Ogras et al. , “Hisim: Analytical performance modeling and design space exploration of 2.5 d/3d integration for ai computing,” IEEE Transactions on Computer- ...

    2025

  22. [30]

    Tacos: Topology-aware collective algorithm synthesizer for distributed machine learning,

    W. Won, M. Elavazhagan, S. Srinivasan, S. Gupta, and T. Krishna, “Tacos: Topology-aware collective algorithm synthesizer for distributed machine learning,” in 2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 2024, pp. 856–870

    2024

  23. [31]

    Towards a standardized representation for deep learning collective algorithms,

    J. Yoo, W. Won, M. Cowan, N. Jiang, B. Klenk, S. Sridharan, and T. Krishna, “Towards a standardized representation for deep learning collective algorithms,” IEEE Micro, 2025

    2025

  24. [32]

    Tileflow: A framework for modeling fusion dataflow via tree-based analysis,

    S. Zheng, S. Chen, S. Gao, L. Jia, G. Sun, R. Wang, and Y . Liang, “Tileflow: A framework for modeling fusion dataflow via tree-based analysis,” in Proceedings of the 56th Annual IEEE/ACM International Symposium on Microarchitecture, 2023, pp. 1271–1288

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.