Pith. sign in

REVIEW 4 major objections 7 minor 37 references

Towards Easy and Realistic Network Infrastructure Testing for Large-scale Machine Learning

T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By replacing GPUs with CPU nodes that inject real network traffic while a workload simulator decides when, GENIE aims to make large-scale ML network testing cheap enough to run anywhere.

desk verdict Honest vision paper for GPU-free network testing, but the central fidelity claim is deferred and unvalidated. read the letter →

arxiv 2504.20854 v1 pith:W7LWWHA2 submitted 2025-04-29 cs.NI cs.AIcs.DCcs.SYeess.SY

Jinsun Yoo , ChonLam Lao , Lianjie Cao , Bob Lantz , Minlan Yu , Tushar Krishna , Puneet Sharma This is my paper · ORCID
classification cs.NIcs.AIcs.DCcs.SYeess.SY
keywords networkinfrastructuretestingdistributedMLtrainingGPU-freeemulationRDMAtrafficgenerationcollectivecommunicationworkloadgraphsimulationhardwaretestbedfaultdiagnosis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GENIE is a proposed testing framework for the network infrastructure of large-scale ML training clusters that does not require GPUs. Its core claim is that CPU nodes can generate the real RDMA traffic patterns of GPU-to-GPU communication, while a distributed simulation of the workload graph decides when each message is sent, so that genuine hardware behavior—congestion, switch buffer pressure, misconfiguration, or NIC degradation—shows up as measurable slowdowns in a full training workload. This matters because GPU clusters are too costly to reserve for network validation, and simulators cannot model proprietary or faulty hardware. The paper lays out the design and positions fidelity validation as the next step; no end-to-end measurements are presented yet.

What carries the argument

The carrying mechanism is a distributed, network-coupled version of the ASTRA-sim workload simulator serving as the "workload feeder," paired with a "traffic generator" acting as its network backend. The workload feeder reads a graph-based representation of the training job (operators as vertices, dependencies as edges), issues compute operations as timed sleeps, and asks the generator to send or receive each collective's constituent messages. The traffic generator translates those requests into real network calls through lower-level transport libraries, so the CPU NIC becomes the stand-in for GPU memory and GPUDirect RDMA. Message completion reported back to the simulator preserves the dependency-driven dynamics, making the real network the arbiter of workload progress.

What would settle it

Run one representative distributed training workload (for example, an AllReduce-heavy LLM or an all-to-all MoE layer) on a GPU cluster and on a CPU-only GENIE testbed with the same topology, NICs, and switches; induce the same network anomaly (a degraded link or disabled congestion control) and compare per-collective completion times, switch queue depths, and packet-drop counts. If the CPU emulation misses the microbursts or flow concurrency of GPU traffic, the slowdowns will diverge, and the fidelity premise fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that GENIE reproduces the interaction between an ML training workload and a real network fabric using only CPU nodes. Each GENIE instance plays the role of one training rank: it walks the operator graph of the workload, sleeps through compute durations, decomposes each collective into point-to-point messages, and injects those messages into the physical network through the NIC using standard transport libraries. Completion of each message is fed back into the simulator, so the dependency chain of the workload—and therefore its critical path—advances only as fast as the real network allows. Because switches cannot tell GENIE traffic from GPU training traffic, the real fabric's buffer behavior, congestion control, and failure modes become part of the experiment. The authors state this as a foundation to be built and validated, not as a measured result.

Load-bearing premise

CPU-initiated network messages with the same sizes and timing reproduce the network-level behavior of real GPU-originated traffic closely enough that congestion, buffer pressure, and fault-induced slowdowns still appear in workload performance.

Editorial extensions

If this is right

  • Operators can validate a physical network fabric—including proprietary transports like Slingshot, Infiniband, or Spectrum-X—before committing GPU time to a training run.
  • Misconfigurations and hardware degradations that simulators cannot predict, such as a disabled congestion-control setting or a degraded NIC, become visible as workload-level slowdowns rather than only as low-level counters.
  • Workload-level effects of collective algorithms, topology choices, and switch buffer behaviors can be studied on real hardware at far lower cost than building a GPU cluster.
  • Because GENIE instances coordinate only through the collectives themselves, no extra synchronization layer is imposed on the testbed, preserving the natural timing of the workload.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If CPU-emulated traffic reproduces GPU traffic's burstiness, the same testbed could be reused as a cheap fault-injection and regression platform: replay a workload graph while intentionally toggling link rates, buffer sizes, or routing policies, and measure which network changes move the critical path.
  • The fidelity question is testable directly: compare GENIE's per-collective completion times and switch queue occupancy against a GPU ground truth under induced incast or congestion; divergence would pinpoint exactly which traffic properties (microbursts, flow concurrency, PCIe pacing) the CPU emulation misses.
  • Because compute is only simulated as delays, GENIE could in principle scale testing to workloads far larger than any available GPU cluster, limited by CPU count and network ports rather than accelerator availability.
  • The dependency-driven message injection could be applied to debugging in reverse: given a production slowdown, replay the workload graph on the suspect fabric and bisect which network component accounts for the added latency.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This paper proposes GENIE, a framework for testing real network infrastructure for large-scale ML training without using GPUs. The design combines a distributed, modified ASTRA-sim as a 'workload feeder' that walks Chakra execution graphs and emulates compute/memory delays, with a 'traffic generator' that uses CPU-side RDMA libraries (libibverbs, libfabric, Gloo, perftest) to inject inter-node messages over the real network fabric. The paper motivates the need with a measurement showing that a packet-level simulator does not capture an unmodeled NIC degradation during repeated AllReduce calls (Figure 1), and it sketches GENIE's architecture and modular transport support. The authors state in Section 5 that the system has not yet been built and that validation of the CPU-to-GPU traffic fidelity is future work.

Significance. If the core assumption were validated, GENIE would be a valuable tool for network vendors and operators, lowering the cost of testing ML network infrastructure. The paper is clearly written, identifies a real problem, and proposes a modular design that leverages existing artifacts (Chakra, ASTRA-sim, Gloo) rather than building from scratch. It also makes a falsifiable hypothesis: CPU-generated traffic with the same message sizes and logical timing should yield the same network-induced workload delays as GPU-generated traffic. The explicit acknowledgement of the open fidelity question in Section 5 is honest. However, the manuscript presents no implementation, no traffic-fidelity comparison, and no end-to-end validation, so the central claim remains a design hypothesis.

major comments (4)
  1. [§4 ('Creating GPU Communication with Traffic Generator') and §5] The central claim of the paper, that GENIE 'captures the impact of real hardware network behavior on ML workload performance' (Abstract), rests entirely on the unvalidated assumption that CPU-initiated RDMA traffic reproduces the network-relevant properties of GPUDirect RDMA traffic. Section 4 states this as a key requirement and proposes using libibverbs, libfabric, Gloo, or perftest, but Section 5 explicitly lists 'carefully study and validate the fidelity of the GPU emulation' as future work. No measurement compares CPU-generated traffic with GPU-generated traffic in terms of burstiness, flow concurrency, queue depth, or switch buffer occupancy. Without such a comparison, the framework may miss the microburst and buffer-pressure effects that determine workload-level slowdowns, so the abstract's claim is unsupported.
  2. [§1 and Figure 1] The only experimental evidence in the paper, Figure 1, demonstrates that a packet-level simulator fails to capture an unmodeled NIC degradation during repeated AllReduce runs. This is motivational evidence for real-hardware testing, but it does not exercise GENIE in any way; therefore it cannot support the claim that GENIE captures network impact on workloads. The figure's relevance to the proposed design would need to be rebuilt around a comparison of GENIE's predictions against ground-truth GPU training runs.
  3. [§4 ('Modeling Workload with ASTRA-sim')] GENIE's workload-level performance is a hybrid of ASTRA-sim's simulated compute/memory durations and real network message completion times, but the paper does not validate this hybrid in the distributed, real-time setting. The graph reader progresses only when ASTRA-sim's collective library reports message completion, so the accuracy of the reported workload time depends on the fidelity of ASTRA-sim's compute/memory model, the correctness of the distributed simulation clock, and the synchronization across nodes. None of these are evaluated or even described in detail. The paper should either restrict its claim to network-level fidelity or provide an end-to-end comparison against a real GPU training run on the same workload.
  4. [§4 ('Modeling Workload with ASTRA-sim')] The paper assumes that ASTRA-sim's collective library produces the same message-level schedule as a real collective runtime such as NCCL. If the simulated chunking, pipeline depth, or collective algorithm differs from the production runtime, the resulting network traffic, and hence the measured network impact, will not match a real training job. No evidence is presented that the collective library's message decomposition matches NCCL's behavior on the target hardware, so this is a second load-bearing fidelity assumption that needs direct validation.
minor comments (7)
  1. [Figure 1] The figure legend and caption are unclear: the y-axis is labeled 'Bus BW (GB/s)', the caption says 'We inject an anomaly', and the text says 'unpredicted NIC degradation'; please clarify what was injected and what the simulated curve represents.
  2. [§4 ('Creating GPU Communication with Traffic Generator')] The phrase 'application agonistic manner' should read 'application-agnostic manner'.
  3. [References] Reference [23] lists 'OpenFabrics Allance'; this should be 'OpenFabrics Alliance'.
  4. [Abstract and Section 5] The Abstract uses present tense ('GENIE captures') while Section 5 states that the system has not been built; rephrase the Abstract to describe a design goal or a hypothesis, or provide the validation that would support the present-tense claim.
  5. [Footnote 1] The footnote dismisses intra-node communication, but many distributed training workloads use a mix of NVLink and network traffic; a sentence on how this scope decision affects the realism of the emulated inter-node traffic would help.
  6. [§4 ('Creating GPU Communication with Traffic Generator')] The claim that perftest 'can generate arbitrary point-to-point messages' is inaccurate: perftest is a fixed-pattern bandwidth/latency benchmark tool, not a library for arbitrary message patterns; please clarify the intended use.
  7. [§4 ('Modeling Workload with ASTRA-sim')] The workload performance metric is not precisely defined; please state whether it is wall-clock time per iteration, end-to-end training time for a fixed number of steps, or another quantity, and explain how it is read out from the distributed GENIE instances.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GENIE is a vision/design paper with no fitted predictions or equations, and its reliance on ASTRA-sim and CPU-emulation fidelity is an unvalidated soundness assumption, not a circular reduction.

full rationale

I walked the paper's derivation chain and found no equation, fitted parameter, or prediction that reduces to an input by construction. The paper is explicitly a vision/position paper: Section 5 says 'First we will build GENIE and carefully study and validate the fidelity of the GPU emulation.' The central claim is conditional on two unvalidated assumptions: (1) CPU-initiated RDMA traffic faithfully reproduces the network-relevant properties of GPUDirect RDMA traffic, and (2) ASTRA-sim's workload model correctly propagates real network delays. Both are soundness concerns, not circularity. There is no fitted value later renamed as a prediction, no definition of one quantity in terms of another equivalent quantity, and no uniqueness theorem imported from the authors' prior work. The self-citations to ASTRA-sim [36], Chakra [32], and related collective-algorithm work [37] are uses of prior tools by overlapping authors, but the paper does not argue that those tools are correct merely because they are cited; it adapts them as components and explicitly defers fidelity validation. Figure 1 motivates the need by showing a simulator gap on AllReduce, but GENIE is not evaluated against that or any other benchmark. Thus the core idea—using CPU traffic on real hardware while simulating workload dependencies—is an untested design hypothesis, not a result derived from its own inputs. No circular step meets the evidence bar in the review rules.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper adds no fitted parameters or new physical entities. It rests on a chain of modeling assumptions about simulator fidelity and CPU traffic equivalence, none of which are validated in the preprint. The only new entity is the GENIE framework itself, which is the subject of the proposal.

assumptions (3)
  • domain assumption ASTRA-sim's graph reader and collective library faithfully represent the dependency structure and per-operator delays of a real training process.
    Invoked in Section 4 'Modeling Workload with ASTRA-sim'. The framework's end-to-end timing depends on the simulator's compute/memory sleep durations and collective decomposition matching reality.
  • domain assumption CPU-generated RDMA traffic with the same message sizes as GPU traffic exhibits equivalent network behavior, including burstiness, flow concurrency, and congestion response.
    Section 4 'Creating GPU Communication with Traffic Generator'. This is the central fidelity premise and is explicitly deferred to future validation in Section 5.
  • domain assumption Chakra execution graphs encode all dependencies and durations needed to reproduce workload performance.
    Sections 2.3 and 4 assume the input graph is a complete representation of the distributed workload, ignoring effects such as kernel-level jitter or host memory contention.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Easy and Realistic Network Infrastructure Testing for Large-scale Machine Learning." pith.science (2026). https://pith.science/paper/W7LWWHA2

@misc{pith2026250420854,
  author       = {Pith},
  title        = {Pith review of: Towards Easy and Realistic Network Infrastructure Testing for Large-scale Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W7LWWHA2}},
  note         = {Machine review of arXiv:2504.20854}
}
read the original abstract

This paper lays the foundation for Genie, a testing framework that captures the impact of real hardware network behavior on ML workload performance, without requiring expensive GPUs. Genie uses CPU-initiated traffic over a hardware testbed to emulate GPU to GPU communication, and adapts the ASTRA-sim simulator to model interaction between the network and the ML workload.

Figures

Figures reproduced from arXiv: 2504.20854 by the authors.

Figure 1
Figure 1. Bandwidth for several AllReduce runs against simulation. We inject an anomaly which the simulation does not anticipate. 2 Background 2.1 ML Training Process Behavior Developers write complex ML training processes with frame￾works such as PyTorch [1] or JAX [8]. These processes inter￾act with various hardware components and run across multiple nodes [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Overall Architecture of GENIE. GENIE can work on a clus￾ter with only CPU nodes connected via the network infrastructure. However, diagnosing these issues are difficult. Inspecting every hardware/software component is tedious in a largescale deployment. On the other hand, it is expensive for network vendors to purchase large numbers of GPUs for the sole pur￾pose of testing their network infrastructure. Simulators ca… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 36 canonical work pages

  1. [1]

    Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael V oznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Des- maison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michae...

    work page 2024

  2. [2]

    UEC White Paper

    Ultra Ethernet Consortium. UEC White Paper. https: //ultraethernet.org/wp-content/uploads/sit es/20/2023/10/23.07.12-UEC-1.0-Overview-F INAL-WITH-LOGO.pdf, 2023

    work page 2023

  3. [3]

    MSCCLang: Microsoft collective communication language

    Meghan Cowan, Saeed Maleki, Madanlal Musuvathi, Olli Saarikivi, and Yifan Xiong. MSCCLang: Microsoft collective communication language. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 502–514, 2023

    work page 2023

  4. [4]

    McMahon, Duncan Roweth, and Torsten Hoefler

    Daniele De Sensi, Salvatore Di Girolamo, Kim H. McMahon, Duncan Roweth, and Torsten Hoefler. An in-depth analysis of the slingshot interconnect. In SC20: International Conference for High Performance Com- puting, Networking, Storage and Analysis, pages 1–14, 2020

    work page 2020

  5. [5]

    Rocksdb: Evolution of development priorities in a key-value store serving large-scale applications

    Siying Dong, Andrew Kryczka, Yanqin Jin, and Michael Stumm. Rocksdb: Evolution of development priorities in a key-value store serving large-scale applications. ACM Trans. Storage, 17(4), October 2021

    work page 2021

  6. [6]

    DeepSeek-V3 Technical Report, 2025

    DeepSeek-AI et al. DeepSeek-V3 Technical Report, 2025

    work page 2025

  7. [7]

    Facebook. Gloo. https://github.com/facebooki ncubator/gloo

    work page

  8. [8]

    Com- piling machine learning programs via high-level trac- ing

    Roy Frostig, Matthew Johnson, and Chris Leary. Com- piling machine learning programs via high-level trac- ing. In Proceedings of Systems for Machine Learning (SysML 18), 2018

    work page 2018

Show all 37 references
  1. [9]

    RDMA over Ethernet for Distributed Training at Meta Scale

    Adithya Gangidi, Rui Miao, Shengbao Zheng, Sai Jayesh Bondu, Guilherme Goes, Hany Morsy, Rohit Puri, Mohammad Riftadi, Ashmitha Jeevaraj Shetty, Jingyi Yang, Shuqiang Zhang, Mikel Jimenez Fernandez, Shashidhar Gandham, and Hongyi Zeng. RDMA over Ethernet for Distributed Traini...

    2024

  2. [10]

    Google Falcon

    Google. Google Falcon. https://cloud.google.com /blog/topics/systems/introducing-falcon-a -reliable-low-latency-hardware-transport , 2023

    2023

  3. [11]

    Infiniband Trade Associ- ation

    Infiniband Trade Association. Infiniband Trade Associ- ation. https://www.infinibandta.org/

  4. [12]

    RDMA over Converged Ethernet

    Infiniband Trade Association. RDMA over Converged Ethernet. https://www.roceinitiative.org/

  5. [13]

    MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

    Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, Yulu Jia, Sun He, Hongmin Chen, Zhihao Bai, Qi Hou, Shipeng Yan, Ding Zhou, Yiyao Sheng, Zhuo Jiang, Haohan Xu, Haoran Wei, Zhang Zhang, Pengfei Nie, Leqi...

    2024

  6. [14]

    Impact of RoCE Congestion Control Policies on Dis- tributed Training of DNNs

    Tarannum Khan, Saeed Rashidi, Srinivas Sridharan, Pallavi Shurpali, Aditya Akella, and Tushar Krishna. Impact of RoCE Congestion Control Policies on Dis- tributed Training of DNNs. In2022 IEEE Symposium on High-Performance Interconnects (HOTI), pages 39–48, 2022

    2022

  7. [15]

    Accelerating Distributed MoE Training and Inference with Lina

    Jiamin Li, Yimin Jiang, Yibo Zhu, Cong Wang, and Hong Xu. Accelerating Distributed MoE Training and Inference with Lina. In 2023 USENIX Annual Technical Conference (USENIX ATC 23), pages 945–959, Boston, MA, jul 2023. USENIX Association

    2023

  8. [16]

    Flor: An Open High Performance RDMA Framework Over Heterogeneous RNICs

    Qiang Li, Yixiao Gao, Xiaoliang Wang, Haonan Qiu, Yanfang Le, Derui Liu, Qiao Xiang, Fei Feng, Peng Zhang, Bo Li, Jianbo Dong, Lingbo Tang, Hongqiang Harry Liu, Shaozong Liu, Weijie Li, Rui Miao, Yaohui Wu, Zhiwu Wu, Chao Han, Lei Yan, Zheng Cao, Zhongjie Wu, Chen Tian, Guihai...

    2023

  9. [17]

    Janus: A Unified Distributed Training Framework for Sparse Mixture-of-Experts Models

    Juncai Liu, Jessie Hui Wang, and Yimin Jiang. Janus: A Unified Distributed Training Framework for Sparse Mixture-of-Experts Models. In Proceedings of the ACM SIGCOMM 2023 Conference , ACM SIGCOMM ’23, page 486–498, New York, NY , USA, 2023. Association for Computing Machinery

    2023

  10. [18]

    Chakra Working Group at MLCommons

    MLCommons. Chakra Working Group at MLCommons. https://mlcommons.org/working-groups/rese arch/chakra

  11. [19]

    Efficient large-scale language model training on gpu clusters using megatron-lm

    Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, and Matei Zaharia. Efficient large-scale language model training on gpu clusters u...

    2021

  12. [20]

    NCCL Tests

    NVIDIA. NCCL Tests. https://github.com/NVIDIA/nccl-tests/tree/master

  13. [21]

    NVIDIA Spectrum-X Network Platform Ar- chitecture

    NVIDIA. NVIDIA Spectrum-X Network Platform Ar- chitecture. https://resources.nvidia.com/en-u s-accelerated-networking-resource-library /nvidia-spectrum-x

  14. [22]

    NVIDIA Infiniband Adaptive Routing Technology—Accelerating HPC and AI Applications

    NVIDIA. NVIDIA Infiniband Adaptive Routing Technology—Accelerating HPC and AI Applications. White Paper. https://resources.nvidia.com/e n-us-cloud-native-supercomputing-dpus-cam paign/infiniband-white-paper-adaptive-rou ting, 2023

    2023

  15. [23]

    libfabric

    OpenFabrics Allance. libfabric. https://ofiwg.gith ub.io/libfabric/

  16. [24]

    Perftest

    OpenFabrics Allance. Perftest. github.com/linux-r dma/perftest

  17. [25]

    Jupiter evolving: trans- forming google’s datacenter network via optical circuit switches and software-defined networking

    Leon Poutievski, Omid Mashayekhi, Joon Ong, Arjun Singh, Mukarram Tariq, Rui Wang, Jianan Zhang, Vir- ginia Beauregard, Patrick Conner, Steve Gribble, Rishi Kapoor, Stephen Kratzer, Nanfang Li, Hong Liu, Karthik Nagaraj, Jason Ornstein, Samir Sawhney, Ryohei Urata, Lorenzo Vic...

    2022

  18. [26]

    Alibaba hpn: A data center network for large language model training

    Kun Qian, Yongqing Xi, Jiamin Cao, Jiaqi Gao, Yichi Xu, Yu Guan, Binzhang Fu, Xuemei Shi, Fangbo Zhu, Rui Miao, Chao Wang, Peng Wang, Pengcheng Zhang, Xianlong Zeng, Eddie Ruan, Zhiping Yao, Ennan Zhai, and Dennis Cai. Alibaba hpn: A data center network for large language mode...

    2024

  19. [27]

    CASSINI: Network-Aware job scheduling in machine learning clusters

    Sudarsanan Rajasekaran, Manya Ghobadi, and Aditya Akella. CASSINI: Network-Aware job scheduling in machine learning clusters. In 21st USENIX Sympo- sium on Networked Systems Design and Implementation (NSDI 24), pages 1403–1420, Santa Clara, CA, April

  20. [28]

    Deepspeed: System optimizations enable training deep learning models with over 100 billion pa- rameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion pa- rameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’...

    2020

  21. [29]

    libibverbs

    rdma-core Maintainers. libibverbs. https://github .com/linux-rdma/rdma-core/tree/master/libi bverbs

  22. [30]

    TACCL: Guiding collective algorithm synthesis using commu- nication sketches

    Aashaka Shah, Vijay Chidambaram, Meghan Cowan, Saeed Maleki, Madan Musuvathi, Todd Mytkowicz, Ja- cob Nelson, Olli Saarikivi, and Rachee Singh. TACCL: Guiding collective algorithm synthesis using commu- nication sketches. In Proceedings of the 20th USENIX Symposium on Networke...

    2023

  23. [31]

    A Cloud-Optimized Transport Protocol for Elastic and Scalable HPC

    Leah Shalev, Hani Ayoub, Nafea Bshara, and Erez Sab- bag. A Cloud-Optimized Transport Protocol for Elastic and Scalable HPC. IEEE Micro, 40(6):67–73, 2020

    2020

  24. [32]

    Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces

    Srinivas Sridharan, Taekyung Heo, Louis Feng, Zhaodong Wang, Matt Bergeron, Wenyin Fu, Shengbao Zheng, Brian Coutinho, Saeed Rashidi, Changhai Man, and Tushar Krishna. Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces. In arXiv:2305.1...

    2023 arXiv

  25. [33]

    TopoOpt: Co-optimizing network topology and parallelization strategy for dis- tributed training jobs

    Weiyang Wang, Moein Khazraee, Zhizhen Zhong, Manya Ghobadi, Zhihao Jia, Dheevatsa Mudigere, Ying Zhang, and Anthony Kewitsch. TopoOpt: Co-optimizing network topology and parallelization strategy for dis- tributed training jobs. In 20th USENIX Symposium on Networked Systems Des...

    2023

  26. [34]

    SimAI: Unifying Architecture Design and Performance Tunning for Large-Scale Large Lan- guage Model Training with Scalability and Precision

    Xizheng Wang, Qingxu Li, Yichi Xu, Gang Lu, Dan Li, Li Chen, Heyang Zhou, Linkang Zheng, Sen Zhang, Yikai Zhu, Yang Liu, Pengcheng Zhang, Kun Qian, Kunling He, Jiaqi Gao, Ennan Zhai, Dennis Cai, and Binzhang Fu. SimAI: Unifying Architecture Design and Performance Tunning for L...

    2025

  27. [35]

    TACOS: Topology-aware collective algorithm synthesizer for dis- tributed machine learning

    William Won, Midhilesh Elavazhagan, Sudarshan Srini- vasan, Swati Gupta, and Tushar Krishna. TACOS: Topology-aware collective algorithm synthesizer for dis- tributed machine learning. In Proceedings of the 57th IEEE/ACM International Symposium on Microarchitec- ture (MICRO), p...

    2024

  28. [36]

    ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale

    William Won, Taekyung Heo, Saeed Rashidi, Srinivas Sridharan, Sudarshan Srinivasan, and Tushar Krishna. ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale. In ISPASS 2023, 2023. 6

    2023

  29. [37]

    Towards a Standardized Representation for Deep Learning Collective Algorithms

    Jinsun Yoo, William Won, Meghan Cowan, Nan Jiang, Benjamin Klenk, Srinivas Sridharan, and Tushar Kr- ishna. Towards a Standardized Representation for Deep Learning Collective Algorithms. IEEE Micro, 45(2):1– 9, 2025. 7

    2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.