An End-to-End DNN Inference Framework for the SpiNNaker2 Neuromorphic MPSoC

Chen Liu; Christian Mayr; Hector A. Gonzalez; Matthias Jobst; Mehmet Alici; Tim Langer

REVIEW 2 major objections 5 minor 15 references

An End-to-End DNN Inference Framework for the SpiNNaker2 Neuromorphic MPSoC

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper tries to show that a single SpiNNaker2 chip can run complete multi-layer DNNs end to end on its own, with an automated quantization and lowering flow from a pretrained model, and demonstrates it on an MLP that runs in 688 µs at…

desk verdict Genuine multi-layer scheduler for SpiNNaker2, but the transformer-scale claim is marketing; the paper's own measurements show DRAM streaming is the bottleneck. read the letter →

arxiv 2507.13736 v1 pith:N32TX3DN submitted 2025-07-18 cs.LG cs.ARcs.DC

Matthias Jobst , Tim Langer , Chen Liu , Mehmet Alici , Hector A. Gonzalez , Christian Mayr This is my paper · ORCID

classification cs.LGcs.ARcs.DC

keywords neuromorphiccomputingSpiNNaker2DNNschedulingmulti-layerpost-trainingquantizationedgeinferenceDRAMorganizationon-chipexecution

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that a single SpiNNaker2 chip can run a complete multi-layer neural network end to end, without the host computer intervening between layers after the model is loaded. The authors extend an existing single-layer scheduler so that one scheduler processor walks a chain of layer configurations stored in off-chip memory, dispatches worker processors, collects their completion flags, and advances to the next layer until a special finish marker. They add an automated front end that quantizes a pretrained model to 8-bit power-of-two values, fuses adjacent operations such as linear layers followed by ReLU, and lowers the graph into the scheduler's layer representation. On a small three-layer MLP for MNIST, the measured on-chip runtime is 688 microseconds and accuracy matches the float baseline within a few hundredths of a percent. If the approach scales, it would give SpiNNaker2 a path to run large edge models, including transformers, without hand-written per-layer code.

What carries the argument

The load-bearing mechanism is the statically organized DRAM layout plus the layer-information linked list that lets the scheduler advance through a model. Each 128-bit header carries the layer type, the number of worker processing elements, the address of the next layer, and the number of parts; a worker, after being triggered, fetches its own configuration and executes the layer-specific operation, possibly iterating over multiple parts. A special 'finish' code in the last layer's header terminates all workers. This design removes model-specific scheduler code and lets one generic scheduler handle diverse layer types such as linear, add, and softmax.

What would settle it

Run a transformer-scale model through this pipeline on a single SpiNNaker2 chip and measure per-layer time while varying the size of the weight matrices. If the execution time grows with the number of weight bytes fetched from off-chip memory instead of with the number of multiply-accumulate operations, then memory bandwidth, not the accelerator, sets the limit, and the transformer-scale claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that multi-layer DNN inference can be executed entirely on one SpiNNaker2 chip after an initial host-side configuration, with the on-chip scheduler iterating over every layer without any host communication. The proof-of-concept is an MLP with 784-512-256-16 neurons, which runs in 688 µs total, with per-layer scheduling overhead of around 13 µs and setup plus cleanup of 12 µs and 9 µs. The quantized 8-bit model reaches 98.34% accuracy on the MNIST validation set, compared with 98.36% for the original float model and 98.33% for the quantized reference, so the measured execution is not a bit-accurate model of the reference but the difference is negligible. The runtime analysis shows that the first linear layer spends 192 of its 323 µs loading weights from off-chip memory and only 29 µs in the matrix-multiply accelerator, identifying data movement as the current bottleneck.

Load-bearing premise

The whole approach depends on off-chip memory being able to feed layer weights to the processor cores fast enough for large models; the reported measurements already show that loading the first layer's weights takes about six times longer than the actual matrix multiplication.

Editorial extensions

If this is right

Any pretrained model that can be quantized to 8-bit power-of-two weights can be executed on a single SpiNNaker2 chip without host interaction between layers, removing a major obstacle to edge deployment.
The per-layer scheduling overhead of about 13 µs is small relative to the hundreds of microseconds of the larger layers, so deeper models should not be swamped by scheduler costs.
Fusing linear layers with the following ReLU reduces the number of scheduled layers and exploits the combined hardware operation, which matters most for small layers where scheduling overhead would otherwise dominate.
The dominant cost shifts from scheduling to loading weights from off-chip memory, so the design points future work toward keeping weights on-chip, overlapping transfers with compute, or spreading a model over multiple chips.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If off-chip memory bandwidth is the real ceiling, the transformer-scale goal likely depends on cache-friendly layer fusion, weight prefetching, or spilling only part of the model to off-chip memory; a natural next experiment is to run a model whose weights exceed the on-chip SRAM and compare actual throughput against a bandwidth-only estimate.
The generic layer-information format could be reused outside deep learning, for example for streaming signal-processing graphs or hybrid spiking-and-artificial networks, since the scheduler does not care what a layer computes.
The fact that measured on-chip accuracy (98.34%) slightly exceeds the quantized reference (98.33%) hints that the non-bit-exact rounding may occasionally be beneficial; checking this across several model families would show whether it is systematic or a single-dataset artifact.
Replacing the 'next layer address' with a remote address would extend the same scheduling scheme across multiple SpiNNaker2 chips, aligning the edge flow with the platform's larger-scale designs.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, what's new: the prior OctopuScheduler handled single layers; this paper adds multi-layer on-chip scheduling with DRAM-resident layer configurations, stand-alone iteration over all layers without host communication, and an automated PyTorch-to-INT8-quantized flow using AMD Quark. That is a real engineering step, and it's verified on hardware: a three-layer MLP runs end-to-end with 98.34% accuracy, matching the quantized ONNX model. The DRAM layout and the scheduler/worker protocol are clearly described, and the runtime breakdown is honest and useful.

What's soft: the abstract's claim about 'up to transformer scale' is not supported by anything in the paper. The only experiment is a 784-512-256-16 MLP. The paper's own measurement shows weight loading from DRAM takes 192 us versus 29 us of compute in the first layer — 9% accelerator utilization — and the discussion explicitly says DRAM transfers are the main bottleneck and lists reducing them as future work. Scaling to transformer-size weights through the same DRAM streaming mechanism is an assumption, not a result. Also, no code is shipped, so the flow is not directly reproducible.

The paper's internal discussion is actually more measured than its abstract; Section VI calls the MLP a dummy model and says it demonstrates preliminary results. The gap is between the contribution as implemented and the contribution as advertised.

For a reader in the SpiNNaker2 / neuromorphic deployment niche, this is a useful enabler: it removes manual per-layer host intervention and gives a standardized on-chip control flow. For a general ML audience, it's mostly a hardware-specific engineering report.

My take: send it to peer review, but the abstract needs to be brought in line with the evidence. A serious referee should ask for at least one larger demonstration — a CNN or a small transformer — or a realistic bandwidth analysis that shows whether the DRAM-centric design can plausibly reach that scale. As is, the central engineering claim holds; the scaling claim does not.

Referee Report

2 major / 5 minor

Summary. The paper presents an extension of the OctopuScheduler framework for the SpiNNaker2 neuromorphic MPSoC, adding multi-layer DNN scheduling that runs entirely on-chip after an initial host configuration. The proposed end-to-end flow converts a PyTorch model to ONNX, applies INT8 post-training quantization and lowering, and generates a hardware-ready S2Model with all weights, activations, and layer configurations placed in the chip's external DRAM. A scheduler PE iterates through layers, dispatching standardized work to worker PEs without host involvement between layers. The authors validate the flow on a three-layer MLP (784-512-256-16) on MNIST, reporting 98.34% accuracy on SpiNNaker2 versus 98.36% for the pre-trained FP32 model, and a total on-chip runtime of 688 microseconds, with DRAM weight loading identified as the dominant cost. The paper claims in the abstract and contributions that this framework enables edge-based execution of DNNs up to transformer scale, but the experiments cover only a small MLP.

Significance. If the multi-layer on-chip scheduling works as described, the paper provides a useful building block for single-chip edge inference on SpiNNaker2: it removes per-layer host involvement, standardizes layer execution so that the scheduler is not model-specific, and provides an automated path from PyTorch to quantized, hardware-mapped inference. The measured accuracy preservation (98.34% vs 98.36%) and the small scheduling overhead (12 us setup, 9 us cleanup, roughly 13 us per layer) are concrete, falsifiable results that give confidence in the implementation for the demonstrated model class. The framework also leverages existing open components (pySpiNNaker2, AMD Quark), which is a practical strength. However, the transformer-scale claim is not supported by the experiments: no transformer or even CNN is evaluated, the design is DRAM-centric, and the paper's own measurements show that DRAM transfer, not compute, dominates the runtime. The contribution is therefore currently an incremental but functional engineering result whose headline claim exceeds its demonstrated scope.

major comments (2)

[Abstract; Section I, Contribution 1; Section VI] The central claim that the framework 'enables the edge-based execution of large and complex DNNs up to transformer scale' is an extrapolation from a single 784-512-256-16 MLP (Sec. V.A). Section VI itself states that the dummy MLP 'demonstrates only preliminary results.' No transformer, CNN, or even larger MLP experiment is reported, and the design stores all weights and activations in DRAM (Sec. III.A-B). The measured runtime shows DRAM weight loading dominates (192 us load vs 29 us compute for FC1, Sec. V.B), and the Discussion identifies DRAM transfers as the main bottleneck. To make the central claim defensible, the authors should either demonstrate a representative larger model on chip or replace the abstract claim with a clearly labeled scaling projection based on measured DRAM bandwidth and a concrete model-size limit.
[Section V.B; Section III.A-B] The runtime analysis quantifying the DRAM bottleneck rests on a single configuration with no repeated measurements or error bars. The paper reports 688 us total runtime and 192 us weight-loading time for the first layer without stating whether these are single measurements, means, or medians. Since the load-bearing conclusion that DRAM streaming is the bottleneck is quantitative, at least a small number of repeated runs with the observed range (or standard deviation) should be reported. This is especially important because the authors use this bottleneck to motivate the future-work direction stated in Section VI.

minor comments (5)

[Section IV.A.3] The name 'Octopuscheduler' appears with inconsistent capitalization; elsewhere in the paper it is 'OctopuScheduler'.
[Section V.A] The word 'accomodate' should be 'accommodate'.
[Table I] The accuracy values appear to be from single runs; for the claim that accuracy is essentially preserved, it would be helpful to state the number of runs and the observed spread, and to specify the size of the calibration dataset used for quantization.
[Section VI] The paper cites prior work on language modeling on SpiNNaker2 [7] but does not compare the proposed framework with that approach; given the transformer-scale claim, a short comparison of model sizes, host interaction, and runtime would help position the contribution.
[General] The paper does not state whether the implementation or configuration files are publicly available; adding a reproducibility/artifact statement would strengthen the end-to-end claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's runtime and accuracy results are direct measurements, and the transformer-scale claim is an unverified extrapolation rather than a derivation from fitted inputs.

full rationale

The paper does not derive any prediction from a fitted parameter or from a self-referential definition. Accuracy (98.34% on SpiNNaker2) and runtime (688 microseconds total, with per-layer breakdowns) are measured on real hardware. The quantization flow uses the external AMD Quark library and standard cross-layer equalization, with no constant fitted to the target result. The multi-layer scheduler is presented as an extension of the authors' prior OctopuScheduler framework, but that prior work is an engineering dependency, not an unverified premise that forces the new results; the new contributions (standalone on-chip iteration, DRAM-based multi-layer configurations, and the PyTorch-to-device flow) are implemented and demonstrated. The claim of enabling execution 'up to transformer scale' is an extrapolation from a small MLP experiment, and the paper itself identifies DRAM transfers as the main bottleneck and lists reducing them as future work. That is a correctness or scalability concern, not circularity: no equation, fitted value, or self-citation is being recycled as the evidence for the conclusion.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper's central claims rest on hardware and quantization assumptions rather than fitted parameters. I listed the SpiNNaker2 resource model, the adequacy of INT8 PTQ, the topological ordering assumption, and the ONNX proxy assumption. No free parameters or invented entities are introduced.

assumptions (4)

domain assumption SpiNNaker2 hardware provides 152 PEs, each with 128 kB SRAM, MLA accelerators, and an external 2 GB DRAM.
The whole scheduler design assumes this memory and accelerator model; introduced in Section I and used throughout.
domain assumption INT8 power-of-two quantization with MSE optimization and cross-layer equalization preserves accuracy for arbitrary DNNs.
Used in Section IV.A.1; only verified on one MNIST MLP, while the paper claims transformer-scale applicability.
standard math Topological sorting of the application graph yields a valid linear execution order respecting data dependencies.
Section IV.A.3 relies on the standard DAG property that a topological order exists and respects edges.
domain assumption The quantized ONNX model is a close enough proxy for SpiNNaker2 execution that measured accuracy reflects model quality.
Section V.A attributes the 98.34 vs 98.33 difference to ONNX not being bit-accurate; this proxy assumption underpins the accuracy claim.

how reviews work

0 comments

Cite this review

Pith. "Pith review of An End-to-End DNN Inference Framework for the SpiNNaker2 Neuromorphic MPSoC." pith.science (2026). https://pith.science/paper/N32TX3DN

@misc{pith2026250713736,
  author       = {Pith},
  title        = {Pith review of: An End-to-End DNN Inference Framework for the SpiNNaker2 Neuromorphic MPSoC},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N32TX3DN}},
  note         = {Machine review of arXiv:2507.13736}
}

read the original abstract

This work presents a multi-layer DNN scheduling framework as an extension of OctopuScheduler, providing an end-to-end flow from PyTorch models to inference on a single SpiNNaker2 chip. Together with a front-end comprised of quantization and lowering steps, the proposed framework enables the edge-based execution of large and complex DNNs up to transformer scale using the neuromorphic platform SpiNNaker2.

Figures

Figures reproduced from arXiv: 2507.13736 by the authors.

**Figure 2.** DRAM Memory Organization: Global Configuration is fixed at start of [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Scheduling flow potentially iterating over multiple parts. After successful completion, the workers indicate the termination of their layer computations to the scheduler and return to sleep mode. When the scheduler receives the completion flags from all workers, it steps on to the next layer, until all layers have been executed. After the final layer, a special ”finish” layer information is sent to all workers to t… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 11 canonical work pages

[1]

Challenging trends in energy of computing for data centers,

S. Shankar, “Challenging trends in energy of computing for data centers,” Computer, vol. 57, no. 12, pp. 134–142, 2024

work page 2024
[2]

Neuromorphic computing at scale,

D. Kudithipudi, C. Schuman, C. M. Vineyard, T. Pandit, C. Merkel, R. Kubendran, J. B. Aimone, G. Orchard, C. Mayr, R. Benosman et al., “Neuromorphic computing at scale,” Nature, vol. 637, no. 8047, pp. 801–812, 2025

work page 2025
[3]

SpiNNaker2: A Large-Scale Neuromor- phic System for Event-Based and Asynchronous Machine Learning,

H. A. Gonzalez, J. Huang, F. Kelber, K. K. Nazeer, T. Langer, C. Liu, M. Lohrmann, A. Rostami, M. Sch ¨one, B. V ogginger, T. C. Wunderlich, Y . Yan, M. Akl, and C. Mayr, “SpiNNaker2: A Large-Scale Neuromor- phic System for Event-Based and Asynchronous Machine Learning,” arXiv preprint arXiv:2401.04491 , Jan. 2024

arXiv 2024
[4]

Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Represen- tation,

S. Arfa, B. V ogginger, C. Liu, J. Partzsch, M. Schone, and C. Mayr, “Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Represen- tation,” arXiv preprint arXiv:2504.06748 , Apr. 2025

arXiv 2025
[5]

Hybrid Spiking and Artificial Neural Networks for Radar-Based Gesture Recog- nition,

P. Gerhards, M. Weih, J. Huang, K. Knobloch, and C. G. Mayr, “Hybrid Spiking and Artificial Neural Networks for Radar-Based Gesture Recog- nition,” in 2023 8th International Conference on Frontiers of Signal Processing (ICFSP), Oct. 2023, pp. 83–87

work page 2023
[6]

Mapping Deep Neural Networks on SpiNNaker2,

F. Kelber, B. Wu, B. V ogginger, J. Partzsch, C. Liu, M. Stolba, and C. Mayr, “Mapping Deep Neural Networks on SpiNNaker2,” in Proceedings of the 2020 Annual Neuro-Inspired Computational Elements Workshop (NICE), Jun. 2020, pp. 1–3

work page 2020
[7]

Language Modeling on a SpiNNaker2 Neuromorphic Chip,

K. K. Nazeer, M. Sch ¨one, R. Mukherji, B. V ogginger, C. Mayr, D. Kappel, and A. Subramoney, “Language Modeling on a SpiNNaker2 Neuromorphic Chip,” in 2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS) , Apr. 2024, pp. 492–496

work page 2024
[8]

Event-based backpropagation on the neuromorphic platform SpiNNaker2,

G. B ´ena, T. Wunderlich, M. Akl, B. V ogginger, C. Mayr, and H. A. Gonzalez, “Event-based backpropagation on the neuromorphic platform SpiNNaker2,” in NeurIPS 2024 Workshop Machine Learning with new Compute Paradigms, 2024

work page 2024

Show all 15 references

[9]

GSPMD: General and Scalable Parallelization for ML Computation Graphs,

Y . Xu, H. Lee, D. Chen, B. Hechtman, Y . Huang, R. Joshi, M. Krikun, D. Lepikhin, A. Ly, M. Maggioni, R. Pang, N. Shazeer, S. Wang, T. Wang, Y . Wu, and Z. Chen, “GSPMD: General and Scalable Parallelization for ML Computation Graphs,” arXiv preprint arXiv:2105.04663, Dec. 2021

2021 arXiv
[10]

LOMA: Fast Auto-Scheduling on DNN Accelerators through Loop-Order-based Memory Allocation,

A. Symons, L. Mei, and M. Verhelst, “LOMA: Fast Auto-Scheduling on DNN Accelerators through Loop-Order-based Memory Allocation,” in 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS) , 06 2021, pp. 1–4

2021
[11]

DeFiNES: Enabling Fast Exploration of the Depth-first Scheduling Space for DNN Accelerators through Analytical Modeling,

L. Mei, K. Goetschalckx, A. Symons, and M. Verhelst, “DeFiNES: Enabling Fast Exploration of the Depth-first Scheduling Space for DNN Accelerators through Analytical Modeling,” arXiv preprint arXiv:2212.05344, 2024

2024 arXiv
[12]

Oc- topuScheduler: On-chip DNN Scheduling on the SpiNNaker2 Neuro- morphic MPSoC,

T. Langer, M. Jobst, C. Liu, B. V ogginger, and C. G. Mayr, “Oc- topuScheduler: On-chip DNN Scheduling on the SpiNNaker2 Neuro- morphic MPSoC,” in Proceedings of the Neuro-Inspired Computational Elements Conference (NICE) , 2025, pp. 1–10

2025
[13]

py-spinnaker2,

B. V ogginger, F. Kelber, M. Jobst, G. B ´ena, S. Arfa, Y . Yan, P. Gerhards, M. Weih, M. Akl, H. A. Gonzalez, and C. Mayr, “py-spinnaker2,” Aug. 2024. [Online]. Available: https://zenodo.org/doi/10.5281/zenodo.10202109

2024 doi
[14]

Quark Model Optimizer,

AMD, “Quark Model Optimizer,” Apr. 2025. [Online]. Available: https://github.com/amd/Quark

2025
[15]

Data-Free Quantization Through Weight Equalization and Bias Correction,

M. Nagel, M. v. Baalen, T. Blankevoort, and M. Welling, “Data-Free Quantization Through Weight Equalization and Bias Correction,” arXiv preprint arXiv:1906.04721, Nov. 2019

1906 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Challenging trends in energy of computing for data centers,

S. Shankar, “Challenging trends in energy of computing for data centers,” Computer, vol. 57, no. 12, pp. 134–142, 2024

work page 2024

[2] [2]

Neuromorphic computing at scale,

D. Kudithipudi, C. Schuman, C. M. Vineyard, T. Pandit, C. Merkel, R. Kubendran, J. B. Aimone, G. Orchard, C. Mayr, R. Benosman et al., “Neuromorphic computing at scale,” Nature, vol. 637, no. 8047, pp. 801–812, 2025

work page 2025

[3] [3]

SpiNNaker2: A Large-Scale Neuromor- phic System for Event-Based and Asynchronous Machine Learning,

H. A. Gonzalez, J. Huang, F. Kelber, K. K. Nazeer, T. Langer, C. Liu, M. Lohrmann, A. Rostami, M. Sch ¨one, B. V ogginger, T. C. Wunderlich, Y . Yan, M. Akl, and C. Mayr, “SpiNNaker2: A Large-Scale Neuromor- phic System for Event-Based and Asynchronous Machine Learning,” arXiv preprint arXiv:2401.04491 , Jan. 2024

arXiv 2024

[4] [4]

Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Represen- tation,

S. Arfa, B. V ogginger, C. Liu, J. Partzsch, M. Schone, and C. Mayr, “Efficient Deployment of Spiking Neural Networks on SpiNNaker2 for DVS Gesture Recognition Using Neuromorphic Intermediate Represen- tation,” arXiv preprint arXiv:2504.06748 , Apr. 2025

arXiv 2025

[5] [5]

Hybrid Spiking and Artificial Neural Networks for Radar-Based Gesture Recog- nition,

P. Gerhards, M. Weih, J. Huang, K. Knobloch, and C. G. Mayr, “Hybrid Spiking and Artificial Neural Networks for Radar-Based Gesture Recog- nition,” in 2023 8th International Conference on Frontiers of Signal Processing (ICFSP), Oct. 2023, pp. 83–87

work page 2023

[6] [6]

Mapping Deep Neural Networks on SpiNNaker2,

F. Kelber, B. Wu, B. V ogginger, J. Partzsch, C. Liu, M. Stolba, and C. Mayr, “Mapping Deep Neural Networks on SpiNNaker2,” in Proceedings of the 2020 Annual Neuro-Inspired Computational Elements Workshop (NICE), Jun. 2020, pp. 1–3

work page 2020

[7] [7]

Language Modeling on a SpiNNaker2 Neuromorphic Chip,

K. K. Nazeer, M. Sch ¨one, R. Mukherji, B. V ogginger, C. Mayr, D. Kappel, and A. Subramoney, “Language Modeling on a SpiNNaker2 Neuromorphic Chip,” in 2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS) , Apr. 2024, pp. 492–496

work page 2024

[8] [8]

Event-based backpropagation on the neuromorphic platform SpiNNaker2,

G. B ´ena, T. Wunderlich, M. Akl, B. V ogginger, C. Mayr, and H. A. Gonzalez, “Event-based backpropagation on the neuromorphic platform SpiNNaker2,” in NeurIPS 2024 Workshop Machine Learning with new Compute Paradigms, 2024

work page 2024