{"id":"e33a060a-126c-4a30-bbce-c07704d899d4","arxiv_id":"2502.07578","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CENT, a simulated GPU-free CXL-connected processing-in-memory system, reports 2.3x higher LLM inference throughput and 5.2x better tokens-per-dollar than a 4x A100 GPU server.","lead":"This paper proposes CENT, a server made of memory chips with processing units built into each memory bank, connected through the CXL interconnect, designed to run large language model inference without GPUs. In simulation, CENT reports 2.3 times higher inference throughput and 5.2 times more generated tokens per dollar than a four-GPU A100 server.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No accuracy validation: BF16 near-bank MACs, BF16 reduction trees, and 10th-order Taylor exp are assumed to preserve LLM quality; if they don't, CENT's throughput/TCO claims are moot.","rationale":"The reader identified numerical accuracy as the weakest assumption, and I agree. The paper compares to GPU baselines on throughput, energy, and TCO, but the claim 'GPU-free system for LLM inference' carries an implicit correctness requirement. The artifact appendix is explicit evidence that no numerical evaluation was performed: model weights are not required and real values are not modeled. The BF16 reduction trees producing BF16 results is particularly concerning for Softmax normalization over thousands of scores and for 8K-dimensional dot products. This is not a stylistic issue; it concerns whether the generated text is usable. Other possible concerns, such as the modified CXL broadcast using reserved header codes or the token-by-token prefill, are implementation or workload-scope issues that can be addressed separately; numerical fidelity is the necessary condition for the entire comparative evaluation to be meaningful. Since the paper is otherwise detailed and artifact-backed, and the concern is addressable with a concrete accuracy study, the appropriate disposition remains conditional acceptance pending that study. The reader's verdict does not need to change.","tokens_in":30440,"tokens_out":3814,"duration_ms":37457,"concrete_test":"Extend or wrap the CENT functional simulator to emulate numerical values: load real Llama2-7B/13B/70B weights, cast weights and activations to BF16 with the same MAC accumulation order, use the 10-order Taylor exponent and BF16 reduction trees for Softmax, and generate text. Compare perplexity on WikiText-2 and accuracy on a standard downstream benchmark (e.g., OpenLLM tasks) against FP16 GPU inference with identical prompts. Quantify the gap: if perplexity increases less than 1% or task accuracy is within 1 point, the concern is resolved; if perplexity increases more than 5% or accuracy drops by more than 2 points on any model, CENT's central claim fails and needs hardware changes (e.g., FP32 accumulation, higher-order or other exponent approximation, selective FP16). Also test a long-context case (16K) because Softmax and attention error can accumulate with sequence length.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that CENT is a GPU-free system for LLM inference, not merely a fast token generator. That requires the produced tokens to be as useful as GPU output. The paper never tests this. Section 4.2 specifies BF16 near-bank MACs, BF16 reduction trees (16 BF16 inputs to one BF16), BF16 accumulators, and a 10-order Taylor Series for exponentiation in the PNM Exponent Accelerators. Softmax is computed with these exponentials and reductions. For Llama2-70B, each GEMV is 8K-dimensional and attention scores have up to 4K elements; repeated BF16 rounding and BF16 reduction-tree accumulation rather than FP32 accumulation can introduce substantial error. The functional simulator, per Section 6 and Appendix A.3.3, verifies instruction-level correctness only; the artifact explicitly states model weights are not required and the performance/power simulator does not model real values. No perplexity, downstream-task, or logit-level comparison against FP16 GPU inference is reported. Thus the design's numerical fidelity is entirely unverified. If the approximations degrade output quality, the 2.3x throughput and 5.2x tokens/dollar numbers describe a device that cannot serve as an LLM replacement. This is the load-bearing assumption: all speed and cost advantages presuppose output fidelity.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"CENT is a proposed GPU-free server for LLM inference built from CXL-attached devices containing GDDR6-PIM channels and near-memory PNM units. The paper describes the hierarchical PIM-PNM microarchitecture, a CXL 3.0-based network with send/receive/broadcast/multicast primitives, mappings for pipeline, tensor, and hybrid parallelism, and an evaluation of Llama2-7B/13B/70B using a modified Ramulator2 simulator plus power and cost models. Against a vLLM baseline on A100 GPUs, the authors report 2.3x higher end-to-end throughput, 2.9x lower energy, and 5.2x more tokens per dollar. The artifact is open source and includes trace generation, simulation, power modeling, and figure-generation scripts.","tokens_in":30781,"tokens_out":6685,"duration_ms":57580,"significance":"If the reported results hold, CENT would be a significant data point showing that memory-centric hardware can serve memory-bound LLM decoding more efficiently than GPUs. The paper's strengths include the open-source artifact, the use of a timing-aware DRAM simulator, the grounding in real PIM prototypes (AiM, UPMEM), the inclusion of CXL-PNM and heterogeneous GPU-PIM baselines, and the scalability exploration to 128 devices. The central limitation is that output fidelity is never validated, so the throughput and TCO claims currently refer to a system whose tokens are not shown to be as useful as GPU-generated tokens.","major_comments":[{"comment":"The numerical fidelity of the proposed datapath is load-bearing and is not evaluated. Section 4.2 specifies BF16 near-bank MACs, BF16 reduction trees that reduce 16 BF16 inputs to one BF16, BF16 accumulators, and 10th-order Taylor-series exponent accelerators in the PNM units. For Llama2-70B, the GEMV dimension is 8K and attention scores reach 4K elements, so repeated BF16 rounding and reduction-tree accumulation can accumulate substantial error. Section 6 and Appendix A.3.3 state that the functional simulator verifies instruction-level correctness only, that model weights are not required, and that the performance/power simulator does not model real values. No perplexity, logit-level, or downstream-task comparison against FP16/FP32 GPU inference is reported. If these approximations degrade model quality, the 2.3x throughput and 5.2x tokens-per-dollar results describe a device that cannot replace GPUs. Please add accuracy validation (e.g., WikiText perplexity, common QA benchmarks, and logit similarity for representative prompts across all three model sizes), or revise the numerical pipeline to use higher-precision accumulation and show that the approximation error is negligible.","section":"§4.2, §6, Appendix A.3.3"},{"comment":"The main throughput comparison uses asymmetric concurrency. The GPU runs vLLM with batch size 128, while CENT uses pipeline parallelism with 32, 40, and 80 concurrent prompts for the 7B, 13B, and 70B models, respectively (Section 5.1 states that CENT does not batch within a pipeline stage; the 'batch' is the pipeline depth). Because reported throughput is aggregated over all concurrent queries, a larger batch directly increases tokens/s. The paper does not show CENT throughput at batch 128 or at matched concurrency levels, nor does it report the maximum feasible batch for the 7B and 13B GPU configurations, which use 1 and 2 GPUs rather than 4. Please report throughput at matched batch sizes and/or provide an SLA-constrained comparison; otherwise the 2.3x geomean does not isolate memory-system efficiency.","section":"§5.1, §7.1, Figure 13"},{"comment":"The TCO and tokens-per-dollar results depend on unvalidated cost assumptions. PIM module cost is set to 10x standard DRAM, the CXL controller production volume is set to 3 million units, and the A100 price is set to $10,000 (Section 6, Table 6, Figure 12). These are plausible inputs but they are point estimates, and the 5.2x tokens-per-dollar figure has no sensitivity analysis. For example, if the PIM cost multiplier is actually 20x or the production volume is 300K, the TCO advantage will shrink substantially. Please provide a one-way (or ideally multi-way) sensitivity analysis over these parameters and report the range of tokens-per-dollar, so readers can see how robust the central TCO claim is.","section":"§6, Table 6, Figure 12"},{"comment":"The CXL network evaluation rests on a protocol-level assumption that is not validated. Section 4.1 repurposes a reserved header code in the PBR flit to implement broadcast/multicast and requires write acknowledgements from all destinations. Section 6 models the multicast-capable switch by halving bandwidth and doubling latency of a baseline switch. No CXL-protocol simulation, credit/ordering analysis, or sensitivity study supports this model. Since the TP and hybrid mappings (Figures 9 and 14) rely on frequent broadcast and gather transactions, and the latency speedups in Figure 13(a) depend on this cost model, the authors should either validate the multicast model with a cycle-level CXL switch simulation or show that the results are insensitive to the factor-of-two bandwidth/latency penalty.","section":"§4.1, §6"}],"minor_comments":[{"comment":"The conclusion states that CENT 'consumes 2.3× less energy,' while the abstract and Section 7.2 report 2.9×; please reconcile the numbers.","section":"§9"},{"comment":"The term 'batch' is used for pipeline depth in the CENT configuration; consider calling it 'concurrent prompts' or 'pipeline width' to avoid confusion with GPU batching.","section":"§5.1, Figure 13"},{"comment":"The 16K and 32K context results use a 1TB CENT configuration with 16Gb GDDR6-PIM modules, but Table 4 lists 512GB; this configuration change should be stated in the main text and not only in the caption.","section":"Figure 14(a)"},{"comment":"The statement that 'model weights and parameters are not required for this appendix' should be reconciled with the claim that functional correctness is verified; as written, it reinforces the missing accuracy validation.","section":"Appendix A.3.3"},{"comment":"Minor typographical issues include 'hierachical' in Section 4, 'Transfmer' in reference [51], and inconsistent use of 'Llama2' versus 'Llama 2'.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The accuracy gap is the primary gate; I would be willing to accept after an accuracy appendix and cost sensitivity analysis. The comparison-fairness issue can be addressed with additional matched-batch experiments. The paper is otherwise thorough and artifact-backed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"CENT is a serious, artifact-backed systems paper. The GPU-free CXL-PIM design with broadcast/multicast/gather primitives and PP/TP/hybrid mappings is genuinely new relative to CXL-PNM, AttAcc, and NeuPIM. The evaluation is detailed: a Ramulator2-based simulator, an activity-based power model with RTL synthesis for the controller, and an explicit cost model. The artifact is open-source, and the authors are transparent that the performance simulator does not model real values.\n\nThe soft spot is load-bearing. The paper claims a GPU-free system for LLM inference, but never checks whether the output is usable. Section 4.2 specifies BF16 near-bank MACs, BF16 reduction trees, and a 10th-order Taylor series for exponentiation in softmax. Attention scores for Llama2-70B have up to 4K elements; BF16 accumulation instead of FP32 can introduce substantial error. The functional simulator only checks instruction-level correctness; Appendix A.3.3 explicitly says model weights are not required. No perplexity, downstream-task, or logit-level comparison against FP16 GPU inference is reported. If these approximations degrade model quality, the 2.3x throughput and 5.2x tokens/dollar numbers describe a device that cannot replace a GPU for LLM serving. This is not a minor omission.\n\nSecondary concerns: the throughput comparison is asymmetric (GPU at batch 128 vs CENT at pipeline-depth batch 32/40/80), and CXL multicast is modeled by halving bandwidth rather than implementing the protocol. Both are stated openly, but they are confounds.\n\nWhat the paper does well: the architecture is novel, the mapping is concrete, and the cost model is reproducible. The artifact is honest, which counts for something.\n\nWho this is for: computer architects working on memory-centric systems, PIM, and LLM serving. I would bring it to a reading group and I would cite the architecture contributions. But I would not accept it as-is. The authors need to add an accuracy study (perplexity on a standard benchmark, or at least logit-level comparison) for the proposed numerical approximations, plus a sensitivity analysis on batch size and CXL multicast assumptions. With that, the system claim could be credible. Without it, the central result is unproven. A serious editor should send it to peer review with major-revision expectations.","headline":"Solid systems work with one load-bearing gap: no accuracy validation for BF16/Taylor softmax, plus an asymmetric GPU comparison; deserves peer review with major-revision expectations.","tokens_in":31295,"tokens_out":2468,"would_cite":true,"duration_ms":22551,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CENT, a GPU-free CXL memory system, claims 2.3x higher LLM inference throughput than A100 GPUs while using 2.9x less energy.","keywords":["Processing-in-memory","Compute Express Link","Large language model inference","Memory-bound workloads","Near-bank processing","GPU-free serving","Pipeline parallelism","Tensor parallelism"],"falsifier":"Run the paper's provided CENT simulator on Llama2-70B with the published traces, measure end-to-end perplexity on a held-out text set, and compare against FP16 A100 inference at batch 128; if perplexity diverges by more than a preset tolerance, the accuracy premise fails. Separately, reproduce the 2.3x throughput and 2.9x energy numbers with the supplied scripts; if they do not reproduce, the headline comparison fails.","tokens_in":30268,"feed_emoji":"⚡","tokens_out":9500,"duration_ms":84097,"temperature":0.7,"pith_summary":"Large language models are memory-bound during token-by-token decoding: they stream weights and key-value caches from DRAM faster than GPUs can feed them, so expensive GPU compute sits underutilized. This paper argues that the right hardware for that workload is not a GPU at all but a memory-centric system: CXL-attached memory devices with processing units placed next to DRAM banks. The proposed system, CENT, claims 2.3x higher end-to-end throughput than an A100-based GPU server at similar average power, 2.9x lower energy, and 5.2x more tokens per dollar. If true, it would make LLM serving cost and energy driven by memory, not by GPU silicon.","feed_headline":"GPU-free CXL memory system claims 2.3x LLM throughput","feed_subtitle":"Near-bank processing in CXL-attached memory serves decoding with 2.9x less energy and 5.2x more tokens per dollar.","key_machinery":"The load-bearing object is the hierarchical PIM-PNM memory device. Each CXL device holds 16 GDDR6-PIM chips; each chip has two channels, and every bank in a channel has a near-bank processing unit with a 16-MAC reduction tree that reads operands directly from the bank at 1 GHz, giving the device 512 TB/s of internal bandwidth. Because near-bank units only do MACs, a second level, PNM units with accumulators, reduction trees, exponent accelerators, and RISC-V cores, finishes the non-MAC operations through a 64 KB shared buffer. Arithmetic uses BF16 values, and exponentiation inside the PNM units uses a 10-order Taylor-series approximation. This hierarchy, plus the CXL communication primitives, is what lets a transformer block run end-to-end inside memory without a GPU or host compute.","core_discovery":"The paper's central claim is that a GPU-free server built from CXL-connected GDDR6-PIM devices can outperform a four-GPU A100 server for LLM inference by moving arithmetic to where the weights live. CENT maps the arithmetic of a transformer block onto two levels: near-bank processing units inside each DRAM bank perform the multiply-accumulate operations that make up over 99% of the arithmetic, while processing-near-memory accelerators and RISC-V cores next to the memory handle softmax, normalization, square roots, and divisions. A CXL 3.0 switch network with broadcast, multicast, gather, send, and receive primitives lets CENT distribute a model across devices using pipeline, tensor, or hybrid parallelism. On Llama2 7B/13B/70B with 4K contexts and maximum supported batch sizes, the paper reports 2.3x geomean throughput, 2.9x lower energy, and 5.2x more tokens per dollar than the GPU baseline; with 32K contexts the decode-throughput advantage grows to 3.3x.","pith_inferences":["If the unverified BF16/Taylor arithmetic preserves model quality, a natural next step is aggressive weight quantization, because the same memory-bound argument suggests CENT would tolerate it at least as well as GPUs.","The cost comparison is a single-server figure: in a multi-tenant fleet, sharing the host CPU and CXL switch across racks could improve or erode the 5.2x tokens-per-dollar number, so the headline TCO claim is not automatically a fleet-level bound.","The manuscript reports 2.9x lower energy in the abstract and results but 2.3x lower energy in the conclusion; this discrepancy is not flagged in the paper and should be resolved before relying on either number."],"forward_implications":["If CENT's numbers hold, LLM serving can shift from compute-optimized GPUs to memory-optimized CXL-attached PIM devices, with the host CPU only orchestrating and sampling.","The advantage grows with context length: at 32K contexts the paper measures 3.3x decoding throughput over GPUs, so long-context and reasoning workloads benefit most.","Prefill stays compute-bound and GPUs remain roughly 2.5x faster there, but because prefill is only about 2% of end-to-end GPU time, the decode-dominated total still favors CENT.","Scaling is feasible up to about 64 CXL devices per server with one switch and 128 devices with two-level switching, giving a path from small models to 70B-class serving without GPUs.","TCO gains are substantial: CENT's owned and rental costs are about 2.5x lower per hour, which is why tokens per dollar reach 5.2x."],"supporting_citations":[{"why":"Supplies the GDDR6-PIM near-bank processing-unit design, internal bandwidth figures, and the MAC-power assumption CENT adopts.","marker":"[56]"},{"why":"Provides the PIM baseline whose memory-density loss motivates CENT's CXL expansion and hierarchical design.","marker":"[13]"},{"why":"Quantifies HBM PIM density loss, supporting the need for CXL-attached memory capacity.","marker":"[57]"},{"why":"Defines the CXL 3.0 protocol underlying CENT's switch network, port design, and inter-device transactions.","marker":"[63]"},{"why":"Sets the GPU serving baseline and batch-128 throughput measurements CENT compares against.","marker":"[54]"},{"why":"Defines the Llama2 model suite (7B/13B/70B) used for all workload evaluations.","marker":"[106]"},{"why":"The DRAM simulator CENT extends to model GDDR6-PIM timing constraints.","marker":"[67]"},{"why":"The CXL processing-near-memory system CENT compares against to show the benefit of bank-level PIM.","marker":"[88]"},{"why":"The open-source RISC-V core used by PNM units for non-MAC operations.","marker":"[10]"}],"fun_headline_variants":["GPU-free CXL-PIM system runs LLM inference 2.3x faster","CXL memory with near-bank compute beats GPUs for LLMs by 2.3x","Replace GPUs with CXL memory for 5.2x tokens per dollar","Near-bank CXL memory cuts LLM energy use 2.9x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The premise that BF16 near-bank MACs and a 10-order Taylor-series exponential produce transformer output indistinguishable from FP16 GPU inference; the functional simulator checks instruction-level behavior but the paper reports no perplexity, accuracy, or downstream-task comparison, so if output quality degrades, CENT cannot replace GPUs regardless of speed or cost.","fun_headline_variants_meta":{"raw":{"variants":["GPU-free CXL-PIM system runs LLM inference 2.3x faster","CXL memory with near-bank compute beats GPUs for LLMs by 2.3x","Replace GPUs with CXL memory for 5.2x tokens per dollar","Near-bank CXL memory cuts LLM energy use 2.9x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000854,"raw_usage":{"total_tokens":3787,"prompt_tokens":1100,"completion_tokens":2687,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":716,"completion_tokens_details":{"reasoning_tokens":2595}},"tokens_in":716,"tokens_out":2687,"duration_ms":19379,"temperature":1.0,"reasoning_tokens":2595,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:15:04.816172+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's provided CENT simulator on Llama2-70B with the published traces, measure end-to-end perplexity on a held-out text set, and compare against FP16 A100 inference at batch 128; if perplexity diverges by more than a preset tolerance, the accuracy premise fails. Separately, reproduce the 2.3x throughput and 2.9x energy numbers with the supplied scripts; if they do not reproduce, the headline comparison fails.","supporting_citations":[{"cited_title":"System architecture and software stack for GDDR6-AiM","cited_arxiv_id":null,"evidence_quote":"Supplies the GDDR6-PIM near-bank processing-unit design, internal bandwidth figures, and the MAC-power assumption CENT adopts."},{"cited_title":"25.4 a 20nm 6gb function-in-memory DRAM, based on HBM2 with a 1.2 tflops pro- grammable computing unit using bank-level parallelism, for machine learning applications","cited_arxiv_id":null,"evidence_quote":"Quantifies HBM PIM density loss, supporting the need for CXL-attached memory capacity."},{"cited_title":"Specification","cited_arxiv_id":null,"evidence_quote":"Defines the CXL 3.0 protocol underlying CENT's switch network, port design, and inter-device transactions."},{"cited_title":"An LPDDR-based CXL-PNM Platform for TCO-efficient Inference of Transformer-based Large Language Models","cited_arxiv_id":null,"evidence_quote":"The CXL processing-near-memory system CENT compares against to show the benefit of bank-level PIM."}],"review_version":1}