{"id":"1704089c-0d29-4265-b8f7-547dc8deb489","arxiv_id":"2507.03522","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A matrix ISA extension that stores matrix tiles in ordinary vector registers and lets software choose tile geometry at runtime reports 1.35x average speedup over Intel AMX in simulation.","lead":"Computer chip makers are adding matrix instructions to CPUs, but the tile shapes are fixed and the register files are small, so real AI workloads run far below peak. This paper proposes MTE, a matrix instruction extension that reuses the normal vector registers and lets software set the tile geometry at runtime, and reports 1.35x average speedups over Intel's AMX in simulations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 1.35x-over-AMX claim rests on an unvalidated cvfma decomposition whose lane-interconnect cost is modeled, not measured; the MTE8s validation only covers the AMX-like configuration.","rationale":"I agree with the reader's verdict and identification of the weakest assumption. The paper's strongest claim, 1.35x over AMX, comes from comparing validated AMX (MTE8s) against unvalidated simulated MTE32v/MTE32s. The MTE8s validation is real evidence and gives credibility to the simulation infrastructure, but it does not validate the new mechanisms: the cvfma decomposition, the lane interconnect cost for A operands, the systolic array path, and the different physical-register/CSR behaviors. This is a correctness risk, not a disagreement with consensus. The proposed concrete test directly targets the modelled cost that would most plausibly erase the 1.35x claim: the lane interconnect flow described in Section IV-A2 and the systolic array latency in Table VII. A parametric latency sweep is feasible with the described trace-driven simulator and would settle whether the claim is robust. No evidence exists to reject the paper outright, so the reader's CONDITIONAL verdict remains appropriate and no adjustment is needed.","tokens_in":22684,"tokens_out":2282,"duration_ms":23263,"concrete_test":"Re-run the MTE32v/MTE32s simulations with the cvfma lane-interconnect cost modeled explicitly: assign a per-lane-transfer cost of 1 cycle (or a sweep of 0.5/1/2 cycles) rather than assuming it is hidden, and keep all other parameters identical. If the average MTE32s speedup over MTE8s drops below 1.10x, the 1.35x headline is not robust. Also report the MTE32s systolic-array MMA latency used in the simulator; if it is below 36 cycles, rerun with the same 36-cycle static latency as MTE32v.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the 1.35x speedup of MTE32s over MTE8s/AMX semantics (Section VI-A, Figure 7). MTE8s is validated against real AMX hardware on 52 convolutions with median 5.0% error (Section V-E, Figure 6), but MTE32s and MTE32v are never validated. The performance of both hinges on microarchitectural assumptions in Section IV-A2: tmul is decomposed into K cvfma micro-operations, where the A operand must move through the lane interconnect from lane to lane. A full 16x16x16 MTE32 multiplication therefore requires 16 sequential lane-to-lane transfers per K step, and the simulation assumes this cost is fully overlapped or accurately captured. If the true cvfma broadcast/shuffle cost is even a fraction of the compute latency, the MTE32v and MTE32s results could degrade below the claimed margins. Additionally, MTE32s uses a systolic array with 8 tile registers and 24 physical registers; the paper does not report the simulated latency of the systolic MMA or validate that model against any hardware or RTL. The reader's concern is correct and specific: the largest claimed speedup is only 1.35x, so a modest change in the unvalidated latency assumptions could erase or invert the headline result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Matrix Tile Extension (MTE), a matrix ISA that reuses vector registers as matrix tiles, defines tile geometry through a 64-bit CSR, and claims to decouple the ISA from the underlying microarchitecture. It describes two possible implementations (one based on a cvfma decomposition in a vector processor, one based on a systolic array), evaluates the designs with a trace-driven simulator, and validates one AMX-compatible configuration (MTE8s) against real Intel AMX hardware on 52 convolutions with a reported median error of 5.0%. The evaluation covers 75 convolution and 18 transformer GEMM workloads, reporting geometric mean speedups of MTE32s over Vector 1KB, Vector 2KB, SiFiveInt, and MTE8s of 2.67x, 2.45x, 2.3x, and 1.35x, respectively, with smaller end-to-end model speedups over MTE8s.","tokens_in":23004,"tokens_out":8260,"duration_ms":93843,"significance":"If the simulated results are reliable, the paper makes a useful contribution to the discussion of CPU matrix ISAs by showing that a geometry-agnostic matrix ISA reusing vector registers can outperform fixed-geometry, dedicated-register matrix ISAs on realistic convolution and transformer workloads. The paper's strengths are its concrete ISA definition, its large workload suite, the use of a JIT-based kernel generator, and the fact that the simulator is anchored to real AMX measurements through the MTE8s configuration. However, the central performance claims rest on microarchitectural latency assumptions for the MTE32v and MTE32s configurations that are not validated against hardware, RTL, or even a sensitivity study, and the comparison to AMX involves a possible precision mismatch. These issues must be addressed before the headline speedup can be accepted.","major_comments":[{"comment":"The headline speedups of MTE32v and MTE32s over MTE8s (1.16x and 1.35x in Section VI-A) are produced entirely by the trace-driven simulator for configurations that are never validated. Only MTE8s is validated against real AMX hardware (Section V-E, Figure 6). The MTE32v result depends on the modeled cvfma decomposition, in which the A operand moves through the lane interconnect from lane to lane, and the MTE32s result depends on an unvalidated systolic-array latency model. Table VII reports static/dynamic latencies of 36/64 for MTE32v and 36/16 for MTE32s, but the paper does not derive these values from a hardware implementation, RTL, or sensitivity analysis. Because the claimed margin over MTE8s is small, a modest increase in the cvfma broadcast/shuffle cost or the systolic MMA latency could erase or invert the reported speedup. Please add a sensitivity study varying these parameters, or validate at least one of the MTE32 configurations against an RTL/FPGA implementation.","section":"Section IV-A2 and Table VII"},{"comment":"The AMX comparison appears to compare different precisions. The paper states that the convolution workloads use 32-bit floating-point datatypes (Section V-B2), but Section II-C1 states that AMX TMUL supports only bf16-to-fp32 floating-point operations, not fp32 inputs. Figure 2 plots AMX efficiency against a bf16 peak while the simulated MTE approaches are evaluated at a 512 SP FLOP/cycle fp32 peak (Section V-A). If AMX executes bf16 and MTE executes fp32, then the reported 1.29x-1.35x speedups are not iso-precision and may partly reflect a data-type advantage rather than an ISA-efficiency advantage. Please clarify whether the AMX runs use the same fp32 inputs as the other approaches, and if conversions to bf16 are used, state this explicitly and discuss the precision implications.","section":"Sections V-B2, V-C, and Figure 9"},{"comment":"The abstract and conclusion describe the 1.35x figure as a speedup over the best state-of-the-art matrix ISA (AMX), but the number is actually a geometric mean speedup of MTE32s over MTE8s, a simulated configuration that reproduces AMX semantics on a different modeled microarchitecture. The only direct comparison to measured AMX is Figure 9, which covers convolutions and reports a 1.29x speedup for MTE32v. The manuscript should state clearly in the abstract and conclusions that the headline speedup is relative to a simulated AMX-semantic baseline, and should either report measured AMX results for the full workload set or provide a quantitative argument that MTE8s faithfully captures AMX performance for the transformer and large-OC workloads as well.","section":"Abstract and Section VI-A"},{"comment":"The simulator validation is limited to 52 convolutions with output-channel counts up to 256, and it does not cover the transformer GEMM workloads or the larger categories in Figure 7 where the largest MTE gains appear. The end-to-end results in Figure 8 show the highest MTE32s speedups for BERT and GPT-2 (1.20x and 1.22x), yet no measured AMX or MTE8s data are provided for transformer GEMMs. A median error of 5.0% on small-OC convolutions does not establish simulator accuracy for the workload classes that drive the paper's main end-to-end claims. Please extend the validation to transformer GEMMs or explicitly bound the simulator uncertainty for those workloads.","section":"Section V-E and Figure 6"}],"minor_comments":[{"comment":"The text refers to the '8084+ processor'; this should be '8480+'.","section":"Section VI-A2"},{"comment":"There are several typos, including 'discuses' for 'discusses' and 'implementetion' for 'implementation'.","section":"Section IV-C"},{"comment":"The CSR table lists ttype[i,o] as 8 bits with two 4-bit fields, but Section III-C1 describes the ttypeio immediate as 3 bits; please clarify how the 3-bit immediate maps to the two 4-bit fields.","section":"Section III-B and III-C1"},{"comment":"The area analysis in Table VIII reports register-file areas but does not include the systolic-array area for MTE8s and MTE32s; since Table VI describes the systolic array as a separate accelerator, please either include its area contribution or state why it is omitted.","section":"Section V-D"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely topic and the workload suite is substantial, but the validation gap for the headline configurations is the key risk. The precision mismatch between AMX and the simulated MTE approaches, if confirmed, would directly affect the interpretation of the speedup claims. I would encourage the editor to ask for a sensitivity analysis and a clear precision statement before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a real ISA proposal with a novel combination—reusing the vector register file for matrix tiles, letting software program tile geometry at runtime through a single CSR, and decomposing MMA into vector micro-ops (cvfma)—and they back it with the right kind of evidence: real AMX measurements, a simulator validated against that hardware on 52 convolutions (median error 5%), and a sizable workload set (75 conv + 18 transformer GEMMs). The motivation is solid: AMX sits at 35.4% average efficiency on those shapes, and the paper explains why (8 tile registers, rigid geometry). If MTE's flexibility works as described, it's a useful contribution to the matrix-ISA design space.\n\nThe soft spot is where the headline lives. The 1.35x speedup over AMX comes from comparing simulated MTE32v/MTE32s against simulated MTE8s (which reproduces AMX semantics). MTE8s is validated, but MTE32v and MTE32s are not. Their performance hinges on modeling choices in Section IV-A2: the cvfma lane-interconnect flow of the A operand and the systolic-array latency, plus the assumed static 36-cycle cost. The margin is small enough that a modest change in those assumptions could erode or flip the result. The stress-test note is right: the largest claimed gain is only 1.35x, and the end-to-end model speedups are even smaller (1.05-1.22x), so the practical performance advantage is fragile as currently evidenced. I also think the abstract overreaches: it states the speedup without saying the MTE numbers are simulated, which a careful reader will spot in Section VI but a casual one won't.\n\nOther concerns are minor: the simulator and rvjit JIT are not released, which hurts reproducibility; the transformer set is only 18 GEMMs; and the energy numbers are McPAT-based, which is standard but loose. The related-work positioning is fair, though 'first completely decoupled' is a bit strong given MME and SiFiveInt already move in that direction.\n\nWho is this for? Computer architects working on matrix/vector ISA extensions and anyone designing GEMM accelerators for AI shapes. It deserves a serious referee: the idea is new, the baseline validation is good practice, and the gaps are fixable in revision. My recommendation: send it out, but require the authors to (1) release the simulator and JIT, (2) add sensitivity analysis on the cvfma and systolic-array costs, and (3) qualify the abstract to say the MTE results are simulated. With those, the claim becomes credible.","headline":"A flexible matrix ISA worth talking about, but the 1.35x headline rests on unvalidated simulation; still deserves review.","tokens_in":23575,"tokens_out":3719,"would_cite":true,"duration_ms":41126,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims a matrix ISA that stores tiles in the existing vector register file and lets software set tile geometry at runtime outperforms fixed-geometry matrix ISAs like Intel AMX on convolution and transformer GEMMs, with 1.35x…","keywords":["matrix instruction set architecture","GEMM","vector register file","tile geometry","AMX","RISC-V V","systolic array","transformer inference"],"falsifier":"Build the MTE32v and MTE32s configurations in RTL (or a cycle-accurate simulator calibrated to a real long-vector core) and measure the actual latency of the tfmul/tmul instructions across the full range of tile shapes; if the measured static plus dynamic cost exceeds the modeled 36-cycle static and active-vector-length-dependent cost, recompute the convolution and transformer speedups against a real AMX baseline and check whether the 1.35x average holds.","tokens_in":22449,"feed_emoji":"🧮","tokens_out":8912,"duration_ms":93311,"temperature":0.7,"pith_summary":"Modern CPUs increasingly add matrix instruction-set extensions to speed up GEMMs, but the paper argues that existing designs--fixed tile geometries and dedicated matrix registers--leave performance on the table for the small, tall, or skinny matrices common in convolution and transformer workloads. It proposes the Matrix Tile Extension (MTE), a matrix ISA that stores matrix tiles in the existing vector register file and lets software program the tile shape (M, N, K) at runtime through a 64-bit control register. Because the ISA is decoupled from the microarchitecture, the same code can run on a long-vector implementation or a systolic-array implementation, and software can use all vector registers for loop unrolling instead of the few dedicated tile registers of current matrix ISAs. On a trace-driven simulator modeling equal peak throughput, MTE is reported to achieve average speedups of 2.67x, 2.45x, 2.3x, and 1.35x over an 8192-bit vector ISA, a 16384-bit vector ISA, a 4x4-tile matrix ISA, and an AMX-style matrix ISA, respectively. The central practical claim is that flexibility in tile geometry plus reuse of the vector register file is worth more than a dedicated fixed-shape matrix unit.","feed_headline":"Matrix ISA that reuses vector registers beats AMX by 1.35x","feed_subtitle":"A geometry-agnostic matrix ISA with a 64-bit CSR claims 1.35x average speedup over Intel's AMX in simulation.","key_machinery":"The load-bearing mechanism is MTE's reinterpretation of the vector register file as matrix tiles plus a runtime-programmable tile geometry held in a 64-bit CSR. The formulas $M = VLEN/RLEN$, $N = RLEN/SEW$, $K = min(M,N)$ (uniform precision) and the transposed-B variant for mixed precision define the largest tile that fully uses a vector register, and the tss instructions grant whatever smaller geometry software requests. On the vector microarchitecture, the tfmul/tmul instructions are decomposed into K cvfma micro-operations: the B operand enters each lane by implicit broadcast, the C operand is laid out lane-locally, and the A operand is shifted across lanes through the lane interconnect; the vector mask and programmable vector length disable inactive rows and columns. On the systolic microarchitecture, the same MTE instructions drive a fixed grid of multiply-accumulate units. This separation of ISA from implementation is what makes the geometry-agnostic claim concrete, and the reuse of the vector register file is what gives software 32 architectural registers instead of AMX's eight tile registers.","core_discovery":"The paper's central claim is that the rigidity of current matrix ISAs, not their peak throughput, is what makes them deliver suboptimal GEMM efficiency, and that a matrix ISA can be made both flexible and cheap. MTE treats each vector register as a rank-2 tile: with a design-time row length RLEN and runtime element width SEW, a vector register of VLEN bits holds VLEN/RLEN rows of RLEN/SEW elements, and the CSR fields tm, tn, and tk encode the active M, N, and K dimensions. Uniform-precision kernels use a row-major tile interpretation with $M = VLEN/RLEN$, $N = RLEN/SEW$, $K = min(M,N)$; mixed-precision kernels store B transposed so that the wider output type does not waste vector capacity. The tile-multiply instruction is implemented either as a systolic array or as a decomposition into K component vector FMAs (cvfma) on a standard vector processor, using implicit broadcast for the B operand and the lane interconnect to move A elements across lanes. The authors report that this design lets MTE vectorize GEMMs across M, N, and K, retire roughly 14x fewer vector/matrix instructions than the 8192-bit vector baseline, and beat an AMX-semantics configuration by 1.35x on average while requiring only a few new instructions and a 64-bit CSR.","pith_inferences":["If the real cost of the lane-interconnect A-operand flow is higher than modeled, the MTE32v advantage over the MTE8s/AMX baseline would shrink; the paper's 1.35x speedup should be read as conditional on the simulator's cost assumptions.","The design suggests a broader principle: dedicated matrix register files may be unnecessary, and future matrix extensions could reinterpret existing vector storage, freeing die area for more registers or compute.","A direct test of the paper's bottleneck analysis is to simulate AMX with more than eight tile registers; the paper predicts efficiency would rise with register count, which would isolate the register-count effect from the tile-geometry effect."],"forward_implications":["If MTE's results hold, a matrix ISA can be added to an existing vector ISA with little extra architectural state, making matrix acceleration available on cores that already have a vector register file.","Software and JIT code generators could choose tile geometry at runtime to match each GEMM's M, N, and K, eliminating the layout transposes and memory round-trips that fixed-geometry AMX kernels need for transformer shapes.","With 32 architectural registers available to hold A, B, and C tiles, compilers can unroll the M loop more deeply and reuse B tiles more often, which is the mechanism behind the claimed speedups.","The same MTE code can target a long-vector implementation and a systolic-array implementation, so hardware vendors could ship either microarchitecture without changing the ISA contract.","Mixed-precision GEMMs can use the full vector register length because storing B transposed removes the output-type constraint on N, which matters for bf16-to-fp32 and int8-to-int32 workloads."],"supporting_citations":[{"why":"Supplies the AMX baseline, its eight-tile register semantics, and the state-of-the-art matrix ISA that MTE8s reproduces and MTE32 must beat.","marker":"[5]"},{"why":"Supplies the SiFiveInt 4x4-tile matrix ISA that serves as the second state-of-the-art matrix comparison.","marker":"[7]"},{"why":"Defines the RISC-V V vector ISA and vector-length-agnostic model that MTE reuses for register storage, masks, and element-wise operations.","marker":"[13]"},{"why":"Is the DNN kernel library through which convolution and transformer workloads are collected and for which the paper's JIT generator produces kernels.","marker":"[45]"},{"why":"Is the Intel Xeon Platinum 8480+ processor used for the real AMX measurement that validates the simulator.","marker":"[35]"},{"why":"Supplies the direct-convolution SIMD kernel recipe and workload classification used to build the vector baselines and convolution workloads.","marker":"[2]"},{"why":"Provides the NEC SX-Aurora vector-core parameters on which the evaluated vector processing unit is based.","marker":"[44]"}],"fun_headline_variants":["Flexible matrix ISA adapts to data formats, beats AMX by 1.35x","Vector-register matrix ISA retires 14x fewer instructions, beats AMX","Flexible matrix ISA decouples from microarchitecture, 1.35x faster than AMX","Reusing vector registers as tiles: matrix ISA beats AMX by 1.35x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simulated microarchitectural costs for the new tile-multiply hardware--the 36-cycle static latency, the vector-length-dependent dynamic latency, and the lane-interconnect flow of A operands--match a real implementation, because only the AMX-like MTE8s configuration was validated against real silicon (median 5% error) and the MTE32 configurations behind the 1.35x claim were not.","fun_headline_variants_meta":{"raw":{"variants":["Flexible matrix ISA adapts to data formats, beats AMX by 1.35x","Vector-register matrix ISA retires 14x fewer instructions, beats AMX","Flexible matrix ISA decouples from microarchitecture, 1.35x faster than AMX","Reusing vector registers as tiles: matrix ISA beats AMX by 1.35x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000862,"raw_usage":{"total_tokens":3825,"prompt_tokens":1115,"completion_tokens":2710,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":731,"completion_tokens_details":{"reasoning_tokens":2614}},"tokens_in":731,"tokens_out":2710,"duration_ms":22066,"temperature":1.0,"reasoning_tokens":2614,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:08:45.789655+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build the MTE32v and MTE32s configurations in RTL (or a cycle-accurate simulator calibrated to a real long-vector core) and measure the actual latency of the tfmul/tmul instructions across the full range of tile shapes; if the measured static plus dynamic cost exceeds the modeled 36-cycle static and active-vector-length-dependent cost, recompute the convolution and transformer speedups against a real AMX baseline and check whether the 1.35x average holds.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the AMX baseline, its eight-tile register semantics, and the state-of-the-art matrix ISA that MTE8s reproduces and MTE32 must beat."},{"cited_title":"Sifive intelligence extensions documentation,","cited_arxiv_id":null,"evidence_quote":"Supplies the SiFiveInt 4x4-tile matrix ISA that serves as the second state-of-the-art matrix comparison."},{"cited_title":"The risc-v vector extension,","cited_arxiv_id":null,"evidence_quote":"Defines the RISC-V V vector ISA and vector-length-agnostic model that MTE reuses for register storage, masks, and element-wise operations."},{"cited_title":"Oneapi deep neural network library,","cited_arxiv_id":null,"evidence_quote":"Is the DNN kernel library through which convolution and transformer workloads are collected and for which the paper's JIT generator produces kernels."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the Intel Xeon Platinum 8480+ processor used for the real AMX measurement that validates the simulator."},{"cited_title":"Anatomy of high-performance deep learning convolutions on simd architectures,","cited_arxiv_id":null,"evidence_quote":"Supplies the direct-convolution SIMD kernel recipe and workload classification used to build the vector baselines and convolution workloads."},{"cited_title":"Vector engine processor of NEC’s brand- new supercomputer SX-Aurora TSUBASA,","cited_arxiv_id":null,"evidence_quote":"Provides the NEC SX-Aurora vector-core parameters on which the evaluated vector processing unit is based."}],"review_version":1}