{"id":"97d2c155-d9c1-4e02-b980-51358381d124","arxiv_id":"2607.25866","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Trixi.jl scales to 61,440 CPU cores with good parallel efficiency, and a precompiled system image removes Julia's startup-load bottleneck at scale.","lead":"This paper tests whether Julia, a fast high-level programming language, can handle huge supercomputer runs; the Julia-based simulation code Trixi.jl scaled to 61,440 CPU cores and kept up with a Fortran solver. It also documents a startup bottleneck when thousands of Julia processes load packages at once, and shows a precompiled system image fixes it.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Superlinear cache speedups distort the single-node baseline; the reported 0.83 parallel efficiency at 61,440 cores overstates scaling, as a cache-corrected baseline yields ~0.43 efficiency.","rationale":"We agree with the reader that the single-node baseline is the weakest and most load-bearing assumption. The claim of 'scales well' is operationalized through the efficiency number, and that number is inflated by superlinear cache speedups. A corrected baseline changes the efficiency from 0.83 to about 0.43, which is a material difference. This is not an internal inconsistency—the paper reports the raw data and even notes the marginal efficiency—but the abstract and conclusions present the 0.83 value without this caveat. Other issues (lack of error bars, mixed Julia versions, no shipped scripts) are secondary because they affect precision and reproducibility but would not change the direction of the result if the baseline were fixed. The proposed test isolates the effect of the baseline choice and would settle whether the headline efficiency is robust.","tokens_in":10406,"tokens_out":7618,"duration_ms":62503,"concrete_test":"Recompute the parallel efficiency for the Trixi.jl Taylor-Green run on JURECA (Fig. 1) using the DUPS at 16,384 ranks as the reference baseline instead of the single-node baseline. Report the efficiency at 61,440 ranks and the marginal efficiency per doubling of rank count from 16,384 to 61,440. If either value is substantially below the reported 0.83/ideal (e.g., corrected efficiency <0.5), the claim of 'good scaling' to 61,440 cores should be revised to acknowledge cache-enhanced early scaling and poor tail scaling. Additionally, plot the per-rank working-set size and identify the core count where the superlinear bump ends to justify the chosen baseline.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Fig. 1 reports a parallel efficiency of 0.83 for Trixi.jl at 61,440 ranks relative to a single-node baseline, and the abstract uses this to claim 'scales well.' However, Fig. 1 also shows a superlinear speedup peaking at efficiency 1.92 at 16,384 ranks, attributed to cache effects. Using the 16,384-rank point as the cache-corrected baseline (where the per-rank working set fits in cache), the efficiency at 61,440 ranks drops to about 0.43, and the marginal efficiency per doubling from 16,384 to 61,440 is only ~0.65, as the authors acknowledge. Thus the headline efficiency is an artifact of comparing against an unusually slow single-node baseline. A claim of 'good scaling' to 61,440 cores is therefore not robust; the data support good scaling only up to roughly 16,384 ranks, with substantial degradation beyond. The central claim in the abstract and conclusions relies on this inflated efficiency, so the burden is on the authors to report a defensible baseline or revise their scaling interpretation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a strong-scaling study of the Julia-based DGSEM code Trixi.jl against the Fortran code FLUXO on three CPU clusters (JURECA, JUWELS Booster, MOGON NHR), using the 3D Taylor-Green vortex and an MHD Alfvén wave test. The central claim, stated in the abstract and conclusions, is that Trixi.jl scales well to 61,440 CPU cores on 480 nodes using standard MPI.jl, with a parallel efficiency of 0.83 relative to a single-node baseline. The paper also documents Julia startup and code-loading bottlenecks and shows that a custom system image alleviates them, and it investigates an MPI all-reduce bottleneck in adaptive time stepping.","tokens_in":10673,"tokens_out":3269,"duration_ms":32126,"significance":"If the scaling result is robust, the paper provides a valuable data point for the HPC community: a high-level, JIT-compiled language can be a viable alternative to Fortran for massively parallel CPU simulations. The comparison to an established Fortran solver on multiple clusters is a strength, and the paper is commendably explicit about startup costs, compilation, and the role of problem size. The authors' transparency about the superlinear speedup and its effect on efficiency is also a positive feature. However, the headline efficiency metric is sensitive to the choice of baseline, and the manuscript's own numbers indicate that the 'scales well up to 61,440 cores' formulation overstates the robustness of the scaling data.","major_comments":[{"comment":"The central claim of 'good scaling up to 61,440 cores' rests on a parallel efficiency of 0.83 measured against a single-node baseline. In the same results paragraph, the authors note that the efficiency peaks at 1.92 at 16,384 ranks because of cache effects and that the marginal efficiency beyond that point is only about 0.65 per doubling. Using the 16,384-rank point as a cache-corrected baseline, the efficiency at 61,440 ranks is approximately 0.43. The abstract and conclusions do not carry this qualification. Please report a cache-corrected efficiency or revise the scaling claim to something like 'good scaling up to about 16,384 ranks, with continued but substantially degraded throughput beyond.'","section":"Results (Fig. 1) and Summary and conclusions"},{"comment":"The manuscript states that JURECA runs use Julia v1.8.3 with bounds checking disabled, while JUWELS and MOGON use Julia v1.10.6 with bounds checking enabled. Because the headline 61,440-core result comes from JURECA only, this inconsistency does not invalidate that specific result, but it is a confound when comparing scaling behavior across clusters and when generalizing the claim that 'Julia scales well.' Please state whether the Julia version or bounds-checking setting materially affects the JURECA results, and explicitly restrict cross-cluster generalizations if they are not controlled.","section":"Experimental setup (software versions)"},{"comment":"Each simulation was run five times and only the minimum is reported. The paper gives no measure of run-to-run variability. Given the superlinear speedup and the observed MPI time-step bottleneck at high rank counts, the minimum can be sensitive to outliers such as OS jitter or MPI startup effects. At minimum, report min-max ranges or a standard deviation for the key data points, and indicate whether the observed trends are stable across the five runs.","section":"Results (all scaling figures)"}],"minor_comments":[{"comment":"The sentence 'This shows the role that the problem size plays in the scalability. Keeping the problem size low increases the overhead...' appears in section 'Results' with a broken reference 'section .' and is repeated near the time-step analysis. The manuscript has several unresolved section cross-reference placeholders ('section .', 'As outlined in section') that must be fixed.","section":"Adaptive Time Step Bottleneck"},{"comment":"The claim that 'at about the break-even point the scaling worsens' is based on visual coincidence. A quantitative statement (e.g., comparing the rate of increase of dt time to the rate of decrease of RHS time) would strengthen the causal argument.","section":"Figure 8 and text"},{"comment":"The custom system image section is introduced as 'The code loading mechanism described in section poses challenges...' with another missing reference; also the text says 'The timing tests are preformed' (typo) and 'MPI paralellized' (typo) in the Introduction.","section":"Package loading / avoiding I/O bottleneck"},{"comment":"Figure 6's caption reads 'compressible MHD and Euler equations', but the text suggests the figure compares the Euler Taylor-Green problem with the MHD Alfvén wave for MOGON. Please clarify the exact configuration and ensure the caption matches the plotted data.","section":"Figures 6 and 7"}],"recommendation":"major_revision","confidential_remarks":"The superlinear-baseline issue is real and is acknowledged inside the paper, which is to the authors' credit. With a revised framing or a cache-adjusted efficiency, the paper's central message would be more defensible. I would not reject the paper because the underlying scaling data are extensive and the comparison to FLUXO is valuable; however, the abstract and conclusions currently overstate the result as 'scales well up to 61,440 cores,' and this needs to be fixed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this is a solid, pragmatic scaling study, not a breakthrough. The new data are real—Trixi.jl running to 61,440 CPU cores on JURECA, plus a concrete measurements of the startup I/O bottleneck and a custom system image that fixes it. That extends the CPU-core scale from the 2,048-core and 4,096-GPU results they cite, so it is a useful data point for the Julia HPC community.\n\nWhat the paper does well: it compares against FLUXO on the same machines, runs a second test case (Alfvén waves), and includes a nice diagnostic—when the global time-step reduction is replaced by a fixed time step, the code scales near-ideally. That isolates the bottleneck and shows Trixi.jl's compute core is not the problem. The authors also explicitly report the superlinear speedup (efficiency 1.92 at 16,384 ranks) and the marginal efficiency of ~0.65 per doubling beyond that. That transparency is better than most benchmark papers.\n\nThe soft spots are real but not fatal. The abstract and conclusion say \"scales well up to 61,440 cores,\" based on a parallel efficiency of 0.83 relative to a single-node baseline. But their own figure shows the single-node baseline is artificially slow due to cache effects. Using their numbers, efficiency at 61,440 relative to the cache-saturated 16,384-rank point is about 0.43, and the marginal efficiency per doubling is 0.65, as they acknowledge. So the data actually say throughput keeps increasing but with strongly diminishing returns—not obviously \"scales well.\" The fix is simple: report efficiency relative to the best (cache-saturated) point, or at least put the superlinear caveat in the abstract.\n\nAlso, the methodology is a bit loose: Julia v1.8.3 with bounds checking disabled on JURECA, v1.10.6 with bounds checking enabled on JUWELS/MOGON; only the minimum of five runs is reported; no error bars; and no benchmark scripts or data are provided. For a measurements-driven paper, that last one is a reproducibility gap.\n\nThe citation pattern is fine; the self-cites describe Trixi.jl and are not the evidence for the central result.\n\nBottom line: this paper deserves a serious referee, but it needs revision before acceptance. Ask for a defensible baseline, consistent Julia configuration across clusters, and the scripts/data. For the right reader—an HPC practitioner wondering whether Julia is ready for CPU clusters—this is worth the time.","headline":"A useful and honest Julia scaling study that oversells its headline efficiency: the superlinear cache baseline inflates the 0.83 number, but the paper itself discloses the issue.","tokens_in":11182,"tokens_out":4939,"would_cite":true,"duration_ms":45933,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65Y05","65M60"],"pacs":[],"model":"deepseek-v4-flash","headline":"Julia, a high-level dynamic language, can run a production CFD solver on 61,440 CPU cores with close-to-ideal scaling.","keywords":["Julia","high-performance computing","parallel scalability","MPI","discontinuous Galerkin","Taylor-Green vortex","magnetohydrodynamics","system image"],"falsifier":"Re-run the Taylor-Green strong-scaling test with a cache-corrected baseline—e.g., using the weak-scaling efficiency curve or padding the per-rank working set so the single-node run is not cache-thrashing—and compare marginal efficiency per doubling of ranks. If the 61,440-core efficiency drops below about 0.5 against that baseline, the headline scaling result is an artifact of the cache-effect speedup rather than true HPC scaling.","tokens_in":10328,"feed_emoji":"🖥️","tokens_out":6968,"duration_ms":60320,"temperature":0.7,"pith_summary":"The paper aims to close a gap in the evidence for Julia as an HPC language: it asks whether Julia's just-in-time compilation and high-level syntax can survive traditional MPI-scale workloads. It answers with a scaling study of Trixi.jl, a Julia solver for conservation laws, on up to 61,440 CPU cores across several large clusters, compared against the established Fortran solver FLUXO. The paper shows that, given enough work per core, Trixi.jl retains a parallel efficiency of 0.83 at 61,440 ranks relative to a single node, and it identifies two practical obstacles—startup code loading and the global time-step reduction—plus a fix for the first. The result matters because it suggests scientist-developers can get developer productivity and Fortran-class scaling in one language.","feed_headline":"Julia solver scales to 61,440 CPU cores","feed_subtitle":"A high-level dynamic language can stand in for Fortran in large MPI runs.","key_machinery":"The load-bearing mechanism is Trixi.jl's MPI communication strategy: one non-blocking message exchange between neighbors per right-hand-side evaluation, overlapped with local computation, at the cost of recomputing some fluxes (whereas the Fortran code FLUXO uses a 'ping-pong' exchange that computes each flux once but needs two messages). The second piece of machinery is a custom system image built with Julia's package compiler, which precompiles all methods for a given simulation into a single large file; this removes both the startup compilation latency and the many-small-files I/O bottleneck that otherwise appears at thousands of MPI processes.","core_discovery":"Julia's standard MPI library is sufficient for large parallel simulations. On the Taylor-Green vortex, Trixi.jl reaches 61,440 MPI ranks on 480 nodes with a parallel efficiency of 0.83 relative to a single-node run, and it outperforms FLUXO in raw throughput while showing slightly weaker scaling at extreme rank counts. A superlinear speedup, with efficiency peaking at 1.92 for 16,384 ranks, is attributed to cache effects; the marginal efficiency beyond that point is about 0.65 per doubling. The adaptive time-step calculation—a global MPI allreduce—becomes a bottleneck around 18,432 ranks, overtaking the right-hand-side computation. With a fixed time step, the code shows near-ideal scaling to","pith_inferences":["If the cache-effect explanation is right, then the reported 0.83 efficiency at 61,440 cores is optimistic: against a baseline that removes cache-thrash gains on small rank counts, the true efficiency would be lower and the headline would change. A simple test would be to measure with per-rank working sets that stay within cache at the baseline.","The system-image fix points to a wider lesson: Julia's JIT startup cost is not an obstacle once deployment artifacts are precompiled per machine—this could translate into HPC-friendly container images or job templates.","The same scaling methodology could be applied to other Julia solvers with different communication patterns; a natural extension is to study whether implicit time integrators or AMR-heavy workloads show the same scaling limits.","Since the bottleneck is one global allreduce per step, replacing it with hierarchical or asynchronous reduction is an immediately testable change that should push the scaling plateau well beyond 18k ranks if communication is the issue."],"forward_implications":["Julia can be a drop-in language for large MPI simulations in scientific domains, at least for explicit high-order DG methods.","Problem size per rank controls scaling: with 8x more elements per rank, Trixi.jl scales to 61,440 ranks with about 83% efficiency; with small problems it flattens near 16k-18k ranks.","The global time-step reduction is a measurable, localizable bottleneck—at around 18,432 ranks it costs more than the RHS itself—so replacing that single allreduce is a concrete optimization target.","Startup code loading at scale can be neutralized with custom system images, removing a barrier that had made Julia's first-run behavior incompatible with short HPC jobs.","Trixi.jl's throughput advantage over FLUXO at moderate rank counts comes with a communication strategy that duplicates some flux computations; at very high rank counts this trade-off starts to hurt."],"fun_headline_variants":["Julia CFD scales to 61K cores, beats Fortran","Trixi.jl: 61,440 MPI ranks at 83% efficiency","Superlinear speedup in Julia on 16K cores","Julia MPI: near-ideal scaling to 61K cores"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim of 0.83 parallel efficiency at 61,440 cores rests on using the single-node run as the baseline, even though the reported superlinear speedup indicates the single-node baseline is partly a cache-thrashing artifact; a cache-corrected baseline would lower the headline efficiency.","fun_headline_variants_meta":{"raw":{"variants":["Julia CFD scales to 61K cores, beats Fortran","Trixi.jl: 61,440 MPI ranks at 83% efficiency","Superlinear speedup in Julia on 16K cores","Julia MPI: near-ideal scaling to 61K cores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000159,"raw_usage":{"total_tokens":1035,"prompt_tokens":684,"completion_tokens":351,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":428,"completion_tokens_details":{"reasoning_tokens":276}},"tokens_in":428,"tokens_out":351,"duration_ms":4073,"temperature":1.0,"reasoning_tokens":276,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T01:14:46.165831+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the Taylor-Green strong-scaling test with a cache-corrected baseline—e.g., using the weak-scaling efficiency curve or padding the per-rank working set so the single-node run is not cache-thrashing—and compare marginal efficiency per doubling of ranks. If the 61,440-core efficiency drops below about 0.5 against that baseline, the headline scaling result is an artifact of the cache-effect speedup rather than true HPC scaling.","supporting_citations":[],"review_version":1}