Pith. sign in

REVIEW 3 major objections 4 minor 3 cited by

Africanus I. Scalable, distributed and efficient radio data processing with Dask-MS and Codex Africanus

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that a data-access layer and algorithm library built on Dask can make radio astronomy pipelines scalable, distributed, and quick to develop, and it demonstrates near-linear strong scaling of a DFT-based visibility…

desk verdict Solid software paper with real libraries and an honest limitation: the scaling evidence covers only a benign compute-bound case, and the paper says so itself. read the letter →

arxiv 2412.12052 v2 pith:YRVNO7GX submitted 2024-12-16 astro-ph.IM

Simon J. Perkins , Jonathan S. Kenyon , Lexy A.L. Andati , Hertzog L. Bester , Oleg M. Smirnov , Benjamin V. Hugo This is my paper · ORCID
classification astro-ph.IM
keywords radiointerferometrydistributedcomputingDaskmeasurementsetsDFTpredictinterferometerequationclouddataaccesslayer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Modern radio interferometers produce data volumes that make single-node processing impossible, but the existing formats and software ecosystems predate distributed computing. This paper introduces two Python libraries, Dask-MS and codex africanus, that together form a data access layer and algorithm library for radio astronomy built on Dask. The claim is that this combination lets developers write flexible, easy-to-maintain techniques that still run with high performance, from a laptop to HPC nodes to cloud clusters, without rewriting code. The quantitative evidence is a distributed DFT-based visibility prediction that shows near-linear strong scaling up to about 80 cloud workers and sustained weak scaling as the problem size grows. If the framework holds up in the more complex calibration and imaging applications described in later papers, it would give the field a common substrate for end-to-end pipelines.

What carries the argument

The load-bearing mechanism is the pairing of Dask-MS, which represents Measurement Set columns as lazily evaluated Dask arrays grouped into XArray datasets, with codex africanus, which implements algorithms as two-layer functions: a Numba-compiled kernel operating on NumPy array chunks, and a thin wrapper that turns the kernel into a Dask graph transformation. This split is what makes the framework work: the developer composes ordinary NumPy-like arrays, while Dask schedules the resulting graph across threads, processes, or cluster nodes. The scheduler plugin is the second mechanism; it reads dimension annotations attached to Dask tasks and stripes row chunks across workers, so related columns such as DATA and WEIGHT_SPECTRUM stay co-located and avoid slow network transfers.

What would settle it

Run QuartiCal and pfb-imaging on a 100-worker cloud cluster with the same data set and chunking; if wall time stops improving well before the task-per-thread limit seen for the DFT predict, or if network transfer dominates runtime, the framework's claim to generalize beyond simple compute-bound algorithms is contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the radio-astronomy data model can be mapped onto lazy Dask arrays and XArray datasets, and that this mapping supports both a familiar programming model and genuine distributed execution. Dask-MS exposes Measurement Set-like data, stored as CASA tables, Zarr, or Arrow, as XArray datasets of Dask arrays, so reads and writes become lazy graph nodes instead of blocking I/O. codex africanus provides algorithms, including a radio interferometer measurement equation (RIME) predictor and w-gridding, as functions that ingest and emit Dask arrays, so applications compose as dataflow graphs. A scheduler plugin uses dimension annotations to co-locate related chunks on the same worker. On a cloud cluster, a Numba-based DFT predictor achieved strong scaling that tracks linear speedup from 10 to roughly 80 workers, with wall time falling from 934 to 145 seconds, and weak scaling held while the problem size grew quadratically with worker count.

Load-bearing premise

The scaling results come from a compute-bound DFT predict with a simple and independent task graph; the paper assumes the same Dask scheduling approach will scale for the communication-heavy calibration and imaging workloads, whose results appear in later papers.

Editorial extensions

If this is right

  • Pipelines written against Dask-MS and codex africanus can run unchanged on a laptop, an HPC cluster, or a cloud cluster, because the scheduling target is selected at run time rather than at development time.
  • The same visibility-prediction algorithm can be reused across calibration, simulation, and imaging, since codex africanus exposes it as a composable Dask operation.
  • A DFT predict on a 345 GB MeerKAT observation reached near-linear strong scaling to about 80 workers, and weak scaling held on problems growing quadratically with worker count, showing that compute-bound algorithms do not hit a Dask scheduling ceiling.
  • The co-location strategy and the Dask Client and actor interfaces provide two complementary routes for applications: collection-based graphs for calibration-style processing, and stateful actor workflows for memory-bound iterative imaging.
  • Exporting archive data to Zarr makes Measurement Set-like data compatible with object stores, which the paper argues is necessary for cloud-native radio astronomy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My editorial inference: the same code paths that run on a laptop and on 100 cloud nodes suggest a development model where new algorithms are prototyped on small subsets and then scaled unchanged; the paper does not explicitly frame this as a workflow benefit.
  • Because Dask collections lack buffered pipelining and copy data on mutation, imaging-style workflows that mutate large visibility arrays are expected to be pushed onto the lower-level Client and actor interface; one testable extension is a benchmark comparing actor-based pfb-imaging with a collection-based equivalent on identical data.
  • The '80 workers' saturation point is not a fundamental limit but a reflection of the number of tasks per thread in that particular run; varying chunk sizes and the number of model components should shift the saturation point, and the co-location plugin may change it.
  • The paper's reported cloud cost for the longest weak-scaling run suggests that the framework could make high-performance radio processing accessible to groups without local HPC, but the estimate omits data ingress and egress charges, which would alter the cost picture in practice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents two open-source Python libraries, Dask-MS (a Dask-based data access layer for Measurement Set-like data) and Codex Africanus (a library of radio astronomy algorithms exposed as NumPy/Dask functions), and argues that they form the data access and algorithmic building blocks for a new generation of distributed, scalable radio astronomy pipelines. The design emphasizes flexibility and ease of development within the PyData ecosystem, using Dask for distributed computing and Numba for performance. Quantitative evidence is provided in Section 8 through strong- and weak-scaling experiments of a DFT-based visibility predictor (crystalball) on AWS. The authors explicitly defer scaling studies of the more complex flagship applications (QuartiCal and pfb-imaging) to companion papers II and III.

Significance. If the framework delivers on its claims, the paper makes a valuable contribution to radio astronomy software by providing a modern, open, PyData-based alternative to legacy CASA/AIPS-centric pipelines, with potential applicability to MeerKAT, SKA, and other next-generation instruments. The strengths are the coherent design decoupling data access from algorithms, the use of widely adopted open-source components, and the availability of reproducible code and performance data. The Section 8 results, despite caveats, show that a compute-bound algorithm can be run in a distributed cloud environment with reasonable scaling up to roughly 80 workers. The paper is also part of a clearly structured series, and the honest discussion of Dask's limitations in Section 9 is a positive feature.

major comments (3)
  1. [§8, Weak Scaling] The experiment labeled 'Weak Scaling' does not implement standard weak scaling, because the amount of work per worker is not held constant. With the row dimension held fixed, the number of rows per worker decreases as 1/W, while the channel and source dimensions grow as (3072+1024W) and 100W respectively; the per-worker number of coherencies is therefore proportional to (3072+1024W) × 100, which grows linearly with W. The sentence 'the problem size ... quadratically increases per worker' is incorrect: the total problem size grows quadratically, but the per-worker size grows linearly. Consequently, the increase in wall time from 936 s at W=10 to 2975 s at W=100 mostly reflects increased per-worker work, not a scaling property of the framework. The authors should either redesign the experiment to keep per-worker work constant (e.g., scale channels as 1024W with a constant row-per-worker distribution) or relabel the experiment and interpret it as a workload-scaling test rather than weak scaling.
  2. [§8, Strong Scaling and Table 1] The claim of 'linear speedup' is not supported by the data. From Table 1, increasing the instance count from 10 to 80 yields a speedup of 934/145 ≈ 6.4x on 8x the resources, and from 10 to 100 gives 934/134 ≈ 7.0x on 10x the resources. These are sub-linear, and the onset of diminishing returns is visible before 80 workers. The text should report the measured speedups and discuss the sub-linearity (e.g., the drop in tasks per thread from 65 to 8.1, scheduling and I/O overhead) rather than labeling the behavior 'linear speedup'. Additionally, the same section contains an internal inconsistency: the dataset is said to correspond to '34TB of predicted source coherencies' and, two paragraphs later, 'the size of the problem (78TB of source coherencies)'. A direct calculation using 5,954,759 rows, 4096 channels, 4 correlations, 100 sources, and 8 bytes per complex64 coherency gives ≈78 TB, so the 34 TB figure appears erroneous and must be corrected.
  3. [§8 and §9.2, Generalizability of the Scaling Claim] The central claim of the paper is that Dask-MS and Codex Africanus provide the building blocks for highly parallel, scalable, and distributed radio astronomy processing, but the only scaling evidence is for a compute-bound, embarrassingly parallel DFT predict implemented with Dask collections. Section 9.2 explicitly states that Dask collections 'provide no mechanism for regulating backpressure' and are 'ill-suited for more complex distributed processing applications', and that the flagship imager pfb-imaging uses the low-level Client interface and stateful Dask actors instead of collections. The presented scaling experiment therefore does not exercise the scheduling pathologies (backpressure, complex dependencies, stateful loops) that the authors themselves identify as the real difficulty for more complex pipelines. Since the scaling of QuartiCal and pfb-imaging is deferred to unpublished companion papers, the paper's broad title and abstract overstate what is currently demonstrated. The authors should either include a scaling result for at least one more communication-heavy or stateful algorithm, or explicitly qualify the title/abstract claims to state that scalable processing is demonstrated here for compute-bound algorithms while complex-application scaling is deferred.
minor comments (4)
  1. [§8, Cost estimate] The cost of the longest weak-scaling run is stated as $84, computed as 100 × $1.1016/hour × 2975 s/3600. This product evaluates to approximately $91; please verify the instance rate or the arithmetic.
  2. [§8, paragraph after Table 1] The phrase 'the size of the problem (78TB of source coherencies) is too small to efficiently solve with further parallelism' is misleading because 78 TB is not small. The intended point is that the number of tasks per thread (8.1 at 80 workers) becomes too small to amortize scheduling and transfer overhead; please rephrase accordingly.
  3. [Table 2 caption] The column header 'Average Tasks/threads' mixes singular and plural; it should be 'Average Tasks/thread' for consistency with Table 1.
  4. [§6, Listing 5] The pseudo-code uses `reduce` without importing or defining it, and the `find_row_id` lambda uses a conditional expression that may be unclear to readers. Adding one line explaining that `reduce` iteratively applies a function is advisable, or the code should be adjusted to a self-contained snippet.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper reports direct wall-clock measurements and software architecture, with no prediction reducing to a fitted input or to a self-citation chain.

full rationale

This is a software and measurement paper rather than a derivation. The central quantitative claims are wall-clock scaling measurements of a DFT predict on AWS EKS (Section 8, Tables 1 and 2, Figure 3), obtained by running the crystalball predict with Dask and Numba. These are direct observations; there is no fitted parameter, no calibrated constant, and no quantity defined in terms of the measurement. Strong and weak scaling are compared against Amdahl's and Gustafson's laws using independent formulas (Eqs. 1 and 2), and the measured speedups are read off timings rather than derived from the formulas. The paper's design claims are architectural: Dask-MS is presented as a Data Access Layer and codex africanus as an algorithm library, with flexibility and ease-of-development favored over performance. Self-citations to the authors' own packages, such as Tricolour, ShadeMS, RAGaVI, QuartiCal, pfb-imaging, and Stimela2, are descriptive pointers to companion papers and prior work, and are not used as evidence for the central scaling result. In particular, scaling of QuartiCal and pfb-imaging is explicitly deferred to Papers II and III; the paper does not claim to have demonstrated it here. Section 9.2 honestly limits the claim by stating that Dask collections provide no backpressure regulation and are 'ill-suited for more complex distributed processing applications', and that pfb-imaging uses the low-level Client interface and stateful actors. These limitations weaken the scope of the general scalability claim but do not make it circular. The reviewer should note the internal inconsistency in Section 8: the text says 100 point-source components correspond to 34TB of predicted source coherencies, while a later paragraph says 78TB, and the scaling tables lack error bars. These are evidence-quality issues, not circularity. No equation in the paper is equivalent to its own input by construction, and no fitted parameter is renamed as a prediction. The result is therefore assessed as having no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central scaling claims rest on four domain assumptions: Dask collections and scheduler are sufficient for radio astronomy workloads; Numba provides near-C performance; the MS v2.0 specification maps cleanly onto Zarr/Arrow; and the DFT-predict benchmark generalizes to the rest of the ecosystem. No free parameters or invented entities are present; the benchmark configuration (instance count, chunk size, source count) is an experimental setup, not a fitted parameter.

assumptions (4)
  • domain assumption Dask collections and the distributed scheduler provide sufficient performance and expressiveness for radio astronomy applications.
    Adopted in Section 3 as the basis for the framework; the scaling tests in Section 8 only cover a compute-bound DFT predict, not the full application suite.
  • domain assumption Numba JIT-compiled functions drop the GIL and achieve near-C/C++ performance.
    Invoked in Sections 3 and 5; the paper provides no direct comparisons of Numba kernels to C++ implementations within this work.
  • domain assumption The Measurement Set v2.0 specification can be mapped without loss onto Zarr and Arrow storage.
    Assumed throughout Section 4 to justify Dask-MS as a general data access layer, with caveats noted for variably-shaped columns and Arrow update support.
  • domain assumption The benchmark results for the DFT predict generalize to other algorithms in the ecosystem.
    Explicitly flagged by the authors as an open question for Papers II and III in Section 8.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Africanus I. Scalable, distributed and efficient radio data processing with Dask-MS and Codex Africanus." pith.science (2026). https://pith.science/paper/YRVNO7GX

@misc{pith2026241212052,
  author       = {Pith},
  title        = {Pith review of: Africanus I. Scalable, distributed and efficient radio data processing with Dask-MS and Codex Africanus},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YRVNO7GX}},
  note         = {Machine review of arXiv:2412.12052}
}
read the original abstract

New radio interferometers such as MeerKAT, SKA, ngVLA, and DSA-2000 drive advancements in software for two key reasons. First, handling the vast data from these instruments requires subdivision and multi-node processing. Second, their improved sensitivity, achieved through better engineering and larger data volumes, demands new techniques to fully exploit it. This creates a critical challenge in radio astronomy software: pipelines must be optimized to process data efficiently, but unforeseen artefacts from increased sensitivity require ongoing development of new techniques. This leads to a trade-off among (1) performance, (2) flexibility, and (3) ease-of-development. Rigid designs often miss the full scope of the problem, while temporary research code is unsuitable for production. This work introduces a framework for developing radio astronomy techniques while balancing the above trade-offs. It prioritizes flexibility and ease-of-development alongside acceptable performance by leveraging Open Source data formats and software. To manage growing data volumes, data is distributed across multiple processors and nodes for parallel processing, utilizing HPC and cloud infrastructure. We present two Python libraries, Dask-MS and Codex Africanus, which enable distributed, high-performance radio astronomy software with Dask. Dask is a lightweight parallelization and distribution framework that integrates with the PyData ecosystem, addressing the "Big Data" challenges of radio astronomy.

Figures

Figures reproduced from arXiv: 2412.12052 by the authors.

Figure 1
Figure 1. dask-ms serves as a Data Access Layer mediating between various data sources and applications. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Extract, transform, load (ETL) achieved using D [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Strong and weak scaling plots for Table 1 and 2. In the strong scaling case, the aggregate clock time is also plotted to show how it stays relatively constant, [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Typical Collection Task Flow stages. Pipelining is implicit in the definition of Dask collec￾tions: functions are mapped over chunks of a collection to pro￾duce chunks of a new collection ( [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Africanus IV. The Stimela2 framework: scalable and reproducible workflows, from local to cloud compute

    astro-ph.IM 2024-12 conditional novelty 6.0 of 10

    Stimela2 introduces a YAML-based workflow framework that combines readable linear recipes with containerization and Kubernetes or Slurm backends for reproducible, scalable radio astronomy data reduction.

  2. Africanus III. pfb-imaging -- a flexible radio interferometric imaging suite

    astro-ph.IM 2024-12 conditional novelty 6.0 of 10

    A flexible imaging library, pfb-imaging, implements a sparsity-based SARA algorithm with interpretable parameters and demonstrates it on terabyte-scale MeerKAT data.

  3. Africanus II. QuartiCal: calibrating radio interferometer data at scale using Numba and Dask

    astro-ph.IM 2024-12 conditional novelty 5.0 of 10

    QuartiCal, a new Python-based radio interferometry calibration package, supports arbitrary chains of gain terms and uses Dask to scale from a laptop to distributed cloud clusters, outperforming CubiCal in memory and speed.

Reference graph

Works this paper leans on

79 extracted references · 51 canonical work pages · cited by 3 Pith papers

  1. [1]

    , author Paul, K

    author Abernathey, R. , author Paul, K. , author Hamman, J. , author Rocklin, M. , author Lepore, C. , author Tippett, M. , author Henderson, N. , author Seager, R. , author May, R. , author Vento, D.D. , year 2017 . title Pangeo NSF Earthcube Proposal . https://figshare.com/articles/journal_contribution/Pangeo_NSF_Earthcube_Proposal/5361094, :10.6084/m9....

    work page arXiv 2017

  2. [2]

    , author Hamman , J

    author Abernathey , R.P. , author Hamman , J. , author Miles , A. , year 2018 . title Beyond netCDF: Cloud Native Climate Data with Zarr and XArray , in: booktitle AGU Fall Meeting Abstracts , pp. pages IN33A--06

    work page 2018

  3. [3]

    , year 2010

    author Alted, F. , year 2010 . title Why modern cpus are starving and what can be done about it . journal Computing in Science & Engineering volume 12 , pages 68--71 . :10.1109/MCSE.2010.51

    work page doi:10.1109/mcse.2010.51 2010

  4. [4]

    , year 1967

    author Amdahl, G.M. , year 1967 . title Validity of the single processor approach to achieving large scale computing capabilities , in: booktitle Proceedings of the April 18-20, 1967, Spring Joint Computer Conference , publisher Association for Computing Machinery , address New York, NY, USA . p. pages 483–485 . https://doi.org/10.1145/1465482.1465560, :1...

    arXiv 1967

  5. [5]

    title Datashader

    author Anaconda , year 2022 . title Datashader . howpublished https://datashader.org/ . note Online; accessed 14-11-2022

    work page 2022

  6. [6]

    , author Smirnov , O.M

    author Andati , L.A.L. , author Smirnov , O.M. , author Makhathini , S. , year 2022 . title RAGaVI: A Radio Astronomy Gains and Visibilities Inspector , in: editor Ruiz , J.E. , editor Pierfedereci , F. , editor Teuben , P. (Eds.), booktitle Astronomical Society of the Pacific Conference Series , p. pages 529

    work page 2022

  7. [7]

    , year 2019

    author Apache Software Foundation. , year 2019 . title Arrow: a cross-language development platform for in-memory data . https://arrow.apache.org

    work page 2019

  8. [8]

    , year 2021

    author Arras, Philipp , author Reinecke, Martin , author Westermann, Rüdiger , author Enßlin, Torsten A. , year 2021 . title Efficient wide-field radio interferometry response . journal Astronomy and Astrophysics volume 646 , pages A58 . :10.1051/0004-6361/202039723

    doi:10.1051/0004-6361/202039723 2021

Show all 79 references
  1. [9]

    , author Lim , P.L

    author Astropy Collaboration , author Price-Whelan , A.M. , author Lim , P.L. , author Earl , N. , author Starkman , N. , author Bradley , L. , author Shupe , D.L. , author Patil , A.A. , author Corrales , L. , author Brasseur , C.E. , author N \"o the , M. , author Donath , A...

    2022 arXiv

  2. [10]

    , author Perkins, S

    author Atemkeng, M. , author Perkins, S. , author Kenyon, J. , author Hugo, B. , author Smirnov, O.M. , year 2022 . title Xova: Baseline-dependent time and channel averaging for radio interferometry , in: editor Ruiz , J.E. , editor Pierfedereci , F. , editor Teuben , P. (Eds....

    2022

  3. [11]

    , author Smirnov , O.M

    author Atemkeng , M.T. , author Smirnov , O.M. , author Tasse , C. , author Foster , G. , author Jonas , J. , year 2016 . title Using baseline-dependent window functions for data compression and field-of-interest shaping in radio interferometry . journal Monthly Notices of the...

    2016 arXiv

  4. [12]

    , author Kenyon, J.S

    author Bester, H.L. , author Kenyon, J.S. , author Repetti, A. , author Perkins, S.J. , author Smirnov, O.M. , author Blecher, T. , author Mhiri, Y. , author Roth, J. , author Heywood, I. , author Wiaux, Y. , author Hugo, B.V. , year 2024 . title Africanus III. pfb-imaging -- ...

    2024 arXiv

  5. [13]

    , author Edelman, A

    author Bezanson, J. , author Edelman, A. , author Karpinski, S. , author Shah, V.B. , year 2017 . title Julia: A fresh approach to numerical computing . journal SIAM R eview volume 59 , pages 65--98 . :10.1137/141000671

    2017 doi

  6. [14]

    , author Frostig, R

    author Bradbury, J. , author Frostig, R. , author Hawkins, P. , author Johnson, M.J. , author Leary, C. , author Maclaurin, D. , author Necula, G. , author Paszke, A. , author Vander P las, J. , author Wanderman- M ilne, S. , author Zhang, Q. , year 2018 . title JAX : composab...

    2018

  7. [15]

    , author Jacobs, D

    author Byrne, R. , author Jacobs, D. , year 2021 . title Development of a high throughput cloud-based data pipeline for 21 cm cosmology . journal Astronomy and Computing volume 34 , pages 100447 . :https://doi.org/10.1016/j.ascom.2021.100447

    2021

  8. [16]

    z y, K.T

    author Chy \. z y, K.T. , author Jurusik, W. , author Piotrowska, J. , author Nikiel-Wroczy \' n ski, B. , author Heesen, V. , author Vacca, V. , author Nowak, N. , author Paladino, R. , author Surma, P. , author Sridhar, S.S. , author Heald, G. , author Beck, R. , author Conw...

    2018

  9. [17]

    , author Perley , R.A

    author Cornwell , T.J. , author Perley , R.A. , year 1992 . title Radio-interferometric imaging of very large fields. The problem of non-coplanar arrays. journal Astronomy and Astrophysics volume 261 , pages 353--364

    1992

  10. [18]

    title Dask: Library for dynamic task scheduling

    author Dask Development Team , year 2016 . title Dask: Library for dynamic task scheduling . https://dask.org

    2016

  11. [19]

    title Scheduling policies

    author Dask Development Team , year 2023 . title Scheduling policies . howpublished https://web.archive.org/web/20230928002741/https://distributed.dask.org/en/stable/scheduling-policies.html . note Accessed: 2023-09-28

    2023

  12. [20]

    title Dask layer annotations

    author Dask Development Team , year 2024 . title Dask layer annotations . howpublished https://github.com/dask/dask/pull/6767 . note Accessed: 2024-02-07

    2024

  13. [21]

    , year 1974

    author Dennis, J.B. , year 1974 . title First version of a data flow procedure language , in: booktitle Programming Symposium, Proceedings Colloque Sur La Programmation , publisher Springer-Verlag , address Berlin, Heidelberg . p. pages 362–376

    1974

  14. [22]

    , author Misunas, D.P

    author Dennis, J.B. , author Misunas, D.P. , year 1974 . title A preliminary architecture for a basic data-flow processor , in: booktitle Proceedings of the 2nd Annual Symposium on Computer Architecture , publisher Association for Computing Machinery , address New York, NY, US...

    1974

  15. [23]

    , author Chalmers , D

    author Di Francesco , J. , author Chalmers , D. , author Denman , N. , author Fissel , L. , author Friesen , R. , author Gaensler , B. , author Hlavacek-Larrondo , J. , author Kirk , H. , author Matthews , B. , author O'Dea , C. , author Robishaw , T. , author Rosolowsky , E. ...

    2019 arXiv

  16. [24]

    , year 2015

    author van Diepen, G. , year 2015 . title Casacore table data system and its use in the measurementset . journal Astronomy and Computing volume 2 . :10.1016/j.ascom.2015.06.002

    2015 doi

  17. [25]

    , year 2006

    author Diepen, G.V. , year 2006 . title NOTE 199 - Table Query Language . type Technical Report . AIPS++ Notes Series

    2006

  18. [26]

    , author Momjian , E

    author Dodson , R. , author Momjian , E. , author Pisano , D.J. , author Luber , N. , author Blue Bird , J. , author Rozgonyi , K. , author Smith , E.T. , author van Gorkom , J.H. , author Lucero , D. , author Hess , K.M. , author Yun , M. , author Rhee , J. , author van der H...

    2022 arXiv

  19. [27]

    , author Vinsen , K

    author Dodson , R. , author Vinsen , K. , author Wu , C. , author Popping , A. , author Meyer , M. , author Wicenec , A. , author Quinn , P. , author van Gorkom , J. , author Momjian , E. , year 2016 . title Imaging SKA-scale data in three different computing environments . jo...

    2016 arXiv

  20. [28]

    , author Bassett , B.A

    author Finlay , C. , author Bassett , B.A. , author Kunz , M. , author Oozeer , N. , year 2023 . title Trajectory-based RFI subtraction and calibration for radio interferometry . journal Monthly Notices of the Royal Astronomical Society volume 524 , pages 3231--3251 . :10.1093...

    2023 arXiv

  21. [29]

    , year 1990

    author Greisen , E.W. , year 1990 . title The Astronomical Image Processing System. , in: booktitle Acquisition, Processing and Archiving of Astronomical Images , pp. pages 125--142

    1990

  22. [30]

    , year 1990

    author Gustafson, J.L. , year 1990 . title Fixed time, tiered memory, and superlinear speedup , in: booktitle Proceedings of the Fifth Distributed Memory Computing Conference (DMCC5) , organization IEEE Press . pp. pages 1255--1260

    1990

  23. [31]

    , author Wise, M.W

    author van Haarlem, M.P. , author Wise, M.W. , author Gunst, A.W. , author Heald, G. , author McKean, J.P. , author Hessels, J.W.T. , author de Bruyn, A.G. , author Nijboer, R. , author Swinbank, J. , author Fallows, R. , author Brentjens, M. , author Nelles, A. , author Beck,...

    2013

  24. [32]

    , author Ravi , V

    author Hallinan , G. , author Ravi , V. , author Weinreb , S. , author Kocz , J. , author Huang , Y. , author Woody , D.P. , author Lamb , J. , author D'Addario , L. , author Catha , M. , author Law , C. , author Kulkarni , S.R. , author Phinney , E.S. , author Eastwood , M.W....

    2019 doi arXiv

  25. [33]

    , author Hoyer, S

    author Hamman, J. , author Hoyer, S. , year 2017 . title Xarray: N-d labeled arrays and datasets in python . journal Journal of Open Research Software

    2017

  26. [34]

    , author Millman, K.J

    author Harris, C.R. , author Millman, K.J. , author van der Walt, S.J. , author Gommers, R. , author Virtanen, P. , author Cournapeau, D. , author Wieser, E. , author Taylor, J. , author Berg, S. , author Smith, N.J. , author Kern, R. , author Picus, M. , author Hoyer, S. , au...

    2020

  27. [35]

    , author Bishop, P

    author Hewitt, C. , author Bishop, P. , author Steiger, R. , year 1973 . title A universal modular actor formalism for artificial intelligence , in: booktitle Proceedings of the 3rd International Joint Conference on Artificial Intelligence , publisher Morgan Kaufmann Publisher...

    1973

  28. [36]

    , author Bunton, J.D

    author Hotan, A.W. , author Bunton, J.D. , author Chippendale, A.P. , author Whiting, M. , author Tuthill, J. , author Moss, V.A. , author McConnell, D. , author Amy, S.W. , author Huynh, M.T. , author Allison, J.R. , author Anderson, C.S. , author Bannister, K.W. , author Bas...

    2021

  29. [37]

    , year 2024

    author Hugo, B. , year 2024 . title A Journey from Commissioning to Science with MeerKAT . Ph.D. thesis. Rhodes University

    2024

  30. [38]

    , author Perkins, S

    author Hugo, B.V. , author Perkins, S. , author Merry, B. , author Mauch, T. , author Smirnov, O.M. , year 2022 . title Tricolour: an optimized sumthreshold flagger for meerkat , in: editor Ruiz , J.E. , editor Pierfedereci , F. , editor Teuben , P. (Eds.), booktitle Astronomi...

    2022

  31. [39]

    , year 2007

    author Hunter, J.D. , year 2007 . title Matplotlib: A 2d graphics environment . journal Computing in Science & Engineering volume 9 , pages 90--95 . :10.1109/MCSE.2007.55

    2007 doi

  32. [40]

    , author Rhinelander, J

    author Jakob, W. , author Rhinelander, J. , author Moldovan, D. , year 2017 . title pybind11 -- Seamless operability between C++11 and Python . note Https://github.com/pybind/pybind11

    2017

  33. [41]

    , author MeerKAT Team , year 2016

    author Jonas , J. , author MeerKAT Team , year 2016 . title The MeerKAT Radio Telescope , in: booktitle MeerKAT Science: On the Pathway to the SKA , p. pages 1 . :10.22323/1.277.0001

    2016 doi

  34. [42]

    , author Andati , L.A.L

    author J \'o zsa , G.I.G. , author Andati , L.A.L. , author de Blok , W.J.G. , author Hugo , B.V. , author Kleiner , D. , author Kamphuis , P. , author Moln \'a r , D.C. , author Makhathini , S. , author Maccagni , F.M. , author Perkins , S.J. , author Ramaila , A. , author Ra...

    2022

  35. [43]

    , author Wieringa , M.H

    author Kemball , A.J. , author Wieringa , M.H. , year 2000 . title Measurement Set definition version 2.0 . howpublished NRAO report, January 21, 2000, 52 pages

    2000

  36. [44]

    , author Perkins, S.J

    author Kenyon, J.S. , author Perkins, S.J. , author Bester, H.L. , author Smirnov, O.M. , author Russeeawon, C. , author Hugo, B.V. , year 2024 . title Africanus II. QuartiCal: calibrating radio interferometer data at scale using Numba and Dask . journal Astronomy and Computin...

    2024 arXiv

  37. [45]

    , author Smirnov , O.M

    author Kenyon , J.S. , author Smirnov , O.M. , author Grobler , T.L. , author Perkins , S.J. , year 2018 . title CUBICAL - fast radio interferometric calibration suite exploiting complex optimization . journal Monthly Notices of the Royal Astronomical Society volume 478 , page...

    2018 arXiv

  38. [46]

    , author van Langevelde , H.J

    author Kettenis , M. , author van Langevelde , H.J. , author Reynolds , C. , author Cotton , B. , year 2006 . title ParselTongue: AIPS Talking Python , in: editor Gabriel , C. , editor Arviset , C. , editor Ponz , D. , editor Enrique , S. (Eds.), booktitle Astronomical Data An...

    2006

  39. [47]

    , author Caserta, J

    author Kimball, R. , author Caserta, J. , year 2004 . title The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning, Conforming, and Delivering Data . publisher Wiley , address Indianapolis, IN

    2004

  40. [48]

    , author Ahmad, I

    author Kwok, Y.K. , author Ahmad, I. , year 1999 . title Static scheduling algorithms for allocating directed task graphs to multiprocessors . journal ACM Comput. Surv. volume 31 , pages 406–471 . https://doi.org/10.1145/344588.344618, :10.1145/344588.344618

    1999

  41. [49]

    , author Pitrou, A

    author Lam, S.K. , author Pitrou, A. , author Seibert, S. , year 2015 . title Numba: A llvm-based python jit compiler , in: booktitle Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC , publisher Association for Computing Machinery , address New Yor...

    2015

  42. [50]

    , year 2018

    author Makhathini, S. , year 2018 . title Advanced radio interferometric simulation and data reduction techniques . Ph.D. thesis. Rhodes University. address Drosty Rd, Grahamstown, 6139, Eastern Cape, South Africa . note Available via http://hdl.handle.net/10962/57348

    2018

  43. [51]

    author W es M c K inney , year 2010 . title D ata S tructures for S tatistical C omputing in P ython , in: editor S t\'efan van der W alt , editor J arrod M illman (Eds.), booktitle P roceedings of the 9th P ython in S cience C onference , pp. pages 56 -- 61 . :10.25080/Majora...

    2010 doi

  44. [52]

    , author Waters , B

    author McMullin , J.P. , author Waters , B. , author Schiebel , D. , author Young , W. , author Golap , K. , year 2007 . title CASA Architecture and Applications , in: editor Shaw , R.A. , editor Hill , F. , editor Bell , D.J. (Eds.), booktitle Astronomical Data Analysis Softw...

    2007

  45. [53]

    title MPI : A Message-Passing Interface Standard Version 4.1

    author Message Passing Interface Forum , year 2023 . title MPI : A Message-Passing Interface Standard Version 4.1 . https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf

    2023

  46. [54]

    , year 2021

    author Molenaar, G.J. , year 2021 . title Design patterns and software techniques for large-scale, open and reproducible data reduction . Ph.D. thesis. Rhodes University

    2021

  47. [55]

    , author Smirnov , O.M

    author Noordam , J.E. , author Smirnov , O.M. , year 2012 . title MeqTrees: Software package for implementing Measurement Equations . http://arxiv.org/abs/1209.010 arXiv:1209.010

    2012

  48. [56]

    , year 2002

    author Norman, D.A. , year 2002 . title The Design of Everyday Things . publisher Basic Books, Inc. , address USA

    2002

  49. [57]

    , year 2010

    author Offringa , A.R. , year 2010 . title Aoflagger: Rfi software . howpublished Astrophysics Source Code Library, record ascl:1010.017 . http://arxiv.org/abs/1010.017 arXiv:1010.017

    2010

  50. [58]

    , year 2012

    author Offringa , A.R. , year 2012 . title The SumThreshold Method: Technical Details . type Technical Report . Rijksuniversiteit Groningen

    2012

  51. [59]

    , author Guy , L

    author O'Mullane , W. , author Guy , L. , author Dubois-Felsmann , G. , author Economou , F. , author AlSayyad , Y. , author Graham , M. , author Slater , C. , year 2023 . title Vera C. Rubin Observatory: Open Science to the core , in: booktitle American Astronomical Society M...

    2023

  52. [60]

    title Openxla: A unified compiler ecosystem for ml workloads

    author OpenXLA Contributors , year 2024 . title Openxla: A unified compiler ecosystem for ml workloads . https://openxla.org

    2024

  53. [61]

    , author Marais , P.C

    author Perkins , S.J. , author Marais , P.C. , author Zwart , J.T.L. , author Natarajan , I. , author Tasse , C. , author Smirnov , O. , year 2015 . title Montblanc ^ 1 : GPU accelerated radio interferometer measurement equations in support of Bayesian inference for radio obse...

    2015 arXiv

  54. [62]

    , author Murgia, M

    author Ramatsoku, M. , author Murgia, M. , author Vacca, V. , author Serra, P. , author Makhathini, S. , author Govoni, F. , author Smirnov, O. , author Andati, L. A. L. , author de Blok, E. , author J\'ozsa, G. I. G. , author Kamphuis, P. , author Kleiner, D. , author Maccagn...

    2020

  55. [63]

    , author Rodriguez Bezos, J.E

    author Rodrigues, J.E. , author Rodriguez Bezos, J.E. , year 1969 . title A GRAPH MODEL FOR PARALLEL COMPUTATIONS . type Technical Report . Massachusetts Institute of Technology. address USA

    1969

  56. [64]

    , author Sánchez-Expósito, S

    author Sabater, J. , author Sánchez-Expósito, S. , author Best, P. , author Garrido, J. , author Verdes-Montenegro, L. , author Lezzi, D. , year 2017 . title Calibration of lofar data on the cloud . journal Astronomy and Computing volume 19 , pages 75--89 . :https://doi.org/10...

    2017 doi

  57. [65]

    , author Dewdney, P.E

    author Schilizzi, R.T. , author Dewdney, P.E. , author Lazio, T.J.W. , year 2008 . title The square kilometre array , in: booktitle SPIE Astronomical Telescopes+ Instrumentation , organization International Society for Optics and Photonics . pp. pages 70121I--70121I

    2008

  58. [66]

    , author Merry , B

    author Schwardt , L. , author Merry , B. , author Perkins , S. , author Richter , L. , author Mauch , T. , author Ratcliffe , S. , year 2023 . title katdal: MeerKAT Data Access Library . howpublished Astrophysics Source Code Library, record ascl:2305.004

    2023

  59. [67]

    , author Athreya, R

    author Sekhar, S. , author Athreya, R. , year 2018 . title Two procedures to flag radio frequency interference in the uv plane . journal The Astronomical Journal volume 156 , pages 9 . https://dx.doi.org/10.3847/1538-3881/aac16e, :10.3847/1538-3881/aac16e

    2018 doi

  60. [68]

    , year 2011

    author Smirnov , O.M. , year 2011 . title Revisiting the radio interferometer measurement equation - I. A full-sky Jones formalism . journal Astronomy and Astrophysics volume 527 , pages A106 . :10.1051/0004-6361/201016082

    2011 doi

  61. [69]

    , author Heywood , I

    author Smirnov , O.M. , author Heywood , I. , author Perkins , S.J. , author van Rooyen , R. , year 2022 . title ShadeMS: Rapid Plotting of Big Radio Interferometry Data , in: editor Ruiz , J.E. , editor Pierfedereci , F. , editor Teuben , P. (Eds.), booktitle Astronomical Soc...

    2022

  62. [70]

    , author Makhathini, S

    author Smirnov, O.M. , author Makhathini, S. , author Kenyon, J.S. , author Bester, H.L. , author Perkins, S.J. , author Ramaila, A.J.T. , author Hugo, B.V. , year 2024 . title Africanus IV. The Stimela2 framework: scalable and reproducible workflows, from local to cloud compu...

    2024 arXiv

  63. [71]

    , author Hugo, B

    author Tasse, C. , author Hugo, B. , author Mirmont, M. , author Smirnov, O. , author Atemkeng, M. , author Bester, L. , author Hardcastle, M.J. , author Lakhoo, R. , author Perkins, S. , author Shimwell, T. , year 2018 . title Faceting for direction-dependent spectral deconvo...

    2018 doi

  64. [72]

    , year 2011

    author Tilmes, C. , year 2011 . title Data Formats: Using Self-Describing Data Formats . type Technical Report . NASA

    2011

  65. [73]

    , author Benn, D

    author Toomey, L. , author Benn, D. , author Chapman, J. , author Dai, S. , author Dempsey, J. , author Hobbs, G. , author Russell, C. , author Wang, C. , author Wang, J. , author Zic, J. , year 2017 . title Processing public pulsar astronomy data in the Amazon Cloud . type Te...

    2017

  66. [74]

    , year 1975

    author Ullman, J.D. , year 1975 . title Np-complete scheduling problems . journal J. Comput. Syst. Sci. volume 10 , pages 384–393 . https://doi.org/10.1016/S0022-0000(75)80008-0, :10.1016/S0022-0000(75)80008-0

    1975 doi

  67. [75]

    , et al., year 2020

    author Virtanen, P. , et al., year 2020 . title SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python . journal Nature Methods volume 17 , pages 261--272 . :10.1038/s41592-019-0686-2

    2020 doi

  68. [76]

    , author Greisen , E.W

    author Wells , D.C. , author Greisen , E.W. , author Harten , R.H. , year 1981 . title FITS - a Flexible Image Transport System . journal A&A volume 44 , pages 363

    1981

  69. [77]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  70. [78]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.stat...

  71. [79]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.stat...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.