{"id":"5805e85b-1fb6-4ae5-b5f5-eeaa2004d5ef","arxiv_id":"2507.18347","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"PyKOALA is a modular, instrument-agnostic Python framework for IFS data reduction, currently applied to KOALA+AAOmega, but lacking quantitative performance validation in this paper.","lead":"PyKOALA is a new Python library for reducing integral field spectroscopy data, aimed at being usable across multiple instruments. This paper describes its architecture and examples from the KOALA+AAOmega instrument, but offers no quantitative validation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The instrument-agnostic claim is not supported: the RSS/Cube abstraction is tested only on KOALA RSS files, with no demonstrated wrapper or end-to-end reduction for any second instrument.","rationale":"The reader's weakest_assumption precisely identifies the same soft spot: the RSS/Cube unified data model is asserted to generalize to any IFS instrument, but the paper offers no operational demonstration beyond KOALA. I read the full text in good faith: this is a proceedings paper describing architecture and initial testing, not a false claim about completed work. The authors explicitly state that PyKOALA 'started as a bespoke data reduction package for KOALA+AAOmega' and only 'has grown to become' instrument-agnostic; Section 3 also says the library will serve as KOALA's primary reduction tool and lists future extensions. The concrete evidence in the paper (Figure 1 and Figure 2) shows real functionality, but both examples use KOALA RSS data. The strongest_claim is therefore the one that is least supported. My recommended verdict remains CONDITIONAL, unchanged from the reader: the paper deserves publication as a software/architecture description, but only with the requested reproducibility and cross-instrument validation evidence. I do not recommend REJECT because the library is open-source (the readthedocs link is provided), the architecture is coherent, and the authors are honest about the project being under development and about the stable release being planned for Q1 2025. The condition should be: add a link to the code repository with a tagged version, provide a minimal working example with public test data, and either provide one non-KOALA wrapper or explicitly narrow the instrument-agnostic claim. This is a modest, actionable bar rather than a demand for new science.","tokens_in":3281,"tokens_out":1887,"duration_ms":17577,"concrete_test":"Ask the authors to provide, in the paper or a linked repository, one complete wrapper for a non-KOALA IFS instrument (e.g., Hector or WEAVE) plus the commands, configuration, and validation plots needed to reproduce one end-to-end reduction: raw input to wavelength-calibrated, sky-subtracted, flux-calibrated RSS or datacube. If a working wrapper cannot be provided because the relevant instrument data are proprietary, an acceptable substitute is a public synthetic IFS dataset with a defined instrument model and a wrapper that ingests it, plus a quantitative comparison (e.g., residual sky line flux or absolute flux accuracy) against the same steps applied to KOALA data. If the wrapper and end-to-end reduction cannot be produced, the central claim should be narrowed to 'a framework designed for instrument-agnostic use, demonstrated on KOALA'.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in Section 3 and echoed in Section 1, is that PyKOALA is an 'instrument-agnostic framework' for IFS data reduction. The load-bearing assumption, stated in Section 2, is that any IFS instrument can be integrated by writing a wrapper that produces an RSS or datacube with fibre spectra, wavelength, and variance arrays. That assumption is plausible but untested here. The only concrete demonstrations in the paper are KOALA+AAOmega RSS data: the twilight wavelength/LSF example (Figure 1) and the datacube interpolation example (Figure 2). The text says the algorithm has been 'tested with other instruments (Hector and WEAVE)', but no wrapper code, no validation figures, and no quantitative accuracy results for those instruments appear. More importantly, the 'tested with' phrase is ambiguous: it could mean the interpolation or correction algorithms ran on data, but there is no evidence that a complete pipeline (sky subtraction, telluric correction, astrometry, flux calibration, cube construction) was executed end-to-end for any non-KOALA instrument. The paper also delays the stable release to Q1 2025, so the architecture described may still be under development. Without evidence of a working non-KOALA wrapper or a description of how instrument-specific metadata (fibre positions, bad fibres, spectral format, LSF variation) is mapped into the unified model, the 'comprehensive, instrument-agnostic solution' claim is an extrapolation from a single-instrument success. This is not an internal inconsistency; it is an unsupported generalization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes PyKOALA, a Python library for integral field spectroscopy (IFS) data reduction, originally developed for the KOALA+AAOmega instrument at the AAT and positioned as an instrument-agnostic, modular framework. It outlines the library's data model (RSS and datacube containers), its Correction classes for atmospheric extinction, sky emission, telluric absorption, astrometry, wavelength offsets, and flux calibration, and its cube-interpolation capabilities. The paper presents two illustrative figures (wavelength/LSF correction in a twilight exposure and datacube interpolation examples) and concludes with claims of sub-pixel wavelength calibration, effective sky subtraction, accurate flux calibration, and a planned stable release in Q1 2025.","tokens_in":3653,"tokens_out":2336,"duration_ms":25125,"significance":"If the instrument-agnostic and high-accuracy claims were substantiated, PyKOALA would fill a genuine gap: a unified, open-source Python IFS reduction framework usable across instruments with modest wrapper effort. The proposed DataContainer abstraction and modular Correction classes are sensible design choices that could promote interoperability and reproducibility in a field dominated by bespoke pipelines. However, the paper's evidence for these claims is limited to qualitative illustrations for a single instrument. The mention of testing with Hector and WEAVE is promising but unquantified. As a proceedings contribution, the paper is an acceptable high-level description, but the strength of the currently worded conclusions goes beyond what is demonstrated.","major_comments":[{"comment":"The sentence 'Initial testing of PyKOALA has demonstrated its capability to achieve precise corrections, including sub-pixel wavelength calibration, effective sky subtraction, and accurate flux calibration' makes quantitative performance claims without presenting any quantitative measure, error bar, or comparison against an existing pipeline. For example, Figure 1 shows fibre-to-fibre wavelength offsets and LSF widths, but the text does not state the achieved precision (e.g., in km/s or pixels) nor how it was validated. Likewise, 'effective sky subtraction' and 'accurate flux calibration' are not accompanied by any residual statistics or comparison with 2dfdr or other reductions. These claims are load-bearing for the paper's central assertion that PyKOALA 'advances the field'; they need either concrete validation numbers, a referenced validation study, or explicit softening of the wording.","section":"Section 3, Conclusions"},{"comment":"The 'comprehensive, instrument-agnostic solution' claim rests on the assumption that any IFS instrument can be integrated via a wrapper that supplies fibre spectra, wavelength, and variance arrays into an RSS/Cube object. The paper states that the implementation 'has been tested with other instruments (Hector and WEAVE)' but provides no details: no wrapper code, no validation figures, no description of how instrument-specific metadata (fibre positions, bad-fibre masks, spectral format, LSF variation) is mapped into the unified model, and no end-to-end reduction demonstration for any non-KOALA instrument. Without such evidence, the instrument-agnostic claim is an extrapolation from a single-instrument deployment. The authors should either provide at least one worked non-KOALA example (even a partial one) or explicitly restrict the claim to 'designed for' rather than 'demonstrated with'.","section":"Section 2, Data model and instrument adaptability"},{"comment":"Figure 2 shows three datacube interpolations (Gaussian, Parabolic, Drizzling) that yield 'slightly distinct cube reconstructions', but no quantitative assessment is given of the relative accuracy or artifacts of each method. Since the choice of interpolation kernel affects scientifically measured quantities (e.g., emission-line fluxes, spatial resolution), the paper should at least report a quantitative comparison on a test case, for example the recovered flux of a point source or the width of a reconstructed unresolved source, or state that such an evaluation is planned. This is necessary to support the claim that the interpolation techniques provide 'flexibility for various scientific applications, ensuring that the resulting data products meet diverse research goals'.","section":"Section 2, Figure 2 and cube interpolation"}],"minor_comments":[{"comment":"The text says 'We plan to release a stable version of PyKOALA in Q1 2025', but the manuscript is dated July 2025 (arXiv:2507.18347). The paper should either report on the status of the release (including a version number or DOI) or be clear about the submission date of the proceedings. As written, this statement is confusing for readers checking the current status.","section":"Section 3, last paragraph"},{"comment":"The link to the PyKOALA documentation is helpful, but the paper should also provide the software repository URL (e.g., GitHub) and a license statement, since the paper highlights the project as an 'open collaboration'.","section":"Footnote 1"},{"comment":"Figure 1 shows wavelength offsets and LSF standard deviations for two approaches, but the axes lack explicit units or scales in the printed caption; please add units (e.g., km/s or Å for the offset, Å or pixels for the LSF width) to the figure or caption.","section":"Figure 1"},{"comment":"The phrase 'sets a significant barrier' is slightly awkward; consider 'poses a significant barrier'. Also, in the abstract and introduction, 'via creating complete three-dimensional view' should be 'via creating a complete three-dimensional view'.","section":"Section 2, first paragraph"},{"comment":"The paper cites the 2dfdr software as 'AAO software team 2015', but the ASCL record is adequately referenced; however, the reference list lacks a DOI for the PyKOALA software itself, which would improve reproducibility.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a short proceedings paper, and the bar for 'accept' in such venues is often lower, but the manuscript makes sweeping claims (sub-pixel calibration, effective sky subtraction, accurate flux calibration, instrument-agnostic framework) that go well beyond the evidence shown. The authors can fix this in revision either by adding a small quantitative validation section (for KOALA and ideally one more instrument) or by carefully qualifying every strong claim as 'designed to achieve' rather than 'demonstrated'. The timeline inconsistency about the Q1 2025 release should also be resolved, as it may indicate the paper was written before the actual release but submitted later. I think major revision is proportionate: the central idea is valuable and likely correct, but the current text overstates what is shown."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PyKOALA is a real, usable-looking Python package for KOALA+AAOmega IFU data reduction, and the paper gives a clear account of its architecture. The modular design — DataContainer, RSS/Cube representations, pluggable Correction classes — is genuinely flexible and should let KOALA users build custom pipelines around 2dfdr products. That is the paper's actual contribution: a well-structured reduction framework, presented honestly as work in progress.\n\nThe soft spot is the breadth of the claim. Calling PyKOALA 'instrument-agnostic' and a 'comprehensive solution for IFS data reduction' goes well beyond what is demonstrated. All concrete examples use KOALA RSS data. Hector and WEAVE are mentioned as 'tested', but no wrapper, no validation, and no end-to-end reduction for those instruments appear in the paper. The abstraction — extract fibre spectra, wavelength, and variance; build an RSS — is plausible, but plausibility is not a demonstration. Section 3 claims sub-pixel wavelength calibration, effective sky subtraction, and accurate flux calibration with zero supporting numbers, no comparison to 2dfdr or other pipelines, and no error analysis. The figures are illustrative, not tests.\n\nI do not think this is a dishonest or confused paper; it reads like an honest proceedings write-up of a library still under development. The stable release is planned for Q1 2025, and the paper includes no repository link, version, or software artifact. That makes independent evaluation impossible right now. If the authors had linked a release and shown even one non-KOALA wrapper working end-to-end, the central claims would mostly hold. As it stands, the conditional verdict is right.\n\nWho gets value: anyone working with KOALA data, and IFS pipeline developers wanting a clean modular reference. For a rigorous journal, the missing validation is disqualifying unless added. For an ADASS-style proceedings, it is borderline acceptable but should be tightened.\n\nMy recommendation: send to review, but with the clear expectation that the authors either add a code repository and validation section, or soften the claims to match the evidence. I would not desk-reject — the design is sensible and the tool is likely to be used — but I would not accept it as-is.","headline":"A genuine and well-structured KOALA reduction library, but the instrument-agnostic claim rests on a single demonstrated instrument and the paper ships no code or validation numbers.","tokens_in":4164,"tokens_out":2368,"would_cite":false,"duration_ms":26182,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PyKOALA is a Python library that aims to make integral-field spectroscopy data reduction a single, instrument-agnostic workflow.","keywords":["integral field spectroscopy","data reduction","PyKOALA","Row-Stacked Spectra","datacubes","wavelength calibration","sky subtraction","instrument-agnostic pipeline"],"falsifier":"Take raw data from an IFS instrument whose design differs strongly from fibre bundles, such as an image-slicer IFU, reduce it using only PyKOALA's public API, and measure the residuals of wavelength calibration, sky subtraction, and flux calibration against a trusted reference; if any residual exceeds the sub-pixel or few-percent tolerances the paper claims, the instrument-agnostic assertion fails.","tokens_in":3098,"feed_emoji":"🔭","tokens_out":5814,"duration_ms":62383,"temperature":0.7,"pith_summary":"PyKOALA is a Python package that aims to turn integral-field spectroscopy (IFS) data reduction—normally a bespoke, instrument-by-instrument chore—into a single modular workflow. The paper argues that by representing every instrument's output as either Row-Stacked Spectra (fibre spectra plus wavelength and variance arrays) or 3D datacubes, and by implementing corrections as interchangeable classes, the same pipeline can reduce data from KOALA+AAOmega, Hector, WEAVE, and, in principle, any IFS instrument. If this holds, astronomers gain a user-friendly, instrument-agnostic reduction tool with sub-pixel wavelength calibration, effective sky subtraction, and accurate flux calibration, instead of maintaining separate pipelines per instrument. The paper reports initial tests on KOALA, Hector, and WEAVE and plans a stable release in Q1 2025.","feed_headline":"PyKOALA: one pipeline to reduce any instrument's integral-field data","feed_subtitle":"One modular reduction workflow could replace per-instrument pipelines for any IFS instrument.","key_machinery":"The load-bearing object is the DataContainer, with two concrete forms: Row-Stacked Spectra (RSS), representing the extracted spectrum of every fibre together with wavelength and variance arrays, and Cube, a 3D grid sampled along one spectral and two spatial dimensions. Every reduction step is implemented as a Correction class that operates on these containers, so once a wrapper converts an instrument's raw files into RSS, the same correction machinery applies. The wavelength-offset and line-spread-function correction works by cross-correlating fibre spectra against a reference solar spectrum, which is the step the paper identifies as critical for reaching sub-pixel calibration accuracy.","core_discovery":"On its own terms, PyKOALA is a comprehensive, instrument-agnostic solution for IFS data reduction. Built originally for the KOALA+AAOmega instrument, it reads the wavelength-calibrated Row-Stacked Spectra produced by the 2dfdr pipeline and then applies a sequence of specialized Correction classes for atmospheric extinction, sky emission, telluric absorption, astrometry offsets, wavelength offsets, and flux calibration. The paper's central evidence is that this modular architecture—combined with a DataContainer abstraction for RSS and datacubes, and user-supplied wrappers for other instruments—lets the same core code handle data from KOALA, Hector, and WEAVE. It reports sub-pixel wavelength calibration and line-spread-function estimation via cross-correlation with a reference solar spectrum, and it supports multiple interpolation schemes, including drizzling, for building 3D datacubes from individual exposures.","pith_inferences":["The instrument-agnostic claim would be put to its sharpest test on an image-slicer or multi-channel IFU, since PyKOALA's data model is demonstrated on fibre-fed instruments; a wrapper would need to serialize sliced data into Row-Stacked Spectra without losing spatial information.","Because the correction classes are modular, the same engine could in principle be reused for fibre-fed spectrographs or time-series spectroscopy, though the paper does not make that claim.","A direct head-to-head comparison against a non-KOALA instrument's official pipeline would turn 'tested with' into a quantitative statement; the paper currently reports no such validation."],"forward_implications":["KOALA+AAOmega observations will be reduced with PyKOALA as the primary tool alongside 2dfdr, making it the default path from raw exposures to science-ready data for that instrument.","Users can assemble custom reduction sequences from Correction classes, letting different science cases choose their own treatment of extinction, sky, telluric, astrometry, wavelength, and flux steps.","For datacube production, users can select inverse-distance weighting or drizzling, which changes spatial resolution and noise properties, giving observers control over the trade-off.","New instruments can be added by writing a data wrapper rather than a full reduction pipeline, which is the core of the instrument-agnostic claim.","A stable release with tutorials is planned for Q1 2025, making the library accessible beyond the KOALA group."],"supporting_citations":[{"why":"Provides the 2dfdr RSS output format that PyKOALA's KOALA readers consume, defining the native data input for the library.","marker":"AAO software team 2015"},{"why":"Describes the KOALA IFU, whose multiple gratings, central wavelengths, and field-of-view configurations motivate the reduction pipeline.","marker":"Ellis et al. 2012"},{"why":"Describes the AAOmega double-beam spectrograph, setting the instrument characteristics that PyKOALA's corrections must handle.","marker":"Smith et al. 2004"},{"why":"Documents a modern IFS reduction pipeline, used as context that IFS data reduction requires sophisticated, multi-step processing.","marker":"Weilbacher et al. 2020"},{"why":"Supplies the drizzling algorithm that PyKOALA offers as one of its datacube interpolation options.","marker":"Fruchter & Hook 2002"}],"fun_headline_variants":["PyKOALA: a single Python library for all IFS reduction","PyKOALA: one workflow for any integral-field spectrograph","PyKOALA: modular Python for instrument-agnostic IFS reduction","PyKOALA: one Python workflow reduces any IFS instrument's data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"A data model built on Row-Stacked Spectra and datacubes, plus user-supplied wrappers, can represent and correctly reduce the data of every IFS instrument.","fun_headline_variants_meta":{"raw":{"variants":["PyKOALA: a single Python library for all IFS reduction","PyKOALA: one workflow for any integral-field spectrograph","PyKOALA: modular Python for instrument-agnostic IFS reduction","PyKOALA: one Python workflow reduces any IFS instrument's data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000599,"raw_usage":{"total_tokens":2722,"prompt_tokens":792,"completion_tokens":1930,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":408,"completion_tokens_details":{"reasoning_tokens":1853}},"tokens_in":408,"tokens_out":1930,"duration_ms":15784,"temperature":1.0,"reasoning_tokens":1853,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:31:21.491954+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take raw data from an IFS instrument whose design differs strongly from fibre bundles, such as an image-slicer IFU, reduce it using only PyKOALA's public API, and measure the residuals of wavelength calibration, sky subtraction, and flux calibration against a trusted reference; if any residual exceeds the sub-pixel or few-percent tolerances the paper claims, the instrument-agnostic assertion fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the 2dfdr RSS output format that PyKOALA's KOALA readers consume, defining the native data input for the library."},{"cited_title":"C., et al","cited_arxiv_id":null,"evidence_quote":"Describes the KOALA IFU, whose multiple gratings, central wavelengths, and field-of-view configurations motivate the reduction pipeline."},{"cited_title":"A., et al","cited_arxiv_id":null,"evidence_quote":"Describes the AAOmega double-beam spectrograph, setting the instrument characteristics that PyKOALA's corrections must handle."}],"review_version":1}