Pith. sign in

REVIEW 3 major objections 5 minor 3 references

Data Quality Monitoring system in the Baikal-GVD experiment

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By fitting event and charge distributions, the Baikal-GVD DQM system assigns a quality code to every optical module and rolls it up to whole-cluster level.

desk verdict A solid, honest description of Baikal-GVD's monitoring system, but the quality-ranking thresholds are unvalidated and the 'very efficiently' conclusion overreaches. read the letter →

arxiv 1908.07270 v1 pith:ES3EJ2AY submitted 2019-08-20 astro-ph.IM astro-ph.HE

Baikal GVD Collaboratio: A.D. Avrorin , A.V. Avrorin , V.M. Aynutdinov , R. Bannash , I.A Belolaptikov , V.B. Brudanin , N.M. Budnev , G.V. Domogatsky
show 44 more authors
A.A. Doroshenko R. Dvornicky A.N. Dyachok Zh.-A.M. Dzhilkibaev L. Fajth S.V Fialkovsky A.R. Gafarov K.V. Golubkov N.S. Gorshkov T.I. Gress R. Ivanov K.G. Kebkal O.G. Kebkal E.V. Khramov M.M. Kolbin K.V. Konischev A.V. Korobchenko A.P. Koshechkin A.V. Kozhin M.V. Kruglov M.K. Kryukov V.F. Kulepov M.B. Milenin R.A. Mirgazov V. Nazari A.I. Panfilov D.P. Petukhov E.N. Pliskovsky M.I. Rozanov E.V. Rjabov V.D. Rushay G.B. Safronov B.A. Shaybonov M.D. Shelepov F. Simkovic A.V. Skurikhin A.G. Solovjev M.N. Sorokovikov I. Stekl O.V. Suvorova E.O. Sushenok V.A. Tabolenko B.A. Tarashansky S.A. Yakovlev
This is my paper · ORCID
classification astro-ph.IMastro-ph.HE PACS 95.55.Vj
keywords Baikal-GVDdataqualitymonitoringneutrinotelescoperun-by-runchargedistributionsopticalmodulecodesunderwaterCherenkovdetector
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents the Data Quality Monitoring system built for the Baikal-GVD neutrino telescope, and claims that it can estimate the quality of experimental data run-by-run, from a single optical module up to the entire cluster. The system fits three statistical distributions, exponential, uniformity, and Poisson, to event timing and rates, and analyzes PMT charge distributions and Gantt plots. Each channel is then assigned one of eight quality codes, and the decision chain aggregates these codes upward through sections, strings, and clusters. The authors show examples from the 2016 season, concluding that the parameters allow efficient, automatic quality estimation.

What carries the argument

The load-bearing mechanism is a threshold-based decision chain over eight quality codes. The fit quality is judged by $\chi^2/\mathrm{NDF}$ with good/normal/bad boundaries at 2 and 4; per-bin deviations must satisfy $R_1 = 25\%$ or $50\%$ with $R_2 = 1\%$ or $5\%$; noise-rate deviations per depth level must be within $3\sigma$ or $5\sigma$; and the cumulative integral in the charge distribution must differ from the basic run by less than $15\%$ or $25\%$. These criteria are applied at channel, section, string, and cluster levels, so a single scalar code propagates upward.

What would settle it

Run the algorithm on a set of 2016 runs that include known LED calibration periods and channels independently judged by a human expert, then check the agreement between the eight quality codes and the known status; if a known-broken channel is marked "good" or a known-good channel is marked "bad" often enough, the threshold choices are not separating the two populations.

Watch

Extended reading notes

Core claim

The central claim is that a compact set of automated fits and comparisons can reliably characterize detector health at every level of Baikal-GVD without manual inspection. For each run, the system checks whether time differences between events follow an exponential, whether event rate is uniform and Poisson-distributed, where the 1 p.e. charge peak sits, whether trigger thresholds are stable relative to the calibration, and whether charge-distribution shapes match a known-good "basic run". Quality is summarized in eight codes, from "excluded by configuration" and "empty data" through "good", "normal", and "bad", each optionally flagged as containing LED-calibration light. The paper's conclusion is that the 2016 examples show this multi-parameter analysis estimates data quality very efficiently from the optical-module level to the whole cluster.

Load-bearing premise

The hand-picked threshold values in Section 5 are assumed to separate genuinely bad detector states from normal seasonal or environmental variation, but the paper does not validate them against a labeled set of broken versus healthy channels.

Editorial extensions

If this is right

  • Detector shifters can see a per-run quality map of every optical module without manually opening event displays.
  • Analysis pipelines can automatically reject runs or channels whose quality code is "bad" or "LED detected".
  • The same run-by-run summary could feed the global multi-messenger alert system, since detector state is known for every time window.
  • Early detection of failing PMTs or calibration anomalies becomes possible within one run rather than after offline reprocessing.
  • Quality codes give an audit trail for the 2016 dataset and later seasons.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed thresholds would likely benefit from calibration against a labeled set of known-broken channels; without that, codes cannot guarantee the same false-positive rate across seasons.
  • The "basic run" comparison may drift as Lake Baikal's water transparency changes seasonally, so the system probably needs periodic re-baselining.
  • A similar channel-to-cluster quality ladder could be adapted to other large underwater or ice neutrino detectors with minimal changes.
  • The method could be tested live by running it simultaneously with human expert inspection during a future season and measuring agreement.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper describes a data quality monitoring (DQM) system for the Baikal-GVD neutrino telescope. The system operates on a run-by-run basis and monitors three statistical parameters (exponential time-difference distribution, uniformity of event rate over time, and Poissonian rate distribution), charge distributions of non-trigger and trigger signals, noise-rate stability, and shape comparison of charge distributions against a reference 'basic run'. Gantt plots are used to identify periods with the LED calibration system active. A quality estimation algorithm combines fit quality (χ²/NDF), bin deviations, noise-rate deviations, and integral-impact deviations into eight quality codes (0–7) at the channel, section, string, and cluster levels. The paper demonstrates the system on examples from the 2016 season, mainly showing that LED calibration runs are flagged as anomalous, and concludes that the system can estimate data quality from the optical-module level up to the whole cluster 'very efficiently'.

Significance. The manuscript addresses a practical and important problem: automated, run-by-run quality assessment for a large underwater neutrino telescope, with output usable by shifters and multi-messenger programs. The strength of the paper is that it presents a concrete, implementable algorithm: the monitored parameters are clearly defined, the expected functional forms are stated, and the quality codes are enumerated. The examples with LED calibration runs provide a useful sanity check that the system can detect an obvious external disturbance. However, the quantitative claim of 'very efficient' quality estimation is not supported by the presented evidence: the threshold values are not validated against labeled data, the aggregation rules for section/string/cluster codes are not specified, and no false-positive/false-negative rates or timing performance are given. Thus the paper is a useful system description, but the performance claim needs either additional validation or a substantial caveat.

major comments (3)
  1. [Section 5, Quality estimation algorithm] The classification into codes 2-7 is fully determined by the hand-picked thresholds (χ²/NDF values 2 and 4; R1=25%/50% and R2=1%/5%; noise-rate deviations of 3σ and 5σ; integral-impact limits of 15% and 25%). The manuscript demonstrates that these parameters respond to LED calibration runs, but it never validates the threshold values against a ground-truth set of known-bad channels or runs, nor does it provide false-positive or false-negative rates. Because these cut values determine every quality code, the conclusion in Section 6 that the system 'allows to estimate the quality ... very efficiently' is not quantitatively supported. The authors should either perform a validation against labeled data or explicitly state that the thresholds are provisional and soften the efficiency claim.
  2. [Section 5, decision chain] The decision chain is stated as 'channel → section → string → cluster levels', but the described criteria apply only to 'channel, section and cluster' and note 'Charge (channel level only)'. The aggregation rule that converts per-channel quality codes into section-, string-, and cluster-level codes is not specified, and the derivation of the LED-related codes 5-7 from the Gantt-plot masks is not described. Without this information, the central claim that the system estimates quality 'from the level of the optical module and up to the whole cluster' cannot be reproduced or tested. Please provide the exact aggregation procedure and the explicit relation between the LED masks and the reported codes.
  3. [Section 3.3] The shape-comparison parameter is defined relative to a 'basic run that is well known to have good channel performance', but the paper does not specify how the basic run is selected, whether it is the same run for all comparisons, or how the 15% and 25% integral-impact thresholds were tuned against this reference. Since this parameter enters directly into the channel quality code, the missing definition of the reference run is a gap in the reproducibility of the algorithm.
minor comments (5)
  1. [Section 5, item 2] The quantities NBins, NBinsNormal, NBinsBad, and NBinsTotal used in the bin-deviation condition are never defined in the text, and the roles of R1 and R2 are not explained; please define them explicitly.
  2. [Section 5, item 4] The threshold conditions leave the boundary values undefined: for χ²/NDF the good/normal boundary is '<4' and the normal/bad boundary is '>4', leaving exactly 4 unassigned; similarly for exactly 5σ noise deviation and exactly 25% integral impact. Please state the convention for ties.
  3. [Section 4] The sentence 'Thus in standard conditions we expect maximum a doubling of the rate during the run' does not follow directly from the preceding statement about the 0.5 entries per bin threshold in the Gantt plot; please clarify the quantitative relationship.
  4. [Section 3.3] The criterion 'Number of events with charge value deposited in channel more than 100 p.e. should not exceed 100 events' is not referenced in the quality estimation algorithm; clarify whether it contributes to codes 5-7 or is a separate masking step.
  5. [Section 6 / Abstract] The abstract and Section 1 list participation in the global multi-messaging system as a design goal, but no latency, interface, or data-volume figures are reported; if this is outside the scope of the paper, this should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DQM quality codes are defined by fixed thresholds and demonstrated on examples, not predicted from fitted inputs.

full rationale

The paper is a detector-monitoring system description rather than a derivation. It defines eight quality codes and a decision chain using fixed thresholds on chi-squared/NDF, per-bin deviations, noise-rate sigma levels, and cumulative-integral deviations (Section 5), then demonstrates the resulting classifications on example runs from the 2016 season (Sections 2-4 and 6). No parameter is fitted to a subset of data and then used to predict a closely related quantity from that same subset; the thresholds are presented as chosen criteria, not as outputs of a fitted model, so there is no self-definitional or fitted-input-called-prediction step. The self-citations [1]-[3] supply detector, data-acquisition, and calibration background, including the expected depth dependence of noise, but none is load-bearing for the quality-code outputs. The skeptical concern that the Section 5 thresholds are not validated against a labeled set of broken versus healthy channels is a correctness or external-validation issue, not circularity: it does not make the claimed demonstration equivalent to its input. No circular step can be exhibited from the paper's own equations or citations, so the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The quality-ranking thresholds are hand-picked, and the system assumes that deviations from expected statistical shapes are caused by detector problems rather than benign environmental variability. No ground-truth validation is provided, so the reliability of the quality codes depends on unverified assumptions.

free parameters (4)
  • χ²/NDF thresholds = 2 (good), 4 (normal)
    Hand-chosen cut values for fit quality in the decision chain (Section 5).
  • Bin deviation thresholds R1 and R2 = R1=25%/50%, R2=1%/5%
    Thresholds for acceptable fraction of deviating bins in the decision chain (Section 5).
  • Noise rate deviation thresholds = 3σ (good), 5σ (normal)
    Channel quality is judged by deviation of noise rate from the average over the depth level (Section 3.1).
  • Integral impact thresholds = 15% (good), 25% (normal)
    Thresholds for relative impact of the cumulative charge distribution compared to the basic run (Section 5).
assumptions (4)
  • domain assumption Time differences between neighbor events follow an exponential distribution under normal conditions
    Used to define the exponential fit in Section 2.1; deviations are interpreted as detector or calibration problems.
  • domain assumption The event rate is linear with possible slope and event counts follow a Poisson distribution
    Sections 2.2 and 2.3 assume these shapes for atmospheric muons and random noise; environmental or detector effects only add a small slope.
  • domain assumption Non-trigger charge distributions are described by two Gaussians plus an exponential
    Section 3.1 uses this model to locate the 1 p.e. peak and to extract thresholds from the charge distribution.
  • ad hoc to paper The selected 'basic run' has good channel performance and is a valid reference for shape comparison
    Section 3.3 compares every run to a 'basic run' that is 'well known to have good channel performance', but no objective criterion for selecting this reference is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Data Quality Monitoring system in the Baikal-GVD experiment." pith.science (2026). https://pith.science/paper/ES3EJ2AY

@misc{pith2026190807270,
  author       = {Pith},
  title        = {Pith review of: Data Quality Monitoring system in the Baikal-GVD experiment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ES3EJ2AY}},
  note         = {Machine review of arXiv:1908.07270}
}
read the original abstract

The quality of the incoming experimental data has a significant importance for both analysis and running the experiment. The main point of the Baikal-GVD DQM system is to monitor the status of the detector and obtained data on the run-by-run based analysis. It should be fast enough to be able to provide analysis results to detector shifter and for participation in the global multi-messaging system.

Figures

Figures reproduced from arXiv: 1908.07270 by the authors.

Figure 1
Figure 1. Exponential distributions for atmospheric muons and random noise for some selected channels. Right: for calibration run with LED matrix. The expected cluster rate is ∼50-200 events/sec. As well as for the exponential distribution the bin width for the corresponding uniformity distribution is chosen to suppress possible bin-to-bin statistical fluctuations (see [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Uniformity distributions for atmospheric muons and random noise for some selected channels. Right: the same for calibration run with LED matrix. 2.3 Poisson distribution The expected value of the event rate for the given cluster size (∼50-200 events/sec per cluster) should follow Poissonian distribution for any fixed time interval. Thus, for each run the time interval is chosen to have ∼20 recorded events on average… view at source ↗
Figure 3
Figure 3. Rate distributions for atmospheric muons and random noise for some selected channels fitted with a Poisson function. Right: the same for calibration run with LED matrix. tions of the charge from the PMT. Therefore the data quality is estimated for given channel only and considered in summarizing algorithm. The low threshold for the charge is the so-called filter threshold that is about half of the single photoelectr… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Left: the charge distribution for some selected non-trigger channel; Right: charge values for some selected channel participating in the trigger. Combined information for all channels in run is used for control of PMTs work stability. As one can see in [PITH_FULL_IMAG…
Figure 5
Figure 5. Figure 5: Top: charge calibration (black), trigger low (blue) and high (red) threshold values for channels in some selected run; Bottom: Ratios of trigger low (blue) and high (red) threshold to charge calibration values for channels in some selected run. 3.2 Distribution for tri…
Figure 6
Figure 6. Figure 6: The noise rates for channels in some selected run. 3.3 Shape comparison for all signals In addition to calibration and threshold values the general comparison of the charge distribu￾tion shapes are performed. Shapes of distributions of any charge deposited in the chann…
Figure 7
Figure 7. Figure 7: Charge distributions (left) and cumulative distributions obtained from them (right) for basic (red) and considered (blue) runs correspondingly. Ratio of charge distributions is shown in black on right panel. 4. Gantt-plots To analyse as much events as possible with the…
Figure 8
Figure 8. Figure 8: Top: the Gantt-plot used for selection of events with the LED matrix switched on in each channel. Bottom: the Gantt-plot used for selection events with stable event rate in each channel. For the second purpose the other Gantt-plot was used. This plot contains 100 thous…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages

  1. [1]

    Avrorin et al, Baikal-GVD: status and prospects , in proceedings of XXth International Seminar on High Energy Physics (QUARKS-2018) , EPJ Web Conf

    A.D. Avrorin et al, Baikal-GVD: status and prospects , in proceedings of XXth International Seminar on High Energy Physics (QUARKS-2018) , EPJ Web Conf. V olume 191, 2018

    work page 2018

  2. [2]

    Avrorin et al, Data acquisition system of the NT1000 Baikal neutrino teles cope, Instruments and Experimental Techniques 3 (2014) 262–273

    A.D. Avrorin et al, Data acquisition system of the NT1000 Baikal neutrino teles cope, Instruments and Experimental Techniques 3 (2014) 262–273

    work page 2014

  3. [3]

    Avrorin, R

    A.D. Avrorin, R. Dvornický et al, Luminescence of water in Lake Baikal observed with the Baikal-GVD neutrino telescope , in proceedings V ery Large V olume Neutrino T elescopes (VLVnT-2018), EPJ Web Conf. V olume 207, 2019. 7

    work page 2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.