{"id":"6f9aa130-f90e-48b8-a8ae-73ae8623e17a","arxiv_id":"2507.12624","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"PaPIS is a pathology-aware full-reference similarity metric for virtual staining, built from cell-morphology segmentation features and Retinex decomposition, demonstrated as both an evaluation score and a training loss.","lead":"This paper introduces PaPIS, a new image-similarity score for virtual staining that uses cell-shape-aware deep features and Retinex decomposition instead of natural-image metrics. The authors also use PaPIS as a training loss in a CycleGAN and report improved staining fidelity on example images.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that PaPIS aligns with pathology-relevant cues is empirically unsupported: no expert or clinical ground truth validates the feature-space premise in §3.1, and the paper's own Section 5 concedes the absence of subjective validation.","rationale":"I read the paper in good faith. The construction is clearly described: PaPIS extracts multi-layer features from a cell-morphology segmentation encoder, applies Retinex decomposition, and combines a high-frequency structure term with a low-frequency illumination MSE term. The idea of using a domain-specific encoder is plausible, and the authors are transparent about the missing expert validation. The reader's weakest_assumption identifies exactly the same load-bearing point: the unsupported premise that the chosen feature space captures pathology-relevant quality differences. My independent reading of Sections 3.1, 4.2, and 5 confirms that no external criterion—expert ratings, diagnostic utility, or a clinical endpoint—is ever used to evaluate PaPIS. The scatter plots and heatmaps only show that PaPIS disagrees with existing metrics in selected regions; disagreement alone is not evidence of superior pathology alignment. The PaPIS-guided training experiment in §4.3 is also evaluated without a blind or independent fidelity measure, so the improved-histological-fidelity claim rests on the same assumption. Because this is a core, load-bearing premise and it is explicitly conceded as unvalidated, the reader's REJECT verdict is appropriate under Pith semantics: the central claim is stated as demonstrated but is not verified. The concrete test I propose—a pathologist reader study with rank-correlation comparison against existing metrics—would settle whether the premise lands, and it is exactly the kind of validation the paper itself identifies as future work. No ad hominem or manufactured concern is involved; the issue is empirical support for the central claim.","tokens_in":10602,"tokens_out":1718,"duration_ms":21864,"concrete_test":"Conduct a reader study in which at least two board-certified pathologists independently rate or rank virtual staining outputs from multiple models on a held-out set of, e.g., 100 image pairs, with blinded and randomized presentation. Compute Spearman rank correlation between PaPIS, SSIM, LPIPS, and DISTS scores and the expert ratings. The central claim would be supported only if PaPIS shows significantly higher correlation with expert scores than the existing metrics do; if PaPIS is not significantly better than LPIPS or DISTS, the claim that it 'more accurately aligns with pathology-relevant visual cues' collapses. As a secondary check, have the same experts assess the PaPIS-guided versus baseline CycleGAN outputs in a forced-choice paradigm, rather than relying on PaPIS-based or qualitative visual claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is that distances computed in the feature space of a nuclei-segmentation encoder (Ignatov et al. [24]), after Retinex decomposition, correspond to pathologically meaningful differences in virtual staining quality. This premise is asserted in §3.1 ('pathologists focus on distinct perceptual attributes that cannot be adequately captured by models trained on natural image datasets') but is never tested against any external ground truth. Section 5 explicitly acknowledges the 'absence of subjective validation by expert pathologists.' The experiments in §4.2 rely on hand-selected scatter points (AH/AL/PD/TD) and qualitative image examples; these illustrate that PaPIS differs from SSIM, LPIPS, and DISTS, but they do not establish that the differences are pathologically relevant rather than merely a reweighting of texture/structure statistics. Similarly, §4.3 evaluates the PaPIS-guided CycleGAN by visual inspection and by PaPIS-related reasoning, not by an independent measure of histological fidelity. Without a correlation study against expert ratings or a diagnostic task, the improved-fidelity and better-alignment claims remain unsupported; PaPIS could be no more than a different perceptual metric with different weights.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces PaPIS (Pathology-Aware Perceptual Image Similarity), a full-reference image quality assessment (FR-IQA) metric for virtual staining in histopathology. The metric uses a pre-trained cell-morphology segmentation encoder to extract multi-layer features, applies multi-scale Retinex decomposition to separate illumination and reflectance components, and computes a perceptual distance as a weighted combination of a low-frequency MSE term and a high-frequency SSIM-like term. The authors also integrate PaPIS as an auxiliary loss in a CycleGAN-based virtual staining model (Eq. 8). Experiments compare PaPIS with SSIM, LPIPS, and DISTS on PARS-to-H&E image pairs via scatter plots and selected examples, and present qualitative evaluations of the PaPIS-guided model, including patch-wise similarity heatmaps.","tokens_in":10874,"tokens_out":6004,"duration_ms":59257,"significance":"The paper addresses a genuine need: standard perceptual metrics such as SSIM, LPIPS, and DISTS are designed for natural images and may not reflect what pathologists care about in virtual staining output. The proposed construction is coherent, and the use of a nuclei-segmentation encoder is a plausible domain-specific choice. The metric is explicitly defined in Section 3.3, which allows independent implementation. The figures provide qualitative evidence that PaPIS can differ from existing metrics in ways interpretable in terms of cellular structure. However, the central empirical claim—that PaPIS aligns more closely with pathology-relevant visual cues—is not supported by any external ground truth. Section 5 explicitly acknowledges the absence of subjective validation by expert pathologists. The paper also uses PaPIS as a training loss and then evaluates the resulting model with PaPIS-related reasoning, which is circular. If validated with expert ratings or a downstream task, PaPIS could be a valuable contribution; as it stands, the evidence is insufficient to support the abstract's claims.","major_comments":[{"comment":"The load-bearing premise of the paper is that distances computed in the feature space of a nuclei-segmentation encoder (Ignatov et al. [24]) reflect pathologically meaningful differences in virtual staining quality. This premise is asserted in Section 3.1, where the authors state that pathologists focus on attributes that cannot be captured by models trained on natural images, but it is never tested against expert ratings or an independent diagnostic task. Section 5 explicitly concedes 'the absence of subjective validation by expert pathologists.' Consequently, the abstract's claim that PaPIS 'more accurately aligns with pathology-relevant visual cues' is not supported by the evidence; the demonstrated differences from SSIM, LPIPS, and DISTS could simply reflect a reweighting of texture and structure statistics. A correlation study with pathologist ratings or a downstream histopathology task is necessary to validate this claim.","section":"Section 3.1 and Section 5"},{"comment":"The evaluation of the PaPIS-guided CycleGAN is circular. The model is trained with L_papis = 1 − PaPIS (Eq. 8), and then the reported improvements in 'histological fidelity' (Section 4.3) and the regional sensitivity analysis (Section 4.3.1) rely on PaPIS-based reasoning or PaPIS-derived heatmaps. Since the training objective directly minimizes PaPIS distance, any improvement in PaPIS-related behavior is partly guaranteed by optimization. Independent evaluation—for example, segmentation performance, expert assessment, or conventional IQA metrics with statistical testing—is required to support the claim that PaPIS guidance improves histological fidelity.","section":"Section 4.3 and Eq. (8)"},{"comment":"The comparative experiments use hand-selected scatter points (AH, AL, PD, TD) and qualitative image examples. These illustrate that PaPIS can disagree with existing metrics, but they do not establish which metric is more accurate with respect to pathology. There are no correlation statistics, no error bars, and no statistical tests. The four categories are defined post hoc from the same data, which creates a risk of confirmation bias. To support the claim that PaPIS 'captures relevant pathological features overlooked by conventional metrics,' the authors should report quantitative agreement with a ground truth, such as expert rankings or an objective histopathology task.","section":"Section 4.2"},{"comment":"The weights α_ij and β_ij are described as 'randomized' with sum 1, making PaPIS stochastic. The paper does not report the random seed, the number of random draws, or the variance of PaPIS scores across draws. This undermines reproducibility and leaves open the possibility that the scatter-plot patterns in Section 4.2 depend on a particular random realization. The authors should either fix the weights with a sensitivity analysis or provide variance estimates for the reported scores.","section":"Section 3.3, Eq. (6)"}],"minor_comments":[{"comment":"The notation µ(i) with a tilde and the channel indices j are not fully defined; the authors should specify that j indexes channels within layer i and that the tilde denotes channel-wise min-max normalized features.","section":"Section 3.3, Eq. (6)"},{"comment":"The WSI dimension '18, 202 × 48, 800' is formatted inconsistently; the spaces around the multiplication sign should be standardized across all dimensions (e.g., 18,202×48,800).","section":"Section 4.1.1"},{"comment":"The loss weight λ2 is reported as 1.0 in Section 4.1.2 but as 2.0 in Section 4.3 for the PaPIS-guided model; please clarify whether these refer to different configurations or whether one is a typographical error.","section":"Sections 4.1.2 and 4.3"},{"comment":"The heading 'P ARS images Acquisition' contains a spacing typo; it should read 'PARS image acquisition.'","section":"Section 3.4.1"},{"comment":"The scatter plots would benefit from axis labels, units, a legend for the AH/AL/PD/TD categories, and a statement of the number of image pairs shown.","section":"Figures 5, 7, and 8"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a relevant problem and the metric construction is reasonable, but the empirical validation is the central weakness. The authors' own Section 5 concedes the lack of expert validation, and the training-loss evaluation in Section 4.3 is circular with respect to the proposed metric. A major revision that adds an external validation study (e.g., correlation with pathologist ratings or a downstream segmentation/diagnostic task) and independent evaluation of the PaPIS-guided model would be necessary to support the claims. The 'randomized' weights in Eq. (6) also need fixing or a stability analysis before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper proposes PaPIS, a full-reference IQA metric for virtual staining that replaces the ImageNet backbone of LPIPS/DISTS with a nuclei-segmentation encoder and adds Retinex-based decomposition of features into illumination and reflectance. That is a legitimate new combination, and the paper describes it clearly. The feature visualizations comparing the histology-trained encoder against VGG are a nice touch, and Section 5 candidly admits that no expert validation was performed. Credit is also due for releasing the training pipeline details, though not the code or data.\n\nThe soft spot is the central claim. The abstract says PaPIS “more accurately aligns with pathology-relevant visual cues,” but there is no external ground truth to back that. The scatter plots rely on hand-picked AH/AL/PD/TD points; they show that PaPIS disagrees with SSIM/LPIPS/DISTS, not that it agrees with pathologists. The heatmap observations are post-hoc interpretations. The training experiment is the most serious issue: L_papis is defined as 1 - PaPIS, and the improvement in the PaPIS-guided model is then discussed using PaPIS-based reasoning, so part of the reported gain is baked into the objective. That is circular for the claim that PaPIS improves histological fidelity.\n\nThe premise in §3.1—that ImageNet features cannot capture pathology-relevant attributes—is asserted but never tested. Free parameters (lambda, alpha, beta, sigma, c1, c2) are many, and no sensitivity analysis is given. These are real gaps, but they are validation gaps rather than flaws in the construction. The metric itself is well-defined and the feature extractor choice has face validity for nuclear morphology.\n\nWho is this for? Researchers building or evaluating virtual staining models, and anyone working on domain-specific IQA. It is also a useful case study for why medical IQA needs human or clinical anchors. I would bring it to a reading group with the question: what would convince you that a perceptual metric is pathology-aware?\n\nMy recommendation: this deserves peer review, but with a strong demand for a reader study—at least a few pathologists rating a set of virtual staining outputs—and correlation statistics between PaPIS and expert scores. Without that, the paper should not be accepted as-is. With it, the metric could be genuinely useful.","headline":"Sensible metric construction, but the pathology-alignment claim is unvalidated and the training evaluation is partly circular; worth reviewing with major revisions.","tokens_in":11406,"tokens_out":1808,"would_cite":false,"duration_ms":21038,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PaPIS, a full-reference metric built on nuclei-segmentation features and Retinex decomposition, claims to capture pathology-relevant similarity better than SSIM, LPIPS, and DISTS, and to improve virtual staining when used as a training…","keywords":["image quality assessment","virtual staining","perceptual similarity","pathology-aware metric","Retinex decomposition","nuclei segmentation features","CycleGAN","histology fidelity"],"falsifier":"A reader study in which board-certified pathologists rank the same virtual-staining output pairs would settle the claim: if PaPIS rankings disagree with expert rankings on pairs where SSIM, LPIPS, or DISTS agree with pathologists, the central claim fails. A cheaper check would be to perturb images so that nuclear boundaries are slightly shifted and measure whether PaPIS changes in the expected direction, while the natural-image metrics stay flat.","tokens_in":10387,"feed_emoji":"🔬","tokens_out":3498,"duration_ms":40094,"temperature":0.7,"pith_summary":"This paper introduces PaPIS, a full-reference similarity metric for virtual staining that scores images by how well cell-level morphology matches the reference, rather than by generic texture or natural-image statistics. The authors argue that standard metrics like SSIM and LPIPS, designed for natural photos, overlook nuclear structure and tissue architecture that pathologists rely on. PaPIS extracts features from a network trained for nuclei segmentation, splits them into reflectance and illumination components via Retinex, and combines high- and low-frequency distances. They show that PaPIS disagrees with SSIM precisely in cases where histology matters, and that adding PaPIS as an extra loss to a CycleGAN-based virtual staining model yields outputs with better-preserved cell shape, position, and size. If the paper is right, pathology-aware metrics of this kind would give automated quality control and training signal for clinical virtual staining.","feed_headline":"Cell-morphology metric rescores virtual staining quality","feed_subtitle":"PaPIS reads nuclei structure instead of texture, and as a loss it sharpens generated H&E stains.","key_machinery":"The load-bearing object is the feature extractor: a pre-trained nuclei-segmentation encoder whose deep feature maps concentrate on nuclear regions rather than natural-image textures, in contrast to ImageNet-trained backbones used by LPIPS and DISTS. On top of that, Multi-Scale Retinex decomposes each feature map into reflectance and illumination components, and PaPIS is defined as a weighted sum of a high-frequency reflectance similarity term (comparing means and standard deviations of reflectance features) and a low-frequency illumination MSE across layers and channels, with a hyperparameter λ balancing the two. The same distance is converted into a training loss, Lpapis = 1 − PaPIS, and added to the CycleGAN objective.","core_discovery":"On its own terms, the paper's central claim is that a perceptual distance computed on cell-morphology features, with Retinex illumination/reflectance separation, captures pathological fidelity that SSIM, LPIPS, and DISTS miss, and that using this distance as a training loss improves virtual H&E generation. The evidence is categorical and qualitative: scatter-plot comparisons label image pairs as PaPIS-dominant or traditionally-dominant, and heatmaps show PaPIS scores dropping in regions with staining artifacts or degraded micro-anatomy even when SSIM stays high. The authors state that PaPIS more accurately aligns with pathology-relevant visual cues and distinguishes subtle cellular structures that conventional metrics tend to overlook.","pith_inferences":["A direct, cheaper validation than a full reader study would be to correlate PaPIS scores with downstream nuclei-segmentation accuracy on the same image pairs; the paper does not report this correlation.","If the feature-space assumption holds, PaPIS could be used as a screening filter that flags low-scoring patches for targeted pathologist review, reducing the workload of manual quality control.","The same Retinex-plus-morphology-feature recipe could generalize to other stain targets by retraining the encoder on IHC or Masson's trichrome segmentation, a testable extension the authors leave implicit.","PaPIS's sensitivity to nuclear organization suggests it may also predict performance of cell-counting or tumor-grading algorithms, though the paper does not test such downstream clinical tasks."],"forward_implications":["PaPIS can flag virtual staining artifacts and morphologically degraded regions that SSIM scores as high-quality, as demonstrated in the patch-wise heatmap analysis.","Adding PaPIS as a perceptual loss to a CycleGAN-based virtual staining model improves preservation of cell position, shape, and size relative to the standard CycleGAN.","Because PaPIS operates on cell-level morphology rather than pixel statistics, it is claimed to be modality-agnostic and applicable to autofluorescence, quantitative phase, and hyperspectral virtual staining.","The modular design allows the segmentation encoder to be replaced or fine-tuned for different stain types and tissue domains, extending PaPIS beyond H&E.","PaPIS provides a quantitative, automated alternative to subjective expert review for routine quality control in virtual staining pipelines."],"supporting_citations":[{"why":"Supplies the pre-trained cell-morphology segmentation encoder whose feature maps define the pathology-aware representation space.","marker":"[24]"},{"why":"Supplies the Multi-Scale Retinex algorithm used to decompose features into reflectance and illumination components.","marker":"[25]"},{"why":"Provides the structural similarity formulation that the high-frequency reflectance distance term is modeled on.","marker":"[16]"},{"why":"Provides the structure-and-texture similarity design that PaPIS adapts for feature statistics comparison.","marker":"[6]"},{"why":"Defines the LPIPS baseline perceptual metric that PaPIS is compared against and contrasted with.","marker":"[5]"},{"why":"Supplies the CycleGAN framework that PaPIS-guided training is built on.","marker":"[15]"},{"why":"Supplies the pixel-to-pixel whole-slide registration and patch extraction pipeline used to build the training and evaluation dataset.","marker":"[26]"},{"why":"Supplies the paired-patch construction methodology and the virtual staining model lineage that the comparative experiments extend.","marker":"[11]"}],"fun_headline_variants":["Pathology-aware metric rethinks virtual stain quality","PaPIS: a metric that sees cell structure, not just texture","Virtual staining gets a pathology-guided scoring metric","New metric trains better virtual stains via cell morphology","PaPIS metric aligns virtual stain scoring with pathology"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that distances computed in the feature space of the nuclei-segmentation network reflect what pathologists care about, and the paper never tests this against expert ratings.","fun_headline_variants_meta":{"raw":{"variants":["Pathology-aware metric rethinks virtual stain quality","PaPIS: a metric that sees cell structure, not just texture","Virtual staining gets a pathology-guided scoring metric","New metric trains better virtual stains via cell morphology","PaPIS metric aligns virtual stain scoring with pathology"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1197,"prompt_tokens":871,"completion_tokens":326,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":487,"completion_tokens_details":{"reasoning_tokens":249}},"tokens_in":487,"tokens_out":326,"duration_ms":4191,"temperature":1.0,"reasoning_tokens":249,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:42:37.684194+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader study in which board-certified pathologists rank the same virtual-staining output pairs would settle the claim: if PaPIS rankings disagree with expert rankings on pairs where SSIM, LPIPS, or DISTS agree with pathologists, the central claim fails. A cheaper check would be to perturb images so that nuclear boundaries are slightly shifted and measure whether PaPIS changes in the expected direction, while the natural-image metrics stay flat.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained cell-morphology segmentation encoder whose feature maps define the pathology-aware representation space."},{"cited_title":"IEEE Transactions on Image processing 6(7), 965–976 (1997)","cited_arxiv_id":null,"evidence_quote":"Supplies the Multi-Scale Retinex algorithm used to decompose features into reflectance and illumination components."},{"cited_title":"IEEE transactions on image processing 13(4), 600–612 (2004)","cited_arxiv_id":null,"evidence_quote":"Provides the structural similarity formulation that the high-frequency reflectance distance term is modeled on."},{"cited_title":"IEEE transactions on pattern analysis and machine intelligence 44(5), 2567–2581 (2020)","cited_arxiv_id":null,"evidence_quote":"Provides the structure-and-texture similarity design that PaPIS adapts for feature statistics comparison."},{"cited_title":"In: Proceedings of the IEEE International Conference on Computer Vision, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the CycleGAN framework that PaPIS-guided training is built on."},{"cited_title":"IEEE Transactions on Biomedical Engineering 71(6), 1901–1912 (2024) 19","cited_arxiv_id":null,"evidence_quote":"Supplies the pixel-to-pixel whole-slide registration and patch extraction pipeline used to build the training and evaluation dataset."},{"cited_title":"Virtual Histology with Photon Absorption Remote Sensing using a Cycle-Consistent Generative Adversarial Network with Weakly Registered Pairs","cited_arxiv_id":"2306.08583","evidence_quote":"Supplies the paired-patch construction methodology and the virtual staining model lineage that the comparative experiments extend."}],"review_version":1}