{"id":"9081ed47-13e3-48cd-930d-3e2b3801ed25","arxiv_id":"2502.02471","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"General-purpose Swin Transformer and ConvNeXt encoders outperformed histopathology-specific ViT foundation models for cell instance segmentation and classification across PanNuke, CoNIC, and CytoDArk0.","lead":"This paper compares frozen image encoders, from general-purpose models like Swin Transformer and ConvNeXt and from histopathology-specific models like UNI2 and Virchow2, on cell segmentation and classification tasks. It finds that general-purpose, non-ViT encoders performed better on three histology datasets, challenging the assumption that pathology-specific foundation models are always the best choice for cell-level analysis.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The causal step from empirical ranking to 'representation learning gap' is underdetermined: all pathology models are DINOv2-style ViTs while the winners are hierarchical non-ViT encoders, and the ViT branch of the decoder depends on 1x1 convolutions and interpolation that Section V concedes may…","rationale":"The paper is a useful, clearly described empirical comparison, and the raw rankings on PanNuke are likely reproducible given the frozen encoders, fixed decoder, and public datasets. The main threat is not the internal consistency of the reported numbers but the interpretation attached to them. The design confounds encoder architecture and pretraining domain, and the decoder's 1x1-convolution adaptation specifically disadvantages the non-hierarchical ViT encoders that all pathology models use. The paper's own Section V acknowledges dimensionality reduction, patchification, and interpolation as additional factors, which weakens the causal claim without providing a control. The proposed feature-pyramid adapter is a feasible single experiment that would determine whether the ViT underperformance is an artifact of decoder compatibility. Because the reader's conditional verdict already demands either a narrowed claim or controlled experiments, my read does not change that verdict; it strengthens the reason for the condition.","tokens_in":11808,"tokens_out":11390,"duration_ms":111822,"concrete_test":"Keep the UNI2 and Virchow2 encoders frozen but replace the 1x1-convolution dimensionality-reduction path with a small feature pyramid that produces the four skip-connection scales from ViT features using strided 3x3 convolutions or bilinear downsampling, leaving the decoder heads, loss, and training schedule unchanged. Re-run Tables I through III for these two encoders with this ViT-friendly adapter. If either pathology ViT reaches the PQ and mPQ+ of Swin2-B-22K or ConvNeXt-B-22K, the reported gap is caused by decoder compatibility rather than encoder representation quality; if the gap persists at comparable FLOPs, the encoder-level interpretation is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Strictly read, the paper's headline result (Swin2-B-22K and ConvNeXt-B-22K outperform ViT-based models) is an empirical ranking, and the PanNuke table supports it for those specific models. The load-bearing problem is the causal step in the conclusion: attributing the ranking to 'superior representation capabilities' of general-purpose encoders. The experiment cannot separate representation quality from (i) architecture class, because all three histopathology models are DINOv2 ViTs while the winners are hierarchical Swin/ConvNeXt encoders; (ii) decoder compatibility, because the UNETR-style skip connections in Section II-B are designed for hierarchical feature pyramids and the ViT branch requires 1x1 convolutions and upsampling to create four scales from same-resolution maps; and (iii) input preprocessing, because encoders have different native resolutions and patch sizes, requiring rescaling or interpolation of patch embeddings as the paper itself notes in Section V. The paper names these as 'not the sole factor,' but does not quantify them. The cross-dataset conclusion is further narrowed by the PanNuke-based top-five selection, which leaves general-purpose ViT baselines out of Tables II and III entirely. Therefore the conclusion that the gap is about pre-training domain or encoder representations is underdetermined by the evidence presented.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates frozen patch-embedding encoders from general-purpose foundation models (ImageNet-22K/LVD-142M pretrained) and histopathology-specific foundation models (UNI2, Virchow2, Prov-GigaPath) for cell instance segmentation and classification. Using a UNETR-style decoder with the CISCA framework, the authors compare 18 encoder/feature-extraction configurations on PanNuke, then select five top models for evaluation on CoNIC and the newly introduced CytoDArk0 dataset. The reported results show that general-purpose non-ViT encoders (Swin2-B-22K, ConvNeXt-B-22K, MaxViT-B-21K) outperform histopathology-specific ViT encoders on most metrics, and the conclusions interpret this as evidence of superior representation capabilities of the general-purpose encoders.","tokens_in":11977,"tokens_out":4158,"duration_ms":40190,"significance":"If the central claim were fully supported, the paper would provide practical guidance for choosing frozen encoders for cell segmentation and classification, and it would challenge the assumption that domain-specific ViT foundation models are always preferable for histopathology tasks. The study has several strengths: it evaluates a broad set of encoders under a consistent decoder, uses three datasets with different staining and tissue types, freezes all encoders to isolate pretraining features, and makes use of recently released pathology foundation models. The cross-dataset evaluation and the inclusion of CytoDArk0 add useful empirical evidence to the field.","major_comments":[{"comment":"The central conclusion that the results reveal the 'superior representation capabilities' of general-purpose Swin Transformer V2 and ConvNeXt encoders is underdetermined because architecture class, pretraining domain, decoder compatibility, and input preprocessing are fully confounded. All histopathology-specific encoders are ViTs (UNI2, Virchow2, Prov-GigaPath), while the best general-purpose encoders are hierarchical non-ViT models (Swin2, ConvNeXt, MaxViT). The paper itself concedes in Section V that 1x1 convolutions for channel reduction, patchification, and interpolation of patch embeddings are additional factors, but it does not quantify their contribution. Without an ablation that controls for architecture class (for example, an ImageNet-pretrained ViT with the same decoder-side handling as the pathology ViTs, or a histopathology-pretrained non-ViT encoder), the observed ranking cannot be attributed to a representation learning gap between general-purpose and domain-specific pretraining.","section":"Sections IV–VI, Tables I–III"},{"comment":"The selection of the top five models on PanNuke before evaluation on CoNIC and CytoDArk0 excludes all general-purpose ViT baselines from the cross-dataset comparison. Consequently, Tables II and III only compare selected non-ViT general-purpose encoders with selected histopathology ViT encoders, so the cross-dataset claim that general-purpose non-ViT models outperform histopathology-specific ViT models is not a full comparison. The generalizability of the conclusion across datasets would require reporting results for the full set of encoders on CoNIC and CytoDArk0, or at least for a comparable general-purpose ViT baseline.","section":"Section III-A, Tables II and III"},{"comment":"No error bars, confidence intervals, or statistical significance tests are reported for any of the performance metrics. Several differences between models are small (for example, PanNuke PQ 63.19 vs. 62.27 between Swin2-B-22K and ConvNeXt-B-22K, and CoNIC PQ 61.15 vs. 60.02 vs. 59.67 among the three non-ViT models). Without multiple runs or uncertainty quantification, it is not possible to assess whether the reported rankings reflect systematic differences or noise, and the fine-grained comparative statements in Section IV therefore need additional support.","section":"Tables I–III, Section III-C"}],"minor_comments":[{"comment":"The abstract and introduction frame the study as measuring a 'representation learning gap,' but the experiments evaluate downstream task performance on frozen encoders. Consider rephrasing to 'task-performance gap' or explicitly stating that representation quality is inferred through downstream performance.","section":"Abstract and Introduction"},{"comment":"There is a typo in the Introduction: 'tipically' should be 'typically.'","section":"Section I"},{"comment":"The sentence 'P Qis the product' is missing a space: 'P Q is the product.'","section":"Section III-C"},{"comment":"For the non-ViT encoders, the 'Feature block' column lists indices such as [2,5,32,35] for ConvNeXt-B-22K and [1,3,21,23] for Swin2-B-22K, but the mapping from these indices to the four hierarchical stages is not explained in the text. A brief note in Section II-B or the table caption would improve interpretability.","section":"Table I"},{"comment":"The text states that Prov-GigaPath exhibits the highest training and validation loss throughout, but Figure 2 appears to plot only a subset of the 18 models. Please clarify which models are included in the figure and whether the loss curves for all models are available.","section":"Section IV, Figure 2"},{"comment":"The description of pretraining datasets is slightly confusing: 'ImageNet-22K' is used for some encoders, while 'ImageNet-21K (Google specific variant of ImageNet-22K)' is used for MaxViT and ViT-L-21K. Please specify the exact checkpoint versions used, since the distinction may matter for reproducibility.","section":"Section II-A"}],"recommendation":"major_revision","confidential_remarks":"The paper is a useful empirical benchmark, but the central causal claim is overstated relative to the evidence. The confound between architecture and pretraining domain, together with the PanNuke-based model selection, are the main load-bearing issues. The manuscript could be made acceptable by substantially softening the conclusions, reanalyzing the data to separate architecture from pretraining effects (e.g., including a general-purpose ViT in all comparisons or adding pathology-specific non-ViT encoders), and reporting uncertainty estimates or at least multiple runs. I do not think the paper should be rejected, because the empirical results are likely to be of interest to the community, but the current framing is not supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi,\n\nThe paper is a useful addition to the cell-segmentation benchmarking literature, but you should read the headline result with a grain of salt. What's genuinely new: a systematic comparison of frozen general-purpose and histopathology-specific encoders under a fixed CISCA decoder, across three datasets including the new CytoDArk0, with code promised. The experiments are clean and the per-dataset tables are informative. The finding that shallow ViT features beat deep ones is a nice, non-obvious result that could guide feature-extraction choices.\n\nThe soft spots are the usual ones, and they land harder than the authors acknowledge. The main claim — that general-purpose encoders have superior representation capabilities — is not actually supported. All three pathology models are DINOv2-style ViTs, while the winners are hierarchical Swin/ConvNeXt/MaxViT. That's an architecture confound, not just a pretraining-domain confound. The decoder's UNETR-style skip connections are built for hierarchical feature pyramids; the ViT branch needs 1x1 convolutions and interpolation to produce four scales. And input preprocessing (resizing to training resolution, patch-size interpolation) differs across models. The paper names these in Section V but never quantifies them, then the conclusion still says 'superior representation capabilities.' That's overreach.\n\nAlso missing: error bars, confidence intervals, or significance tests. And the top-five selection on PanNuke means CoNIC and CytoDArk0 only see the winners plus UNI2/Virchow2. General-purpose ViTs (ViT-L-22K, ViT-H-142M) are not evaluated on those datasets, so we have no idea whether the general-purpose-ViT-vs-pathology-ViT comparison generalizes beyond PanNuke. That leaves a hole in the cross-dataset argument.\n\nNone of this kills the paper. The empirical ranking of specific models in this specific decoder is valuable, and practitioners who want to pick a frozen encoder for a cell-segmentation pipeline can use it. The authors are honest about the caveats; the discussion section is balanced. But the conclusion should be narrowed to something like 'in this CISCA setup, frozen hierarchical non-ViT encoders outperform frozen ViT-based pathology models.' If they want to claim a representation learning gap, they should add an architecture-matched control (e.g., a pathology-specific Swin or a general-purpose ViT with matched tuning), report variance across splits, and run all models on all datasets.\n\nWho's the audience? Anyone choosing encoders for cell segmentation, and people building pathology foundation models. It deserves a serious referee; I'd send it out. With revisions, it could be a solid archival paper.\n\nBest,","headline":"Useful benchmark, but the 'representation learning gap' conclusion is overreaching given the architecture confound and missing baselines.","tokens_in":12587,"tokens_out":2834,"would_cite":true,"duration_ms":26796,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"General-purpose encoders beat histopathology-specific ViT foundation models for cell instance segmentation and classification across three datasets, including the new Nissl-stained CytoDArk0.","keywords":["cell instance segmentation","cell classification","foundation models","histopathology","vision transformers","Swin Transformer V2","ConvNeXt","CytoDArk0"],"falsifier":"Re-run the comparison with a decoder and preprocessing pipeline that removes the asymmetries for ViT encoders—for example, by consuming ViT patch embeddings directly without 1x1 channel reduction and by matching native inference resolution—and check whether the gap between Swin2-B-22K/ConvNeXt-B-22K and UNI2/Virchow2 persists; if it closes, the result is an artifact of decoder treatment rather than pretraining quality.","tokens_in":11521,"feed_emoji":"🔬","tokens_out":6009,"duration_ms":56856,"temperature":0.7,"pith_summary":"This paper asks whether foundation models pretrained on histopathology whole-slide images give better cell-level features than general-purpose models pretrained on natural images, when both are used as frozen encoders for cell instance segmentation and cell classification. On PanNuke, CoNIC, and the newly introduced CytoDArk0 brain-cell dataset, the answer is no: Swin Transformer V2 and ConvNeXt encoders pretrained on ImageNet-22K outperform UNI2, Virchow2, and Prov-GigaPath. The paper argues that inductive biases such as locality and hierarchical feature learning matter more for small-cell localization than pretraining domain. If this holds, model selection for cell-focused histopathology should not assume that domain-specific ViT encoders are the strongest choice.","feed_headline":"General-purpose encoders beat pathology AI on cell tasks","feed_subtitle":"Frozen Swin V2 and ConvNeXt outperform UNI, Virchow2, and Prov-GigaPath for segmenting and classifying cells.","key_machinery":"The comparison is carried out by an encoder-decoder architecture built on the CISCA framework: one frozen encoder supplies feature maps from four levels, skip connections fuse them into a shared decoder, and three convolutional heads predict a three-class semantic map, four directional distance maps, and a cell-type map, with post-processing converting these into instance masks and cell labels. Every encoder is kept frozen and paired with the same decoder and training recipe, so performance differences are interpreted as differences in the pretrained representations. For ViT encoders, feature maps are extracted from shallow, deep, or mixed blocks and passed through 1x1 convolutions and upsampling to match the decoder's channel and resolution schedule.","core_discovery":"The central claim is that frozen, general-purpose encoders with convolutional or hierarchical attention designs—Swin Transformer V2 and ConvNeXt pretrained on ImageNet-22K—produce better patch embeddings for cell instance segmentation and classification than frozen ViT encoders from histopathology foundation models, across all three evaluated datasets. On PanNuke, Swin2-B-22K achieves the highest panoptic quality and multi-class panoptic quality, and on CoNIC and CytoDArk0, ConvNeXt-B-22K leads. The paper attributes the gap to ViTs lacking locality and hierarchical representation, which diffuses fine-grained information needed for precise detection and segmentation, while also listing dimensionality reduction of ViT feature maps, patchification, interpolation, and supervised versus self-supervised pretraining as additional factors requiring further experiments.","pith_inferences":["If inductive bias rather than pretraining data is the deciding factor, then a histopathology-specific Swin or ConvNeXt encoder trained on pathology patches could combine domain knowledge with locality and plausibly outperform both families tested here.","The results imply that evaluations of pathology foundation models should include dense prediction tasks such as cell segmentation, not only classification or patch retrieval, before claiming general superiority.","The shallow-versus-deep ViT finding suggests a simple testable recipe for adapting a frozen ViT to cell segmentation: extract features from early blocks, since the paper's mixed strategies only partially recover the gap.","The paper leaves open whether supervised ImageNet-22K pretraining itself, rather than the non-ViT architecture, drives the advantage; a same-architecture comparison between supervised and self-supervised ViTs would separate those factors."],"forward_implications":["Frozen general-purpose hierarchical encoders can serve as strong, parameter-efficient backbones for cell instance segmentation and classification without fine-tuning.","Domain-specific ViT foundation models do not necessarily transfer their advantages to dense cell-level tasks; their strengths may lie in patch-level or whole-slide-level tasks.","Shallow ViT features are more useful than deep ViT features for segmentation, indicating that localization information is concentrated in early blocks of these self-supervised ViTs.","Swin2-B-22K tops the PanNuke benchmark while using the fewest parameters among the tested models, so the performance gain does not require extra compute.","On the Nissl-stained brain dataset, the same ordering holds, suggesting the result generalizes beyond H&E-stained tissue to a different stain and cell type."],"supporting_citations":[{"why":"Supplies UNI, the precursor to the UNI2 histopathology-specific ViT encoder used as a specialized baseline.","marker":"[2]"},{"why":"Supplies Virchow2, the histopathology-specific ViT encoder used as a specialized baseline.","marker":"[30]"},{"why":"Supplies Prov-GigaPath, the histopathology-specific ViT encoder used as a specialized baseline.","marker":"[29]"},{"why":"Provides the CISCA framework, including the loss functions, post-processing, and data-splitting protocol.","marker":"[23]"},{"why":"Supplies the base implementation of the shared decoder and encoder loading used for all models.","marker":"[12]"},{"why":"Provides the pretrained general-purpose encoders and the block-level feature extraction used across models.","marker":"[28]"},{"why":"Supplies Swin Transformer V2, the top-performing general-purpose encoder on PanNuke.","marker":"[13]"},{"why":"Supplies ConvNeXt, the top-performing general-purpose encoder on CoNIC and CytoDArk0.","marker":"[15]"},{"why":"Provides the PanNuke dataset, one of the three cell segmentation and classification benchmarks.","marker":"[4]"},{"why":"Provides the CoNIC dataset and the panoptic-quality metrics used to evaluate the models.","marker":"[6]"}],"fun_headline_variants":["Frozen general-purpose encoders top pathology models on cell tasks","Swin V2 and ConvNeXt beat UNI, Virchow2, Prov-GigaPath for cells","General-purpose Swin/ConvNeXt trump histopathology ViTs in cell analysis","Frozen ImageNet-22K encoders outperform UNI, Virchow2, Prov-GigaPath"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the fixed decoder, feature-extraction choices, and preprocessing treat all encoders fairly, so observed performance differences measure differences in what the encoders learned; the paper itself notes this is not the only factor, since architecture and pretraining domain are entangled.","fun_headline_variants_meta":{"raw":{"variants":["Frozen general-purpose encoders top pathology models on cell tasks","Swin V2 and ConvNeXt beat UNI, Virchow2, Prov-GigaPath for cells","General-purpose Swin/ConvNeXt trump histopathology ViTs in cell analysis","Frozen ImageNet-22K encoders outperform UNI, Virchow2, Prov-GigaPath"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000943,"raw_usage":{"total_tokens":4066,"prompt_tokens":1017,"completion_tokens":3049,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":2953}},"tokens_in":633,"tokens_out":3049,"duration_ms":20990,"temperature":1.0,"reasoning_tokens":2953,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T11:59:21.799450+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the comparison with a decoder and preprocessing pipeline that removes the asymmetries for ViT encoders—for example, by consuming ViT patch embeddings directly without 1x1 channel reduction and by matching native inference resolution—and check whether the gap between Swin2-B-22K/ConvNeXt-B-22K and UNI2/Virchow2 persists; if it closes, the result is an artifact of decoder treatment rather than pretraining quality.","supporting_citations":[{"cited_title":"Towards a general-purpose foundation model for computational pathology","cited_arxiv_id":null,"evidence_quote":"Supplies UNI, the precursor to the UNI2 histopathology-specific ViT encoder used as a specialized baseline."},{"cited_title":"A whole-slide foundation model for digital pathology from real-world data","cited_arxiv_id":null,"evidence_quote":"Supplies Prov-GigaPath, the histopathology-specific ViT encoder used as a specialized baseline."},{"cited_title":"cellseg models.pytorch: Cell/nuclei segmentation models and benchmark","cited_arxiv_id":null,"evidence_quote":"Supplies the base implementation of the shared decoder and encoder loading used for all models."},{"cited_title":"Pytorch image models","cited_arxiv_id":null,"evidence_quote":"Provides the pretrained general-purpose encoders and the block-level feature extraction used across models."},{"cited_title":"Swin transformer v2: Scaling up capacity and resolution","cited_arxiv_id":null,"evidence_quote":"Supplies Swin Transformer V2, the top-performing general-purpose encoder on PanNuke."},{"cited_title":"Conic challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting","cited_arxiv_id":null,"evidence_quote":"Provides the CoNIC dataset and the panoptic-quality metrics used to evaluate the models."}],"review_version":1}