Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Tile-Based ViT Inference with Visual-Cluster Priors for Zero-Shot Multi-Species Plant Identification

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A fully training-free pipeline reaches 0.348 macro-F1 on multi-species plant identification by tiling quadrat images to match a ViT's input size and reweighting predictions with visual-cluster priors.

desk verdict The 4x4 tiling result is the real story; the cluster-prior gain is transductive and split-dependent, so treat the +0.004 as a hypothesis, not a result. read the letter →

arxiv 2507.06093 v1 pith:GIDKX5NY submitted 2025-07-08 cs.CV cs.IRcs.LG

Murilo Gustineli , Anthony Miyaguchi , Adrian Cheung , Divyansh Khattak This is my paper · ORCID
classification cs.CVcs.IRcs.LG MSC 68T45
keywords plantidentificationmulti-labelclassificationVisionTransformertilinginferencezero-shotadaptationBayesianpriorsK-Meansclusteringmacro-F1
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper shows that a fully training-free pipeline can produce competitive multi-species plant identification on high-resolution vegetation quadrats. Cutting each test image into a 4x4 grid, so each tile roughly matches the 518-pixel input size of a PlantCLEF-fine-tuned Vision Transformer, lifts macro-F1 from 0.006 to 0.344. Re-weighting tile predictions with a Bayesian prior estimated from unsupervised visual clusters of test embeddings adds another 0.004, for a private-leaderboard macro-F1 of 0.348 and second place. The result matters because it suggests that domain shift from single-label training data to multi-label field images can be partly absorbed by inference-time geometry and priors, without retraining or additional labels.

What carries the argument

The load-bearing mechanism is the ViTD2PC24All transformer, fine-tuned on single-label 2024 data, whose 518x518 input demands that high-resolution quadrats be split into a 4x4 grid for patch-level inference. On top of tiling, the pipeline projects test [CLS] token embeddings with PaCMAP, groups them by K-Means into three visual clusters, and forms an empirical prior P(y|c) by averaging the model's own per-image class-probability vectors across each cluster; tile probabilities are reweighted by this prior during inference. A geolocation filter, which keeps species whose nearest observation lies in France, Spain, Italy, or Switzerland, is tested as an alternative prior.

What would settle it

Recompute the prior on a held-out subset of test quadrats, or on independent floristic occurrence data for the same regions, and compare macro-F1 against the reported 0.3483; if the gain over the 0.3442 tiling baseline disappears or turns negative, the transductive prior is the cause. A second check: permute cluster labels and re-estimate priors; if random clusters reproduce the same improvement, the visual clustering itself is not doing the work.

Watch

Extended reading notes

Core claim

The central claim is that matching inference tile scale to a ViT's receptive field plus cluster-aware prior reweighting is enough to make a publicly released single-label model performant on a multi-label identification task. Concretely, the paper reports that a 4x4 tiling of roughly 2000-pixel quadrat images, with top-9 tile predictions aggregated by majority vote, reaches a private macro-F1 of 0.3442. Adding cluster-specific Bayesian priors from PaCMAP and K-Means on [CLS] token embeddings reaches 0.3483, while geolocation filtering reaches 0.3449 private and 0.3160 public. All reported gains come without updating any model weights.

Load-bearing premise

The reported gain from Bayesian priors assumes that averaging the model's own predictions over the very test images being scored yields a regional species prior that transfers to those same images, rather than just echoing the model's systematic errors.

Editorial extensions

If this is right

  • If the reported numbers hold, competitive multi-label plant identification is achievable without training on the 2025 task's multi-label data.
  • Tile size should be matched to the backbone's input resolution rather than chosen arbitrarily.
  • Unsupervised clustering of test embeddings can supply a cheap domain prior for reweighting model outputs.
  • Geolocation filtering and visual-cluster priors are complementary: geolocation leads the public leaderboard, cluster priors lead the private one.
  • The full pipeline, including code and configuration files, is released so the ablation can be reproduced and extended.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the prior is estimated on the same test images that are scored, the +0.004 improvement is at risk of transductive overfitting; a held-out estimate would be a stronger test.
  • Non-overlapping square tiles likely cut plants at boundaries, so sliding windows or token merging may recover additional accuracy beyond 0.348.
  • The same recipe of tiling to a receptive field plus cluster priors could transfer to other high-resolution ecological surveys where a single-label pretrained model already exists.
  • The three clusters are labelled as coastal, alpine, and alpine-grassland communities; whether these ecological interpretations are accurate could be tested against ground-truth species lists.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript reports DS@GT's second-place solution to the PlantCLEF 2025 multi-species plant identification challenge. The pipeline is fully inference-only: a DINOv2-based ViT (ViTD2PC24All) fine-tuned on PlantCLEF 2024 data is applied to 4×4 tiles of each test quadrat; tile predictions are aggregated by top-K voting; and a cluster-specific Bayesian prior, derived by PaCMAP+K-Means clustering of test-image embeddings, is used to reweight tile probabilities. Geolocation filtering is tested as an alternative. On the official private leaderboard, the 4×4 top-9 tiling baseline reaches 0.3442 macro-F1, and adding the visual-cluster prior yields 0.3483, good for second place. Code and configuration are public.

Significance. The tiling result is a genuinely useful, reproducible engineering insight: matching the tile size to the ViT's 518px receptive field improves performance by two orders of magnitude over full-image inference, and the open-sourced code lets others verify it. However, the paper's second claimed contribution—visual-cluster Bayesian priors—is not yet established as a generalizable gain. The prior is estimated transductively from the same test images it reweights, and the public leaderboard score decreases when the prior is applied, whereas the private score increases. If the prior can be validated on held-out data, the contribution would be meaningful; as presented, only the tiling claim is fully supported.

major comments (3)
  1. [§4.3, Table 3] The empirical prior P(y|c) in §4.3 is computed by averaging ViTD2PC24All's own predicted probability vectors over all test images assigned to each K-Means cluster, and these same images are then reweighted during inference. This transductive procedure means the scored images contribute directly to the prior, so the reported +0.004 private improvement (0.34834 vs. 0.34420) may reflect the model amplifying its own biases rather than a generalizable regional prior. The concern is supported by Table 3: the prior variant's public macro-F1 drops to 0.29293 from 0.30810, while the no-prior tiling baseline stays at 0.34420 private and 0.30810 public. To support the claim, the prior should be estimated on held-out images (e.g., out-of-fold or from the training set) or from independent floristic data, and the Section 6.1 limitation paragraph should acknowledge this transductive estimation.
  2. [§6, Table 3] The sentence in §6 that 'Adding visual-cluster Bayesian priors yields a further +0.004 improvement' reports only the private split. The same variant loses 0.015 on the public split, so the effect is split-dependent and could be noise given the small absolute difference. Please provide a significance test, confidence intervals, or a per-transect breakdown, or temper the claim to describe the prior as an exploratory variant that requires further validation.
  3. [§4.1, §5] The grid size N=4 and the top-K operating point (top-9) appear to be selected after inspecting the reported private/public results in Table 3, but the paper does not state what data were used for model selection. If any of the ablation choices were made using the private leaderboard, the reported private score is optimistic. Please clarify the validation protocol used to choose N, K, and the number of clusters, and whether the final configuration was frozen before private evaluation.
minor comments (6)
  1. [§4.3] Typo: 'Custer 2' should be 'Cluster 2' in the bullet list.
  2. [Table 1] The table header reads 'PlantCLEF 20204'; it should read 'PlantCLEF 2024'.
  3. [§4.2] The geolocation filter computes squared Euclidean distance directly on latitude/longitude coordinates (§4.2); this overweights longitude differences at high latitudes. A note on this approximation or the use of a projected coordinate system would improve reproducibility.
  4. [§4.3] The cluster labels are generated by submitting the most frequent species to ChatGPT; this is not fully reproducible. Please provide the species lists and prompt, or replace the ChatGPT summaries with deterministic ecological descriptions.
  5. [Declaration on Generative AI] The declaration states that no generative AI tools were used for writing the paper, but §4.3 describes using ChatGPT to generate cluster descriptions. Please clarify whether the declaration covers only the prose writing or also the content-generation step.
  6. [Figure 3 caption] The Figure 3 caption describes the pipeline as selecting top-K species by frequency count, but the final pipeline also includes the Bayesian prior reweighting described in §4.3; the caption should be updated to reflect the complete method.

Circularity Check

1 steps flagged · score 6.0 of 10

Cluster-prior 'gain' is an in-sample fit: P(y|c) is computed from the model's own test-set probabilities and then used to reweight those same images; the tiling result remains a valid external benchmark.

  1. fitted input called prediction [Section 4.3 (Visual-Cluster Bayesian prior adaptation), applied in Table 3]
    "The PaCMAP + K-Means step yields, for every cluster 𝑐, an empirical prior distribution 𝑃 (𝑦|𝑐) obtained by averaging the model’s predicted probability vectors across all images in that cluster. During inference, we re-weight each tile’s class probabilities by this prior."

    P(y|c) is computed by averaging the model's own class-probability vectors over the exact test images that are later re-weighted, so the 'prior' contains no independent floristic information. The reweighted predictions are a deterministic transformation of the model's test-set outputs, and the +0.004 private gain (0.34834 vs 0.34420) is an in-sample result on the data used to build the prior. The public split shows the opposite sign (-0.015: 0.29293 vs 0.30810), confirming that the claimed improvement is split-dependent and not a validated prediction. This is fitting a parameter (P(y|c)) to the target of evaluation and reporting the fit as a predictive gain.

full rationale

The paper's core tiling result is an external leaderboard score and is not circular: matching tile size to the ViT receptive field produces a large, reproducible gain (0.3442 private) that is independently benchmarked. The only circularity concern is Section 4.3, where the empirical prior is derived from the model's own test-set probabilities and evaluated on the same test set. This makes the +0.004 prior gain an in-sample estimate rather than a prediction, and Table 3 confirms the instability across public and private splits. No load-bearing self-citations or imported uniqueness theorems are present; self-citations are limited to related work and infrastructure. Therefore, the tiling and leaderboard claim stands, but the cluster-prior contribution is partially circular and not established.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The pipeline depends on the design choices listed above and the modeling assumptions. No new physical or conceptual entities are introduced beyond the empirical clusters, which are groupings of existing images.

free parameters (5)
  • Grid size N = 4
    Empirically chosen in Section 4.1 to match the ViT's 518x518 input resolution; no ablation table for other N is shown, but the text claims smaller or larger grids degrade performance.
  • Top-K predictions per tile = 9 (PRIORS), 10 (GEO)
    Selected via ablation in Section 5; top-9 gives the best tiling baseline private F1 (0.3442), and the prior model uses top-9 while the GEO model uses top-10.
  • Number of K-Means clusters = 3
    Chosen by visual inspection of the PaCMAP projection in Section 4.3; not selected by a quantitative criterion.
  • Georeference point for filtering = (44N, 4E)
    Hand-picked reference point in Southern France in Section 4.2.
  • Allowed countries for geofilter = France, Spain, Italy, Switzerland
    Hand-selected in Section 4.2; species with no observation near those countries are removed.
assumptions (4)
  • domain assumption Averaging the model's predicted probability vectors over test-set images in a visual cluster yields an empirical prior P(y|c) that approximates the true species distribution for that region.
    In Section 4.3, the prior is used to reweight tile probabilities; this assumes the model's softmax outputs are well-calibrated enough that their per-cluster average tracks the ground-truth label distribution, which is untested.
  • domain assumption Each 4x4 tile of a quadrat photo is similar enough to the single-label training images that the fine-tuned ViT's predictions on tiles are meaningful local evidence.
    Section 4.1 relies on this to bridge the single-label to multi-label domain shift; plants cut at tile boundaries can violate the assumption.
  • domain assumption The geolocation metadata and region identifiers in the test set are accurate and reflect the true origin of each image.
    Used in Sections 4.2 and 4.3 to filter species and assign clusters; noisy metadata would misalign priors and filters.
  • standard math K-Means clustering on PaCMAP-reduced embeddings with Euclidean distance produces ecologically meaningful groups.
    Section 4.3 uses K-Means without validating cluster stability or the Euclidean metric on the PaCMAP projection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tile-Based ViT Inference with Visual-Cluster Priors for Zero-Shot Multi-Species Plant Identification." pith.science (2026). https://pith.science/paper/GIDKX5NY

@misc{pith2026250706093,
  author       = {Pith},
  title        = {Pith review of: Tile-Based ViT Inference with Visual-Cluster Priors for Zero-Shot Multi-Species Plant Identification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GIDKX5NY}},
  note         = {Machine review of arXiv:2507.06093}
}
read the original abstract

We describe DS@GT's second-place solution to the PlantCLEF 2025 challenge on multi-species plant identification in vegetation quadrat images. Our pipeline combines (i) a fine-tuned Vision Transformer ViTD2PC24All for patch-level inference, (ii) a 4x4 tiling strategy that aligns patch size with the network's 518x518 receptive field, and (iii) domain-prior adaptation through PaCMAP + K-Means visual clustering and geolocation filtering. Tile predictions are aggregated by majority vote and re-weighted with cluster-specific Bayesian priors, yielding a macro-averaged F1 of 0.348 (private leaderboard) while requiring no additional training. All code, configuration files, and reproducibility scripts are publicly available at https://github.com/dsgt-arc/plantclef-2025.

Figures

Figures reproduced from arXiv: 2507.06093 by the authors.

Figure 1
Figure 1. Single-label training images displaying the following six species: Veronica urticifolia Jacq., Tripleuros￾permum inodorum (L.) Sch.Bip., Geum reptans L., Carex spicata Huds., Oenothera rosea L’Hér. ex Aiton, Lamium bifidum Cirillo [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Subset of twelve test set images showcasing the significant domain shift between different quadrats. 3.3. Fine-tuned models The ViTD2 and ViTD2PC24 models are Vision Transformers (ViTs) pretrained using the DINOv2 Self￾Supervised Learning (SSL) approach on the LVD-142M dataset, which contains 142 million images [8]. These models were fine-tuned on the PlantCLEF 2024 dataset to address plant species identification [3… view at source ↗
Figure 3
Figure 3. Overview of our proposed transfer learning method. We perform a tiling approach on the test set, classify each tile using the ViTD2PC24All model, and aggregate the results by selecting the top-K species based on their frequency count. Our approach leverages the embedding space learned by the ViTD2PC24All model as a generalized feature representation of images, which is used for classification ( [PITH_FULL_IMAGE:fig… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Geolocation of plant species based on their latitude and longitude metadata. Using a reference point (latitude=44, longitude=4) located in the Southwestern European region, we computed the squared Euclidean distance between each plant species geolocation and the refere…
Figure 5
Figure 5. Figure 5: PaCMAP projections of the test set [CLS] token embeddings. 5a Two-dimensional projection of test image embeddings using PaCMAP, with each point colored by its region label (13 regions total). The plot reveals that quadrats from the same region tend to cluster together,…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Decoupled Pipeline with Proposal Reranking and Score Fusion for Positive-Unlabeled Marine Species Detection

    cs.CV 2026-07 conditional novelty 4.0 of 10

    A decoupled find-then-name pipeline with frozen detector proposals, a LoRA-tuned DINOv3 classifier, and weighted geometric score fusion reached 0.1757 mAP and 12th place in FathomNetCLEF 2026's positive-unlabeled, sou...

  2. Multi-Scale ViT Inference with Habitat-Fit Priors and kNN Retrieval for Multi-Species Plant Identification

    cs.CV 2026-07 conditional novelty 4.0 of 10

    A multi-scale DINOv2 tile classifier with habitat-fit geographic/altitude priors and kNN retrieval scored 0.439 macro-F1 (third place) on PlantCLEF 2026.

Reference graph

Works this paper leans on

21 extracted references · 15 canonical work pages · cited by 2 Pith papers

  1. [1]

    Martellucci, H

    G. Martellucci, H. Goëau, P. Bonnet, F. Vinatier, A. Joly, Overview of PlantCLEF 2025: Multi-species plant identification in vegetation quadrat images, in: Working Notes of CLEF 2025 - Conference and Labs of the Evaluation Forum, 2025

    work page 2025

  2. [2]

    Picek, S

    L. Picek, S. Kahl, H. Goëau, L. Adam, T. Larcher, C. Leblanc, M. Servajean, K. Janoušková, J. Matas, V. Čermák, K. Papafitsoros, R. Planqué, W.-P. Vellinga, H. Klinck, T. Denton, J. S. Cañas, G. Martel- lucci, F. Vinatier, P. Bonnet, A. Joly, Overview of lifeclef 2025: Challenges on species presence prediction and identification, and individual animal ide...

    2025

  3. [3]

    Goëau, V

    H. Goëau, V. Espitalier, P. Bonnet, A. Joly, Overview of PlantCLEF 2024: Multi-species plant identification in vegetation plot images, in: Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, 2024

    work page 2024

  4. [4]

    S. Foy, S. McLoughlin, Utilising dinov2 for domain adaptation in vegetation plot analysis, in: Conference and Labs of the Evaluation Forum, 2024

    work page 2024

  5. [5]

    Gustineli, A

    M. Gustineli, A. Miyaguchi, I. Stalter, Multi-label plant species classification with self-supervised vision transformers, arXiv preprint arXiv:2407.06298 (2024)

    arXiv 2024

  6. [6]

    Chulif, H

    S. Chulif, H. A. Ishrat, Y. L. Chang, S. H. Lee, Patch-wise inference using pre-trained vision transformers: Neuon submission to plantclef 2024, in: Conference and Labs of the Evaluation Forum, 2024

    work page 2024

  7. [7]

    Goëau, P

    H. Goëau, P. Bonnet, A. Joly, V. Bakić, J. Barbe, I. Yahiaoui, S. Selmi, J. Carré, D. Barthélémy, N. Boujemaa, et al., Pl@ ntnet mobile app, in: Proceedings of the 21st ACM international conference on Multimedia, 2013, pp. 423–424

    work page 2013

  8. [8]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al., Dinov2: Learning robust visual features without supervision, arXiv preprint arXiv:2304.07193 (2023)

    arXiv 2023

Show all 21 references
  1. [9]

    L. Wu, W. Zhang, T. Jiang, W. Yang, X. Jin, W. Zeng, [cls] token is all you need for zero-shot semantic segmentation, arXiv preprint arXiv:2304.06212 (2023)

    2023 arXiv

  2. [10]

    Y. Wang, H. Huang, C. Rudin, Y. Shaposhnik, Understanding how dimension reduction tools work: An empirical approach to deciphering t-sne, umap, trimap, and pacmap for data visualization, Journal of Machine Learning Research 22 (2021) 1–73. URL: http://jmlr.org/papers/v22/20-1061. html

    2021

  3. [11]

    Leroy, J

    V. Leroy, J. Revaud, T. Lucas, P. Weinzaepfel, Win-win: Training high-resolution vision transform- ers from two windows, arXiv preprint arXiv:2310.00632 (2023)

    2023 arXiv

  4. [12]

    Z. Li, S. F. Bhat, P. Wonka, Patchfusion: An end-to-end tile-based framework for high-resolution monocular metric depth estimation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 10016–10025

    2024

  5. [13]

    B. An, S. Zhu, M.-A. Panaitescu-Liess, C. K. Mummadi, F. Huang, Perceptionclip: Visual classifica- tion by inferring and conditioning on contexts, arXiv preprint arXiv:2308.01313 (2023)

    2023 arXiv

  6. [14]

    Y. Miao, Y. Lei, F. Zhou, Z. Deng, Bayesian exploration of pre-trained models for low-shot image classification, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 23849–23859

    2024

  7. [15]

    Z. Ji, X. Chai, Y. Yu, Z. Zhang, Reweighting and information-guidance networks for few-shot learning, Neurocomputing 423 (2021) 13–23

    2021

  8. [16]

    A. Dede, H. Nunoo-Mensah, E. T. Tchao, A. S. Agbemenu, P. E. Adjei, F. A. Acheampong, J. J. Kponyo, Deep learning for efficient high-resolution image processing: A systematic review, Intelligent Systems with Applications (2025) 200505

    2025

  9. [17]

    Y. Niu, Z. Song, Q. Luo, G. Chen, M. Ma, F. Li, Atmformer: An adaptive token merging vision transformer for remote sensing image scene classification, Remote Sensing 17 (2025) 660

    2025

  10. [18]

    Fayyaz, S

    M. Fayyaz, S. A. Koohpayegani, F. R. Jafari, S. Sengupta, H. R. V. Joze, E. Sommerlade, H. Pirsiavash, J. Gall, Adaptive token sampling for efficient vision transformers, in: European Conference on Computer Vision, Springer, 2022, pp. 396–414

    2022

  11. [19]

    Q. Fan, Q. You, X. Han, Y. Liu, Y. Tao, H. Huang, R. He, H. Yang, Vitar: Vision transformer with any resolution, arXiv preprint arXiv:2403.18361 (2024)

    2024 arXiv

  12. [20]

    Hussain, M

    W. Hussain, M. F. Mushtaq, M. Shahroz, U. Akram, E. S. Ghith, M. Tlija, T.-h. Kim, I. Ashraf, Ensemble genetic and cnn model-based image classification by enhancing hyperparameter tuning, Scientific Reports 15 (2025) 1003

    2025

  13. [21]

    URL: http://www

    PACE, Partnership for an Advanced Computing Environment (PACE), 2017. URL: http://www. pace.gatech.edu

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.