{"id":"e6398f96-59d7-49d3-a804-37bc305d3644","arxiv_id":"2508.10457","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A frozen DINOv2 transformer with species, genus, and family heads, multi-scale tiling, and threshold tuning placed 3rd in the PlantCLEF 2025 multi-label plant identification benchmark.","lead":"A computer vision system trained on single-plant photos was adapted to list all plant species visible in photos of small vegetation plots. It ranked 3rd in the 2025 PlantCLEF challenge with a private-test score of 0.33655.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Internal contradiction: §3.3 says ViT-L training is infeasible (>1500 h), yet Table 2's 3rd-place submission (0.33655) includes 'Vitlarge' — the described pipeline is not reproducible as written.","rationale":"The paper is a competition notebook whose central claim is a leaderboard rank. That claim is externally verifiable, but the paper's internal description of the methodology is what allows readers to assess and reuse it. I find a direct internal contradiction far more load-bearing than the frozen-backbone transfer assumption: §3.3 (DINOv2 ViT-L) explicitly argues ViT-L is infeasible, yet Table 2 lists 'Vitlarge' as one of the three model components in the selected submission that scored 0.33655 and placed 3rd. The paper offers no explanation. This is not a matter of external consensus or domain shift; it is an inconsistency within the manuscript's own evidence. The transfer assumption is a plausible external risk, but the paper's own public/private gap and the abandoned LUCAS adaptation already show the authors are aware of it. The ViT-L discrepancy is concrete and resolvable by inspecting the code. It affects the reproducibility of the reported 3rd-place result. I therefore disagree with the reader's choice of weakest assumption: while the frozen-backbone transfer is fragile, the ViT-L contradiction is more specific and more damaging. Because the manuscript cannot be faithfully reproduced as written, I recommend UNVERDICTED pending clarification.","tokens_in":8741,"tokens_out":6077,"duration_ms":56917,"concrete_test":"Inspect the public GitHub repository (tag v1.0.0) for a ViT-L checkpoint, config, or training logs. Run the inference script to see whether 'Vitlarge' is instantiated and what weights it loads. If no ViT-L artifacts exist, the Table 2 row is erroneous and the method description is false. If a ViT-L checkpoint exists, compare its training time/compute with §3.3's infeasibility claim; if it wasn't trained here, identify its provenance. This check resolves whether the reported pipeline actually produced the 0.33655 score.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim (3rd place, 0.33655, Table 2) rests on the selected submission 'Hydra + 5h1l + Vitlarge'. However, §3.3 (DINOv2 ViT-L) states that ViT-L training requires ~30 h/iteration × 50 iterations = >1500 h, 'rendering this approach infeasible within the project's resource constraints.' This directly contradicts the presence of a ViT-L model in the final ensemble. The paper gives no explanation of how this model was trained or obtained. If ViT-L was not actually used, the description of the winning submission is inaccurate; if it was used, the feasibility argument is false. Either way, the method section cannot be relied upon to reproduce the result. Additionally, no single-head baseline is reported, and the selected private score (0.33655) is below five of the authors' own unselected submissions (e.g., 0.34575), indicating the final model choice was not the best available—so the '3rd best' claim is not robust evidence of the method's effectiveness.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a multi-label plant species prediction pipeline for the PlantCLEF 2025 challenge. The authors use a frozen DINOv2 ViT-B/14 backbone provided by the organizers, equipped with separate MLP heads for species, genus, and family prediction. Taxonomic probabilities are multiplied and invalid triples are discarded. Inference combines multi-scale tiling, top-n/max-length constraints, dynamic logit thresholding, and ensemble strategies (Hydra and bagging). The system is reported to have achieved macro-F1 0.33655 on the private leaderboard, placing third, and 0.37555 on the public leaderboard, placing second. The paper includes a detailed description of the data, preprocessing, training setup, and inference heuristics, as well as a critical discussion of methods that did not work (e.g., LUCAS pre-training, plant/non-plant filtering, kernel smoothing).","tokens_in":9029,"tokens_out":4854,"duration_ms":56359,"significance":"If the reported leaderboard results are accurate, the paper demonstrates a competitive, externally validated pipeline for a difficult domain-shift task: training on single-species images while testing on multi-species vegetation plots. The result is directly supported by public leaderboard records, and the authors provide a public code repository. The paper is honest about negative results and does not overclaim local validation. However, the technical novelty is modest; the main contributions are a combination of known components (multi-head taxonomic classification, tiling, thresholding, ensembling) rather than a new architecture or algorithm. The value of the paper lies primarily in its reproducible engineering and in the detailed reporting of what did and did not work in a constrained competition setting.","major_comments":[{"comment":"There is a direct contradiction in the description of the submitted pipeline. §3.3 states that training a DINOv2 ViT-L/14 model requires approximately 30 h per iteration and more than 1,500 h total, 'rendering this approach infeasible within the project's resource constraints.' Yet Table 2 lists a selected submission containing 'Vitlarge' as one of the five chosen submissions, and this is the submission with private score 0.33655 reported as the 3rd-place result. The paper gives no explanation of how the ViT-L model was trained, whether it was a pretrained backbone with only the heads trained, or whether the label in the table denotes something else. This is load-bearing: the central reproduction claim requires the exact composition of the submitted ensemble to be unambiguous. Please either provide the missing training details for ViT-L or correct Table 2 and the surrounding text.","section":"§3.3 and Table 2"},{"comment":"The paper states, 'Our primary multi-head classification approach achieved a substantial improvement over the baseline, which relied on simple single-head plant species classification,' and claims that Table 2 highlights this improvement. However, Table 2 contains no single-head baseline row at all; every listed submission uses multi-head species/genus/family prediction. Without a single-head control under otherwise matched configurations, the paper's central component claim about multi-head taxonomic fusion is not supported by any reported evidence. This is not a request for exhaustive ablations, but a baseline for the headline claim is essential in a methods paper.","section":"§4, Table 2"},{"comment":"Most component-level claims (cropping percentage, tiling scales, top-n filtering, dynamic thresholding, bagging) are supported only by comparing full submissions on the public and private leaderboards, and the paper itself acknowledges that the public/private sets are imbalanced and that leaderboard-driven optimization likely led to public-set overfitting. For example, §3.2 says 10% cropping was best on public and 5% best on private, but Table 2 does not provide a controlled comparison between these two variants. The contribution list in the abstract thus rests on observations that are partly self-confirmatory: the same public scores used to select hyperparameters are later cited as evidence of their value. Given the small differences among private scores (e.g., 0.34575 vs. 0.33655) and the absence of error bars, the paper should temper its causal language and clearly separate 'what work","section":"§3.2, §3.4, §4"}],"minor_comments":[{"comment":"The transect-averaged macro-F1 formula is not displayed cleanly in the text. Please ensure it renders correctly and define all subscripts explicitly (N, T_i) in the caption or surrounding text.","section":"Equation (1)"},{"comment":"The table formatting is difficult to parse: the model column contains multi-line entries such as 'Hydra 5h1l Vitlarge', and the columns 'min', 'mean', 'max', 'crop %' are not clearly explained in the caption. In particular, 'crop %' values like '8,10,12' and '1,2,4,5' need a precise definition (does 8,10,12 denote three separate crop experiments?).","section":"Table 2"},{"comment":"The sentence 'The 10% cropping strategy yielded the best results on the public leaderboard, while the 5% strategy was more effective on the private one' is stated as a conclusion, but Table 2 does not provide a direct 10% vs. 5% comparison under the same model. Consider presenting such a comparison explicitly, or state that this is an informal observation.","section":"§3.2"},{"comment":"Reference [4] appears to be the ViT paper 'An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale', but the author list is incorrectly ordered/incomplete (Kolesnikov is listed first, and the actual citation style is non-standard for ICLR). Please correct the citation.","section":"References"},{"comment":"The abstract emphasizes 'metadata-enhanced' and 'metadata' in the title, but the metadata merging method described in §3.4 was not used in the final pipeline. Clarify that the 'metadata' used is the taxonomic labels (species/genus/family) at training time, not additional test-time metadata, to avoid misleading readers.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The paper is a competition working note, and the externally verifiable leaderboard position is a meaningful positive. However, the ViT-L contradiction is substantial because it directly affects the reproducibility of the reported 3rd-place submission; editors should require a clear explanation or correction. The missing single-head baseline is also necessary to support the paper's central methodological claim. With these addressed, the paper could be acceptable as a working-notes contribution. I would suggest the authors also add a short 'limitations' statement about the lack of local validation and the risk of leaderboard overfitting, which is currently only implicit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a clear, honest competition write-up. The authors report what worked and what didn't, and the central score (0.33655, 3rd on the PlantCLEF 2025 private leaderboard) is externally verifiable. The small new idea—dynamically setting the logit threshold via bisection on the mean prediction length—is neat and plausibly useful. The multi-head taxonomic fusion is a sensible extension of existing hierarchical-classification ideas, and the code is available. Credit where due: the paper does not oversell itself; it openly discusses public/private overfitting and abandoned ideas like LUCAS pre-training and the plant/non-plant filter.\n\nBut the soft spots are real, and one is load-bearing. Section 3.3 states that training a ViT-L backbone is infeasible because a single iteration takes ~30 hours and 50 iterations would exceed 1,500 hours, 'rendering this approach infeasible within the project's resource constraints.' Yet Table 2 lists a selected submission, the one that produced the paper's headline 3rd-place score, as 'Hydra + 5h1l + Vitlarge.' No explanation is given. That is an internal contradiction: either ViT-L was used (and the feasibility argument is false) or it wasn't (and the method description is wrong). Either way, the method as written cannot be reproduced.\n\nA second issue: the reported 3rd-place score of 0.33655 is not the best score among the authors' own selected submissions. Their selected list includes a private score of 0.33947, and their top-5 private submissions go up to 0.34575. The paper never explains why 0.33655 is the headline number. It may be that the team had to choose one final submission for the leaderboard and chose this one for diversity, but the paper doesn't say that. As written, it makes the '3rd best' claim look arbitrary and undercuts the significance of that specific result.\n\nAlso, there are no controlled ablations and no error bars. All design choices were tuned on the public leaderboard, and the missing single-head baseline means the claimed improvement from multi-head classification is not actually demonstrated in the table.\n\nThe reader's conditional verdict is fair, and the stress-test note correctly catches the ViT-L problem. This is not a paper with fraudulent results—the leaderboard score is public and credible—but the method section has a serious unresolved inconsistency. A good referee would force the authors to clarify what was actually run. If they fix that, this is a useful working note for the PlantCLEF community; if they can't, it's not.\n\nRecommendation: send it to peer review, but flag the ViT-L issue in your comments. It deserves referee time because the competition result is real and the flaw is fixable, but it needs major revision before it can be trusted.","headline":"Candid competition note with an externally real 3rd-place score, but the paper contradicts itself on whether ViT-L was used, and the reported winning score isn't their own best selected submission.","tokens_in":9501,"tokens_out":3683,"would_cite":false,"duration_ms":37810,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A frozen DINOv2 backbone with species, genus, and family heads fusing probabilities reaches third place in PlantCLEF 2025 with macro-F1 0.33655.","keywords":["multi-label classification","plant species identification","DINOv2","vision transformer","taxonomic hierarchy","multi-scale tiling","vegetation plot images","PlantCLEF 2025"],"falsifier":"Train the same multi-head, tiling, and thresholding pipeline on a backbone that has been continually pre-trained on the LUCAS quadrat-like images, and compare its private macro-F1 to the frozen backbone's 0.33655; if the adapted backbone does not beat the frozen one, the score is carried by the taxonomic heads and thresholds, not by transfer of the frozen embeddings.","tokens_in":8620,"feed_emoji":"🌿","tokens_out":8263,"duration_ms":82765,"temperature":0.7,"pith_summary":"The paper attempts to show that multi-label plant-species recognition in dense vegetation-plot images can be handled without training a new visual backbone. It uses a frozen DINOv2 ViT-B/14 encoder pretrained on single-species images, with separate classification heads for species, genus, and family whose probabilities are multiplied so only taxonomically valid combinations survive. On the PlantCLEF 2025 private leaderboard this pipeline reaches macro-F1 0.33655, ranking third, and the authors identify multi-scale tiling and dynamic thresholding as the key inference ingredients. The result matters because vegetation-plot inventories are slow expert work, and a pipeline built from a frozen backbone plus lightweight heads is far cheaper to deploy than a fully trained multi-species model. At the same time, every submitted model scored lower on the private than on the public set, so the paper's own evidence shows the single-to-multi-species domain shift is only partially bridged.","feed_headline":"Frozen DINOv2 ranks 3rd in PlantCLEF 2025","feed_subtitle":"Multiple taxonomic heads share one frozen encoder to identify every plant in a plot photo.","key_machinery":"Multi-head taxonomic fusion over a frozen DINOv2 ViT-B/14 backbone. The frozen encoder produces one embedding; separate lightweight heads predict species, genus, and family; the probabilities are multiplied and impossible taxonomic combinations are masked out. Multi-scale tiling (splitting each quadrat into non-overlapping grids of size 4 and 5) lets the same encoder see plants at different scales, while a bisection search sets a global logit threshold so each image gets about four species predictions.","core_discovery":"The central claim is that taxonomic hierarchy can be injected into a frozen self-supervised vision transformer to turn a single-species classifier into a competitive multi-species presence predictor. From one DINOv2 embedding, the model predicts species, genus, and family, multiplies the three probability vectors, and discards combinations absent from the training metadata; this fusion moves it past the single-head baseline. With multi-scale tiling, dynamic logit thresholds (about four species per image), and bagged/Hydra ensembles, the method reports private macro-F1 0.33655, third place. The paper also documents a consistent public-to-private score drop, which it reads as domain-shift sens","pith_inferences":["If the frozen-backbone transfer is the real driver, continually pre-training the same ViT on the LUCAS quadrat-like images would be a direct test: a backbone adapted to plot imagery should beat the frozen one on the private split, and failing to do so would indicate the heads and thresholds, not the backbone, carry the method.","The probability-multiplication trick is a form of hierarchical constraint decoding that should transfer to other strongly hierarchical domains, such as fungi or insect identification, where single-species training sets and multi-species field images coexist.","A cheap local validation set built from LUCAS images with pseudo-labels would let future teams do model selection without public-leaderboard overfitting, directly addressing the public-to-private drop the authors observed."],"forward_implications":["Vegetation-plot inventories could be partially automated without training a new backbone; a frozen encoder plus lightweight taxonomic heads is the deployable unit.","Taxonomic probability multiplication is a transferable post-processing layer: any single-species classifier with genus and family outputs can be adapted to multi-label presence prediction.","Public-leaderboard tuning is risky: all five selected submissions scored lower on the private set, so competition-style model selection needs validation data drawn from the quadrat distribution.","Strategies like plant/non-plant filtering, metadata merging, and kernel smoothing were tried and did not help under this pipeline, so future systems can focus effort on tiling and thresholding.","Multi-scale tiling appears to absorb the benefit of kernel-based smoothing, since plants spanning tile boundaries are already seen at multiple resolutions."],"supporting_citations":[{"why":"Defines the PlantCLEF 2025 challenge: single-species training images, multi-species quadrat test images, and the leaderboard.","marker":"[1, 2, 3]"},{"why":"Supplies the DINOv2 self-supervised method and the exact organizer-provided ViT-B/14 weights used frozen.","marker":"[5, 8]"},{"why":"Defines the transect-averaged macro-F1 metric and the prior PlantCLEF 2024 task this work extends.","marker":"[6, 7]"},{"why":"Adds registers to the ViT-B/14 backbone, the specific architecture variant whose embeddings are used.","marker":"[9]"},{"why":"Records PlantCLEF 2024 evidence that ViT-B architectures perform well on this data, motivating the backbone choice.","marker":"[6, 10, 11, 12]"},{"why":"Supplies the LUCAS unannotated quadrat-like images that would have been used for domain adaptation, abandoned for hardware reasons.","marker":"[13]"},{"why":"Provides the bagging ensemble method used to average logits across trained models.","marker":"[14]"}],"fun_headline_variants":["Frozen DINOv2 with taxonomic heads ranks 3rd in PlantCLEF 2025","Taxonomic hierarchy fuses probabilities for multi-species plant prediction","One frozen ViT, three heads: species, genus, family from plot photos","Multi-head transformer leverages metadata for 3rd place in PlantCLEF 2025","Frozen DINOv2 plus taxonomic heads predicts multiple plants per image"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that a frozen DINOv2 backbone trained on single-species images produces embeddings from dense multi-species quadrat images that are informative enough for linear heads trained on single-species crops to recognize the species present.","fun_headline_variants_meta":{"raw":{"variants":["Frozen DINOv2 with taxonomic heads ranks 3rd in PlantCLEF 2025","Taxonomic hierarchy fuses probabilities for multi-species plant prediction","One frozen ViT, three heads: species, genus, family from plot photos","Multi-head transformer leverages metadata for 3rd place in PlantCLEF 2025","Frozen DINOv2 plus taxonomic heads predicts multiple plants per image"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000242,"raw_usage":{"total_tokens":1353,"prompt_tokens":728,"completion_tokens":625,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":518}},"tokens_in":472,"tokens_out":625,"duration_ms":6251,"temperature":1.0,"reasoning_tokens":518,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T20:25:49.110017+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same multi-head, tiling, and thresholding pipeline on a backbone that has been continually pre-trained on the LUCAS quadrat-like images, and compare its private macro-F1 to the frozen backbone's 0.33655; if the adapted backbone does not beat the frozen one, the score is carried by the taxonomic heads and thresholds, not by transfer of the frozen embeddings.","supporting_citations":[{"cited_title":"Darcet, M","cited_arxiv_id":null,"evidence_quote":"Adds registers to the ViT-B/14 backbone, the specific architecture variant whose embeddings are used."},{"cited_title":"d’Andrimont, M","cited_arxiv_id":null,"evidence_quote":"Supplies the LUCAS unannotated quadrat-like images that would have been used for domain adaptation, abandoned for hardware reasons."}],"review_version":1}