{"id":"6b9c7a1a-a9b8-46d0-9842-b655024ecf01","arxiv_id":"2504.18286","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A comparative study reports that continuously expanding the gallery of known pallet images raises long-term re-identification accuracy, and introduces the pallet-block-2696 aged-pallet dataset.","lead":"The authors test whether synthetic images and regularly refreshing the reference gallery keep a warehouse pallet recognition system accurate as real pallets age and suffer damage. They release a new open dataset of 2,696 pallet-block photos taken over four months and report that updating the gallery improves mean Rank-1 accuracy by 24 percentage points.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 13% synthetic-data gain is confounded: A and R models differ in real training corpora, so the Rank-1 gap cannot be attributed to synthetic images without a matched retraining ablation.","rationale":"The reader's weakest_assumption correctly identifies the A-vs-R confound, and I agree that it is the load-bearing issue. The gallery-updating experiment is well-defined: T01 versus T00 mean Rank-1 (0.72 vs 0.48) survives even when excluding T01's extra day-2 query (mean approximately 0.72). The new dataset is a useful contribution, and the qualitative comparison of synthetic vs real aging is explicitly framed as validation evidence rather than as the primary claim. The only place where the paper's headline could be wrong is the synthetic-data comparison, because the design does not isolate synthetic images as the causal factor. A matched retraining ablation would settle this. I would not reject the paper: the confound is removable, the gallery claim stands independently, and the code and data are released. The reader's CONDITIONAL verdict is therefore appropriate, and my stress test does not change it.","tokens_in":9770,"tokens_out":7438,"duration_ms":71432,"concrete_test":"Retrain PCB-R (and ideally ResNet/OSNet) on exactly the same 206,306 real images and 62,963 IDs used for the A models, with synthetic images excluded; train a second model on the same real subset plus the 21,255 synthetic images, repeating with at least three seeds. Evaluate both on pallet-block-2696 under T00/T01/T02. If the T01 Rank-1 gap stays near 13 points across seeds, the synthetic-data claim is supported; if it collapses, the original comparison was confounded by the real-data composition. Also verify that no pallet-block-2696 IDs appear in the pretraining datasets.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The decisive weakness is in the synthetic-data comparison, not in the gallery-updating experiment. Section III-C defines A as pretrained on 206,306 real images from 62,963 IDs plus 21,255 synthetic images from 1,635 IDs, while R is pretrained on 330,089 real images from 102,068 IDs with no synthetic data. The reported PCB Rank-1 advantage in T01 (0.72 vs 0.59) and the 'up to 13%' claim therefore conflates two changes: the addition of synthetic images and a different, smaller real corpus with fewer IDs. If the A real corpus is more similar to pallet-block-2696, the gap could appear even if synthetic images contribute nothing. No error bars or repeated-seed results are given, so run-to-run noise is also unquantified. The 24% gallery-updating result uses only the PCB A model and compares T01 with T00; that claim is not affected by this confound, but the synthetic-data conclusion in the abstract and Section V is.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces pallet-block-2696, a new re-identification dataset of Euro pallet blocks photographed over four months with natural aging and induced mechanical damage, and evaluates three gallery-updating strategies (fixed T00, expanding T01, and latest-image T02) using three pretrained architectures (ResNet50, PCB, OSNet) trained with and without synthetic data. The authors report a 24 percentage-point increase in mean Rank-1 accuracy for the expanding gallery (T01) over the fixed gallery (T00) using the PCB-A model, and up to 13 percentage-point Rank-1 gains for models trained with additional synthetic data (A) over real-only models (R). The dataset, models, and code are publicly released.","tokens_in":9970,"tokens_out":6305,"duration_ms":55643,"significance":"The dataset and the gallery-updating experiment are valuable and well aligned with an important operational problem: re-identification of objects whose appearance changes over time. The T00/T01/T02 comparisons are clean for the PCB-A model and support the claim that periodically updating the gallery improves long-term re-identification. The public release of dataset, models, and code is a strength, and the qualitative comparison of real versus synthetic aging provides a starting point for future work. However, the synthetic-data comparison is currently not supported because the A and R model families are not trained on matched real data; a controlled retraining ablation is needed before the 13% claim can be attributed to synthetic data.","major_comments":[{"comment":"The comparison between models trained with additional synthetic data (A) and models trained only on real data (R) does not isolate the effect of synthetic data. In Section III-C, A models use 206,306 real images from 62,963 IDs plus 21,255 synthetic images, while R models use 330,089 real images from 102,068 IDs. The reported Rank-1 advantage of PCB A over PCB R in T01 (0.72 vs 0.59) and the 'up to 13%' claim in the abstract and conclusion therefore conflate the addition of synthetic images with a reduction in the size of the real training corpus and a change in the number of IDs. A controlled ablation, retraining the models on matched real data with and without the synthetic images, is necessary to support the synthetic-data claim.","section":"Section III-C and Table IV"},{"comment":"The headline 'up to 13%' improvement is based on a single model pair (PCB A vs PCB R in T01) without any measure of variance, bootstrap confidence intervals, or repeated training runs. Since the pretrained A and R models come from different previous publications, the observed difference could also reflect differences in the training procedure, hyperparameters, or random initialization rather than the data recipe. The authors should either provide repeated-seed experiments or explicitly frame the result as a point estimate from two pretrained model families rather than a statistically validated improvement.","section":"Abstract and Section V"}],"minor_comments":[{"comment":"The abstract states 'trained with 10% artificial training data'; with 21,255 synthetic images among 206,306+21,255 total images, the synthetic fraction is approximately 9.3%, while 21,255/206,306 is approximately 10.3%. Please clarify the intended definition of the 10% figure.","section":"Abstract"},{"comment":"The '24%' increase in mean Rank-1 accuracy is a change in percentage points, not a relative improvement; consider writing '24 percentage points' to avoid ambiguity.","section":"Abstract and Section V"},{"comment":"The means for T00 and T01 are computed over different query sets (13 vs 14 recording days, respectively). Reporting the mean over the common days 03-14a in addition would make the comparison cleaner, although the direction and approximate magnitude of the improvement appear to remain.","section":"Section IV-B, Tables I and II"},{"comment":"The statement that the qualitative comparison 'validating that the data synthesis process yielded realistic results' is an overstatement; the comparison is visual and the synthetic process comes from the same group. It is a useful qualitative observation but not a quantitative validation.","section":"Section IV-A2"},{"comment":"The caption reads 'Overview of the µ mean Average Precision'; the µ symbol is redundant and should be removed.","section":"Table IV caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study for the gallery-updating part, and the dataset release is a useful contribution. The main risk is that the synthetic-data claim is inherited from previous work without a matched ablation; the authors should either retrain matched models or substantially weaken the claim to state that models trained on a different, smaller real corpus plus synthetic data outperformed models trained on a larger real-only corpus. I would not reject because the core gallery experiment is sound and the dataset is likely to be reused by the community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the pallet-block-2696 dataset and the gallery-updating comparison are worth taking seriously; the synthetic-data claim is not yet supported. The paper releases a genuinely new open dataset of 2,696 images of Euro pallet blocks photographed over four months with natural aging and induced damage. That fills a real gap — there is no other public benchmark for long-term re-identification of industrial wooden objects. The three gallery strategies (fixed, expanding, and rolling one-day) are clearly defined, and the comparison on the PCB A model is internally consistent: T01 raises mean Rank-1 from 0.48 (T00) to 0.72, which matches the advertised 24-point gain, and T02 gives higher mAP but lower Rank-1. The qualitative side-by-side of real versus synthetic aging in Figure 3 is useful and shows the synthetic process captures branding decay and surface change, though it misses moss and mechanical damage. Credit also for releasing code and data.\n\nThe weak spot is the \"up to 13%\" synthetic-data result. The A and R models are not matched: A trains on 206,306 real images from 62,963 IDs plus 21,255 synthetic images; R trains on 330,089 real images from 102,068 IDs with no synthetic data. So the Rank-1 gap in Table IV conflates synthetic data with a different, smaller real corpus. Without an ablation that holds the real data fixed, the synthetic contribution cannot be isolated. The paper does not explicitly acknowledge this confound, and the abstract states the synthetic claim as a finding. Also, there are no error bars or repeated runs, so run-to-run noise is unquantified. The gallery-updating conclusion does not depend on that confound, but the synthetic-data conclusion does.\n\nThe qualitative validation of synthetic aging is partially self-referential — the synthetic images come from the same group's prior GAN, and the comparison is visual only — but that is a minor issue, not a fatal one.\n\nWho this is for: anyone working on re-identification under appearance change, especially industrial or logistics settings. The paper is worth refereeing because the dataset and the gallery-updating benchmark are useful contributions, even if the synthetic-data analysis needs a matched retraining experiment before the claim can stand. I would ask the authors to rerun or report a controlled A-vs-R comparison on identical real data, add seed-level variance, and soften the abstract. That is a heavy revision, not a desk reject.","headline":"A useful open dataset and a clean gallery-updating experiment, but the synthetic-data payoff claim is confounded by unmatched training corpora.","tokens_in":10511,"tokens_out":2054,"would_cite":false,"duration_ms":18956,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A rolling gallery that absorbs each new recording day keeps pallet re-identification accurate as the pallets age.","keywords":["re-identification","synthetic training data","gallery updating","material aging","pallet-block-2696","GAN-generated aging","Rank-1 accuracy","logistics computer vision"],"falsifier":"Train the same PCB model twice on the same 206,306 real images, once with and once without the 21,255 synthetic images, and evaluate both on pallet-block-2696 under the T01 gallery; if the Rank-1 gap is smaller than the reported 13 points, the synthetic-data advantage is not isolated as claimed.","tokens_in":9601,"feed_emoji":"🪵","tokens_out":11784,"duration_ms":107919,"temperature":0.7,"pith_summary":"The paper claims that re-identification of industrial objects must be treated as a time-varying problem: as a physical object wears, its appearance drifts away from any fixed reference image. On a new dataset of 2,696 Euro pallet block images captured over four months of natural weathering plus induced mechanical damage, it shows that keeping the reference gallery fixed loses the object over time, with per-day Rank-1 accuracy (the share of queries whose top match is correct) dropping from 0.97 to 0.11, while a rolling gallery that adds each recording day's images holds mean Rank-1 at 0.72—a 24-percentage-point gain over the fixed baseline. It further claims that models pretrained with about 10 percent synthetic images outperform models pretrained on real data alone by up to 13 percentage points Rank-1. The work contributes the open pallet-block-2696 dataset, which also allows a qualitative check of synthetic aging: synthetic data reflects loss of branding and surface change, but not moss buildup or mechanical damage.","feed_headline":"Rolling gallery lifts pallet re-identification accuracy by 24","feed_subtitle":"On a new 2,696-image dataset of Euro pallet blocks, the rolling gallery beats a fixed one by 24 percentage points.","key_machinery":"The central mechanism is the gallery-update strategy, benchmarked in three configurations: T00 (fixed gallery from the first two recording days), T01 (rolling gallery that grows by one recording day at a time), and T02 (gallery holding only the most recent day). The paper uses these to isolate what the reference set contributes over time. The companion object is pallet-block-2696, 2,696 images of 60 pallet-block IDs photographed from three perspectives on 14 days over four months, with natural weathering and an artificially damaged final session. The models compared are ResNet50, PCB, and OSNet, each pretrained either on real images only or on real images plus roughly 10 percent GAN-generated synthetic aged images.","core_discovery":"The core discovery is that a continuously updated gallery is what preserves identity under material aging, not a better feature extractor alone. In the rolling experiment T01, the gallery is expanded with every previous recording day and tested against the next day; mean Rank-1 accuracy reaches 0.72, and even the deliberately damaged final-day images are identified at Rank-1 0.75, whereas the fixed gallery T00 drops to 0.11 on the same damaged images. The models are not retrained on the new dataset, so the results measure generalization to previously unseen aging rather than adaptation. The paper also reports that synthetic-augmented models beat real-only models consistently except for OSNet's mean Average Precision, with the largest Rank-1 gap being 13 percentage points for the PCB model in the rolling setup. On the visual side, the real aging data show four distinct processes—branding loss, surface-structure change, moss buildup, and mechanical damage—of which the synthetic generator from prior work reproduces the first two.","pith_inferences":["A controlled ablation on the identical real corpus—with and without the 21,255 synthetic images—would be needed to confirm the 13-point synthetic-data gain, because the A and R models in the paper were pretrained on different real datasets.","The rolling gallery grows in size as well as recency, so a follow-up that adds only one newest image per ID per day could separate the benefit of more reference views from the benefit of recent appearance.","The same gallery-update logic should transfer to other slowly changing identities, such as clothing-change person re-identification or machine parts subject to wear; the dataset provides a template for controlled long-horizon evaluation.","Quantitative comparison of synthetic versus real aged images with distribution metrics could turn the qualitative observation about missing moss and mechanical damage into a direct evaluation target for the GANs."],"forward_implications":["With a fixed gallery, per-day Rank-1 on the new dataset falls from 0.97 on the third recording day to 0.11 on the damaged final day, so any long-term re-identification deployment needs gallery refresh.","A rolling gallery raises mean Rank-1 from 0.48 to 0.72 over the fixed baseline and keeps damaged-object identification at 0.75 Rank-1.","Synthetic-augmented training adds up to 13 percentage points Rank-1 over real-only training for PCB; the same direction holds for ResNet, while OSNet does not show an mAP gain from synthetic data.","Most-recent-day gallery (T02) gives the highest mAP, 0.61 for PCB A, but lower Rank-k accuracy than the rolling gallery, so the update strategy should depend on whether one or several matches are needed.","The released dataset gives other researchers a real benchmark for material aging and a target for validating GAN-generated aged images."],"supporting_citations":[{"why":"Supplies the synthetic aging generation method whose images are used in the A models and whose realism is qualitatively checked against real aging.","marker":"[5]"},{"why":"Provides the ResNet50 architecture evaluated as a re-identification backbone.","marker":"[7]"},{"why":"Provides the PCB part-based architecture that achieves the best Rank-1 results in the experiments.","marker":"[9]"},{"why":"Provides the OSNet architecture evaluated as a lightweight omni-scale re-identification model.","marker":"[10]"},{"why":"Earlier evidence that synthetic pallet images can augment training data while preserving per-ID characteristics.","marker":"[17]"},{"why":"One of the real pallet-block corpora used to pretrain the models, defining part of the real-data side of the A and R comparisons.","marker":"[19]"},{"why":"The other real pallet-block corpus used for pretraining; together with [19] it defines the real-only training set for the R models and part of the real set for the A models.","marker":"[20]"},{"why":"The new pallet-block-2696 dataset release, including images, code, and models, on which all experiments are run.","marker":"[21]"}],"fun_headline_variants":["Rolling gallery lifts pallet re-ID accuracy 24%","Aging pallets re-ID improved 24% by rolling gallery","Continuous gallery updates beat fixed set on aging pallets","Synthetic data plus rolling gallery boost long-term re-ID","Rolling gallery beats fixed one by 24% on aging pallets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The synthetic-data result assumes that the real-only and synthetic-augmented models differ only by the synthetic images, but the two model sets were pretrained on different real-image collections (206,306 real images for A versus 330,089 for R), so the reported 13-point gap might come from the real-data composition instead.","fun_headline_variants_meta":{"raw":{"variants":["Rolling gallery lifts pallet re-ID accuracy 24%","Aging pallets re-ID improved 24% by rolling gallery","Continuous gallery updates beat fixed set on aging pallets","Synthetic data plus rolling gallery boost long-term re-ID","Rolling gallery beats fixed one by 24% on aging pallets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000705,"raw_usage":{"total_tokens":3187,"prompt_tokens":959,"completion_tokens":2228,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":2142}},"tokens_in":575,"tokens_out":2228,"duration_ms":14533,"temperature":1.0,"reasoning_tokens":2142,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:19:54.239306+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same PCB model twice on the same 206,306 real images, once with and once without the 21,255 synthetic images, and evaluate both on pallet-block-2696 under the T01 gallery; if the Rank-1 gap is smaller than the reported 13 points, the synthetic-data advantage is not isolated as claimed.","supporting_citations":[{"cited_title":"Re- identification Robustness Over Time - The Case for Synthetic Training Data,","cited_arxiv_id":null,"evidence_quote":"Supplies the synthetic aging generation method whose images are used in the A models and whose realism is qualitatively checked against real aging."},{"cited_title":"Beyond Part Models: Person Retrieval with Refined Part Pooling (And a Strong Convolutional Baseline),","cited_arxiv_id":null,"evidence_quote":"Provides the PCB part-based architecture that achieves the best Rank-1 results in the experiments."},{"cited_title":"Omni-scale Feature Learning for Person Re-identification,","cited_arxiv_id":null,"evidence_quote":"Provides the OSNet architecture evaluated as a lightweight omni-scale re-identification model."},{"cited_title":"On the Applicability of Synthetic Data for Re- Identification,","cited_arxiv_id":null,"evidence_quote":"Earlier evidence that synthetic pallet images can augment training data while preserving per-ID characteristics."},{"cited_title":"pallet- block-98382 3270,","cited_arxiv_id":null,"evidence_quote":"One of the real pallet-block corpora used to pretrain the models, defining part of the real-data side of the A and R comparisons."},{"cited_title":"pallet-block-32965 – A Chipwood Re-Identification Dataset,","cited_arxiv_id":null,"evidence_quote":"The other real pallet-block corpus used for pretraining; together with [19] it defines the real-only training set for the R models and part of the real set for the A models."},{"cited_title":"pallet-block-2696,","cited_arxiv_id":null,"evidence_quote":"The new pallet-block-2696 dataset release, including images, code, and models, on which all experiments are run."}],"review_version":1}