{"id":"8750163e-2fbf-42d4-93c3-71238ff29600","arxiv_id":"2501.01710","paper_version":2,"verdict":"REJECT","confidence":"LOW","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A frozen LVM with an ASSP head is trained using a loss built from ground-truth-guided logit trajectories (POTGui), reportedly reaching 99.99 mIoU on Cityscapes, a result that is almost certainly invalid.","lead":"A driving-perception paper freezes a large pretrained vision model, adds a small segmentation head, and trains it with a loss that uses ground-truth labels to simulate future optimization steps. It reports near-perfect segmentation scores and six times faster convergence, but the numbers are implausible and the evaluation protocol is undefined.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never states whether POTGen (Eqs. 4–12), which requires ground-truth labels P_Y, is active at inference; if it is, the 99.99 mIoU is label leakage, and if it is not, the reported metric is unexplained.","rationale":"The reader's verdict of REJECT is appropriate, and the weakest_assumption identified is the same protocol gap I find most load-bearing. The strongest claim is empirical; its support is Table II and Fig. 4. The single most load-bearing condition is that the evaluated model does not receive ground-truth labels. That condition is never stated. The method's only label-free forward path is Eq. (2); everything after it in Sections III-B and III-C is constructed from P_Y and hence cannot be part of a valid test-time prediction unless the protocol says otherwise. I do not base the concern on the implausibility of 99.99 mIoU alone, since that is outside consensus but could in principle stem from a reporting or metric error; the internal gap is that neither Section IV nor Algorithm 1 defines an inference-time forward pass. The proposed concrete test would settle it: run the trained model with POTGen disabled and compare the resulting mIoU to Table II. If the number collapses, the advertised gain is label leakage; if it survives, the authors still need to release evaluation code/checkpoints to make the result credible. No code, no error bars, and no ablation of sigma at inference are additional weaknesses, but they are secondary to the undefined evaluation protocol.","tokens_in":11980,"tokens_out":5583,"duration_ms":59313,"concrete_test":"Evaluate the trained LVM+POTGui model on the Cityscapes validation split in a purely feed-forward manner: freeze the head, compute O = omega_su(iGPT_Middle_4 features) (Eq. 2) with no ground-truth labels, and disable POTGen (Eqs. 4–12) and Eq. (13). Compute mIoU over the 500 validation images. If this number is materially below 99.99, the reported metric depended on label-fed POTGen at inference. As a second check, re-run with sigma=1 during inference (using O only) and compare to Table II; if the gap is large, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract; Section IV-B) is that LVM+POTGui attains 99.99 mIoU on Cityscapes and 82.06 on CamVid (Table II) with convergence within 10 epochs. The load-bearing assumption is an unstated inference protocol. In the method, Eq. (12) defines O_POT as the result of K gradient-descent steps on logits toward the ground-truth one-hot P_Y (Eqs. 4 and 11), and Eq. (13) forms O_POTGui by mixing O with O_POT. Algorithm 1 describes only training: O_0 <- model(D_v), then K label-dependent updates, then a loss LCE(P_Y, O_POTGui). It never specifies what the deployed model outputs at evaluation. If O_POT or O_POTGui is used at inference, the near-perfect Table II scores are trivial leakage, because P_Y enters the computation directly. If instead the model outputs the plain logits O = omega_su(F_h) from Eq. (2), then the paper must explain why optimizing a loss that includes a label-dependent lookahead term transfers to the unaugmented forward pass; no such argument or ablation (e.g., sigma=1 vs. sigma=0.5 at eval) is given. Either way, the headline result is not well-defined, and comparison to baselines evaluated without label-dependent lookahead is not apples-to-apples. Missing code/checkpoints and single-run metrics compound this, but the protocol gap is decisive.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a street-scene semantic segmentation system built from a frozen iGPT backbone plus an ASSP head, and a training-time acceleration scheme called POTGui. POTGen unrolls K gradient-descent steps on the predicted logits toward the ground-truth one-hot labels, then mixes the refined logits with the original logits to form the cross-entropy training target. The authors report 99.99 mIoU on Cityscapes and 82.06 mIoU on CamVid with convergence within a few epochs, and claim a 66.48% improvement and over 6x faster convergence relative to the compared baselines. The paper includes layer-selection experiments, an ablation on the number of unrolled layers, and qualitative real-driving tests.","tokens_in":12365,"tokens_out":5379,"duration_ms":52213,"significance":"The proposed training-time lookahead idea—unrolling a few optimization steps with learnable step sizes and using the result as a training target—is potentially interesting, and the paper is unusually explicit in presenting its algorithm. However, the central experimental claims are not credible as reported, and the paper never defines the inference-time protocol for a module that requires ground-truth labels. If the method were re-evaluated with a clearly label-free inference protocol and realistic baselines, it might have value as a convergence-acceleration technique; in its current form, the evidence does not support the stated claims.","major_comments":[{"comment":"The inference-time protocol is never specified. O_POT in Eq. (12) is produced by K gradient-descent steps toward the ground-truth one-hot labels P_Y (Eqs. (4) and (11)), and O_POTGui in Eq. (13) is a weighted blend of the original logits with O_POT. Table II is titled \"average inference performance,\" but Algorithm 1 describes only the training loop and does not state what the deployed model outputs at evaluation. If O_POT or O_POTGui is evaluated, ground-truth labels enter the forward computation directly and the 99.99 mIoU is trivial label leakage; if only the plain logits O from Eq. (2) are evaluated, no argument or ablation (e.g., sigma=1 vs. sigma=0.5 at eval) explains why optimizing Eq. (14) transfers to the unaugmented forward pass. The headline result is therefore not well-defined.","section":"Section IV-B and Algorithm 1"},{"comment":"The central experimental numbers are not credible. LVM+POTGui reaches 99.99 mIoU on Cityscapes, while the same table reports DeepLabv3+ at 69.04, SegFormer at 39.37, and BiSeNetV2 at 33.63. These baseline values are far below their published results on the same datasets, and 99.99 mIoU implies near-perfect per-pixel agreement across 19 classes, which no published segmentation method achieves. The paper must provide the full evaluation setup—input resolution, crop/stride, validation split, test-time augmentation, number of runs, and standard deviations—before any comparison can be interpreted.","section":"Table II and Section IV-B"},{"comment":"The derivation of the softmax cross-entropy gradient is incorrect as written. Eq. (8) omits the 1/N normalization, Eq. (9) keeps only the diagonal term of the softmax Jacobian, and Eq. (10) mixes the class index c on the left with the summation index t. The subsequent identity sum_t P_X,t P_Y,t = P_X,c only holds when P_Y is one-hot at class c, which is not articulated. Although Eq. (11) is the standard final result for one-hot labels, the presented steps do not rigorously establish it, so a central technical derivation needs correction.","section":"Section III-B, Eqs. (7)-(10)"},{"comment":"The method is described as a \"posterior optimization trajectory,\" but O_POT is not a predicted trajectory: it is constructed from the ground-truth labels P_Y at every unrolled step and is then used in a cross-entropy loss against the same P_Y. This makes the training signal partly answer-dependent, and the paper provides no theoretical analysis or ablation (e.g., detaching O_POT, using predicted pseudo-labels, or varying sigma during training/evaluation) to show whether the claimed speedup comes from this label-conditioned lookahead or from some other effect.","section":"Section III-C and Eq. (14)"}],"minor_comments":[{"comment":"There are typos such as \"the the AD model\" in the abstract and \"P osterior\" in Section I; the manuscript needs a careful proofreading pass.","section":"Abstract and Section I"},{"comment":"The heading \"LVM+POTGui pk Benchmarks\" contains a stray \"pk\" and should be corrected.","section":"Section IV-B"},{"comment":"The class name \"Signsymbol\" should be checked against the standard CamVid class list; the paper also reports 0.00 IoU for several classes in baselines, which is unusual and should be explained (e.g., class never predicted due to evaluation protocol).","section":"Table III"},{"comment":"The ablation on the number of POTGen layers has no axis labels or numerical values, so claims such as \"50 layers performs almost the same as 60 layers\" are not quantitatively supported.","section":"Figure 5"},{"comment":"Calling iGPT a Large Vision Model is unconventional, and the paper does not compare with more recent frozen backbones used for semantic segmentation; a brief positioning or comparison with such models would strengthen the claims.","section":"Section III-A and Related Work"},{"comment":"No code, checkpoints, random seeds, or per-run statistics are provided, which limits reproducibility of a paper whose main claims are experimental.","section":"Implementation Details"}],"recommendation":"reject","confidential_remarks":"The undefined evaluation protocol is the decisive issue: either the reported numbers come from label leakage and the experiments collapse, or the numbers are unexplained under a label-free protocol. Given that 99.99 mIoU is far outside the range of published Cityscapes results and the baselines are also anomalously low, the experimental section would need to be redone rather than locally patched. I would not recommend revision unless the authors can supply code and clarify the inference protocol with non-leaking evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth a look for its training scheme, but the headline results should not be taken at face value. The new bit is POTGui: at training time, it unrolls K gradient-descent steps on the segmentation logits, using the ground-truth labels to define those steps, and then trains the head to match a weighted blend of the original and refined logits. I haven't seen this exact idea in the cited literature, and it is a legitimate candidate for a faster-converging objective. The paper also does a clean comparison of which iGPT feature layers work best for street-scene segmentation.\n\nThe soft spots are load-bearing. The abstract and Table II claim 99.99 mIoU on Cityscapes and 82.06 on CamVid. That is beyond any published result on Cityscapes and far above the paper's own DeepLabv3+ baseline of 69.04. The paper never says whether the POTGen module, which requires ground-truth labels P_Y to compute gradient steps, is active at inference. If it is, the near-perfect scores are trivial label leakage, because the answer enters the computation directly. If it is not, the paper owes an explanation for why training against a label-dependent lookahead target transfers to the plain forward pass. No ablation on sigma at evaluation is provided. This omission makes the central empirical claim undefined, not merely under-supported.\n\nThere are also smaller issues. The derivation of the softmax cross-entropy gradient (Eqs. 7-10) mishandles the cross terms in the Jacobian, although the final gradient expression is the standard one. The baseline numbers are weak—DeepLabv3+ at 69.04 mIoU on Cityscapes is well below what published implementations achieve—so the claimed relative improvement is not apples-to-apples. There are no error bars, no code, and no checkpoints.\n\nOverall: the training idea is novel enough to deserve a serious referee, but the experimental evidence as presented is not credible. If the authors can clarify the inference protocol and provide reproducible code with believable numbers, there may be a salvageable contribution. As is, I would not cite it or trust the results. I'd send it to review rather than desk-reject, because the method is genuinely new and the flaw is empirical clarity rather than a nonsense idea.","headline":"Interesting training-time logit-refinement idea, but the 99.99 mIoU claim is unsupported because the paper never specifies whether label-dependent POTGen runs at inference.","tokens_in":12909,"tokens_out":4476,"would_cite":false,"duration_ms":44816,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a frozen iGPT large vision model with a small ASSP head, trained through a Posterior Optimization Trajectory-guided scheme, reaches 99.99 mIoU on Cityscapes and 82.06 mIoU on CamVid while converging within 10 epochs.","keywords":["semantic segmentation","autonomous driving","large vision models","iGPT","posterior optimization trajectory","training acceleration","street scene understanding","POT-guided optimization"],"falsifier":"Take the trained LVM+POTGui model and run a pure forward pass on Cityscapes validation with the POTGen label-guided refinement disabled; if mIoU falls far below the reported 99.99 (e.g., toward the plain LVM's 45.81), then the headline metric depended on ground-truth information injected during evaluation. Conversely, if 99.99 persists without any ground-truth input, the result stands as a genuine near-saturation segmentation score.","tokens_in":11779,"feed_emoji":"🚗","tokens_out":7544,"duration_ms":66306,"temperature":0.7,"pith_summary":"The paper tries to establish that the under-fitting a vehicle's semantic-segmentation model faces as onboard data accumulates can be avoided by keeping a pretrained large vision model (iGPT) frozen and training only a small downstream head, and that the cost of training that head can be slashed by a Posterior Optimization Trajectory (POT)-guided scheme. The central experimental claim is that the resulting LVM+POTGui system reaches 99.99 mIoU on Cityscapes and 82.06 mIoU on CamVid, outperforming the plain LVM baseline by 118.27% and 66.48% respectively, and converges within 10 epochs, about 6 times faster (15 times on Cityscapes). If true, this would mean a near-perfect street-scene segmenter can be obtained by training only a small head on frozen large-model features, a practical recipe for onboard autonomous-driving updates. The method works by unrolling the cross-entropy training loss into K synthetic gradient-descent layers, generating 'future' logits from the ground truth, and blending those logits into the current loss to guide optimization.","feed_headline":"Frozen iGPT plus small head hits 99.99 mIoU on Cityscapes","feed_subtitle":"POT-guided trick trains the perception head in under 10 epochs, six times faster than the baseline.","key_machinery":"The central object is the Posterior Optimization Trajectory (POT) Generator, a learned unrolling of the cross-entropy loss into K gradient-descent 'layers' that produce future logits O_POT. It defines per-layer updates O_k = O_{k-1} - eta_k alpha_k grad_O LCE(P_Y, O_{k-1}) with learnable step sizes alpha_k and eta_k, and a closed-form gradient (Eq. 11) equal to the averaged difference between predicted and ground-truth class probabilities. The POT-guided optimization (POTGui) loss then replaces the raw logits with a weighted blend sigma O + (1-sigma) O_POT before backpropagating (Eq. 14). This carries the argument by injecting information about K future optimization steps into the current gradient, which the paper argues both accelerates convergence and regularizes the learning.","core_discovery":"The paper's discovery claim is that posterior optimization trajectories make a frozen LVM sufficient for street-scene semantic understanding. Concretely, using hidden features from the middle four layers of a pretrained iGPT (zero-shot, parameters frozen) and an ASSP head, the POTGui training scheme reaches 99.99 mIoU/mF1/mPrecision/mRecall on Cityscapes validation and 82.06 mIoU on CamVid test, with class-wise IoU above 80% even for slender classes like Pole and Fence that all compared baselines miss entirely. The mechanism: POTGen unfolds one optimization step into K layers, computes the gradient of cross-entropy w.r.t. the logits at each layer (Eq. 11), performs K learnable gradient-descent updates (Eq. 4), and the final refined logits O_POT (Eq. 12) are blended with the original logits as O_POTGui = sigma O + (1-sigma) O_POT (Eq. 13); the head is then trained against the cross-entropy of these blended logits (Eq. 14). The paper reports that more POTGen layers improve both accuracy and convergence speed up to saturation around 50 layers, and that the method stays accurate and low-variance in real-driving tests on ApolloScapes and CARLA across fog, rain, cloud, and dark conditions.","pith_inferences":["If the reported Cityscapes mIoU survives a no-label inference check, the practical consequence goes beyond this paper: onboard AD updates could skip backbone fine-tuning entirely and retrain only a small head on frozen LVM features for each new data batch, drastically cutting compute and communication.","POTGui is a generic training accelerator: the same unrolled-gradient 'future logits' trick could be applied to detection heads, depth heads, or federated-learning client updates, wherever the loss is differentiable and ground-truth labels are available during training; whether the speedups transfer beyond segmentation is untested here.","The comparison set stops at older real-time baselines and the LVM-without-POTGui control; a natural next test is to pit this recipe against modern segmentation transformers and domain-adaptation methods on challenging domain-shift benchmarks, and to report class-wise results on Cityscapes with the same protocol."],"forward_implications":["On the paper's numbers, a frozen iGPT backbone plus an ASSP head reaches 99.99 mIoU on Cityscapes and 82.06 mIoU on CamVid, making the LVM+POTGui combination the top row in every metric on both datasets.","POTGui converges in at most 10 epochs: about 6 times faster than the LVM baseline on CamVid and 15 times faster on Cityscapes (first epoch vs 16th epoch), which is the claimed computational relief for onboard training.","Rare and slender classes on CamVid (Pole, Fence, Bicyclist) jump from near 0 IoU in the baselines to 85.27, 93.24, and 91.59 IoU respectively, a class-level gain the paper attributes to the LVM's pretraining plus POTGui's optimization.","Increasing the number of POTGen layers improves performance and convergence speed up to a saturation point around 50 layers, after which additional layers add nothing (Fig. 5).","In real-driving tests on ApolloScapes and CARLA_ADV across foggy, rainy, cloudy, and dark conditions, LVM+POTGui outperforms DeepLabv3+ on all metrics with smaller frame-to-frame variance."],"supporting_citations":[{"why":"Supplies the frozen iGPT backbone whose middle-four-layer hidden features feed the perception head.","marker":"[28]"},{"why":"Defines the ASSP head architecture used for multi-scale context aggregation.","marker":"[29]"},{"why":"Provides the layer-wise unrolling perspective that POTGen adapts to treat optimization iterations as layers.","marker":"[30]"},{"why":"Same unrolled-optimization inspiration, cited alongside [30] for the POTGen design.","marker":"[31]"},{"why":"Provides the Cityscapes dataset used for the main 99.99 mIoU result.","marker":"[32]"},{"why":"Provides the CamVid dataset used for the 82.06 mIoU and class-wise comparisons.","marker":"[33]"},{"why":"Supplies the ApolloScapes real-driving test set for the qualitative and metric comparisons.","marker":"[34]"},{"why":"Supplies the CARLA simulator used to generate the CARLA_ADV adverse-weather driving test.","marker":"[35]"},{"why":"Serves as a real-time segmentation baseline (BiSeNetV2) in the main comparison table.","marker":"[36]"},{"why":"Serves as the strongest compared baseline (DeepLabv3+) and the main competitor in the real-driving tests.","marker":"[38]"}],"fun_headline_variants":["Frozen iGPT + tiny head: 99.99 mIoU on Cityscapes","POT-guided training cuts convergence to 10 epochs","Posterior trajectory trick trains vision head 6x faster","Frozen LVM beats under-fitting in street scenes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the validation numbers in Table II were computed with the label-guided POTGen refinement switched off at inference; the paper never states whether the ground-truth-dependent K-step logit updates used in the training loss are active during evaluation, and the reported near-perfect scores are only meaningful under that unstated protocol.","fun_headline_variants_meta":{"raw":{"variants":["Frozen iGPT + tiny head: 99.99 mIoU on Cityscapes","POT-guided training cuts convergence to 10 epochs","Posterior trajectory trick trains vision head 6x faster","Frozen LVM beats under-fitting in street scenes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000894,"raw_usage":{"total_tokens":3934,"prompt_tokens":1104,"completion_tokens":2830,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":720,"completion_tokens_details":{"reasoning_tokens":2758}},"tokens_in":720,"tokens_out":2830,"duration_ms":17437,"temperature":1.0,"reasoning_tokens":2758,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:21:44.192464+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained LVM+POTGui model and run a pure forward pass on Cityscapes validation with the POTGen label-guided refinement disabled; if mIoU falls far below the reported 99.99 (e.g., toward the plain LVM's 45.81), then the headline metric depended on ground-truth information injected during evaluation. Conversely, if 99.99 persists without any ground-truth input, the result stands as a genuine near-saturation segmentation score.","supporting_citations":[{"cited_title":"Communication-efficient activity detection for cell-free massive mimo: An augmented model-driven end-to-end learning framework,","cited_arxiv_id":null,"evidence_quote":"Provides the layer-wise unrolling perspective that POTGen adapts to treat optimization iterations as layers."},{"cited_title":"Communication- efficient joint signal compression and activity detection in cell-free massive mimo,","cited_arxiv_id":null,"evidence_quote":"Same unrolled-optimization inspiration, cited alongside [30] for the POTGen design."},{"cited_title":"Carla: An open urban driving simulator,","cited_arxiv_id":null,"evidence_quote":"Serves as a real-time segmentation baseline (BiSeNetV2) in the main comparison table."}],"review_version":1}