{"id":"605316d1-dd45-424a-9390-ffafc65a229e","arxiv_id":"2606.30875","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A task-agnostic Turing Test Network prunes zero-shot pseudo-labels across vision tasks, yielding F1 gains and enabling category recovery on transfer-vulnerable classes.","lead":"The paper presents a Turing Test Network trained via a Label Imitation Game to prune hallucinated pseudo-labels from zero-shot vision-language model outputs. A smart generalist might read it for a practical way to scale clean training data without manual labeling or model retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Zero-shot transfer claim hinges on untested assumption that classification-trained semantic logic detects detection hallucinations without spatial cues.","rationale":"The reader's weakest assumption matches the load-bearing point exactly. The abstract asserts empirical success on detection transfer, but the argument's security rests on whether that success generalizes beyond semantic errors; the proposed test isolates this without requiring full re-training.","tokens_in":1749,"tokens_out":281,"duration_ms":20496,"concrete_test":"Construct a controlled test set of object-detection pseudo-labels with two error types: (1) semantic (wrong class label, accurate box) and (2) spatial (correct class, inaccurate box). Run the pre-trained TTN on both; if F1 on spatial errors falls below 0.6 while semantic errors exceed 0.85, the transfer claim is limited to non-spatial hallucinations.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that a TTN trained exclusively on image classification (real vs. imitated labels) can prune object-detection pseudo-labels by identifying hallucinations via dataset-wide semantic context alone. This implicitly assumes detection errors (including localization failures) produce detectable semantic signatures that transfer without any geometric or spatial features. No mechanism is described for how classification training encodes localization sensitivity, and the abstract positions this as a strength rather than a risk.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces the Label Imitation Game (LIG) framework, which formalizes pseudo-label pruning as an adversarial interrogation process. It trains a task-agnostic Turing Test Network (TTN) to distinguish real labels from imitated ones using dataset-wide semantic context rather than isolated thresholds. Experiments across four datasets show the TTN improves label accuracy for three state-of-the-art vision-language models without additional supervision or retraining. The central claim is zero-shot task transfer: a TTN trained exclusively on image classification data can prune object detection pseudo-labels by identifying hallucinations via semantic logic alone, yielding F1-score gains of 28% on the worst-performing baseline categories and 44% with task-specific fine-tuning, plus Category Revival where downstream models recover from zero recall on vulnerable classes. Pre-trained TTN models and code are released.","tokens_in":1847,"tokens_out":554,"duration_ms":40919,"significance":"If the empirical claims hold, the work would be significant for scalable pseudo-labeling in foundation model pipelines. The demonstration of cross-task transfer from classification to detection without spatial or geometric features would provide a new, supervision-free alternative to conventional verification methods and could improve handling of hallucinations at scale.","major_comments":[{"comment":"Abstract: The zero-shot transfer claim—that a classification-trained TTN reliably detects object detection hallucinations (including localization failures) via semantic context alone—is load-bearing for the paper's novelty and central contribution. No mechanism is described for how classification-only training encodes sensitivity to spatial or geometric errors, and the abstract positions this as a strength rather than a risk requiring explicit validation.","section":"Abstract"},{"comment":"Experiments section (referenced via reported F1 gains and Category Revival): The 28% and 44% F1 improvements and Category Revival are presented as key evidence for the transfer capability, but without visible details on dataset splits, statistical tests, baseline definitions, or controls for task-specific effects, it is not possible to assess whether these gains are robust or confounded by the choice of detection datasets.","section":"Experiments"}],"minor_comments":[{"comment":"The abstract states experiments on 'four diverse datasets' but does not name them; naming the datasets (and providing a table of their characteristics) would strengthen the generalizability claim.","section":null},{"comment":"The GitHub link for pre-trained models and code is a positive step for reproducibility; ensure the release includes the exact training configurations and evaluation scripts used for the cross-task results.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. The comments identify areas where the presentation of our zero-shot transfer claims and experimental details can be strengthened. We address each point below and will revise the manuscript accordingly.","responses":[{"response":"We agree that the abstract presents the zero-shot transfer as a key strength and that an explicit mechanism discussion would improve clarity. The TTN operates on dataset-wide semantic context rather than per-instance features; our hypothesis is that localization hallucinations in detection often produce semantically implausible label sets (e.g., object co-occurrence violations or category mismatches within a scene) that the classification-trained model can flag via learned semantic logic. The manuscript reports this as an empirical outcome rather than a theoretically derived spatial encoding. In revision we will (a) add a short subsection in Methods explaining this semantic-proxy argument with supporting examples from the detection results and (b) revise the abstract to frame the transfer as an observed capability whose underlying mechanism merits further investigation.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The zero-shot transfer claim—that a classification-trained TTN reliably detects object detection hallucinations (including localization failures) via semantic context alone—is load-bearing for the paper's novelty and central contribution. No mechanism is described for how classification-only training encodes sensitivity to spatial or geometric errors, and the abstract positions this as a strength rather than a risk requiring explicit validation."},{"response":"We accept that additional experimental transparency is required. The reported F1 gains and Category Revival results were obtained on standard public splits (ImageNet/COCO-style partitions) with baselines defined as the unpruned VLM pseudo-labels; Category Revival was measured as recovery from zero recall on specific classes after TTN pruning. In the revised manuscript we will expand the Experiments section to include: explicit train/validation/test split descriptions, statistical significance testing (bootstrap confidence intervals and paired tests across seeds), precise baseline definitions, and controls that isolate task-specific effects (e.g., comparison against a detection-only trained TTN). We will also make the evaluation scripts and split files available in the released repository.","revision_made":"yes","referee_comment":"[Experiments] Experiments section (referenced via reported F1 gains and Category Revival): The 28% and 44% F1 improvements and Category Revival are presented as key evidence for the transfer capability, but without visible details on dataset splits, statistical tests, baseline definitions, or controls for task-specific effects, it is not possible to assess whether these gains are robust or confounded by the choice of detection datasets."}],"tokens_in":1473,"tokens_out":547,"duration_ms":30748,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that this work turns pseudo-label pruning into an adversarial game where a separate Turing Test Network learns to flag bad labels from foundation models, and it reports gains plus a zero-shot transfer from classification training to object detection pruning.\n\nThe new element is the LIG framing itself. Most prior work on pseudo-labels relies on confidence thresholds or simple filters; training a judge network on real versus imitated labels across a dataset is a different setup. The experiments cover four datasets and three vision-language models, with reported F1 lifts of 28% on weak categories and 44% after fine-tuning, plus the category revival observation where cleaned labels let downstream models recover recall on hard classes. Releasing the pre-trained models and code is a concrete plus for anyone wanting to test it.\n\nThe soft spot is the zero-shot transfer claim. The abstract states that a TTN trained only on classification data can prune detection pseudo-labels by semantic context alone, without spatial or geometric checks. That requires detection hallucinations to leave detectable semantic signatures that transfer directly, but the description gives no mechanism for how classification training would encode sensitivity to localization failures. The stress-test note correctly flags this as the load-bearing assumption, and without visible ablations or training details it is hard to judge whether the reported gains actually demonstrate the transfer or something narrower.\n\nThis is for researchers working on scaling vision datasets with foundation model labels, especially those dealing with rare classes or noisy detection outputs. A reader focused on practical label cleaning could extract the idea and the numbers even if the transfer needs more scrutiny.\n\nIt deserves peer review so the transfer mechanism and experimental controls can be examined directly.","headline":"The paper introduces an adversarial Label Imitation Game and TTN for pruning pseudo-labels, with a claimed zero-shot transfer from classification to detection, but that transfer rests on an untested assumption about semantic cues catching localization errors.","tokens_in":2294,"tokens_out":426,"would_cite":false,"duration_ms":29370,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A Turing Test Network trained only on image classification data can prune object detection pseudo-labels and improve F1 scores by up to 44%.","keywords":["pseudo-label pruning","zero-shot learning","Turing Test Network","Label Imitation Game","object detection","image classification","vision-language models","hallucination detection"],"falsifier":"Run the TTN on a collection of object detection pseudo-labels known to be spatially incorrect yet semantically plausible and measure whether it prunes them at the expected rate.","tokens_in":2656,"feed_emoji":"🤖","tokens_out":670,"duration_ms":51317,"temperature":0.7,"pith_summary":"The paper introduces the Label Imitation Game as a way to train a Turing Test Network that judges whether candidate pseudo-labels are real or imitated. This judge operates on dataset-wide context rather than isolated thresholds to remove hallucinations from zero-shot labels produced by vision-language models. The central result is zero-shot task transfer: the same network, trained strictly on classification data, successfully cleans up labels for the more complex object detection task. This yields measurable gains in label quality and lets downstream models recover performance on categories that started with zero recall.","feed_headline":"Turing Test Network prunes detection pseudo-labels from classification data","feed_subtitle":"Trained only on image classification, the judge raises F1 by 28% on worst categories and revives classes that started at zero recall.","key_machinery":"The Turing Test Network (TTN), a task-agnostic judge trained through the Label Imitation Game to distinguish real labels from imitated ones inside a dataset-wide context.","core_discovery":"The paper establishes that a TTN trained via the LIG on image classification datasets can prune erroneous pseudo-labels in object detection, producing F1-score gains of 28% for the worst-performing baseline categories and 44% with task-specific fine-tuning, while also producing Category Revival in which downstream models recover from zero recall on transfer-vulnerable classes.","pith_inferences":["Semantic patterns learned across an entire dataset may be sufficient for label quality control even when geometric structure is ignored.","A single pre-trained TTN could serve as a reusable filter for pseudo-labels in additional vision tasks such as segmentation.","The imitation-game training procedure might generalize to other foundation-model outputs where hallucinations appear without task-specific retraining."],"forward_implications":["Enhances label accuracy for three state-of-the-art vision-language models across four diverse datasets without additional supervision or retraining.","Produces 28% F1-score gains for the worst-performing baseline categories when pruning object detection labels.","Produces 44% F1-score gains when the TTN receives task-specific fine-tuning.","Enables Category Revival so that downstream models recover from zero recall on classes vulnerable to transfer.","Supplies learned semantic-contextual logic as a workable substitute for spatial-geometric verification."],"fun_headline_variants":["TTN trained on classification prunes detection pseudo-labels","Classification TTN achieves 28% F1 gains for worst detection categories","TTN pruning leads to category revival from zero recall in detection","Turing Test Network transfers pruning from classification to detection"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"A network trained only to tell real labels from imitated ones on classification data will reliably detect hallucinations in object detection pseudo-labels without any spatial or geometric verification.","fun_headline_variants_meta":{"raw":{"variants":["TTN trained on classification prunes detection pseudo-labels","Classification TTN achieves 28% F1 gains for worst detection categories","TTN pruning leads to category revival from zero recall in detection","Turing Test Network transfers pruning from classification to detection"]},"model":"grok-4.3","cost_usd":0.006014,"raw_usage":{"total_tokens":2864,"prompt_tokens":702,"num_sources_used":0,"completion_tokens":68,"cost_in_usd_ticks":60137000,"prompt_tokens_details":{"text_tokens":702,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2094,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":702,"tokens_out":68,"duration_ms":27903,"temperature":1.0,"reasoning_tokens":2094,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T01:58:36.270590+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Run the TTN on a collection of object detection pseudo-labels known to be spatially incorrect yet semantically plausible and measure whether it prunes them at the expected rate.","supporting_citations":[],"review_version":1}