Pith. sign in

REVIEW 3 major objections 2 minor 107 references

SPARSE Data, Rich Results: Few-Shot Semi-Supervised Learning via Class-Conditioned Image Translation

T0 review · 3 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A three-network GAN framework that translates unlabeled medical images between classes is claimed to beat six state-of-the-art semi-supervised methods at 5 to 50 labeled samples per class.

desk verdict The supplied full text is a different paper (cs.RO), so I can judge only the abstract; on that basis it's a plausible but unverifiable empirical claim, and the editor should get the correct manuscript before deciding. read the letter →

arxiv 2508.06429 v1 pith:RY7XO7AD submitted 2025-08-08 cs.CV cs.AI

Guido Manni , Clemente Lauretti , Loredana Zollo , Paolo Soda This is my paper
classification cs.CVcs.AI
keywords few-shotlearningsemi-supervisedGANimage-to-imagetranslationpseudo-labelingmedicalimageclassificationMedMNISTclass-conditionedgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes SPARSE, a GAN-based semi-supervised framework for medical image classification when only 5–50 labeled samples per class exist. It uses a generator that performs class-conditioned image-to-image translation on unlabeled images, a joint discriminator/classifier, and a dedicated classifier, trained in alternating supervised and unsupervised phases. Ensemble pseudo-labels combine confidence-weighted predictions of the discriminator and classifier with exponential-moving-average temporal consistency. The paper claims statistically significant gains over six GAN-based semi-supervised baselines on eleven MedMNIST datasets, with the strongest edge at 5 shots. If true, this would make deep medical imaging classification practical with very few annotations.

What carries the argument

Class-conditioned image-to-image translation generator: instead of synthesizing images from random noise, the generator maps an unlabeled image into another class's appearance, providing dense supervisory signal. It cooperates with a joint discriminator/classifier and a dedicated classifier; pseudo-labels are computed by ensembling confidence-weighted predictions of the two classifiers with exponential moving average (EMA) temporal consistency.

What would settle it

Track the accuracy of the EMA-ensembled pseudo-labels on a held-out portion of the unlabeled set during training at 5 shots. If pseudo-label accuracy falls at or below chance in early epochs, or if removing the translation phase (an ablation) does not reduce final classification accuracy, the claim that translation-driven pseudo-labels drive the gains is falsified.

Watch

Extended reading notes

Core claim

The paper claims that semi-supervised learning in extreme data scarcity can be driven by image-to-image translation rather than generation from noise. In the SPARSE framework, the generator is conditioned on class labels and learns to translate an unlabeled image into a different class while preserving structure; the discriminator simultaneously judges authenticity and predicts class, while a separate classifier adds a second prediction. Pseudo-labels for unlabeled data are formed by ensembling the discriminator's and classifier's confidence-weighted predictions with exponential moving averaging for temporal consistency. Trained through three phases—supervised, unsupervised translation, and

Load-bearing premise

At 5–50 labeled examples per class, the ensemble pseudo-labels stay accurate enough that the unsupervised image-translation phase adds signal rather than noise—the abstract claims the mechanism but does not report pseudo-label accuracy or training stability.

Editorial extensions

If this is right

  • If correct, medical imaging classifiers can be built with only five labeled examples per class while using abundant unlabeled scans, substantially lowering annotation cost.
  • The method's strongest gains at 5 shots suggest translation-based augmentation is more label-efficient than generation-from-noise GANs in extreme low-data regimes.
  • The reported consistency across eleven MedMNIST datasets implies the recipe may transfer across different medical imaging modalities and classification tasks.
  • The framework's superiority at 10, 20, and 50 shots also suggests a usable semi-supervised training recipe beyond the most extreme label-scarcity setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The supplied full text is a different paper (about shortcut learning in robot policies), so the experimental evidence behind the abstract could not be inspected in this pass; this summary restates the abstract's claims rather than verified experimental details.
  • If the pseudo-labeling ensemble is the real source of gain, an ablation that replaces the translation generator with simpler augmentations would reveal whether translation itself or the label-ensembling mechanism drives the reported improvement.
  • A natural testable extension is applying the same three-network recipe to non-medical few-shot image benchmarks to see whether the 5-shot edge is specific to medical image statistics or a general phenomenon.
  • The EMA temporal-consistency pseudo-label design could be sharpened by measuring pseudo-label accuracy during training; the abstract does not report it, and that measurement would directly test the mechanism's plausibility.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The paper, as described by its abstract, introduces SPARSE, a GAN-based semi-supervised learning framework for medical image classification in low-label regimes (5 to 50 labeled samples per class). The proposed method combines a class-conditioned image-translation generator, a joint discriminator/classifier, a dedicated classifier, and ensemble pseudo-labeling with exponential moving averaging, and claims statistically significant improvements over six GAN-based semi-supervised baselines on eleven MedMNIST datasets. The full text supplied for review, however, is not the SPARSE paper: it is a separate manuscript titled 'Shortcut Learning in Generalist Robot Policies' (arXiv:2508.06426v1, cs.RO), which contains no SPARSE framework, no MedMNIST experiments, no pseudo-label algorithm, and no comparison with the claimed baselines. Consequently, the abstract is the only content from the target manuscript available, and the central empirical claims cannot be checked from the submitted materials.

Significance. If the abstract's claims were substantiated, the contribution would be significant for few-shot semi-supervised medical image classification: the design of using class-conditioned translation rather than generation-from-noise, combined with ensemble discriminator/classifier pseudo-labels and temporal EMA consistency, is a plausible and potentially useful approach, and the availability of code is a positive step. However, because the body of the submitted manuscript is an unrelated paper, the current submission provides no verifiable evidence for any of these claims. The claimed statistical superiority, the stability of the adversarial training at 5-shot settings, and the accuracy of the pseudo-labeling mechanism all remain unexamined. As submitted, this is not a technically reviewable paper.

major comments (3)
  1. [Full Text (entire document)] The manuscript supplied for review is not the paper announced in the abstract. The abstract describes SPARSE, a three-network GAN-based semi-supervised framework evaluated on MedMNIST; the Full Text is 'Shortcut Learning in Generalist Robot Policies' (Xing et al., arXiv:2508.06426, cs.RO). This body contains no SPARSE framework, no MedMNIST experiments, no six GAN-based baselines, no pseudo-label algorithm, and no statistical protocol. The central claim of the paper is therefore entirely unsupported by the submitted text.
  2. [Abstract] The headline claim of 'statistically significant improvements over six state-of-the-art GAN-based semi-supervised methods' and 'superiority across all evaluated settings' is presented without any statistical protocol. There are no error bars, seed counts, test statistics, confidence intervals, or multiple-comparison corrections. Even if the correct full text were provided, this claim would require a concrete comparison protocol to be verifiable; in the current submission it is an unsupported assertion.
  3. [Abstract (proposed mechanism)] The load-bearing premise is that at 5-50 labeled samples per class the ensemble EMA pseudo-labels remain accurate enough and the joint adversarial training avoids collapse, so that the unsupervised image-translation phase adds signal rather than noise. The abstract asserts this mechanism but provides no evidence on pseudo-label accuracy, training stability, or mode-collapse behavior in the extreme 5-shot regime. The supplied full text does not address these issues, so the core mechanism is unverified.
minor comments (2)
  1. [Abstract] The code link (github.com/GuidoManni/SPARSE) is a welcome reproducibility element; a pinned commit or versioned release would make the claims more easily checkable once the correct manuscript is provided.
  2. [Full Text] The supplied full text has its own title, authors, and reference list, all pertaining to robot learning and the OXE dataset; none of the cited works relate to MedMNIST or GAN-based semi-supervised classification. If this is a packaging error, the submission must be replaced with the correct manuscript.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity detectable from the abstract; the supplied full text is a different paper, which creates an evidentiary gap but not circularity.

full rationale

The only in-scope evidence from arXiv:2508.06429 is the abstract, which makes an empirical claim: SPARSE achieves statistically significant improvements over six GAN-based semi-supervised baselines on eleven MedMNIST datasets at 5-50 shots per class. The abstract contains no derivation, no equations, and no fitted-parameter-to-prediction chain. The claimed improvements are comparisons against external baselines; nothing in the abstract defines a performance metric in terms of the method's own outputs or imports a uniqueness result from the authors' prior work. The supplied full text is a different paper (arXiv:2508.06426, cs.RO, 'Shortcut Learning in Generalist Robot Policies'), so the mechanism details, baseline identities, and statistical protocol cannot be checked. That is an evidentiary mismatch, not a circularity. Under the hard rules, circularity may be claimed only with a quoted reduction or a fitted input renamed as prediction; no such step is present. The reader's separate concerns about pseudo-label accuracy, training stability, and per-dataset hyperparameter tuning are empirical risks that cannot be assessed from the abstract and, even if substantiated, would concern correctness rather than circularity. Therefore the honest finding is no significant circularity, score 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

Only the abstract was available because the supplied full text is a different manuscript. This ledger records the load-bearing design choices and premises that the abstract leaves unspecified.

free parameters (3)
  • EMA momentum coefficient
    Exponential moving averaging over pseudo-label predictions requires a momentum hyperparameter; not stated in the abstract and typically tuned per dataset.
  • Ensemble fusion weight for discriminator vs classifier confidence
    The abstract says pseudo-labels combine confidence-weighted predictions from the discriminator and classifier but does not specify the weighting; this controls how much each head influences the unsupervised phase.
  • Unlabeled pool size and per-phase training budget
    The abstract says 'abundant unlabeled images' and a three-phase schedule, but volumes and iteration counts are unspecified; these choices materially affect the claimed gains at 5-50 shots.
assumptions (3)
  • domain assumption The three networks train stably in the three-phase schedule with only 5-50 real labeled examples per class, without mode collapse or discriminator collapse.
    Adversarial training in extreme few-shot regimes is the known instability point; the abstract provides no evidence for convergence.
  • domain assumption Ensemble pseudo-labels on the unlabeled pool are accurate enough to avoid confirmation bias and error accumulation across the unsupervised phases.
    Pseudo-label feedback loops are the standard failure mode of few-shot SSL; the abstract asserts the EMA/ensemble mechanism mitigates this without evidence.
  • domain assumption The significance testing behind 'statistically significant improvements' across 11 datasets x 4 settings is valid, with seeds, paired tests, and multiple-comparison handling.
    The abstract gives no statistical protocol; at 5 shots per class variance is high, and 44 dataset-setting comparisons invite false positives without correction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SPARSE Data, Rich Results: Few-Shot Semi-Supervised Learning via Class-Conditioned Image Translation." pith.science (2026). https://pith.science/paper/RY7XO7AD

@misc{pith2026250806429,
  author       = {Pith},
  title        = {Pith review of: SPARSE Data, Rich Results: Few-Shot Semi-Supervised Learning via Class-Conditioned Image Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RY7XO7AD}},
  note         = {Machine review of arXiv:2508.06429}
}
read the original abstract

Deep learning has revolutionized medical imaging, but its effectiveness is severely limited by insufficient labeled training data. This paper introduces a novel GAN-based semi-supervised learning framework specifically designed for low labeled-data regimes, evaluated across settings with 5 to 50 labeled samples per class. Our approach integrates three specialized neural networks -- a generator for class-conditioned image translation, a discriminator for authenticity assessment and classification, and a dedicated classifier -- within a three-phase training framework. The method alternates between supervised training on limited labeled data and unsupervised learning that leverages abundant unlabeled images through image-to-image translation rather than generation from noise. We employ ensemble-based pseudo-labeling that combines confidence-weighted predictions from the discriminator and classifier with temporal consistency through exponential moving averaging, enabling reliable label estimation for unlabeled data. Comprehensive evaluation across eleven MedMNIST datasets demonstrates that our approach achieves statistically significant improvements over six state-of-the-art GAN-based semi-supervised methods, with particularly strong performance in the extreme 5-shot setting where the scarcity of labeled data is most challenging. The framework maintains its superiority across all evaluated settings (5, 10, 20, and 50 shots per class). Our approach offers a practical solution for medical imaging applications where annotation costs are prohibitive, enabling robust classification performance even with minimal labeled data. Code is available at https://github.com/GuidoManni/SPARSE.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

107 extracted references · 27 canonical work pages

  1. [1]

    Kaplan, S

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Rad- ford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    arXiv 2001

  2. [2]

    Hoffmann, S

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022

    arXiv 2022

  3. [3]

    X. Zhai, A. Kolesnikov, N. Houlsby, and L. Beyer. Scaling vision transformers. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12104–12113, 2022

    2022

  4. [4]

    X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kir- mani, et al. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.05941, 2024

    arXiv 2024

  5. [5]

    H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V . Myers, M. J. Kim, M. Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning, pages 1723–1736. PMLR, 2023

    2023

  6. [6]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022

    arXiv 2022

  7. [7]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Haus- man, B. Ichter, et al. π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    arXiv 2024

  8. [8]

    Bharadhwaj, J

    H. Bharadhwaj, J. Vakil, M. Sharma, A. Gupta, S. Tulsiani, and V . Kumar. Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 4788–4795. IEEE, 2024

    2024

Show all 107 references
  1. [9]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024

    2024 arXiv

  2. [10]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In 2024 IEEE International Conference on Robotics and Automat...

    2024

  3. [11]

    Q. Bu, J. Cai, L. Chen, X. Cui, Y . Ding, S. Feng, S. Gao, X. He, X. Huang, S. Jiang, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent em- bodied systems. arXiv preprint arXiv:2503.06669, 2025

    2025 arXiv

  4. [12]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  5. [13]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024. 12

    2024 arXiv

  6. [14]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246, 2024

    2024 arXiv

  7. [15]

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864, 2024

    2024 arXiv

  8. [16]

    Pertsch, K

    K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025

    2025 arXiv

  9. [17]

    G. R. Team, S. Abeyruwan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, T. Armstrong, A. Balakr- ishna, R. Baruch, M. Bauza, M. Blokzijl, et al. Gemini robotics: Bringing ai into the physical world. arXiv preprint arXiv:2503.20020, 2025

    2025 arXiv

  10. [18]

    J. Gao, S. Belkhale, S. Dasari, A. Balakrishna, D. Shah, and D. Sadigh. A taxonomy for evaluating generalist robot policies. arXiv preprint arXiv:2503.01238, 2025

    2025

  11. [19]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  12. [20]

    Bjorck, F

    J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025

    2025 arXiv

  13. [21]

    K. Wu, C. Hou, J. Liu, Z. Che, X. Ju, Z. Yang, M. Li, Y . Zhao, Z. Xu, G. Yang, et al. Robo- mind: Benchmark on multi-embodiment intelligence normative data for robot manipulation. arXiv preprint arXiv:2412.13877, 2024

    2024 arXiv

  14. [22]

    S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, Y . Sulsky, J. Kay, J. T. Springenberg, et al. A generalist agent. arXiv preprint arXiv:2205.06175, 2022

    2022 arXiv

  15. [23]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  16. [24]

    X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer. Sigmoid loss for language image pre- training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023

    2023

  17. [25]

    S. Tong, Z. Liu, Y . Zhai, Y . Ma, Y . LeCun, and S. Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568–9578, 2024

    2024

  18. [26]

    Karamcheti, S

    S. Karamcheti, S. Nair, A. Balakrishna, P. Liang, T. Kollar, and D. Sadigh. Prismatic vlms: Investigating the design space of visually-conditioned language models. In Forty-first Inter- national Conference on Machine Learning, 2024

    2024

  19. [27]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language super- vision. In International conference on machine learning, pages 8748–8763. PmLR, 2021

    2021

  20. [28]

    Wang and P

    T. Wang and P. Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International conference on machine learning, pages 9929–9939. PMLR, 2020

    2020

  21. [29]

    Higgins, A

    I. Higgins, A. Pal, A. Rusu, L. Matthey, C. Burgess, A. Pritzel, M. Botvinick, C. Blundell, and A. Lerchner. Darla: Improving zero-shot transfer in reinforcement learning. In International conference on machine learning, pages 1480–1490. PMLR, 2017. 13

    2017

  22. [30]

    Arpit, S

    D. Arpit, S. Jastrzebski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fis- cher, A. C. Courville, Y . Bengio, and S. Lacoste-Julien. A closer look at memorization in deep networks. In ICML, pages 233–242. PMLR, 2017

    2017

  23. [31]

    Rahaman, A

    N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. A. Hamprecht, Y . Bengio, and A. C. Courville. On the spectral bias of neural networks. InICML, pages 5301–5310. PMLR, 2019

    2019

  24. [32]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776–44791, 2023

    2023

  25. [33]

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Dif- fusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, page 02783649241273668, 2023

    2023

  26. [34]

    Belkhale and D

    S. Belkhale and D. Sadigh. Minivla: A better vla with a smaller footprint, 2024. URL https://github.com/Stanford-ILIAD/openvla-mini

    2024

  27. [35]

    S. Tian, B. Wulfe, K. Sargent, K. Liu, S. Zakharov, V . Guizilini, and J. Wu. View-invariant policy learning via zero-shot novel view synthesis. arXiv preprint arXiv:2409.03685, 2024

    2024 arXiv

  28. [36]

    L. Y . Chen, C. Xu, K. Dharmarajan, M. Z. Irshad, R. Cheng, K. Keutzer, M. Tomizuka, Q. Vuong, and K. Goldberg. Rovi-aug: Robot and viewpoint augmentation for cross- embodiment robot learning. arXiv preprint arXiv:2409.03403, 2024

    2024 arXiv

  29. [37]

    Sargent, Z

    K. Sargent, Z. Li, T. Shah, C. Herrmann, H.-X. Yu, Y . Zhang, E. R. Chan, D. Lagun, L. Fei- Fei, D. Sun, et al. Zeronvs: Zero-shot 360-degree view synthesis from a single image. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9420–...

    2024

  30. [38]

    T. Yu, T. Xiao, A. Stone, J. Tompson, A. Brohan, S. Wang, J. Singh, C. Tan, J. Peralta, B. Ichter, et al. Scaling robot learning with semantically imagined experience. arXiv preprint arXiv:2302.11550, 2023

    2023 arXiv

  31. [39]

    Z. Chen, Z. Mandi, H. Bharadhwaj, M. Sharma, S. Song, A. Gupta, and V . Kumar. Semanti- cally controllable augmentations for generalizable robot learning. The International Journal of Robotics Research, page 02783649241273686, 2024

    2024

  32. [41]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al.π0�5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025

    2025 arXiv

  33. [42]

    Kuang, J

    F. Kuang, J. You, Y . Hu, T. Zhang, C. Wen, and Y . Gao. Adapt your body: Mitigating proprioception shifts in imitation learning. arXiv preprint arXiv:2506.23944, 2025

    2025 arXiv

  34. [43]

    Doshi, H

    R. Doshi, H. Walke, O. Mees, S. Dasari, and S. Levine. Scaling cross-embodied learn- ing: One policy for manipulation, navigation, locomotion and aviation. arXiv preprint arXiv:2408.11812, 2024

    2024 arXiv

  35. [44]

    L. Wang, X. Chen, J. Zhao, and K. He. Scaling proprioceptive-visual learning with hetero- geneous pre-trained transformers. Advances in Neural Information Processing Systems , 37: 124420–124450, 2024

    2024

  36. [45]

    Mandlekar, Y

    A. Mandlekar, Y . Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay, et al. Roboturk: A crowdsourcing platform for robotic skill learning through imi- tation. In Conference on Robot Learning, pages 879–893. PMLR, 2018. 14

    2018

  37. [46]

    Zawalski, W

    M. Zawalski, W. Chen, K. Pertsch, O. Mees, C. Finn, and S. Levine. Robotic control via embodied chain-of-thought reasoning. arXiv preprint arXiv:2407.08693, 2024

    2024 arXiv

  38. [47]

    Cheng, Y

    A.-C. Cheng, Y . Ji, Z. Yang, Z. Gongye, X. Zou, J. Kautz, E. Bıyık, H. Yin, S. Liu, and X. Wang. Navila: Legged robot vision-language-action model for navigation. arXiv preprint arXiv:2412.04453, 2024

    2024 arXiv

  39. [48]

    Belkhale, T

    S. Belkhale, T. Ding, T. Xiao, P. Sermanet, Q. Vuong, J. Tompson, Y . Chebotar, D. Dwibedi, and D. Sadigh. Rt-h: Action hierarchies using language. arXiv preprint arXiv:2403.01823, 2024

    2024 arXiv

  40. [49]

    L. X. Shi, B. Ichter, M. Equi, L. Ke, K. Pertsch, Q. Vuong, J. Tanner, A. Walling, H. Wang, N. Fusai, et al. Hi robot: Open-ended instruction following with hierarchical vision-language- action models. arXiv preprint arXiv:2502.19417, 2025

    2025 arXiv

  41. [50]

    Huang, F

    H. Huang, F. Liu, L. Fu, T. Wu, M. Mukadam, J. Malik, K. Goldberg, and P. Abbeel. Otter: A vision-language-action model with text-aware visual feature extraction. arXiv preprint arXiv:2503.03734, 2025

    2025

  42. [51]

    M. J. Kim, C. Finn, and P. Liang. Fine-tuning vision-language-action models: Optimizing speed and success. arXiv preprint arXiv:2502.19645, 2025

    2025 arXiv

  43. [52]

    H.-S. Fang, H. Fang, Z. Tang, J. Liu, C. Wang, J. Wang, H. Zhu, and C. Lu. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot. arXiv preprint arXiv:2307.00595, 2023

    2023 arXiv

  44. [53]

    Jiang, Y

    Z. Jiang, Y . Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. Fan, and Y . Zhu. Dexmimicgen: Automated data generation for bimanual dexterous manipulation via imitation learning.arXiv preprint arXiv:2410.24185, 2024

    2024 arXiv

  45. [54]

    Geirhos, J.-H

    R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wich- mann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665– 673, 2020

    2020

  46. [55]

    Izmailov, P

    P. Izmailov, P. Kirichenko, N. Gruver, and A. G. Wilson. On feature learning in the presence of spurious correlations. Advances in Neural Information Processing Systems , 35:38516– 38532, 2022

    2022

  47. [56]

    W. Ye, G. Zheng, X. Cao, Y . Ma, and A. Zhang. Spurious correlations in machine learning: A survey. arXiv preprint arXiv:2402.12715, 2024

    2024

  48. [57]

    K. Xiao, L. Engstrom, A. Ilyas, and A. Madry. Noise or signal: The role of image back- grounds in object recognition. arXiv preprint arXiv:2006.09994, 2020

    2006 arXiv

  49. [58]

    Sagawa, P

    S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019

    1911 arXiv

  50. [59]

    X. Luo, L. Wei, L. Wen, J. Yang, L. Xie, Z. Xu, and Q. Tian. Rectifying the shortcut learning of background for few-shot learning. Advances in Neural Information Processing Systems , 34:13073–13085, 2021

    2021

  51. [60]

    Moayeri, P

    M. Moayeri, P. Pope, Y . Balaji, and S. Feizi. A comprehensive study of image classification model sensitivity to foregrounds, backgrounds, and visual attributes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19087–19097, 2022

    2022

  52. [61]

    Kolesnikov and C

    A. Kolesnikov and C. H. Lampert. Improving weakly-supervised object localization by micro-annotation. arXiv preprint arXiv:1605.05538, 2016. 15

    2016 arXiv

  53. [62]

    Rosenfeld, R

    A. Rosenfeld, R. Zemel, and J. K. Tsotsos. The elephant in the room. arXiv preprint arXiv:1808.03305, 2018

    2018 arXiv

  54. [63]

    Singla and S

    S. Singla and S. Feizi. Salient imagenet: How to discover spurious features in deep learning? arXiv preprint arXiv:2110.04301, 2021

    2021 arXiv

  55. [64]

    Shetty, B

    R. Shetty, B. Schiele, and M. Fritz. Not using the car to see the sidewalk–quantifying and controlling the effects of context in classification and segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8218–8226, 2019

    2019

  56. [65]

    M. A. Alcorn, Q. Li, Z. Gong, C. Wang, L. Mai, W.-S. Ku, and A. Nguyen. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4845–4854, 2019

    2019

  57. [66]

    S. Mo, H. Kang, K. Sohn, C.-L. Li, and J. Shin. Object-aware contrastive learning for debi- ased scene representation. Advances in Neural Information Processing Systems , 34:12251– 12264, 2021

    2021

  58. [67]

    Geirhos, P

    R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel. Imagenet- trained cnns are biased towards texture; increasing shape bias improves accuracy and robust- ness. In International conference on learning representations, 2018

    2018

  59. [68]

    Brendel and M

    W. Brendel and M. Bethge. Approximating cnns with bag-of-local-features models works surprisingly well on imagenet. arXiv preprint arXiv:1904.00760, 2019

    1904 arXiv

  60. [69]

    Y . Li, Y . Li, and N. Vasconcelos. Resound: Towards action recognition without representation bias. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 513– 528, 2018

    2018

  61. [70]

    M. T. Ribeiro, T. Wu, C. Guestrin, and S. Singh. Beyond accuracy: Behavioral testing of nlp models with checklist. arXiv preprint arXiv:2005.04118, 2020

    2005 arXiv

  62. [71]

    Y . Yuan, L. Zhao, K. Zhang, G. Zheng, and Q. Liu. Do llms overcome shortcut learning? an evaluation of shortcut challenges in large language models.arXiv preprint arXiv:2410.13343, 2024

    2024 arXiv

  63. [72]

    R. Tang, D. Kong, L. Huang, and H. Xue. Large language models can be lazy learners: Analyze shortcuts in in-context learning. arXiv preprint arXiv:2305.17256, 2023

    2023 arXiv

  64. [73]

    M. Du, F. He, N. Zou, D. Tao, and X. Hu. Shortcut learning of large language models in natural language understanding. Communications of the ACM, 67(1):110–120, 2023

    2023

  65. [74]

    W. Ding, L. Shi, Y . Chi, and D. Zhao. Seeing is not believing: Robust reinforcement learning against spurious correlation. Advances in Neural Information Processing Systems, 36:66328– 66363, 2023

    2023

  66. [75]

    Grandien, Q

    N. Grandien, Q. Delfosse, and K. Kersting. Interpretable end-to-end neurosymbolic rein- forcement learning agents. arXiv preprint arXiv:2410.14371, 2024

    2024

  67. [76]

    Hoftijzer, G

    D. Hoftijzer, G. Burghouts, and L. Spreeuwers. Language-based augmentation to address shortcut learning in object-goal navigation. In 2023 Seventh IEEE International Conference on Robotic Computing (IRC), pages 1–8. IEEE, 2023

    2023

  68. [77]

    Z. Deng, J. Jiang, G. Long, and C. Zhang. Causal reinforcement learning: A survey. arXiv preprint arXiv:2307.01452, 2023

    2023 arXiv

  69. [78]

    R. Tian, C. Xu, M. Tomizuka, J. Malik, and A. Bajcsy. What matters to you? towards visual representation alignment for robot learning. arXiv preprint arXiv:2310.07932, 2023. 16

    2023 arXiv

  70. [79]

    R. R. Sanchez, H. Nemlekar, S. Sagheb, C. M. Nunez, and D. P. Losey. Recon: Reducing causal confusion with human-placed markers. arXiv preprint arXiv:2409.13607, 2024

    2024 arXiv

  71. [80]

    De Haan, D

    P. De Haan, D. Jayaraman, and S. Levine. Causal confusion in imitation learning. Advances in neural information processing systems, 32, 2019

    2019

  72. [81]

    J. Park, Y . Seo, C. Liu, L. Zhao, T. Qin, J. Shin, and T.-Y . Liu. Object-aware regulariza- tion for addressing causal confusion in imitation learning. Advances in Neural Information Processing Systems, 34:3029–3042, 2021

    2021

  73. [82]

    I. Bica, D. Jarrett, and M. van der Schaar. Invariant causal imitation learning for generalizable policies. Advances in Neural Information Processing Systems, 34:3952–3964, 2021

    2021

  74. [83]

    Y . Chen, Y . Zhang, G. D’urso, N. Lawrance, and B. Tidd. Improving generalization ability of robotic imitation learning by resolving causal confusion in observations. arXiv preprint arXiv:2507.22380, 2025

    2025 arXiv

  75. [84]

    Zhang, S

    J. Zhang, S. Wu, X. Luo, H. Wu, L. Gao, H. T. Shen, and J. Song. Inspire: Vision-language- action models with intrinsic spatial reasoning. arXiv preprint arXiv:2505.13888, 2025

    2025

  76. [85]

    S. Wu, J. Zhang, X. Luo, J. Xie, J. Song, H. T. Shen, and L. Gao. Policy contrastive decoding for robotic foundation models. arXiv preprint arXiv:2505.13255, 2025

    2025 arXiv

  77. [86]

    Hejna, S

    J. Hejna, S. Mirchandani, A. Balakrishna, A. Xie, A. Wahid, J. Tompson, P. Sanketi, D. Shah, C. Devin, and D. Sadigh. Robot data curation with mutual information estimators. arXiv preprint arXiv:2502.08623, 2025

    2025 arXiv

  78. [87]

    S. Bai, W. Zhou, P. Ding, W. Zhao, D. Wang, and B. Chen. Rethinking latent redundancy in behavior cloning: An information bottleneck approach for robot manipulation.arXiv preprint arXiv:2502.02853, 2025

    2025 arXiv

  79. [88]

    Belkhale, Y

    S. Belkhale, Y . Cui, and D. Sadigh. Data quality in imitation learning. Advances in neural information processing systems, 36:80375–80395, 2023

    2023

  80. [89]

    Hejna, C

    J. Hejna, C. Bhateja, Y . Jiang, K. Pertsch, and D. Sadigh. Re-mix: Optimizing data mixtures for large scale imitation learning. arXiv preprint arXiv:2408.14037, 2024

    2024 arXiv

  81. [90]

    J. Gao, A. Xie, T. Xiao, C. Finn, and D. Sadigh. Efficient data collection for robotic manipu- lation via compositional generalization. arXiv preprint arXiv:2403.05110, 2024

    2024 arXiv

  82. [91]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierar- chical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  83. [92]

    Kuznetsova, H

    A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci, A. Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International journal of com...

    1956

  84. [93]

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ar, and C. L. Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th Euro- pean conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13 , pages 740–...

    2014

  85. [94]

    B. Zhou, H. Zhao, X. Puig, T. Xiao, S. Fidler, A. Barriuso, and A. Torralba. Semantic un- derstanding of scenes through the ade20k dataset. International Journal of Computer Vision, 127:302–321, 2019. 17

    2019

  86. [95]

    Van Horn, O

    G. Van Horn, O. Mac Aodha, Y . Song, Y . Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8769–8778, 2018

    2018

  87. [96]

    B. A. Plummer, L. Wang, C. M. Cervantes, J. C. Caicedo, J. Hockenmaier, and S. Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision , pages 2641–2649, 2015

    2015

  88. [97]

    D. A. Hudson and C. D. Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700–6709, 2019

    2019

  89. [98]

    Z. Lin, C. Liu, R. Zhang, P. Gao, L. Qiu, H. Xiao, H. Qiu, C. Lin, W. Shao, K. Chen, et al. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXiv preprint arXiv:2311.07575, 2023

    2023 arXiv

  90. [99]

    Krishna, Y

    R. Krishna, Y . Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y . Kalantidis, L.-J. Li, D. A. Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123:32–73, 2017

    2017

  91. [100]

    Schuhmann, R

    C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T. Coombes, J. Jitsev, and A. Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image- text pairs. arXiv preprint arXiv:2111.02114, 2021

    2021 arXiv

  92. [101]

    Sharma, N

    P. Sharma, N. Ding, S. Goodman, and R. Soricut. Conceptual captions: A cleaned, hyper- nymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 2556–...

    2018

  93. [102]

    First, we build a comprehensive object mask library (D)

    Step 1: Object Mask Library Creation. First, we build a comprehensive object mask library (D). To do this, we apply Grounded-SAM2 to the initial frame of every episode in our dataset. This allows us to extract high-quality segmentation masks for all target objects as seen from...

  94. [103]

    Next, for each image (o�) in a trajectory, we identify the target object specified by the language instruction ( L)

    Step 2: Scene Preparation (Object Removal and Inpainting). Next, for each image (o�) in a trajectory, we identify the target object specified by the language instruction ( L). Us- ing Grounded-SAM2 again, we segment this specific object to obtain its mask (m���� ) and record i...

  95. [104]

    Finally, we create the augmented image

    Step 3: Object Swapping and Augmented Dataset Generation. Finally, we create the augmented image. For a clean scene that originally contained a specific object (e.g., a banana with mask d���� ), we randomly sample a mask of adifferent object from our library (e.g., a watermelo...

  96. [105]

    Each video captures the complete sequence of actions taken by the robot policy from the start to the end of an episode

    Video Review: Human evaluators are presented with video recordings of every evaluation trial for a given experimental setup. Each video captures the complete sequence of actions taken by the robot policy from the start to the end of an episode

  97. [106]

    shortcut

    Behavioral Judgment: For each video, the evaluator judges whether the policy’s actions correspond to the given language instruction or if they revert to a “shortcut” behavior learned from spurious correlations in the training data

  98. [107]

    For example, when instructed to interact with object A from a viewpoint previously associated with object B, the policy ignores A and attempts to interact with B

    Scoring Rubric: A score is assigned to each trial based on a predefined rubric: • Score = 1.0 (Clear Shortcut): The policy unequivocally ignores the instruction and performs a clear shortcut action. For example, when instructed to interact with object A from a viewpoint previo...

  99. [108]

    Degree of Shortcut Learning

    Final Score Calculation: The final “Degree of Shortcut Learning” for a model is calcu- lated by averaging the scores from all of its evaluation trials. A score closer to 1.0 indicates a strong tendency to rely on shortcuts, while a score closer to 0.0 indicates that the policy...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.