Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Grounding action labels in the observing camera's coordinate frame makes vision-language-action policies converge faster and generalize across viewpoints.

desk verdict A simple, plausible coordinate-relabeling trick with an overstated cross-view-generalization claim that needs the ablations it doesn't show. read the letter →

arxiv 2508.13103 v1 pith:TJL5VLZ3 submitted 2025-08-18 cs.RO cs.CV

Tianyi Zhang , Haonan Duan , Haoran Hao , Yu Qiao , Jifeng Dai , Zhi Hou This is my paper · ORCID
classification cs.ROcs.CV
keywords vision-language-actionmodelsrobotmanipulationcameracoordinateframeextrinsiccalibrationcross-viewgeneralizationend-effectorposepredictionobservation-centricactiongrounding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language-action policies usually predict end-effector poses in the robot base frame, but training data comes from many cameras, so the same action gets different labels depending on which camera saw it. This paper proposes labeling each action in the coordinate frame of the camera that observes the scene, using the camera's extrinsic calibration matrix to transform base-frame poses into camera-frame poses. The authors argue this unifies the prediction targets across heterogeneous viewpoints, aligning actions with the pixels the policy sees. They report that this lightweight, plug-and-play change accelerates convergence, raises success rates, and improves robustness to camera viewpoint changes on both simulated and real manipulation tasks. If correct, it would make VLA policies more reliable when cameras move or are swapped.

What carries the argument

The central object is the camera extrinsic calibration matrix: a rigid rotation-and-translation transform that maps a point from the robot base frame into the camera frame. OC-VLA applies this matrix to every end-effector pose label before training, so the policy's regression targets share the same coordinate system as its visual observations. This single deterministic transform does the work, which is why the method is described as lightweight and plug-and-play.

What would settle it

Train a VLA with camera-frame targets on data from several cameras, then evaluate it at a held-out viewpoint never seen in training and compare against the same policy trained with base-frame targets: if the camera-frame policy does not clearly outperform the base-frame baseline, the cross-view generalization claim is falsified. A second check is to perturb the extrinsic matrices during training or deployment by a few centimeters and degrees; under the paper's logic, this should corrupt the camera-frame labels and degrade OC-VLA more than a base-frame policy.

Watch

Extended reading notes

Core claim

The central claim is that action targets should be expressed in the observing camera's coordinate system rather than in the robot base frame. For each training episode, the end-effector pose is transformed from base coordinates to camera coordinates using the camera's extrinsic calibration matrix. This makes labels from different cameras comparable and aligned with the visual input, so the policy learns a more consistent mapping from pixels to actions. The method, named Observation-Centric VLA (OC-VLA), requires no new network modules or auxiliary losses; it is applied as a preprocessing step and is therefore compatible with existing VLA architectures. The paper reports that this coordinate change alone accelerates convergence, improves task success, and strengthens cross-view generalization in simulations and real-world experiments.

Load-bearing premise

Every camera used in training and deployment must have an accurate, stable extrinsic calibration matrix; if calibration is noisy, stale, or missing, the camera-frame targets are misaligned with the pixels and the claimed gains vanish.

Editorial extensions

If this is right

  • A VLA trained with camera-frame action targets should reach a given success level with fewer training steps than the same architecture trained on base-frame targets.
  • Policies trained this way should retain success when the deployment camera is moved, re-oriented, or replaced.
  • The transform can be added as an input preprocessing step to existing VLA pipelines without architectural changes.
  • Multi-camera training data no longer carries viewpoint-dependent label inconsistencies, so the policy sees a more coherent action distribution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the central claim holds, viewpoint augmentation and domain randomization might become less necessary, because the label distribution is already invariant to the base frame; the paper does not explicitly draw this conclusion.
  • The benefit should be small when all training data comes from a single fixed camera, since base-frame and camera-frame targets differ only by a fixed rigid transform; this makes the claim testable by varying camera diversity.
  • Extending the method to eye-in-hand cameras would require online extrinsic updates, since the camera moves with the arm; the static-calibration assumption would need revisiting in that setting.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes OC-VLA, a method that relabels end-effector action targets from the robot base coordinate frame into the camera coordinate frame of the observation camera, using the camera's extrinsic calibration matrix. The authors argue that this 'observation-centric' relabeling aligns action prediction with the pixel observation and thereby unifies prediction targets across heterogeneous viewpoints. They claim that OC-VLA accelerates convergence, improves task success rates, and improves cross-view generalization, and that it is a lightweight, plug-and-play modification compatible with existing VLA architectures. The paper reports evaluations on simulated and real-world robotic manipulation tasks.

Significance. If the empirical claims hold, the contribution is a simple and practical training-time relabeling that could improve the viewpoint robustness of VLA policies. The coordinate transform itself is mathematically straightforward and unlikely to be incorrect, and the method does not introduce free parameters fitted to the evaluation data, so the circularity burden is low. The main value of the paper would be the empirical demonstration of a causal benefit from the relabeling, and that demonstration is currently missing from the manuscript. The paper would be publishable if it provides clean ablations that isolate the relabeling mechanism from confounding factors such as target normalization, training budget, and data mixture.

major comments (4)
  1. [§4 (Experiments)] The central claim that camera-frame targets improve cross-view generalization is not supported by an ablative test that isolates relabeling from other factors. The experiments do not appear to include a baseline trained on the same multi-view dataset with base-frame targets, matched in architecture, number of steps, batch size, and data augmentation. Without such a control, the reported gains in convergence and success rate in Tables 1–3 could be caused by better-conditioned target scales, additional optimization steps, or the use of a single camera instead of multiple cameras during training. Please add this baseline and report per-seed mean and standard deviation.
  2. [§3 (coordinate transform)] The paper describes the method as 'unifying prediction targets across heterogeneous viewpoints,' but camera-frame targets are not camera-invariant: for a fixed physical end-effector pose, the numerical label changes when the camera pose changes. The model must still infer the observer's viewpoint from the image. The actual contribution is a relabeling strategy that changes the target coordinate frame, not a unified or view-invariant representation. This overstatement matters because it leads to an evaluation design that may not test true cross-view generalization; the cross-view experiment should use a held-out camera pose never seen during training, rather than a held-out scene with the same training viewpoints.
  3. [§4 (calibration robustness)] The method relies on accurate extrinsic calibration for every camera used in training and deployment, but the manuscript does not analyze sensitivity to calibration error. In real-world robotic setups, camera mounts can shift or drift after calibration. A perturbation study that adds, for example, 1–5 cm translation error and 1–5 degrees rotation error to the extrinsics used to relabel the evaluation targets would be needed to determine whether the claimed robustness to viewpoint variations is robust to calibration noise. Without this, the practical scope of the claim is unclear.
  4. [§4 (Tables 2–3)] The reported success rates are not accompanied by error bars, number of seeds, or statistical significance tests. Because the abstract claims substantial improvements, the lack of variance information makes it impossible to assess whether the differences are meaningful. Please report means over multiple seeds with standard deviations, and state the number of evaluation episodes per task.
minor comments (4)
  1. [Abstract] The name 'Observation-Centric VLA' suggests that the observation representation is changed, but the method actually changes the action label frame. Consider renaming the method or clarifying in the introduction that the relabeling is in the action space, not in the representation.
  2. [§3 (Method)] The coordinate transform equations should define all symbols consistently, for example T_camera^base and p_ee^camera = T_camera^base * p_ee^base, and should specify whether orientation is represented as Euler angles, quaternions, or rotation matrices, because the choice affects target continuity and the scale of the regression loss.
  3. [§2 (Related Work)] The related work discussion should cite recent VLA systems such as RT-2, OpenVLA, and π0, and should compare against prior work on action-space normalization, canonicalization, or frame-invariant action representations.
  4. [§4 (Tables)] The tables would be easier to read if each task name were spelled out and if the baseline rows were explicitly labeled with the action-frame convention (base frame vs. camera frame) and the number of training views.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: OC-VLA's camera-frame action re-labeling is an input transformation, and the convergence and cross-view gains are empirical claims evaluated against benchmarks.

full rationale

The paper's central operation is a coordinate transform of end-effector targets from the robot base frame to the camera frame using the camera extrinsic calibration matrix. This is an input/label preprocessing step, not a fitted parameter or a derived prediction. The claimed benefits (faster convergence, higher success, better cross-view generalization) are empirical outcomes measured in simulation and real-robot evaluations, and are not asserted to follow from the transform by construction. No equation in the supplied text reduces a predicted quantity to a fitted value, and no load-bearing premise is justified solely by a self-citation chain. The potential fragility of extrinsic calibration is a correctness/robustness concern about the method's assumptions, not a circularity in its derivation. Therefore no specific circular step can be quoted, and the appropriate finding is no significant circularity (score 0).

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities are identified from the abstract. The method's correctness rests on accurate camera calibration and on the empirical assumption that VLA policies learn camera-frame targets effectively.

assumptions (3)
  • domain assumption Extrinsic calibration matrices for all cameras are known accurately and remain valid during data collection and deployment.
    The camera-frame transform is applied to every action label; any calibration error directly corrupts the unified action space. Stated in the abstract as 'Leveraging the camera's extrinsic calibration matrix'.
  • standard math End-effector poses are rigid-body quantities that can be expressed equivalently in base or camera coordinates.
    The method relies on standard rigid-body coordinate transformation, which is mathematically valid if calibration is correct.
  • domain assumption VLA models can learn from camera-frame action targets without architectural changes and can execute them after inverse transform.
    The abstract claims plug-and-play compatibility and no substantial modifications; this is an empirical assumption about representation learning and controller compatibility.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy." pith.science (2026). https://pith.science/paper/TJL5VLZ3

@misc{pith2026250813103,
  author       = {Pith},
  title        = {Pith review of: Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TJL5VLZ3}},
  note         = {Machine review of arXiv:2508.13103}
}
read the original abstract

Vision-Language-Action (VLA) models frequently encounter challenges in generalizing to real-world environments due to inherent discrepancies between observation and action spaces. Although training data are collected from diverse camera perspectives, the models typically predict end-effector poses within the robot base coordinate frame, resulting in spatial inconsistencies. To mitigate this limitation, we introduce the Observation-Centric VLA (OC-VLA) framework, which grounds action predictions directly in the camera observation space. Leveraging the camera's extrinsic calibration matrix, OC-VLA transforms end-effector poses from the robot base coordinate system into the camera coordinate system, thereby unifying prediction targets across heterogeneous viewpoints. This lightweight, plug-and-play strategy ensures robust alignment between perception and action, substantially improving model resilience to camera viewpoint variations. The proposed approach is readily compatible with existing VLA architectures, requiring no substantial modifications. Comprehensive evaluations on both simulated and real-world robotic manipulation tasks demonstrate that OC-VLA accelerates convergence, enhances task success rates, and improves cross-view generalization. The code will be publicly available.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cross-View Action Consistency for Camera-Robust Vision-Language-Action Policies

    cs.RO 2026-08 conditional novelty 7.0 of 10

    Regularizing flow-VLA action-velocity predictions across action-equivalent camera views improves held-out camera success on LIBERO-Plus and a real robot while keeping single-scene-RGB inference.

  2. CorridorVLA: Explicit Spatial Constraints for Generative Action Heads via Sparse Anchors

    cs.RO 2026-04 unverdicted novelty 6.0 of 10

    Sparse action-aligned spatial anchors impose an explicit tolerance corridor that improves flow-matching VLA success rates by 4–12 points on LIBERO and LIBERO-Plus.

Reference graph

Works this paper leans on

66 extracted references · 27 canonical work pages · cited by 2 Pith papers

  1. [1]

    4" FUNCTION default.is.dash.repeated.names #1 FUNCTION default.name.format.string

    11em plus .33em minus .07em @technote 4000 4000 100 4000 4000 500 `\.=1000 = #1 #1 #1 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEauthorblockAstyle \@IEEEauthordefaulttextstyle \@IEEEauthorblockconfadjspace -0.25em \@IEEEauthorblockNtopspace 0.0ex \@IEEEauthorblockAtopspace 0.0ex \@IEEEauthorblockNinterlinespace 2.6ex \@IEEEauthorblockAinte...

    work page 2003

  2. [2]

    write newline

    " write newline "" initialize.prev.this.status FUNCTION begin.bib preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@rmstyle " write newline " " write newline " [2] #2 " write newline " =0pt " write newline " " ALTinterwordstretchfactor * " " * write newline " = 2 plus " wr...

  3. [3]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. J. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, I. Leal, K.-H. Lee, S. Levine, Y. Lu, U. Malla, D. Manjunath, I. Mordatch, O. Nachum, C. Parada, J. Peralta, E. Perez, K. Pertsch, J. Q...

    arXiv 2023

  4. [4]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al., ``Openvla: An open-source vision-language-action model,'' arXiv preprint arXiv:2406.09246, 2024

    arXiv 2024

  5. [5]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al., ``Octo: An open-source generalist robot policy,'' arXiv preprint arXiv:2405.12213, 2024

    arXiv 2024

  6. [6]

    Belkhale, T

    S. Belkhale, T. Ding, T. Xiao, P. Sermanet, Q. Vuong, J. Tompson, Y. Chebotar, D. Dwibedi, and D. Sadigh, ``Rt-h: Action hierarchies using language,'' arXiv preprint arXiv:2403.01823, 2024

    arXiv 2024

  7. [7]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al., `` pi_0 : A vision-language-action flow model for general robot control,'' arXiv preprint arXiv:2410.24164, 2024

    arXiv 2024

  8. [8]

    Z. Hou, T. Zhang, Y. Xiong, H. Duan, H. Pu, R. Tong, C. Zhao, X. Zhu, Y. Qiao, J. Dai, and Y. Chen, ``Dita: Scaling diffusion transformer for generalist vision-language-action policy,'' arXiv preprint arXiv:2503.19757, 2025

    arXiv 2025

Show all 66 references
  1. [9]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al., ``Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0,'' in 2024 IEEE International Conference on Robotics and Au...

    2024

  2. [10]

    H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V. Myers, M. J. Kim, M. Du, et al., ``Bridgedata v2: A dataset for robot learning at scale,'' in Conference on Robot Learning. 1em plus 0.5em minus 0.4em PMLR, 2023, pp. 1723--1736

    2023

  3. [11]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al., ``Droid: A large-scale in-the-wild robot manipulation dataset,'' arXiv preprint arXiv:2403.12945, 2024

    2024 arXiv

  4. [12]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, P. Florence, C. Fu, M. G. Arenas, K. Gopalakrishnan, K. Han, K. Hausman, A. Herzog, J. Hsu, B. Ichter, A. Irpan, N. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, I. Lea...

    2023 arXiv

  5. [13]

    Kroemer, S

    O. Kroemer, S. Niekum, and G. Konidaris, ``A review of robot learning for manipulation: Challenges, representations, and algorithms,'' Journal of machine learning research, vol. 22, no. 30, pp. 1--82, 2021

    2021

  6. [14]

    Dalal, T

    M. Dalal, T. Chiruvolu, D. Chaplot, and R. Salakhutdinov, ``Plan-seq-learn: Language model guided rl for solving long horizon robotics tasks,'' arXiv preprint arXiv:2405.01534, 2024

    2024 arXiv

  7. [15]

    Yamada, Y

    J. Yamada, Y. Lee, G. Salhotra, K. Pertsch, M. Pflueger, G. Sukhatme, J. Lim, and P. Englert, ``Motion planner augmented reinforcement learning for robot manipulation in obstructed environments,'' in Conference on Robot Learning. 1em plus 0.5em minus 0.4em PMLR, 2021, pp. 589--603

    2021

  8. [16]

    F. Xia, C. Li, R. Mart \' n-Mart \' n, O. Litany, A. Toshev, and S. Savarese, ``Relmogen: Leveraging motion generation in reinforcement learning for mobile manipulation,'' arXiv preprint arXiv:2008.07792, 2020

    2008 arXiv

  9. [17]

    Shridhar, L

    M. Shridhar, L. Manuelli, and D. Fox, ``Cliport: What and where pathways for robotic manipulation,'' in Conference on robot learning. 1em plus 0.5em minus 0.4em PMLR, 2022, pp. 894--906

    2022

  10. [18]

    1em plus 0.5em minus 0.4em PMLR, 2023, pp

    ------, ``Perceiver-actor: A multi-task transformer for robotic manipulation,'' in Conference on Robot Learning. 1em plus 0.5em minus 0.4em PMLR, 2023, pp. 785--799

    2023

  11. [19]

    X. Li, M. Zhang, Y. Geng, H. Geng, Y. Long, Y. Shen, R. Zhang, J. Liu, and H. Dong, ``Manipllm: Embodied multimodal large language model for object-centric robotic manipulation,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 1...

    2024

  12. [20]

    Y. Jin, D. Li, J. Shi, P. Hao, F. Sun, J. Zhang, B. Fang, et al., ``Robotgpt: Robot manipulation learning from chatgpt,'' IEEE Robotics and Automation Letters, vol. 9, no. 3, pp. 2543--2550, 2024

    2024

  13. [21]

    Singh, V

    I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, ``Progprompt: Generating situated robot task plans using large language models,'' in 2023 IEEE International Conference on Robotics and Automation (ICRA). 1em plus 0.5em minus 0....

    2023

  14. [22]

    X. Li, M. Liu, H. Zhang, C. Yu, J. Xu, H. Wu, C. Cheang, Y. Jing, W. Zhang, H. Liu, et al., ``Vision-language foundation models as effective robot imitators,'' arXiv preprint arXiv:2311.01378, 2023

    2023 arXiv

  15. [23]

    H. Zhen, X. Qiu, P. Chen, J. Yang, X. Yan, Y. Du, Y. Hong, and C. Gan, ``3d-vla: A 3d vision-language-action generative world model,'' arXiv preprint arXiv:2403.09631, 2024

    2024 arXiv

  16. [24]

    D. Qu, H. Song, Q. Chen, Y. Yao, X. Ye, Y. Ding, Z. Wang, J. Gu, B. Zhao, D. Wang, et al., ``Spatialvla: Exploring spatial representations for visual-language-action model,'' arXiv preprint arXiv:2501.15830, 2025

    2025 arXiv

  17. [25]

    Cheang, G

    C.-L. Cheang, G. Chen, Y. Jing, T. Kong, H. Li, Y. Li, Y. Liu, H. Wu, J. Xu, Y. Yang, et al., ``Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation,'' arXiv preprint arXiv:2410.06158, 2024

    arXiv

  18. [26]

    H.-S. Fang, H. Fang, Z. Tang, J. Liu, J. Wang, H. Zhu, and C. Lu, ``Rh20t: A robotic dataset for learning diverse skills in one-shot,'' in RSS 2023 Workshop on Learning for Task and Motion Planning, 2023

    2023

  19. [27]

    Driess, F

    D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al., ``Palm-e: An embodied multimodal language model,'' arXiv preprint arXiv:2303.03378, 2023

    2023 arXiv

  20. [28]

    H. Wu, Y. Jing, C. Cheang, G. Chen, J. Xu, X. Li, M. Liu, H. Li, and T. Kong, ``Unleashing large-scale video generative pre-training for visual robot manipulation,'' arXiv preprint arXiv:2312.13139, 2023

    2023 arXiv

  21. [29]

    P. Li, H. Wu, Y. Huang, C. Cheang, L. Wang, and T. Kong, ``Gr-mg: Leveraging partially-annotated data via multi-modal goal-conditioned policy,'' IEEE Robotics and Automation Letters, 2025

    2025

  22. [30]

    Huang, L

    S. Huang, L. Chen, P. Zhou, S. Chen, Z. Jiang, Y. Hu, P. Gao, H. Li, M. Yao, and G. Ren, ``Enerverse: Envisioning embodied future space for robotics manipulation,'' arXiv preprint arXiv:2501.01895, 2025

    2025

  23. [31]

    Lynch and P

    C. Lynch and P. Sermanet, ``Language conditioned imitation learning over unstructured data,'' arXiv preprint arXiv:2005.07648, 2020

    2005 arXiv

  24. [32]

    Reuss, M

    M. Reuss, M. Li, X. Jia, and R. Lioutikov, ``Goal-conditioned imitation learning using score-based diffusion policies,'' arXiv preprint arXiv:2304.02532, 2023

    2023 arXiv

  25. [33]

    H. Ha, P. Florence, and S. Song, ``Scaling up and distilling down: Language-guided robot skill acquisition,'' in Conference on Robot Learning. 1em plus 0.5em minus 0.4em PMLR, 2023, pp. 3766--3777

    2023

  26. [34]

    Myers, A

    V. Myers, A. W. He, K. Fang, H. R. Walke, P. Hansen-Estruch, C.-A. Cheng, M. Jalobeanu, A. Kolobov, A. Dragan, and S. Levine, ``Goal representations for instruction following: A semi-supervised language interface to control,'' in Conference on Robot Learning. 1em plus 0.5em mi...

    2023

  27. [35]

    Zhang, Y

    E. Zhang, Y. Lu, W. Wang, and A. Zhang, ``Language control diffusion: Efficiently scaling through space, time, and tasks,'' arXiv preprint arXiv:2210.15629, 2022

    2022 arXiv

  28. [36]

    L. Chen, S. Bahl, and D. Pathak, ``Playfusion: Skill acquisition via diffusion from language-annotated play,'' in Conference on Robot Learning. 1em plus 0.5em minus 0.4em PMLR, 2023, pp. 2012--2029

    2023

  29. [37]

    Y. Tian, S. Yang, J. Zeng, P. Wang, D. Lin, H. Dong, and J. Pang, ``Predictive inverse dynamics models are scalable learners for robotic manipulation,'' arXiv preprint arXiv:2412.15109, 2024

    2024 arXiv

  30. [38]

    J. Ho, A. Jain, and P. Abbeel, ``Denoising diffusion probabilistic models,'' Advances in neural information processing systems, vol. 33, pp. 6840--6851, 2020

    2020

  31. [39]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, ``High-resolution image synthesis with latent diffusion models,'' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10\,684--10\,695

    2022

  32. [40]

    Dhariwal and A

    P. Dhariwal and A. Nichol, ``Diffusion models beat gans on image synthesis,'' Advances in neural information processing systems, vol. 34, pp. 8780--8794, 2021

    2021

  33. [41]

    Peebles and S

    W. Peebles and S. Xie, ``Scalable diffusion models with transformers,'' in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4195--4205

    2023

  34. [42]

    Brooks, B

    T. Brooks, B. Peebles, C. Holmes, W. DePue, Y. Guo, L. Jing, D. Schnurr, J. Taylor, T. Luhman, E. Luhman, C. Ng, R. Wang, and A. Ramesh, ``Video generation models as world simulators,'' 2024. [Online]. Available: https://openai.com/research/video-generation-models-as-world-simulators

    2024

  35. [43]

    Liang, Y

    Z. Liang, Y. Mu, H. Ma, M. Tomizuka, M. Ding, and P. Luo, ``Skilldiffuser: Interpretable hierarchical planning via skill abstractions in diffusion-based task execution,'' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16\,467--16\,476

    2024

  36. [44]

    Z. Wang, Z. Li, A. Mandlekar, Z. Xu, J. Fan, Y. Narang, L. Fan, Y. Zhu, Y. Balaji, M. Zhou, et al., ``One-step diffusion policy: Fast visuomotor policies via diffusion distillation,'' arXiv preprint arXiv:2410.21257, 2024

    2024 arXiv

  37. [45]

    J. Cao, Q. Zhang, J. Sun, J. Wang, H. Cheng, Y. Li, J. Ma, Y. Shao, W. Zhao, G. Han, et al., ``Mamba policy: Towards efficient 3d diffusion policy with hybrid selective state models,'' arXiv preprint arXiv:2409.07163, 2024

    2024 arXiv

  38. [46]

    Y. Wang, Y. Zhang, M. Huo, R. Tian, X. Zhang, Y. Xie, C. Xu, P. Ji, W. Zhan, M. Ding, et al., ``Sparse diffusion policy: A sparse, reusable, and flexible policy for robot learning,'' arXiv preprint arXiv:2407.01531, 2024

    2024 arXiv

  39. [47]

    B. Chen, D. M. Monso, Y. Du, M. Simchowitz, R. Tedrake, and V. Sitzmann, ``Diffusion forcing: Next-token prediction meets full-sequence diffusion,'' arXiv preprint arXiv:2407.01392, 2024

    2024 arXiv

  40. [48]

    C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, ``Diffusion policy: Visuomotor policy learning via action diffusion,'' arXiv preprint arXiv:2303.04137, 2023

    2023 arXiv

  41. [49]

    Y. Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, ``3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,'' in ICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024

    2024

  42. [50]

    T.-W. Ke, N. Gkanatsios, and K. Fragkiadaki, ``3d diffuser actor: Policy diffusion with 3d scene representations,'' arXiv preprint arXiv:2402.10885, 2024

    2024 arXiv

  43. [51]

    Reuss, \"O

    M. Reuss, \"O . E. Ya g murlu, F. Wenzel, and R. Lioutikov, ``Multimodal diffusion transformer: Learning versatile behavior from multimodal goals,'' in First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024, 2024

    2024

  44. [52]

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, ``Rdt-1b: a diffusion foundation model for bimanual manipulation,'' arXiv preprint arXiv:2410.07864, 2024

    2024 arXiv

  45. [53]

    J. Wen, Y. Zhu, J. Li, Z. Tang, C. Shen, and F. Feng, ``Dexvla: Vision-language model with plug-in diffusion expert for general robot control,'' arXiv preprint arXiv:2502.05855, 2025

    2025 arXiv

  46. [54]

    J. Wen, M. Zhu, Y. Zhu, Z. Tang, J. Li, Z. Zhou, C. Li, X. Liu, Y. Peng, C. Shen, et al., ``Diffusion-vla: Scaling robot foundation models via unified diffusion and autoregression,'' arXiv preprint arXiv:2412.03293, 2024

    2024 arXiv

  47. [55]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, ``Attention is all you need,'' in Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garn...

    2017

  48. [56]

    Dasari, O

    S. Dasari, O. Mees, S. Zhao, M. K. Srirama, and S. Levine, ``The ingredients for robotic diffusion transformers,'' arXiv preprint arXiv:2410.10088, 2024

    2024 arXiv

  49. [57]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., ``Learning transferable visual models from natural language supervision,'' in International conference on machine learning. 1em plus 0.5em minus 0.4em PMLR, 20...

    2021

  50. [58]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al., ``Dinov2: Learning robust visual features without supervision,'' arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  51. [59]

    J. Li, D. Li, S. Savarese, and S. Hoi, ``Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,'' in International conference on machine learning. 1em plus 0.5em minus 0.4em PMLR, 2023, pp. 19\,730--19\,742

    2023

  52. [60]

    Perez, F

    E. Perez, F. Strub, H. De Vries, V. Dumoulin, and A. Courville, ``Film: Visual reasoning with a general conditioning layer,'' in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018

    2018

  53. [61]

    J. Ho, A. Jain, and P. Abbeel, ``Denoising diffusion probabilistic models,'' in Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, H. Larochelle, M. Ranzato, R. Hads...

    2020

  54. [62]

    J. Song, C. Meng, and S. Ermon, ``Denoising diffusion implicit models,'' in 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . 1em plus 0.5em minus 0.4em OpenReview.net, 2021. [Online]. Available: https://openreview.net...

    2021

  55. [63]

    Loshchilov and F

    I. Loshchilov and F. Hutter, ``Decoupled weight decay regularization,'' in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . 1em plus 0.5em minus 0.4em OpenReview.net, 2019. [Online]. Available: https://openreview.net/fo...

    2019

  56. [64]

    J. Gu, F. Xiang, X. Li, Z. Ling, X. Liu, T. Mu, Y. Tang, S. Tao, X. Wei, Y. Yao, et al., ``Maniskill2: A unified benchmark for generalizable manipulation skills,'' arXiv preprint arXiv:2302.04659, 2023

    2023 arXiv

  57. [65]

    T. Mu, Z. Ling, F. Xiang, D. Yang, X. Li, S. Tao, Z. Huang, Z. Jia, and H. Su, ``Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations,'' arXiv preprint arXiv:2107.14483, 2021

    2021 arXiv

  58. [66]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi \`e re, N. Goyal, E. Hambro, F. Azhar, et al., ``Llama: Open and efficient foundation language models,'' arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.