Pith. sign in

REVIEW 4 major objections 6 minor 34 references

CapsDT: Diffusion-Transformer for Capsule Robot Manipulation

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CapsDT, a vision-language diffusion transformer, controls a magnetic capsule robot in a stomach simulator, and the authors report it beats generalist robot baselines by 21.25 percentage points on average.

desk verdict The main results in Table II are impossible under the stated trial counts, so the SOTA claim is unsupported as written, but the application area and dataset have real value. read the letter →

arxiv 2506.16263 v1 pith:QGJ27ADO submitted 2025-06-19 cs.RO cs.AI

Xiting He , Mingwu Su , Xinqi Jiang , Long Bai , Jiewen Lai , Hongliang Ren This is my paper · ORCID
classification cs.ROcs.AI
keywords vision-language-actiondiffusiontransformercapsuleendoscopymagneticactuationrobotmanipulationimitationlearningstomachsimulator
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a generalist vision-language-action policy, pre-trained at scale and fine-tuned on a small collection of capsule-robot demonstrations, can control a magnetically actuated capsule endoscope in a stomach. Its model, CapsDT, ingests camera images and natural-language instructions and outputs motion commands for a robotic arm that moves an external magnet. On four tasks in a silicone stomach simulator, the authors report a 26.25 percent average success rate, 21.25 percentage points above the best baseline. If the result holds, operators could drive capsule endoscopy with typed or spoken instructions rather than manual joystick control, and the same recipe could transfer to other magnetic medical robots.

What carries the argument

The load-bearing machinery is the conditional diffusion policy: a learnable denoising network $f_\theta(\ell, o_t, \tilde a_t, k)$ that predicts the clean action $a_t^0$ from a noisy action sample $\tilde a_t$, conditioned on language $\ell$ and observation $o_t$, trained by the mean-squared error of equation (3). Actions are predicted as chunks $a_{t:t+T_a}$ to enforce temporal consistency and reduce accumulated error, and each modality is encoded into a shared latent space before the diffusion transformer mixes them. Physically, the action commands move a 7-degree-of-freedom arm carrying a permanent magnet, and the force on the capsule's internal dipole follows the magnetic dipole-dipole law $F_m = 3\mu_0 \|M\| \|m\|/(2\pi \|r\|^4)$, so the learned policy effectively controls a magnetic field rather than a rigid link.

What would settle it

Re-run the four tasks with at least 30 trials per condition, scoring each trial as binary success or failure, and verify the reported 66 percent for 'Navigate to Esophagus' is consistent with a 5-trial count; if the larger-sample success rates no longer exceed the baselines by the reported margin, the state-of-the-art claim is not supported.

Watch

Extended reading notes

Core claim

CapsDT is a diffusion-transformer policy trained to model the conditional distribution of robot actions given language and vision, rather than learning a single deterministic action. At inference it samples a noisy action chunk and denoises it over $K$ steps, conditioned on encoded images from two cameras and a text instruction, emitting a 7-dimensional action vector of translational and rotational velocities plus a gripper signal. The authors claim this design, after pre-training and a short fine-tuning phase on a self-collected dataset of around 1,000 trajectories with LLM-rewritten instructions, reaches state-of-the-art success rates across four endoscopy tasks, including a 40 percent rate on the rotation task and 20 percent on the two most difficult combined tasks, while two open-source generalist robot policies score at or near zero on the hardest conditions. Ablations attribute the result to pre-training, diffusion modeling, and the fine-tuned model size.

Load-bearing premise

The central claim depends on treating the reported success rates as accurate measures of model ability, even though tasks are evaluated with only 5 or 10 trials each, where a single trial outcome shifts a rate by 10 to 20 percentage points.

Editorial extensions

If this is right

  • One policy, not a stack of separate controllers, can handle navigation, rotation, water-level view adjustment, and the combination of both, using a single action space.
  • A few dozen demonstrations per skill (5 to 30) are enough to adapt a pre-trained VLA model to a new robot embodiment, which matters where collecting data is expensive.
  • Diffusion-based action generation beats deterministic regression on tasks with multi-modal action distributions, and pre-training on broad data enables zero-shot transfer to unseen instructions and water conditions.
  • If the simulator results transfer to real anatomy, natural-language command of capsule endoscopy becomes a plausible interface for diagnostics and therapy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An unstated corollary is that the same pre-train-then-fine-tune recipe should transfer to other magnetically actuated medical robots, since the policy only sees images, text, and a 7-dimensional action vector.
  • The paper's evaluation would be a stronger benchmark with per-trial binary logs and confidence intervals; at 5 trials per condition, the published percentages are unstable, and the navigation numbers contain a possible internal inconsistency.
  • A natural stress test is to vary magnet size, capsule dipole moment, or stomach geometry and check whether success degrades smoothly, since the magnetic force law fixes a physical prior that the policy must respect.
  • The LLM-rewritten instruction set could also be used to measure instruction-following robustness directly, for example by comparing performance on seen versus unseen paraphrases.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes CapsDT, a diffusion-transformer vision-language-action policy for controlling a magnetically actuated capsule endoscopy robot in a stomach simulator. The system comprises a 7-DoF KUKA arm holding a permanent magnet, a capsule robot with an internal magnetic dipole, and four progressively harder tasks: navigation, rotation, view adjustments in changing water lines, and view adjustments with rotation. The authors claim a self-collected dataset of over 1,000 trajectories, a 26.25% average success rate for CapsDT, and a 21.25% improvement over Octo and OpenVLA baselines. The main claim is that CapsDT is the first diffusion-based endoscopy capsule robotic model and achieves state-of-the-art performance.

Significance. If the reported results were reliable, the paper would make a useful contribution: it brings diffusion-transformer VLA policies to capsule endoscopy, provides a concrete hardware setup and task taxonomy, and compares against established generalist policies. The architecture sensibly follows the RDT design, and the use of DPM-Solver++ for real-time inference is a practical choice. However, the empirical evaluation contains internal inconsistencies in the central success-rate table, so the significance of the claimed SOTA result cannot be assessed from the manuscript as written.

major comments (4)
  1. [V.A, Table II] The success rates reported in Table II are not attainable under the stated trial counts. Navigation is tested with 5 trials, yet CapsDT is reported as 66%, 50%, and 25% for the three sub-tasks; with 5 binary trials the only possible rates are multiples of 20 percentage points. Similarly, 'View adjustments' reports 70% and 'View adjustments with rotation' reports 60% for conditions described as 5 trials per water line. Since the headline 26.25% average and the 21.25% improvement over baselines (Abstract, Section V.B) are computed from these impossible numbers, the central SOTA claim is unsupported as written. The authors must report exact per-trial outcomes, correct trial counts, and confidence intervals, or rerun the evaluation with a protocol that can support the claimed effect sizes.
  2. [I, III.A, V.A] The paper claims a dataset of 'over 1k trajectories' (Introduction) and '1,000 trajectories' (Section III.A), but Section V.A lists 12 + 20 + 10 + 10 + 10 + 5 + 5 + 5 = 77 fine-tuning demonstrations. The relationship between the 1,000 trajectories and the 77 demos is never explained. If the 1,000 trajectories constitute the pre-training set and the 77 are the fine-tuning set, this must be stated explicitly with a clear account of how each set is used; otherwise the dataset claim is contradicted by the experimental section.
  3. [V.A, Table II] The baseline evaluation is underspecified. The manuscript does not state whether Octo and OpenVLA were fine-tuned on the same 77 demonstrations, with the same camera views, action chunking, proprioceptive inputs, and evaluation trials, nor are any hyperparameters or fine-tuning durations given for the baselines. Without this information, the comparison in Table II is not interpretable and the claimed average improvement of 21.25% cannot be verified.
  4. [V.C, Table III] The ablation study is difficult to interpret because the variants are not fully defined and no trial counts or confidence intervals are reported. In particular, the sentence 'LLM takes the position of diffusion modeling in CapsDT (Regress)' is unclear, and the 'Pretrained' variant is said to be not fine-tuned yet still achieves nonzero success rates. The authors should rewrite the variant definitions, specify how each is trained and evaluated, and report per-task trial counts and uncertainties.
minor comments (6)
  1. [Table II] In the Rotation block, 'CapsD (Ours)' should be 'CapsDT (Ours)' for consistency.
  2. [V.A] 'Pour View adjustments are tested with 5 trials' appears to be a typo for 'View adjustments'; please correct this.
  3. [I] The contributions list contains the typo 'endocopy' instead of 'endoscopy'.
  4. [IV] SigLIP is attributed to reference [29], which is the CLIP paper by Radford et al.; the proper SigLIP reference (Zhai et al., 2023) should be cited instead.
  5. [References] References [13] and [14] appear to be incomplete duplicate entries of [11] and [12]; they should be removed or corrected.
  6. [III.A] The paper states that textual augmentation generates 100 distinct instruction sets, but Section V.A's fine-tuning list does not indicate where these augmented instructions are used; please clarify their role in training and evaluation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical success rates are measured against external baselines and are not forced by the model's own equations or by self-citation.

full rationale

The central claim is empirical: CapsDT achieves state-of-the-art success rates on four capsule endoscopy tasks in a stomach simulator, compared against Octo and OpenVLA. These baselines are external open-source policies, not the authors' own prior results, and the success rates are obtained by executing the trained policy on test trials rather than by evaluating a fitted quantity. The diffusion objective in Eq. (3) trains the denoiser to predict clean actions from noisy actions, but the reported success rates are not defined in terms of that objective's training loss, so there is no self-definitional reduction. The architecture is explicitly adapted from RDT [6], an external prior work, and the citations to the authors' own endoscopy papers appear in the introduction as background motivation, not as justifications for the claimed performance. No uniqueness theorem, ansatz, or fitted parameter is smuggled in via self-citation. The manuscript does contain a serious internal inconsistency in the experimental protocol (5-trial Navigation conditions cannot yield 66%, 50%, or 25% under binary per-trial scoring), but that is a data-reporting or metric-definition error, not a circularity: an unreliable measurement is not the same as a prediction that is equivalent to its inputs by construction. Therefore no circular step is exhibited, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim is an empirical benchmark result, not an analytical derivation. There are no derived constants or invented entities. The main unstated premises are the physical fidelity of the silicone stomach and magnetic model, and the sufficiency of the small demonstration set.

assumptions (4)
  • domain assumption The silicone gastric model is a valid physical proxy for the human stomach for the purpose of measuring capsule control success.
    The entire evaluation is conducted inside a silicone model (Section III), and clinical relevance is asserted from these results without in-vivo validation.
  • domain assumption Equation (1), the point-dipole magnetic force model from [21], accurately describes the forces on the capsule in the experimental setup.
    The system design uses this equation to justify magnetic actuation, but no direct force validation is reported.
  • standard math Diffusion model training and sampling theory from [23] and [26] is assumed correct; no novel theory is introduced.
    Equations (2) and (3) are standard DDPM/diffusion policy equations taken from cited prior work.
  • domain assumption Joystick demonstrations with the specified task definitions provide sufficient coverage of the target behavior distribution.
    The model is trained by imitation on 77 demonstrations; the paper does not quantify coverage or state randomizations per demonstration.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CapsDT: Diffusion-Transformer for Capsule Robot Manipulation." pith.science (2026). https://pith.science/paper/QGJ27ADO

@misc{pith2026250616263,
  author       = {Pith},
  title        = {Pith review of: CapsDT: Diffusion-Transformer for Capsule Robot Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QGJ27ADO}},
  note         = {Machine review of arXiv:2506.16263}
}
read the original abstract

Vision-Language-Action (VLA) models have emerged as a prominent research area, showcasing significant potential across a variety of applications. However, their performance in endoscopy robotics, particularly endoscopy capsule robots that perform actions within the digestive system, remains unexplored. The integration of VLA models into endoscopy robots allows more intuitive and efficient interactions between human operators and medical devices, improving both diagnostic accuracy and treatment outcomes. In this work, we design CapsDT, a Diffusion Transformer model for capsule robot manipulation in the stomach. By processing interleaved visual inputs, and textual instructions, CapsDT can infer corresponding robotic control signals to facilitate endoscopy tasks. In addition, we developed a capsule endoscopy robot system, a capsule robot controlled by a robotic arm-held magnet, addressing different levels of four endoscopy tasks and creating corresponding capsule robot datasets within the stomach simulator. Comprehensive evaluations on various robotic tasks indicate that CapsDT can serve as a robust vision-language generalist, achieving state-of-the-art performance in various levels of endoscopy tasks while achieving a 26.25% success rate in real-world simulation manipulation.

Figures

Figures reproduced from arXiv: 2506.16263 by the authors.

Figure 1
Figure 1. Endoscopy Capsule Robot System. CapsDT generates control commands for the robotic arm to adjust the magnet’s position and orienta￾tion, creating a controlled magnetic field that actuates the endoscopy capsule robot, enabling its navigation within the silicone stomach. encode both translational velocities (∆x,∆y,∆z in mm/s) and rotational velocities (∆θroll,∆θpitch,∆θyaw in rad/s), along with a normalized gripper act… view at source ↗
Figure 2
Figure 2. CapsDT framework. For multi-robot training, the action spaces of different robots are integrated into a single action environment. As denoising inputs, the following inputs are used: proprioception, noisy action chunk, control frequency, and diffusion time step; as conditions, picture inputs (a collection of images from the gripper camera and the exterior); and as conditions, language inputs. Denoised action is the … view at source ↗
Figure 3
Figure 3. Task definitions and visualizations. For 4 challenging tasks, we describe their language instruction, randomization, and definitions of each sub-task. The figures include the view of the gripper, exterior, and capsule robot. The photos of the gripper and the exterior camera will be the inputs of CapsDT. The lights in the picture realistically represent the position of the capsule robot. manipulation, such as Octo [3… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 20 canonical work pages

  1. [1]

    Rethinking exemplars for continual semantic segmentation in endoscopy scenes: Entropy- based mini-batch pseudo-replay,

    G. Wang, L. Bai, Y . Wu, T. Chen, and H. Ren, “Rethinking exemplars for continual semantic segmentation in endoscopy scenes: Entropy- based mini-batch pseudo-replay,”Computers in Biology and Medicine, vol. 165, p. 107412, 2023

    work page 2023

  2. [2]

    Endoscopic diagnosis and management of esoph- agogastric variceal hemorrhage: European society of gastrointestinal endoscopy (esge) guideline,

    I. M. Gralnek, M. C. Duboc, J. C. Garcia-Pagan, L. Fuccio, J. G. Karstensen, T. Hucl, I. Jovanovic, H. Awadie, V . Hernandez-Gea, M. Tantauet al., “Endoscopic diagnosis and management of esoph- agogastric variceal hemorrhage: European society of gastrointestinal endoscopy (esge) guideline,”Endoscopy, vol. 54, no. 11, pp. 1094– 1120, 2022

    work page 2022

  3. [3]

    Surgical-vqla: Trans- former with gated vision-language embedding for visual question localized-answering in robotic surgery,

    L. Bai, M. Islam, L. Seenivasan, and H. Ren, “Surgical-vqla: Trans- former with gated vision-language embedding for visual question localized-answering in robotic surgery,” in2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 6859–6865

    2023

  4. [4]

    Wireless capsule endoscopy,

    G. Iddan, G. Meron, A. Glukhovsky, and P. Swain, “Wireless capsule endoscopy,”Nature, vol. 405, no. 6785, pp. 417–417, 2000

    work page 2000

  5. [5]

    Endoood: Uncertainty-aware out-of-distribution detection in capsule endoscopy diagnosis,

    Q. Tan, L. Bai, G. Wang, M. Islam, and H. Ren, “Endoood: Uncertainty-aware out-of-distribution detection in capsule endoscopy diagnosis,” in2024 IEEE International Symposium on Biomedical Imaging (ISBI). IEEE, 2024, pp. 1–5

    work page 2024

  6. [6]

    Rdt-1b: a diffusion foundation model for bimanual manipulation,

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “Rdt-1b: a diffusion foundation model for bimanual manipulation,”arXiv preprint arXiv:2410.07864, 2024

    arXiv 2024

  7. [7]

    V 2-sfmlearner: Learning monocular depth and ego-motion for multimodal wireless capsule endoscopy,

    L. Bai, B. Cui, L. Wang, Y . Li, S. Yao, S. Yuan, Y . Wu, Y . Zhang, M. Q.-H. Meng, Z. Liet al., “V 2-sfmlearner: Learning monocular depth and ego-motion for multimodal wireless capsule endoscopy,” IEEE Transactions on Automation Science and Engineering, 2025

    work page 2025

  8. [8]

    Endovla: Dual-phase vision-language-action model for autonomous tracking in endoscopy,

    C. K. Ng, L. Bai, G. Wang, Y . Wang, H. Gao, K. Yuan, C. Jin, T. Zeng, and H. Ren, “Endovla: Dual-phase vision-language-action model for autonomous tracking in endoscopy,”arXiv preprint arXiv:2505.15206, 2025

    arXiv 2025

Show all 34 references
  1. [9]

    Deep reinforcement learning-based control for stomach coverage scanning of wireless capsule endoscopy,

    Y . Zhang, L. Bai, L. Liu, H. Ren, and M. Q.-H. Meng, “Deep reinforcement learning-based control for stomach coverage scanning of wireless capsule endoscopy,” in2022 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 2022, pp. 01–06

    2022

  2. [10]

    Endoscopy robotics: current and future applications,

    I. Bo ˇskoski and G. Costamagna, “Endoscopy robotics: current and future applications,”Digestive Endoscopy, vol. 31, no. 2, pp. 119– 124, 2019

    2019

  3. [11]

    Platform development for a paradigm shift in flexible endoscopy,

    L. L. Swanstrom, “Platform development for a paradigm shift in flexible endoscopy,”Gastroenterology, vol. 140, no. 4, pp. 1150–1154, 2011

    2011

  4. [12]

    Flexible robotic endoscopy for treating gastrointestinal neoplasms,

    K. Kume, “Flexible robotic endoscopy for treating gastrointestinal neoplasms,”World Journal of Gastrointestinal Endoscopy, vol. 15, no. 6, p. 434, 2023

    2023

  5. [13]

    Notes: Platform development for a paradigm shift in flexible endoscopy,

    S. LL, “Notes: Platform development for a paradigm shift in flexible endoscopy,”Gastroenterology, vol. 140, pp. 1150–1154, 2011

    2011

  6. [14]

    Flexible robotic endoscopy for treating gastrointestinal neo- plasms,

    K. K, “Flexible robotic endoscopy for treating gastrointestinal neo- plasms,”World Journal of Gastrointestinal Endoscopy, vol. 15, no. 6, p. 434, 2023

    2023

  7. [15]

    Initial clinical experience with a novel flexible endoscopic robot for transanal surgery,

    A. A. Morino M, Forcignan `o E, “Initial clinical experience with a novel flexible endoscopic robot for transanal surgery,”Techniques in Coloproctology, vol. 26, no. 4, pp. 301–308, 2022

    2022

  8. [16]

    A novel distal hybrid pneumatic/cable-driven continuum joint with variable stiffness capacity for flexible gastrointestinal endoscopy,

    X. Luo, D. Song, Z. Zhang, S. Wang, and C. Shi, “A novel distal hybrid pneumatic/cable-driven continuum joint with variable stiffness capacity for flexible gastrointestinal endoscopy,”Advanced Intelligent Systems, vol. 5, no. 6, p. 2200403, 2023

    2023

  9. [17]

    Robotic endoscope control via autonomous instrument tracking,

    C. Gruijthuijsen, L. C. Garcia-Peraza-Herrera, G. Borghesan, D. Rey- naerts, J. Deprest, S. Ourselin, T. Vercauteren, and E. Vander Poorten, “Robotic endoscope control via autonomous instrument tracking,” Frontiers in Robotics and AI, vol. 9, p. 832208, 2022

    2022

  10. [18]

    Develop- ment of reconfigurable electromagnetic actuation system with large workspaces: Design, optimization, and validation,

    M. Cai, Z. Qi, Y . Cao, X. Wu, T. Xu, and L. Zhang, “Develop- ment of reconfigurable electromagnetic actuation system with large workspaces: Design, optimization, and validation,”IEEE Transactions on Automation Science and Engineering, pp. 1–12, 2024

    2024

  11. [19]

    On reciprocally rotating magnetic actuation of a robotic capsule in unknown tubular environments,

    Y . Xu, K. Li, Z. Zhao, and M. Q.-H. Meng, “On reciprocally rotating magnetic actuation of a robotic capsule in unknown tubular environments,”IEEE Transactions on Medical Robotics and Bionics, vol. 3, no. 4, pp. 919–927, 2021

    2021

  12. [20]

    Future directions for robotic en- doscopy–artificial intelligence (ai), three-dimensional (3d) imaging, and natural orifice transluminal endoscopic surgery,

    A. H. Simsek C, Kaan H L, “Future directions for robotic en- doscopy–artificial intelligence (ai), three-dimensional (3d) imaging, and natural orifice transluminal endoscopic surgery,”Techniques and innovations in gastrointestinal endoscopy, vol. 25, no. 1, pp. 95–101, 2023

    2023

  13. [21]

    Five-degree-of-freedom manip- ulation of an untethered magnetic device in fluid using a single permanent magnet with application in stomach capsule endoscopy,

    A. W. Mahoney and J. J. Abbott, “Five-degree-of-freedom manip- ulation of an untethered magnetic device in fluid using a single permanent magnet with application in stomach capsule endoscopy,” The International Journal of Robotics Research, vol. 35, no. 1-3, pp. 129–147, 2016

    2016

  14. [22]

    Deepseek-v2: A strong, economi- cal, and efficient mixture-of-experts language model,

    A. Liu, B. Feng, B. Wang, B. Wang, B. Liu, C. Zhao, C. Dengr, C. Ruan, D. Dai, D. Guoet al., “Deepseek-v2: A strong, economi- cal, and efficient mixture-of-experts language model,”arXiv preprint arXiv:2405.04434, 2024

    2024 arXiv

  15. [23]

    Diffusion policy: Visuomotor policy learning via ac- tion diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,”The International Journal of Robotics Research, p. 02783649241273668, 2023

    2023

  16. [24]

    Capturing the spatio-temporal continu- ity for video semantic segmentation,

    Y . Han, X. Chen, and A. Wu, “Capturing the spatio-temporal continu- ity for video semantic segmentation,”IET Image Processing, vol. 13, no. 14, pp. 2813–2820, 2019

    2019

  17. [25]

    C. C. de Wit, B. Siciliano, and G. Bastin,Theory of robot control. Springer Science & Business Media, 2012

    2012

  18. [26]

    Improved denoising diffusion prob- abilistic models,

    A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion prob- abilistic models,” inInternational conference on machine learning. PMLR, 2021, pp. 8162–8171

    2021

  19. [27]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,”arXiv preprint arXiv:2304.13705, 2023

    2023 arXiv

  20. [28]

    Fourier features let networks learn high frequency functions in low dimen- sional domains,

    M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Ragha- van, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimen- sional domains,”Advances in neural information processing systems, vol. 33, ...

    2020

  21. [29]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  22. [30]

    Bart: Denoising sequence-to- sequence pre-training for natural language generation, translation, and comprehension,

    M. Lewis, Y . Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V . Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to- sequence pre-training for natural language generation, translation, and comprehension,”arXiv preprint arXiv:1910.13461, 2019

    1910 arXiv

  23. [31]

    Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models,

    C. Lu, Y . Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models,” arXiv preprint arXiv:2211.01095, 2022

    2022 arXiv

  24. [32]

    Octo: An open-source generalist robot policy,

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xuet al., “Octo: An open-source generalist robot policy,”arXiv preprint arXiv:2405.12213, 2024

    2024 arXiv

  25. [33]

    Open- vla: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketiet al., “Open- vla: An open-source vision-language-action model,”arXiv preprint arXiv:2406.09246, 2024

    2024 arXiv

  26. [34]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.