Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Mini Diffuser: Fast Multi-task Diffusion Policy Training Using Two-level Mini-batches

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Mini Diffuser trains multi-task diffusion robot policies at about 5 percent of the usual cost, while keeping roughly 95 percent of the performance.

desk verdict A practical, believable efficiency method for training multi-task diffusion policies on a single GPU; the headline time ratio is not fully controlled, but the core mechanism checks out. read the letter →

arxiv 2505.09430 v2 pith:5X65VTB3 submitted 2025-05-14 cs.RO cs.LG

Yutong Hu , Pinhao Song , Kehan Wen , Renaud Detry This is my paper · ORCID
classification cs.ROcs.LG
keywords multi-taskroboticmanipulationdiffusionpolicytrainingefficiencytwo-levelmini-batchingRLBenchvision-language-actiontransformerattentionmaskingimitationlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that diffusion-based robotic policies are expensive to train for the wrong reason: unlike image diffusion, the action output is low-dimensional while the conditioning input (visual scene, language, proprioception) is high-dimensional. Mini Diffuser exploits this asymmetry with two-level mini-batching, pairing many noised action samples with each shared condition instead of the usual one-to-one sampling. On the 18-task RLBench benchmark it reports 95.4 percent of the average success of 3D Diffuser Actor while using 4.8 percent of the training time and 6.6 percent of the memory, allowing end-to-end training on a single RTX 4090 in about 13 hours. If correct, this removes a major practical barrier to training multi-task diffusion policies, making them feasible for labs without multi-GPU clusters.

What carries the argument

The mechanism is two-level mini-batching formalized in equations (6) through (10): sample B state-action pairs, then for each pair sample M independent noise-timestep tuples, yielding a flattened batch of B times M noised actions that share condition tokens. The supporting architecture replaces standard transformer self-attention with a masked attention matrix M_QK in equation (13), where action tokens attend within their own sample and to shared visual and context tokens, while condition tokens do not attend to actions; per-sample FiLM modulation in equation (18) conditions each sample on its noise step, and a local 3D kernel-based query in equation (17) gathers spatial context around query positions. These non-invasive components keep samples independent while processing many samples per condition in one forward pass, which is what cuts per-sample compute and memory.

What would settle it

A direct test is to fix the Level-1 batch and increase the number of action tokens per sample (for example, predicting longer action horizons) while holding M fixed; if per-step time and memory scale with the total action-token count instead of staying near the condition-token cost, the core efficiency claim fails.

Watch

Extended reading notes

Core claim

The central claim is that in action diffusion the condition is much larger than the target, so re-encoding the same condition for every noised action sample wastes nearly all the training cost. Mini Diffuser organizes training into Level-1 batches of B state-action pairs and Level-2 batches of M independent noise-timestep samples per condition, then flattens them into one batch of B times M samples while encoding each condition only once. A masked global attention mechanism lets action tokens attend to their own tokens and to shared condition tokens, but never to other action samples, and condition tokens never attend to actions, preventing information leakage. The paper reports that with M equal to 64, it processes 64 times more samples per gradient step for roughly 3 percent extra memory and 7 percent extra time, and that the resulting policy reaches 77.6 percent average success on RLBench versus 81.3 percent for 3D Diffuser Actor, with real-world experiments showing preserved multimodal behavior.

Load-bearing premise

The speed-up depends on condition tokens dominating the compute and memory cost of a training batch, so adding more action samples per condition adds only a few percent overhead; if action sequences grow long enough to dominate the token count, the savings shrink or disappear.

Editorial extensions

If this is right

  • Multi-task diffusion policies become trainable end-to-end on a consumer GPU in about 13 hours instead of requiring multi-GPU clusters for days.
  • Effective batch size can be scaled by increasing M without a proportional increase in memory or compute, accelerating convergence in gradient steps.
  • The efficiency gain is orthogonal to inference accelerations like DDIM or flow matching, so deployment speed can be improved independently of training cost.
  • The approach preserves diffusion's multimodal action generation, as shown in real-world trials where the same visual scene with different language instructions produces distinct valid actions.
  • The remaining small performance gap relative to state-of-the-art methods shrinks further on several individual tasks, where Mini Diffuser outperforms 3D Diffuser Actor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same two-level batching idea could accelerate other conditional generative models whose target space is much smaller than the condition space, such as long-context control or video-conditioned action prediction.
  • A testable extension is to measure how the speed-up degrades as the action token sequence length L grows; if action tokens start to dominate the batch, the condition-reuse premise fails and conventional batching should be restored.
  • Because Level-2 batches share conditions, they reduce gradient diversity; reinvesting the saved compute into more Level-1 conditions or more demonstrations may close more of the remaining 5 percent performance gap than pushing M to very large values.
  • The paper's architecture suggests a general recipe: when the target is low-dimensional, batch over target samples per condition and design the network with masks and per-sample modulation to prevent leakage across those samples.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript introduces Mini-Diffuser, a diffusion policy for multi-task vision-language manipulation that pairs B state-action conditions with M noised action samples per condition, a scheme the authors call two-level minibatching. To make this efficient, the architecture uses masked global attention to prevent inter-sample leakage, a local kernel-based query for 3D grounding, and per-sample FiLM modulation, with a PTv3-style U-Net backbone and a decoupled action head. On RLBench, the authors report 77.6% average success (95.4% of 3D Diffuser Actor) with normalized training time 1.9 vs 39 and memory 16 vs 240 GB, and they include real-world success rates on twelve tasks. Ablations vary the Level-2 batch size M and remove components such as 3D RoPE, the PTv3 backbone, and the local convolution.

Significance. If the efficiency claims hold, the result is practically significant: it suggests near-state-of-the-art multi-task manipulation policies can be trained on a single consumer GPU in about 13 hours, which would substantially lower the barrier to reproducing and extending diffusion-policy methods. The paper's strengths include evaluation on an external benchmark, real-world experiments, an ablation that isolates the Level-2 batching contribution, and Fig. 4, which shows faster convergence per gradient step as M increases. However, the headline time and memory ratios in Table 1 are not controlled comparisons: the memory column appears to aggregate across multiple GPUs for baselines, and the time comparison does not report total gradient steps or epochs. The central efficiency claim is plausible but needs to be substantiated with step counts, epochs, and per-device memory before it can be accepted at face value.

major comments (3)
  1. [Table 1; §5.1.3] The headline efficiency ratios (4.8% time, 6.6% memory) are not controlled comparisons. The memory column appears to be aggregate across multiple GPUs: PerAct trained on 8 V100s reports 128 GB (16 GB per GPU), and 3D Diffuser Actor trained on 6 A100s reports 240 GB, while Mini-Diffuser reports 16 GB on a single RTX 4090. Comparing aggregate multi-GPU memory with single-GPU memory conflates algorithmic memory savings with hardware scaling. Please report per-device peak memory and total aggregate memory for every method, and state explicitly whether the normalized time unit already accounts for GPU count and type.
  2. [§5.1.3; §5.2.1] The 4.8% time claim is not controlled for training budget. The paper does not report the total number of gradient steps or epochs for the Mini-Diffuser run in Table 1, nor the corresponding numbers for 3D Diffuser Actor. Since Table 3 shows only 106.3% per-iteration time for B=100, M=64 relative to B=100, M=1, the roughly 20x reduction in normalized time must arise primarily from requiring far fewer optimizer steps; Fig. 4 suggests this, but the actual step counts, data repeats, and learning-rate schedules are missing. Please provide a same-hardware or same-step-budget comparison, or at minimum report the total steps, epochs, and unique demonstrations for every run in Table 1.
  3. [§5.2.1; Eq. (10)] The statement that M=64 processes 64 times more training samples per step is not equivalent to seeing 64 times more independent demonstrations, because Level-2 samples reuse the same condition and the paper itself calls them 'fake' batches. Please report the number of unique (state, action) pairs processed per step for each configuration, and report wall-clock time to reach a fixed success threshold for M=1, 4, 16, 64, 128, and 256, so that the trade-off between per-step cost and convergence speed can be evaluated directly.
minor comments (6)
  1. [Abstract; Table 1; §5.1.3] The performance ratio appears as 95.4% in Table 1, 95.6% in the text of §5.1.3, and roughly 95% in the abstract; these numbers should be made consistent.
  2. [§2.1] There are several typos in this section, including 'foundition', 'spaitial', and 'set if tasks'; a careful proofreading pass is needed.
  3. [Table 3] The caption says 'Duo-Level batches' instead of 'Two-Level batches', and the column header 'per decent' should be 'per descent'; the meaning of the first column header 'consistent' is unclear and should be defined.
  4. [Figure 2 caption] The caption contains the typo 'baed' and the bubble percentages duplicate numbers in the legend; please clarify what is being plotted.
  5. [Table 4] Real-world results are based on only 10 trials per task and no error bars are reported; the table caption should state this limitation explicitly.
  6. [§1; §5.2 (Table 3)] The introduction claims the architecture adds 'no additional memory usage or computational overhead,' but Table 3 reports 102.2% memory and 106.3% time at B=100, M=64 relative to M=1; please rephrase to 'near-zero' or 'a few percent' overhead.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Mini-Diffuser's efficiency claims are measured against external benchmarks, and no equation reduces its predictions to fitted inputs or self-citations.

full rationale

No circularity found. The paper's central claim (Sec. 5.1.3) is an empirical comparison against RLBench baselines and real-world tasks; the efficiency ratios are measured, not derived from parameters fitted to the headline success rate. The Level-2 loss in Eq. (10) is the standard conditional denoising objective with shared state conditions, and the claimed compute/memory savings follow from the masked-attention architecture (Sec. 4.1) rather than from defining the saved cost into existence. The per-step cost claim is verified in Table 3, where M=64 raises iteration time to 106.3% and memory to 102.2% relative to M=1, and the Level-2 batch size M is a hyperparameter selected in ablations, not a fitted constant. There is no load-bearing self-citation: the paper builds on external baselines such as 3D Diffuser Actor and PTv3, and it explicitly states that pretrained PTv3 weights are not reused. The acknowledged statement in Sec. 5.2.1 that Level-2 batches are 'fake' because they reuse the same condition is a stated limitation on sample diversity, not a circular reduction of the efficiency claim. The concern that the Table 1 time comparison may not control for gradient steps or hardware configuration is an experimental-control and correctness risk, not circularity, because no equation or fitted parameter makes the headline ratio true by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard diffusion training, the low-dimensional action assumption, the masked-attention design, the decoupled action head, and the benchmark proxy. Hyperparameters M=64, B=100, and the voxel bin size 24 are hand-chosen; no new physical entities are introduced.

free parameters (3)
  • Level-2 batch size M = 64
    Number of noised action samples paired with each condition; chosen from ablation in Sec. 5.2.1 where larger M accelerates convergence but saturates beyond 128 (Fig. 4, Table 3).
  • Level-1 batch size B = 100
    Number of state-action pairs per mini-batch; fixed in Sec. 5.2 for the main experiments.
  • Voxel downsampling factor = 24
    Bin size for local kernel-based query in Sec. 4.2; chosen by hand and affects the locality of spatial feature aggregation.
assumptions (5)
  • standard math The conditional denoising objective in Eq. (4) is a valid training target for the action diffusion policy.
    Follows from Ho et al. [1]; used without modification in Sec. 3.2.
  • domain assumption The action target is low-dimensional compared to the condition (dim(a) << dim(s)) for the target manipulation tasks.
    Assumed in Sec. 1 and Sec. 3.3; if false, the compute savings of reusing conditions shrink.
  • domain assumption The masked attention pattern in Eq. (13) prevents inter-sample leakage while giving each action token full access to condition tokens.
    Central architectural claim in Sec. 4.1; validated only empirically through ablations, not formally proven.
  • domain assumption RLBench success rate with 100 demonstrations per task and 300 test episodes is a reliable proxy for multi-task policy quality.
    Standard benchmark assumption used in Sec. 5.1; no proof that results transfer to other benchmarks.
  • domain assumption Denoising only end-effector position while classifying rotation and gripper states preserves the multimodal action distribution of full action diffusion.
    Design choice in Sec. 4.4.2; supported by real-world qualitative results in Sec. 5.3 but not by a quantitative comparison against full-pose denoising.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mini Diffuser: Fast Multi-task Diffusion Policy Training Using Two-level Mini-batches." pith.science (2026). https://pith.science/paper/5X65VTB3

@misc{pith2026250509430,
  author       = {Pith},
  title        = {Pith review of: Mini Diffuser: Fast Multi-task Diffusion Policy Training Using Two-level Mini-batches},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5X65VTB3}},
  note         = {Machine review of arXiv:2505.09430}
}
read the original abstract

We present a method that reduces, by an order of magnitude, the time and memory needed to train multi-task vision-language robotic diffusion policies. This improvement arises from a previously underexplored distinction between action diffusion and the image diffusion techniques that inspired it: In image generation, the target is high-dimensional. By contrast, in action generation, the dimensionality of the target is comparatively small, and only the image condition is high-dimensional. Our approach, \emph{Mini Diffuser}, exploits this asymmetry by introducing \emph{two-level minibatching}, which pairs multiple noised action samples with each vision-language condition, instead of the conventional one-to-one sampling strategy. To support this batching scheme, we introduce architectural adaptations to the diffusion transformer that prevent information leakage across samples while maintaining full conditioning access. In RLBench simulations, Mini-Diffuser achieves 95\% of the performance of state-of-the-art multi-task diffusion policies, while using only 5\% of the training time and 7\% of the memory. Real-world experiments further validate that Mini-Diffuser preserves the key strengths of diffusion-based policies, including the ability to model multimodal action distributions and produce behavior conditioned on diverse perceptual inputs. Code available at mini-diffuse-actor.github.io

Figures

Figures reproduced from arXiv: 2505.09430 by the authors.

Figure 1
Figure 1. Difference between image diffusion (bottom), state-of-art action diffusion [7] (middle) and our mini-diffuser (top). A semantically meaningful image is denoised from fully random pixels, while structured and meaningful actions are denoised from random samples. At token level, the denoised target (image) dominates token space in image diffusion (bottom row). By contrast, in action diffusion (middle and top rows), the… view at source ↗
Figure 2
Figure 2. Comparison with state of the arts in RLbench Peract-18 benchmark. Our method by far takes least time and memory to train, while maintain 95% of the performance of currently SOTA diffusion baed model. multimodal action-generation capabilities that diffusion models are known for, ensuring reliable performance across diverse perceptual inputs. 2 Related Works 2.1 Robot Learning from demonstration Earlier works on learn… view at source ↗
Figure 3
Figure 3. Mini-diffuser model structure. (a) During training phase, B samples of the states form a Level-1 batch, where M noise actions are sampled independently under the same state conditions, building a Level-2 batch. Tokens are flattened and feed into a multi-layer model contains Masked attention module, local query module, and FiLM layers. (b) During inference phase, denoising is applied only to the end-effector position… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Policy Learning Efficiency. The y-axis shows the proportion of generated actions with errors below specified thresholds (e.g., < 1cm or <3°), indicating successful diffusion. Increasing the Level-2 batch size accelerates convergence at the same number of gradient steps…
Figure 5
Figure 5. Figure 5: Real World Mini-Diffuser. We visualize 10 action candidates along the denoising trajectory, though only one is executed. Mini-Diffuser preserves the core strengths of diffusion-based actors. In the top row, when multiple actions are valid under the same instruction, Mi…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Robot Trajectron V3: A Probabilistic Shared Control Framework for SE(3) Manipulation

    cs.RO 2026-07 accept novelty 6.0 of 10

    RT-V3 learns a transformer-CVAE prior over multi-modal SE(3) trajectories conditioned on scene geometry and grasps, then continuously fuses it with noisy user twists via Bayesian posterior estimation for shared graspi...

  2. Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges

    cs.RO 2025-08 conditional novelty 4.0 of 10

    A survey that taxonomizes robotic manipulation policies trained by imitation learning, traces their evolution, and compiles benchmark comparisons.

Reference graph

Works this paper leans on

39 extracted references · 35 canonical work pages · cited by 2 Pith papers

  1. [1]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” inAdvances in Neural Information Processing Systems, vol. 33. Curran Associates, Inc., 2020, pp. 6840–6851

    work page 2020

  2. [2]

    High-resolution image synthe- sis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthe- sis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 684–10 695

    work page 2022

  3. [3]

    Is conditional gen- erative modeling all you need for decision making?

    A. Ajay, Y . Du, A. Gupta, J. B. Tenenbaum, T. S. Jaakkola, and P. Agrawal, “Is conditional gen- erative modeling all you need for decision making?” in The Eleventh International Conference on Learning Representations, Sep. 2022

    work page 2022

  4. [4]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,”Int. J. Rob. Res., p. 2783649241273668, Oct. 2024

    work page 2024

  5. [5]

    Planning with diffusion for flexible behavior synthesis,

    M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” Dec. 2022

    work page 2022

  6. [6]

    3D diffusion policy: Generalizable visuomotor policy learning via simple 3D representations,

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3D diffusion policy: Generalizable visuomotor policy learning via simple 3D representations,” in 2nd Workshop on Dexterous Manipulation: Design, Perception and Control (RSS), Jul. 2024

    work page 2024

  7. [7]

    3D Diffuser Actor: Policy Diffusion with 3D Scene Representations,

    T.-W. Ke, N. Gkanatsios, and K. Fragkiadaki, “3D Diffuser Actor: Policy Diffusion with 3D Scene Representations,” Conference on Robot Learning, vol. abs/2402.10885, Feb. 2024

    arXiv 2024

  8. [8]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in International Conference on Learning Representations, Oct. 2020

    work page 2020

Show all 39 references
  1. [9]

    Consistency models,

    Y . Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency models,” inProceedings of the 40th International Conference on Machine Learning. PMLR, Jul. 2023, pp. 32 211–32 252

    2023

  2. [10]

    Flowpolicy: Enabling fast and robust 3d flow-based policy via consistency flow matching for robot manipulation,

    Q. Zhang, Z. Liu, H. Fan, G. Liu, B. Zeng, and S. Liu, “Flowpolicy: Enabling fast and robust 3d flow-based policy via consistency flow matching for robot manipulation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, 2025, pp. 14 754–14 762

    2025

  3. [11]

    $ π_0$: A vision- language-action flow model for general robot control,

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky, “$ π_0$: A vis...

    2024

  4. [12]

    Alvinn: An autonomous land vehicle in a neural network,

    D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” Adv. Neural Inf. Process. Syst., vol. 1, 1988

    1988

  5. [13]

    Transporter networks: Rearranging the visual world for robotic manipulation,

    A. Zeng, P. Florence, J. Tompson, S. Welker, J. Chien, M. Attarian, T. Armstrong, I. Krasin, D. Duong, and V . Sindhwani, “Transporter networks: Rearranging the visual world for robotic manipulation,” in Conference on Robot Learning. PMLR, 2021, pp. 726–747

    2021

  6. [14]

    PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation,

    S. Chen, R. Garcia, C. Schmid, and I. Laptev, “PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation,” Conference on Robot Learning, vol. abs/2309.15596, Sep. 2023

    2023 arXiv

  7. [15]

    Towards generalizable vision-language robotic manipula- tion: A benchmark and LLM-guided 3D policy,

    R. Garcia, S. Chen, and C. Schmid, “Towards generalizable vision-language robotic manipula- tion: A benchmark and LLM-guided 3D policy,” Arxiv.org, vol. abs/2410.1345, Oct. 2024

    2024

  8. [16]

    Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data,

    A. Mandlekar, F. Ramos, B. Boots, S. Savarese, L. Fei-Fei, A. Garg, and D. Fox, “Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data,” in 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE,...

    2020

  9. [17]

    Autoregressive action sequence learning for robotic manipulation,

    X. Zhang, Y . Liu, H. Chang, L. Schramm, and A. Boularias, “Autoregressive action sequence learning for robotic manipulation,” IEEE Robot. Autom. Lett., 2025

    2025

  10. [18]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” Apr. 2023

    2023

  11. [19]

    SAM2Act: Integrating visual foundation model with a memory architecture for robotic manipulation,

    H. Fang, M. Grotz, W. Pumacay, Y . R. Wang, D. Fox, R. Krishna, and J. Duan, “SAM2Act: Integrating visual foundation model with a memory architecture for robotic manipulation,” Feb. 2025

    2025

  12. [20]

    Coarse-to-fine Q-attention: Efficient learning for visual robotic manipulation via discretisation,

    S. James, K. Wada, T. Laidlow, and A. J. Davison, “Coarse-to-fine Q-attention: Efficient learning for visual robotic manipulation via discretisation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 13 739–13 748

    2022

  13. [21]

    Perceiver-actor: A multi-task transformer for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Perceiver-actor: A multi-task transformer for robotic manipulation,” in Conference on Robot Learning. PMLR, 2023, pp. 785–799

    2023

  14. [22]

    Act3D: 3D feature field transformers for multi-task robotic manipulation,

    T. Gervet, Z. Xian, N. Gkanatsios, and K. Fragkiadaki, “Act3D: 3D feature field transformers for multi-task robotic manipulation,” in Proceedings of the 7th Conference on Robot Learning. PMLR, Dec. 2023, pp. 3949–3965

    2023

  15. [23]

    Rvt: Robotic view transformer for 3d object manipulation,

    A. Goyal, J. Xu, Y . Guo, V . Blukis, Y .-W. Chao, and D. Fox, “Rvt: Robotic view transformer for 3d object manipulation,” in Conference on Robot Learning. PMLR, 2023, pp. 694–710

    2023

  16. [24]

    RVT-2: Learning precise manipulation from few demonstrations,

    A. Goyal, V . Blukis, J. Xu, Y . Guo, Y .-W. Chao, and D. Fox, “RVT-2: Learning precise manipulation from few demonstrations,” in RSS 2024 Workshop: Data Generation for Robotics, Jul. 2024

    2024

  17. [25]

    RT-1: Robotics transformer for real-world control at scale,

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. J. Joshi, R. Julian, D. Kalashnikov, Y . Kuang, I. Leal, K.-H. Lee, S. Levine, Y . Lu, U. Malla, D....

    2023

  18. [26]

    RT-2: Vision-language-action models transfer web knowledge to robotic control,

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, P. Florence, C. Fu, M. G. Arenas, K. Gopalakrishnan, K. Han, K. Hausman, A. Herzog, J. Hsu, B. Ichter, A. Irpan, N. Joshi, R. Julian, D. Kalashnikov, Y . Kuang, I. L...

    2023

  19. [27]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0,

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, and A. Jain, “Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0,” in 2024 IEEE International Conference on Robotics and Automati...

    2024

  20. [28]

    Octo: An open-source generalist robot policy,

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, Y . L. Tan, L. Y . Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine, “Octo: An open-source generalist robot policy,” May 2024

    2024

  21. [29]

    OpenVLA: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn, “OpenVLA: An open-source vision-language-action model,” in 8th Annual C...

    2024

  22. [30]

    Goal-conditioned imitation learning using score- based diffusion policies,

    M. Reuss, M. Li, X. Jia, and R. Lioutikov, “Goal-conditioned imitation learning using score- based diffusion policies,” 2023

    2023

  23. [31]

    Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation,

    Z. Xian and N. Gkanatsios, “Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation,” in Conference on Robot Learning/Proceedings of Machine Learning Research. Proceedings of Machine Learning Research, 2023

    2023

  24. [32]

    Hierarchical diffusion policy for kinematics-aware multi-task robotic manipulation,

    X. Ma, S. Patidar, I. Haughton, and S. James, “Hierarchical diffusion policy for kinematics-aware multi-task robotic manipulation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 18 081–18 090

    2024

  25. [33]

    Boosting continuous control with consistency policy,

    Y . Chen, H. Li, and D. Zhao, “Boosting continuous control with consistency policy,” inProceed- ings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, ser. AAMAS ’24. Richland, SC: International Foundation for Autonomous Agents and Multiagent Sy...

    2024

  26. [34]

    RoFormer: Enhanced transformer with rotary position embedding,

    J. Su, M. Ahmed, Y . Lu, S. Pan, W. Bo, and Y . Liu, “RoFormer: Enhanced transformer with rotary position embedding,” Neurocomputing, vol. 568, p. 127063, Feb. 2024

    2024

  27. [35]

    Film: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “Film: Visual reasoning with a general conditioning layer,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, 2018

    2018

  28. [36]

    Point transformer v3: Simpler faster stronger,

    X. Wu, L. Jiang, P.-S. Wang, Z. Liu, X. Liu, Y . Qiao, W. Ouyang, T. He, and H. Zhao, “Point transformer v3: Simpler faster stronger,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 4840–4851

    2024

  29. [37]

    U-net: Convolutional networks for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-assisted Intervention – MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, Eds. Cham: Springer International Pub...

    2015

  30. [38]

    Rlbench: The robot learning benchmark & learning environment,

    S. James, Z. Ma, D. R. Arrojo, and A. J. Davison, “Rlbench: The robot learning benchmark & learning environment,” IEEE Robot. Autom. Lett., vol. 5, no. 2, pp. 3019–3026, 2020

    2020

  31. [39]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, and W.-Y . Lo, “Segment anything,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.