Pith. sign in

REVIEW 2 major objections 6 minor 68 references

Diffusion-Based Imaginative Coordination for Bimanual Manipulation

T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A diffusion policy trained to also predict future video frames outperforms ACT on bimanual tasks by 24.9%.

desk verdict The auxiliary video-prediction idea is solid and well ablated, but the headline ALOHA and real-world numbers have baseline and arithmetic problems that need correcting before the effect sizes are trusted. read the letter →

arxiv 2507.11296 v1 pith:LXK5WQAQ submitted 2025-07-15 cs.RO

Huilin Xu , Jian Ding , Jiakun Xu , Ruixiang Wang , Jun Chen , Jinjie Mai , Yanwei Fu , Bernard Ghanem
show 2 more authors
Feng Xu Mohamed Elhoseiny
This is my paper
classification cs.RO
keywords bimanualmanipulationimitationlearningdiffusionpolicyvideopredictionactionchunkingworldmodelstransformerrobot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that a bimanual visuomotor policy trained to predict future video frames alongside actions coordinates its two arms better than policies trained on actions alone, and that the gain costs nothing at run time. Its framework trains a diffusion transformer to denoise an action chunk together with a compressed latent representation of future frames, using a unidirectional attention mask so video tokens attend to actions but not vice versa; action prediction therefore never depends on video tokens, and video prediction can be switched off at inference. On the ALOHA and RoboTwin simulators and a four-task real-robot setup, the paper reports success-rate gains over ACT of 24.9, 11.1, and 32.5 percentage points respectively. If the effect holds, video prediction during training is a cheap, task-agnostic inductive bias for bimanual coordination, and sequentially coordinated tasks show the largest benefit.

What carries the argument

The load-bearing mechanism is a unidirectional action-conditioned attention mask inside the transformer denoising decoder: action tokens attend only to themselves, while future-frame latent tokens attend to both themselves and historical action tokens. This enforces a causal action-to-future flow during training, lets future prediction guide action fitting, and allows video tokens to be dropped at inference without changing action predictions. The second component is multi-frame latent prediction: future frames are uniformly sampled, compressed by a pretrained Cosmos video tokenizer into latent tokens, patchified, and denoised together with the action chunk, so the model learns long-horizon visual dynamics in a compact space.

What would settle it

Run ACT again in this paper's exact evaluation harness on ALOHA, with the same 3 seeds, 100 episodes per seed, averaging the three best checkpoints, and the same observation preprocessing, then compare its average with the 47% quoted from the ACT paper; if ACT's re-run average is close to the proposed method's 71.9%, the claimed 24.9-point improvement would not reproduce in that comparison. Separately, training the proposed model with and without the video-prediction branch under identical seeds would test whether the auxiliary task itself is the cause of the gap.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that future imagination can serve as a coordination mechanism for bimanual manipulation without task-specific roles, symmetry assumptions, or pretrained skill primitives. The model jointly optimizes action generation and multi-frame latent video prediction inside one diffusion transformer; the unidirectional attention mask makes the video branch a pure training-time auxiliary that shapes action features through the shared decoder while remaining absent at inference. The authors interpret the consistent success-rate gains, largest on sequentially coordinated tasks such as block handover and put apple cabinet, as evidence that predicting future visual states gives both arms a common imagination of the interaction, aligning their objectives implicitly.

Load-bearing premise

The headline ALOHA gain assumes that ACT's published success rates, obtained under its original evaluation protocol, are directly comparable to success rates measured by this paper's evaluation protocol; if the protocols differ, the reported 24.9-point gap could change.

Editorial extensions

If this is right

  • Because video prediction is dropped at inference, the coordination benefit costs no extra latency; on the real-robot setup the paper measures 35.8 Hz control frequency, above ACT's 33.9 Hz.
  • The gain is concentrated where coordination is sequential: in the paper's RoboTwin task categories, adding video prediction raises success by about 6.2 points on Seq-coordinate tasks while leaving sync-bimanual tasks roughly flat.
  • Multi-frame latent prediction outperforms next-frame pixel and next-frame latent variants, so compressing future states into tokens is part of the design's effectiveness.
  • The method extends to a language-conditioned multi-task policy, where it beats RDT-1B by 8.0 points on a three-task RoboTwin subset.
  • Under a 20-demonstration data-efficiency setting, the method remains the strongest 2D-image policy and outperforms the point-cloud baseline on coordinated tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reader's inference: the ALOHA ACT numbers are quoted from the original ACT paper rather than re-run in this paper's harness, so a matched rerun of ACT under this protocol would be the cleanest check on the 24.9-point headline gap; the method's mechanism could hold even if the magnitude shifts.
  • A reader's inference: since Cosmos is pretrained on broad video, the gain from the video-prediction branch could come partly from the pretrained representation rather than the prediction task; retraining the model with a randomly initialized tokenizer would separate the two.
  • A reader's inference: the paper reports a small decline on simultaneous sync-bimanual tasks, suggesting that future prediction can introduce unnecessary dependencies; an arm-level ablation that removes the video head for only one arm would localize this effect and test the coordination story.
  • A reader's inference: the unidirectional recipe is not tied to diffusion or to bimanual hardware; any action-prediction model could in principle add a one-way video-prediction head during training, a transfer the paper does not test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes a diffusion-based policy for bimanual manipulation that jointly trains action prediction and latent future-frame prediction. A unidirectional attention mask lets video tokens attend to action tokens but not vice versa, so video prediction can be omitted at inference. Experiments are reported on ALOHA, RoboTwin, and a real Mobile Trossen platform, with comparisons against ACT, Diffusion Policy, 3D Diffusion Policy, RDT-1B, and GR-MG. The central empirical claim is that adding a video-prediction auxiliary objective improves average success rates, with the main quantitative support being +24.9 pp on ALOHA, +11.4 pp on RoboTwin, and +32.5 pp in real-world experiments, plus a consistent internal ablation showing gains from video prediction.

Significance. If the quantitative claims hold, the paper makes a useful contribution: it gives a simple way to add a video-prediction auxiliary task to a diffusion policy for bimanual control without paying inference-time cost, it releases code and models, and its ablation evidence for the auxiliary task is consistent across benchmarks. The multi-frame latent prediction design and the unidirectional attention mechanism are clearly described and are sensible engineering choices. However, the headline numbers currently rest on an unverified external ACT baseline on ALOHA and on an internally inconsistent real-world average, so the magnitude of the claimed gains cannot be taken at face value until those measurement issues are fixed.

major comments (2)
  1. [Appendix B.3 and Table 1] The headline ALOHA improvement (+24.9 pp over ACT, Table 1) is load-bearing, but Appendix B.3 states that the ACT row is taken directly from the original ACT paper rather than trained and evaluated under the protocol used for the proposed method. The manuscript's protocol uses 3 seeds, 100 episodes per seed, and the average of the highest 3 checkpoints, with specific image preprocessing, chunk size, and normalization; the original ACT paper's protocol is not shown to match on these dimensions. As a result, the +24.9 pp claim is not yet established. The authors should either re-run ACT in their own harness and report those numbers, or explicitly demonstrate that the original ACT protocol is equivalent and state the comparison as cross-paper. The ambiguity is increased by Table 10, which lists ACT hyperparameters for ALOHA even though B.3 says ACT results are taken from the original paper.
  2. [Table 8 and Abstract] Table 8 reports per-task real-world success rates for the proposed method of 40, 70, 70, and 20 across Water Wipe, Cup Stack, Can Handover, and Coffee Stir, which average to 50.0%, not the reported 60.0%. The abstract's claimed +32.5% real-world gain over ACT (27.5%) uses the 60.0% value; with the corrected mean the gain would be +22.5 pp. The discrepancy must be corrected in Table 8, the abstract, and Section 4.4, and any conclusions about real-world superiority should be re-evaluated after the correction.
minor comments (6)
  1. [Abstract and Section 4.2.2] The abstract reports an 11.1% increase on RoboTwin, while Section 4.2.2 reports +11.4% over ACT and +5.2% over RDT-1B; please specify which baseline is used for the abstract number and reconcile the percentages.
  2. [Section 4.3] The text says the tasks in Figure 5(b) are categorized into three types and refers to '20 tasks', but the RoboTwin benchmark used for that categorization has 16 tasks; please clarify whether additional tasks are included.
  3. [Table 2] The GR-MG entry in Table 2 appears to contain a formatting error ('GR-MG [32]4.31'), which should be corrected to the method name followed by the numerical value.
  4. [Section 4.4] The statement that the proposed method 'consistently achieves the best performance' is slightly overstated because on Coffee Stir the proposed method ties with ACT at 20% success; please qualify the claim.
  5. [Appendix C.1] The comparisons with InterACT and ARP in Table 12 use results taken from their respective papers. As with the ALOHA ACT baseline, please state the evaluation protocol used in those papers or re-run the baselines, since cross-paper comparisons are only valid when the protocols match.
  6. [Appendix D.2] The heading contains a typo, 'RoboTiwn', which should read 'RoboTwin'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; the central claim is supported by an internal ablation and does not reduce to its inputs by construction.

full rationale

The paper's derivation chain is self-contained. The video-prediction auxiliary loss (Sec. 3.3) is an extra MSE term on future frame tokens produced by a pretrained Cosmos tokenizer, while the action loss is a separate L1 term on action chunks; neither term is defined in terms of the other's output. The unidirectional attention mechanism (Sec. 3.2) explicitly prevents action tokens from attending to video tokens, so action prediction is not defined from video prediction either during training or at inference. The central contribution is isolated empirically by the no-video-prediction ablation (Table 4 and Fig. 5), which compares the full model against a variant without the video loss under the same architecture and protocol. This makes the claimed improvement an experimental result rather than a tautology. No load-bearing self-citation appears in the reference chain; baseline numbers are either reproduced from external code or taken from external publications, which is standard comparative practice and not circular. The review concerns about the ALOHA ACT baseline being quoted from the original paper (Appendix B.3) and the arithmetic inconsistency in Table 8's real-world average are measurement and reporting issues, not cases where a prediction reduces by construction to its inputs. Per the review rules, such protocol concerns do not increase the circularity score.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests mainly on empirical evaluation. The only non-standard inputs are the choice of the Cosmos video tokenizer, the tuned video prediction weight, the number of predicted frames, and the baseline evaluation protocols. No new physical or mathematical entities are postulated.

free parameters (2)
  • video prediction loss weight w = 0.2
    Tuned on the ALOHA benchmark (Table 6); the reported ALOHA result is the best over the tested values (0.05, 0.2, 0.5). The qualitative conclusion that video prediction helps is robust across these values, but the headline magnitude is the tuned maximum.
  • number of predicted future frames = 40 (ALOHA, real-world); 20 (RoboTwin)
    Ablated on ALOHA (Table 7); 40 frames was chosen for ALOHA and the real robot, 20 for RoboTwin. Performance varies by several points with this choice.
assumptions (4)
  • domain assumption The pretrained Cosmos video tokenizer preserves task-relevant visual dynamics in its latent tokens.
    The video prediction loss is computed on Cosmos latents (Section 3.2, Appendix B.4). If the tokenizer discards task-relevant information, the auxiliary loss would not provide useful signal. The paper shows reconstruction visualizations (Fig. 8) but no quantitative measure of task-relevant information retention.
  • domain assumption The evaluation protocols for baselines are comparable, including ACT results taken from the original paper.
    Appendix B.3 states ACT results on ALOHA are reported directly from the original paper [65]. If the original evaluation protocol differs from the one used here, the claimed improvements may be inflated.
  • standard math Standard diffusion model mathematics (DDIM, cosine schedule) is correct and applicable.
    The method relies on standard conditional diffusion training and inference (Section 3.3, Appendix B.1). No new mathematical claims are made.
  • domain assumption The unidirectional attention mask, where action tokens do not attend to video tokens, does not degrade training and enables skipping video prediction at inference.
    This is a design choice validated by ablation (Table 5), but the architectural premise that the action branch can be trained independently of the video branch is an assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion-Based Imaginative Coordination for Bimanual Manipulation." pith.science (2026). https://pith.science/paper/LXK5WQAQ

@misc{pith2026250711296,
  author       = {Pith},
  title        = {Pith review of: Diffusion-Based Imaginative Coordination for Bimanual Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LXK5WQAQ}},
  note         = {Machine review of arXiv:2507.11296}
}
read the original abstract

Bimanual manipulation is crucial in robotics, enabling complex tasks in industrial automation and household services. However, it poses significant challenges due to the high-dimensional action space and intricate coordination requirements. While video prediction has been recently studied for representation learning and control, leveraging its ability to capture rich dynamic and behavioral information, its potential for enhancing bimanual coordination remains underexplored. To bridge this gap, we propose a unified diffusion-based framework for the joint optimization of video and action prediction. Specifically, we propose a multi-frame latent prediction strategy that encodes future states in a compressed latent space, preserving task-relevant features. Furthermore, we introduce a unidirectional attention mechanism where video prediction is conditioned on the action, while action prediction remains independent of video prediction. This design allows us to omit video prediction during inference, significantly enhancing efficiency. Experiments on two simulated benchmarks and a real-world setting demonstrate a significant improvement in the success rate over the strong baseline ACT using our method, achieving a \textbf{24.9\%} increase on ALOHA, an \textbf{11.1\%} increase on RoboTwin, and a \textbf{32.5\%} increase in real-world experiments. Our models and code are publicly available at https://github.com/return-sleep/Diffusion_based_imaginative_Coordination.

Figures

Figures reproduced from arXiv: 2507.11296 by the authors.

Figure 1
Figure 1. Performance overview on bimanual manipulation tasks. Our method demonstrates significant improvements over previous approaches on ALOHA [65], RoboTwin [41], and real￾world evaluations. offer superior dexterity, efficiency, and versatility by en￾abling coordinated, simultaneous actions with two arms, making them well-suited for complex, heavy, or precision￾oriented tasks. However, bimanual manipulation presents signi… view at source ↗
Figure 2
Figure 2. Model Overview. We formulate bimanual manipulation as a conditional generation problem and propose a unified transformer￾based diffusion model for action prediction and video forecasting, optimizing the perception-prediction-control procedure in an end-to-end manner. Taking a single-view image and the robot’s proprioceptive state as input, our method simultaneously predicts the future action sequence and correspondi… view at source ↗
Figure 3
Figure 3. Illustration of unidirectional attention mechanism. Action tokens attend exclusively to themselves, while future frame tokens attend to both themselves and historical action tokens. As for decoupled attention, actions and visual latent tokens attend only within their own modality. It shows a scenario predicting four actions and two frames. reconstructs the denoised action sequence, and (2) the pre￾diction head, a mu… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Visualizations of bimanual tasks among two simula￾tion benchmarks, including ALOHA [65] and RoboTwin [41]. Avg.↑ Transfer Cube Insertion Method Success (%) Scripted Human Scripted Human ACT [65] 47 86 50 32 20 DP [7] 33 54 4 74 0 Ours 71.9 95.9 78.1 83.2 30.2 [PITH_FU…
Figure 5
Figure 5. Figure 5: Effectiveness of video prediction. Video prediction consistently improves performance across multiple benchmarks, highlighting its effectiveness. For data efficiency setting, each task in RoboTwin is provided with only 20 demonstrations. across all categories, outperfo…
Figure 6
Figure 6. Figure 6: The impact of chunk size on performance.The optimal chunk size depends on the task, but video prediction consistently improves performance across all chunk sizes [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Real-Robot setup and 4 real-world bimanual tasks. We collect 50 demos for each task by human teleoperation. erate chunk size yields the optimal performance, as high￾frequency tasks benefit from action chunking, improving control stability. However, longer chunks limit …
Figure 8
Figure 8. Figure 8: Visualization of video reconstruction using Cosmos Tokenizer on ALOHA [65] and RoboTwin [41] benchmark (17 frame sub-clip, DV4x8x8 version). Method Avg. SR ↑ Transfer Cube (Human) Insertion (Human) InterACT [31] 63.0 82 44 ARP [64] 59.4 94 24.8 Ours 65.3 84 46.7 [PITH…
Figure 10
Figure 10. Figure 10: Task definition of real-world experiments. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 52 canonical work pages

  1. [1]

    Exploiting symmetries in reinforcement learning of bimanual robotic tasks

    Fabio Amadio, Adri `a Colom´e, and Carme Torras. Exploiting symmetries in reinforcement learning of bimanual robotic tasks. IEEE Robotics and Automation Letters , 4(2):1838– 1845, 2019. 1, 2

    work page 2019

  2. [2]

    Roboa- gent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking

    Homanga Bharadhwaj, Jay Vakil, Mohit Sharma, Abhinav Gupta, Shubham Tulsiani, and Vikash Kumar. Roboa- gent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 4788–4795. IEEE, 2024. 13

    work page 2024

  3. [3]

    π0 : A vision-language- action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0 : A vision-language- action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. 2

    arXiv

  4. [4]

    Zero-shot robotic manipulation with pre-trained image-editing diffusion models

    Kevin Black, Mitsuhiko Nakamoto, Pranav Atreya, Homer Rich Walke, Chelsea Finn, Aviral Kumar, and Sergey Levine. Zero-shot robotic manipulation with pre-trained image-editing diffusion models. InThe Twelfth International Conference on Learning Representations, 2024. 3

    work page 2024

  5. [5]

    A dual-arm collaborative robot system for the smart factories of the future

    Jens F Buhl, Rune Grønhøj, Jan K Jørgensen, Guilherme Mateus, Daniela Pinto, Jacob K Sørensen, Simon Bøgh, and Dimitrios Chrysostomou. A dual-arm collaborative robot system for the smart factories of the future. Procedia manu- facturing, 38:333–340, 2019. 1

    work page 2019

  6. [6]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. In European confer- ence on computer vision, pages 213–229. Springer, 2020. 5

    work page 2020

  7. [7]

    Diffusion policy: Visuomotor policy learning via action dif- fusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action dif- fusion. The International Journal of Robotics Research, page 02783649241273668, 2023. 1, 2, 5, 6, 8, 12, 14

    work page 2023

  8. [8]

    Efficient bimanual manipulation using learned task schemas

    Rohan Chitnis, Shubham Tulsiani, Saurabh Gupta, and Ab- hinav Gupta. Efficient bimanual manipulation using learned task schemas. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 1149–1155. IEEE,

    work page 2020

Show all 68 references
  1. [9]

    Large language models for orchestrating bimanual robots

    Kun Chu, Xufeng Zhao, Cornelius Weber, Mengdi Li, Wen- hao Lu, and Stefan Wermter. Large language models for orchestrating bimanual robots. In 2024 IEEE-RAS 23rd In- ternational Conference on Humanoid Robots (Humanoids) , pages 328–334. IEEE, 2024. 2

    2024

  2. [10]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  3. [11]

    Learning universal policies via text-guided video genera- tion

    Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video genera- tion. Advances in neural information processing systems , 36:9156–9172, 2023. 2, 3

    2023

  4. [12]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 3

    2021

  5. [13]

    NVIDIA et. al. Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575, 2025. 3, 12, 14

    2025 arXiv

  6. [14]

    Interactive imitation learn- ing of bimanual movement primitives

    Giovanni Franzese, Leandro de Souza Rosa, Tim Verburg, Luka Peternel, and Jens Kober. Interactive imitation learn- ing of bimanual movement primitives. IEEE/ASME Trans- actions on Mechatronics, 2023. 2

    2023

  7. [15]

    Zhao, and Chelsea Finn

    Zipeng Fu, Tony Z. Zhao, and Chelsea Finn. Mobile aloha: Learning bimanual mobile manipulation with low- cost whole-body teleoperation. In Conference on Robot Learning (CoRL), 2024. 8

    2024

  8. [16]

    Bi-kvil: Keypoints-based visual imitation learning of bimanual manipulation tasks

    Jianfeng Gao, Xiaoshu Jin, Franziska Krebs, No ´emie Jaquier, and Tamim Asfour. Bi-kvil: Keypoints-based visual imitation learning of bimanual manipulation tasks. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 16850–16857. IEEE, 2024. 2

    2024

  9. [17]

    Dag-plan: Generating directed acyclic dependency graphs for dual-arm cooperative plan- ning

    Zeyu Gao, Yao Mu, Jinye Qu, Mengkang Hu, Lingyue Guo, Ping Luo, and Yanfeng Lu. Dag-plan: Generating directed acyclic dependency graphs for dual-arm cooperative plan- ning. arXiv preprint arXiv:2406.09953, 2024. 2

    2024 arXiv

  10. [18]

    Stabilize to act: Learning to coordinate for bimanual manip- ulation

    Jennifer Grannen, Yilin Wu, Brandon Vu, and Dorsa Sadigh. Stabilize to act: Learning to coordinate for bimanual manip- ulation. In Conference on Robot Learning, pages 563–576. PMLR, 2023. 1, 2

    2023

  11. [19]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vi- sio...

    2022

  12. [20]

    Peract2: Benchmarking and learn- ing for robotic bimanual manipulation tasks

    Markus Grotz, Mohit Shridhar, Yu-Wei Chao, Tamim As- four, and Dieter Fox. Peract2: Benchmarking and learn- ing for robotic bimanual manipulation tasks. In CoRL 2024 Workshop on Whole-body Control and Bimanual Manipula- tion: Applications in Humanoids and Beyond, 2024. 2

    2024

  13. [21]

    Maniskill2: A unified benchmark for generalizable manipulation skills

    Jiayuan Gu, Fanbo Xiang, Xuanlin Li, Zhan Ling, Xiqiang Liu, Tongzhou Mu, Yihe Tang, Stone Tao, Xinyue Wei, Yun- chao Yao, Xiaodi Yuan, Pengwei Xie, Zhiao Huang, Rui Chen, and Hao Su. Maniskill2: A unified benchmark for generalizable manipulation skills. In International Confe...

    2023

  14. [22]

    Prediction with action: Visual policy learning via joint denoising pro- cess

    Yanjiang Guo, Yucheng Hu, Jianke Zhang, Yen-Jen Wang, Xiaoyu Chen, Chaochao Lu, and Jianyu Chen. Prediction with action: Visual policy learning via joint denoising pro- cess. In The Thirty-eighth Annual Conference on Neural In- formation Processing Systems, 2024. 3

    2024

  15. [23]

    Dream to control: Learning behaviors by la- tent imagination

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Moham- mad Norouzi. Dream to control: Learning behaviors by la- tent imagination. In International Conference on Learning Representations, 2020. 2, 4

    2020

  16. [24]

    Td-mpc2: Scalable, robust world models for continuous control

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control. In In- ternational Conference on Learning Representations , 2024. 13 9

    2024

  17. [25]

    Hardware and software architecture of a bimanual mobile manipulator for industrial application

    Andreas Hermann, Zhixing Xue, Steffen W R ¨uhl, and R¨udiger Dillmann. Hardware and software architecture of a bimanual mobile manipulator for industrial application. In 2011 IEEE International Conference on Robotics and Biomimetics, pages 2282–2288. IEEE, 2011. 1

    2011

  18. [26]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  19. [27]

    Learning manipulation by predicting inter- action

    Zeng Jia, Bu Qingwen, Wang Bangjun, Xia Wenke, Chen Li, Dong Hao, Song Haoming, Wang Dong, Hu Di, Luo Ping, Cui Heming, Zhao Bin, Li Xuelong, Qiao Yu, and Li Hongyang. Learning manipulation by predicting inter- action. In Proceedings of Robotics: Science and Systems (RSS), 202...

    2024

  20. [28]

    3d diffuser actor: Policy diffusion with 3d scene rep- resentations

    Tsung-Wei Ke, Nikolaos Gkanatsios, and Katerina Fragki- adaki. 3d diffuser actor: Policy diffusion with 3d scene rep- resentations. arXiv preprint arXiv:2402.10885, 2024. 12

    2024 arXiv

  21. [29]

    Tenenbaum

    Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, and Joshua B. Tenenbaum. Learning to act from actionless videos through dense correspondences. In The Twelfth In- ternational Conference on Learning Representations , 2024. 2, 3

    2024

  22. [30]

    A bimanual manipula- tion taxonomy

    Franziska Krebs and Tamim Asfour. A bimanual manipula- tion taxonomy. IEEE Robotics and Automation Letters, 7(4): 11031–11038, 2022. 1

    2022

  23. [31]

    Interact: Inter-dependency aware action chunking with hierarchical attention transformers for biman- ual manipulation

    Andrew Choong-Won Lee, Ian Chuang, Ling-Yuan Chen, and Iman Soltani. Interact: Inter-dependency aware action chunking with hierarchical attention transformers for biman- ual manipulation. In 8th Annual Conference on Robot Learn- ing, 2024. 13

    2024

  24. [32]

    Gr-mg: Leveraging partially- annotated data via multi-modal goal-conditioned policy

    Peiyan Li, Hongtao Wu, Yan Huang, Chilam Cheang, Liang Wang, and Tao Kong. Gr-mg: Leveraging partially- annotated data via multi-modal goal-conditioned policy. IEEE Robotics and Automation Letters , 2025. 2, 5, 6, 7, 12

    2025

  25. [33]

    Planning-guided diffusion policy learning for generalizable contact-rich bimanual manipula- tion

    Xuanlin Li, Tong Zhao, Xinghao Zhu, Jiuguang Wang, Tao Pang, and Kuan Fang. Planning-guided diffusion policy learning for generalizable contact-rich bimanual manipula- tion. arXiv preprint arXiv:2412.02676, 2024. 1, 2

    2024 arXiv

  26. [34]

    Efficient bimanual handover and rearrangement via symmetry-aware actor-critic learning

    Yunfei Li, Chaoyi Pan, Huazhe Xu, Xiaolong Wang, and Yi Wu. Efficient bimanual handover and rearrangement via symmetry-aware actor-critic learning. In 2023 IEEE Inter- national Conference on Robotics and Automation (ICRA) , pages 3867–3874. IEEE, 2023. 2

    2023

  27. [35]

    Sukhatme

    I-Chun Arthur Liu, Sicheng He, Daniel Seita, and Gaurav S. Sukhatme. V oxact-b: V oxel-based acting and stabilizing policy for bimanual manipulation. In Conference on Robot Learning, 2024. 1, 2

    2024

  28. [36]

    Robot cooking with stir-fry: Bimanual non-prehensile manipulation of semi-fluid objects

    Junjia Liu, Yiting Chen, Zhipeng Dong, Shixiong Wang, Syl- vain Calinon, Miao Li, and Fei Chen. Robot cooking with stir-fry: Bimanual non-prehensile manipulation of semi-fluid objects. IEEE Robotics and Automation Letters, 7(2):5159– 5166, 2022. 2

    2022

  29. [37]

    Rdt-1b: a diffusion foundation model for bimanual manipu- lation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipu- lation. arXiv preprint arXiv:2410.07864 , 2024. 1, 2, 5, 6, 12, 13

    2024 arXiv

  30. [38]

    Anybimanual: Transferring unimanual policy for general bimanual manipulation

    Guanxing Lu, Tengbo Yu, Haoyuan Deng, Season Si Chen, Yansong Tang, and Ziwei Wang. Anybimanual: Transferring unimanual policy for general bimanual manipulation. arXiv preprint arXiv:2412.06779, 2024. 1, 2

    2024 arXiv

  31. [39]

    Latte: Latent diffusion transformer for video generation

    Xin Ma, Yaohui Wang, Gengyun Jia, Xinyuan Chen, Zi- wei Liu, Yuan-Fang Li, Cunjian Chen, and Yu Qiao. Latte: Latent diffusion transformer for video generation. arXiv preprint arXiv:2401.03048, 2024. 3

    2024 arXiv

  32. [40]

    Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncer- tainty (rss 2016 best student paper award)

    Seyed Sina Mirrazavi Salehian, Nadia Barbara Figueroa Fer- nandez, and Aude Billard. Coordinated multi-arm motion planning: Reaching for moving objects in the face of uncer- tainty (rss 2016 best student paper award). In2016 Robotics: Science and Systems Conference, 2016. 1

    2016

  33. [41]

    Robotwin: Dual-arm robot benchmark with generative digi- tal twins (early version)

    Yao Mu, Tianxing Chen, Shijia Peng, Zanxin Chen, Zeyu Gao, Yude Zou, Lunkai Lin, Zhiqiang Xie, and Ping Luo. Robotwin: Dual-arm robot benchmark with generative digi- tal twins (early version). arXiv preprint arXiv:2409.02920,

    arXiv

  34. [42]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR,

  35. [43]

    Action-conditioned benchmarking of robotic video prediction models: a comparative study

    Manuel Serra Nunes, Atabak Dehban, Plinio Moreno, and Jos´e Santos-Victor. Action-conditioned benchmarking of robotic video prediction models: a comparative study. In 2020 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 8316–8322. IEEE, 2020. 4

    2020

  36. [44]

    Action-conditional video prediction us- ing deep networks in atari games

    Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard L Lewis, and Satinder Singh. Action-conditional video prediction us- ing deep networks in atari games. Advances in neural infor- mation processing systems, 28, 2015. 4

    2015

  37. [45]

    Bimanual long-horizon manipulation via temporal- context transformer rl

    Ji-Heon Oh, Ismael Espinoza, Danbi Jung, and Tae-Seong Kim. Bimanual long-horizon manipulation via temporal- context transformer rl. IEEE Robotics and Automation Let- ters, 2024. 2

    2024

  38. [46]

    Film: Visual reasoning with a general conditioning layer

    Ethan Perez, Florian Strub, Harm De Vries, Vincent Du- moulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI con- ference on artificial intelligence, 2018. 13

    2018

  39. [47]

    Distributed consensus in multi-vehicle cooperative control

    Wei Ren and Randal W Beard. Distributed consensus in multi-vehicle cooperative control. Springer, 2008. 2

    2008

  40. [48]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  41. [49]

    Dual arm manipulation—a survey

    Christian Smith, Yiannis Karayiannidis, Lazaros Nalpan- tidis, Xavi Gratal, Peng Qi, Dimos V Dimarogonas, and Dan- ica Kragic. Dual arm manipulation—a survey. Robotics and Autonomous systems, 60(10):1340–1353, 2012. 1

    2012

  42. [50]

    Learning structured output representation using deep conditional gen- erative models

    Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional gen- erative models. Advances in neural information processing systems, 28, 2015. 1

    2015

  43. [51]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021. 12 10

    2021

  44. [52]

    Predictive inverse dynam- ics models are scalable learners for robotic manipulation

    Yang Tian, Sizhe Yang, Jia Zeng, Ping Wang, Dahua Lin, Hao Dong, and Jiangmiao Pang. Predictive inverse dynam- ics models are scalable learners for robotic manipulation. In The Thirteenth International Conference on Learning Rep- resentations, 2025. 3

    2025

  45. [53]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033. IEEE, 2012. 5

    2012

  46. [54]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information pro- cessing systems, 30, 2017. 3

    2017

  47. [55]

    Embodied ai with two arms: Zero-shot learning, safety and modularity

    Jake Varley, Sumeet Singh, Deepali Jain, Krzysztof Choro- manski, Andy Zeng, Somnath Basu Roy Chowdhury, Avinava Dubey, and Vikas Sindhwani. Embodied ai with two arms: Zero-shot learning, safety and modularity. In 2024 IEEE/RSJ International Conference on Intelligent Robots an...

    2024

  48. [56]

    Vidman: Exploiting implicit dynamics from video diffusion model for effective robot ma- nipulation

    Youpeng Wen, Junfan Lin, Yi Zhu, Jianhua Han, Hang Xu, Shen Zhao, and Xiaodan Liang. Vidman: Exploiting implicit dynamics from video diffusion model for effective robot ma- nipulation. Advances in Neural Information Processing Sys- tems, 37:41051–41075, 2025. 3

    2025

  49. [57]

    Unleashing large-scale video generative pre-training for visual robot manipulation

    Hongtao Wu, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li, Minghuan Liu, Hang Li, and Tao Kong. Unleashing large-scale video generative pre-training for visual robot manipulation. In International Conference on Learning Representations, 2024. 2, 3, 5

    2024

  50. [58]

    Coordinated control of a dual-arm robot for surgical instrument sorting tasks

    Qihan Wu, Meng Li, Xiaozhi Qi, Ying Hu, Bing Li, and Jianwei Zhang. Coordinated control of a dual-arm robot for surgical instrument sorting tasks. Robotics and Autonomous Systems, 112:1–12, 2019. 1

    2019

  51. [59]

    Flexible control and trajectory planning of medical two- arm surgical robot

    Yanchun Xie, Xue Zhao, Yang Jiang, Yao Wu, and Hailong Yu. Flexible control and trajectory planning of medical two- arm surgical robot. Frontiers in Neurorobotics, 18:1451055,

  52. [60]

    Latent action pretraining from videos

    Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Se June Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, Lars Liden, Kimin Lee, Jian- feng Gao, Luke Zettlemoyer, Dieter Fox, and Minjoon Seo. Latent action pretraining from videos. In The Thirt...

    2025

  53. [61]

    Bikc: Keypose-conditioned consistency policy for biman- ual robotic manipulation

    Dongjie Yu, Hang Xu, Yizhou Chen, Yi Ren, and Jia Pan. Bikc: Keypose-conditioned consistency policy for biman- ual robotic manipulation. arXiv preprint arXiv:2406.10093,

    arXiv

  54. [62]

    3d diffusion policy: Gen- eralizable visuomotor policy learning via simple 3d repre- sentations

    Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3d diffusion policy: Gen- eralizable visuomotor policy learning via simple 3d repre- sentations. In Proceedings of Robotics: Science and Systems (RSS), 2024. 2, 5, 6, 12, 14

    2024

  55. [63]

    Empowering embodied manipulation: A bimanual-mobile robot manipulation dataset for household tasks

    Tianle Zhang, Dongjiang Li, Yihang Li, Zecui Zeng, Lin Zhao, Lei Sun, Yue Chen, Xuelong Wei, Yibing Zhan, Lu- song Li, et al. Empowering embodied manipulation: A bimanual-mobile robot manipulation dataset for household tasks. arXiv preprint arXiv:2405.18860, 2024. 2

    2024 arXiv

  56. [64]

    Autoregressive action se- quence learning for robotic manipulation

    Xinyu Zhang, Yuhan Liu, Haonan Chang, Liam Schramm, and Abdeslam Boularias. Autoregressive action se- quence learning for robotic manipulation. arXiv preprint arXiv:2410.03132, 2024. 1, 8, 13

    2024 arXiv

  57. [65]

    Learning fine-grained bimanual manipulation with low-cost hardware

    Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Proceedings of Robotics: Science and Systems (RSS), 2023. 1, 2, 5, 6, 8, 12, 13, 14

    2023

  58. [66]

    Zhao, Jonathan Tompson, Danny Driess, Pete Flo- rence, Seyed Kamyar Seyed Ghasemipour, Chelsea Finn, and Ayzaan Wahid

    Tony Z. Zhao, Jonathan Tompson, Danny Driess, Pete Flo- rence, Seyed Kamyar Seyed Ghasemipour, Chelsea Finn, and Ayzaan Wahid. ALOHA unleashed: A simple recipe for robot dexterity. In 8th Annual Conference on Robot Learn- ing, 2024. 2

    2024

  59. [67]

    Dualafford: Learning collaborative visual affordance for dual-gripper manipulation

    Yan Zhao, Ruihai Wu, Zhehuan Chen, Yourong Zhang, Qingnan Fan, Kaichun Mo, and Hao Dong. Dualafford: Learning collaborative visual affordance for dual-gripper manipulation. In International Conference on Learning Rep- resentations, 2023. 1, 2 11 A. Differences between Our Mode...

    2023

  60. [2024]

    1, 2, 5, 6, 8, 12, 13, 14

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.