Pith. sign in

REVIEW 3 major objections 40 references

A frozen video model, queried once at pure noise, yields a directional interaction prior that improves low-data robot manipulation without future-frame rollout.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 15:44 UTC pith:E2VDPPUU

load-bearing objection Clean, reusable idea—single-step frozen Flow Matching velocity as a policy prior—with solid gains and a real ablation, but the “directional beyond localization” claim is under-identified and the numbers are single-run sim. the 3 major comments →

arxiv 2607.04652 v1 pith:E2VDPPUU submitted 2026-07-06 cs.RO

KAM-WM: Kinematic Affordance Maps from Latent World Models for Robot Manipulation

Xinyu Shao , Keru Zhou , Guowei Huang , Yajun Gao , Tongtong Cao , Xiu Li This is my paper
classification cs.RO
keywords robot manipulationlatent world modelsimitation learningkinematic affordance mapsflow matchingdiffusion policyvisual priorsfew-demonstration learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Learning robot manipulation from few demonstrations needs visual guidance that says not only where to touch but how the motion should begin. Static masks only mark places; they leave approach direction to the policy. This paper claims that a frozen Flow Matching image-to-video model already encodes that first-order cue: its single-step latent velocity at the high-noise endpoint, conditioned on the first observation and the language instruction, highlights task-relevant contact regions and coarse motion structure. The authors treat that field as a Kinematic Affordance Map, compress it into a handful of tokens, and condition a diffusion policy on those tokens together with RGB and proprioception. On standard low-data benchmarks the method raises success rates relative to diffusion and vision-language-action baselines, and controlled mask ablations suggest part of the gain is directional information beyond localization alone. The practical point is that a large video model can be reused as a cheap, non-rolled-out prior rather than as a future-frame generator or a fine-tuned backbone.

Core claim

In the evaluated low-data settings, the single-step latent velocity of a frozen Flow Matching image-to-video backbone, read once at the noise endpoint from the first head-camera frame and language instruction, is a useful first-order visual prior for manipulation: it supplies task-conditioned interaction regions plus coarse directional structure, and when compressed into tokens that condition a diffusion policy it improves success over strong baselines without multi-step video rollout or world-model fine-tuning. Controlled comparisons with instruction-conditioned masks indicate that part of the improvement comes from directional information beyond spatial localization alone.

What carries the argument

Kinematic Affordance Map (KAM): the dense single-step latent velocity field produced by one query of a frozen Flow Matching image-to-video model at the pure-noise endpoint, whose magnitude marks task-conditioned response regions and whose normalized response carries coarse orientation; a lightweight Perceiver then compresses this field into a few tokens that condition the policy.

Load-bearing premise

That a single velocity field extracted from the first head-camera frame and held fixed for the whole episode stays informative enough for closed-loop control, even when the scene changes, the horizon is long, or the contact view becomes occluded.

What would settle it

On the same 50-demo RoboTwin and LIBERO protocols, re-run the exact policy architecture with KAM refreshed at every critical contact or after major visual change, versus the default once-per-episode fixed KAM: if success does not rise on long-horizon and occlusion-heavy tasks, the claim that a single fixed first-frame prior is sufficient collapses; if a pure mask prior then matches or beats KAM on direction-sensitive tasks under multi-seed evaluation, the directional-beyond-localization claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. KAM-WM proposes reading a single-step latent velocity field from a frozen Flow Matching image-to-video model (Wan 2.2) at the noise endpoint t=1.0 and treating it as a Kinematic Affordance Map (KAM)—a task-conditioned first-order visual prior that encodes interaction regions and coarse motion structure without future-frame rollout or world-model fine-tuning. A lightweight Perceiver compresses the dense field into K=8 tokens that condition a 1D U-Net Diffusion Policy together with multi-view RGB and proprioception. On LIBERO (50 demos) the method reports 90.6% average success; on RoboTwin 2.0 (50 demos, 50 tasks) it reports 65.7% Easy and 22.4% Hard success. Controlled ablations against instruction-conditioned SAM masks under a fixed policy interface, plus a low-data diagnostic and timestep study, are used to argue that part of the gain is directional structure beyond zero-order localization.

Significance. If the result holds, the paper offers a practically attractive design point: reuse a large frozen video model as a one-query, first-order visual prior for low-data imitation learning without test-time video generation or backbone updates. That is a useful complement to mask/affordance priors and to rollout-based world-model policies. Strengths include a clear extraction interface (Eqs. 1–3), a controlled same-architecture mask ablation (Table 3 / Appendix C), efficiency accounting (one-time ~895 ms extraction amortized over the episode; ~2.1M KAM-specific trainable parameters), and large reported gains on standard low-data suites. The work is empirical rather than circular by construction, and the limitations section candidly flags staleness, simulation-only main evaluation, and single-run reporting.

major comments (3)
  1. Abstract, §1, §4.4, and conclusion claim that “part of the gains comes from directional information beyond spatial localization alone.” The load-bearing control is Table 3 / Appendix C Table 7 (same Perceiver + diffusion policy; only conditioning tensor swapped). That comparison shows Easy 67.8%→77.0% and Hard 24.0%→28.6% on a 9-task subset, with large Easy lifts on Hanging Mug and Place Empty Cup, but Hard is mixed or worse for KAM on precise-contact tasks (Click Bell 34→18; Place Shoe 22→10). Critically, the policy consumes the dense field V_prior (Eq. 3), not the separated magnitude A_kam vs. normalized response bV_prior (Eq. 4). Without a magnitude-only / soft-heatmap control of matched spatial support and channel capacity, sharper multi-channel localization remains a viable alternative explanation. Please add that control (or an explicit orientation-scrambled / direction-randomized
  2. §4.1 Evaluation protocol and Limitations §5: all main success rates (Tables 1–2, full 50-task Table 8) are single training runs with no multi-seed means or error bars. In the low-data regime the paper targets, training variance is material; several RoboTwin Hard rates are low enough that single-run differences of a few points are hard to interpret. At minimum, report multi-seed statistics for the main aggregate claims (LIBERO suite averages; RoboTwin 50-task Easy/Hard averages) and for the prior-type ablation subset, or clearly demote leaderboard-style point estimates to exploratory and restate confidence accordingly.
  3. §3.4 default protocol extracts KAM once from the first head-camera frame and holds it fixed for the episode. Limitations §5 correctly notes staleness under long horizons, occlusion, or major scene change—precisely the regimes where Long-suite LIBERO and Hard RoboTwin gains are most interesting. The paper does not quantify how often the fixed prior becomes mismatched, nor does it report a refresh-at-key-events or periodic re-query ablation. A small controlled study (e.g., refresh every N steps or at contact events on Long / Hard tasks) is needed to bound how much of the reported gain depends on the “once and fixed” axiom versus a still-valid prior.

Circularity Check

0 steps flagged

No significant circularity: KAM-WM is an empirical control interface whose claims are measured on external benchmarks, not forced by definition or self-citation.

full rationale

The paper does not present a first-principles derivation that forces its performance claims. KAM is defined operationally as the frozen Flow Matching single-step latent velocity V_prior = v_θ(x1, 1.0, [o1, l]) (Eqs. 1–3); success rates on LIBERO and RoboTwin 2.0 are external empirical measurements under a fixed 50-demo protocol, not quantities recovered from a fit to the same targets. The Flow Matching identity at t=1.0 (Eq. 2) is a standard property of the training objective and is used only to motivate reading a conditional latent field, not to equate that field with robot success. The controlled mask ablation (Table 3 / Appendix C) holds the Perceiver + diffusion policy fixed and swaps only the conditioning tensor; any under-identification of “direction vs. sharper localization” is an experimental-design limitation, not a by-construction reduction of the claimed gain to the input. The sole author-overlapping citation of note is SVP [38], which supplies the mask-baseline protocol (SAM-3 instruction masks through the same Perceiver), not the success metric or a uniqueness theorem that forbids alternatives. No uniqueness is imported, no ansatz is smuggled as a forced result, and no fitted free parameter is renamed as a prediction. The work is therefore self-contained against external benchmarks with no circular derivation chain.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The central claim rests on standard Flow Matching identities, the empirical premise that Wan 2.2’s noise-endpoint velocity carries task-conditioned interaction structure, several architectural free choices (token count, extraction timestep, single-query-per-episode), and the invented interpretive entity “KAM.” No new physical constants; the free parameters are design knobs that affect the reported numbers.

free parameters (4)
  • KAM extraction timestep t
    Default t=1.0 is chosen as the only strictly rollout-free point; Table 4 shows task-dependent better timesteps, so the operating point is a design choice that affects reported success.
  • Number of Perceiver tokens K
    K=8 is fixed by design; it controls how much spatial/directional structure reaches the policy and is not derived from first principles.
  • Action horizon Ha and diffusion inference steps
    Ha=16 and 10 inference denoising steps are standard DP hyperparameters chosen for the experiments; they affect latency and success but are not predicted by theory.
  • Checkpoint selection epochs {100,300,600} in low-data diagnostic
    Best-of-three-epoch rule is applied uniformly but still selects post-training; reported 20-demo gains depend on this selection rule (Table 5).
axioms (4)
  • standard math Under Flow Matching / rectified flow with straight paths, the model velocity at t=1.0 approximates x1 − E[x0|c], i.e., a conditional latent displacement up to a constant offset (Eqs. 1–2).
    Standard Flow Matching identity used to justify reading the single-step field as a task-conditioned prior rather than calibrated optical flow.
  • domain assumption A frozen general-purpose video model (Wan 2.2) encodes motion and interaction structure useful for robot manipulation when queried only at the noise endpoint.
    Load-bearing empirical premise of the whole framework; not proven, only motivated by qualitative response maps and downstream success.
  • ad hoc to paper Extracting KAM once from the first head-camera frame and holding it fixed for the episode is sufficient for the evaluated tasks.
    Default protocol in §3.4; Limitations §5 notes staleness risk under long horizons or scene change.
  • domain assumption A lightweight Perceiver can compress the dense latent velocity into a few tokens without destroying the directional cue the policy needs.
    Architectural premise of the control interface (§3.3); supported only by end-to-end success, not by intermediate reconstruction metrics.
invented entities (1)
  • Kinematic Affordance Map (KAM) no independent evidence
    purpose: Name and operational definition for the single-step latent velocity field used as a first-order where-and-how visual prior for the policy.
    New interpretive construct: Vprior is a standard model output; calling it KAM and routing it through a Perceiver into a diffusion policy is the paper’s contribution. Independent evidence is only the downstream task success and mask ablations, not an external physical measurement of the field.

pith-pipeline@v1.1.0-grok45 · 21861 in / 3638 out tokens · 34194 ms · 2026-07-11T15:44:52.978360+00:00 · methodology

0 comments
read the original abstract

Learning manipulation from few demonstrations requires visual priors that capture not only where to interact, but also how the interaction should begin; static priors such as segmentation masks encode only the former. We present KAM-WM, a framework that extracts a coarse directional interaction cue from a frozen latent video world model without rollout or world-model fine-tuning. KAM-WM queries a Flow Matching image-to-video backbone once and interprets its single-step latent velocity as a Kinematic Affordance Map (KAM), which provides task-conditioned interaction regions and coarse motion structure. A lightweight Perceiver compresses KAM into tokens that condition a diffusion policy together with RGB observations and proprioception. Across LIBERO and RoboTwin2.0, KAM-WM reaches 90.6% average success on LIBERO and achieves 65.7% and 22.4% success rates in the Easy and Hard settings on RoboTwin2.0, respectively. Controlled comparisons against a zero-order mask prior suggest that part of the gains comes from directional information beyond spatial localization alone. These results indicate that, in the evaluated settings, a frozen video model can provide a useful first-order visual prior for control without the test-time cost of future rollout.

Figures

Figures reproduced from arXiv: 2607.04652 by Guowei Huang, Keru Zhou, Tongtong Cao, Xinyu Shao, Xiu Li, Yajun Gao.

Figure 1
Figure 1. Figure 1: KAM-WM provides where-and-how cues for low-data manipulation. (a) KAM highlights interaction-relevant regions and motion cues beyond object masks. (b) These tokens condition the diffusion policy together with RGB and proprioception. (c) KAM-WM improves performance on LIBERO and RoboTwin 2.0. This observation is consistent with the Flow Matching training objective [1, 2]. For a conditional video model, the … view at source ↗
Figure 2
Figure 2. Figure 2: KAM-WM framework. Given the initial observation and language instruction, KAM-WM queries a frozen Flow Matching image-to-video backbone once at t=1.0, with no future-frame generation, and reads the resulting single-step latent velocity field as a Kinematic Affordance Map. A Perceiver compresses this dense field into K dynamic tokens, which condition a 1D U-Net Diffusion Policy together with current RGB obs… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison of mask and KAM priors. Mask priors localize target regions but remain zero-order spatial prior. KAM highlights more selective interaction regions and adds coarse orientation prior from the latent velocity field under the same policy interface. Prior type. We compare KAM with a zero-order mask prior while keeping the policy architecture fixed. The mask baseline follows the spatial vi… view at source ↗
Figure 4
Figure 4. Figure 4: Real-world setup and representative execution sequences. Key frames from four real￾world tasks: (a) cup stacking, (b) bowl-to-plates, (c) place bottle, and (d) apple-to-basket [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 15 linked inside Pith

  1. [1]

    Lipman, R

    Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for gen- erative modeling. InInternational Conference on Learning Representations (ICLR), 2023. arXiv:2210.02747

    Pith/arXiv arXiv 2023

  2. [2]

    X. Liu, C. Gong, and Q. Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  3. [3]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning.Advances in Neural Information Processing Systems (NeurIPS), 36:44776–44791, 2023

    2023

  4. [4]

    T. Chen, Z. Chen, B. Chen, Z. Cai, Y . Liu, Z. Li, Q. Liang, X. Lin, Y . Ge, Z. Gu, et al. RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning (ICML), pages 8748–8763. PMLR, 2021

    2021

  6. [6]

    K. He, X. Chen, S. Xie, Y . Li, P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16000–16009, 2022

    2022

  7. [7]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Soyer, V . Kinnunen, H. Touvron, et al. DINOv2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4015–4026, 2023

    2023

  9. [9]

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta. R3M: A universal visual representation for robot manipulation. InConference on Robot Learning (CoRL), pages 892–909. PMLR, 2022

    2022

  10. [10]

    Radosavovic, T

    I. Radosavovic, T. Xiao, S. James, P. Abbeel, J. Malik, and T. Darrell. Real-world robot learning with masked visual pre-training. InConference on Robot Learning (CoRL), 2022

    2022

  11. [11]

    Majumdar, K

    A. Majumdar, K. Yadav, S. Lin, S. Peralta, et al. Where are we in the search for an artificial visual cortex for embodied intelligence? InAdvances in Neural Information Processing Systems (NeurIPS), volume 36, 2023

    2023

  12. [12]

    Bardes, Q

    A. Bardes, Q. Garrido, J. Ponce, X. Chen, M. Rabbat, Y . LeCun, M. Assran, and N. Ballas. Revisiting feature prediction for learning visual representations from video.arXiv preprint arXiv:2404.08471, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    K. Mo, L. J. Guibas, M. Mukadam, A. Gupta, and S. Tulsiani. Where2Act: From pixels to actions for articulated 3d objects. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6813–6823, 2021

    2021

  14. [14]

    R. Wu, Y . Zhao, K. Mo, Z. Guo, Y . Wang, T. Wu, Q. Fan, X. Chen, L. Guibas, and H. Dong. V AT-Mart: Learning visual action trajectory proposals for manipulating 3d articulated objects. InInternational Conference on Learning Representations (ICLR), 2022

    2022

  15. [15]

    S. Bahl, R. Mendonca, L. Chen, U. Jain, and D. Pathak. Affordances from human videos as a versatile representation for robotics. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13778–13790, 2023. 9

    2023

  16. [16]

    A. Zeng, P. Florence, J. Tompson, J. Welker, J. Chien, M. Attarian, T. Armstrong, I. Krasin, D. Duong, V . Sindhwani, and J. Lee. Transporter networks: Rearranging the visual world for robotic manipulation. InProceedings of the Conference on Robot Learning (CoRL), volume 155 ofProceedings of Machine Learning Research, pages 726–747. PMLR, 2021

    2021

  17. [17]

    Shridhar, L

    M. Shridhar, L. Manuelli, and D. Fox. CLIPort: What and where pathways for robotic manipulation. InProceedings of the Conference on Robot Learning (CoRL), volume 164 of Proceedings of Machine Learning Research, pages 894–906. PMLR, 2022

    2022

  18. [18]

    X. Shao, Y . Tang, P. Xie, K. Zhou, Y . Zhuang, et al. More than a point: Capturing uncertainty with adaptive affordance heatmaps for spatial grounding in robotic tasks.arXiv preprint arXiv:2510.10912, 2025

    arXiv 2025

  19. [19]

    Y . Du, M. Yang, P. Florence, and A. Zeng. UniPi: Learning universal policies via text-guided video generation. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  20. [20]

    Y . Hu, Y . Guo, P. Wang, X. Chen, Y .-J. Wang, J. Zhang, K. Sreenath, C. Lu, and J. Chen. Video prediction policy: A generalist robot policy with predictive visual representations.arXiv preprint arXiv:2412.14803, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Black, M

    K. Black, M. Nakamoto, P. Atreya, H. Walke, C. Finn, A. Kumar, and S. Levine. Zero- shot robotic manipulation with pre-trained image-editing diffusion models. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  22. [22]

    X. Shao, J. Zhang, H. Wang, L. M. Brunswic, K. Zhou, J. Dong, K. Guo, Z. Chen, J. Wang, J. Hao, X. Li, and Y . Li. Generative models in decision making: A survey.arXiv preprint arXiv:2502.17100, 2025

    arXiv 2025

  23. [23]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, X. Chen, K. Choromanski, T. Ding, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control.arXiv preprint arXiv:2307.15818, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Ghosh, H

    Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, et al. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky.π0: A vision-language- action flow model for general robot control.arXiv preprint arXiv:2410.24...

    Pith/arXiv arXiv 2024

  26. [26]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. OpenVLA: An open-source vision-language-action model. InProceedings of The 8th Conference on Robot Learning, volume 270 ofProceedings of Machine Learni...

    2025

  27. [27]

    C. Wen, X. Lin, J. I. R. So, K. Chen, Q. Dou, Y . Gao, and P. Abbeel. Any-point trajectory modeling for policy learning. InProceedings of Robotics: Science and Systems (RSS), 2024. arXiv:2401.00025

    Pith/arXiv arXiv 2024

  28. [28]

    Bharadhwaj, R

    H. Bharadhwaj, R. Mottaghi, A. Gupta, and S. Tulsiani. Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation. InEuropean Conference on Computer Vision (ECCV), 2024. arXiv:2405.01527

    Pith/arXiv arXiv 2024

  29. [29]

    M. Xu, Z. Xu, Y . Xu, C. Chi, G. Wetzstein, M. Veloso, and S. Song. Flow as the cross- domain manipulation interface. InProceedings of The 8th Conference on Robot Learning, volume 270 ofProceedings of Machine Learning Research, pages 2475–2499. PMLR, 2025. arXiv:2407.15208. 10

    Pith/arXiv arXiv 2025

  30. [30]

    C. Yuan, C. Wen, T. Zhang, and Y . Gao. General flow as foundation affordance for scalable robot learning. InProceedings of The 8th Conference on Robot Learning, volume 270 ofProceedings of Machine Learning Research, pages 1541–1566. PMLR, 2025. arXiv:2401.11439

    Pith/arXiv arXiv 2025

  31. [31]

    Wan Team, A. Wang, B. Ai, B. Wen, C. Mao, C.-W. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  32. [32]

    Jaegle, F

    A. Jaegle, F. Gimeno, A. Brock, O. Vinyals, A. Zisserman, and J. Carreira. Perceiver: General perception with iterative attention. InInternational Conference on Machine Learning (ICML), pages 4651–4664. PMLR, 2021

    2021

  33. [33]

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. InProceedings of Robotics: Science and Systems (RSS), 2023

    2023

  34. [34]

    Perez, F

    E. Perez, F. Strub, H. de Vries, V . Dumoulin, and A. Courville. FiLM: Visual reasoning with a general conditioning layer. InProceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2018

    2018

  35. [35]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. InProceedings of Robotics: Science and Systems (RSS), 2023

    2023

  36. [36]

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu. RDT-1B: A diffusion foundation model for bimanual manipulation. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  37. [37]

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu. 3D diffusion policy: Generalizable visuomotor policy learning via simple 3D representations. InRobotics: Science and Systems (RSS), 2024

    2024

  38. [38]

    Y . Tang, X. Shao, P. Xie, J. Li, J. Li, Y . Gao, G. Huang, T. Cao, and X. Li. Decoupling semantics and geometric grounding: Spatial visual prompts for language-conditioned imitation learning. arXiv preprint arXiv:2606.25360, 2026

    Pith/arXiv arXiv 2026

  39. [39]

    Carion, L

    N. Carion, L. Gustafson, Y .-T. Hu, S. Debnath, R. Hu, D. Suris Coll-Vinent, C. Ryali, K. V . Alwala, H. Khedr, A. Huang, et al. SAM 3: Segment anything with concepts. InInternational Conference on Learning Representations (ICLR), 2026. arXiv:2511.16719. 11 Appendix Overview This appendix provides implementation details, an auxiliary real-world feasibilit...

    Pith/arXiv arXiv 2026

  40. [40]

    For RoboTwin 2.0, proprioception includes the 14-DoF dual-arm state and end-effector poses

    The global condition concatenates language-modulated visual features, flattened KAM tokens (K×d k = 2,048 dimensions), and proprioceptive features. For RoboTwin 2.0, proprioception includes the 14-DoF dual-arm state and end-effector poses. For LIBERO, it uses the corresponding single-arm state. The action chunk horizon is Ha = 16 . We use a DDPM scheduler...