{"work":{"id":"94cf4d33-9a24-4155-b8ea-0a2f2d71fadc","openalex_id":null,"doi":null,"arxiv_id":"2406.14540","raw_key":null,"title":"IRASim: A Fine-Grained World Model for Robot Manipulation","authors":null,"authors_text":"Fangqi Zhu, Hongtao Wu, Song Guo, Yuxiao Liu, Chilam Cheang, and Tao Kong","year":2024,"venue":"cs.RO","abstract":"World models allow autonomous agents to plan and explore by predicting the visual outcomes of different actions. However, for robot manipulation, it is challenging to accurately model the fine-grained robot-object interaction within the visual space using existing methods which overlooks precise alignment between each action and the corresponding frame. In this paper, we present IRASim, a novel world model capable of generating videos with fine-grained robot-object interaction details, conditioned on historical observations and robot action trajectories. We train a diffusion transformer and introduce a novel frame-level action-conditioning module within each transformer block to explicitly model and strengthen the action-frame alignment. Extensive experiments show that: (1) the quality of the videos generated by our method surpasses all the baseline methods and scales effectively with increased model size and computation; (2) policy evaluations using IRASim exhibit a strong correlation with those using the ground-truth simulator, highlighting its potential to accelerate real-world policy evaluation; (3) testing-time scaling through model-based planning with IRASim significantly enhances policy performance, as evidenced by an improvement in the IoU metric on the Push-T benchmark from 0.637 to 0.961; (4) IRASim provides flexible action controllability, allowing virtual robotic arms in datasets to be controlled via a keyboard or VR controller.","external_url":"https://arxiv.org/abs/2406.14540","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-07-10T04:16:48.676072+00:00","pith_arxiv_id":"2406.14540","created_at":"2026-05-10T23:38:46.194777+00:00","updated_at":"2026-07-10T04:16:48.676072+00:00","title_quality_ok":true,"display_title":"Irasim: A fine-grained world model for robot manipulation","render_title":"Irasim: A fine-grained world model for robot manipulation"},"hub":{"state":{"work_id":"94cf4d33-9a24-4155-b8ea-0a2f2d71fadc","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":25,"external_cited_by_count":null,"distinct_field_count":4,"first_pith_cited_at":"2025-01-07T06:55:50+00:00","last_pith_cited_at":"2026-07-09T16:15:43+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-08-22T14:49:38.809871+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":4}],"polarity_counts":[{"context_polarity":"background","n":4}],"runs":{},"summary":{},"graph":{},"authors":[]}}