REVIEW 5 major objections 5 minor 83 references
HelloWorld: Enabling Socially Interactive Characters in Video World Models
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read HelloWorld is a video world model whose characters can be prompted to wave, nod, or speak to the viewer at a chosen moment.
desk verdict First video world model with viewer-directed social interactions; the temporal mask is a clever idea and the benchmark is useful, but the evaluation needs error bars and a clearer CamCtrl protocol before I'd trust the camera-control numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing piece is the warp-video condition: a pseudo-video that lifts the first frame into a point cloud and reprojects it along the target camera path, giving the diffusion transformer an explicit, frame-aligned geometric history condition after visible-token selection discards reprojection holes. The second carrier is the self-distillation loop, in which the frozen base model's own interaction-rich clips are annotated for camera trajectory and point cloud, then reconstructed by a LoRA-finetuned DiT under the flow-matching loss, with the camera-text prompt excluded so camera control is carried solely by the warp video. The third is the temporal cross-attention mask, a training-free modification of Eq. (4) that sets attention to $-\infty$ for query tokens outside the interaction window attending to interaction text tokens, thereby temporally localizing the character's response.
What would settle it
Run the same pose-recovery tool on clips rendered with a known ground-truth camera path, such as a synthetic 3D scene with an animated camera, and measure per-frame pose error; if the error is large on stylized or synthetic content, or if adding small noise to the recovered poses significantly degrades HelloWorld's CamCtrl and TimeAcc, then the self-distillation geometry pipeline is not doing the load-bearing work claimed.
Extended reading notes
Core claim
The paper's central claim is that social interaction can be made a first-class input to a video world model, on par with camera trajectory and text prompt. HelloWorld takes a first-frame image, a text prompt, a camera trajectory, and an interaction window, and generates a video in which the character both follows the camera and performs the prompted viewer-directed action within the window. The method is self-distillation: the frozen base model LTX-2.3 synthesizes clips containing both interactions and camera motion; Pi3X recovers each clip's camera path and first-frame point cloud; a re-rendered warp video becomes the geometric condition; and a rank-32 LoRA is finetuned under flow matching, with the camera text prompt deliberately removed so camera control comes only from the warp video. At inference, a temporal cross-attention mask blocks frames outside the press window from attending to the interaction prompt, localizing the response in time. On HelloWorldBench, HelloWorld reports TimeAcc 81.7% versus 30.9–52.6% for the compared world models, GazeDev 40.2 degrees versus 52.8–77.2 degrees, and the highest CamCtrl at 82.9, while staying competitive on background consistency and aesthetic score; on ActAcc its 41.4% trails LingBot-World's 50.5%, which the paper reads as an untimed behavior that ActAcc rewards regardless of when the action occurs.
Load-bearing premise
The load-bearing premise is that the frozen base model generates videos that genuinely contain both social interactions and natural camera motion, and that the off-the-shelf geometry tool recovers accurate camera poses and first-frame point clouds from those specific generated videos; if pose recovery is wrong on synthetic content, the warp-video condition injects incorrect guidance and the claimed joint preservation of interaction quality and camera control collapses.
Editorial extensions
If this is right
- Users can specify not only what a character does but when it does it, with interaction timing accuracy of 81.7% compared with roughly random 30–52% for trajectory-controlled baselines.
- Camera-pose following does not have to be traded for interaction quality: HelloWorld keeps the best CamCtrl score (82.9) and competitive aesthetic and background-consistency scores among the compared models.
- The system needs no manually collected or annotated interaction video: all training data are generated by the base model itself and processed automatically.
- Training-free temporal control extends to audio: masking both video and audio cross-attention streams raises SpeechInWin from 52.5% to 69.1%, localizing spoken greetings to the interaction window.
- HelloWorldBench introduces three metrics (ActAcc, TimeAcc, GazeDev) that separate what action occurs, when it occurs, and whether it is directed at the viewer, making future social-interaction world models measurable.
Reading between the lines
- The self-distillation recipe should transfer to other base video generators, since it only needs the base model to sample interaction-rich clips and an off-the-shelf geometry tool to annotate them; the cost is a single lightweight fine-tuning run per base model.
- The temporal cross-attention mask is a general event-localization mechanism: the same $-\infty$ masking applied to interaction-related tokens could gate other user-specified events, such as object changes, weather effects, or sound cues, without retraining.
- If pose recovery from generated videos is the bottleneck, a stronger or self-supervised geometry estimator should directly improve GazeDev and CamCtrl; the benchmark could also be extended with adversarial timing windows, such as sub-second presses or overlapping windows, to expose the limits of the mask.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HelloWorld, a video world model that lets users prompt an on-screen character to perform viewer-directed social interactions (waving, nodding, speaking, etc.) within a user-specified time window. The method has two components: a self-distillation finetuning stage in which the base LTX-2.3 model is finetuned on its own generated interaction-rich videos under a warp-video camera condition, and a training-free temporal cross-attention mask that suppresses interaction-related text tokens outside the button-press window. The authors also introduce HelloWorldBench, a 400-sample benchmark with three interaction metrics (ActAcc, TimeAcc, GazeDev) and three conventional metrics (BgCons, Aesthetic, CamCtrl). Experiments report large gains in TimeAcc (81.7% vs. roughly 30–52% for baselines) and GazeDev, competitive ActAcc, and strong user-study preferences.
Significance. If the claims hold, the paper makes a useful contribution: it is the first to address social interaction with the viewer as a first-class capability of video world models, and the temporal cross-attention mask is a clean, training-free mechanism for event timing. The benchmark is a substantial new resource, with 400 samples and interaction-specific metrics, and the user study provides evidence beyond automatic metrics. The main uncertainty is evaluative: the headline quantitative claims rest on metrics that are incompletely specified, one of which excludes failures by construction, and the central camera-control result depends on an unvalidated pose estimator used in the training loop.
major comments (5)
- [§5.3, Table 1] The claim in Section 5.3 that HelloWorld outperforms existing methods on all three interaction metrics is contradicted by the reported ActAcc values: HelloWorld scores 41.4, while LTX-2.3 scores 42.5 and LingBot-World scores 50.5. The text later acknowledges that LingBot-World attains the highest ActAcc, but the introductory and abstract-level claim is not qualified. Please restate the claim to make clear that ActAcc is not state-of-the-art, and discuss the trade-off between temporal localization and action accuracy, as the no-mask ablation in Table 3 already suggests.
- [§3.2, Eq. (1)–(3)] The training loop uses Pi3X to recover per-frame camera poses and first-frame point clouds from LTX-2.3's own synthetic generations, yet the paper provides no validation of Pi3X on that distribution. Since the warp video is the only camera-conditioning input, systematic pose or depth errors on interaction-rich, non-rigid content would propagate directly into the learned conditioning and could explain part of the reported camera-control performance. Please add a quantitative validation, e.g., comparing Pi3X-estimated trajectories on held-out self-generated clips against known synthetic trajectories or human-annotated correspondences, and report the resulting pose error.
- [§4.2, Table 1] CamCtrl is never defined in the manuscript. If it is computed with the same Pi3X estimator that generates the training conditions, the reported 82.9 could partly reflect estimator bias rather than true camera following. Please state the exact computation, identify the estimator and any alignment step, and, if possible, evaluate camera following with an independent measure (e.g., optical-flow consistency or a different pose estimator) to rule out circularity.
- [§4.2, TimeAcc definition] The TimeAcc computation excludes videos in which the VLM judge selects the 'no interaction' option. This can inflate the metric, because a model that never performs the interaction is never penalized for timing; it also makes the 81.7% figure not directly comparable across methods if they produce different shares of no-interaction outputs. Please report TimeAcc computed over all samples (treating 'no interaction' as incorrect) and report the per-method exclusion rates, or justify why the excluded cases are uninformative.
- [§5.2, Table 1] The paper states that each sample is run with three seeds and that averaged results are reported, but the tables contain no error bars, confidence intervals, or significance tests. Without these, several headline gaps (e.g., ActAcc 41.4 for HelloWorld vs. 42.5 for LTX-2.3, and differences among TimeAcc values near 30–40%) could be within run-to-run noise. Please report variance or confidence intervals for the main metrics and, where appropriate, a paired significance test across the three seeds.
minor comments (5)
- [§4.2, GazeDev] GazeDev is computed only on the 217 human samples, but the paper does not explain why 217 of the 400 samples are human or how the non-human samples contribute to the headline GazeDev number; please clarify the sample split and the aggregation.
- [§5.2, VLM judge] The evaluation uses 'Qwen3.6-35B-A3B' as the VLM judge; please provide a precise model identifier or checkpoint and include the full evaluation prompts used for ActAcc, TimeAcc, and GazeDev in an appendix to make the benchmark reproducible.
- [§4.2, §5.5] SpeechInWin and the user-study question 'Action Naturalness' are used without a formal definition in the main text; please define them explicitly, including how Whisper transcriptions are matched to the interaction window.
- [§5.4, Table 2] The 'Real-video' ablation setting is said to train on real data from Warp-as-History [60], but the number of real clips, their resolution, and the training steps are not stated; please report these details so that the comparison with the self-distillation data is interpretable.
- [§3.3, Fig. 1] The interaction button F is described as a press that opens a window [τs, τe], but Figure 1 and the text do not clarify whether a single click or a press-and-hold defines the window; please state how users specify both the start and the duration of the interaction.
Circularity Check
No demonstrated circularity: the only self-citation is corroborated in-paper, and the Pi3X/CamCtrl loop is an unspecified risk, not a shown reduction.
full rationale
Walking the claimed derivation chain, no step reduces a reported result to its own input by definition. (i) Self-distillation (Sec. 3.2) is a reconstruction objective: Eq. (3) trains the LoRA to predict the velocity field of the base model's own generated video given the warp-video condition derived from that same video; the reported interaction metrics are then measured on external Unsplash first frames (Sec. 4.1) with an external VLM judge and UniGaze, so the evaluation is not fitted to the training signal. (ii) The temporal gate of Eq. (4) directly suppresses cross-attention to interaction tokens outside the press window; high TimeAcc is therefore partly a check that the architectural mechanism operates as specified, and the ablation in Table 3 (no-mask TimeAcc 52.5 vs. masked 81.7) shows the measurement is not vacuous, but this is an architectural intervention rather than a fitted input renamed as prediction. (iii) The only load-bearing self-citation is [42] for 'The base video generation model itself is capable of producing rich social interactions'; Table 1 independently corroborates this with LTX-2.3's ActAcc of 42.5, so the argument does not reduce to a self-citation chain. (iv) The Pi3X loop is a legitimate correctness risk: if CamCtrl is scored by the same pose estimator used to build the warp-video training conditions, then camera-control numbers could be inflated; however, the paper never defines how CamCtrl is computed, so applying the hard rule that circularity must be exhibited by quoted equations, this remains an unverified risk rather than a demonstrated self-reduction. Overall the benchmark and the main interaction metrics are externally grounded; score 2 reflects only the minor, non-load-bearing self-citation and the unquantified estimator-alignment risk.
Assumptions & free parameters
free parameters (4)
- LoRA rank =
32
- Learning rate =
1e-4
- Training steps =
2000
- Warp attention strength =
0.3
assumptions (4)
- domain assumption Base model LTX-2.3 reliably generates interaction-rich videos when prompted
- domain assumption Pi3X recovers accurate first-frame point clouds and per-frame camera poses from generated videos
- domain assumption Warp-video conditioning from [60] works for LTX-2.3 with LoRA finetuning
- domain assumption Qwen3.6-35B-A3B VLM judge correctly identifies actions and timing
Cite this review
Pith. "Pith review of HelloWorld: Enabling Socially Interactive Characters in Video World Models." pith.science (2026). https://pith.science/paper/XHFOXYAO
@misc{pith2026260805070,
author = {Pith},
title = {Pith review of: HelloWorld: Enabling Socially Interactive Characters in Video World Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/XHFOXYAO}},
note = {Machine review of arXiv:2608.05070}
}
read the original abstract
Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: https://github.com/AlayaLab/HelloWorld
Reference graph
Works this paper leans on
-
[1]
Cosmos 3: Omnimodal world models for physical ai.arXiv preprint arXiv:2606.02800, 2026
Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, et al. Cosmos 3: Omnimodal world models for physical ai.arXiv preprint arXiv:2606.02800, 2026
arXiv 2026
-
[2]
From masks to worlds: A hitchhiker’s guide to world models.arXiv preprint arXiv:2510.20668, 2025
Jinbin Bai, Yu Lei, Hecong Wu, Yuchen Zhu, Shufan Li, Yi Xin, Xiangtai Li, Molei Tao, Aditya Grover, and Ming-Hsuan Yang. From masks to worlds: A hitchhiker’s guide to world models.arXiv preprint arXiv:2510.20668, 2025
-
[3]
Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025
arXiv 2025
-
[4]
Philip J. Ball, Jakob Bauer, Frank Belletti, Bethanie Brownfield, Ariel Ephrat, Shlomi Fruchter, Agrim Gupta, Kristian Holsheimer, Aleksander Holynski, Jiri Hron, Christos Kaplanis, Marjorie Limont, Matt McGill, Yanko Oliveira, Jack Parker-Holder, Frank Perbet, Guy Scully, Jeremy Shar, Stephen Spencer, Omer Tov, Ruben Villegas, Emma Wang, Jessica Yung, Ci...
2025
-
[5]
Ambigest: A dataset of social gestures with inter-class similarity and intra-class variability
Hajra Anwar Beg, Mohamed Daoudi, and Angela Bartolo. Ambigest: A dataset of social gestures with inter-class similarity and intra-class variability. In2026 IEEE 20th International Conference on Automatic Face and Gesture Recognition (FG), pages 1–5. IEEE, 2026
2026
-
[6]
Video generation models as world simulators
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. URL https://openai.com/research/ video-generation-models-as-world-simulators
2024
-
[7]
Genie: Generative interactive environments
Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. InForty-first International Conference on Machine Learning, 2024
2024
-
[8]
Socialgesture: Delving into multi-person gesture understanding
Xu Cao, Pranav Virupaksha, Wenqi Jia, Bolin Lai, Fiona Ryan, Sangmin Lee, and James M Rehg. Socialgesture: Delving into multi-person gesture understanding. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 19509–19519, 2025
2025
Show all 83 references
-
[9]
Gamegen-x: Interactive open- world game video generation
Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x: Interactive open- world game video generation. InInternational Conference on Learning Representations, volume 2025, pages 37546–37593, 2025
2025
-
[10]
Unils: End-to-end audio-driven avatars for unified listening and speaking
Xuangeng Chu, Ruicong Liu, Yifei Huang, Yun Liu, Yichen Peng, and Bo Zheng. Unils: End-to-end audio-driven avatars for unified listening and speaking. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 25142–25152, 2026
2026
-
[11]
Oasis: A universe in a transformer.URL: https://oasis-model
Etched Decart, Quinn McIntyre, Spruce Campbell, Xinlei Chen, and Robert Wachen. Oasis: A universe in a transformer.URL: https://oasis-model. github. io, 2(3):6, 2024
2024
-
[12]
Worldscore: A unified evaluation benchmark for world generation
Haoyi Duan, Hong-Xing Yu, Sirui Chen, Li Fei-Fei, and Jiajun Wu. Worldscore: A unified evaluation benchmark for world generation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 27713–27724, 2025
2025
-
[13]
The matrix: Infinite-horizon world generation with real-time moving control.Advances in Neural Information Processing Systems, 38:87318–87344, 2026
Ruili Feng, Han Zhang, Zhilei Shu, Zhantao Yang, Longxiang Tang, Zhicai Wang, Andy Zheng, Jie Xiao, Zhiheng Liu, Ruihang Chu, et al. The matrix: Infinite-horizon world generation with real-time moving control.Advances in Neural Information Processing Systems, 38:87318–87344, 2026
2026
-
[14]
Longvie 2: Multimodal controllable ultra-long video world model
Jianxiong Gao, Zhaoxi Chen, Xian Liu, Junhao Zhuang, Chengming Xu, Jianfeng Feng, Yu Qiao, Yanwei Fu, Chenyang Si, and Ziwei Liu. Longvie 2: Multimodal controllable ultra-long video world model. arXiv preprint arXiv:2512.13604, 2025
2025
-
[15]
Mineworld: a real-time and open-source interactive world model on minecraft.arXiv preprint arXiv:2504.08388, 2025
Junliang Guo, Yang Ye, Tianyu He, Haoyu Wu, Yushu Jiang, Tim Pearce, and Jiang Bian. Mineworld: a real-time and open-source interactive world model on minecraft.arXiv preprint arXiv:2504.08388, 2025
2025 arXiv
-
[16]
Ltx-2: Efficient joint audio-visual foundation model.arXiv preprint arXiv:2601.03233, 2026
Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, Andrew Kvochko, Avishai Berkowitz, Daniel Shalem, Daphna Lifschitz, Dudu Moshe, Eitan Porat, et al. Ltx-2: Efficient joint audio-visual foundation model.arXiv preprint arXiv:2601.03233, 2026
2026 arXiv
-
[17]
Voyager: Long-range and world-consistent video diffusion for explorable 3d scene generation.ACM Transactions on Graphics (TOG), 44(6):1–15, 2025
Tianyu Huang, Wangguandong Zheng, Tengfei Wang, Yuhao Liu, Zhenwei Wang, Junta Wu, Jie Jiang, Hui Li, Rynson Lau, Wangmeng Zuo, et al. Voyager: Long-range and world-consistent video diffusion for explorable 3d scene generation.ACM Transactions on Graphics (TOG), 44(6):1–15, 2025. 11
2025
-
[18]
Bind-your-avatar: Multi- character-talking video generation with dynamic 3d-mask-based embedding router
Yubo Huang, Weiqiang Wang, Sirui Zhao, Tong Xu, Lin Liu, and Enhong Chen. Bind-your-avatar: Multi- character-talking video generation with dynamic 3d-mask-based embedding router. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4440–4449, 2026
2026
-
[19]
Can mllms read the room? a multimodal benchmark for assessing deception in multi-party social interactions.arXiv preprint arXiv:2511.16221, 2025
Caixin Kang, Yifei Huang, Liangyang Ouyang, Mingfang Zhang, Ruicong Liu, and Yoichi Sato. Can mllms read the room? a multimodal benchmark for assessing deception in multi-party social interactions.arXiv preprint arXiv:2511.16221, 2025
2025
-
[20]
Grasp: Learning to ground social reasoning in multi-person non-verbal interactions
Junho Kim, Xu Cao, Houze Yang, Bikram Boote, Ana Jojic, Fiona Ryan, Bolin Lai, Sangmin Lee, and James M Rehg. Grasp: Learning to ground social reasoning in multi-person non-verbal interactions. arXiv preprint arXiv:2605.15764, 2026
2026 arXiv
-
[21]
Let them talk: Audio-driven multi-person conversational video generation.Advances in Neural Information Processing Systems, 38:70990–71013, 2026
Zhe Kong, Feng Gao, Yong Zhang, Zhuoliang Kang, Xiaoming Wei, Xunliang Cai, Guanying Chen, and Wenhan Luo. Let them talk: Audio-driven multi-person conversational video generation.Advances in Neural Information Processing Systems, 38:70990–71013, 2026
2026
-
[22]
Modeling multimodal social interactions: new challenges and baselines with densely aligned representations
Sangmin Lee, Bolin Lai, Fiona Ryan, Bikram Boote, and James M Rehg. Modeling multimodal social interactions: new challenges and baselines with densely aligned representations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14585–14595, 2024
2024
-
[23]
Worldmodelbench: Judging video generation models as world models.Advances in Neural Information Processing Systems, 38, 2026
Dacheng Li, Yunhao Fang, Yukang Chen, Shuo Yang, Shiyi Cao, Justin Wong, Michael Luo, Xiaolong Wang, Hongxu Yin, Joseph Gonzalez, et al. Worldmodelbench: Judging video generation models as world models.Advances in Neural Information Processing Systems, 38, 2026
2026
-
[24]
Hunyuan-gamecraft: High-dynamic interactive game video generation with hybrid history condition.arXiv preprint arXiv:2506.17201, 2(3):6, 2025
Jiaqi Li, Junshu Tang, Zhiyong Xu, Longhuang Wu, Yuan Zhou, Shuai Shao, Tianbao Yu, Zhiguo Cao, and Qinglin Lu. Hunyuan-gamecraft: High-dynamic interactive game video generation with hybrid history condition.arXiv preprint arXiv:2506.17201, 2(3):6, 2025
2025 arXiv
-
[25]
Towards online multi-modal social interaction understanding.Transactions on Machine Learning Research (TMLR), 2026
Xinpeng Li, Shijian Deng, Bolin Lai, Weiguo Pian, James M Rehg, and Yapeng Tian. Towards online multi-modal social interaction understanding.Transactions on Machine Learning Research (TMLR), 2026
2026
-
[26]
Omni-mmsi: Toward identity-attributed social interaction understanding
Xinpeng Li, Bolin Lai, Hardy Chen, Shijian Deng, Cihang Xie, Yuyin Zhou, James M Rehg, and Yapeng Tian. Omni-mmsi: Toward identity-attributed social interaction understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8684–8696, 2026
2026
-
[27]
Sekai: A video dataset towards world exploration
Zhen Li, Chuanhao Li, Xiaofeng Mao, Shaoheng Lin, Ming Li, Shitian Zhao, Zhaopan Xu, Xinyue Li, Yukang Feng, Jianwen Sun, Zizhen Li, Fanrui Zhang, Jiaxin Ai, Zhixiang Wang, Yuwei Wu, Tong He, Yunde Jia, and Kaipeng Zhang. Sekai: A video dataset towards world exploration. In D....
2025
-
[28]
Wildworld: A large-scale dataset for dynamic world modeling with actions and explicit state toward generative arpg.arXiv preprint arXiv:2603.23497, 2026
Zhen Li, Zian Meng, Shuwei Shi, Wenshuo Peng, Yuwei Wu, Bo Zheng, Chuanhao Li, and Kaipeng Zhang. Wildworld: A large-scale dataset for dynamic world modeling with actions and explicit state toward generative arpg.arXiv preprint arXiv:2603.23497, 2026
2026
-
[29]
Intergen: Diffusion-based multi- human motion generation under complex interactions.International Journal of Computer Vision, 132(9): 3463–3483, 2024
Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, and Lan Xu. Intergen: Diffusion-based multi- human motion generation under complex interactions.International Journal of Computer Vision, 132(9): 3463–3483, 2024
2024
-
[30]
Polyslgen: Online multimodal speaking-listening reaction generation in polyadic interaction
Zhi-Yi Lin, Thomas Markhorst, Jouh Yeong Chew, and Xucong Zhang. Polyslgen: Online multimodal speaking-listening reaction generation in polyadic interaction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 29379–29390, 2026
2026
-
[31]
Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pa...
2024
-
[32]
Uvagaze: Unsupervised 1-to-2 views adaptation for gaze estimation
Ruicong Liu and Feng Lu. Uvagaze: Unsupervised 1-to-2 views adaptation for gaze estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 3693–3701, 2024. 12
2024
-
[33]
Pnp-ga+: Plug-and-play domain adaptation for gaze estimation using model variants.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46 (5):3707–3721, 2024
Ruicong Liu, Yunfei Liu, Haofei Wang, and Feng Lu. Pnp-ga+: Plug-and-play domain adaptation for gaze estimation using model variants.IEEE Transactions on Pattern Analysis and Machine Intelligence, 46 (5):3707–3721, 2024
2024
-
[34]
Single-to-dual-view adaptation for egocentric 3d hand pose estimation
Ruicong Liu, Takehiko Ohkawa, Mingfang Zhang, and Yoichi Sato. Single-to-dual-view adaptation for egocentric 3d hand pose estimation. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 677–686. IEEE, 2024
2024
-
[35]
Sfhand: A streaming framework for language-guided 3d hand forecasting and embodied manipulation.arXiv e-prints, pages arXiv–2511, 2025
Ruicong Liu, Yifei Huang, Liangyang Ouyang, Caixin Kang, and Yoichi Sato. Sfhand: A streaming framework for language-guided 3d hand forecasting and embodied manipulation.arXiv e-prints, pages arXiv–2511, 2025
2025
-
[36]
Generalizing gaze estimation with outlier-guided collaborative adaptation
Yunfei Liu, Ruicong Liu, Haofei Wang, and Feng Lu. Generalizing gaze estimation with outlier-guided collaborative adaptation. InProceedings of the IEEE/CVF international conference on computer vision, pages 3835–3844, 2021
2021
-
[37]
Playmate2: Training-free multi-character audio-driven animation via diffusion transformer with reward feedback.arXiv preprint arXiv:2510.12089, 2025
Xingpei Ma, Shenneng Huang, Jiaran Cai, Yuansheng Guan, Shen Zheng, Hanfeng Zhao, Qiang Zhang, and Shunsi Zhang. Playmate2: Training-free multi-character audio-driven animation via diffusion transformer with reward feedback.arXiv preprint arXiv:2510.12089, 2025
-
[38]
Xiaofeng Mao, Zhen Li, Chuanhao Li, Xiaojie Xu, Kaining Ying, and Kaipeng Zhang. Yume1. 5: A text-controlled interactive world generation model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7752–7761, 2026
2026
-
[39]
Multimediate: Multi-modal group behaviour analysis for artificial mediation
Philipp Müller, Michael Dietz, Dominik Schiller, Dominike Thomas, Guanhua Zhang, Patrick Gebhard, Elisabeth André, and Andreas Bulling. Multimediate: Multi-modal group behaviour analysis for artificial mediation. InProceedings of the 29th ACM International Conference on Multim...
2021
-
[40]
Leadership assessment in pediatric intensive care unit team training.arXiv preprint arXiv:2505.24389, 2025
Liangyang Ouyang, Yuki Sakai, Ryosuke Furuta, Hisataka Nozawa, Hikoro Matsui, and Yoichi Sato. Leadership assessment in pediatric intensive care unit team training.arXiv preprint arXiv:2505.24389, 2025
2025 arXiv
-
[41]
Multi-speaker attention alignment for multimodal social interaction
Liangyang Ouyang, Yifei Huang, Mingfang Zhang, Caixin Kang, Ryosuke Furuta, and Yoichi Sato. Multi-speaker attention alignment for multimodal social interaction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24608–24619, 2026
2026
-
[42]
Socialdirector: Training- free social interaction control for multi-person video generation.arXiv preprint arXiv:2605.10079, 2026
Liangyang Ouyang, Ruicong Liu, Caixin Kang, Yifei Huang, and Yoichi Sato. Socialdirector: Training- free social interaction control for multi-person video generation.arXiv preprint arXiv:2605.10079, 2026
2026 arXiv
-
[43]
Svbench: Evaluation of video generation models on social reasoning
Wenshuo Peng, Gongxuan Wang, Tianmeng Yang, Chuanhao Li, Xiaojie Xu, Hui He, and Kaipeng Zhang. Svbench: Evaluation of video generation models on social reasoning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 32872–32881, 2026
2026
-
[44]
Dyadit: A multi-modal diffusion transformer for socially favorable dyadic gesture generation.arXiv preprint arXiv:2602.23165, 2026
Yichen Peng, Jyun-Ting Song, Siyeol Jung, Ruofan Liu, Haiyang Liu, Xuangeng Chu, Ruicong Liu, Erwin Wu, Hideki Koike, and Kris Kitani. Dyadit: A multi-modal diffusion transformer for socially favorable dyadic gesture generation.arXiv preprint arXiv:2602.23165, 2026
2026
-
[45]
Actavatar: Temporally-aware precise action control for talking avatars
Ziqiao Peng, Yi Chen, Yifeng Ma, Guozhen Zhang, Zhiyao Sun, Zixiang Zhou, Youliang Zhang, Zhengguang Zhou, Zhaoxin Fan, Hongyan Liu, et al. Actavatar: Temporally-aware precise action control for talking avatars. InProceedings of the IEEE/CVF Conference on Computer Vision and P...
2026
-
[46]
Unigaze: Towards universal gaze estimation via large-scale pre-training
Jiawei Qin, Xucong Zhang, and Yusuke Sugano. Unigaze: Towards universal gaze estimation via large-scale pre-training. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5809–5820, 2026
2026
-
[47]
Qwen3.6-35B-A3B: Agentic coding power, now open to all, April 2026
Qwen Team. Qwen3.6-35B-A3B: Agentic coding power, now open to all, April 2026. URL https: //qwen.ai/blog?id=qwen3.6-35b-a3b. 13
2026
-
[48]
Robust speech recognition via large-scale weak supervision
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. InInternational conference on machine learning, pages 28492–28518. PMLR, 2023
2023
-
[49]
Interact2ar: Full-body human-human interaction generation via autoregressive diffusion models
Pablo Ruiz-Ponce, Sergio Escalera, José García-Rodríguez, Jiankang Deng, and Rolandos Alexandros Potamias. Interact2ar: Full-body human-human interaction generation via autoregressive diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...
2026
-
[50]
Solaris: Building a multiplayer video world model in minecraft.arXiv preprint arXiv:2602.22208, 2026
Georgy Savva, Oscar Michel, Daohan Lu, Suppakit Waiwitlikhit, Timothy Meehan, Dhairya Mishra, Srivats Poddar, Jack Lu, and Saining Xie. Solaris: Building a multiplayer video world model in minecraft.arXiv preprint arXiv:2602.22208, 2026
2026
-
[51]
Prisma-world: Camera-controllable multi-agent video world model
Huiqiang Sun, Zhan Peng, Size Wu, Kun Wang, Kang Liao, Dianyi Wang, Xingyu Zeng, Sheng Jin, Yangguang Li, Zhiguo Cao, et al. Prisma-world: Camera-controllable multi-agent video world model. arXiv preprint arXiv:2606.09507, 2026
2026 arXiv
-
[52]
Worldplay: Towards long-term geometric consistency for real-time interactive world modeling.arXiv preprint arXiv:2512.14614, 2025
Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. Worldplay: Towards long-term geometric consistency for real-time interactive world modeling.arXiv preprint arXiv:2512.14614, 2025
2025 arXiv
-
[53]
Alayaworld: Long-horizon and playable video world generation,
AlayaWorld Team, Kaipeng Zhang, Chuanhao Li, Yifan Zhan, Yongtao Ge, Yuanyang Yin, Jiaming Tan, Kang He, Liaoyuan Fan, Ruicong Liu, Xiaojie Xu, Xuangeng Chu, Zhen Li, Zhengyuan Lin, Zhixiang Wang, Zian Meng, and Zihui Gao. Alayaworld: Long-horizon and playable video world generation,
-
[54]
Advancing open-source world models.arXiv preprint arXiv:2601.20540, 2026
Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, Jiapeng Zhu, Ka Leong Cheng, Yixuan Li, Hanlin Wang, Yinghao Xu, Shuailei Ma, et al. Advancing open-source world models.arXiv preprint arXiv:2601.20540, 2026
2026 arXiv
-
[55]
Unsplash: Beautiful, free images and photos
Unsplash. Unsplash: Beautiful, free images and photos. https://unsplash.com, 2026. Accessed: 2026-07-17
2026
-
[56]
Diffusion models are real-time game engines
Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. InInternational Conference on Learning Representations, volume 2025, pages 73754–73776, 2025
2025
-
[57]
Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025
2025 arXiv
-
[58]
Spatialvid: A large-scale video dataset with spatial annotations
Jiahao Wang, Yufeng Yuan, Rujie Zheng, Youtian Lin, Jian Gao, Lin-Zhuo Chen, Yajie Bao, Chang Zeng, Yanxi Zhou, Xiao-Xiao Long, et al. Spatialvid: A large-scale video dataset with spatial annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...
2026
-
[59]
Fantasyportrait: Enhancing multi-character portrait animation with expression-augmented diffusion transformers
Qiang Wang, Mengchao Wang, Fan Jiang, Yaqi Fan, Yonggang Qi, and Mu Xu. Fantasyportrait: Enhancing multi-character portrait animation with expression-augmented diffusion transformers. arXiv preprint arXiv:2507.12956, 2025
2025 arXiv
-
[60]
Warp-as-history: Generalizable camera-controlled video generation from one training video.arXiv preprint arXiv:2605.15182, 2026
Yifan Wang and Tong He. Warp-as-history: Generalizable camera-controlled video generation from one training video.arXiv preprint arXiv:2605.15182, 2026
2026 arXiv
-
[61]
π3: Permutation-equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025
Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. π3: Permutation-equivariant visual geometry learning.arXiv preprint arXiv:2507.13347, 2025
2025 arXiv
-
[62]
Reactivegwm: Steering npc in reactive game world models.arXiv preprint arXiv:2605.15256, 2026
Zeqing Wang, Danze Chen, Zhaohu Xing, Zizhao Tong, Yinhan Zhang, Xingyi Yang, and Yeying Jin. Reactivegwm: Steering npc in reactive game world models.arXiv preprint arXiv:2605.15256, 2026
2026 arXiv
-
[63]
Interacthuman: Multi-concept human animation with layout-aligned audio conditions.arXiv preprint arXiv:2506.09984, 2025
Zhenzhi Wang, Jiaqi Yang, Jianwen Jiang, Chao Liang, Gaojie Lin, Zerong Zheng, Ceyuan Yang, Yuan Zhang, Mingyuan Gao, and Dahua Lin. Interacthuman: Multi-concept human animation with layout-aligned audio conditions.arXiv preprint arXiv:2506.09984, 2025. 14
2025
-
[64]
Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026
Zile Wang, Zexiang Liu, Jiaxing Li, Kaichen Huang, Baixin Xu, Fei Kang, Mengyin An, Peiyu Wang, Biao Jiang, Yichen Wei, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026
2026 arXiv
-
[65]
Actworld: From explorable to interactive world model via action-aware memory.arXiv preprint arXiv:2606.17730, 2026
Zhexiao Xiong, Yizhi Song, Hao Kang, Qing Yan, Liming Jiang, Jenson Yang, Zhoujie Fu, Stathi Fotiadis, Angtian Wang, Zichuan Liu, et al. Actworld: From explorable to interactive world model via action-aware memory.arXiv preprint arXiv:2606.17730, 2026
2026 arXiv
-
[66]
Inter-x: Towards versatile human-human interaction analysis
Liang Xu, Xintao Lv, Yichao Yan, Xin Jin, Shuwen Wu, Congsheng Xu, Yifan Liu, Yizhou Zhou, Fengyun Rao, Xingdong Sheng, et al. Inter-x: Towards versatile human-human interaction analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pag...
2024
-
[67]
Worldmark: A unified benchmark suite for interactive video world models.arXiv preprint arXiv:2604.21686, 2026
Xiaojie Xu, Zhengyuan Lin, Kang He, Yukang Feng, Xiaofeng Mao, Yuanyang Yin, Kaipeng Zhang, and Yongtao Ge. Worldmark: A unified benchmark suite for interactive video world models.arXiv preprint arXiv:2604.21686, 2026
2026 arXiv
-
[68]
Holo-world: Unified camera, object and weather control for video world model.arXiv preprint arXiv:2606.20083, 2026
Xiangchen Yin, Wenzhang Sun, Jiahui Yuan, Zijie Liu, Yinda Chen, Wei Li, Dachun Kai, Chunfeng Wang, and Xiaoyan Sun. Holo-world: Unified camera, object and weather control for video world model.arXiv preprint arXiv:2606.20083, 2026
2026 arXiv
-
[69]
Wbench: A comprehensive multi-turn benchmark for interactive video world model evaluation.arXiv preprint arXiv:2605.25874, 2026
Kaining Ying, Hengrui Hu, Siyu Ren, Jiamu Li, Fengjiao Chen, Ziwen Wang, Xuezhi Cao, Xunliang Cai, and Henghui Ding. Wbench: A comprehensive multi-turn benchmark for interactive video world model evaluation.arXiv preprint arXiv:2605.25874, 2026
2026 arXiv
-
[70]
Building social world models with large language models.arXiv preprint arXiv:2606.11482, 2026
Haofei Yu, Yining Zhao, Guanyu Lin, and Jiaxuan You. Building social world models with large language models.arXiv preprint arXiv:2606.11482, 2026
2026 arXiv
-
[71]
Socialgen: Modeling multi-human social interaction with language models
Heng Yu, Juze Zhang, Changan Chen, Tiange Xiang, Yusu Fang, Juan Carlos Niebles, and Ehsan Adeli. Socialgen: Modeling multi-human social interaction with language models. In2026 International Conference on 3D Vision (3DV), pages 1–17. IEEE, 2026
2026
-
[72]
Gamefactory: Creating new games with generative interactive videos
Jiwen Yu, Yiran Qin, Xintao Wang, Pengfei Wan, Di Zhang, and Xihui Liu. Gamefactory: Creating new games with generative interactive videos. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11590–11599, 2025
2025
-
[73]
Helios: Real real-time long video generation model.arXiv preprint arXiv:2603.04379, 2026
Shenghai Yuan, Yuanyang Yin, Zongjian Li, Xinwei Huang, Xiao Yang, and Li Yuan. Helios: Real real-time long video generation model.arXiv preprint arXiv:2603.04379, 2026
2026
-
[74]
Social-iq: A question answering benchmark for artificial social intelligence
Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, and Louis-Philippe Morency. Social-iq: A question answering benchmark for artificial social intelligence. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8807–8817, 2019
2019
-
[75]
Socioverse: A world model for social simulation powered by llm agents and a pool of 10 million real-world users.arXiv preprint arXiv:2504.10157, 2025
Xinnong Zhang, Jiayu Lin, Xinyi Mou, Shiyue Yang, Xiawei Liu, Libo Sun, Hanjia Lyu, Yihang Yang, Weihong Qi, Yue Chen, et al. Socioverse: A world model for social simulation powered by llm agents and a pool of 10 million real-world users.arXiv preprint arXiv:2504.10157, 2025
2025 arXiv
-
[76]
Anytalker: Scaling multi-person talking video generation with interactivity refinement.arXiv preprint arXiv:2511.23475, 2025
Zhizhou Zhong, Yicheng Ji, Zhe Kong, Yiying Liu, Jiarui Wang, Jiasun Feng, Lupeng Liu, Xiangyi Wang, Yanjia Li, Yuqing She, et al. Anytalker: Scaling multi-person talking video generation with interactivity refinement.arXiv preprint arXiv:2511.23475, 2025
2025
-
[77]
Stereo magnification: learning view synthesis using multiplane images.ACM Transactions on Graphics (TOG), 37(4):1–12, 2018
Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: learning view synthesis using multiplane images.ACM Transactions on Graphics (TOG), 37(4):1–12, 2018
2018
-
[78]
Social world models.arXiv preprint arXiv:2509.00559, 2025
Xuhui Zhou, Jiarui Liu, Akhila Yerukola, Hyunwoo Kim, and Maarten Sap. Social world models.arXiv preprint arXiv:2509.00559, 2025
2025
-
[79]
Omniworld: A multi-domain and multi-modal dataset for 4d world modeling.arXiv preprint arXiv:2509.12201, 2025
Yang Zhou, Yifan Wang, Jianjun Zhou, Wenzheng Chang, Haoyu Guo, Zizun Li, Kaijing Ma, Xinyue Li, Yating Wang, Haoyi Zhu, et al. Omniworld: A multi-domain and multi-modal dataset for 4d world modeling.arXiv preprint arXiv:2509.12201, 2025
2025
-
[80]
Evaltalker: Learning to evaluate real-portrait-driven multi-subject talking humans.arXiv preprint arXiv:2512.01340, 2025
Yingjie Zhou, Xilei Zhu, Siyu Ren, Ziyi Zhao, Ziwen Wang, Farong Wen, Yu Zhou, Jiezhang Cao, Xiongkuo Min, Fengjiao Chen, et al. Evaltalker: Learning to evaluate real-portrait-driven multi-subject talking humans.arXiv preprint arXiv:2512.01340, 2025. 15
2025
-
[81]
Sana-wm: Efficient minute-scale world modeling with hybrid linear diffusion transformer
Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye, Junsong Chen, Jincheng Yu, Tong He, Song Han, and Enze Xie. Sana-wm: Efficient minute-scale world modeling with hybrid linear diffusion transformer. arXiv preprint arXiv:2605.15178, 2026
2026 arXiv
-
[82]
Astra: General interactive world model with autoregressive denoising.arXiv preprint arXiv:2512.08931, 2025
Yixuan Zhu, Jiaqi Feng, Wenzhao Zheng, Yuan Gao, Xin Tao, Pengfei Wan, Jie Zhou, and Jiwen Lu. Astra: General interactive world model with autoregressive denoising.arXiv preprint arXiv:2512.08931, 2025. 16
2025
-
[2026]
URLhttps://arxiv.org/abs/2607.06291
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.