REVIEW 5 cited by
Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment
read the original abstract
We introduce a novel task within the field of 3D dance generation, termed dance accompaniment, which necessitates the generation of responsive movements from a dance partner, the "follower", synchronized with the lead dancer's movements and the underlying musical rhythm. Unlike existing solo or group dance generation tasks, a duet dance scenario entails a heightened degree of interaction between the two participants, requiring delicate coordination in both pose and position. To support this task, we first build a large-scale and diverse duet interactive dance dataset, DD100, by recording about 117 minutes of professional dancers' performances. To address the challenges inherent in this task, we propose a GPT-based model, Duolando, which autoregressively predicts the subsequent tokenized motion conditioned on the coordinated information of the music, the leader's and the follower's movements. To further enhance the GPT's capabilities of generating stable results on unseen conditions (music and leader motions), we devise an off-policy reinforcement learning strategy that allows the model to explore viable trajectories from out-of-distribution samplings, guided by human-defined rewards. Based on the collected dataset and proposed method, we establish a benchmark with several carefully designed metrics.
Forward citations
Cited by 5 Pith papers
-
InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation
A first large multimodal 4D human–dog interaction dataset (6.8M frames) plus an autoregressive model that generates dog motion from human body/hand gestures and audio.
-
Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generation
A hierarchical global-keyframe then local-refinement diffusion pipeline produces stable 720p/30fps music-to-dance videos longer than one minute across five genres.
-
Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generation
A hierarchical global-keyframe then local-refinement diffusion pipeline produces stable 720p/30fps music-to-dance videos longer than one minute across five genres.
-
Contact Matrix: Enhancing Dance Motion Synthesis with Precise Interaction Modeling
The contact matrix approach in a diffusion model, paired with specialized VQ-VAE, enables more precise and realistic generation of interactive duet dance motions compared to prior methods.
-
Agentic MPC for Semantic Control System Resynthesis
Introduces an agentic MPC framework that uses LLM-based agents to resynthesize control specifications from semantic inputs, demonstrated in an autonomous driving scenario.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.