Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modeling

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EnvPoser estimates full-body VR motion by letting a pre-scanned room disambiguate sparse head-and-hand tracking.

desk verdict Useful environment-aware integration for sparse VR motion tracking, but the uncertainty sampler's inference protocol is unspecified and the SOTA claim is not yet falsifiably pinned down. read the letter →

arxiv 2412.10235 v2 pith:QMVQ25DS submitted 2024-12-13 cs.CV

Songpengcheng Xia , Yu Zhang , Zhuo Su , Xiaozheng Zheng , Zheng Lv , Guidong Wang , Yongjie Zhang , Qi Wu
show 2 more authors
Lei Chu Ling Pei
This is my paper · ORCID
classification cs.CV
keywords humanmotionestimationsparsetrackingVR/ARuncertaintymodelingscene-awarepointcloudmulti-hypothesisSMPL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EnvPoser claims that full-body motion can be estimated far more reliably from the sparse head-and-hand tracking signals of VR devices when a pre-scanned 3D model of the surrounding environment is also available. The paper argues that sparse observations are inherently one-to-many: many full-body poses fit the same three tracking points, so the right strategy is to explicitly model that ambiguity as uncertainty, sample multiple plausible motions, and then let environmental constraints select the most realistic one. On the EgoBody and GIMO benchmarks, the method reports the best numbers across rotation error, position error, velocity error, and jitter, with gains such as an 18.5% reduction in position error over the strongest non-environment baseline. The authors position the contribution as showing that environment context, treated with both semantic and geometric constraints, is a practical substitute for extra body-worn sensors.

What carries the argument

The carrying mechanism is the two-stage uncertainty-and-environment loop. Stage I uses an autoregressive transformer over sparse observations and historical motion to predict a mean motion $\hat{\theta}$ and per-joint uncertainty $\delta$, then applies the reparameterization trick $\theta = \hat{\theta} + \delta \cdot \epsilon$ with $\epsilon \sim \mathcal{N}(0,1)$ to generate multi-hypothesis samples; the model is pre-trained on AMASS with a mean-squared-error loss and a heteroscedastic uncertainty loss. Stage II crops a 1-meter-radius circular region from the pre-scanned point cloud, encodes it with PointNet++, and refines the sampled motions through environment-motion cross-attention with a spatial salience term, contact probability prediction, and a set of geometric losses including the COAP-based collision loss and foot-ground constraints. The uncertainty sampling is what lets the refinement stage choose among plausible poses rather than committing to a single regressed mean.

What would settle it

Record the learned uncertainty values $\delta$ produced by the Stage I head on the EgoBody or GIMO test sets: if any component of $\delta$ is negative, the Gaussian $\mathcal{N}(\hat{\theta}, \delta)$ is not a valid distribution and the multi-hypothesis sampling strategy is formally broken. A second check would be to run the method in a scene with interactions farther than one meter from the body or with a moving object; if performance collapses, the static-scene and one-meter-crop assumptions are the limiting factors.

Watch

Extended reading notes

Core claim

The central claim is that incorporating pre-scanned environmental point cloud information constrains the uncertainty in motion reconstruction caused by sparse tracking signals. EnvPoser does this in two stages: first an autoregressive transformer with a heteroscedastic uncertainty head predicts a mean pose and per-joint uncertainty, and the model resamples multiple pose hypotheses from a Gaussian around that mean; then a second stage refines these hypotheses with a point-cloud encoder, a motion-environment cross-attention that injects spatial salience, contact estimation, and geometric losses including collision, foot contact, foot height, and ground penetration. The paper reports that this combined pipeline outperforms sparse-only methods and the scene-aware S2Fusion baseline on both EgoBody and GIMO, and that ablations show each component contributes, with the full system best.

Load-bearing premise

The method depends on the uncertainty head's output $\delta$ being a valid standard deviation for the Gaussian resampling $\theta = \hat{\theta} + \delta \cdot \epsilon$, but no positivity constraint is applied to $\delta$, so the multi-hypothesis sampling distribution can be undefined; the framework also assumes a static, pre-scanned environment and that a one-meter-radius crop contains the interaction-relevant context.

Editorial extensions

If this is right

  • Using a pre-scanned environment measurably reduces full-body pose error from sparse head-and-hand tracking, with the paper reporting MPJPE drops from 91.7 mm to 74.7 mm on EgoBody and from 70.7 mm to 57.6 mm on GIMO versus AvatarJLM.
  • Uncertainty estimation and resampling by themselves improve a sparse-tracking baseline, and their benefit grows when the sampled hypotheses are later refined by environmental constraints.
  • Semantic and geometric environment constraints are complementary: removing either one degrades accuracy, and removing both leaves substantial error.
  • The full pipeline also produces smoother motion than baselines, with lower jitter, which matters for VR experiences where jerky avatars break immersion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the uncertainty head lacks a positivity constraint, the model may be learning a scale-like quantity rather than a true variance; forcing $\delta$ positive with softplus or exponentiation would make the probabilistic interpretation sound and could be tested for accuracy changes.
  • The one-meter-radius crop and static-scene assumption imply the method is tuned for nearby, fixed furniture; extending the crop radius or fusing online scene updates is the natural next test, and the paper's own limitations section flags dynamic multi-user scenes as unhandled.
  • The two-stage 'sample many, then let the scene select' recipe could transfer to other ill-posed sensor-to-pose problems, such as estimating body pose from a single camera or from sparse IMUs, wherever a coarse 3D model of the allowed space exists.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. EnvPoser proposes a two-stage method for estimating full-body SMPL motion from sparse VR tracking signals (head and two hands) using a pre-scanned environment point cloud. Stage I is an autoregressive transformer with a joint uncertainty head that produces an initial motion estimate and per-joint uncertainty, resampled via the reparameterization trick to represent multiple hypotheses. Stage II refines the sampled motion by cross-attending to a cropped environment point cloud, predicting contact probabilities, and applying semantic and geometric losses including a COAP-based collision loss and foot-ground constraints. The method is trained on AMASS and fine-tuned on EgoBody and GIMO, and reports state-of-the-art MPJRE/MPJPE/MPJVE/Jitter values on both datasets compared to AvatarPoser, AGRoL, AvatarJLM, and S2Fusion.

Significance. If the empirical claims hold, EnvPoser is a useful contribution to sparse-observation human motion estimation: it explicitly introduces environment context into the estimation loop, separates uncertainty modeling from environment refinement, and provides ablations on both components. Strengths include the two-stage design, the use of pre-scanned point clouds with both semantic and geometric constraints, a public project page, and disclosure of hyperparameters and limitations. However, the central SOTA claim is currently weakened by an unspecified inference-time use of the uncertainty sampler, an inconsistent treatment of the S2Fusion baseline, and a missing statement on variance across runs. These issues are fixable and do not invalidate the approach, but they must be addressed before the results can be treated as conclusive.

major comments (3)
  1. [Sec. 3.2-3.3, Eq. (2), Tables 1-3] The inference protocol for the uncertainty-guided sampler is not specified. The paper defines theta = theta_hat + delta * epsilon during training, but never states whether at test time epsilon is set to 0 (deterministic mean), drawn once per frame, or averaged over multiple samples. If one random draw is used, every reported metric is a single realization of a random variable and comparisons without variance estimates are not reliable; if epsilon = 0 is used, the multi-hypothesis refinement is not exercised at inference and the claimed advantage of uncertainty modeling needs a different justification. Please state the exact inference protocol and, if randomness is involved, report means and standard deviations over multiple seeds or samples.
  2. [Sec. 4, Table 1] The comparison with S2Fusion is inconsistent. The text states 'We re-train all competing methods on the EgoBody and GIMO datasets until convergence,' but then states 'For the results on the GIMO dataset, we reused the values reported in previous studies [35].' Moreover, S2Fusion* on EgoBody is a partial-code reimplementation rather than the original method. This asymmetry, combined with the absence of error bars or multiple seeds, makes the reported SOTA margins (e.g., MPJRE 6.00 vs 6.42 for AvatarJLM on EgoBody) difficult to verify. Please clarify exactly which methods were retrained on which datasets and provide variance estimates or a clear statement that all methods were evaluated under identical conditions.
  3. [Sec. 3.1 vs Supplementary Sec. A] The description of Stage II training conflicts. Section 3.1 says 'we jointly train both modules (Stage II) on the motion-environment interaction datasets,' while Supplementary Sec. A says 'At the beginning of this phase, the parameters of the first module are fixed, and only the environment refinement module is trained.' These statements cannot both be true unless the first module is later unfrozen, which is not specified. Please clarify whether Stage I parameters are updated in Stage II and, if so, at what training step. This affects reproducibility and the interpretation of the ablation results.
minor comments (4)
  1. [Eq. (2)] The uncertainty loss is not a well-formed negative log-likelihood as written: N(theta_hat, delta) requires delta to be a positive scale, and the standard Kendall and Gal NLL contains a factor 1/2 in both terms. The reparameterized sampling is still Gaussian with variance delta^2, so the sign of delta is not load-bearing, but the notation should be corrected (e.g., define variance as delta^2 or apply softplus/exp to delta) to match the claimed probabilistic interpretation.
  2. [Sec. 4, Metrics] The metric definition says MPJPE is in centimeters, but the values in Table 1 (e.g., 74.7, 91.7, 57.8) are consistent with millimeters. Please correct the unit or the values.
  3. [Table 3] In Table 3, uncertainty estimation slightly worsens MPJVE on EgoBody (Baseline w/o UNC 333.0 vs Baseline 342.9; EnvPoser w/o UNC 172.7 vs EnvPoser 174.0). The text claims uncertainty estimation 'improves full-body motion estimation through additional supervision,' but this is not true for all metrics; please discuss or contextualize this.
  4. [Eq. (9)] The COAP-based loss uses a hard indicator I[f_Theta(V_Si|G)>0] inside a sum with a sigmoid. Please clarify how gradients are computed through the indicator (e.g., stop-gradient on the indicator or a smooth approximation), since a hard step function is not differentiable.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the EnvPoser pipeline is an empirical, loss-supervised system; no reported prediction reduces to a fitted input or to a self-citation chain.

full rationale

Walking the derivation chain from Sec. 3.1 through Eq. (12), I find no step in which a claimed prediction is defined in terms of the target or fitted to the test quantity. Stage I is trained on AMASS with L_M = ||theta_hat - theta||^2 and L_delta (Eqs. 1-2) against ground-truth SMPL motion; Stage II adds environment-derived losses L_coap, L_contact, L_fc, L_gfh, L_gp, L_posi, and L_hAL, all supervised by ground truth or external scene geometry. The uncertainty head outputs delta, which is learned jointly and used for resampling, but delta is never a fitted constant that is then relabeled as a predicted result; the headline MPJRE/MPJPE numbers come from test-set evaluation and ablations (Tables 1-3) that vary components rather than reinserting fitted values. Citations to the authors' prior work ([41], [42], [54], [55]) appear in Related Work and baseline comparisons, not as the load-bearing justification for EnvPoser's environment-refinement claim; the GIMO baseline numbers are reused from prior studies and S2Fusion* is a partial-code reimplementation, which are evaluation-fairness and reproducibility caveats, not circular reductions. The unspecified inference-time epsilon protocol for the uncertainty sampler is a falsifiability and reporting gap, not a self-referential derivation. No circular step meets the quoted-evidence bar.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard deep-learning training assumptions plus domain-specific choices. The most consequential is treating delta as a Gaussian standard deviation without positivity enforcement. The rest are typical hyperparameters and dataset assumptions.

free parameters (8)
  • lambda_M = 1.0
    Stage I motion loss weight in Eq. (3), chosen manually.
  • lambda_delta = 0.001
    Stage I uncertainty loss weight in Eq. (3).
  • lambda_1..lambda_7 = {2.0, 1.0, 0.75, 0.75, 0.75, 1.0, 0.1}
    Stage II loss weights in Eq. (12), set by hand and listed in Supplement A.
  • Environment crop radius = 1 meter
    Cropping radius around the human for point cloud sampling, stated in Sec. 3.3.
  • Sampled environment points N_S = 1000
    Number of points after circular sampling, chosen after comparing 500, 1000, and 2000 in Supplement C.
  • Sequence window length = 40
    Input window length for the transformer, stated in Supplement A.
  • Transformer attention heads = 8
    Architecture hyperparameter, stated in Supplement A.
  • Motion embedding latent dimension = 256
    Latent dimension for motion and contact embeddings, stated in Supplement A.
assumptions (5)
  • domain assumption SMPL with 22 joints and 6D rotation representation faithfully represents full-body pose.
    Used throughout; Sec. 3.1 defines theta as R^{T x 132}.
  • domain assumption AMASS provides a sufficient motion prior for sparse-to-full-body regression.
    Stage I pretraining on AMASS, Sec. 3.2.
  • domain assumption EgoBody and GIMO ground-truth motion and environment scans are accurately aligned.
    Evaluation relies on these benchmarks, Sec. 4.
  • domain assumption COAP can act as a differentiable collision proxy between SMPL body and point clouds.
    L_coap in Eq. (9) uses COAP, cited from [29].
  • ad hoc to paper The predicted delta is a valid standard deviation for Gaussian resampling.
    Sec. 3.2 assumes N(theta_hat, delta) and theta = theta_hat + delta * epsilon, but no positivity constraint is specified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modeling." pith.science (2026). https://pith.science/paper/QMVQ25DS

@misc{pith2026241210235,
  author       = {Pith},
  title        = {Pith review of: EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QMVQ25DS}},
  note         = {Machine review of arXiv:2412.10235}
}
read the original abstract

Estimating full-body motion using the tracking signals of head and hands from VR devices holds great potential for various applications. However, the sparsity and unique distribution of observations present a significant challenge, resulting in an ill-posed problem with multiple feasible solutions (i.e., hypotheses). This amplifies uncertainty and ambiguity in full-body motion estimation, especially for the lower-body joints. Therefore, we propose a new method, EnvPoser, that employs a two-stage framework to perform full-body motion estimation using sparse tracking signals and pre-scanned environment from VR devices. EnvPoser models the multi-hypothesis nature of human motion through an uncertainty-aware estimation module in the first stage. In the second stage, we refine these multi-hypothesis estimates by integrating semantic and geometric environmental constraints, ensuring that the final motion estimation aligns realistically with both the environmental context and physical interactions. Qualitative and quantitative experiments on two public datasets demonstrate that our method achieves state-of-the-art performance, highlighting significant improvements in human motion estimation within motion-environment interaction scenarios.

Figures

Figures reproduced from arXiv: 2412.10235 by the authors.

Figure 1
Figure 1. EnvPoser can estimate the full-body motion using three [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of EnvPoser: A Two-Stage Motion Estimation Model. Stage I involves training the uncertainty-aware initial estimation [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of motion estimation on three test sequences from EgoBody Dataset [ [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparison of Interaction Details. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 1
Figure 1. Figure 1: Environmental point cloud with different sampling [PITH_FULL_IMAGE:figures/full_fig_p012_1.png]
Figure 2
Figure 2. Figure 2: Qualitative results of lower-body MPJPE box plot for [PITH_FULL_IMAGE:figures/full_fig_p013_2.png]
Figure 3
Figure 3. Figure 3: Visualization of various sitting motions from EgoBody and GIMO Datasets. [PITH_FULL_IMAGE:figures/full_fig_p014_3.png]
Figure 4
Figure 4. Figure 4: Visualization of full-body estimation on three test sequences from GIMO Datasets. [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results on real data from VR device. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VolumetricSMPL: A Neural Volumetric Body Model for Efficient Interactions, Contacts, and Collisions

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A new SMPL-compatible signed distance field body model, VolumetricSMPL, uses neural blend weights to cut inference time and memory about 10x and 6x versus COAP while matching or improving accuracy.

Reference graph

Works this paper leans on

57 extracted references · 52 canonical work pages · cited by 1 Pith paper

  1. [35]

    A unified diffusion framework for scene-aware human motion estimation from sparse signals

    Jiangnan Tang, Jingya Wang, Kaiyang Ji, Lan Xu, Jingyi Yu, and Ye Shi. A unified diffusion framework for scene-aware human motion estimation from sparse signals. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21251–21262, 2024. 2, 3, 6, 7

    work page 2024

  2. [1]

    Flag: Flow- based 3d avatar generation from sparse observations

    Sadegh Aliakbarian, Pashmina Cameron, Federica Bogo, Andrew Fitzgibbon, and Thomas J Cashman. Flag: Flow- based 3d avatar generation from sparse observations. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13253–13262, 2022. 2

    work page 2022

  3. [2]

    Hmd-nemo: Online 3d avatar motion generation from sparse observations

    Sadegh Aliakbarian, Fatemeh Saleh, David Collier, Pash- mina Cameron, and Darren Cosker. Hmd-nemo: Online 3d avatar motion generation from sparse observations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9622–9631, 2023. 1, 2

    work page 2023

  4. [3]

    Mhentropy: Entropy meets multiple hypotheses for pose and shape re- covery

    Rongyu Chen, Linlin Yang, and Angela Yao. Mhentropy: Entropy meets multiple hypotheses for pose and shape re- covery. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 14840–14849, 2023. 2

    work page 2023

  5. [4]

    Hmd-poser: On-device real-time human motion tracking from scalable sparse observations

    Peng Dai, Yang Zhang, Tao Liu, Zhen Fan, Tianyuan Du, Zhuo Su, Xiaozheng Zheng, and Zeming Li. Hmd-poser: On-device real-time human motion tracking from scalable sparse observations. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 874–884, 2024. 2

    work page 2024

  6. [5]

    Full- body motion from a single head-mounted device: Generat- ing smpl poses from partial observations

    Andrea Dittadi, Sebastian Dziadzio, Darren Cosker, Ben Lundell, Thomas J Cashman, and Jamie Shotton. Full- body motion from a single head-mounted device: Generat- ing smpl poses from partial observations. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 11687–11697, 2021. 2

    work page 2021

  7. [6]

    Avatars grow legs: Generating smooth human motion from sparse tracking in- puts with diffusion model

    Yuming Du, Robin Kips, Albert Pumarola, Sebastian Starke, Ali Thabet, and Artsiom Sanakoyeu. Avatars grow legs: Generating smooth human motion from sparse tracking in- puts with diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 481–490, 2023. 1, 2, 6, 7, 3

    work page 2023

  8. [7]

    Emhi: A multimodal egocentric human motion dataset with hmd and body-worn imus

    Zhen Fan, Peng Dai, Zhuo Su, Xu Gao, Zheng Lv, Jiarui Zhang, Tianyuan Du, Guidong Wang, and Yang Zhang. Emhi: A multimodal egocentric human motion dataset with hmd and body-worn imus. arXiv preprint arXiv:2408.17168,

    arXiv

Show all 57 references
  1. [8]

    Stratified avatar generation from sparse observations

    Han Feng, Wenchao Ma, Quankai Gao, Xianwei Zheng, Nan Xue, and Huijuan Xu. Stratified avatar generation from sparse observations. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 153–163, 2024. 1, 2

    2024

  2. [9]

    Diffpose: Toward more reliable 3d pose estimation

    Jia Gong, Lin Geng Foo, Zhipeng Fan, Qiuhong Ke, Hos- sein Rahmani, and Jun Liu. Diffpose: Toward more reliable 3d pose estimation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 13041–13051, 2023. 2

    2023

  3. [10]

    Resolving 3d human pose ambiguities with 3d scene constraints

    Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, and Michael J Black. Resolving 3d human pose ambiguities with 3d scene constraints. In Proceedings of the IEEE/CVF international conference on computer vision , pages 2282– 2292, 2019. 3, 6

    2019

  4. [11]

    Capturing and inferring dense full-body human-scene contact

    Chun-Hao P Huang, Hongwei Yi, Markus H ¨oschle, Matvey Safroshkin, Tsvetelina Alexiadis, Senya Polikovsky, Daniel Scharstein, and Michael J Black. Capturing and inferring dense full-body human-scene contact. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2022

  5. [12]

    Diffusion- based generation, optimization, and planning in 3d scenes

    Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, and Song-Chun Zhu. Diffusion- based generation, optimization, and planning in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16750–16761, 2023. 3

    2023

  6. [13]

    Deep inertial poser: learning to reconstruct human pose from sparse iner- tial measurements in real time

    Yinghao Huang, Manuel Kaufmann, Emre Aksan, Michael J Black, Otmar Hilliges, and Gerard Pons-Moll. Deep inertial poser: learning to reconstruct human pose from sparse iner- tial measurements in real time. ACM Trans. Graph. (TOG), 37(6):1–15, 2018. 1, 2

    2018

  7. [14]

    Avatarposer: Ar- ticulated full-body pose tracking from sparse motion sens- ing

    Jiaxi Jiang, Paul Streli, Huajian Qiu, Andreas Fender, Larissa Laich, Patrick Snape, and Christian Holz. Avatarposer: Ar- ticulated full-body pose tracking from sparse motion sens- ing. In European conference on computer vision, pages 443–

  8. [15]

    Egoposer: Robust real-time egocentric pose estimation from sparse and intermittent observations everywhere

    Jiaxi Jiang, Paul Streli, Manuel Meier, and Christian Holz. Egoposer: Robust real-time egocentric pose estimation from sparse and intermittent observations everywhere. In Eu- ropean Conference on Computer Vision , pages 277–294. Springer, 2025. 2

    2025

  9. [16]

    Scaling up dynamic human-scene interaction mod- eling

    Nan Jiang, Zhiyuan Zhang, Hongjie Li, Xiaoxuan Ma, Zan Wang, Yixin Chen, Tengyu Liu, Yixin Zhu, and Siyuan Huang. Scaling up dynamic human-scene interaction mod- eling. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 1737–1747,

  10. [17]

    Transformer inertial poser: Real-time human motion reconstruction from sparse imus with simultaneous terrain generation

    Yifeng Jiang, Yuting Ye, Deepak Gopinath, Jungdam Won, Alexander W Winkler, and C Karen Liu. Transformer inertial poser: Real-time human motion reconstruction from sparse imus with simultaneous terrain generation. In SIGGRAPH Asia 2022 Conference Papers(SA’ 22), pages 1–9, 202...

    2022

  11. [18]

    Em-pose: 3d human pose estimation from sparse electromagnetic trackers

    Manuel Kaufmann, Yi Zhao, Chengcheng Tang, Lingling Tao, Christopher Twigg, Jie Song, Robert Wang, and Otmar Hilliges. Em-pose: 3d human pose estimation from sparse electromagnetic trackers. In Proceedings of the IEEE/CVF international conference on computer vision(ICCV) , pag...

    2021

  12. [19]

    What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017

    Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017. 3, 4

    2017

  13. [20]

    Mocap everyone everywhere: Lightweight motion capture with smartwatches and a head- mounted camera

    Jiye Lee and Hanbyul Joo. Mocap everyone everywhere: Lightweight motion capture with smartwatches and a head- mounted camera. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1091–1100, 2024. 5

    2024

  14. [21]

    Questenvsim: Environment-aware simulated motion tracking from sparse sensors

    Sunmin Lee, Sebastian Starke, Yuting Ye, Jungdam Won, and Alexander Winkler. Questenvsim: Environment-aware simulated motion tracking from sparse sensors. InACM SIG- GRAPH 2023 Conference Proceedings, pages 1–9, 2023. 3

    2023

  15. [22]

    Generating multiple hypotheses for 3d human pose estimation with mixture density network

    Chen Li and Gim Hee Lee. Generating multiple hypotheses for 3d human pose estimation with mixture density network. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 9887–9895, 2019. 2

    2019

  16. [23]

    Pose-oriented transformer with uncertainty- guided refinement for 2d-to-3d human pose estimation

    Han Li, Bowen Shi, Wenrui Dai, Hongwei Zheng, Botao Wang, Yu Sun, Min Guo, Chenglin Li, Junni Zou, and Hongkai Xiong. Pose-oriented transformer with uncertainty- guided refinement for 2d-to-3d human pose estimation. In Proceedings of the AAAI Conference on Artificial Intelli- ...

    2023

  17. [24]

    Mhformer: Multi-hypothesis transformer for 3d human pose estimation

    Wenhao Li, Hong Liu, Hao Tang, Pichao Wang, and Luc Van Gool. Mhformer: Multi-hypothesis transformer for 3d human pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13147–13156, 2022. 2

    2022

  18. [25]

    Swin transformer v2: Scaling up capacity and resolution

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 12009–12019, 2022. 5

    2022

  19. [26]

    Smpl: A skinned multi- person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi- person linear model. ACM Transactions on Graphics, 34(6),

  20. [27]

    Multimodal sense-informed forecasting of 3d human motions

    Zhenyu Lou, Qiongjie Cui, Haofan Wang, Xu Tang, and Hong Zhou. Multimodal sense-informed forecasting of 3d human motions. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 2144–2154, 2024. 3, 5

    2024

  21. [28]

    Amass: Archive of motion capture as surface shapes

    Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Ger- ard Pons-Moll, and Michael J Black. Amass: Archive of motion capture as surface shapes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5442–5451, 2019. 1, 4, 2

    2019

  22. [29]

    Coap: Compositional articulated occupancy of people

    Marko Mihajlovic, Shunsuke Saito, Aayush Bansal, Michael Zollhoefer, and Siyu Tang. Coap: Compositional articulated occupancy of people. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 13201–13210, 2022. 6

    2022

  23. [30]

    Sparseposer: Real-time full- body motion reconstruction from sparse data

    Jose Luis Ponton, Haoran Yun, Andreas Aristidou, Carlos Andujar, and Nuria Pelechano. Sparseposer: Real-time full- body motion reconstruction from sparse data. ACM Trans- actions on Graphics, 43(1):1–14, 2023. 2

    2023

  24. [31]

    Dragposer: Motion re- construction from variable sparse tracking signals via latent space optimization

    Jose Luis Ponton, Eduard Pujol, Andreas Aristidou, Car- los Andujar, and Nuria Pelechano. Dragposer: Motion re- construction from variable sparse tracking signals via latent space optimization. arXiv preprint arXiv:2406.14567, 2024. 2

    2024 arXiv

  25. [32]

    Pointnet++: Deep hierarchical feature learning on point sets in a metric space

    Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017. 5, 1

    2017

  26. [33]

    Diffusion-based 3d human pose estimation with multi- hypothesis aggregation

    Wenkang Shan, Zhenhua Liu, Xinfeng Zhang, Zhao Wang, Kai Han, Shanshe Wang, Siwei Ma, and Wen Gao. Diffusion-based 3d human pose estimation with multi- hypothesis aggregation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14761– 14771, 2023. 2

    2023

  27. [34]

    Learning human mesh recovery in 3d scenes

    Zehong Shen, Zhi Cen, Sida Peng, Qing Shuai, Hujun Bao, and Xiaowei Zhou. Learning human mesh recovery in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17038– 17047, 2023. 3, 5

    2023

  28. [36]

    Diffusionposer: Real-time human motion reconstruction from arbitrary sparse sensors using autoregressive diffusion

    Tom Van Wouwe, Seunghwan Lee, Antoine Falisse, Scott Delp, and C Karen Liu. Diffusionposer: Real-time human motion reconstruction from arbitrary sparse sensors using autoregressive diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ...

    2024

  29. [37]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 4, 1

    2017

  30. [38]

    Probabilistic monocular 3d human pose es- timation with normalizing flows

    Tom Wehrbein, Marco Rudolph, Bodo Rosenhahn, and Bas- tian Wandt. Probabilistic monocular 3d human pose es- timation with normalizing flows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 11199–11208, 2021. 2

    2021

  31. [39]

    Quest- sim: Human motion tracking from sparse sensors with simu- lated avatars

    Alexander Winkler, Jungdam Won, and Yuting Ye. Quest- sim: Human motion tracking from sparse sensors with simu- lated avatars. In SIGGRAPH Asia 2022 Conference Papers, pages 1–8, 2022. 1

    2022

  32. [40]

    Accurate and steady inertial pose estimation through sequence structure learning and modulation

    Yinghao Wu, chaoran wang, Lu Yin, Shihui Guo, and Yipeng Qin. Accurate and steady inertial pose estimation through sequence structure learning and modulation. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. 2

    2024

  33. [41]

    Learning disentangled rep- resentation for mixed-reality human activity recognition with a single imu sensor

    Songpengcheng Xia, Lei Chu, Ling Pei, Zixuan Zhang, Wenxian Yu, and Robert C Qiu. Learning disentangled rep- resentation for mixed-reality human activity recognition with a single imu sensor. IEEE Transactions on Instrumentation and Measurement, 70:1–14, 2021. 2

    2021

  34. [42]

    Timestamp-supervised wearable- based activity segmentation and recognition with contrastive learning and order-preserving optimal transport

    Songpengcheng Xia, Lei Chu, Ling Pei, Jiarui Yang, Wenx- ian Yu, and Robert C Qiu. Timestamp-supervised wearable- based activity segmentation and recognition with contrastive learning and order-preserving optimal transport. IEEE Transactions on Mobile Computing, 2024. 1

    2024

  35. [43]

    Scorehypo: Probabilistic human mesh estimation with hypothesis scoring

    Yuan Xu, Xiaoxuan Ma, Jiajun Su, Wentao Zhu, Yu Qiao, and Yizhou Wang. Scorehypo: Probabilistic human mesh estimation with hypothesis scoring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 979–989, 2024. 2

    2024

  36. [44]

    Narrator: Towards natural control of human-scene interaction generation via relation- ship reasoning

    Haibiao Xuan, Xiongzheng Li, Jinsong Zhang, Hongwen Zhang, Yebin Liu, and Kun Li. Narrator: Towards natural control of human-scene interaction generation via relation- ship reasoning. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 22268–22278,

  37. [45]

    Smgdiff: Soccer motion generation using diffusion probabilistic mod- els

    Hongdi Yang, Chengyang Li, Zhenxuan Wu, Gaozheng Li, Jingya Wang, Jingyi Yu, Zhuo Su, and Lan Xu. Smgdiff: Soccer motion generation using diffusion probabilistic mod- els. arXiv preprint arXiv:2411.16216, 2024. 3

    2024 arXiv

  38. [46]

    Spatial-related sensors matters: 3d human motion reconstruction assisted with textual semantics

    Xueyuan Yang, Chao Yao, and Xiaojuan Ban. Spatial-related sensors matters: 3d human motion reconstruction assisted with textual semantics. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pages 10225–10233, 2024. 3

    2024

  39. [47]

    Mime: Human- aware 3d scene generation

    Hongwei Yi, Chun-Hao P Huang, Shashank Tripathi, Lea Hering, Justus Thies, and Michael J Black. Mime: Human- aware 3d scene generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12965–12976, 2023. 3, 5

    2023

  40. [48]

    Transpose: Real-time 3d human translation and pose estimation with six inertial sensors

    Xinyu Yi, Yuxiao Zhou, and Feng Xu. Transpose: Real-time 3d human translation and pose estimation with six inertial sensors. ACM Transactions On Graphics (TOG) , 40(4):1– 13, 2021. 2

    2021

  41. [49]

    Phys- ical inertial poser (pip): Physics-aware real-time human mo- tion tracking from sparse inertial sensors

    Xinyu Yi, Yuxiao Zhou, Marc Habermann, Soshi Shimada, Vladislav Golyanik, Christian Theobalt, and Feng Xu. Phys- ical inertial poser (pip): Physics-aware real-time human mo- tion tracking from sparse inertial sensors. In Proceedings of the IEEE/CVF Conference on Computer Visio...

    2022

  42. [50]

    Learning motion priors for 4d human body capture in 3d scenes

    Siwei Zhang, Yan Zhang, Federica Bogo, Marc Pollefeys, and Siyu Tang. Learning motion priors for 4d human body capture in 3d scenes. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 11343– 11353, 2021. 6

    2021

  43. [51]

    Ego- body: Human body shape and motion of interacting peo- ple from head-mounted devices

    Siwei Zhang, Qianli Ma, Yan Zhang, Zhiyin Qian, Taein Kwon, Marc Pollefeys, Federica Bogo, and Siyu Tang. Ego- body: Human body shape and motion of interacting peo- ple from head-mounted devices. In European conference on computer vision, pages 180–200. Springer, 2022. 6, 7, 2

    2022

  44. [52]

    Probabilistic human mesh recovery in 3d scenes from egocentric views

    Siwei Zhang, Qianli Ma, Yan Zhang, Sadegh Aliakbarian, Darren Cosker, and Siyu Tang. Probabilistic human mesh recovery in 3d scenes from egocentric views. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 7989–8000, 2023. 3, 6

    2023

  45. [53]

    Body knowledge and uncertainty modeling for monoc- ular 3d human body reconstruction

    Yufei Zhang, Hanjing Wang, Jeffrey O Kephart, and Qiang Ji. Body knowledge and uncertainty modeling for monoc- ular 3d human body reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9020–9032, 2023. 3

    2023

  46. [54]

    Dynamic inertial poser (dynaip): Part- based motion dynamics learning for enhanced human pose estimation with sparse inertial sensors

    Yu Zhang, Songpengcheng Xia, Lei Chu, Jiarui Yang, Qi Wu, and Ling Pei. Dynamic inertial poser (dynaip): Part- based motion dynamics learning for enhanced human pose estimation with sparse inertial sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2024

  47. [55]

    Realistic full-body tracking from sparse ob- servations via joint-level modeling

    Xiaozheng Zheng, Zhuo Su, Chao Wen, Zhou Xue, and Xiaojie Jin. Realistic full-body tracking from sparse ob- servations via joint-level modeling. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 14678–14688, 2023. 1, 2, 6, 7, 3

    2023

  48. [56]

    Gimo: Gaze-informed human motion prediction in context

    Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, C Karen Liu, and Leonidas J Guibas. Gimo: Gaze-informed human motion prediction in context. In Eu- ropean Conference on Computer Vision , pages 676–694. Springer, 2022. 6, 2 EnvPoser: Environment-aware Realist...

    2022

  49. [460]

    1, 2, 6, 7, 3

    Springer, 2022. 1, 2, 6, 7, 3

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.