Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling

Edmond S. L. Ho; He Wang; Hubert P. H. Shum; Zhanxing Zhu

REVIEW 1 major objections 1 minor 1 cited by

Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling

T0 review · 1 major / 1 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read STRNN learns a natural motion manifold from unlabeled skeletal data and predicts far beyond 20,000 frames in open loop.

desk verdict A solid architectural contribution to motion manifold learning, undermined by a vague H3.6M protocol and a self-referential quality metric, but fixable. read the letter →

arxiv 1908.07214 v1 pith:YO5SDE3O submitted 2019-08-20 cs.GR cs.LG

He Wang , Edmond S. L. Ho , Hubert P. H. Shum , Zhanxing Zhu This is my paper · ORCID

classification cs.GRcs.LG

keywords humanmotionsynthesismanifoldskeletalanimationrecurrentneuralnetworklong-horizonpredictiondenoisingspatio-temporalmodelingunsupervisedlearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes STRNN, a deep network that learns a manifold of natural human motions directly from unlabeled 3D skeletal data, and claims that, given a 20-frame prefix, the network can keep generating coherent motion for far more than 20,000 frames in an open-loop setting with no error moderation. The authors argue that previous learned motion models fail in two ways: they underestimate motion multimodality and collapse to a featureless average posture, or they overestimate variance and produce jitter. STRNN counters the first failure with a hierarchical spatial encoder that groups joints into body parts, and the second with a batch prediction scheme that forecasts many frames at once and is trained with a long-horizon smoothness loss. The same learned manifold is also applied to denoising corrupted motion capture data and to synthesizing motion that follows control signals, so one unsupervised model is claimed to cover several tasks.

What carries the argument

The carrying mechanism is the TBTN (Two-way Bidirectional Temporal Network) combined with a hierarchical part-based spatial encoder and a long-horizon acceleration penalty. The temporal network is a hybrid of encoding/decoding and sequence-to-sequence learning: an LSTM encoder reads $m+1$ past frames, its state seeds both a decoder that reconstructs backward in time and a predictor that forecasts $n$ frames forward, so the training objective covers a long temporal window. The spatial encoder treats the skeleton as seven body-part groups (root, torso, legs, arms, foot contacts) and merges them level by level through fully connected layers, making group-wise posture variance and cross-group correlation explicit. The long-horizon cost is $C_s = \frac{1}{m+n}\sum \|\hat{X}^{t+1}_{body} - 2\hat{X}^t_{body} + \hat{X}^{t-1}_{body}\|^2 + \sum \|\hat{X}^t_{root} - \hat{X}^{t-1}_{root}\|^2$, which penalizes acceleration and root drift rather than position. A residual subnetwork is pretrained separately and then fine-tuned to cancel the periodic jumps that appear in iterative prediction.

What would settle it

Take a generated 20,000-frame sequence and run a sliding-window nearest-neighbor search of its 40-frame windows against the exact training corpus; if a large fraction of windows match training clips at near-zero distance, the long prediction is replay rather than manifold-based generation. Alternatively, compute D1nn between held-out real test motions and the training set and compare it with D1nn for generated motions; if generated motions are not closer to the manifold than real test motions, the metric is not evidence of naturalness.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that motion multimodality and long-range temporal dependencies can be jointly handled by replacing single-frame prediction with batch prediction and by preserving spatial variance with a part-based skeleton encoding. The network splits a skeleton into seven part groups, merges them hierarchically through fully connected layers, and decodes in reverse; temporally, a two-way bidirectional LSTM network reads a long context, decodes backward to force short-term learning first, and predicts forward in batches. A long-horizon smoothness cost penalizes acceleration rather than position, which the authors say keeps dynamics while avoiding mean-posture collapse, and a residual network trained separately cancels periodic high-frequency noise. Quantitative comparisons on H3.6M report lower per-frame prediction errors than LSTM 3LR, CRBMs, 6GRAM, GPDM, and ERD at every tested horizon from 80 to 560 ms.

Load-bearing premise

The load-bearing premise is that the motion-capture data used for training are dense and representative enough that nearest-neighbor distance to those data measures naturalness; if the data are sparse or the model memorizes clips, low D1nn no longer means the output is natural.

Editorial extensions

If this is right

Given only a 20-frame prefix, the model can keep generating coherent motion for over 20,000 frames in open loop, which would make long synthetic sequences practical without runtime error correction.
The same learned manifold can denoise corrupted motion capture data and synthesize motions that follow control signals, so one unsupervised model covers multiple tasks.
Training with a long-horizon acceleration cost instead of frame-wise error alone avoids both mean-posture collapse and jitter, a failure mode common in recurrent motion models.
On the H3.6M benchmark, STRNN reports lower per-frame prediction error than LSTM 3LR, CRBMs, 6GRAM, GPDM, and ERD at every horizon from 80 to 560 ms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A stricter evaluation of the long-horizon claim would check whether generated windows are near-exact copies of training clips; if they are, the reported D1nn closeness reflects memorization rather than manifold learning.
The paper's own limitation note says action-level control is absent because motions are unlabeled; conditioning the same architecture on action labels or environmental geometry is the most direct next step its results support.
Because the long-horizon cost penalizes acceleration rather than position, it could be transferred to other time-series generation tasks where over-smoothing is a problem, though the paper does not test this.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things before spending time on this one. First, the architecture is a genuine combination—hierarchical part-based spatial encoding, a sequence-to-sequence LSTM with backward decoding, a long-horizon smoothness loss, and a residual filter—and the ablations actually support the design choices. Second, the headline numbers in Table 1 are not yet reproducible: the H3.6M comparison is described in two sentences with no subject/action split and no error bars.

What is new is the specific system, STRNN, and the claim that it can generate 20,000 frames open-loop without run-time error moderation. That stability claim is the kind of thing the community would care about, and the paper backs it with qualitative video plus the D1nn manifold-distance metric. The ablations in Section 5 are the strongest part: batch prediction, the long-horizon loss, the spatial hierarchy, and the residual component each get tested, and the differences mostly line up with the narrative.

Now the soft spots. The Table 1 comparison follows the protocol in [6] and uses eight random prefixes, but never says which subjects or actions those come from, how many runs, or what the variance is. The stress-test worry about train/test leakage doesn't land here, because H3.6M is not in the training corpus listed in Section 3; the model is evaluated zero-shot. But the lack of protocol detail still matters, and the error curve from 0.36 to 0.51 over 480 ms is oddly flat, which makes you wonder whether the model is returning a near-ground-truth local clip rather than generalizing. Table 2, the comparison against Martinez et al., is better documented: they use code from the authors and the standard subject 5 test split. That comparison is credible.

The D1nn metric is a real weakness. It measures distance to the nearest training segment, so a model that memorizes training clips will score well by construction. The paper explicitly assumes the training data are dense enough to represent the ground-truth manifold, and the authors acknowledge the limitation, but that means the 20,000-frame stability claim rests on a metric that cannot distinguish natural motion from memorized snippets. Videos help, but they are not a substitute for a sharp metric. No code or data is released, so the reader cannot check the long-horizon behavior either.

Overall, the central argument holds up: the architecture and training objective do improve on earlier RNN-based motion models, and the limitations section is honest about transition hesitations and foot sliding. The paper deserves a serious referee, but it needs a revised evaluation section with a precise H3.6M protocol, error bars, and a better naturalness metric before it can be trusted. I would engage with it, and I'd send it to review, but I'd ask for those fixes.

Referee Report

1 major / 1 minor

Summary. The paper proposes STRNN, a deep network for learning a manifold of 3D skeletal human motion from unlabeled motion-capture data. The architecture combines a hierarchical spatial encoder/decoder that partitions the skeleton into body parts, a two-way bidirectional temporal LSTM encoder-decoder-predictor that performs batch prediction, and a residual network trained to remove high-frequency artifacts. A long-horizon (LH) loss penalizing acceleration and root velocity is used to preserve dynamics. The authors evaluate the model on long-horizon open-loop motion extrapolation (up to 20,000 frames), motion denoising, and control-signal-conditioned synthesis, reporting lower prediction errors than several baselines on H3.6M and on the [25] protocol, and extensive ablations over the network components and loss terms.

Significance. If the reported results hold, the paper makes a useful contribution: the hierarchical spatial model directly addresses a known weakness of RNN-based motion synthesis (convergence to mean postures), and the batch-prediction plus long-horizon loss is a sensible way to mitigate error accumulation. The ablation study in Section 5.1.2 (Figure 6) provides credible evidence that the spatial hierarchy, batch prediction, LH loss, and residual network each contribute to the final quality, which is a strength. The claims of stable open-loop generation are qualitatively demonstrated in the accompanying video, and the reported inference speed suggests practical value. However, the central quantitative claim (Table 1) currently lacks a reproducible protocol, and the D1nn metric (Eq. 12) has a known vulnerability to memorization; these gaps limit how strongly the results can be assessed at this stage.

major comments (1)

[Section 6.2, Table 1] Section 5.2 states that STRNN "can predict far longer than 20000 frames in practice," but no quantitative error or stability measure is reported beyond the first 2000 frames shown in Figure 1. Since the claim of long-horizon stability is central to the paper's contribution, this section should report error accumulation (e.g., per-frame error against ground truth where available, or D1nn computed over disjoint windows) as a function of horizon, and compare with the closest prior long-horizon method [26]. Without such a measure, the stability claim is only anecdotal.

minor comments (1)

[General] Section 5.1.3 reports experiments with 10- and 20-frame segments but does not provide the corresponding D1nn or error numbers, only a statement that visual qualities are similar; adding quantitative results would strengthen the claim that the balanced encode/predict setting generalizes.

Circularity Check

1 steps flagged · score 4.0 of 10

The main prediction-error comparisons are independent, but the motion-naturalness and long-horizon claims rest in part on a nearest-neighbor metric that is defined as closeness to the same data used for training.

self definitional [Section 5.1.2, Eq. 12; Section 4.2.1, Eqs. 5-6]
"Since designing a metric that directly reflects visual quality is difficult, we use a metric that evaluates how close the generated motions are from the motion manifold, which has been previously used to measure motion style similarity [20]. Also, we assume that the existing data are representative samples and are dense enough to represent the ’ground-truth’ motion manifold. ... D1nn = min(dist(Dg,Dm)) (12) ... Minimizing Cr results in a tight approximation of the motion manifold."

With the paper's own assumption, the 'ground-truth motion manifold' is operationalized as the existing motion data, and D1nn measures the minimum per-frame L2 distance from a generated motion to that data. The training objective Cr in Eqs. 5-6 is also a per-frame MSE against ground-truth motion segments. Thus a network that stores or closely reproduces data segments will obtain a low D1nn by construction, so D1nn is not an independent test of naturalness or open-loop stability. The claim in Section 5.2 that STRNN 'can predict far longer than 20000 frames in practice' is supported largely by this metric and qualitative video, making that part of the motion-quality claim partly definitional. The external prediction-error comparisons in Tables 1 and 2 are not affected by this circularity.

full rationale

This is an empirical deep-learning system paper rather than a formal derivation, so most of the claimed results are measured rather than deduced. The central numerical claims against external baselines are independent of the training manifold: Table 1 compares STRNN with LSTM 3LR, CRBMs, 6GRAM, GPDM, and ERD on H3.6M following the protocol of [6], and Table 2 compares against [25] using their downloaded code and a subject-based training/testing split. Those comparisons give genuine, external support to the superiority claim. The main circularity concern is evaluative. The D1nn metric in Eq. 12 is explicitly defined as the distance from a generated motion to the nearest existing data segment, and the paper states that it assumes the existing data are dense enough to represent the ground-truth manifold. Since the network is trained with per-frame MSE against the same type of ground-truth data, D1nn rewards behaviors that reproduce or memorize data patterns; it cannot by itself establish that the generated motions are natural or that the open-loop 20000-frame extrapolation truly generalizes. The citation [20] used to justify D1nn is also a self-citation by the paper's own co-authors, though it is not the sole support for the headline prediction results. The insufficiently documented H3.6M protocol in Section 6.2 is a reproducibility and correctness risk, not a circularity, so it does not increase the circularity score. Overall, the paper has partial circularity in its motion-quality evaluation but retains independent external comparisons for its central prediction-accuracy claims, warranting a score of 4 rather than 0 or 6.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

This is an empirical architecture paper, so its load-bearing inputs are hyperparameters and modeling assumptions rather than mathematical axioms. No invented physical or conceptual entities are introduced. The main free parameters are the batch prediction horizon, the loss weights, the corruption noise schedule, and the spatial network capacities; the main assumptions concern manifold representativeness, body-part group independence, and the benefit of decreasing corruption noise.

free parameters (4)

encode_len and predict_len = 20 frames each
Chosen empirically in Section 5.1.1 as a balanced encoding/prediction pair; this defines the batch prediction horizon that the long-horizon loss acts on.
Loss weights w_r and w_s
Equation 4 balances reconstruction Cr and long-horizon smoothness Cs, but the paper never reports the actual weight values, despite crediting the LH term as the key to preserving motion variance.
Corruption noise schedule = initial std 0.1, decreasing by 0.001 per iteration
Used in the hybrid training strategy in Section 4.2.4; the schedule is hand-designed and is claimed to reduce error accumulation.
Spatial network widths and residual width = L1-L4: 64, 128, 256, 512; residual: 512
The spatial encoder capacity is hand-selected; the paper reports that lower-dimensional variants failed to learn the manifold, so these capacities are load-bearing for the central result.

assumptions (4)

domain assumption Natural human motions constitute a low-dimensional motion manifold.
Section 1 and Section 4.1 parameterize the motion manifold as a conditional time series. If this framing is wrong, the entire generative approach is questionable, though the assumption is inherited from earlier work [1].
domain assumption Existing motion data are representative and dense enough to stand in for the ground-truth motion manifold in the D1nn metric.
Stated explicitly in Section 5.1.2. The D1nn validation depends on this assumption to interpret nearest-neighbor distance as a quality score.
domain assumption Body-part semantics define relatively independent groups of Dofs, so hierarchical part-wise encoding captures spatial correlations.
Section 4.1.1 bases the spatial encoder design on this assumption. If the group independence is false, the spatial hierarchy provides no benefit.
ad hoc to paper Decreasing corruption noise during training maps a small neighborhood of each ground-truth frame to the ground truth and reduces error accumulation.
Section 4.2.4 asserts this benefit without an ablation or theoretical justification, and it is central to the hybrid training strategy.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling." pith.science (2026). https://pith.science/paper/YO5SDE3O

@misc{pith2026190807214,
  author       = {Pith},
  title        = {Pith review of: Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YO5SDE3O}},
  note         = {Machine review of arXiv:1908.07214}
}

read the original abstract

Data-driven modeling of human motions is ubiquitous in computer graphics and computer vision applications, such as synthesizing realistic motions or recognizing actions. Recent research has shown that such problems can be approached by learning a natural motion manifold using deep learning to address the shortcomings of traditional data-driven approaches. However, previous methods can be sub-optimal for two reasons. First, the skeletal information has not been fully utilized for feature extraction. Unlike images, it is difficult to define spatial proximity in skeletal motions in the way that deep networks can be applied. Second, motion is time-series data with strong multi-modal temporal correlations. A frame could be followed by several candidate frames leading to different motions; long-range dependencies exist where a number of frames in the beginning correlate to a number of frames later. Ineffective modeling would either under-estimate the multi-modality and variance, resulting in featureless mean motion or over-estimate them resulting in jittery motions. In this paper, we propose a new deep network to tackle these challenges by creating a natural motion manifold that is versatile for many applications. The network has a new spatial component for feature extraction. It is also equipped with a new batch prediction model that predicts a large number of frames at once, such that long-term temporally-based objective functions can be employed to correctly learn the motion multi-modality and variances. With our system, long-duration motions can be predicted/synthesized using an open-loop setup where the motion retains the dynamics accurately. It can also be used for denoising corrupted motions and synthesizing new motions with given control signals. We demonstrate that our system can create superior results comparing to existing work in multiple applications.

Figures

Figures reproduced from arXiv: 1908.07214 by the authors.

**Figure 2.** The network architecture of the Spatio-temporal Recurrent Neural Networks. Detailed network structures of the Temporal En [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Two-way Bidirectional Temporal Network (TBTN). Rectangles are [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: From left to right: training error and validation error. Spa [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 7.** Figure 7: Denoising comparisons. Red: original, White: corrupted, Green: [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

**Figure 8.** Figure 8: X Axis: Experiments. Y Axis: Reconstruction error of STRNN [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Motion Generation Review: Exploring Deep Learning for Lifelike Animation with Manifold
cs.CV 2024-12 conditional novelty 4.0 of 10

A survey of manifold learning techniques for human motion generation, covering extraction, synthesis, control, and in-betweening methods.

Reference graph

Works this paper leans on

52 extracted references · 43 canonical work pages · cited by 1 Pith paper

[6]

Recurrent network models for human dynamics,

K. Fragkiadaki, S. Levine, P . Felsen, and J. Malik, “Recurrent network models for human dynamics,” in Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV) , ser. ICCV ’15. Washington, DC, USA: IEEE Computer Society, 2015, pp. 4346–4354

work page 2015
[25]

On human motion prediction using recurrent neural networks

J. Martinez, M. J. Black, and J. Romero, “On human motion predic- tion using recurrent neural networks,” CoRR, vol. abs/1705.02445, 2017

work page Pith review arXiv 2017
[26]

Auto- conditioned recurrent networks for extended complex human motion synthesis,

Y. Zhou, Z. Li, S. Shao, C. He, Z. Huang, and H. Li, “Auto- conditioned recurrent networks for extended complex human motion synthesis,” in International Conference on Learning Represen- tations, 2018

work page 2018
[1]

A deep learning framework for character motion synthesis and editing,

D. Holden, J. Saito, and T. Komura, “A deep learning framework for character motion synthesis and editing,” ACM T rans. Graph., vol. 35, no. 4, pp. 138:1–138:11, Jul. 2016

work page 2016
[2]

Motion graphs,

L. Kovar, M. Gleicher, and F. Pighin, “Motion graphs,” ACM T rans. Graph., vol. 21, no. 3, pp. 473–482, Jul. 2002

work page 2002
[3]

Modeling human motion using binary latent variables,

G. W. Taylor, G. E. Hinton, and S. Roweis, “Modeling human motion using binary latent variables,” in Proceedings of the 19th International Conference on Neural Information Processing Systems , ser. NIPS’06. Cambridge, MA, USA: MIT Press, 2006, pp. 1345– 1352

work page 2006
[4]

Motion graphs++: a compact generative model for semantic motion analysis and synthesis,

J. Min and J. Chai, “Motion graphs++: a compact generative model for semantic motion analysis and synthesis,” ACM T ransactions on Graphics (TOG), vol. 31, no. 6, p. 153, 2012

work page 2012
[5]

Phase-functioned neural networks for character control,

D. Holden, T. Komura, and J. Saito, “Phase-functioned neural networks for character control,” ACM T rans. Graph., vol. 36, no. 4, pp. 42:1–42:13, Jul. 2017

work page 2017

Show all 52 references

[7]

In- teractive control of avatars animated with human motion data,

J. Lee, J. Chai, P . S. A. Reitsma, J. K. Hodgins, and N. S. Pollard, “In- teractive control of avatars animated with human motion data,” ACM T rans. Graph., vol. 21, no. 3, pp. 491–500, Jul. 2002

2002
[8]

Planning biped locomotion using motion capture data and probabilistic roadmaps,

M. G. Choi, J. Lee, and S. Y. Shin, “Planning biped locomotion using motion capture data and probabilistic roadmaps,” ACM T rans. Graph., vol. 22, no. 2, pp. 182–203, Apr. 2003

2003
[10]

Natural character posing from a large motion database,

X. Wu, M. Tournier, and L. Reveret, “Natural character posing from a large motion database,” IEEE Computer Graphics and Appli- cations, vol. 31, no. 3, pp. 69–77, May 2011

2011
[11]

Spectral style transfer for human motion between independent actions,

M. E. Yumer and N. J. Mitra, “Spectral style transfer for human motion between independent actions,” ACM T rans. Graph., vol. 35, no. 4, pp. 137:1–137:8, Jul. 2016

2016
[12]

Realtime style transfer for unlabeled heterogeneous human motion,

S. Xia, C. Wang, J. Chai, and J. Hodgins, “Realtime style transfer for unlabeled heterogeneous human motion,” ACM T rans. Graph., vol. 34, no. 4, pp. 119:1–119:10, Jul. 2015

2015
[13]

Spectral-based group formation control,

S. Takahashi, K. Yoshida, T. Kwon, K. H. Lee, J. Lee, and S. Y. Shin, “Spectral-based group formation control,” Computer Graphics Forum, vol. 28, no. 2, pp. 639–648, 2009

2009
[14]

Controllable data sampling in the space of humanposes,

K. Yang, K. Youn, K. Lee, and J. Lee, “Controllable data sampling in the space of humanposes,” Comput. Animat. Virtual Worlds , vol. 26, no. 3-4, pp. 457–467, May 2015

2015
[15]

Realtime human motion control with a small number of inertial sensors,

H. Liu, X. Wei, J. Chai, I. Ha, and T. Rhee, “Realtime human motion control with a small number of inertial sensors,” in Symposium on Interactive 3D Graphics and Games , ser. I3D ’11. New York, NY, USA: ACM, 2011, pp. 133–140

2011
[16]

Synthesizing physically realistic human motion in low-dimensional, behavior- speciﬁc spaces,

A. Safonova, J. K. Hodgins, and N. S. Pollard, “Synthesizing physically realistic human motion in low-dimensional, behavior- speciﬁc spaces,” ACM T rans. Graph. , vol. 23, no. 3, pp. 514–521, Aug. 2004

2004
[17]

Motion synthesis and editing in low- dimensional spaces: Research articles,

H. J. Shin and J. Lee, “Motion synthesis and editing in low- dimensional spaces: Research articles,” Comput. Animat. Virtual Worlds, vol. 17, no. 3-4, pp. 219–227, Jul. 2006

2006
[18]

Performance animation from low- dimensional control signals,

J. Chai and J. K. Hodgins, “Performance animation from low- dimensional control signals,” ACM T rans. Graph. , vol. 24, no. 3, pp. 686–696, Jul. 2005

2005
[19]

Motion reconstruction using sparse accelerometer data,

J. Tautges, A. Zinke, B. Kr ¨uger, J. Baumann, A. Weber, T. Helten, M. M ¨uller, H.-P . Seidel, and B. Eberhardt, “Motion reconstruction using sparse accelerometer data,” ACM T rans. Graph., vol. 30, no. 3, pp. 18:1–18:12, May 2011

2011
[20]

Real-time posture reconstruction for microsoft kinect,

H. P . H. Shum, E. S. L. Ho, Y. Jiang, and S. Takagi, “Real-time posture reconstruction for microsoft kinect,” IEEE T ransactions on Cybernetics, vol. 43, no. 5, pp. 1357–1369, 2013

2013
[21]

Kinect posture re- construction based on a local mixture of gaussian process models,

Z. Liu, L. Zhou, H. Leung, and H. P . H. Shum, “Kinect posture re- construction based on a local mixture of gaussian process models,” IEEE T ransactions on Visualization and Computer Graphics , vol. 22, no. 11, pp. 2437–2450, Nov 2016

2016
[22]

Example-based human motion denoising,

H. Lou and J. Chai, “Example-based human motion denoising,” IEEE T ransactions on Visualization and Computer Graphics , vol. 16, no. 5, pp. 870–879, Sept 2010

2010
[23]

Imagenet classi- ﬁcation with deep convolutional neural networks,

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classi- ﬁcation with deep convolutional neural networks,” in Proceedings of the 25th International Conference on Neural Information Processing Systems, ser. NIPS’12. USA: Curran Associates Inc., 2012, pp. 1097–1105

2012
[24]

Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement 12 learning,

X. B. Peng, G. Berseth, K. Yin, and M. van de Panne, “Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement 12 learning,” ACM T ransactions on Graphics (Proc. SIGGRAPH 2017) , vol. 36, no. 4, 2017

2017
[27]

Hierarchical recurrent neural network for skeleton based action recognition,

Y. Du, W. Wang, and L. Wang, “Hierarchical recurrent neural network for skeleton based action recognition,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2015, pp. 1110–1118

2015
[28]

Long-term recurrent convolutional networks for visual recognition and description,

J. Donahue, L. A. Hendricks, M. Rohrbach, S. Venugopalan, S. Guadarrama, K. Saenko, and T. Darrell, “Long-term recurrent convolutional networks for visual recognition and description,” IEEE T ransactions on Pattern Analysis and Machine Intelligence , vol. 39, no. 4, pp. 677–69...

2017
[29]

Recurrent attention models for depth-based person identiﬁcation,

A. Haque, A. Alahi, and L. Fei-Fei, “Recurrent attention models for depth-based person identiﬁcation,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016, pp. 1229–1238

2016
[30]

Interactive character animation by learning multi-objective control,

K. Lee, S. Lee, and J. Lee, “Interactive character animation by learning multi-objective control,” in T o appear in SIGGRAPH Asia 2018, Dec 2018

2018
[31]

Li and R

Y. Li and R. Nevatia, Key Object Driven Multi-category Object Recognition, Localization and T racking Using Spatio-temporal Context . Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 409–422

2008
[32]

Learning spatio-temporal structure from rgb-d videos for human activity detection and anticipa- tion,

H. S. Koppula and A. Saxena, “Learning spatio-temporal structure from rgb-d videos for human activity detection and anticipa- tion,” in Proceedings of the 30th International Conference on Inter- national Conference on Machine Learning - Volume 28 , ser. ICML’13. JMLR.org, 2013...

2013
[33]

(2016) Cargnie mellon university motion database

CMU. (2016) Cargnie mellon university motion database. [Online]. Available: http://mocap.cs.cmu.edu/

2016
[34]

Documentation mocap database hdm05,

M. M ¨uller, T. R ¨oder, M. Clausen, B. Eberhardt, B. Kr ¨uger, and A. Weber, “Documentation mocap database hdm05,” Universit ¨at Bonn, Tech. Rep. CG-2007-2, June 2007

2007
[35]

Berkeley mhad: A comprehensive multimodal human action database,

F. Oﬂi, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy, “Berkeley mhad: A comprehensive multimodal human action database,” in 2013 IEEE Workshop on Applications of Computer Vision (WACV) , Jan 2013, pp. 53–60

2013
[36]

Action recognition based on a bag of 3d points,

W. Li, Z. Zhang, and Z. Liu, “Action recognition based on a bag of 3d points,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops , June 2010, pp. 9–14

2010
[37]

Modeling spatial and temporal variation in motion data,

M. Lau, Z. Bar-Joseph, and J. Kuffner, “Modeling spatial and temporal variation in motion data,” ACM T ransactions on Graphics, vol. 28, no. 5, 12 2009

2009
[38]

Gaussian process dy- namical models for human motion,

J. M. Wang, D. J. Fleet, and A. Hertzmann, “Gaussian process dy- namical models for human motion,” IEEE T ransactions on Pattern Analysis and Machine Intelligence , vol. 30, no. 2, pp. 283–298, Feb 2008

2008
[39]

Long short-term memory,

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997

1997
[40]

Towards end-to-end speech recognition with deep convolutional neural networks,

Y. Zhang, M. Pezeshki, P . Brakel, S. Zhang, C. Laurent, Y. Bengio, and A. C. Courville, “Towards end-to-end speech recognition with deep convolutional neural networks,” CoRR, vol. abs/1701.02720, 2017

2017 arXiv
[41]

Sequence to sequence learning with neural networks,

I. Sutskever, O. Vinyals, and Q. V . Le, “Sequence to sequence learning with neural networks,” CoRR, vol. abs/1409.3215, 2014

2014 arXiv
[42]

Unsuper- vised learning of video representations using lstms,

N. Srivastava, E. Mansimov, and R. Salakhutdinov, “Unsuper- vised learning of video representations using lstms,” CoRR, vol. abs/1502.04681, 2015

2015 arXiv
[43]

Style- based inverse kinematics,

K. Grochow, S. L. Martin, A. Hertzmann, and Z. Popovi ´c, “Style- based inverse kinematics,” ACM T rans. Graph., vol. 23, no. 3, pp. 522–531, Aug. 2004

2004
[44]

Dropout: A simple way to prevent neural networks from overﬁtting,

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overﬁtting,” Journal of Machine Learning Research , vol. 15, pp. 1929–1958, 2014

1929
[45]

Batch normalization: Accelerating deep network training by reducing internal covariate shift,

S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” CoRR, vol. abs/1502.03167, 2015

2015 arXiv
[46]

Fast and accurate deep network learning by exponential linear units (elus),

D. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (elus),” CoRR, vol. abs/1511.07289, 2015

2015 arXiv
[47]

ADADELTA: an adaptive learning rate method,

M. D. Zeiler, “ADADELTA: an adaptive learning rate method,” CoRR, vol. abs/1212.5701, 2012

2012 arXiv
[48]

Real-time physical modelling of character movements with microsoft kinect,

H. P . H. Shum and E. S. L. Ho, “Real-time physical modelling of character movements with microsoft kinect,” in Proceedings of the 18th ACM Symposium on Virtual Reality Software and T echnology, ser. VRST ’12. New York, NY, USA: ACM, Dec 2012, pp. 17–24

2012
[49]

Hu- man3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments,

C. Ionescu, D. Papava, V . Olaru, and C. Sminchisescu, “Hu- man3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments,” IEEE T ransactions on Pattern Analysis and Machine Intelligence , 2014

2014
[50]

Learning motion manifolds with convolutional autoencoders,

D. Holden, J. Saito, T. Komura, and T. Joyce, “Learning motion manifolds with convolutional autoencoders,” in SIGGRAPH Asia 2015 T echnical Briefs, ser. SA ’15. New York, NY, USA: ACM, 2015, pp. 18:1–18:4

2015
[51]

On the properties of neural machine translation: Encoder-decoder approaches,

K. Cho, B. van Merrienboer, D. Bahdanau, and Y. Bengio, “On the properties of neural machine translation: Encoder-decoder approaches,” CoRR, vol. abs/1409.1259, 2014

2014 arXiv
[52]

Empirical evalua- tion of gated recurrent neural networks on sequence modeling,

J. Chung, C ¸ . G¨ulc ¸ehre, K. Cho, and Y. Bengio, “Empirical evalua- tion of gated recurrent neural networks on sequence modeling,” CoRR, vol. abs/1412.3555, 2014

2014 arXiv
[53]

Latent structured models for human pose estimation,

C. S. Catalin Ionescu, Fuxin Li, “Latent structured models for human pose estimation,” in International Conference on Computer Vision, 2011. He Wang is an Assistant Professor (Lecturer in UK) at School of Computing, University of Leeds, UK. His research interest is computer gr...

2011

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [6]

Recurrent network models for human dynamics,

K. Fragkiadaki, S. Levine, P . Felsen, and J. Malik, “Recurrent network models for human dynamics,” in Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV) , ser. ICCV ’15. Washington, DC, USA: IEEE Computer Society, 2015, pp. 4346–4354

work page 2015

[2] [25]

On human motion prediction using recurrent neural networks

J. Martinez, M. J. Black, and J. Romero, “On human motion predic- tion using recurrent neural networks,” CoRR, vol. abs/1705.02445, 2017

work page Pith review arXiv 2017

[3] [26]

Auto- conditioned recurrent networks for extended complex human motion synthesis,

Y. Zhou, Z. Li, S. Shao, C. He, Z. Huang, and H. Li, “Auto- conditioned recurrent networks for extended complex human motion synthesis,” in International Conference on Learning Represen- tations, 2018

work page 2018

[4] [1]

A deep learning framework for character motion synthesis and editing,

D. Holden, J. Saito, and T. Komura, “A deep learning framework for character motion synthesis and editing,” ACM T rans. Graph., vol. 35, no. 4, pp. 138:1–138:11, Jul. 2016

work page 2016

[5] [2]

Motion graphs,

L. Kovar, M. Gleicher, and F. Pighin, “Motion graphs,” ACM T rans. Graph., vol. 21, no. 3, pp. 473–482, Jul. 2002

work page 2002

[6] [3]

Modeling human motion using binary latent variables,

G. W. Taylor, G. E. Hinton, and S. Roweis, “Modeling human motion using binary latent variables,” in Proceedings of the 19th International Conference on Neural Information Processing Systems , ser. NIPS’06. Cambridge, MA, USA: MIT Press, 2006, pp. 1345– 1352

work page 2006

[7] [4]

Motion graphs++: a compact generative model for semantic motion analysis and synthesis,

J. Min and J. Chai, “Motion graphs++: a compact generative model for semantic motion analysis and synthesis,” ACM T ransactions on Graphics (TOG), vol. 31, no. 6, p. 153, 2012

work page 2012

[8] [5]

Phase-functioned neural networks for character control,

D. Holden, T. Komura, and J. Saito, “Phase-functioned neural networks for character control,” ACM T rans. Graph., vol. 36, no. 4, pp. 42:1–42:13, Jul. 2017

work page 2017