REVIEW 4 major objections 2 minor 43 references
Improving Facial Rig Semantics for Tracking and Retargeting
T0 review · 4 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A fine-tuned rig turns facial tracking into animation controls that transfer to another person or character.
desk verdict As delivered, the manuscript contains no part of the claimed facial-rig paper — the full text is an unrelated contour-tree paper — so the method cannot be reviewed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Implicit differentiation through the tracker: instead of differentiating through every iteration of the tracking optimizer, the fine-tuning step differentiates the stationarity condition of the tracker's optimization problem, so the tracker can be treated as a black box. Around it sits the Simon-Says calibration procedure, which fits each rig to a performer's or character's distinctive motion signature, and volumetric morphing, which fits the rig geometry to each subject.
What would settle it
Take a fixed performance, track it with a Simon-Says-calibrated rig before and after fine-tuning, and compare the output controls against a ground-truth set of semantic pose labels for the same performance; if the fine-tuned controls are not closer to the intended expressions, or if retargeting to a second subject does not improve, the central claim is falsified.
Extended reading notes
Core claim
The central claim is that retargeting fidelity is limited by an inverse problem: a rig may be well-behaved as a forward function from controls to expressions, yet ill-conditioned as an inverse mapping from performance to controls, so a well-calibrated rig still produces semantically noisy controls when tracking. The paper proposes fine-tuning the rig parameters using the tracking objective so that the tracker outputs more meaningful controls, and does this via implicit differentiation of the tracker's optimization, so the tracker itself does not need to be differentiable. The fine-tuned rig, built on the same rig framework for performer and target and fitted by volumetric morphing with Simon
Load-bearing premise
The load-bearing premise is that implicit differentiation through the tracker gives a usable gradient for fine-tuning even when the tracker itself is non-differentiable; if that gradient cannot be computed or is too noisy in practice, the fine-tuning step collapses.
Editorial extensions
If this is right
- Tracking with the fine-tuned rig should yield controls that map to semantically named expressions, making retargeting a matter of replaying controls rather than remapping them between rigs.
- Same-framework retargeting person-to-person means one calibration protocol can be reused across subjects, while person-to-character retargeting needs character-specific Simon-Says expressions that capture the character's motion signature.
- Implicit differentiation opens the fine-tuning to trackers with non-differentiable components, such as collision or contact handling, without modifying the tracker itself.
- The method should reduce manual cleanup of facial animation in game and VR pipelines, since controls correspond to meaningful poses rather than arbitrary parameter combinations.
Reading between the lines
- If the fine-tuning is applied to the target character's rig rather than the performer's, it may also serve as a rig-authoring tool, exposing which controls are actually observable in the character's motion signature.
- A direct testable extension: track the same performance with and without fine-tuning, then measure retargeting error on held-out expressions; if control semantics improve, retargeting error should drop even when geometry error is unchanged.
- Because calibration uses Simon-Says expressions, the method's ceiling depends on how well that finite set covers the motion signature; extending to procedurally generated or data-driven expression sets is a natural next step.
- The supplied full text is a separate paper on distributed contour trees, so the fine-tuning derivation, implementation details, and experimental comparisons for the facial-rig claim are not present in the material given; those claims are supported only by the abstract.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes to improve facial-performance retargeting by using the same rig framework for tracked performer and target character, fitting rigs via volumetric morphing, calibrating rigs with a Simon-Says expression set, and then fine-tuning the tracking rig so that the animation controls produced during tracking are semantically meaningful. The abstract states that fine-tuning relies on implicit differentiation and can treat the tracker as a '(potentially non-differentiable) black box.' The supplied full text, however, is not the facial-rig paper but an unrelated contour-tree manuscript (arXiv:2508.08433). No equations, algorithmic details, comparisons, or experiments for the claimed facial-rig method are present, so none of the central claims can be checked from the submission.
Significance. If the method worked as described, it would be practically useful: retargeting within a shared rig framework avoids the difficult cross-framework semantic-mapping problem, and a fine-tuning step that improves control semantics without degrading geometry reconstruction would be a meaningful contribution for game and VR character animation. The paper also points to a real phenomenon: a well-calibrated forward rig can still have an ill-conditioned inverse, so control outputs during tracking need not be semantically clean. However, as submitted, the significance cannot be assessed. There is no method section, no reproduction information, no fabricated or measured results, and the central mathematical claim about implicit differentiation is unstated in a checkable form. The submission therefore does not currently support a scientific verdict.
major comments (4)
- [Abstract, final paragraph] The claim that fine-tuning 'relies on implicit differentiation so that the tracker can be treated as a (potentially non-differentiable) black box' is internally problematic as stated. Implicit differentiation of an optimization-based tracker requires the tracker's output to be a differentiable function of the rig parameters: at a minimum, smooth stationary-point conditions with an invertible Jacobian. A genuinely non-differentiable tracker (with argmax, clipping, discrete decisions, or other non-smooth operations) does not satisfy those conditions, and the implicit-function theorem does not apply. If a differentiable surrogate or a relaxed tracker is intended, that must be stated and its gradient accuracy validated. This is the load-bearing premise of the whole fine-tuning step, and the submission contains no equations or regularity assumptions to support it.
- [Supplied full text (whole submission)] The full text attached to this submission is arXiv:2508.08433, 'Extremely Scalable Distributed Computation of Contour Trees via Pre-Simplification' — a different paper with a different title, authors, and subject. None of the claimed volumetric morphing, Simon-Says calibration, fine-tuning objective, implicit differentiation, or retargeting evaluation appears anywhere in the supplied text. This is not a minor formatting defect; it means the manuscript as submitted is not the paper under review, and every claim in the abstract is unverified.
- [Abstract, calibration paragraph] The evaluation circularity concern is real and needs to be addressed explicitly. The abstract says the Simon-Says expressions are used to calibrate each rig, and the same rig is then fine-tuned to 'promote the output of more semantically meaningful animation controls.' If the fine-tuning objective and the calibration protocol use the same Simon-Says expressions, improved control semantics may be tied to those expressions rather than to general retargeting. The submission must specify how evaluation separates calibration/fine-tuning expressions from held-out expressions and target characters, and report retargeting metrics on expressions not used in the optimization.
- [Abstract, ill-conditioned-inverse remark] The paper itself notes that well-calibrated rigs can still lead to undesirable controls when tracking because a well-behaved function can have an arbitrarily ill-conditioned inverse. If the proposed fine-tuning computes implicit gradients through such an inverse, the same ill-conditioning can make those gradients numerically unstable or useless. The submission provides no gradient checks, no condition-number estimates, and no regularization analysis. This is particularly important because the final paragraph asserts the fine-tuning works in 'real-world scenarios' without any experimental evidence being present.
minor comments (2)
- [Abstract and title] The term 'Simon-Says expressions' is used without definition. For a general graphics and VR audience, the set should be defined in the introduction, along with why it is expected to capture the motion signature of a rig.
- [Abstract, paragraph 2] The phrase 'the same framework (3DMM, FLAME, MetaHuman, etc.) for both subjects' is not precise: it is unclear whether 'framework' means a shared parameterization, a shared topology, or a shared rig interface. This ambiguity affects the interpretation of the retargeting claim.
Circularity Check
No demonstrated circularity; supplied full text is a different paper, so the claimed derivation chain is not available for audit.
full rationale
The only text attributable to the cited facial-rig paper (arXiv:2508.08429) is the abstract. The supplied 'FULL TEXT' is arXiv:2508.08433 on distributed contour trees, a different paper with different authors and topic, so no equations, loss functions, or evaluation protocols from the facial-rig paper are available for analysis. The abstract's potential circularity concern—that the rigs are both calibrated and fine-tuned on Simon-Says expressions—is not demonstrated by any quoted equation or explicit statement that the fine-tuning objective equals the calibration target. In fact, the abstract distinguishes the two stages: calibrated rigs already produce desired expressions, while fine-tuning is motivated by undesirable controls arising during tracking. The asserted use of implicit differentiation through a 'potentially non-differentiable' tracker is a missing justification and a feasibility/correctness risk, not a circular reduction of the paper's output to its input. Without the paper's own equations or experimental details, no load-bearing step can be exhibited as equivalent to its own inputs by construction. This is an honest non-finding under the evidentiary rule that circularity must be quoted and explicitly reduced.
Assumptions & free parameters
free parameters (2)
- Simon-Says expression set =
not specified; 'carefully chosen'
- Fine-tuning objective (implicit) =
not specified
assumptions (4)
- standard math A well-behaved function can have an arbitrarily ill-conditioned inverse
- domain assumption Using the same rig framework for performer and target suffices to avoid cross-rig semantic mapping problems during retargeting
- domain assumption Volumetric morphing can fit the chosen rig faithfully to performer and target
- domain assumption Implicit differentiation through a (potentially non-differentiable) black-box tracker yields usable gradients for fine-tuning
Cite this review
Pith. "Pith review of Improving Facial Rig Semantics for Tracking and Retargeting." pith.science (2026). https://pith.science/paper/KO2NVWIQ
@misc{pith2026250808429,
author = {Pith},
title = {Pith review of: Improving Facial Rig Semantics for Tracking and Retargeting},
year = {2026},
howpublished = {\url{https://pith.science/paper/KO2NVWIQ}},
note = {Machine review of arXiv:2508.08429}
}
read the original abstract
In this paper, we consider retargeting a tracked facial performance to either another person or to a virtual character in a game or virtual reality (VR) environment. We remove the difficulties associated with identifying and retargeting the semantics of one rig framework to another by utilizing the same framework (3DMM, FLAME, MetaHuman, etc.) for both subjects. Although this does not constrain the choice of framework when retargeting from one person to another, it does force the tracker to use the game/VR character rig when retargeting to a game/VR character. We utilize volumetric morphing in order to fit facial rigs to both performers and targets; in addition, a carefully chosen set of Simon-Says expressions is used to calibrate each rig to the motion signatures of the relevant performer or target. Although a uniform set of Simon-Says expressions can likely be used for all person to person retargeting, we argue that person to game/VR character retargeting benefits from Simon-Says expressions that capture the distinct motion signature of the game/VR character rig. The Simon-Says calibrated rigs tend to produce the desired expressions when exercising animation controls (as expected). Unfortunately, these well-calibrated rigs still lead to undesirable controls when tracking a performance (a well-behaved function can have an arbitrarily ill-conditioned inverse), even though they typically produce acceptable geometry reconstructions. Thus, we propose a fine-tuning approach that modifies the rig used by the tracker in order to promote the output of more semantically meaningful animation controls, facilitating high efficacy retargeting. In order to better address real-world scenarios, the fine-tuning relies on implicit differentiation so that the tracker can be treated as a (potentially non-differentiable) black box.
Reference graph
Works this paper leans on
-
[1]
A. Acharya and V . Natarajan. A parallel and memory efficient al- gorithm for constructing the contour tree. In Proceedings of the 2015 IEEE Pacific Visualization Symposium (PacificVis), pp. 271–278. IEEE, New York, 2015. doi: 10.1109/PACIFICVIS.2015.7156387 3
-
[2]
J. Ahrens, M. Arienti, U. Ayachit, J. Bennett, R. Binyahib, A. Biswas, P.-T. Bremer, E. Brugger, R. Bujack, H. Carr, et al. The ECP ALPINE project: In situ and post hoc visualization infrastructure and analysis capabilities for exascale. International Journal of High Performance Computing Applications (IJHPCA), 39(1):32–51, 2025. doi: 10.1177/ 10943420241286521 1
work page 2025
-
[3]
A. S. Almgren, J. B. Bell, M. J. Lijewski, Z. Luki ´c, and E. Van An- del. Nyx: A massively parallel AMR code for computational cosmol- ogy. The Astrophysical Journal, 765(1):39, 2013. doi: 10.1088/0004 -637X/765/1/39 5
doi:10.1088/0004 2013
-
[4]
C. L. Bajaj, V . Pascucci, and D. R. Schikore. The contour spectrum. In Proceedings of Visualization 1997, pp. 167–173. IEEE, New York,
work page 1997
-
[5]
T. F. Banchoff. Critical Points and Curvature for Embedded Poly- hedra. Journal of Differential Geometry , 1:245–256, 1967. doi: 10. 4310/jdg/1214428092 2
arXiv 1967
-
[6]
R. L. Boyell and H. Ruston. Hybrid techniques for real-time radar simulation. In Proceedings of the 1963 Fall Joint Computer Confer- ence, pp. 445–458. ACM, New York, 1963. doi: 10.1145/1463822. 1463869 2
- [7]
- [8]
Show all 43 references
-
[9]
H. Carr, C. Sewell, L.-T. Lo, and J. Ahrens. Hybrid Data-Parallel Contour Tree Computation. Technical Report LA-UR-15-24579, Los Alamos National Laboratory, 2015. 3
2015
-
[10]
Carr and J
H. Carr and J. Snoeyink. Path seeds and flexible isosurfaces: Using topology for exploratory visualization. In Proceedings of Eurograph- ics Visualization Symposium 2003 , pp. 49–58, 285, 2003. doi: 10. 5555/769922.769927 2
2003
-
[11]
Carr and J
H. Carr and J. Snoeyink. Representing interpolant topology for con- tour tree computation. In H.-C. Hege, K. Polthier, and G. Scheuer- mann, eds., Topology-Based Methods in Visualization II, Mathematics and Visualization, pp. 59–73. Springer, Berlin, Heidelberg, 2009. doi: 10....
2009 doi
-
[12]
H. Carr, J. Snoeyink, and U. Axen. Computing contour trees in all dimensions. Computational Geometry Theory and Applications , 24(2):75–94, 2003. doi: 10.1016/S0925-7721(02)00093-7 2, 7
2003 doi
-
[13]
H. Carr, J. Snoeyink, and M. van de Panne. Flexible isosurfaces: Sim- plifying and displaying scalar topology using the contour tree. Com- putational Geometry: Theory and Applications , 43(1):42–58, 2010. doi: 10.1016/j.comgeo.2006.05.009 2, 3, 4
2010 doi
-
[15]
H. A. Carr, G. H. Weber, C. M. Sewell, O. Rubel, P. Fasel, and J. P. Ahrens. Scalable contour tree computation by data parallel peak prun- ing. IEEE Transactions on Visualization and Computer Graphics , 27(4):2437–2454, 2021. doi: 10.1109/TVCG.2019.2948616 1
2021
-
[16]
Consortium, J
M. Consortium, J. A. Bae, M. Baptiste, A. L. Bodor, D. Brittain, J. Buchanan, D. J. Bumbarger, M. A. Castro, B. Celii, E. Cobos, F. Collman, N. M. da Costa, S. Dorkenwald, L. Elabbady, P. G. Fahey, T. Fliss, E. Froudarakis, J. Gager, C. Gamlin, A. Halageri, J. Heb- ditch, Z. J...
2025
-
[17]
Dagum and R
L. Dagum and R. Menon. OpenMP: an industry standard API for shared-memory programming. IEEE Computational Science and En- gineering, 5(1):46–55, 1998. doi: 10.1109/99.660313 5
1998 doi
-
[18]
Edelsbrunner, D
H. Edelsbrunner, D. Letscher, and A. Zomorodian. Topological Per- sistence and Simplification. In Proceedings of the 41st Annual Sympo- sium on Foundations of Computer Science, pp. 454–463. IEEE, 2000. doi: 10.1109/SFCS.2000.892133 3
-
[19]
Edelsbrunner and E
H. Edelsbrunner and E. P. M ¨ucke. Simulation of simplicity: A tech- nique to cope with degenerate cases in geometric algorithms. ACM Transactions on Graphics , 9(1):66–104, 1990. doi: 10.1145/77635. 77639 2
1990 doi
-
[20]
Freeman and S
H. Freeman and S. P. Morse. On Searching A Contour Map for a Given Terrain Elevation Profile. Journal of the Franklin Institute, 284(1):1– 25, 1967. doi: 10.1016/0016-0032(67)90568-6 2
1967 doi
-
[21]
Gold and S
C. Gold and S. Cormack. Spatially Ordered Networks and Topo- graphic Reconstruction. In Proceedings of the 2nd International ACM Symposium on Spatial Data Handling, pp. 74–85, 1986. doi: 10.1080/ 02693798708927800 2
1986
-
[23]
Gueunet, P
C. Gueunet, P. Fortin, J. Jomier, and J. Tierny. Task-based augmented merge trees with Fibonacci heaps. In 2017 IEEE 7th Symposium on Large Data Analysis and Visualization (LDAV), pp. 6–15. IEEE, New York, 2017. doi: 10.1109/LDA V.2017.8231846 3
2017
-
[24]
Harvey and Y
W. Harvey and Y . Wang. Topological landscape ensembles for vi- sualization of scalar-valued functions. Computer Graphics Forum , 29(3):993–1002, 2010. doi: 10.1111/j.1467-8659.2009.01706.x 2
2010
-
[25]
Hristov and H
P. Hristov and H. Carr. W-structures in contour trees. In I. Hotz, T. Bin Masood, F. Sadlo, and J. Tierny, eds., Topological Methods in Data Analysis and Visualization VI, pp. 3–18. Springer, Cham, 2021. doi: 10.1007/978-3-030-83500-2 1 3, 4
2021 doi
-
[26]
Hristov, G
P. Hristov, G. H. Weber, H. Carr, O. R ¨ubel, and J. Ahrens. Data par- allel hypersweeps for in situ topological analysis. In 2020 IEEE 10th Symposium on Large Data Analysis and Visualization (LDAV), pp. 12–
2020
-
[27]
Itoh and K
T. Itoh and K. Koyamada. Isosurface generation by using extrema graphs. In Proceedings Visualization ’94, pp. 77–83, 1994. doi: 10. 1109/VISUAL.1994.346334 2
1994
-
[28]
doi: 10.1109/LDA V51489.2020.00008 1, 4, 7, 12
IEEE, New York, 2020. doi: 10.1109/LDA V51489.2020.00008 1, 4, 7, 12
2020
-
[29]
M. Li, H. Carr, O. R ¨ubel, B. Wang, and G. H. Weber. Distributed Augmentation, Hypersweeps, and Branch Decomposition of Contour Trees for Scientific Exploration. IEEE Transactions on Visualization and Computer Graphics (TVCG), 31(1):152–162, 2025. doi: 10.1109/ TVCG.2024.3456...
2025
-
[30]
A. G. Landge, V . Pascucci, A. Gyulassy, J. C. Bennett, H. Kolla, J. Chen, and P.-T. Bremer. In-situ feature extraction of large scale combustion simulations using segmented merge trees. InSC14: Inter- national Conference for High Performance Computing, Networking, Storage and...
2014 doi
-
[31]
Moreland, C
K. Moreland, C. Sewell, W. Usher, L. ta Lo, J. Meredith, D. Pugmire, J. Kress, H. Schroots, K.-L. Ma, H. Childs, M. Larsen, C.-M. Chen, R. Maynard, and B. Geveci. VTK-m: Accelerating the visualization toolkit for massively threaded architectures.IEEE Computer Graphics and Appl...
2016 doi
-
[32]
G. L. Miller and J. H. Reif. Parallel Tree Contraction Part I: Funda- mentals. Advances in Computing Research, 5:47–72, 1989. 4
1989
-
[33]
Morozov and G
D. Morozov and G. Weber. Distributed merge trees. In Proceedings of the 18th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 93–102. ACM, New York, 2013. doi: 10. 1145/2442516.2442526 4
2013
-
[34]
Morozov and T
D. Morozov and T. Peterka. Block-parallel data analysis with DIY2. In 2016 IEEE 6th Symposium on Large Data Analysis and Visualiza- tion (LDAV), pp. 29–36. IEEE, New York, 2016. doi: 10.1109/LDA V. 2016.7874307 5
2016
-
[35]
Nigmetov and D
A. Nigmetov and D. Morozov. Local-Global Merge Tree Computa- tion with Local Exchanges . In SC19: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–13. IEEE Computer Society, Los Alamitos, CA, USA, Nov 2019. doi: 10.1145/3295500.3...
2019
-
[36]
LAMBDA - Matter Density, 2019
NASA / LAMBDA Archive Team. LAMBDA - Matter Density, 2019. Accessed Apr 10th, 2025,https://lambda.gsfc.nasa.gov/ education/graphic_history/matterd.html. 5
2019
-
[37]
G. Reeb. Sur les points singuliers d’une forme de Pfaff compl `etement int´egrable ou d’une fonction num ´erique. Comptes Rendus de l’Acad`emie des Sciences de Paris, 222:847–849, 1946. 2
1946
-
[38]
Pascucci and K
V . Pascucci and K. Cole-McLaughlin. Parallel computation of the topology of level sets. Algorithmica, 38(1):249–268, 2003. doi: 10. 1007/s00453-003-1052-3 2, 3, 4
2003
-
[39]
S. P. Tarasov and M. N. Vyalyi. Construction of Contour Trees in 3D in O(n log n) steps. In Proceedings, 14th ACM Symposium on Com- putational Geometry, pp. 68–75, 1998. doi: 10.1145/276884.276892 2
1998
-
[40]
Smirnov and D
D. Smirnov and D. Morozov. Triplet merge trees. In Topological Methods in Data Analysis and Visualization V . TopoInVis 2017., Math- ematics and Visualization, pp. 19–36. Springer, Cham, 2020. doi: 10. 1007/978-3-030-43036-8 2 3, 4
2017
-
[41]
Weber, S
G. Weber, S. Dillard, H. Carr, V . Pascucci, and B. Hamann. Topology- controlled volume rendering. IEEE Transactions on Visualization and Computer Graphics, 13(2):330–341, 2007. doi: 10.1109/TVCG.2007 .47 2
2007 doi
-
[42]
van Kreveld, R
M. van Kreveld, R. van Oostrum, C. L. Bajaj, V . Pascucci, and D. R. Schikore. Contour trees and small seed sets for isosurface traversal. In Proceedings of the 13th ACM Symposium on Computational Ge- ometry, pp. 212–220. ACM, New York, 1997. doi: 10.1145/262839. 269238 2
1997 doi
-
[43]
Zhang, C
X. Zhang, C. L. Bajaj, and N. Baker. Fast matching of volumetric functions using multi-resolution dual contour trees. Technical report, Texas Institute for Computational and Applied Mathematics, Austin, Texas, 2004. 2 A D ISCUSSION : PARAMETER ESTIMATION In Sec. 4.4, we conduc...
2004
-
[44]
Wester, W
B. Wester, W. Gray-Roncal, S. Hider, T. Gion, J. Matelsky, J. Downs, D. Xenes, T. Rose, K. Romero, L. Kitchell, D. Ramsden, M. Sanchez, and D. Moore. The brain observatory storage service & database (BossDB). Accessed March 30th, 2022, https://bossdb.org/ project/microns-minnie. 5
2022
-
[1997]
doi: 10.1109/VISUAL.1997.663875 3
1997
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.