Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Reward driven workflows for unsupervised explainable analysis of phases and ferroic variants from atomically resolved imaging data

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that the hidden choices in unsupervised atomically resolved image analysis can be selected automatically by rewarding segmentations whose detected domain walls are straight and continuous.

desk verdict Incremental but useful extension of the group's reward-driven workflow to phase and ferroic segmentation; the central accuracy claim needs quantitative backing before I'd trust it. read the letter →

arxiv 2411.12612 v1 pith:32XF3R3S submitted 2024-11-19 cond-mat.mtrl-sci cs.HCcs.LG

Kamyar Barakati , Yu Liu , Chris Nelson , Maxim A. Ziatdinov , Xiaohang Zhang , Ichiro Takeuchi , Sergei V. Kalinin This is my paper · ORCID
classification cond-mat.mtrl-scics.HCcs.LG
keywords reward-drivenworkflowunsupervisedsegmentationscanningtransmissionelectronmicroscopyferroelectricdomainwallsSm-dopedBiFeO3Gaussianmixturemodelvariationalautoencoderhyperparameteroptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the hidden choices in unsupervised analysis of atomically resolved images, such as how large a local image patch to use and which clustering or autoencoder settings to adopt, can be selected automatically by rewarding the outcome for looking physically plausible. In Sm-doped BiFeO3 thin films, the workflow defines a good segmentation as one whose detected domain walls are straight and continuous, then searches descriptor size and covariance-type space for the settings that maximize those rewards. The result is a segmentation and latent representation that recovers the material's polarization and phase structure without ground-truth labels. The authors extend the same logic to a rotationally invariant variational autoencoder to disentangle structural factors of variation, and they show that the reward choice determines what the workflow can see, for example failing to find walls at 7% Sm doping where the real domain structure does not meet the straightness criterion.

What carries the argument

The load-bearing object is the reward-driven workflow: a search over the product space of descriptor window size and the unsupervised model's hyperparameters, scored by two reward functions computed from detected domain-wall lines. Wall lines are extracted by edge detection followed by the Hough transform; Reward_1 is the negative average curvature of these lines, favoring straight walls, and Reward_2 is the total wall length divided by the number of segments, favoring long continuous walls. The optimizer produces a Pareto front of solutions from which an operator chooses. The same reward structure is applied to a conditional rotationally invariant variational autoencoder, where the reward governs descriptor size and the rotationally invariant latent angle tracks polarization rotation at domain walls.

What would settle it

Take a material with intentionally curved or wavy ferroelectric domain walls, run the reward-driven workflow, and compare the reward-optimal segmentation to a physics-based polarization map; if the workflow's selected descriptors consistently miss curved walls while a low-reward solution matches the ground truth, the straightness reward is steering the analysis away from the true structure. A cheaper check within the paper's own data is the 7% Sm sample, where the workflow finds no walls: test whether any descriptor setting in the searched space produces a segmentation correlated with the ground-truth polarization map, and if one does while scoring poorly on the rewards, the reward is the wrong objective.

Watch

Extended reading notes

Core claim

The central claim is that explainable unsupervised segmentation of atomically resolved scanning transmission electron microscopy data can be reduced from a laborious manual hyperparameter search to an optimization over a reward function. The reward function encodes the physics that ferroelectric domain walls are nearly straight and continuous: Reward_1 minimizes the average curvature of Hough-transformed wall lines, and Reward_2 maximizes total wall length per segment. Optimizing the descriptor window size $w_1,w_2$ and the Gaussian mixture model covariance type against these rewards selects a descriptor of size [34,50] with tied covariance, whose segmentation agrees with the physics-based ground-truth polarization map and visualizes both domain walls and a mis-tilt boundary. The paper further embeds a conditional rotationally invariant variational autoencoder in the same reward loop, obtaining latent variables whose spatial maps reproduce the ferroelectric-to-nonferroelectric transition and the $\pi/2$ rotational symmetry of domain walls, with dropout-based uncertainty maps identifying unreliable regions.

Load-bearing premise

The premise that keeps the whole optimization meaningful is that physically relevant domain walls in the material under study are straight and continuous; if that geometric prior does not match the true structure, the optimizer selects settings that hide the real phases, as the paper itself reports for 7% Sm-doped BFO.

Editorial extensions

If this is right

  • The clustering workflow with reward-optimized descriptors yields segmentation that matches the physics-based ground truth, so phase and ferroic variant maps can be produced without labeled training data.
  • The same reward loop applied to a variational autoencoder gives latent variables whose spatial maps reproduce the ferroelectric-to-nonferroelectric phase transition and the rotational symmetry of domain walls.
  • Because the search is automated, the workflow can be embedded in real-time microscope operation, replacing days or weeks of manual analysis.
  • The reward design transfers across the Sm-doping series, and at 7% doping the workflow correctly reports that no walls meet the straight/continuous criteria, exposing the reward definition's limitation for morphotropic compositions.
  • The approach generalizes to other physics-discovery tasks whenever physics-based or human-heuristic reward functions can be formulated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the authors leave implicit is that the straightness and continuity prior is not a universal law: materials with curved, wavy, or charged domain walls would need different rewards, and the paper's 7% Sm result is direct evidence of that boundary.
  • The Pareto front offers a calibration tool: comparing reward-selected solutions against a ground-truth correlation map could separate imaging artifacts, such as the mis-tilt effect, from genuine structural features without labels.
  • A testable extension is to plug the same reward functions into other descriptor types, such as atomic-coordinate vectors or four-dimensional STEM data, and check whether the reward-optimal settings still match physics-based ground truth.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes reward-driven workflows for unsupervised analysis of atomically resolved STEM images of Sm-doped BiFeO3, where descriptor window size and GMM covariance type are optimized against two geometric rewards (domain-wall straightness and continuity) before clustering, and a similar reward-driven approach is applied to a conditional rotationally invariant variational autoencoder (CrVAE) for disentangling latent factors. The authors claim that the resulting segmentation shows excellent agreement with physics-based ground-truth polarization maps, and that the approach is robust, explainable, unsupervised, and suitable for real-time instrument operation.

Significance. If the central claim is established, the paper would make a useful contribution to automated microscopy: it directly addresses hyperparameter sensitivity in unsupervised descriptor-based segmentation, provides a concrete workflow with publicly available code, and uses a dataset with independently computed polarization ground truth. The reward-driven framing is appealing because it makes the human-biased choices in the pipeline explicit and optimizable. However, the paper's load-bearing claim—that reward-optimized solutions recover the physically correct phase and polarization structure—is currently supported mainly by visual comparison and by the same geometric rewards used for optimization; the only quantitative agreement metric in the paper is computed for manually chosen window sizes, not for the reward-selected Pareto solutions. The 7% Sm result in Section IV is an honest acknowledgment that the reward prior is not universally valid, but it also shows why the missing quantitative validation matters.

major comments (4)
  1. [Section II, Figures 2 and 3] The central claim that the reward-optimized clustering shows 'excellent agreement with physics-based ground truth' is not quantitatively demonstrated. The correlation heatmap in Figure 2(A) is computed for GMM with a fixed 5-component setting across manually chosen window sizes, not for the reward-selected Pareto solutions in Figure 3. The solutions selected on the Pareto front (e.g., window [34,50] with 'tied' covariance) are evaluated only by Reward_1 and Reward_2, which encode a geometric prior and not label fidelity. To establish that the reward tracks physical accuracy, the authors should report quantitative segmentation metrics (e.g., adjusted Rand index, normalized mutual information, or correlation with the k-means-quantized Pxy ground truth) for the selected Pareto solutions, and compare against non-reward baselines such as random parameter choices or all evaluated configurations.
  2. [Section IV and Summary] The 7% Sm result is explicitly acknowledged in the text as a limitation: the workflow finds no domain walls because the observed domain structures do not satisfy the straight-and-continuous reward criteria. This is not a minor caveat; it means the reward definitions encode a material-specific prior that is invalid at the morphotropic phase boundary composition. The Summary nevertheless states that the approach 'shows excellent agreement with physics-based ground truth analysis' without conditioning on the composition/phase regime. The manuscript should state the validity conditions for the reward functions, and either provide a fallback or an automatic diagnostic that detects when the reward prior is violated, rather than reporting a null result as a successful optimization.
  3. [Equation (1), Section II] The printed formula for Reward_1 is corrupted and is not a well-defined curvature expression as typeset. Curvature of a planar curve parameterized by (x(s), y(s)) is normally κ = |x'y'' − y'x''| / (x'^2 + y'^2)^(3/2), but the equation in the manuscript has mismatched parentheses and garbled exponents, making it impossible for a reader to reproduce the calculation from the text. Please provide a clean, correctly typeset equation and explicitly define the parameterization of the detected wall segments and how derivatives are estimated from discrete points.
  4. [Section III, Figure 5] The CrVAE reward-optimized workflow is validated only visually against the ground-truth polarization map in Figure 5(D); no quantitative agreement measure, uncertainty interval, or comparison to a non-rewarded baseline is given. In addition, the number of KDE peaks ('top 5 peaks') used to classify domain walls from the latent z2 variable is a free hyperparameter that is not included in the reward optimization, yet it directly determines the detected domain-wall geometry and hence the computed rewards. The authors should either include this parameter in the optimization/search or provide a sensitivity analysis, and they should quantify how well the reward-selected [50,58] descriptor recovers the true domain structure.
minor comments (4)
  1. [Figure 1 caption] The caption states 'GMM with 5 fixed components' in (D) and '6 fixed components' in (E), but the text in Section II says Figure 1 shows 'how different window sizes influence region segmentation ... using GMM clustering' without specifying that the component count differs; please clarify the variables that are fixed in this figure.
  2. [Figure 3 caption and text] The phrases 'both rewards successfully achieved' and 'both rewards are at their minimum' in the discussion of Figure 3 are confusing because one reward is minimized and the other is maximized; please use consistent language (e.g., 'optimized' versus 'minimized') when describing the Pareto front.
  3. [Section II, Hough transform] The Hough transform has internal thresholds and accumulator parameters that affect the detected line segments and therefore the computed rewards; these are not listed among the workflow hyperparameters, so it is unclear whether they were fixed or optimized.
  4. [General] The reference list contains numerous OCR artifacts (e.g., 'M', 'e', 'A' substituted for letters), and some entries are malformed; these should be cleaned for a final version.

Circularity Check

2 steps flagged · score 4.0 of 10

The reward-driven selection loop is partially self-referential: the same straightness/continuity rewards used to choose hyperparameters are used to declare the optimized solution successful, while ground-truth agreement for those selected solutions is only asserted visually.

  1. self definitional [Section II, Figure 3 paragraph (page 9)]
    "The green point in Figure 3(A) represents the optimal solution with the highest rewards. This is shown in Figure 3(B), where both predefined objectives straightness, continuous lines with minimal curvature have been successfully achieved"

    Reward_1 is defined as minimizing domain-wall curvature and Reward_2 as maximizing line length/continuity (Eqs. 1-2). The optimizer chooses w1, w2, and GMM covariance type by maximizing these two rewards, so reporting that the selected Pareto solution 'successfully achieved' the same objectives is a restatement of the optimization target rather than an independent validation of physical accuracy. The Summary's claim of 'excellent agreement with physics-based ground truth analysis' is not backed by a quantitative correlation for the reward-selected [34,50] tied-covariance solution; the only quantitative ground-truth heat map (Figure 2A) is computed for a fixed 5-component GMM across window sizes, before the reward optimization.

  2. self definitional [Section III, Figure 5 paragraph (pages 12-13)]
    "As shown in Figure 5(A), where Pareto-optimal points illustrate the trade-off between straightness and lines length, the goal is to achieve a solution closest to the origin [0, 0], representing optimal rewards in both objectives. Figure 5(B) shows that both objectives straightness and continuous length were successfully achieved"

    As in the clustering case, the VAE descriptor size is selected by optimizing the same two rewards (orientation consistency/straightness and continuous wall length). Asserting that Figure 5(B) 'successfully achieved' these objectives is therefore a description of the optimization result, not an independent check of physical fidelity. The comparison to the ground truth polarization map in Figure 5(D) is visual and qualitative, so this second 'success' also reduces to the reward definition by construction.

full rationale

The underlying computations are largely self-contained: GMM clustering and VAE are standard unsupervised methods, hyperparameters are explicitly optimized against defined rewards, and an independent physics-based Pxy ground truth exists. Citations to the authors' prior reward-driven workflow concept (refs 50, 51) and to pyroVED (ref 71) are ordinary method provenance and are not load-bearing in a circular sense; no uniqueness theorem or ansatz is imported from self-citations. The genuine circularity is the validation loop: the straightness/continuity rewards used to select hyperparameters are also used to declare the solution successful (Figures 3B and 5B), so the optimality claim is definitional. The paper's own Section IV result for 7% Sm-doped BFO, where no domain walls are found because the actual structure does not meet the reward criteria, confirms that the reward is a modeling prior rather than an emergent discovery, which limits generality but also shows the limitation is disclosed. Because the central phase/ferroic-variant discovery still has independent content in the clustering/VAE outputs and the visual ground truth, the circularity is moderate, not complete.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities; the reward functions are scoring rules, not postulates. The main borrowed assumptions are the domain-wall geometry prior, the accuracy of the atomic coordinates, and the correctness of the ground truth polarization maps.

free parameters (3)
  • Descriptor window size (w1, w2) = [34, 50] for the pure BFO GMM solution; [50, 58] for the CrVAE solution
    Selected by reward-driven optimization; all displayed segmentation maps depend on this choice and the paper shows results vary strongly with it.
  • GMM covariance type = tied in the selected pure BFO solution
    A categorical hyperparameter in the reward optimization space (full, tied, diagonal, spherical); it affects the shape of clusters.
  • Number of KDE peaks used for CrVAE domain-wall classification = 5 peaks (highest density)
    Manual choice in Section III to convert the z2 latent variable into domain-wall markers; no sensitivity analysis is reported.
assumptions (4)
  • domain assumption Ferroelectric domain walls in the studied images are continuous, straight, and low-curvature.
    Defines Reward_1 and Reward_2 in Section II; Section IV reports this assumption fails for 7% Sm BFO, where the workflow finds no domain walls.
  • domain assumption Atomic column coordinates provided with the dataset are accurate.
    All descriptors are patches centered on known atomic positions (Section II); coordinate errors propagate into every descriptor and label.
  • domain assumption Ground truth polarization maps from ref 54 are correct.
    Used as the validation benchmark in Figures 2 to 6; the paper does not independently verify these maps.
  • domain assumption Edges found by edge detection and Hough transform correspond to physical domain walls.
    The rewards are computed from these line fits; the paper notes in Figure 2(E) that a mistilt artifact can appear as an extra cluster or wall, so the mapping is not always physical.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reward driven workflows for unsupervised explainable analysis of phases and ferroic variants from atomically resolved imaging data." pith.science (2026). https://pith.science/paper/32XF3R3S

@misc{pith2026241112612,
  author       = {Pith},
  title        = {Pith review of: Reward driven workflows for unsupervised explainable analysis of phases and ferroic variants from atomically resolved imaging data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/32XF3R3S}},
  note         = {Machine review of arXiv:2411.12612}
}
read the original abstract

Rapid progress in aberration corrected electron microscopy necessitates development of robust methods for the identification of phases, ferroic variants, and other pertinent aspects of materials structure from imaging data. While unsupervised methods for clustering and classification are widely used for these tasks, their performance can be sensitive to hyperparameter selection in the analysis workflow. In this study, we explore the effects of descriptors and hyperparameters on the capability of unsupervised ML methods to distill local structural information, exemplified by discovery of polarization and lattice distortion in Sm doped BiFeO3 (BFO) thin films. We demonstrate that a reward-driven approach can be used to optimize these key hyperparameters across the full workflow, where rewards were designed to reflect domain wall continuity and straightness, ensuring that the analysis aligns with the material's physical behavior. This approach allows us to discover local descriptors that are best aligned with the specific physical behavior, providing insight into the fundamental physics of materials. We further extend the reward driven workflows to disentangle structural factors of variation via optimized variational autoencoder (VAE). Finally, the importance of well-defined rewards was explored as a quantifiable measure of success of the workflow.

Figures

Figures reproduced from arXiv: 2411.12612 by the authors.

Figure 2
Figure 2. A) Heat map of the correlation coefficient between ground truth images of polarization component Pxy with the label images for the pairs of (w1, w2), B) Ground truth labeling with color-coded polarization regions, C) to F) Segmentation of regions based on the descriptors sizes and fixed 5 number of components in GMM clustering [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. (A) Pareto front solutions representing the trade-off between two rewards, R1 (Straightness) and R2 (Length). The red and green markers highlight the solutions selected by operator during analysis according to be prerequisites of the experiment, (B) Solution acquired by the workflow, showing the best possible trade-off between objectives of the experiment, (C) Another option in pareto front solution that operator ha… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mic-hackathon 2024: Hackathon on Machine Learning for Electron and Scanning Probe Microscopy

    cond-mat.mtrl-sci 2025-06 unverdicted novelty 3.0 of 10

    A hackathon report summarizing 19 machine-learning projects for electron and scanning probe microscopy, with code and data releases but no single testable scientific claim.

Reference graph

Works this paper leans on

12 extracted references · 9 canonical work pages · cited by 1 Pith paper

  1. [1]

    #−𝑑".𝑑!#|(𝑑!#⁄+𝑑

    The centroids for the GMM clusters are available in the associated Colab notebook. 5 Figure 1: (A) HAADF image showing the selected region of interest used for analysis. (B) Ground truth polarization map of the region, illustrating the actual polarization distribution. (C) Selected descriptor examples. (D) Clustered regions using GMM with 5 fixed componen...

    work page 2003

  2. [10]

    A.; Minor, A

    (52) Volkert, C. A.; Minor, A. M. Focused ion beam microscopy and micromachining. MRS bulleAn 2007, 32 (5), 389-399. (53) Damjanovic, D. A morphotropic phase boundary system based on polarizaMon rotaMon and polarizaMon extension. Applied Physics Le7ers 2010, 97 (6). 19 (54) Ziatdinov, M.; Nelson, C.; Vasudevan, R. K.; Chen, D.; Kalinin, S. V. Building fer...

    arXiv 2007

  3. [11]

    J.; Paige, B.; Hernández-Lobato, J

    (70) Kusner, M. J.; Paige, B.; Hernández-Lobato, J. M. Grammar variaMonal autoencoder. In InternaAonal conference on machine learning, 2017; PMLR: pp 1945-1954. (71) Ziatdinov, M. A. pyroVED. h]ps://github.com/ziatdinovmax/pyroVED (accessed. (72) ValleM, M.; Ziatdinov, M.; Liu, Y .; Kalinin, S. V. Physics and chemistry from parsimonious representaMons: im...

    work page 2017

  4. [13]

    Physics-based reward driven image analysis in microscopy

    (49) Kang, S.; Park, J.; Lee, M. Machine learning-enabled autonomous operaMon for atomic force microscopes. Review of ScienAfic Instruments 2023, 94 (12). (50) BarakaM, K.; Yuan, H.; Goyal, A.; Kalinin, S. V. Physics-based reward driven image analysis in microscopy. arXiv preprint arXiv:2404.14146

    work page Pith review arXiv 2023

  5. [183]

    Kernel density esMmaMon and its applicaMon

    (73) Węglarczyk, S. Kernel density esMmaMon and its applicaMon. In ITM web of conferences, 2018; EDP Sciences: Vol. 23, p 00037. (74) Sezan, M. I. A peak detecMon algorithm and its applicaMon to histogram-based image data reducMon. Computer vision, graphics, and image processing 1990, 49 (1), 36-51

    work page 2018

  6. [197]

    Interface-induced ferroelectric domains and charged domain walls in Bi Fe O 3/Sr Ti O 3 superlasces

    (23) Fu, Z.; Chen, H.; Liu, Y .; Liu, M.; Liu, W.-M. Interface-induced ferroelectric domains and charged domain walls in Bi Fe O 3/Sr Ti O 3 superlasces. Physical Review B 2021, 103 (19), 195301. (24) Borisevich, A. Y .; Eliseev, E. A.; Morozovska, A. N.; Cheng, C. J.; Lin, J. Y .; Chu, Y . H.; Kan, D.; Takeuchi, I.; Nagarajan, V.; Kalinin, S. V. Atomic-s...

    2021

  7. [1987]

    X-ray and neutron sca]ering from rough surfaces

    (3) Sinha, S.; Sirota, E.; Garoff; Stanley, H. X-ray and neutron sca]ering from rough surfaces. Physical Review B 1988, 38 (4),

    work page 1988

  8. [2013]

    AberraMon correcMon in the STEM

    (6) Krivanek, O.; Dellby, N.; Spence, A.; Camps, R.; Brown, L. AberraMon correcMon in the STEM. In Electron Microscopy and Analysis 1997, Proceedings of the InsAtute of Physics Electron Microscopy and Analysis Group Conference, University of Cambridge, 2-5 September 1997, 2022; CRC Press: pp 35-40. (7) de Jonge, N.; Houben, L.; Dunin-Borkowski, R. E.; Ros...

    work page 1997

Show all 12 references
  1. [2015]

    B.; Berkels, B.; Dahmen, W.; Binev, P .; Sanchez, S

    (9) Yankovich, A. B.; Berkels, B.; Dahmen, W.; Binev, P .; Sanchez, S. I.; Bradley, S. A.; Li, A.; Szlufarska, I.; Voyles, P. M. Picometre-precision analysis of scanning transmission electron microscopy images of plaMnum nanocatalysts. Nat. Commun. 2014, 5, 4155, ArMcle. DOI: ...

    2014 doi

  2. [2016]

    G.; Prentza, A.; Koutsouris, D.; Chatziioannou, A

    (46) Sifakis, E. G.; Prentza, A.; Koutsouris, D.; Chatziioannou, A. A. EvaluaMng the effect of various background correcMon methods regarding noise reducMon, in two-channel microarray data. Computers in biology and medicine 2012, 42 (1), 19-29. (47) Chen, S. F.; Rosenfeld, R. A...

    2012

  3. [2024]

    C.; Duscher, G.; Kalinin, S

    (51) BarakaM, K.; PraMush, U.; Houston, A. C.; Duscher, G.; Kalinin, S. V. Unsupervised Reward-Driven Image SegmentaMon in Automated Scanning Transmission Electron Microscopy Experiments. arXiv preprint arXiv:2409.12462

    arXiv

  4. [6361]

    M.; Al-Najjar, A.; Ghosh, A.; Hinkle, J

    (31) Mukherjee, D.; Roccapriore, K. M.; Al-Najjar, A.; Ghosh, A.; Hinkle, J. D.; Lupini, A. R.; Vasudevan, R. K.; Kalinin, S. V.; Ovchinnikova, O. S.; Ziatdinov, M. A. A roadmap for edge compuMng enabled automated mulMdimensional transmission electron microscopy. Microscopy To...

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.