Pith. sign in

REVIEW 4 major objections 6 minor 24 references

Domain Adaptation of Drag Reduction Policy to Partial Measurements

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A learned reconstruction map lets a full-state drag-reduction policy run on base-pressure sensors alone, matching the wake-measurement optimum in simulation.

desk verdict Useful empirical adaptation of full-state RL drag control to partial base-pressure measurements; the 'optimal' claim is not yet established, but the action-history finding is a solid contribution. read the letter →

arxiv 2507.04309 v1 pith:HR2S2SED submitted 2025-07-06 cs.LG cs.SYeess.SY

Anton Plaksin , Georgios Rigas This is my paper · ORCID
classification cs.LGcs.SYeess.SY
keywords dragreductionpartialmeasurementsdomainadaptationreinforcementlearningactiveflowcontrolDSFTPOMDPbluffbody
open problems The Measurement Problem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that an optimal feedback controller trained on dense wake measurements can be ported to a vehicle that only has pressure sensors on its base. The method trains a Domain-Specific Feature Transfer map that reconstructs the current full wake measurement from a history of partial measurements and past actions, then composes that map with the already-optimal full-state policy. In a two-dimensional direct numerical simulation of flow past a square bluff body, the composed policy reaches drag close to the full-state optimum and beats a policy trained directly on partial measurements. If this holds, real vehicles could inherit simulation-trained drag-reduction policies using only onboard sensors.

What carries the argument

The load-bearing object is the Domain-Specific Feature Transfer (DSFT) map $T_\theta$, a supervised neural network trained by equation (1) to map the history of base-pressure measurements and past actions to the current wake measurement vector. Its job is to make the partial observation look like the full observation to the existing optimal policy. The second ingredient is the choice of a deliberately simple optimal full-state policy, an affine function of the measurements with no hidden layers, which the paper finds to be more robust to reconstruction errors than a deep policy.

What would settle it

Run the composed PDA policy on the same bluff-body flow but with a Reynolds number or actuator authority outside the training dataset; if the reconstruction error on those histories is large and drag degrades toward the uncontrolled value, the claim that the composition is optimal for the partial-measurement problem fails. A simpler check: compare the composed policy against an oracle that receives the true full measurement at every step; the optimality claim predicts the two drag values should coincide.

Watch

Extended reading notes

Core claim

The central claim is that for the partial-observation drag-reduction problem, an optimal policy is obtained without any new reinforcement-learning training: train a map $T_\theta$ on stored trajectories so that $T_\theta(o^\mathrm{PM}_{t-n:t}, a_{t-m:t-1}) \approx o^\mathrm{FM}_t$, then define the deployed policy as $\pi^*(T_\theta(\cdot), a_{t-1})$, the composition of the full-state optimal policy with the reconstructed measurement. In the simulated bluff-body flow at Reynolds number 100, this composed policy matches the drag of the full-measurement optimal policy, exceeds the drag reduction of a policy trained directly on partial measurements, and needs only one hidden layer in the reconstruction network.

Load-bearing premise

The argument assumes that a finite history of base-pressure measurements and past actions contains enough information to reconstruct the current full wake measurement, and that the dataset used to train that reconstruction covers the states the composed policy visits.

Editorial extensions

If this is right

  • Partial-measurement drag control can reach full-state optimality without retraining, as long as a reconstruction map can be trained from stored trajectories.
  • The optimal history length $n$ can be selected offline by evaluating the composed policy, and the paper finds that longer base-pressure histories improve performance.
  • Action history barely affects performance, suggesting the controlled flow can be modelled without an explicit delay in the MDP.
  • A single hidden layer in the reconstruction map suffices; deeper maps add no gain and can overfit, while linear maps are insufficient.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reconstruction premise generalises, the same composition strategy could be applied to other actuator-sensor pairs in fluid control, such as transferring policies between differently instrumented wind-tunnel models.
  • The optimality result is shown at one Reynolds number and in two dimensions; a natural test is whether the same map length and architecture survive three-dimensional or turbulent wakes, where the full measurement is higher-dimensional.
  • The DSFT map is trained on a fixed dataset; online adaptation of $T_\theta$ with minimal labels during deployment would be a direct extension if the flow conditions drift.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Policy Domain Adaptation (PDA) for transferring a full-state feedback control policy to a setting with only partial measurements. A Domain-Specific Feature Transfer (DSFT) map T_theta is trained by supervised regression to reconstruct full wake pressure measurements from a history of base pressure measurements and past actions, and the resulting policy is defined as the composition of T_theta with a pre-trained full-state optimal policy (Eq. 2). The method is demonstrated on a two-dimensional DNS of flow past a square bluff body at Re=100, where it reportedly achieves drag close to the full-state optimal policy and outperforms a directly trained partial-observation RL policy. The paper also uses the approach to study the influence of measurement and action history lengths and draws conclusions about the needed policy architecture.

Significance. If the central optimality claim were established, the paper would offer a practically useful recipe for adapting simulation-trained RL flow-control policies to realistic sensor configurations without retraining, together with guidance on minimal history length. The idea is simple and clearly stated, and the empirical comparison uses 65 runs, which is a reasonable effort for a DNS-based study. However, the paper provides no proof or quantitative validation of the reconstruction accuracy needed for the composition to be optimal, does not describe how the dataset D is collected, and does not report statistical uncertainty beyond min-max shading. The strength of the paper lies in the demonstrated empirical trend; the weakness is that the main claim is considerably stronger than the evidence.

major comments (4)
  1. [Section 1, bullet 1 and Eq. (2)] The statement that composing the trained DSFT map with the full-state optimal policy 'we obtain the optimal policy for the partial measurement case' is not supported by the provided evidence. The composition is optimal only if T_theta reconstructs o_FM_t accurately on the support of the states visited when the composed policy is deployed. The paper minimizes MSE on dataset D in Eq. (1) but never reports the reconstruction error or its distribution, never describes how D is collected (under which behavior policy, over which time horizon, and from how many trajectories), and never validates that D covers the deployment distribution. I request that the data-collection protocol be specified, reconstruction error be reported on training and held-out data, and closed-loop performance under the composed policy be validated. Otherwise the claim should be weakened to 'empirically near-optimal' rather than 'optimal'.
  2. [Section 4, Fig. 2b] The claimed optimal history length n=m=48 is selected from the same performance sweep that is then reported as the finding. Because n and m are hyperparameters chosen from the curve, the statement that the method 'enables determination of the optimal history length' overstates the evidence. A separate holdout validation, cross-validation, or a plateau analysis is needed to distinguish a genuine optimum from overfitting to the evaluation set. The paper should also report whether the reconstruction error of T_theta depends monotonically on n and m, since that is the mechanism presumed to explain the policy performance.
  3. [Section 3, Eqs. (1) and (2)] There is an inconsistency in the action-history index ranges: Eq. (1) uses a_{t-m:t} in the input to T_theta, while Eq. (2) uses a_{t-m:t-1}. If the current action a_t is part of the reconstruction input, the deployment-time availability of a_t must be discussed, because the composed policy in Eq. (2) cannot know a_t before computing it. In addition, Eq. (2) carries the trajectory index i in the policy expression, which is a formatting error that should be removed. Please correct these and ensure all index ranges are consistent throughout.
  4. [Section 4, Fig. 2a and Appendix A] The claim that the composed policy outperforms the directly trained partial-observation RL policy is based only on min-max shading over 65 runs, with no standard deviations, confidence intervals, or significance tests reported. Moreover, the 'optimal' full-state policy pi* is itself only empirically optimal with respect to a specific TQC training run, a particular architecture (no hidden layers), and fixed hyperparameters. I request that the paper quantify uncertainty (for example, with confidence intervals or paired statistical tests) and clearly state that 'optimal' means 'optimal within the studied policy class and training setup' rather than a proven global optimum.
minor comments (6)
  1. [Figure 2 and surrounding text] The color descriptions in the text are inconsistent: the text refers to 'blue and green lines' for the PDA approach, then to an 'orange line' for the no-action-history policy, while the green line was earlier identified as the full-state optimal policy. Please align the legend, figure colors, and the textual references.
  2. [Appendix B] The phrase 'tree type of neural networks' should read 'three types of neural networks'.
  3. [General] The paper does not state whether code, trained models, or the dataset D will be made available, which limits reproducibility; please add a data and code availability statement.
  4. [Section 2] The time discretization is not specified: please state the length of the action interval in simulation time units and the episode length over which returns are computed.
  5. [Abstract and Section 1] The abstract refers to 'a simplified road vehicle,' but the environment is a two-dimensional square bluff body; 'simplified two-dimensional bluff body' would be more precise and avoid overgeneralization.
  6. [Section 4] The statement that 'all policies converge to a reduced drag state' is supported only by the last-100-steps average; please specify the convergence criterion used or show the full time series for representative runs.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the PDA composition is an empirical supervised reconstruction plus policy composition, not an identity; self-citation is replicated, not load-bearing.

full rationale

No circular derivation is present. The central pipeline trains a DSFT map T_theta by minimizing the squared reconstruction error in Eq. (1), then forms the composed policy in Eq. (2). The claimed optimality is conditional on T_theta accurately reconstructing full measurements on the deployment distribution, and it is validated empirically by comparing against a directly trained partial-observation RL policy and the full-state optimal baseline. Neither equation is equivalent to the target result by construction: Eq. (1) is a supervised regression loss, Eq. (2) is a composition, and the optimality claim is not embedded in the training objective. The paper's reference to Xia et al. (2024) includes a co-author, but the paper explicitly states it replicates those findings and uses them as a baseline, so the self-citation is not load-bearing; it is supported by the paper's own experiments. The selection of n=m=48 as the best history length comes from the performance sweep shown in Fig. 2b and is reported as an empirical optimum, not as a prediction; this is hyperparameter fitting rather than a circular step. The strongest concern is that the optimality claim in Section 1 is stronger than the evidence without a quantitative bound on reconstruction error over the deployment distribution, but that is a correctness/assumption issue, not a circularity issue.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on standard CFD and RL tools, but adds several empirical choices: a learned reconstruction map, selected history lengths, and an assumption about the sufficiency of partial observations. No new physical entities are introduced.

free parameters (4)
  • T_θ model architecture and weights = one hidden layer with 128 neurons; weights fitted by Adam on Eq. (1)
    Chosen by comparing linear, [128], and [64,64] networks (Appendix B); the weights are fitted to the paired dataset D.
  • Measurement and action history lengths n and m = n=m=48 selected from Fig. 2b
    The 'minimum history length giving the optimal result' is obtained by sweeping n and m on the same performance metric, so it is a selected value rather than a prediction.
  • Full-state policy matrices A and B = learned by TQC RL, no hidden layers
    The policy π*(o_FM,a_prev)=a* tanh(A o_FM + B a_prev) is trained by RL; the no-hidden-layer class was selected because it is more robust to input imprecision (Appendix A).
  • RL and supervised training hyperparameters = learning rates 1e-4 and 1e-3, batch sizes 128 and 10000, epochs 10000, discount 0.99
    Taken mostly from Xia et al. (2024) for TQC and from Appendix B for T_θ; the central result depends on these tuned choices.
assumptions (5)
  • standard math Navier-Stokes equations accurately model the 2D flow around the square body at Re=100
    The DNS solves the Navier-Stokes equations with FEniCS/Dolfin (Section 2).
  • domain assumption The optimal full-state policy lies in the class π(o_FM_t, a_{t-1})
    Inherited from Chen et al. (2023) and Xia et al. (2024) and replicated here; the infinite-dimensional state is compressed to wake pressure probes (Section 2).
  • ad hoc to paper A finite history of partial measurements and actions is sufficient to reconstruct the current full measurement
    Assumed in Section 3, Eq. (1); the map T_θ is trained to do this, but the sufficiency of the history is not proven and dataset D is not described.
  • domain assumption Full measurements are available during training, only partial during deployment
    Explicitly stated as a limitation in Section 5; the method cannot be applied if no full-state data exist at training time.
  • ad hoc to paper The collected dataset D is representative of states visited by the composed policy
    T_θ is trained on D via Eq. (1); if D is collected under a different policy, distribution shift may break the reconstruction; this is not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Domain Adaptation of Drag Reduction Policy to Partial Measurements." pith.science (2026). https://pith.science/paper/HR2S2SED

@misc{pith2026250704309,
  author       = {Pith},
  title        = {Pith review of: Domain Adaptation of Drag Reduction Policy to Partial Measurements},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HR2S2SED}},
  note         = {Machine review of arXiv:2507.04309}
}
read the original abstract

Feedback control of fluid-based systems poses significant challenges due to their high-dimensional, nonlinear, and multiscale dynamics, which demand real-time, three-dimensional, multi-component measurements for sensing. While such measurements are feasible in digital simulations, they are often only partially accessible in the real world. In this paper, we propose a method to adapt feedback control policies obtained from full-state measurements to setups with only partial measurements. Our approach is demonstrated in a simulated environment by minimising the aerodynamic drag of a simplified road vehicle. Reinforcement learning algorithms can optimally solve this control task when trained on full-state measurements by placing sensors in the wake. However, in real-world applications, sensors are limited and typically only on the vehicle, providing only partial measurements. To address this, we propose to train a Domain Specific Feature Transfer (DSFT) map reconstructing the full measurements from the history of the partial measurements. By applying this map, we derive optimal policies based solely on partial data. Additionally, our method enables determination of the optimal history length and offers insights into the architecture of optimal control policies, facilitating their implementation in real-world environments with limited sensor information.

Figures

Figures reproduced from arXiv: 2507.04309 by the authors.

Figure 1
Figure 1. Flow pressure fields with the jet actuators and measurement locations for the following [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Time series and an average of the last 100 values of the Drag Coefficient (negative reward) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Time series and average of the last 100 values of the Drag Coefficient (negative reward) for [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Average of the last 100 values of the Drag Coefficient for the policies obtained by PDA [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 22 canonical work pages

  1. [1]

    Bertsekas

    D. Bertsekas. Dynamic Programming and Optimal Control: Volume I. Athena Scientific, Belmont, Massachusetts, 2012

    work page 2012

  2. [2]

    S. L. Brunton, B. R. Noack, and P. Koumoutsakos. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics, 52: 0 477--508, 2020

    work page 2020

  3. [3]

    L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch. Decision transformer: Reinforcement learning via sequence modeling. Proceedings of 35th Conference on Neural Information Processing Systems (NeurIPS 2021), 2021

    work page 2021

  4. [4]

    W. Chen, Q. Wang, L. Yan, G. Hu, and B. R. Noack. Deep reinforcement learning-based active flow control of vortex-induced vibration of a square cylinder. Physics of Fluids, 35, 2023

    work page 2023

  5. [5]

    H. Choi, J. Lee, and H. Park. Aerodynamics of heavy vehicles. Annual Review of Fluid Mechanics, 46: 0 441--468, 2014

    work page 2014

  6. [6]

    Duriez, S

    T. Duriez, S. L. Brunton, and B. R. Noack. Machine Learning Control --- Taming Nonlinear Dynamics and Turbulence. Springer, 2016

    work page 2016

  7. [7]

    Garnier, J

    P. Garnier, J. Viquerat, J. Rabault, A. Larcher, A. Kuhnle, and E. Hachem. A review on deep reinforcement learning for fluid mechanics: an update. Computers and Fluids, 225, 2021

    work page 2021

  8. [8]

    Hafner, J

    D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering diverse domains through world models. ArXiv:2301.04104, 2024

    arXiv 2024

Show all 24 references
  1. [9]

    K. V. Katsikopoulos and S. E. Engelbrecht. Markov decision processes with delays and asynchronous cost collection. IEEE Transactions on Automatic Control, 48 0 (4): 0 568--574, 2003

    2003

  2. [10]

    Kuznetsov, P

    A. Kuznetsov, P. Shvechikov, A. Grishin, and D. Vetrov. Controlling overestimation bias with truncated mixture of continuous distributional quantile critics. Proceedings of the 37 th International Conference on Machine Learning (ICML 2020), 119: 0 5556--5566, 2020

    2020

  3. [11]

    Li and M

    J. Li and M. Zhang. Reinforcement-learning-based control of confined cylinder wakes with stability analyses. Journal of Fluid Mechanics, 932: 0 A44, 2022

    2022

  4. [12]

    Logg, K.-A

    A. Logg, K.-A. Mardal, and G. Wells. Automated Solution of Differential Equations by the Finite Element Method. Springer, 2012

    2012

  5. [13]

    Y. Mao, S. Zhong, and H. Yin. Active flow control using deep reinforcement learning with time-delays in markov decision process and autoregressive policy. Physics of Fluids, 34 0 (5), 2022

    2022

  6. [14]

    Paris, S

    R. Paris, S. Beneddine, and J. Dandois. Robust flow control and optimal sensor placement using deep reinforcement learning. Journal of Fluid Mechanics, 913: 0 A25, 2021

    2021

  7. [15]

    Rabault and A

    J. Rabault and A. Kuhnle. Accelerating deep reinforcement learning strategies of flow control through a multi- environment approach. Physics of Fluids, 31 0 (9), 2019

    2019

  8. [16]

    Rabault, M

    J. Rabault, M. Kuchta, A. Jensen, U. Reglade, and N. Cerardi. Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. Journal of Fluid Mechanics, 865: 0 281--302, 2019

    2019

  9. [17]

    Rabault, F

    J. Rabault, F. Ren, W. Zhang, H. Tang, and H. Xu. Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization. Journal of Hydrodynamics, 32: 0 234--246, 2020

    2020

  10. [18]

    Raffin, A

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22 0 (268): 0 1--8, 2021. URL http://jmlr.org/papers/v22/20-1364.html

    2021

  11. [19]

    Sharma, W

    P. Sharma, W. T. Chung, B. Akoush, and M. Ihme. A review of physics-informed machine learning in fluid mechanics. Energies, 16 0 (5), 2023

    2023

  12. [20]

    M. N. Sudin, M. A. Abdullah, S. A. Shamsuddin, F. R. Ramli, and M. M. Tahir. Review of research on vehicles aerodynamic drag reduction methods. International Journal of Mechanical and Mechatronics Engineering, 14 0 (2): 0 37--47, 2014

    2014

  13. [21]

    R. S. Sutton and A. G. Barto. Reinforcement Learning An Introduction (Second Edition). The MIT Press, Cambridge, Massachusetts, 2018

    2018

  14. [22]

    H. Tang, J. Rabault, A. Kuhnle, Y. Wang, and T. Wang. Robust active flow control over a range of reynolds numbers using an artificial neural network trained through deep reinforcement learning. Physics of Fluids, 32 0 (5), 2020

    2020

  15. [23]

    P. Wei, Y. Ke, and C. K. Goh. A general domain specific feature transfer framework for hybrid domain adaptation. IEEE Transactions on Knowledge and Data Engineering, 31 0 (8): 0 1440--1451, 2018

    2018

  16. [24]

    C. Xia, J. Zhang, E. C. Kerrigan, and G. Rigas. Active flow control for bluff body drag reduction using reinforcement learning with partial measurements. Journal of Fluid Mechanics, 981: 0 A17, 2024

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.