Pith. sign in

REVIEW 3 major objections 7 minor 49 references

Deep reinforcement learning for separation control in turbulent wind-tunnel flow

T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A PPO agent trained directly in a turbulent wind tunnel learns a low-duty-cycle separation-control policy in about two minutes; truncating the reward horizon to the convective time is what makes the learning fast and stable.

desk verdict Genuine online DRL in a fully turbulent wind tunnel, with a useful but confounded return-horizon comparison. read the letter →

arxiv 2608.10829 v1 pith:BO7YTP5Q submitted 2026-08-11 physics.flu-dyn

Sofia Avdiiv , Andre Weiner , Ben Steinfurth This is my paper · ORCID
classification physics.flu-dyn
keywords deepreinforcementlearningactiveseparationcontrolturbulentboundarylayerwindtunnelexperimentProximalPolicyOptimizationtemporalcreditassignmentwallshearstresssensorpulsedjetactuators
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper shows that a reinforcement-learning controller can be trained online, in real time, inside a fully turbulent wind-tunnel flow over a one-sided diffuser, using only the signal from a single wall-shear-stress sensor and binary open/close commands to three pulsed jet actuators. The central methodological claim is that the temporal credit-assignment horizon matters: an agent whose return is truncated to the convective response time of the flow (four interaction steps, about 24 ms) converges to a stable low-duty-cycle control policy within roughly 20 episodes (about two minutes of interaction), whereas an agent using a discounted return over the full trajectory keeps fluctuating. Deployed, the fast-converging controller achieves a local forward-flow fraction of about 53% at the feedback sensor, about one percentage point above optimized periodic open-loop forcing at a comparable duty cycle of 12.5%. The authors argue this makes reinforcement learning a practical tool for discovering robust and interpretable separation-control strategies directly from physical experiments.

What carries the argument

The mechanism is the finite-horizon return of Eq. (5), which sums the next N_c=4 instantaneous sign rewards after an action without discounting, constraining credit assignment to the convective time scale of 24 ms; this is contrasted with the infinite-horizon discounted return of Eq. (4). This truncation is what reduces the variance of the PPO advantage estimates and, according to the paper, what allows the actor-critic updates to settle quickly on a stable policy. The rest of the machinery is the experimental loop: a single bidirectional MEMS wall-shear-stress sensor at the transitory detachment point supplies the scalar state, a Bernoulli policy over the binary valve action is sampled each 6 ms step, and the reward is the sign of the next sensor state.

What would settle it

Run the full-trajectory agent with gamma=1 (no discounting) and check whether it still converges within about 20 episodes; if it does, the advantage the paper attributes to the convective-time horizon actually comes from removing discounting, not from truncation.

Watch

Extended reading notes

Core claim

The discovery is that when Proximal Policy Optimization is coupled directly to a wind-tunnel experiment, restricting the return to a finite horizon aligned with the characteristic convective response time of the flow (N_c=4 steps, gamma=1, Eq. 5) yields faster convergence, lower policy entropy, and a steadier low-duty-cycle actuation pattern than the standard discounted full-trajectory return (gamma=0.995, Eq. 4). The resulting policy commands valve openings with a mean duty cycle near 12.5%, matching the previously identified optimal open-loop duty cycle for this configuration, and produces a local forward-flow fraction at the reference sensor of approximately 0.53, slightly exceeding the best periodic open-loop control. The controller is interpretable: it maps the measured wall-shear-stress sign to a probability of valve opening, and for the majority of visited states it keeps that probability below 0.5. The authors therefore claim that a model-free policy can be learned directly from interactions with a fully turbulent experimental flow, and that temporal credit assignment is the key design choice that makes that learning efficient.

Load-bearing premise

The comparison that anchors the central claim changes both the return horizon and the discount factor at the same time (N_c=4 with gamma=1 versus full trajectory with gamma=0.995), so attributing the faster, more stable learning to the horizon length alone is an untested assumption; the reward is also the sign of the same sensor used as the performance metric.

Editorial extensions

If this is right

  • If the finite-horizon return is the cause of faster convergence, then online RL for turbulent flow control should set the return horizon by the physical response time rather than by the episode length.
  • The trained DRL policy discovers the same optimal duty cycle (about 12.5%) that open-loop parametric sweeps identified, suggesting that the agent recovers known actuation physics without a model.
  • The roughly 1% improvement over optimized periodic forcing at the sensor location is accompanied by lower forward-flow fractions upstream, meaning the current single-sensor objective trades global performance for local attachment.
  • Training in about two minutes of flow interaction makes online experimental RL practical for other turbulent-flow control tasks with sparse sensing and binary actuation.
  • The low entropy and narrow duty cycle of the converged policy indicate that the learned controller is deterministic and interpretable rather than a fragile stochastic artifact.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper compares two agents that differ in both horizon length and discount factor (N_c=4 with gamma=1 versus full trajectory with gamma=0.995), so the causal role of horizon alignment is not isolated; a sweep over (N_c, gamma) would be needed to confirm the mechanism.
  • Because the reward is the sign of the same RS4 sensor used to define the performance metric, the agent is optimizing the metric on which it is later judged; a reward based on a downstream sensor or spanwise-integrated shear stress could change the discovered policy and the comparison with open-loop control.
  • The roughly 1% margin over periodic open-loop control may or may not be statistically robust across the reported autocorrelation-corrected confidence intervals; replicating the deployment with more trajectories would sharpen whether the closed-loop advantage is real.
  • The same truncation principle could be transferred to other flow-control tasks with a known convective or shedding time scale, with N_c set by that physically estimated time rather than by trial and error.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The manuscript reports an experimental implementation of Proximal Policy Optimization (PPO) for active separation control on a one-sided diffuser in a fully turbulent wind tunnel. The policy maps the voltage of a single wall-shear-stress sensor to a Bernoulli distribution over binary valve commands for three in-phase pulse-jet actuators. Two return formulations are compared: a discounted full-trajectory return (Eq. 4, gamma=0.995) and a truncated undiscounted return over four steps (Eq. 5, gamma=1), corresponding to 24 ms, which the authors relate to the convective response time of the flow. The finite-horizon agent is reported to converge in about 20 episodes to a low-duty-cycle policy, while the infinite-horizon agent continues to fluctuate; the deployed finite-horizon controller achieves a forward-flow fraction of about 0.53 at sensor RS4, roughly one percentage point above periodic open-loop control at a 12.5% duty cycle. The authors conclude that a return horizon matched to the flow response time improves the efficiency and stability of online learning.

Significance. If the results hold, the paper is a valuable demonstration that model-free DRL can be trained directly in a fully turbulent experiment with minimal sensing and actuation. The experimental effort is substantial, the setup is realistic, a public code repository is cited (Ref. [45]), and the comparison against open-loop control provides a useful reference point. The central feasibility claim—that PPO can identify a low-duty-cycle separation-control policy from online interactions with a turbulent wind-tunnel flow—is credible and supported by the reported data. The more specific methodological claim about the benefit of a convective-time-aligned return horizon is, however, not yet cleanly established because the two agents differ in both horizon length and discount factor, and the performance metric is the same sensor signal used in the reward. With additional experiments or a more modest framing, the paper could make a solid contribution to experimental DRL-based flow control.

major comments (3)
  1. [II.B, Eqs. (4)-(5), Table 1] The comparison between the two agents varies two quantities at once: the finite-horizon agent uses N_c=4 with gamma=1, whereas the infinite-horizon agent uses the full trajectory with gamma=0.995. With gamma=0.995, a reward 24 ms after an action is still weighted by approximately 0.98, so the practical difference between the two returns is mainly the removal of the long tail rather than a direct contrast between aligned and misaligned horizons. The faster convergence and stability of the finite-horizon agent could therefore be due to truncation, to the removal of discounting, or to an interaction of the two, rather than specifically to aligning the horizon with the convective time. The interpretation offered in Section III.A is plausible but untested; an additional experiment or analysis that changes one factor at a time (for example, gamma=0.995 with N_c=4, or gamma=1 with a full trajectory) is needed to support the causal claim.
  2. [II.B, Eq. (2); III.B, Fig. 7] The reward is the sign of the RS4 sensor signal, and the reported performance metric gamma+ at RS4 is the fraction of positive samples of the same sensor. The DRL controller is therefore trained to maximize exactly the metric against which it is later compared with open-loop control. The roughly 1% advantage over periodic open-loop control in Fig. 7a is thus partly a statement about matching the reward to the benchmark, not an independent validation of closed-loop superiority. The authors should either benchmark against a metric not used in the reward (for example, gamma+ at the downstream sensors or an integrated quantity such as pressure recovery) or explicitly frame the result as an optimization of the RS4 metric rather than a general performance advantage.
  3. [III.A, Fig. 4] The faster-convergence and stability claims are based on a single 500-episode run for each agent, with only two additional 100-episode repetition runs. Given the run-to-run variability typical of online turbulent-flow experiments, this is a thin basis for the quantitative convergence comparison. Reporting the mean and spread of the learning curves across several independent runs, or at least a clear statistical summary of the existing repeats at matching episode counts, would substantially strengthen the claim that the finite-horizon agent converges more reliably.
minor comments (7)
  1. [Table 1] The entry 'Return upper limit' lists 'T' for the infinite-horizon agent and '4 steps' for the finite-horizon agent; since Eq. (4) uses the remaining trajectory length T-t, the label 'T' is misleading and should be rephrased as 'remaining trajectory'.
  2. [Abstract] The phrase 'more robust control strategy' is stronger than what the data support, given the limited number of independent runs; consider 'more stable in the tested runs' or similar.
  3. [III.A, Fig. 6] The statement that 'both agents have 50% of all visited states lying within the same narrow interval around zero' is ambiguous because the intervals are specific to each agent; please report the numerical 50% and 90% intervals for each agent.
  4. [III.B] The claim that consecutive valve closures in the learned policy exceed the characteristic separation time should be supported by the measured distribution of off-intervals in the deployed policy; currently this is a qualitative statement.
  5. [Introduction, Ref. [26]] The sentence reporting '20% skin-friction drag reduction at Re=100' should be checked for consistency with the original reference, as the Reynolds number and flow configuration are not clearly specified.
  6. [Fig. 3] The gray curves labeled 'loss per update' are hard to distinguish in the printed figure; a logarithmic scale or smoothed curves with shaded confidence bands would improve readability.
  7. [Refs. [45]] The GitHub repository is cited without a version or commit identifier; adding a commit hash or release tag would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RL results are direct experimental measurements, and the return-horizon comparison is confounded but not circular.

full rationale

The paper's load-bearing claims are empirical outcomes of an online PPO experiment: the measured forward-flow fraction at RS4 and the observed convergence difference between two return formulations. No parameter is fitted to the reported gamma+ values and then presented as a prediction. The reward in Eq. (2) is the sign of the RS4 wall-shear-stress signal, and gamma+ is the fraction of positive values of that same signal; as the paper itself notes, 'the evolution of the mean gamma+ also roughly translates to the evolution of the cumulative return R.' This is an alignment of training objective and evaluation metric by design, not a hidden fit or a derivation from the target result. The finite-horizon choice N_c=4 and the open-loop duty cycle of 12.5% are taken from the prior experimental study by Steinfurth & Weiss [39], which overlaps with a co-author; however, that study is an externally published, experimentally falsifiable source of flow-response times, so the self-citation is real evidence rather than an unverified load-bearing premise. The main weakness is that the finite-horizon and infinite-horizon agents differ in both horizon and discount factor (Eqs. 4-5 and Table 1: gamma=1 vs 0.995), so the causal attribution to convective-time alignment is confounded; this is a correctness and experimental-design limitation, not circularity. The paper also explicitly limits its claim to local optimization at the feedback sensor, further reducing any risk of overreaching interpretation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central result rests on hyperparameter choices, especially N_c and gamma, and on the assumption that the RS4 sensor provides a sufficient and faithful reward and performance signal. No new physical entities are introduced. The finite-horizon return is a design variant of standard n-step returns, not an invented entity.

free parameters (5)
  • Finite-horizon length N_c = 4 steps (24 ms)
    Set to match the characteristic convective and transient wall-shear-stress response time from prior work; central to the claimed advantage. No neighboring values were tested.
  • Discount factor gamma (infinite-horizon agent) = 0.995
    Chosen as the standard discounted-return design; it differs jointly with the horizon from the finite-horizon agent, confounding the comparison.
  • Discount factor gamma (finite-horizon agent) = 1
    No discounting in the truncated return; this is the second input that changes between the two agents.
  • PPO clip factor epsilon = 0.1
    Author-chosen deviation from the common 0.2, justified for stability; peripheral to the central claim.
  • Entropy regularization factor beta = 0
    Author chose to omit the entropy bonus because environment noise provides exploration; peripheral to the central claim.
assumptions (4)
  • standard math PPO clipped surrogate objective provides a valid policy improvement mechanism
    Assumed from Schulman et al.; not re-derived in this paper.
  • domain assumption The sign of the RS4 voltage is a useful reward proxy for separation-control quality
    The reward r=sgn(s_t+1) is also the raw signal behind the evaluation metric gamma+, so the controller optimizes the same quantity on which it is judged.
  • domain assumption The 24 ms response time from prior work applies to the present configuration
    N_c=4 with dt=6 ms is justified by a characteristic time identified in a closely related prior setup; no in-situ measurement is reported.
  • domain assumption The natural transitory detachment point and sensor placement are stable across training
    The state and reward rely on RS4 being near the detachment point under actuation, as in Fig. 2b; drift over hours is not discussed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep reinforcement learning for separation control in turbulent wind-tunnel flow." pith.science (2026). https://pith.science/paper/BO7YTP5Q

@misc{pith2026260810829,
  author       = {Pith},
  title        = {Pith review of: Deep reinforcement learning for separation control in turbulent wind-tunnel flow},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BO7YTP5Q}},
  note         = {Machine review of arXiv:2608.10829}
}
abstract

This work investigates Deep Reinforcement Learning (DRL) as a tool for model-free closed-loop active separation control in a fully turbulent wind tunnel flow over a one-sided diffuser. The agent controls an array of magnetic valves (on/off) that eject compressed air into the boundary layer, while the environmental state is reduced to the signal from a single wall-shear-stress sensor placed near the natural transitory detachment point. The control law is learned in real time using Proximal Policy Optimization. Compared to the standard learning design based on the weighted sum of all rewards following an action, we demonstrate that a horizon aligned with the convective time of the flow leads to faster convergence and a more robust control strategy. The resulting control law corresponds to a low-duty-cycle actuation pattern that yields a forward-flow fraction of approximately $53\%$. This compares favorably with conventional and optimized periodic open-loop control ($\sim 40\%$ and $\sim 51\%$, respectively). The findings of this article indicate that, when embedded into an online experiment, DRL represents an efficient tool to identify robust and interpretable active separation control strategies.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 34 canonical work pages

  1. [45]

    AFC_with_PPO,

    Avdeeva,S.,“AFC_with_PPO,”https://github.com/dushuytilo/AFC_with_PPO,2026. GitHubrepository,accessed2026-04-26

    work page 2026

  2. [1]

    V.,Boundary Layer and Flow Control: Its Principles and Application, Vol

    Lachmann, G. V.,Boundary Layer and Flow Control: Its Principles and Application, Vol. 1, Pergamon Press, Oxford, 1961

    work page 1961

  3. [2]

    Separation control: Review,

    Gad-el Hak, M., and Bushnell, D. M., “Separation control: Review,”Journal of Fluids Engineering, Vol. 113, No. 1, 1991, pp. 5–30

    work page 1991

  4. [3]

    The Formation and Evolution of Synthetic Jets,

    Smith, B. L., and Glezer, A., “The Formation and Evolution of Synthetic Jets,”Physics of Fluids, Vol. 10, No. 9, 1998, pp. 2281–2297. https://doi.org/10.1063/1.869828

    doi:10.1063/1.869828 1998

  5. [4]

    Synthetic Jets,

    Glezer, A., and Amitay, M., “Synthetic Jets,”Annual Review of Fluid Mechanics, Vol. 34, No. 1, 2002, pp. 503–529. https://doi.org/10.1146/annurev.fluid.34.090501.094913

    work page arXiv 2002

  6. [5]

    Fluidic Oscillators for Flow Control,

    Raghu, S., “Fluidic Oscillators for Flow Control,”Experiments in Fluids, Vol. 54, No. 2, 2013, pp. 1–11. https://doi.org/10. 1007/s00348-012-1455-5

    work page 2013

  7. [6]

    A Review of Fluidic Oscillator Development and Application for Flow Control,

    Gregory, J. W., and Tomac, M. N., “A Review of Fluidic Oscillator Development and Application for Flow Control,”43rd AIAA Fluid Dynamics Conference, 2013. https://doi.org/10.2514/6.2013-2474, aIAA Paper 2013-2474

    work page doi:10.2514/6.2013-2474 2013

  8. [7]

    The Interaction Between a Spatially Oscillating Jet Emitted by a Fluidic Oscillator and a Cross-Flow,

    Ostermann, F., Woszidlo, R., Nayeri, C. N., and Paschereit, C. O., “The Interaction Between a Spatially Oscillating Jet Emitted by a Fluidic Oscillator and a Cross-Flow,”Journal of Fluid Mechanics, Vol. 863, 2019, pp. 215–241

    work page 2019

Show all 49 references
  1. [8]

    Active Separation Control on the Flap of a Two-Dimensional Generic High-Lift Configuration,

    Petz, R., and Nitsche, W., “Active Separation Control on the Flap of a Two-Dimensional Generic High-Lift Configuration,” Journal of Aircraft, Vol. 44, No. 3, 2007, pp. 865–874. https://doi.org/10.2514/1.25425

    2007 doi

  2. [9]

    Pulsed Air-Jet Actuators for Flow Separation Control,

    Warsop, C., Hucker, M., Press, A. J., and Dawson, P., “Pulsed Air-Jet Actuators for Flow Separation Control,”Flow, Turbulence and Combustion, Vol. 78, No. 3, 2007, pp. 255–281. https://doi.org/10.1007/s10494-006-9060-4

    2007 doi

  3. [10]

    Suction and Oscillatory Blowing Actuator Modeling and Validation,

    Arwatz, G., Fono, I., and Seifert, A., “Suction and Oscillatory Blowing Actuator Modeling and Validation,”AIAA Journal, Vol. 46, No. 5, 2008, pp. 1107–1118. https://doi.org/10.2514/1.30468. 15

    2008 doi

  4. [11]

    Bluff Body Drag Manipulation Using Pulsed Jets and Coanda Effect,

    Barros, D., Borée, J., Noack, B. R., Spohn, A., and Ruiz, T., “Bluff Body Drag Manipulation Using Pulsed Jets and Coanda Effect,”Journal of Fluid Mechanics, Vol. 805, 2016, pp. 422–459. https://doi.org/10.1017/jfm.2016.508

    2016 doi

  5. [12]

    Boundary-LayerControlwithAtmosphericPlasmaDischarges,

    Font,G.I.,“Boundary-LayerControlwithAtmosphericPlasmaDischarges,”AIAA Journal,Vol.44,No.7,2006,pp.1572–1578. https://doi.org/10.2514/1.18542

    2006 doi

  6. [13]

    Dielectric Barrier Discharge Plasma Actuators for Flow Control,

    Corke, T. C., Enloe, C. L., and Wilkinson, S. P., “Dielectric Barrier Discharge Plasma Actuators for Flow Control,”Annual Review of Fluid Mechanics, Vol. 42, 2010, pp. 505–529

    2010

  7. [14]

    Experimental Study for Momentum Transfer in a Dielectric Barrier Discharge Plasma Actuator,

    Abe, P., Takizawa, Y., Sato, S., and Kimura, N., “Experimental Study for Momentum Transfer in a Dielectric Barrier Discharge Plasma Actuator,”AIAA Journal, Vol. 46, No. 9, 2008, pp. 2248–2256. https://doi.org/10.2514/1.30985

    2008 doi

  8. [15]

    The Taming of the Shrew: Why Is It so Difficult to Control Turbulence?

    Gad-el Hak, M., “The Taming of the Shrew: Why Is It so Difficult to Control Turbulence?”Active Flow Control, 2006, pp. 1–19. URL http://www.people.vcu.edu/âĹijgadelhak

    2006

  9. [16]

    Fluidic-Oscillator-Based Pulsed Jet Actuators for Flow Separation Control,

    Löffler, S., Ebert, C., and Weiss, J., “Fluidic-Oscillator-Based Pulsed Jet Actuators for Flow Separation Control,”Fluids, Vol. 6, No. 4, 2021. https://doi.org/10.3390/fluids6040166

    2021 doi

  10. [17]

    The control of flow separation by periodic excitation,

    Greenblatt, D., and Wygnanski, I. J., “The control of flow separation by periodic excitation,”Progress in Aerospace Sciences, Vol. 36, No. 7, 2000, pp. 487–545. https://doi.org/10.1016/S0376-0421(00)00008-7

    doi

  11. [18]

    Delay of airfoil stall by periodic excitation,

    Seifert, A., Darabi, A., and Wyganski, I., “Delay of airfoil stall by periodic excitation,”Journal of Aircraft - J AIRCRAFT, Vol. 33, 1996, pp. 691–698. https://doi.org/10.2514/3.47003

    1996 doi

  12. [19]

    DragReductionofaBluffBodyUsingAdaptiveControlMethods,

    Beaudoin,J.-F.,Cadot,O., Aider,J.-L.,andWesfreid, J.E.,“DragReductionofaBluffBodyUsingAdaptiveControlMethods,” Physics of Fluids, Vol. 18, 2006, p. 085107. https://doi.org/10.1063/1.2339023

    2006 doi

  13. [20]

    Robust Multivariable Closed-Loop Control of a Turbulent Backward-Facing Step Flow,

    Henning, L., and King, R., “Robust Multivariable Closed-Loop Control of a Turbulent Backward-Facing Step Flow,”Journal of Aircraft, Vol. 44, No. 3, 2007, pp. 791–800. https://doi.org/10.2514/1.23712

    2007 doi

  14. [21]

    Effect of Base Flow Variation in Noise Amplifiers: The Flat-Plate Boundary Layer,

    Brandt, L., Sipp, D., Pralits, J., and Marquet, O., “Effect of Base Flow Variation in Noise Amplifiers: The Flat-Plate Boundary Layer,”Journal of Fluid Mechanics, Vol. 687, 2011, pp. 503–528. https://doi.org/10.1017/jfm.2011.347

    2011 doi

  15. [22]

    Effects of Pulsed Actuation Upstream a Backward-Facing Step,

    Gautier, N., and Aider, J.-L., “Effects of Pulsed Actuation Upstream a Backward-Facing Step,”Comptes Rendus Mécanique, Vol. 342, No. 6–7, 2013, pp. 382–388. https://doi.org/10.1016/j.crme.2013.03.002, proceedings GDR 2502 Controle des Decollements

    2013 doi

  16. [23]

    Model Reduction for Flow Analysis and Control,

    Rowley, C. W., and Dawson, S. T. M., “Model Reduction for Flow Analysis and Control,”Annual Review of Fluid Mechanics, Vol. 49, 2017, pp. 387–417. https://doi.org/10.1146/annurev-fluid-010816-060042

    2017 doi

  17. [24]

    Closed-Loop Turbulence Control: Progress and Challenges,

    Brunton, S. L., and Noack, B. R., “Closed-Loop Turbulence Control: Progress and Challenges,”Applied Mechanics Reviews, Vol. 67, No. 5, 2015, p. 050801. https://doi.org/10.1115/1.4031175, URL https://doi.org/10.1115/1.4031175. 16

    2015 doi

  18. [25]

    Machine Learning for Fluid Mechanics,

    Brunton, S. L., Noack, B. R., and Koumoutsakos, P., “Machine Learning for Fluid Mechanics,”Annual Review of Fluid Mechanics, Vol. 52, 2020, pp. 477–508. https://doi.org/10.1146/annurev-fluid-010719-060214

    2020 doi

  19. [26]

    Application of Neural Networks to Turbulence Control for Drag Reduction,

    Lee, C., Kim, J., Babcock, D., and Goodman, R., “Application of Neural Networks to Turbulence Control for Drag Reduction,” Physics of Fluids, Vol. 9, 1997, pp. 1740–1747

    1997

  20. [27]

    S., and Barto, A

    Sutton, R. S., and Barto, A. G.,Reinforcement learning: An introduction, 2nd ed., Vol. 1, MIT press, Cambridge, MA, 2018

    2018

  21. [28]

    A Statistical Learning Strategy for Closed-Loop Control of Fluid Flows,

    Guéniat, F., Mathelin, L., and Hussaini, M. Y., “A Statistical Learning Strategy for Closed-Loop Control of Fluid Flows,” Theoretical and Computational Fluid Dynamics,Vol.30,No.6,2016,pp.497–510. https://doi.org/10.1007/s00162-016-0392-y

    2016 doi

  22. [29]

    Artificial Neural Networks Trained through Deep Reinforcement Learning Discover Control Strategies for Active Flow Control,

    Rabault, J., Kuchta, M., Jensen, A., Reglade, U., and Cerardi, N., “Artificial Neural Networks Trained through Deep Reinforcement Learning Discover Control Strategies for Active Flow Control,”arXiv preprint arXiv:1808.07664, 2018. https://doi.org/10.48550/arXiv.1808.07664, URL...

    2018 doi arXiv

  23. [30]

    Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization,

    Rabault1, J., Ren, F., Zhang, W., Tang, H., and Xu, H., “Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization,”Journal of Hydrodynamics, Vol. 32, No. 2, 2020, pp. 234–246

    2020

  24. [31]

    Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning,

    Tang, H., Rabault, J., Kuhnle, A., Wang, Y., and Wang, T., “Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning,”Physics of Fluids, Vol. 32, No. 5, 2020, p. 053605. https://doi.org/10.1063/5...

    2020 doi

  25. [32]

    Applying Deep Reinforcement Learning to Active Flow Control in Turbulent Conditions,

    Ren, F., Rabault, J., and Tang, H., “Applying Deep Reinforcement Learning to Active Flow Control in Turbulent Conditions,” Physics of Fluids, Vol. 33, No. 3, 2021, p. 037121. https://doi.org/10.1063/5.0037371

    2021 doi

  26. [33]

    Reinforcement Learning for Bluff Body Active Flow Control in Experiments and Simulations,

    Fan, D., Yang, L., Wang, Z., Triantafyllou, M. S., and Karniadakis, G. E., “Reinforcement Learning for Bluff Body Active Flow Control in Experiments and Simulations,”Proceedings of the National Academy of Sciences, Vol. 117, No. 42, 2020, pp. 26091–26098. https://doi.org/10.10...

    2020 doi

  27. [34]

    Experimental Study on Application of Distributed Deep Reinforcement Learning to Closed-loop Flow Separation Control over an Airfoil,

    Shimomura, S., Sekimoto, S., Oyama, A., Fujii, K., and Nishida, H., “Experimental Study on Application of Distributed Deep Reinforcement Learning to Closed-loop Flow Separation Control over an Airfoil,”AIAA Scitech 2020 Forum, 2020. https://doi.org/10.2514/6.2020-0579, aIAA Pa...

    2020 doi

  28. [35]

    Transformer-based in-context Policy Learning for Efficient Active Flow Control Across Various Airfoils,

    Zheng, C., Xie, F., Ji, T., Zhou, H., and Zheng, Y., “Transformer-based in-context Policy Learning for Efficient Active Flow Control Across Various Airfoils,”Journal of Fluid Mechanics, Vol. 1001, 2024, p. A53

    2024

  29. [36]

    Deep reinforcement learning for active flow control in a turbulent separation bubble,

    Font, B., Alcántara-Ávila, F., Rabault, J., Vinuesa, R., and Lehmkuhl, O., “Deep reinforcement learning for active flow control in a turbulent separation bubble,”Natural Communications, Vol. 16, 2025, 1422

    2025

  30. [37]

    Experimental deep reinforcement learning control of a turbulent boundary layer with plasma actuators for skin-friction drag reduction,

    Fang, Z., Zong, H., Wu, Y., Li, J., Su, Z., and Wei, B., “Experimental deep reinforcement learning control of a turbulent boundary layer with plasma actuators for skin-friction drag reduction,”Journal of Fluid Mechanics, Vol. 1027, 2026. https://doi.org/10.1017/jfm.2025.11086. 17

    2026

  31. [38]

    Surrogate-Based Exploration of Active Separation Control Parameters: An Experimental Study,

    Löffler, S., Steinfurth, B., and Weiss, J., “Surrogate-Based Exploration of Active Separation Control Parameters: An Experimental Study,” 2023. https://doi.org/10.2514/6.2023-0076

    2023 doi

  32. [39]

    Efficiency Enhancement in Active Separation Control Through Optimizing the Duty Cycle of Pulsed Jets,

    Steinfurth, B., and Weiss, J., “Efficiency Enhancement in Active Separation Control Through Optimizing the Duty Cycle of Pulsed Jets,”AIAA Journal, 2022. https://doi.org/10.2514/1.J061667

    2022 doi

  33. [40]

    Boundary-layer control by means of pulsed jets at different inclination angles,

    Steinfurth, B., and Weiss, J., “Boundary-layer control by means of pulsed jets at different inclination angles,”AIAA Journal, Vol. 59, No. 8, 2021. https://doi.org/10.2514/1.J060474

    2021 doi

  34. [41]

    Vortex rings produced by non-parallel planar starting jets,

    Steinfurth, B., and Weiss, J., “Vortex rings produced by non-parallel planar starting jets,”Journal of Fluid Mechanics, Vol. 903, 2020, p. A16. https://doi.org/10.1017/jfm.2020.637

    2020 doi

  35. [42]

    Velocity ratio effect on flow structures of non-parallel planar starting jets in cross-flow,

    Steinfurth, B., and Weiss, J., “Velocity ratio effect on flow structures of non-parallel planar starting jets in cross-flow,”Journal of Fluid Mechanics, Vol. 915, 2021, p. A11. https://doi.org/10.1017/jfm.2021.40

    2021 doi

  36. [43]

    Simulation and Testing of a MEMS Calorimetric Shear-Stress Sensor,

    Weiss, J., Schwaab, Q., Boucetta, Y., Giani, A., Guigue, C., Combette, P., and Charlot, B., “Simulation and Testing of a MEMS Calorimetric Shear-Stress Sensor,”Sensors and Actuators A: Physical, Vol. 253, 2017, pp. 210–217. https://doi.org/10.1016/j.sna.2016.11.018

    2017 doi

  37. [44]

    Proximal Policy Optimization Algorithms,

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O., “Proximal Policy Optimization Algorithms,”ArXiv, Vol. abs/1707.06347, 2017. URL https://api.semanticscholar.org/CorpusID:28695052

    2017 arXiv

  38. [46]

    Actor-Critic Algorithms,

    Konda, V., and Tsitsiklis, J., “Actor-Critic Algorithms,”Advances in Neural Information Processing Systems, Vol. 12, edited by S. Solla, T. Leen, and K. Müller, MIT Press, 1999. URL https://proceedings.neurips.cc/paper_files/paper/1999/file/ 6449f44a102fde848669bdd9eb6b76fa-Paper.pdf

    1999

  39. [47]

    Deep reinforcement learning for turbulent drag reduction in channel flows,

    Guastoni, L., Rabault, J., Schlatter, P., Azizpour, H., and Vinuesa, R., “Deep reinforcement learning for turbulent drag reduction in channel flows,”The European Physical Journal, Vol. 46, 2023, 27. https://doi.org/10.1140/epje/s10189-023-00285-8

    2023 doi

  40. [48]

    OptimizingPulsedBlowingParametersforActiveSeparationControlina One-SidedDiffuserUsingReinforcementLearning,

    Müller,A.,Schesny,T.,Steinfurth,B.,andWeiss,J.,“OptimizingPulsedBlowingParametersforActiveSeparationControlina One-SidedDiffuserUsingReinforcementLearning,”New Results in Numerical and Experimental Fluid Mechanics XV,Noteson NumericalFluidMechanicsandMultidisciplinaryDesign,Vo...

    2026 doi

  41. [49]

    Proximal Policy Optimization,

    OpenAI, “Proximal Policy Optimization,” https://spinningup.openai.com/en/latest/algorithms/ppo.html, 2018. Accessed 14 April 2026. 18

    2018

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.