{"id":"e119ef78-ca0e-4b64-938d-e2512b13e3cf","arxiv_id":"2507.04309","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A supervised feature-transfer map lets a full-state RL drag-reduction policy be reused with only partial pressure measurements, recovering near-optimal performance in 2D DNS.","lead":"This paper trains a neural network that reconstructs full flow measurements from a history of partial sensor readings, then combines it with an already-trained optimal drag-reduction policy. The result is a partial-measurement policy that nearly matches the full-measurement optimum in simulation and outperforms directly trained partial-observation RL.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The optimality claim for the composed policy in Eq. (2) is not supported unless the DSFT reconstruction error is shown to be uniformly small over the deployment distribution, which the paper does not establish.","rationale":"The reader identified exactly this: the reconstructability and coverage premise is unproven. The paper's central claim is conditional on Tθ being accurate on the deployment distribution, which requires (a) the partial history to be sufficient for reconstruction, and (b) the training data D to cover the relevant states. Neither is demonstrated. The paper acknowledges a primary limitation about full measurements at training time, but does not address the reconstruction error distribution or data collection protocol. This is the load-bearing concern because Eq. (2) is only optimal if the composition is exact; any nonzero reconstruction error can, in principle, degrade performance arbitrarily depending on the sensitivity of π*. The experiments show the method works in one setting, which is supportive but not sufficient to establish the general claim. Hence the verdict remains CONDITIONAL.","tokens_in":5807,"tokens_out":1311,"duration_ms":13241,"concrete_test":"Re-run the PDA pipeline but explicitly collect D under the composed policy itself (or a mixture that provably covers the composed policy's state distribution), then compute and report the reconstruction error e(t) = ||Tθ(o_PM_{t-n:t}, a_{t-m:t-1}) - o_FM_t|| on these deployment trajectories. If the mean or worst-case reconstruction error is not small relative to the scale of o_FM (e.g., not within the observed sensitivity of π* to input perturbations), the composed policy's optimality claim fails. Alternatively, train Tθ on D collected under a random/exploratory policy and evaluate reconstruction error on a rollout of the composed policy; if error is large or drag degrades, the coverage assumption is violated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that composing the trained DSFT map with the full-state optimal policy yields the optimal partial-measurement policy. This holds only if Tθ is an exact reconstruction on the support of the deployment state distribution. The paper trains Tθ by minimizing mean squared error on a dataset D (Eq. 1) but never reports the reconstruction error's distribution, worst-case magnitude, or location. Critically, the training set D is collected under some behavior policy, and the paper never specifies that policy or argues that D covers the states visited by the composed policy (reader's concern). Moreover, the policy π* is an approximation (no hidden layers), and Fig. 3 shows it is more robust but still an approximation; the composition in Eq. (2) compounds approximation errors. Without a quantitative bound or validation of reconstruction error on trajectories under the composed policy, the optimality claim is an assumption, not an established result. The paper's own experiments are suggestive but only show one configuration; no sensitivity to n and m for reconstruction error, and no evidence of closed-loop stability beyond 100 time units. The optimality claim in Section 1 is therefore stronger than the evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Policy Domain Adaptation (PDA) for transferring a full-state feedback control policy to a setting with only partial measurements. A Domain-Specific Feature Transfer (DSFT) map T_theta is trained by supervised regression to reconstruct full wake pressure measurements from a history of base pressure measurements and past actions, and the resulting policy is defined as the composition of T_theta with a pre-trained full-state optimal policy (Eq. 2). The method is demonstrated on a two-dimensional DNS of flow past a square bluff body at Re=100, where it reportedly achieves drag close to the full-state optimal policy and outperforms a directly trained partial-observation RL policy. The paper also uses the approach to study the influence of measurement and action history lengths and draws conclusions about the needed policy architecture.","tokens_in":6093,"tokens_out":4558,"duration_ms":54773,"significance":"If the central optimality claim were established, the paper would offer a practically useful recipe for adapting simulation-trained RL flow-control policies to realistic sensor configurations without retraining, together with guidance on minimal history length. The idea is simple and clearly stated, and the empirical comparison uses 65 runs, which is a reasonable effort for a DNS-based study. However, the paper provides no proof or quantitative validation of the reconstruction accuracy needed for the composition to be optimal, does not describe how the dataset D is collected, and does not report statistical uncertainty beyond min-max shading. The strength of the paper lies in the demonstrated empirical trend; the weakness is that the main claim is considerably stronger than the evidence.","major_comments":[{"comment":"The statement that composing the trained DSFT map with the full-state optimal policy 'we obtain the optimal policy for the partial measurement case' is not supported by the provided evidence. The composition is optimal only if T_theta reconstructs o_FM_t accurately on the support of the states visited when the composed policy is deployed. The paper minimizes MSE on dataset D in Eq. (1) but never reports the reconstruction error or its distribution, never describes how D is collected (under which behavior policy, over which time horizon, and from how many trajectories), and never validates that D covers the deployment distribution. I request that the data-collection protocol be specified, reconstruction error be reported on training and held-out data, and closed-loop performance under the composed policy be validated. Otherwise the claim should be weakened to 'empirically near-optimal' rather than 'optimal'.","section":"Section 1, bullet 1 and Eq. (2)"},{"comment":"The claimed optimal history length n=m=48 is selected from the same performance sweep that is then reported as the finding. Because n and m are hyperparameters chosen from the curve, the statement that the method 'enables determination of the optimal history length' overstates the evidence. A separate holdout validation, cross-validation, or a plateau analysis is needed to distinguish a genuine optimum from overfitting to the evaluation set. The paper should also report whether the reconstruction error of T_theta depends monotonically on n and m, since that is the mechanism presumed to explain the policy performance.","section":"Section 4, Fig. 2b"},{"comment":"There is an inconsistency in the action-history index ranges: Eq. (1) uses a_{t-m:t} in the input to T_theta, while Eq. (2) uses a_{t-m:t-1}. If the current action a_t is part of the reconstruction input, the deployment-time availability of a_t must be discussed, because the composed policy in Eq. (2) cannot know a_t before computing it. In addition, Eq. (2) carries the trajectory index i in the policy expression, which is a formatting error that should be removed. Please correct these and ensure all index ranges are consistent throughout.","section":"Section 3, Eqs. (1) and (2)"},{"comment":"The claim that the composed policy outperforms the directly trained partial-observation RL policy is based only on min-max shading over 65 runs, with no standard deviations, confidence intervals, or significance tests reported. Moreover, the 'optimal' full-state policy pi* is itself only empirically optimal with respect to a specific TQC training run, a particular architecture (no hidden layers), and fixed hyperparameters. I request that the paper quantify uncertainty (for example, with confidence intervals or paired statistical tests) and clearly state that 'optimal' means 'optimal within the studied policy class and training setup' rather than a proven global optimum.","section":"Section 4, Fig. 2a and Appendix A"}],"minor_comments":[{"comment":"The color descriptions in the text are inconsistent: the text refers to 'blue and green lines' for the PDA approach, then to an 'orange line' for the no-action-history policy, while the green line was earlier identified as the full-state optimal policy. Please align the legend, figure colors, and the textual references.","section":"Figure 2 and surrounding text"},{"comment":"The phrase 'tree type of neural networks' should read 'three types of neural networks'.","section":"Appendix B"},{"comment":"The paper does not state whether code, trained models, or the dataset D will be made available, which limits reproducibility; please add a data and code availability statement.","section":"General"},{"comment":"The time discretization is not specified: please state the length of the action interval in simulation time units and the episode length over which returns are computed.","section":"Section 2"},{"comment":"The abstract refers to 'a simplified road vehicle,' but the environment is a two-dimensional square bluff body; 'simplified two-dimensional bluff body' would be more precise and avoid overgeneralization.","section":"Abstract and Section 1"},{"comment":"The statement that 'all policies converge to a reduced drag state' is supported only by the last-100-steps average; please specify the convergence criterion used or show the full time series for representative runs.","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads as a workshop paper with a promising empirical idea but with a central claim ('optimal policy') that goes beyond what is demonstrated. The missing dataset description and reconstruction-error validation are the main archival-blocking issues; both are fixable without changing the method. I would advise the editor that the paper should be reframed as an empirical study of a transfer heuristic unless the authors can provide the requested validation and uncertainty quantification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a useful empirical adaptation—train a supervised reconstructor from base-pressure history to full wake sensors, then compose it with the full-state RL policy—and the experiments honestly show it beats directly-trained partial-observation RL in the 2D laminar case. The genuinely new findings are that action history is unnecessary and a single-hidden-layer reconstructor is enough; both are concrete and worth knowing. Credit also for including architecture sweeps and a direct baseline.\n\nWhere it gets shaky is the word 'optimal.' Eq. (2) is a composition of two approximations. It is optimal only if Tθ is an accurate reconstruction on the states the composed policy actually visits. The paper never shows that: dataset D (how collected, under what behavior policy, coverage) is unspecified; reconstruction error is never reported in distribution or worst case; the results are min-max shading over 65 runs with no significance tests; and n=m=48 is chosen from the same performance sweep that 'finds' it. So the central claim in Section 1 is stronger than the evidence. The scope is also narrow—one geometry, Re=100, 2D DNS, ~100 time units—so the introduction's gesture toward 3D real-world deployment is hand-waving.\n\nI don't think this is a fatal flaw. The mechanism is plausible, the baseline comparison is honest, and the action-history finding is a genuine counterpoint to the delayed-MDP framing in Xia et al. (2024). The paper deserves peer review; a serious referee should ask for the dataset protocol, reconstruction error curves along deployed trajectories, and a robustness or significance check on the main comparisons. For a workshop paper it is fine as is; for a journal those additions would be required.\n\nRecommendation: send it out, with expectation of revision. The optimality language needs softening regardless.","headline":"Useful empirical adaptation of full-state RL drag control to partial base-pressure measurements; the 'optimal' claim is not yet established, but the action-history finding is a solid contribution.","tokens_in":6557,"tokens_out":2061,"would_cite":true,"duration_ms":24324,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A learned reconstruction map lets a full-state drag-reduction policy run on base-pressure sensors alone, matching the wake-measurement optimum in simulation.","keywords":["drag reduction","partial measurements","domain adaptation","reinforcement learning","active flow control","DSFT","POMDP","bluff body"],"falsifier":"Run the composed PDA policy on the same bluff-body flow but with a Reynolds number or actuator authority outside the training dataset; if the reconstruction error on those histories is large and drag degrades toward the uncontrolled value, the claim that the composition is optimal for the partial-measurement problem fails. A simpler check: compare the composed policy against an oracle that receives the true full measurement at every step; the optimality claim predicts the two drag values should coincide.","tokens_in":5628,"feed_emoji":"🚗","tokens_out":5400,"duration_ms":52995,"temperature":0.7,"pith_summary":"The paper tries to establish that an optimal feedback controller trained on dense wake measurements can be ported to a vehicle that only has pressure sensors on its base. The method trains a Domain-Specific Feature Transfer map that reconstructs the current full wake measurement from a history of partial measurements and past actions, then composes that map with the already-optimal full-state policy. In a two-dimensional direct numerical simulation of flow past a square bluff body, the composed policy reaches drag close to the full-state optimum and beats a policy trained directly on partial measurements. If this holds, real vehicles could inherit simulation-trained drag-reduction policies using only onboard sensors.","feed_headline":"Base sensors alone reach full-wake drag control","feed_subtitle":"A learned reconstruction map lets the optimal wake-measurement policy run on onboard pressure sensors.","key_machinery":"The load-bearing object is the Domain-Specific Feature Transfer (DSFT) map $T_\\theta$, a supervised neural network trained by equation (1) to map the history of base-pressure measurements and past actions to the current wake measurement vector. Its job is to make the partial observation look like the full observation to the existing optimal policy. The second ingredient is the choice of a deliberately simple optimal full-state policy, an affine function of the measurements with no hidden layers, which the paper finds to be more robust to reconstruction errors than a deep policy.","core_discovery":"The central claim is that for the partial-observation drag-reduction problem, an optimal policy is obtained without any new reinforcement-learning training: train a map $T_\\theta$ on stored trajectories so that $T_\\theta(o^\\mathrm{PM}_{t-n:t}, a_{t-m:t-1}) \\approx o^\\mathrm{FM}_t$, then define the deployed policy as $\\pi^*(T_\\theta(\\cdot), a_{t-1})$, the composition of the full-state optimal policy with the reconstructed measurement. In the simulated bluff-body flow at Reynolds number 100, this composed policy matches the drag of the full-measurement optimal policy, exceeds the drag reduction of a policy trained directly on partial measurements, and needs only one hidden layer in the reconstruction network.","pith_inferences":["If the reconstruction premise generalises, the same composition strategy could be applied to other actuator-sensor pairs in fluid control, such as transferring policies between differently instrumented wind-tunnel models.","The optimality result is shown at one Reynolds number and in two dimensions; a natural test is whether the same map length and architecture survive three-dimensional or turbulent wakes, where the full measurement is higher-dimensional.","The DSFT map is trained on a fixed dataset; online adaptation of $T_\\theta$ with minimal labels during deployment would be a direct extension if the flow conditions drift."],"forward_implications":["Partial-measurement drag control can reach full-state optimality without retraining, as long as a reconstruction map can be trained from stored trajectories.","The optimal history length $n$ can be selected offline by evaluating the composed policy, and the paper finds that longer base-pressure histories improve performance.","Action history barely affects performance, suggesting the controlled flow can be modelled without an explicit delay in the MDP.","A single hidden layer in the reconstruction map suffices; deeper maps add no gain and can overfit, while linear maps are insufficient."],"supporting_citations":[{"why":"Supplies the Domain-Specific Feature Transfer method that the reconstruction map is based on.","marker":"Wei et al. [2018]"},{"why":"Defines the partial-measurement drag-reduction task and provides the directly trained partial-observation baseline policy and TQC hyperparameters.","marker":"Xia et al. [2024]"},{"why":"Provides the bluff-body flow environment used for all simulations.","marker":"Rabault et al. [2019]"},{"why":"The TQC algorithm used to obtain the optimal full-state policy $\\pi^*$.","marker":"Kuznetsov et al. [2020]"},{"why":"Demonstrates that optimal performance is achievable within the policy class using wake measurements, which the full-state baseline relies on.","marker":"Chen et al. [2023]"},{"why":"Gives the POMDP formulation used to frame the partial-measurement control problem.","marker":"Bertsekas [2012]"}],"fun_headline_variants":["Reconstruct wake, keep optimal drag control","Partial sensors, full drag reduction via learned map","One hidden layer maps partial data to optimal drag","Optimal drag policy from onboard sensors, no retraining","Wake reconstruction unlocks full-state drag control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that a finite history of base-pressure measurements and past actions contains enough information to reconstruct the current full wake measurement, and that the dataset used to train that reconstruction covers the states the composed policy visits.","fun_headline_variants_meta":{"raw":{"variants":["Reconstruct wake, keep optimal drag control","Partial sensors, full drag reduction via learned map","One hidden layer maps partial data to optimal drag","Optimal drag policy from onboard sensors, no retraining","Wake reconstruction unlocks full-state drag control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000159,"raw_usage":{"total_tokens":1199,"prompt_tokens":885,"completion_tokens":314,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":244}},"tokens_in":501,"tokens_out":314,"duration_ms":3681,"temperature":1.0,"reasoning_tokens":244,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:50:50.270622+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the composed PDA policy on the same bluff-body flow but with a Reynolds number or actuator authority outside the training dataset; if the reconstruction error on those histories is large and drag degrades toward the uncontrolled value, the claim that the composition is optimal for the partial-measurement problem fails. A simpler check: compare the composed policy against an oracle that receives the true full measurement at every step; the optimality claim predicts the two drag values should coincide.","supporting_citations":[],"review_version":1}