{"id":"ea556100-c09d-44f8-b93a-00090a126be4","arxiv_id":"2607.15910","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A DDPG agent tuned PID coefficients on a physical free-space-optics deflector over a 200 km UDP link, reducing radial tracking error range by 31% on a pseudo-random trajectory while leaving static-position performance unchanged.","lead":"Researchers used a reinforcement-learning agent to tune six PID control gains for an optical fiber positioning device from 200 km away, cutting dynamic tracking error range by 31% in one test regime. The improvement did not hold for a fixed target, so the tuning is regime-specific rather than universal.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"31% reduction is asserted from one evaluation of best-of-5000 coefficient sets on a trajectory from the same generator; selection bias and lack of repetition are not ruled out.","rationale":"The reader correctly identified the same trajectory family and the lack of validation on a different signal class as the weakest assumption. I partially agree, but I would sharpen the concern: even within the same trajectory family, the protocol is vulnerable to selection bias because the coefficient sets are chosen as the best of 5000 noisy training episodes and then evaluated on a single test trajectory. This makes the numerical improvement less trustworthy than the paper presents. The paper has genuine strengths: it uses a real physical optical deflector, reports the static-regime failure explicitly, and the DDPG-PID framework is a reasonable hybrid approach. However, the headline 31% reduction is the quantitative core of the central claim, and it is not backed by repeated independent measurements or a proper validation split. The appropriate verdict remains CONDITIONAL: the result is plausible but not yet established at the level of a quantitative claim. No change to the reader's verdict is needed, though the rationale for conditionality could be sharpened toward selection bias and single-test evaluation rather than only trajectory-family matching.","tokens_in":8047,"tokens_out":4551,"duration_ms":56838,"concrete_test":"Run a confirmatory protocol: generate 50 fresh independent PRS target sequences from the same generator (new random seeds) and, separately, 50 sequences from a different trajectory family (e.g., band-limited noise with a different cutoff or a recorded vibration profile). For each sequence, measure D_R and σ_R for a_baseline, a_opt1, and a_opt2 in randomized order under identical UDP conditions. Report per-set means, paired differences, and 95% bootstrap confidence intervals. If the 31% reduction is not reproduced (e.g., CI includes zero or the effect is <10%), the central claim weakens. Additionally, perform a proper selection-by-validation split: choose coefficients on a training subset and evaluate on held-out sessions to quantify selection bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III.B: the agent runs 5000 communication sessions, each with a fresh 362-point pseudo-random target sequence, and the two coefficient sets are selected as the actions with maximum episodic reward from the saved history (Table II). Because the reward is noisy (UDP failures, sensor noise, random target generation), the best of 5000 is expected to look better than its true mean—this is best-of-N selection bias. The reported comparison (Table IV) then evaluates these selected sets on a single 65535-point PRS generated by Eq. (3) using the same OD-matched low-pass filter. Thus the observed change in D_R (119→82) and σ_R (15→12) is one draw, with no confidence intervals, no paired replication, and no held-out trajectory family. If the test PRS is independent of training, the improvement is subject to regression to the mean; if it is not independent, the improvement is partly an artifact of optimizing and evaluating on the same signal class. The 'variance of D_R' column in Table IV implies repeated tests, but the number of repetitions is never stated, so no statistical significance can be assessed. The central claim—that the 31% reduction demonstrates RL-assisted PID tuning—requires the selected gains to be systematically better, which the current protocol does not establish.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a DDPG agent that tunes six coefficients of cascaded position and velocity PID controllers for an optical deflector in a free-space optical positioning stand. Training is performed over 5000 remote UDP communication sessions against the physical plant. Two coefficient sets are selected from the recorded training history as the highest-reward actions. These sets, along with a manually tuned baseline, are evaluated on two target regimes: a constant {0,0} target and a 12-kHz pseudo-random trajectory matched to the OD frequency response (Eq. 3). The best RL-tuned set yields a 31% reduction in radial error range (119 to 82) and a standard-deviation reduction (15 to 12) for the PRS, while no improvement over baseline is observed for the static target. The paper concludes that DDPG-assisted PID tuning is feasible for the tested dynamic regime but does not generalize universally.","tokens_in":8403,"tokens_out":4184,"duration_ms":48915,"significance":"The experimental setup is unusual and valuable: the RL agent and the physical plant are separated by 200 km with UDP transport, so the training includes real network delays and failures. The authors are honest about the null result for a static target and explicitly state that the coefficients are not optimal for all operating conditions. Strengths include a real-hardware test rather than simulation, a clear description of the stand and protocol, and an explicit call for repeatability experiments. However, the central quantitative claim (31% improvement) rests on a best-of-5000 selection and a single evaluation on the same trajectory family used in training. The missing statistical replication and the lack of a held-out trajectory class prevent the paper from establishing that the measured improvement is systematic rather than a selection artifact.","major_comments":[{"comment":"The two coefficient sets are chosen as the maximum-reward actions from 5000 training sessions, and each session uses a fresh PRS from Eq. (3) with noisy reward (UDP failures, sensor noise, random target generation). The reported D_R reduction (119→82) is from a single 65535-point PRS test. With no repeated independent evaluations, this is exactly the kind of best-of-N selection bias that would make a noisy sample look better than its true mean. Please report the number of repetitions behind the 'Variance of D_R' column, provide confidence intervals, and include a paired statistical comparison against the baseline.","section":"Section III.B, Table II, Table IV"},{"comment":"The PRS used for training and the PRS used for testing are generated by the same stochastic process: a uniform polar random vector followed by a low-pass filter whose frequency response is 'identical to the frequency response of the OD.' Thus the selected gains are optimized for the same signal class on which they are evaluated. To support a general dynamic-regime claim, test on a distinct trajectory family (e.g., chirp, multi-sine, or recorded operational disturbance) or at least on independent realizations from a different distribution. The static-target test is a useful contrast but is only one alternative regime.","section":"Section III.B, Eq. (3), Table IV"},{"comment":"Eq. (2) defines the reward as the sum of Euclidean distances, but the Fig. 3 caption states that a 'self-preserving penalty' is applied when the error exceeds the baseline PID error, and that a penalty is also applied for connection failures. This penalty is not in Eq. (2), so the reward actually used for action selection is not the quantity defined in the text. Since the coefficient sets are selected by maximum reward, the paper must specify the full reward function, including the penalty and its scale.","section":"Section II, Eq. (2), Fig. 3 caption"},{"comment":"The units of σ and D are not stated (presumably micrometers or device-specific units). More importantly, 'Variance of D_R' implies replicate measurements, but the number of repetitions is never given. Without this information, the statement that a_opt,1 is 'more optimal' than baseline because its D_R variance is halved is not statistically testable. Report the number of repeated runs, the distribution of D_R, and an appropriate significance test.","section":"Section IV, Table IV"}],"minor_comments":[{"comment":"Grammar and typos: 'authors was investigated' (Section I), 'is chose' (Section II), 'position positions' (Section III.A). A careful proofread is needed.","section":"Throughout"},{"comment":"The notation Q(a_t, s_t) is nonstandard; DDPG typically uses Q(s, a). Also, showing a max over a_t' ignores the target-network and deterministic-policy details; please clarify the notation.","section":"Section II, Eq. (1)"},{"comment":"The formula y_target(t_i) = U(t_i) + P(t_i) is presented with 'U(x) is a step periodic function' and 'P(x) is a pseudo-random sequence'. For the static target, the text says the target is {0,0}, which is not a special case of this formula; clarify how the PRS and the constant target relate to Eq. (3).","section":"Section III.B, Eq. (3)"},{"comment":"The vertical axis label '-101' appears to be a typo; also, the log-scale normalization should be defined explicitly.","section":"Figure 3"},{"comment":"Several references are incomplete (e.g., [10] lacks venue/pages, [13] is an arXiv preprint without arXiv number, [1] is a Russian-language proceedings paper). Please complete the bibliographic information.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is an engineering report with an interesting distributed-hardware experimental setup. The statistical weaknesses (best-of-N selection, single evaluation, same trajectory family for training and testing) are load-bearing for the claimed 31% improvement. I believe they are fixable: the authors could rerun the evaluation with several independent trajectories from different families, report repetition counts and confidence intervals, and clearly separate the selection and evaluation phases. If the authors cannot provide such evidence, the paper's central claim would remain unsubstantiated. Also, the reward function needs clarification regarding the self-preserving penalty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a real experiment: DDPG tunes six cascaded PID coefficients on a physical optical deflector over a 200 km UDP link, with 12 kHz target updates. That combination is new, and the authors deserve credit for building it. Second, the headline 31% reduction in radial error range is not as solid as it reads. The two coefficient sets were selected as the best-of-5000 training sessions, and the evaluation ran once on a trajectory generated by the same filter used in training. No confidence intervals, no replication count, no held-out trajectory class. So the quantitative claim is a demonstration, not a measurement.\n\nWhat the paper does well: the setup is genuinely physical, not simulated. They report the static-target failure openly: the RL sets do not improve D_R there, and Table IV even shows one set increases variance. That honesty is real. They also keep the claim modest — feasibility of DDPG-assisted PID tuning — and explicitly say the results don't generalize across regimes. The DDPG-PID hybrid is not new (they cite the prior work), and the novelty is the application, which is legitimate.\n\nWhere it's soft: the stress-test concern about best-of-N selection bias lands. With noisy rewards over 5000 sessions, the maximum is expected to overstate the true performance of that coefficient set. The test uses a single 65535-point PRS from the same Eq. (3) generator, so it's not an independent check. Table IV's 'Variance of D_R' column implies repeated tests, but the number of repetitions is never stated. Code and data are 'available upon request,' which in practice means unavailable. These are fixable: run the selected sets on several independent trajectories from a different family, report means and standard deviations across repeats, and give the actual repetition count.\n\nThe paper is worth refereeing because the distributed physical setup is unusual and the negative result is informative. But the quantitative claim should be treated with caution. I'd ask for clarity on repetitions, a held-out trajectory test, and ideally more transparent data before accepting the 31% figure.\n\nFor peer review: send it, but expect heavy revision on the statistics.","headline":"A real distributed RL-PID tuning experiment on a physical deflector, with an honest negative result — but the 31% improvement is a single best-of-5000 draw with no stats, so judge it as a feasibility demo.","tokens_in":8948,"tokens_out":2402,"would_cite":false,"duration_ms":27505,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that a DDPG reinforcement-learning agent, acting over a 200-km UDP link, can tune six PID coefficients of a physical optical deflector so that radial tracking-error range falls from 119 to 82 (31%) on a dynamic pseudo-random","keywords":["free-space optical communication","PID control","reinforcement learning","DDPG","optical deflector","beam alignment","distributed control","positioning error"],"falsifier":"Run the selected coefficient set aopt,1 on a trajectory family that differs from the training PRS (for example, a chirp, a recorded platform-vibration sequence, or band-limited noise with a different cutoff), and compare radial error range against the baseline. If the range reduction disappears, reverses, or is not reproduced across repeated trials, the reported 31% gain is specific to the generator rather than a general property of the tuned gains.","tokens_in":7922,"feed_emoji":"🎯","tokens_out":4185,"duration_ms":40465,"temperature":0.7,"pith_summary":"This paper tries to establish that reinforcement learning can be used as an experimental, data-driven tuning layer for conventional PID controllers in a real free-space optical positioning system, rather than as a replacement for the controller. A DDPG agent interacts with a physical optical deflector that moves the tip of an optical fiber in the focal plane; agent and device are 200 km apart and exchange data through UDP packets, so the learning process includes genuine communication delays and losses. After 5000 sessions the agent found two fixed six-coefficient sets. In a dynamic pseudo-random trajectory test, the better set reduced the radial positioning-error range from 119 to 82 (31%) and its standard deviation from 15 to 12; in a constant-target test, neither RL-tuned set improved on manual tuning. The claim is therefore not universal superiority, but feasibility and a measurable gain in one important operating regime.","feed_headline":"RL-tuned PID cuts optical tracking error by 31%","feed_subtitle":"A DDPG agent tuned six gains on a physical deflector over a 200-km UDP link, beating manual tuning on dynamic targets.","key_machinery":"The load-bearing mechanism is a Deep Deterministic Policy Gradient (DDPG) agent operating in a closed loop over a distributed network. Its action is a six-dimensional vector of PID coefficients (Pp, Ip, Dp for the position loop, Pv, Iv, Dv for the velocity loop), transmitted through UDP to the optical deflector. Its state is a sequence of 362 measured and target coordinate pairs, its reward is the sum of radial distances between position and target over those 362 samples, and epsilon-greedy noise provides exploration. This arrangement lets the agent perform black-box, policy-guided search over controller gains directly on hardware, including the device's nonlinearities, sensor noise, and net","core_discovery":"On the paper's own terms, the central discovery is that DDPG-assisted PID tuning is feasible on a physical stand: the RL agent adjusted the six gains of cascaded position and velocity PID controllers over thousands of real communication sessions, and the selected coefficients aopt,1 improved dynamic tracking (radial range 82 vs 119, sigma 12 vs 15) compared with the manually tuned baseline. For the stationary zero-target regime, the RL-tuned sets did not improve radial error range. The authors interpret this as evidence for the potential of RL-based experimental PID tuning in dynamic tasks and as a warning that the training trajectory and reward must match the intended operating regime.","pith_inferences":["The improvement is only shown for the same signal class used in training; the paper never tests a different trajectory family, so the 31% figure may partly reflect specialization to the PRS generator rather than a robust improvement.","A plausible next experiment is to train on a reward that mixes dynamic and stationary regimes, which the paper itself suggests; this would likely recover zero-target performance while retaining dynamic gains.","Because the agent only updates coefficients between sessions, the approach is episodic black-box optimization; interpreting DDPG as online is a stretch—this is closer to guided search over a low-dimensional parameter space, and simpler baselines such as Bayesian optimization could provide a stronger benchmark.","The remote 200-km setup implies that long-latency, lossy channels do not prevent data-driven tuning; this could extend to tuning controllers on platforms that are hard to access physically, like telescopes or orbital systems."],"forward_implications":["If the result holds, RL-based tuning can be done in the field from a remote location, enabling automated commissioning of precision positioning systems without a local engineer.","The 31% range reduction in dynamic tracking implies the selected PID gains better compensate the deflector's frequency response for fast target sequences, which could improve coupling stability in FSO links under vibration.","The failure on constant targets suggests single-regime rewards can overfit: any deployment should specify a mixture of operating modes in the reward to avoid degrading static accuracy.","The method is general: the same DDPG-over-UDP approach can be applied to other cascaded PID plants, provided a safe exploration range and a self-preserving penalty are defined."],"fun_headline_variants":["DDPG tunes 6 PID gains over 200-km link, cuts dynamic error 31%","RL-tuned PID: 31% lower tracking error on dynamic targets, no gain on static","Remote DDPG optimizes fiber deflector PID, 31% error cut on moving targets","Six PID gains tuned by RL over 200 km: dynamic error drops 31%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The pseudo-random target trajectory used for both training and testing is generated by filtering uniform polar noise through a low-pass filter whose frequency response is matched to the optical deflector; if this generator does not represent real operating conditions, or if the evaluation uses the same signal family as training, the observed 31% gain is an artifact of optimizing and testing within the same narrow signal class.","fun_headline_variants_meta":{"raw":{"variants":["DDPG tunes 6 PID gains over 200-km link, cuts dynamic error 31%","RL-tuned PID: 31% lower tracking error on dynamic targets, no gain on static","Remote DDPG optimizes fiber deflector PID, 31% error cut on moving targets","Six PID gains tuned by RL over 200 km: dynamic error drops 31%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000749,"raw_usage":{"total_tokens":3174,"prompt_tokens":748,"completion_tokens":2426,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":492,"completion_tokens_details":{"reasoning_tokens":2327}},"tokens_in":492,"tokens_out":2426,"duration_ms":30951,"temperature":1.0,"reasoning_tokens":2327,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T21:56:12.310136+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the selected coefficient set aopt,1 on a trajectory family that differs from the training PRS (for example, a chirp, a recorded platform-vibration sequence, or band-limited noise with a different cutoff), and compare radial error range against the baseline. If the range reduction disappears, reverses, or is not reproduced across repeated trials, the reported 31% gain is specific to the generator rather than a general property of the tuned gains.","supporting_citations":[],"review_version":1}