{"id":"7aa99bcb-63ad-4344-bd4b-d5d83eb0bf35","arxiv_id":"2502.07326","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A deep-reinforcement-learning agent that dynamically tunes PI control gains reduced deflection errors by 26% to 90% on commercial scanning probe microscope scans.","lead":"Researchers built PICTS, a system that uses deep reinforcement learning to adjust the proportional-integral gains of a scanning probe microscope in real time. On three test materials, it reported 26% to 90% lower deflection errors than a fixed-gain commercial controller, pointing to more stable nanoscale imaging.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed DRL advantage is confounded by a hardware change: the DRL arm uses a custom FPGA PI loop, while the baseline uses the MFP-3D internal controller, and no fixed-gain FPGA control was run.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing concern: the comparison conflates the DRL gain-adaptation mechanism with the custom FPGA control hardware. I considered other limitations—the absence of error bars, the unavailable SI, and unreported reward/hyperparameter values—but the hardware confound is more fundamental because it breaks the causal link between the DRL policy and the reported improvements. If the paper's claim were only 'the integrated PICTS system outperforms the commercial controller,' a hardware confound would be less damaging; however, the abstract and introduction consistently emphasize DRL's adaptive gain adjustment as the source of the improvement. The proposed check—a fixed-gain FPGA control arm—is the minimal experiment that would settle whether the learned gains, rather than the higher-bandwidth FPGA loop, are responsible. Because this experiment is missing, the current evidence supports at most a conditional acceptance, consistent with the reader's verdict. I therefore recommend no change to the verdict: the conditional status appropriately reflects a plausible but not yet causally isolated central claim.","tokens_in":12173,"tokens_out":3759,"duration_ms":37549,"concrete_test":"Add a fixed-gain control arm on the identical FPGA path: disable Cycle 2 (the agent decision-making loop) so the FPGA PI controller holds P and I at fixed values, using both the initial FPGA gains and values matched to the commercial baseline, and scan the same grating, PS-LDPE, and HOPG samples at the same speeds and scan ranges with at least five independent scans per condition. If the fixed-gain FPGA PI arm already achieves the bulk of the 26–90% error reduction relative to the MFP-3D controller, the DRL attribution is unsupported; if DRL still shows a significant reduction with non-overlapping confidence intervals relative to fixed-gain FPGA PI, the central claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that the DRL policy's gain adaptation reduces deflection errors by 26–90% across samples—requires the DRL gain adjustment to be the causal factor. But Sections 2.1.1 and 4.1.1 show that the DRL arm runs a custom FPGA-based PI controller at 128 kHz using NI-9775/9263 ADC/DAC modules and a low-pass filter, while the baseline is the MFP-3D's internal commercial controller with fixed P-I gains. The two arms differ in two independent variables: adaptive versus fixed gains, and FPGA-based versus commercial control hardware. The paper never reports a fixed-gain run on the same FPGA path. Without that control arm, the observed improvements in Section 2.2.2 and Figure 5 could largely reflect the faster FPGA loop, the additional analog filtering, or the different DAC/ADC signal path, rather than the learned P-I adjustment. This is the load-bearing gap because it prevents attribution of the headline 26–90% error reduction specifically to DRL. In addition, Section 2.2.2 reports no repeated-scan statistics or error bars, so the magnitude of the effect is also uncertain. The weaker claim that the complete PICTS system outperforms the commercial controller may still hold, but the paper's emphasized DRL-specific causal claim is not established by the current comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents PICTS, a parallel control and training system that integrates a soft actor-critic deep reinforcement learning agent with an FPGA-based proportional-integral controller for scanning probe microscopy. The agent adjusts P-I gains in real time using a reward function that penalizes deflection error, overshoot, oscillation, and probe-sample disengagement. The authors compare the DRL-based controller with the commercial MFP-3D controller with fixed P-I gains on a calibration grating, PS-LDPE, HOPG, aerogel, and CFRP samples. They report qualitative improvements in topography and deflection images and quantitative reductions in mean deflection error of 26% to 90% across conditions, with error histograms more concentrated near zero. The paper claims that PICTS balances real-time control and computational load by offloading control to an FPGA and training to a host computer.","tokens_in":12436,"tokens_out":4308,"duration_ms":40147,"significance":"If the claimed improvements are robust, this is a practically significant demonstration of DRL-based adaptive control in commercial SPM hardware, addressing a real limitation of fixed-gain P-I controllers on sharp edges, soft multiphase materials, and large topographical variations. The main strengths are the physical integration with an MFP-3D system, the explicit three-cycle architecture, the use of a standard SAC algorithm, and the evaluation across multiple challenging samples with quantitative error histograms. The paper is also careful to scan with the commercial controller first to avoid attributing sample damage to the DRL controller. However, the central quantitative claim is currently undercut by a hardware confound and by the absence of repeated-scan statistics, so the significance is contingent on additional control experiments.","major_comments":[{"comment":"The comparison is confounded by hardware differences between the two control arms. The DRL path uses a custom FPGA-based PI loop (cRIO-9064 with NI-9775/9263 and a low-pass filter) at 128 kHz, while the baseline uses the MFP-3D internal commercial controller with fixed P-I gains. The two arms therefore differ in both the gain-adaptation strategy and the control hardware/signal chain. No fixed-gain PI control run on the same FPGA path is reported, so the observed 26% to 90% deflection-error reduction cannot be attributed specifically to DRL gain adaptation; it may reflect the faster loop rate, analogue filtering, or different ADC/DAC path. The authors should add a fixed-gain control condition on the same FPGA hardware, or explicitly reframe the claim as a system-level comparison of PICTS with the commercial controller and temper the DRL-specific causal language in the abstract and Section 2.2.2.","section":"Section 2.1.1 and Section 4.1.1"},{"comment":"The quantitative claim lacks uncertainty quantification and repeated-scan statistics. Figure 5B reports one mean deflection-error value per condition with no error bars, no number of repeated scans, and no per-line or per-image spread; the histograms in Figure 5A similarly do not report counts, bin widths, or vertical-axis units. The 26% to 90% improvement range is therefore not statistically grounded. Please provide repeated-scan statistics (at least n, mean, and standard deviation or confidence interval) for each condition, a formal comparison between controllers, and a clear description of how the deflection-error signal was sampled and aggregated.","section":"Section 2.2.2 and Figure 5"},{"comment":"The primary evaluation metric, mean deflection error, is also the main term optimized by the reward function. The reported improvement is thus partly the optimized objective rather than an independent validation of imaging quality or stability. This does not invalidate the measured engineering result, but the paper should acknowledge the overlap and report at least one independent metric, such as trace-retrace correlation, image sharpness, edge fidelity, or contact-loss rate, to support the broader claims of improved stability and scanning precision.","section":"Section 2.1.3 and Section 2.2.2"}],"minor_comments":[{"comment":"The statement that hyperparameters were determined by grid search in a simulated environment is not accompanied by any description of the simulator or a table of the chosen hyperparameter values; providing these details is necessary for reproducibility.","section":"Section 4.2.2"},{"comment":"Reference 22 (Arulkumaran et al.) appears twice, and the reference numbering after that entry is offset; please correct the numbering and remove the duplicate.","section":"Reference list"},{"comment":"The sentence 'Analogue Input-Output Communication: This operates via TCP over Ethernet' is unclear for analogue signals; please clarify the actual signal path, the role of TCP/IP, and the reported 100 MB/s speed and 1000 μs delay.","section":"Section 4.1.1"},{"comment":"The SI is described as 'available upon requirements'; for a methods-heavy paper with additional samples and analyses, the SI should be provided with the manuscript.","section":"Supplementary Information"},{"comment":"The fixed-gain baseline is described only as integral gain values of 10 and 20; please specify the corresponding proportional gain and any other relevant controller settings so the baseline is reproducible.","section":"Section 2.2.1(i)"}],"recommendation":"major_revision","confidential_remarks":"This is a promising applied-control paper with real hardware integration and a clear practical motivation. The main barrier is the confounded comparison: a same-FPGA fixed-gain baseline is essential for the DRL-specific claim in the abstract. The missing repeated-scan statistics are also a straightforward fix with additional data collection. The reward/evaluation overlap is a lesser concern for an engineering demonstration, but it should be acknowledged. The paper fits an applied instrumentation or SPM methods venue more than a traditional materials-science journal, though it is within scope given the SPM development focus."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zoe, here is my read on Wei et al. The genuinely new thing is the system: they put a SAC agent in the loop on a commercial MFP-3D, with FPGA-based PI control at 128 kHz, host-side decision at 512 Hz, and online training at 1 Hz. That integration—DRL tuning P-I gains on real hardware, not in simulation—is new as far as the cited literature goes. The architecture is thoughtful, and the qualitative images on grating, PS-LDPE, and HOPG show a real working system. I believe the scans happened and the system does what they say.\n\nThe soft spot is exactly the one the stress-test flags. The DRL arm uses a custom FPGA loop with its own ADC/DAC, low-pass filter, and 128 kHz rate; the baseline is the MFP-3D internal controller. That is two variables changed at once: gain adaptation and control hardware. Without a fixed-gain run on the same FPGA path, the 26–90% error reduction cannot be attributed specifically to the learned P-I adjustment. A faster loop and cleaner analog path could easily account for a large chunk of it. That weakens the central causal claim, though not the weaker claim that the whole PICTS system outperforms the commercial controller.\n\nThe second issue is that Fig. 5 reports mean error reductions with no error bars, no repeated-scan statistics, and no per-scan scatter. Given the qualitative images show strong effects, I suspect the effect is real, but the magnitude—90% vs 26%—is not pinned down. Third, there is no code, no data, and the SI is “obtained upon requirements”; explicit reward weights, SAC hyperparameters, and thresholds for the overshoot/oscillation detector are all missing. For an ML-for-control paper, that makes reproducibility weak. Minor: references 21 and 22 are duplicated.\n\nWho this is for: the SPM instrumentation community and people working on RL for real-time control. It deserves a serious referee, but the referee should ask for a same-hardware fixed-gain control experiment, repeated-scan error bars, and full hyperparameters/data. I would engage with it, but I would not cite the headline improvement numbers until the confounding control is done.\n\nReading group: maybe—good for discussing experimental design confounds in RL-for-control work.\n\nRecommendation: send to peer review with major revision, not desk reject.","headline":"A genuinely new hardware integration of DRL into a commercial SPM, but the headline 26–90% error reduction is confounded by a hardware change and needs a same-hardware fixed-gain control arm before it can be believed.","tokens_in":13011,"tokens_out":3031,"would_cite":false,"duration_ms":27511,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A deep-reinforcement-learning agent that retunes a microscope's feedback loop in real time reduces deflection errors by 26–90% on difficult samples.","keywords":["scanning probe microscopy","deep reinforcement learning","proportional-integral control","adaptive control","field-programmable gate array","deflection error","Soft Actor-Critic","topography imaging"],"falsifier":"Run a fixed-gain P-I controller on the same FPGA board with the same gains the commercial controller used, and compare deflection errors; if the fixed-gain FPGA loop alone already matches the DRL controller's low errors, then the DRL adaptation is not the driver of the reported improvement.","tokens_in":11947,"feed_emoji":"🔬","tokens_out":8006,"duration_ms":62115,"temperature":0.7,"pith_summary":"Scanning probe microscopy's height-control loop normally runs with fixed proportional-integral (P-I) gains, and those fixed gains produce artefacts on sharp edges, soft multiphase structures, and large topographical steps. The paper presents a system, PICTS, that replaces the fixed gains with a deep-reinforcement-learning agent that adjusts the P and I gains at every scan point based on the current error, control output, actuator history, and current gains. On a calibration grating, a biphasic PS-LDPE film, and exfoliated HOPG, the adaptive controller reduces mean deflection error by 26% to 90% relative to the commercial fixed-gain controller, with error distributions more concentrated near zero. The paper argues this makes SPM control practical for difficult samples without a high-performance computer.","feed_headline":"AI-tuned control cuts microscope deflection errors by 26–90%","feed_subtitle":"A deep-reinforcement-learning agent retunes the feedback loop in real time, keeping the probe tracking sharp edges and soft materials.","key_machinery":"The load-bearing mechanism is a three-cycle parallel control-and-training architecture. A field-programmable gate array (FPGA) runs the high-speed P-I control loop at 128 kHz, computes the error signal, and detects overshoots and oscillations; a host computer runs a Soft Actor-Critic agent that at 512 Hz reads the current state and outputs small increments to the P and I gains; and a training thread at 1 Hz continually updates the agent's policy. The reward function combines a squared-error accuracy term, penalties for overshoot and oscillation derived from the FPGA-based detector, and a penalty for losing probe-sample contact, which together steer the agent toward gain combinations that keep the probe precisely tracking. The incremental action space is designed to produce smooth gain changes, one adjustment per scan point.","core_discovery":"The central claim is that a deep reinforcement learning controller that dynamically adjusts the proportional and integral gains of the SPM height loop outperforms a commercial fixed-gain P-I controller, reducing deflection errors by 26% to 90% depending on the sample, with fewer outliers and error values clustered near zero. The agent infers surface context from the error signal, control output, and actuator history, and changes the gains accordingly: it raises P on downward slopes to prevent the probe from disengaging, lowers I when transitioning onto soft material to avoid oscillations, and reduces gains after a feature to prevent overcompensation. These behaviours are demonstrated on a calibration grating with sharp edges, a soft biphasic PS-LDPE polymer, and HOPG with 4–6 micrometre height variations, with additional aerogel and carbon-fibre-reinforced polymer tests reported in the supplementary material.","pith_inferences":["A natural next experiment would be to run the learned gain schedule on the commercial controller's own hardware, or a fixed-gain P-I loop on the FPGA, to separate the benefit of the learned adaptation from the benefit of the FPGA implementation itself.","The same three-cycle architecture could be applied to other SPM feedback loops—such as amplitude control in tapping mode or the lateral force loop—or to other probe-based instruments that rely on fast P-I control.","The online-training design suggests a path to controllers that co-adapt with tip degradation: as the probe wears, the agent's policy could keep adjusting indefinitely, though the paper only demonstrates stable operation over a four-hour window.","A testable prediction is that on samples with extremely high topographical slope, the adaptive controller's deflection-error advantage over fixed-gain control should grow monotonically with slope magnitude, because the agent explicitly changes gains on downward slopes to prevent disengagement."],"forward_implications":["If the reported error reductions hold, SPM users could scan sharp-edged or soft, multiphase samples without manually retuning P and I gains for each region, and would see improved trace-retrace consistency.","The system's FPGA-plus-CPU split means the adaptive control loop could be added to existing commercial SPM hardware without a high-performance computer, lowering the barrier for labs.","The largest reductions—up to 90%—occur on the most challenging surfaces (HOPG with large height variations), so the method is most valuable where conventional fixed-gain control breaks down.","Because training runs concurrently with scanning, the agent's policy continues to adapt over time, which should also absorb slow drifts such as tip wear or thermal drift during long scans."],"supporting_citations":[{"why":"Supplies the Soft Actor-Critic algorithm with automatic entropy adjustment that trains the DRL agent.","marker":"[26]"},{"why":"Provides the off-policy maximum-entropy SAC objective used for action selection in continuous space.","marker":"[27]"},{"why":"Supplies the prioritized experience replay that biases training toward informative transitions.","marker":"[30]"},{"why":"Earlier simulation-based DRL scan control that this work extends to a commercial SPM with quantitative error comparison.","marker":"[25]"},{"why":"RL-based asymmetric scan control concept referenced for achieving smooth, safe tip-sample interaction.","marker":"[24]"},{"why":"Course-report source cited for the idea of reinforcement learning adjusting SPM scan parameters.","marker":"[21]"}],"fun_headline_variants":["AI tunes microscope feedback, cutting errors up to 90%","Deep RL adjusts gains on the fly, slicing probe errors by 26–90%","SPM control gets smarter: RL agent cuts errors by 26–90%","Probe error drops up to 90% with adaptive AI gains","Reinforcement learning cuts probe errors by up to 90% in real time"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim rests on the assumption that the error reduction comes from the learned gain adaptation and not from the different hardware (custom FPGA loop vs. commercial controller) used in the two compared arms.","fun_headline_variants_meta":{"raw":{"variants":["AI tunes microscope feedback, cutting errors up to 90%","Deep RL adjusts gains on the fly, slicing probe errors by 26–90%","SPM control gets smarter: RL agent cuts errors by 26–90%","Probe error drops up to 90% with adaptive AI gains","Reinforcement learning cuts probe errors by up to 90% in real time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001148,"raw_usage":{"total_tokens":4656,"prompt_tokens":733,"completion_tokens":3923,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":349,"completion_tokens_details":{"reasoning_tokens":3823}},"tokens_in":349,"tokens_out":3923,"duration_ms":23853,"temperature":1.0,"reasoning_tokens":3823,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T13:04:51.893517+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a fixed-gain P-I controller on the same FPGA board with the same gains the commercial controller used, and compare deflection errors; if the fixed-gain FPGA loop alone already matches the DRL controller's low errors, then the DRL adaptation is not the driver of the reported improvement.","supporting_citations":[{"cited_title":"& Gonzalez -Martinez, J","cited_arxiv_id":null,"evidence_quote":"Supplies the Soft Actor-Critic algorithm with automatic entropy adjustment that trains the DRL agent."},{"cited_title":"& Zeng, K","cited_arxiv_id":null,"evidence_quote":"Provides the off-policy maximum-entropy SAC objective used for action selection in continuous space."},{"cited_title":"P., Brundage, M","cited_arxiv_id":null,"evidence_quote":"Supplies the prioritized experience replay that biases training toward informative transitions."},{"cited_title":"General Resolution Enhancement Method in Atomic Force Microscopy (AFM) Using Deep Learning","cited_arxiv_id":"1809.03704","evidence_quote":"Earlier simulation-based DRL scan control that this work extends to a commercial SPM with quantitative error comparison."},{"cited_title":"V., Rar, A","cited_arxiv_id":null,"evidence_quote":"Course-report source cited for the idea of reinforcement learning adjusting SPM scan parameters."}],"review_version":1}