{"id":"109d11a0-a9c5-4b8d-a96d-d5ba97073703","arxiv_id":"2501.16485","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A MOESP-identified state-space model plus Kalman filter is applied to the JIGSAWS dataset, but the reported position estimation accuracy falls below 95 percent under several of the paper's own simulated network scenarios.","lead":"This paper combines MOESP system identification with a Kalman filter to estimate the position of a surgical robot arm over a simulated lossy network. The headline claim of over 95 percent accuracy is contradicted by the paper's own results table for moderate and severe network impairments.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract claims over 95% estimation accuracy, but Table II itself reports accuracies as low as 85.95%, with nine of fifteen per-axis entries below 95%; the central claim is contradicted by the paper's own data.","rationale":"The stress-test pass confirms the reader's rejection is justified. The paper's central claim is not merely unsupported by external evidence; it is contradicted by the paper's own Table II. Under moderate, high, and severe network impairments, all three per-axis accuracies fall below 95%, reaching as low as 85.95%. This is an internal inconsistency in the reported results and is sufficient to falsify the abstract and conclusion statements. The reader's rationale already identifies this contradiction, but the reader's listed weakest assumption emphasizes the LTI/MOESP modeling premise and Q/R circularity. I agree that those are important, but I find the direct numerical contradiction more decisive: even if the linear model and the noise covariances were perfectly appropriate, the table still fails to support the headline number. The undefined accuracy metric compounds the problem, since no reader can verify what the percentages mean. Algorithm 1's scalar-update loop is a further internal flaw: it reuses the a priori prediction for each output component, so the filter does not actually fuse all three position measurements, making the reported per-axis results mechanistically suspect. A concrete arithmetic audit and a request for code and the accuracy definition would settle whether any version of the claim can be maintained. Because the reader already recommended REJECT, my assessment does not change the verdict; hence UNCHANGED.","tokens_in":13486,"tokens_out":3430,"duration_ms":32853,"concrete_test":"Perform an arithmetic audit of Table II: for each of the five network scenarios, compare every per-axis accuracy entry against the 95% threshold. Since rows with moderate, high, and severe impairments contain entries of 88.46%, 85.95%, 90.66%, and similar, the abstract's 'above 95%' claim fails directly. To go further, request the authors' code and the exact formula for 'Est%' (the accuracy metric). Then re-implement Algorithm 1 exactly as written, including the scalar-update loop that reuses x_pred and P_pred, and compare the resulting estimates with Table II. If the code instead chains updates correctly, the reproduction will show which version produced the table; if no code is supplied, the reported numbers remain unverifiable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim, stated in the abstract and repeated in the conclusion, is that the MOESP-based Kalman filter achieves over 95% estimation accuracy under network-induced uncertainties. The only quantitative evidence, Table II, contradicts this. For the moderate row (jitter 1.0 ms, delay 1 ms, loss 0.001%), the reported accuracies are x=90.66%, y=88.67%, z=93.47%. For the high row (jitter 2.0 ms, delay 5 ms, loss 0.001%), they are 88.46%, 89.32%, 93.65%. For the severe row (jitter 3.0 ms, delay 200–5000 ms, loss 1%), they are 85.95%, 87.68%, 90.95%. Thus, nine of the fifteen entries in Table II are below 95%, and the conclusion's statement that the filter 'maintains accuracies above 95% for all positions' is false on the paper's own numbers. This is an internal inconsistency, not a matter of external consensus. Separately, the accuracy metric is never defined: Section VI says only that accuracy 'reflects the similarity between the estimated and acPSM states,' with no formula, so even the entries above 95% cannot be audited. A compounding implementation concern is that Algorithm 1's inner loop (lines 9–13) computes each scalar update using the same a priori x_pred and P_pred rather than the updated state and covariance; consequently, only the last output dimension's update survives, and the filter does not correctly fuse the three position measurements. This undermines the mechanism that supposedly produces the reported estimates. Without code or a precise accuracy definition, the table cannot be reproduced or reconciled with the claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a Kalman filter (KF) for estimating the patient-side manipulator (PSM) position in teleoperated robotic surgery. The state-space model for the PSM dynamics is identified offline from the JIGSAWS dataset using the MOESP subspace identification method, and the KF is then evaluated under simulated network delay, jitter, and packet loss. The paper claims that the proposed approach achieves over 95% estimation accuracy despite network-induced uncertainties, with a linear-time KF making it suitable for Tactile Internet applications. The evaluation is performed in simulation using JIGSAWS kinematic data, with accuracy and RMSE reported for the x, y, and z positions under several network impairment scenarios.","tokens_in":13779,"tokens_out":4385,"duration_ms":40527,"significance":"If the claims were supported, the contribution would be a useful data-driven alternative to hand-derived kinematic models for PSM position estimation, with clear computational advantages and a falsifiable accuracy target. The paper uses a public dataset and explicitly states the algorithm, which are positive features for reproducibility. However, the central accuracy claim is contradicted by the paper's own Table II, the accuracy metric is never defined, and the Kalman update in Algorithm 1 does not implement a correct multi-output fusion. These issues make the quantitative results unreliable in their current form, so the contribution cannot be assessed as presented.","major_comments":[{"comment":"The abstract and conclusion claim over 95% estimation accuracy, but Table II reports accuracies below 95% in nine of the fifteen per-axis entries. For the moderate row (jitter 1.0 ms, delay 1 ms, loss 0.001%) the values are x=90.66%, y=88.67%, z=93.47%; for the high row (2.0 ms, 5 ms, 0.001%) they are 88.46%, 89.32%, 93.65%; for the severe row (3.0 ms, 200–5000 ms, 1%) they are 85.95%, 87.68%, 90.95%. The statement in Section IX that \"the KF maintains accuracies above 95% for all positions\" is therefore false on the paper's own data. This is an internal inconsistency in the central claim and must be resolved.","section":"Abstract, Section IX, Table II"},{"comment":"The \"percentage accuracy\" metric is never defined. The text says only that accuracy \"reflects the similarity between the estimated and acPSM states,\" with no formula. Without a precise definition, the entries in Table II cannot be audited, and the reported accuracies are not even internally consistent with the RMSE values: for example, the row with jitter 0.5 ms, delay 0.5–2 ms, and loss 0.01% has RMSE_x=0.0363 and accuracy 96.67%, while the row with the same jitter/delay but loss 0.001% has a lower RMSE_x=0.0356 yet a lower accuracy of 95.58%. Please define the metric, for instance as 100*(1 - RMSE/range) or a normalized error measure, and report it consistently.","section":"Section VI"},{"comment":"The inner update loop recomputes the Kalman gain from the same a priori x_pred and P_pred for each output dimension d and overwrites x_est[k] and P on every iteration. As written, only the last output dimension's update survives, so the filter does not fuse the three position measurements; it effectively performs a single scalar update. A correct multi-output Kalman update should use the full measurement vector with the matrix gain K = P_pred C^T (C P_pred C^T + R)^{-1}, or a sequential update that propagates the updated state and covariance into each subsequent scalar update. The reported estimation results therefore cannot be attributed to the algorithm presented.","section":"Algorithm 1, lines 9–13"},{"comment":"The noise covariance matrices Q and R are estimated empirically from residuals of an initial KF run on the JIGSAWS data (Section III-C), while the final evaluation in Section VI appears to use the same dataset. Section V.B.3 states that a cross-validation methodology was used, but the manuscript does not specify which trials were used for MOESP identification, which for Q/R estimation, and which for final evaluation. If the same trials are used for all three stages, the reported accuracy is optimistically biased. Please specify the exact trial split and report results on a held-out test set that is disjoint from both model identification and noise covariance estimation.","section":"Section III-C and Section VI"},{"comment":"The system order selection is ambiguous. The text says the knee point around mode 10 indicates the practical order, while Algorithm 2 line 19 instructs to \"Choose n based on singular values (e.g., 85% energy criterion).\" These are different criteria that can yield different model orders. Please state the exact rule used and report the resulting model order n for each experiment, since the identified model order is a free parameter that directly affects the Kalman filter's behavior.","section":"Section V.A and Algorithm 2, line 19"}],"minor_comments":[{"comment":"JIGSAWS is described as having 76 features, but the paper does not specify which MTM signals are used as the control input vector u in Eq. (1), nor how the three PSM position outputs are selected and extracted. Please clarify the exact input and output variables and their indices in the dataset.","section":"Section IV and Section III.A"},{"comment":"The symbol n is used both as the width of the Hankel matrix and later as the system order; this is confusing. Please use distinct symbols for the Hankel dimension and the state dimension.","section":"Section V.B, Eq. (14)"},{"comment":"Since Min-Max normalization is applied to the data, please state whether the reported RMSE values are computed in normalized coordinates or in original physical units, and provide the corresponding units.","section":"Table II and Section V.C"},{"comment":"The comparison table claims \"Robustness to Noise: Excellent\" and \"Real-Time Performance: High\" for the proposed method without any quantitative runtime or noise-sensitivity measurements. Please provide measured computation times and statistical variability across trials, or temper these qualitative claims.","section":"Section VIII, Table III"},{"comment":"Reference [9] is the authors' own prior conference paper, which is mentioned only in the introduction. Please add an explicit quantitative comparison with this earlier KF approach in Section II or Section VI, since a central claim of the paper is that MOESP improves on it.","section":"Section II"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an extended version of the authors' arXiv preprint 2406.04503, but the relationship is not clearly disclosed beyond a brief mention. The paper is evaluated entirely in simulation with synthetic network impairments; there is no real-time experiment, no comparison with existing estimation methods on identical data, and no code release, despite the emphasis on computational efficiency. Given the internal contradiction in the headline accuracy claim, the undefined accuracy metric, and the incorrect multi-output update in Algorithm 1, I do not see a straightforward revision that would make the quantitative claims credible. The paper may be more suited to a communications or signal-processing venue than to IEEE Transactions on Robotics, unless the robotic estimation contribution is substantially strengthened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is an incremental, reasonable-in-principle idea — MOESP system identification on the JIGSAWS dataset to build a linear state-space model, then a Kalman filter for PSM position estimation under network impairments — but the paper's own numbers contradict its headline claim. The abstract and conclusion say the filter achieves over 95% accuracy; Table II shows nine of fifteen per-axis entries below 95%, with the severe-impairment row at 85.95%, 87.68%, and 90.95%. That's not a nuance, it's a load-bearing inconsistency. The accuracy metric is never defined ('similarity between the estimated and acPSM states'), so even the good rows can't be audited. Algorithm 1 has a real bug: the inner loop computes each scalar Kalman update from the same a priori x_pred and P_pred, so only the last output dimension's update survives; the filter is not actually fusing the three position measurements. Also, the additive network model in Eq. (2) doesn't match the time-delay/packet-drop simulation in Algorithm 1. There's a cross-validation sentence, but no split is given, and Q/R are estimated from the same data used for evaluation, so the circularity burden is real. No code or parameter files are provided.\n\nTo be fair, the paper is clear about its lineage (a direct extension of their prior KF paper, now with MOESP), the public-dataset choice is sensible, and the writing is readable. The idea of deriving a lightweight model for a KF from data in this setting is not crazy, and the comparison table's complexity discussion, while hand-wavy, points in the right direction. But the experiments as reported do not support the claims.\n\nWho gets value from this? Someone doing a survey of data-driven estimation for teleoperation might cite it as an early attempt, but as it stands the numbers are not reliable. My recommendation: don't send this to a serious peer review as is. It needs a major revision that reconciles the abstract with Table II, defines and reports the accuracy metric, fixes Algorithm 1, specifies the train/test split, and releases code or at least exact experimental parameters. Until then, I wouldn't rely on it.","headline":"Reasonable-in-principle MOESP plus Kalman filter applied to JIGSAWS, but the paper's own Table II contradicts the >95% accuracy claim and Algorithm 1's update loop is incorrect; needs major revision before serious review.","tokens_in":14399,"tokens_out":5645,"would_cite":false,"duration_ms":48425,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Kalman filter whose state-space model is learned from surgical kinematic data via MOESP keeps patient-side arm position estimates above 95 percent accuracy under network delay, jitter, and packet loss.","keywords":["Tactile Internet","remote robotic surgery","Kalman filter","MOESP","subspace identification","JIGSAWS dataset","patient-side manipulator","network delay and packet loss"],"falsifier":"Run the same MOESP identification on one JIGSAWS trial, estimate $Q$ and $R$ from that trial only, then evaluate the Kalman filter on a held-out trial from a different surgeon and task; if the average accuracy over $x$, $y$, $z$ falls below 95 percent under the mild-impairment settings, the claimed generalization of the identified model is not supported.","tokens_in":13213,"feed_emoji":"🤖","tokens_out":11512,"duration_ms":101550,"temperature":0.7,"pith_summary":"This paper aims to establish that a Kalman filter can keep estimating the patient-side surgical arm's position accurately even when its commands travel over a delayed, lossy network, provided the filter's state-space model is learned from data instead of from proprietary robot dynamics. The authors learn the model with MOESP subspace identification applied to the JIGSAWS kinematic dataset, using master-tool inputs and patient-side outputs, and then put the identified matrices into a network-aware Kalman filter. In simulations with delay, jitter, and packet loss, the reported estimation accuracy ranges from about 86 to 99 percent depending on severity, with the operating points the authors emphasize staying above 95 percent. If the claim holds, remote surgery over Tactile Internet links can have precise arm-position feedback without needing the manufacturer's dynamics model.","feed_headline":"Filter trained on surgery data keeps arm-position accuracy above 95%","feed_subtitle":"MOESP identification gives the Kalman filter a model, cutting reliance on known robot dynamics under network stress.","key_machinery":"MOESP (Multivariable Output-Error State Space) is a subspace identification method that reconstructs the discrete-time state-space matrices $A$, $B$, $C$, $D$ directly from input-output data without a prescribed model structure. The pipeline builds block Hankel matrices of normalized master-tool (input) and patient-side (output) kinematic data, applies LQ decomposition and an oblique projection, and then an SVD; the singular-value spectrum shows a knee around mode 10, and that knee sets the model order. The identified matrices become the Kalman filter's prediction and measurement model, and the filter's noise covariances $Q$ and $R$ are estimated from residuals of an initial filter run. The load-bearing mechanism is replacing unknown proprietary robot dynamics with an identified linear model, which is what lets the Kalman filter predict the patient-side state between delayed or lost network updates.","core_discovery":"The paper's central claim is that the MOESP-based Kalman filter recovers the patient-side arm's $x$, $y$, $z$ positions from master-tool commands with high fidelity despite network-induced corruption. Using the JIGSAWS kinematic data, the authors identify a linear state-space model of order about 10, selected at the knee of the singular-value spectrum, and place that model inside a Kalman filter whose noise covariances $Q$ and $R$ are estimated empirically from residuals. They then simulate transmission through a network with specified delay, jitter, and packet loss. Under the operating points they emphasize as representative of Tactile Internet use, the filter maintains estimation accuracy above 95 percent for all three position coordinates, with RMSE in the 0.02 to 0.04 range; under severe impairments (200 to 5000 ms delay, 1 percent packet loss) accuracy falls to roughly 86 to 91 percent. The authors further claim that the filter's linear-time complexity makes it practical for real-time control, in contrast to heavier alternatives.","pith_inferences":["Editorial inference: the empirically estimated $Q$ and $R$ are computed from residuals of the same data used to evaluate the filter, so the reported accuracies may partly reflect in-sample tuning; a hold-out estimation protocol would test whether the above-95-percent figure survives on unseen trials.","Editorial inference: the model-order choice, a knee near mode 10 under an 85 percent energy criterion, is made once; sweeping the order on validation trajectories and measuring RMSE would reveal whether the identified model is genuinely minimal or simply adequate for these tasks.","Editorial inference: the network simulation treats delay, jitter, and packet loss as independent per-sample effects, whereas Tactile Internet traffic often has bursty loss and correlated delay, so a burst-loss model would be a stiffer test of the claimed resilience.","Editorial inference: if the linear-time-invariant assumption holds only for the slow, structured motions in the JIGSAWS tasks, the same filter may need adaptation for fast or contact-rich maneuvers; a test with abrupt trajectory changes would define its operating envelope."],"forward_implications":["If a Kalman filter with MOESP-identified matrices keeps above-95-percent accuracy under mild delay, jitter, and packet loss, then remote surgery position feedback can be maintained without a proprietary dynamic model of the da Vinci patient-side arm.","The $O(n)$ per-step filter cost means the same estimator can run on the 30 Hz kinematic data with margin inside a 1 to 10 ms Tactile Internet latency budget.","Accuracy degrades predictably with impairment severity, so the approach could serve as a graceful-degradation layer that stays high-fidelity under near-ideal links and remains usable under moderate ones.","Because MOESP learns from a recorded trial of MTM/PSM data, the pipeline could be re-trained quickly for other teleoperated arms or new setup configurations, as long as kinematic data are available.","Compared with GPR and transformer alternatives, the identified linear model plus Kalman filter avoids costly matrix inversions and is better suited to real-time operation."],"supporting_citations":[{"why":"Supplies the JIGSAWS kinematic dataset of MTM and PSM motion at 30 Hz, used for both system identification and evaluation.","marker":"[19]"},{"why":"Provides the MOESP/N4SID subspace identification formulation, including the Hankel-matrix and decomposition steps.","marker":"[27]"},{"why":"Supplies the public implementation of MOESP used to compute the state-space matrices.","marker":"[28]"},{"why":"Provides the reliability and latency reference values used to set the simulated network delay, jitter, and packet loss.","marker":"[29]"},{"why":"Foundational reference for the Kalman filter equations used in the prediction and update steps.","marker":"[17]"}],"fun_headline_variants":["MOESP-Kalman filter keeps robotic arm position accurate over noisy links","Surgical robot arm tracking: 95%+ accuracy even with network jitter","Data-driven Kalman filter beats network chaos for remote surgery","Remote surgery positioning stays above 95% with MOESP-based filter","Forget robot models: MOESP+Kalman nails arm position under delay"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a single linear time-invariant model of order about 10, learned from one segment of JIGSAWS data, captures the patient-side arm's dynamics well enough to extrapolate under network impairments, with $Q$ and $R$ estimated from that same data reflecting true noise rather than in-sample fit.","fun_headline_variants_meta":{"raw":{"variants":["MOESP-Kalman filter keeps robotic arm position accurate over noisy links","Surgical robot arm tracking: 95%+ accuracy even with network jitter","Data-driven Kalman filter beats network chaos for remote surgery","Remote surgery positioning stays above 95% with MOESP-based filter","Forget robot models: MOESP+Kalman nails arm position under delay"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000424,"raw_usage":{"total_tokens":2174,"prompt_tokens":945,"completion_tokens":1229,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":1148}},"tokens_in":561,"tokens_out":1229,"duration_ms":9263,"temperature":1.0,"reasoning_tokens":1148,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T12:55:01.054259+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same MOESP identification on one JIGSAWS trial, estimate $Q$ and $R$ from that trial only, then evaluate the Kalman filter on a held-out trial from a different surgeon and task; if the average accuracy over $x$, $y$, $z$ falls below 95 percent under the mild-impairment settings, the claimed generalization of the identified model is not supported.","supporting_citations":[{"cited_title":"Da Vinci Surgical System,","cited_arxiv_id":null,"evidence_quote":"Supplies the JIGSAWS kinematic dataset of MTM and PSM motion at 30 Hz, used for both system identification and evaluation."},{"cited_title":"W., Wahab, N","cited_arxiv_id":null,"evidence_quote":"Provides the MOESP/N4SID subspace identification formulation, including the Hankel-matrix and decomposition steps."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the public implementation of MOESP used to compute the state-space matrices."},{"cited_title":"Towards enabling critical mMTC: A review of URLLC within mMTC,","cited_arxiv_id":null,"evidence_quote":"Provides the reliability and latency reference values used to set the simulated network delay, jitter, and packet loss."},{"cited_title":"Kalman filter: historical overview and review of its use in robotics 60 years after its creation,","cited_arxiv_id":null,"evidence_quote":"Foundational reference for the Kalman filter equations used in the prediction and update steps."}],"review_version":1}