{"id":"3fa744ba-8b15-4016-a4c6-631605d90989","arxiv_id":"2605.28033","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"User study finds kinesthetic teaching yields shortest demonstrations and lowest workload on complex tasks, joystick wins on simple peg picking, and gestures are less reliable but sometimes comparable.","lead":"This paper reports a user study with eight participants comparing three robot teaching methods—kinesthetic guidance, joystick teleoperation, and hand gestures—across three manipulation tasks using replay success, NASA-TLX workload, and error counts. Kinesthetic guidance showed advantages on orientation-sensitive and contact-rich tasks while joystick performed best on simple picking.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.3","headline":"Small N=8 and three tasks limit generalizability of modality rankings","rationale":"Reader's weakest_assumption directly identifies the representativeness issue; full-text access does not alter the small-scale nature of the evidence, so no adjustment to UNVERDICTED is warranted.","tokens_in":1586,"tokens_out":239,"duration_ms":18829,"concrete_test":"Re-run the identical protocol with 20+ new participants (balanced demographics) on the same three tasks; recompute success rates, demonstration durations, and TLX scores with paired statistical tests—if modality rankings or significance levels change, the original claim does not generalize.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim (kinesthetic guidance yields shortest demos, lowest workload, highest success on orientation/contact-rich tasks) is derived from a within-subjects user study with only eight participants. No power analysis, effect-size reporting, or cross-validation across additional tasks/users is referenced; results on replay success and NASA-TLX could be driven by individual motor skill differences or the specific peg/insertion/contact properties of the three tasks rather than modality per se.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper reports a within-subjects user study with eight participants comparing three robot teaching modalities—kinesthetic guidance, joystick teleoperation, and hand gestures—across three manipulation tasks. It measures demonstration duration, replay success rate, modified NASA-TLX workload scores, and common teaching errors. The central empirical finding is that kinesthetic guidance yields the shortest demonstrations, lowest workload, and highest success rates on orientation-sensitive and contact-rich tasks, while joystick performs best on simple peg-picking and gestures are less reliable overall but occasionally comparable.","tokens_in":1648,"tokens_out":450,"duration_ms":23685,"significance":"If the results hold under larger samples, the work supplies concrete, multi-metric evidence on usability trade-offs among common teaching interfaces for robot programming by demonstration. The within-subjects design and use of both objective (success, duration) and subjective (NASA-TLX) measures strengthen the comparison; such data can directly inform interface selection in HRI and robotics applications.","major_comments":[{"comment":"User study section (participant count and analysis): the central claim that kinesthetic guidance is superior on orientation- and contact-rich tasks rests on data from only eight participants. No power analysis, effect-size reporting, or confidence intervals are referenced, so the modality rankings could be driven by individual differences rather than modality per se; this directly limits the generalizability asserted in the abstract and conclusion.","section":null},{"comment":"Task selection and results discussion: the three chosen tasks (peg picking, orientation-sensitive, contact-rich) are not shown to be representative of broader manipulation scenarios; without additional tasks or cross-validation, the task-specific performance ordering cannot be treated as a general recommendation for teaching modality choice.","section":null}],"minor_comments":[{"comment":"Abstract: statistical details (p-values, error bars, or participant demographics) are absent, making it difficult to assess the strength of the reported differences.","section":null},{"comment":"Methods: clarify the exact definition and scoring of 'common teaching errors' and whether the modified NASA-TLX was administered after each trial or per modality.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address each major comment below and indicate where revisions will be made to the manuscript.","responses":[{"response":"We acknowledge that N=8 is a small sample and that the absence of power analysis, effect sizes, or confidence intervals weakens the statistical claims. This is a genuine limitation of the current study. In revision we will compute and report effect sizes (e.g., Cohen’s d) and 95% confidence intervals for the key duration, success-rate, and NASA-TLX comparisons, add a dedicated limitations paragraph, and moderate the language in the abstract and conclusion to present the results as preliminary rather than definitive.","revision_made":"partial","referee_comment":"User study section (participant count and analysis): the central claim that kinesthetic guidance is superior on orientation- and contact-rich tasks rests on data from only eight participants. No power analysis, effect-size reporting, or confidence intervals are referenced, so the modality rankings could be driven by individual differences rather than modality per se; this directly limits the generalizability asserted in the abstract and conclusion."},{"response":"The three tasks were selected to isolate distinct manipulation challenges (free-space pick-and-place, orientation precision, and contact-rich insertion) that recur across many pbd applications. We agree, however, that they do not exhaustively represent all manipulation domains. In the revised discussion we will (1) cite prior HRI literature justifying these task categories and (2) explicitly qualify that the observed modality rankings are task-dependent and should not be read as universal recommendations.","revision_made":"yes","referee_comment":"Task selection and results discussion: the three chosen tasks (peg picking, orientation-sensitive, contact-rich) are not shown to be representative of broader manipulation scenarios; without additional tasks or cross-validation, the task-specific performance ordering cannot be treated as a general recommendation for teaching modality choice."}],"tokens_in":1261,"tokens_out":419,"duration_ms":25094,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper runs a within-subjects comparison of three established teaching methods on three manipulation tasks. Eight participants each used kinesthetic guidance, joystick teleoperation, and hand gestures, then the authors measured demonstration length, modified NASA-TLX workload, replay success rate, and error patterns.\n\nWhat the work actually delivers is a set of concrete head-to-head measurements. Kinesthetic guidance produced shorter demonstrations, lower workload, and higher success on the orientation-sensitive and contact-rich tasks. Joystick performed best on the simple peg pick. Gestures were less reliable overall but occasionally reached comparable results. These are direct empirical observations rather than new algorithms or frameworks.\n\nThe study uses standard metrics and a controlled design, which keeps the claims grounded. The abstract states the main patterns clearly without overreaching.\n\nThe soft spot is the sample. Eight participants and three tasks leave room for individual motor differences or task-specific quirks to dominate the rankings. No power analysis, effect sizes, or detailed demographics appear in the provided abstract, and the stress-test note correctly flags that the strongest claim could shift with more users or different tasks. That limits how far the modality ordering can be taken.\n\nThis is the sort of paper that belongs in a robotics conference track on human-robot interaction. Readers who need quick empirical benchmarks on these three interfaces will get some value; anyone looking for broad design rules or statistical robustness will not.\n\nI would send it to peer review. The comparison is straightforward and the metrics are relevant, so referees can usefully press on the statistics and scope without the paper being fundamentally unsound.","headline":"Eight-person study gives usable numbers on kinesthetic vs joystick vs gesture teaching but the rankings rest on too narrow a base to generalize.","tokens_in":2131,"tokens_out":392,"would_cite":false,"duration_ms":25120,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Kinesthetic guidance yields shorter demonstrations, lower workload, and higher success than joystick or gestures on orientation-sensitive robot tasks.","keywords":["robot teaching","kinesthetic guidance","joystick teleoperation","hand gestures","user study","manipulation tasks","workload assessment","demonstration replay"],"falsifier":"A larger study or different set of tasks in which joystick or gesture methods produce shorter demonstrations, lower workload, or higher success rates than kinesthetic guidance on orientation-sensitive or contact-rich tasks.","tokens_in":2488,"feed_emoji":"🤖","tokens_out":653,"duration_ms":20397,"temperature":0.7,"pith_summary":"The paper compares three ways to teach robots movements by demonstration: physically guiding the arm by hand, operating it with a joystick, and using hand gestures. Eight participants performed three manipulation tasks, and the study measured demonstration length, mental and physical workload via a modified NASA-TLX scale, and how reliably the robot replayed the taught motion. Kinesthetic guidance produced the shortest teaching sessions with the lowest workload and best success rates on tasks that required careful orientation or physical contact, while the joystick performed best on the simplest peg-picking task and gestures showed mixed but sometimes competitive results. These trade-offs matter because the choice of teaching method directly affects how quickly and accurately humans can program robots for practical work.","feed_headline":"Kinesthetic teaching beats joystick and gestures on complex robot tasks","feed_subtitle":"Eight-participant study finds shortest demos and lowest workload for orientation and contact tasks.","key_machinery":"A controlled user study measuring replay success, modified NASA-TLX workload scores, and common teaching errors for three teaching modalities across three manipulation tasks.","core_discovery":"In a user study with eight participants comparing kinesthetic guidance, joystick teleoperation, and hand-gesture teaching across three manipulation tasks, kinesthetic guidance produced the shortest demonstrations, lowest workload, and highest replay success on the more orientation-sensitive and contact-rich tasks, joystick teleoperation performed best on simple peg picking, and hand-gesture teaching, although less reliable overall, achieved results comparable to kinesthetic guidance in some cases.","pith_inferences":["Hybrid interfaces that switch modalities based on detected task type could combine the speed of kinesthetic teaching with the accessibility of gestures.","The advantage of shorter, higher-quality demonstrations may compound when the taught motions are later used to train learned robot controllers.","Repeating the comparison with users who have no prior robotics experience could reveal different workload and error patterns."],"forward_implications":["Kinesthetic guidance is the stronger choice for tasks that require precise orientation or physical contact.","Joystick teleoperation is preferable for simple picking operations.","Hand-gesture teaching can serve as a usable alternative when it matches kinesthetic performance despite lower overall reliability.","Teaching modality should be selected according to the orientation and contact demands of the target task."],"fun_headline_variants":["Kinesthetic guidance produces shortest demonstrations lowest workload","Kinesthetic highest replay success on orientation contact tasks","Joystick teleoperation highest success on simple peg picking","Hand gesture teaching less reliable but sometimes comparable"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That results from three specific manipulation tasks performed by eight participants can be generalized to broader robot teaching scenarios.","fun_headline_variants_meta":{"raw":{"variants":["Kinesthetic guidance produces shortest demonstrations lowest workload","Kinesthetic highest replay success on orientation contact tasks","Joystick teleoperation highest success on simple peg picking","Hand gesture teaching less reliable but sometimes comparable"]},"model":"grok-4.3","cost_usd":0.009707,"raw_usage":{"total_tokens":4188,"prompt_tokens":557,"num_sources_used":0,"completion_tokens":55,"cost_in_usd_ticks":97065500,"prompt_tokens_details":{"text_tokens":557,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3576,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":557,"tokens_out":55,"duration_ms":36238,"temperature":1.0,"reasoning_tokens":3576,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T12:18:53.390351+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A larger study or different set of tasks in which joystick or gesture methods produce shorter demonstrations, lower workload, or higher success rates than kinesthetic guidance on orientation-sensitive or contact-rich tasks.","supporting_citations":[],"review_version":1}