{"id":"8cb1cf6f-8f90-4124-bb5e-5ae104154fda","arxiv_id":"2412.10599","paper_version":1,"verdict":"UNVERDICTED","confidence":"HIGH","novelty_score":0.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey of Transformer applications in robotic perception, planning, control, human-robot interaction, and reinforcement learning.","lead":"This paper reviews how Transformer models are used in robotics, covering pre-trained foundation models, reinforcement learning, and perception, planning, and control. It maps recent work in the field but does not present new experiments or results.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified; the broad survey claim is uncontroversial, but the review's internal errors mean the 'UNVERDICTED' verdict stands.","rationale":"The reader identified the weakest assumption as reliance on the fidelity of cited primary results and the representativeness of informal paper selection. I agree with that assessment. The paper's central claim—that Transformers are being adopted in three major ways in robotics—is a descriptive survey claim, not a falsifiable technical contribution. The condition that must hold for this claim to be true is that the cited papers genuinely support the described trends. Nothing in the manuscript gives me a reason to doubt the broad trend: the transformers-in-robotics literature is large and well-documented, and the review's own citation list includes canonical works (RT-1, RT-2, OpenVLA, Decision Transformer, Trajectory Transformer, Perceiver, SAM, CLIP). However, the paper does contain internal errors that prevent me from treating it as a reliable secondary source: the MarineFormer claim appears misattributed, several figure references are broken ('see Figure ??'), the efficient-transformer table duplicates Routing Transformer, and the sparse/adaptive table also contains a duplicate Routing Transformer entry. These errors do not falsify the central claim, but they do mean the review should not be verified as a trustworthy compilation of specific numbers. Hence, UNVERDICTED is the right verdict, and no verdict adjustment is needed. The concrete test I propose would distinguish between mere typographical sloppiness (which would not change the verdict) and systemic misreporting (which would make the paper actively misleading as a survey).","tokens_in":22176,"tokens_out":1847,"duration_ms":14450,"concrete_test":"Verify a random sample of the review's specific quantitative claims against the cited primary sources. For each of the five most prominent claims (MarineFormer 20% improvement, 92.73% fault-instruction detection, 99.95% simulation success, TransformerMPC 6.8x-34.9x speedup, OpenVLA outperforming RT-2-X at smaller footprint), locate the corresponding passage in the cited paper and check (a) that the number appears there, (b) that it is quoted in context, and (c) that the cited reference is actually the source of that result. If at least four of the five check out, the survey's factual reporting is trustworthy despite formatting issues; if several fail, the paper's reliability as a secondary source is materially weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"This is a narrative review with no new experiments, no systematic search protocol, and no formal derivation. The strongest claim—that Transformers are being adopted in robotics across foundation models, DRL variants, and perception/planning/control—is supported by numerous cited primary papers and is broadly consistent with the field's consensus. For a review, the load-bearing condition is that the cited numbers and characterizations faithfully represent the cited sources. I checked several specific claims against the visible citation list. The paper states MarineFormer 'demonstrated 20% success rate improvement' but cites reference [101] as Parisotto and Salakhutdinov, 'Efficient transformers in reinforcement learning using actor-learner distillation'—which is not MarineFormer. The paper states '92.73% detection of fault instructions' [104] and '99.95% success rate in simulation' [105]; both appear to be citations to construction-HRI papers, but the exact numbers are not verifiable from the reference list alone. The paper attributes TransformerMPC speedups of 6.8x to 34.9x to [172], which is plausibly consistent with the cited arXiv paper, but no independent replication is possible from this manuscript. These are citation-level concerns; they do not undermine the broad central claim, because the broad claim would survive even if a few specific numbers were wrong or misattributed. The review itself is explicit that it is a broad overview and it acknowledges limitations such as dataset imbalance and sim-to-real gaps. Therefore, the reader's 'UNVERDICTED' verdict is appropriate: there is no central technical claim to accept or reject, and the paper's value is as an informal survey. The internal errors, broken figure references, and duplicate table entries are execution-quality issues, not logical flaws in the central assertion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a narrative review of transformer architectures for robotic applications. After a background section on the original transformer and its efficient, multimodal, and sparse variants, it organizes robotic applications into three threads: pre-trained foundation models (including vision-language-action models and human-robot interaction), transformer-based deep reinforcement learning variants, and transformer use in perception, planning, and control. It closes with challenges and future directions. The paper presents no new experiments or derivations; its central descriptive claim is that transformers are now broadly adopted across robotic perception, planning, and control, which is consistent with the current literature.","tokens_in":22500,"tokens_out":9908,"duration_ms":81284,"significance":"If the survey's attributions were reliable, the paper would serve as a useful broad overview of a fast-moving area, with a sensible tripartite organization and up-to-date coverage (through late 2024) of models such as RT-2, OpenVLA, Octo, π0, and TransformerMPC. The authors explicitly acknowledge limitations in Section 4 (dataset imbalance, sim-to-real gap, safety, memory footprint) and propose concrete future directions such as quantization for resource-constrained deployment and cross-embodiment training. The main weakness is that several specific quantitative and architectural claims are mis-cited or unverifiable from the reference list; because a review's value depends on readers being able to trace claims to sources, these issues are load-bearing and must be corrected before the paper can be relied upon as a survey.","major_comments":[{"comment":"The same work (Parisotto and Salakhutdinov, 'Efficient transformers in reinforcement learning using actor-learner distillation') appears as both [101] and [102]. Reference [101] is then used for two incompatible claims: that transformers are effective as world models, and that 'MarineFormer [101] ... demonstrated 20% success rate improvement' for marine environments. The cited paper is neither primarily a world-model paper nor MarineFormer; MarineFormer appears to be a different work that is missing from the bibliography. These mis-citations make specific quantitative claims untraceable and must be fixed, either by correcting the citations or by inserting the correct references.","section":"§3.2, references [101]/[102]"},{"comment":"The table row 'Synthesizer [138]' cites reference [138], which is Tay et al., 'Sparse Sinkhorn Attention.' The Synthesizer architecture is a separate paper (Tay et al., 'Synthesizer: Rethinking Self-Attention for Transformer Models') and is not in the bibliography, while the Sinkhorn Transformer is correctly associated with [138] in Table 2 of Section 2.3.3. The two attention mechanisms are conflated; please replace the citation and/or add the Synthesizer reference.","section":"§2.3.1, Table 1 (Efficient Transformers)"},{"comment":"The table lists 'Routing Transformer [121]' twice with different asymptotic complexities: O(n log n) and O(n^{1.5} d^*). This is internally inconsistent, and neither entry is adequately justified. Please consolidate the duplicate entries into a single row and state the complexity reported in the original Routing Transformer paper (Roy et al., 2020), or explain why two different forms are listed.","section":"§2.3.3, Table 2 (Sparse and Adaptive Transformers)"},{"comment":"The sentence 'Gated Transformer-XL [23] for Long-Term Memory' cites reference [23], which is Transformer-XL by Dai et al.; the Gated Transformer-XL model is reference [103] (Parisotto et al., 'Stabilizing Transformers for Reinforcement Learning'), already correctly cited in Section 3.2. In the same subsection, the ViNT model is discussed ('ViNT, a Transformer-based model for visual navigation, demonstrates significant potential...') without any citation at all. Please correct the attribution and add a citation for ViNT.","section":"§3.3.2, Planning"},{"comment":"The paragraph on human-robot collaboration reports specific success rates: '92.73% detection of fault instructions by humans' [104] and '99.95% success rate in simulation' [105]. From the reference list alone, I cannot confirm that these exact numbers appear in the cited papers, since [104] and [105] are construction-HRI papers but no page numbers, sections, tables, or experiment descriptions are given. Because these figures are the only quantitative evidence in that paragraph, please verify the attributions or qualify the claims as reported in the cited sources.","section":"§3.1, Human-Robot Collaboration"}],"minor_comments":[{"comment":"Both sections contain unresolved 'see Figure ??' cross-references (after the ViLBERT description and after the OpenVLA description, respectively); these must be fixed before publication.","section":"§2.3.2 and §3.1"},{"comment":"The table entry 'VisualGPT [74]' is mis-cited: reference [74] is the VisualBERT paper, which is correctly named in the bullet list immediately below the table. Please correct the table entry or add the actual VisualGPT reference.","section":"§2.3.2, Table 3"},{"comment":"There is a typo in the last paragraph: 'zero-short generalization' should read 'zero-shot generalization.'","section":"§4"},{"comment":"The phrase 'having low inference times while maintaining low sample effeciency of transformers' appears to state the opposite of the intended meaning; the actor-learner distillation approach is designed to retain the sample efficiency of transformers, so please rephrase to 'high sample efficiency' or similar.","section":"§3.2"},{"comment":"The caption is grammatically tangled and the subfigure references are unclear: 'while Franka has the most number of scenes see 2a x-Arm and Google Robot have the biggest contribution to trajectory data see 2b and 2c' does not map cleanly to panels (a)–(e). Please rewrite the caption to describe each panel explicitly.","section":"Figure 2 caption"},{"comment":"Reference [93] is an incomplete URL-like entry containing a sentence fragment rather than a full citation, and reference [59] has a garbled author list and ordering. Both need to be reformatted according to the journal style.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a narrative review with no original experiments, so the soundness of its central claim depends entirely on the fidelity of its citations. The broad claim—that transformers are now standard across robotic perception, planning, and control—is consistent with the literature and would survive correction of the specific errors. However, the number of demonstrable mis-citations (MarineFormer attributed to [101], Synthesizer attributed to [138], GTrXL attributed to [23], duplicate Routing Transformer entries) and the unresolved figure cross-references indicate that the manuscript has not been carefully checked. I do not recommend rejection, but the paper needs a thorough fact-checking pass before it can serve as a trustworthy survey."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one so you know what it is: a narrative review of transformer uses in robotics, current through late 2024. There is no new method, dataset, or analysis, so there is nothing to accept or reject in the usual sense. The broad claim—that transformers enter robotics through foundation models, DRL variants, and perception/planning/control—is true and uncontroversial, and the paper does a decent job of organizing recent work in those buckets. Coverage of Open X-Embodiment, RT-1/RT-2, OpenVLA, π0, SAM, and the decision-transformer line is reasonably up to date, and the authors cite earlier surveys rather than pretending to be first.\n\nThe soft spots are execution-level, but for a review they are the product. The MarineFormer '20% improvement' claim is attributed to [101], which is actually Parisotto and Salakhutdinov's actor-learner distillation paper, not MarineFormer. The '92.73% fault-instruction detection' and '99.95% simulation success' numbers point at construction-HRI papers, but the references do not let a reader verify them. The efficient transformer table cites Synthesizer as [138] when [138] is the Sparse Sinkhorn Attention paper, and the Routing Transformer appears twice with different complexity claims. There are multiple 'see Figure ??' broken cross-references. None of this kills the central descriptive claim, but it undercuts the one function a survey should serve: pointing accurately at the primary literature.\n\nThe selection is also informal—no search protocol, no inclusion/exclusion criteria—so the picture may be skewed toward whatever the authors happened to read. The limitations section is honest about dataset imbalance and sim-to-real gaps, which is good.\n\nWho is this for? A newcomer who wants a quick map of the landscape and is prepared to chase the actual papers. It is not a scholarly reference in its current state. I would not cite it, and I would not bring it to a reading group except as an example of why citation hygiene matters. If the authors fixed the references and broken figures, it could be a serviceable tutorial review, but as is I would desk-reject it rather than spend referee time.","headline":"Broad, current-through-2024 survey of transformers in robotics; the organizing claim is fine, but citation errors and broken references make it unsafe as a reference and not yet worth refereeing.","tokens_in":22984,"tokens_out":2946,"would_cite":false,"duration_ms":27186,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This review argues that Transformers have become a mainstream architecture across robotic perception, planning, and control, advancing human-robot interaction, long-horizon planning, and system performance.","keywords":["Transformers","robotics","foundation models","vision-language-action models","deep reinforcement learning","perception","planning","control"],"falsifier":"Run a systematic search of robotics papers from 2022 through 2024 using the review's topical keywords and check whether transformer-based methods appear in a substantial share of perception, planning, and control systems; then attempt to reproduce the headline numbers on the original benchmarks, including TransformerMPC's 6.8x to 34.9x speedups, MarineFormer's 20% success-rate gain, and the 92.73% fault-instruction detection rate. If transformer use is concentrated in a narrow slice of robotics or those numbers do not reproduce, the review's characterization of adoption is unsubstantiated.","tokens_in":21945,"feed_emoji":"🤖","tokens_out":7348,"duration_ms":58375,"temperature":0.7,"pith_summary":"This review aims to establish that Transformers are now a broadly adopted component in robotics, spanning perception, planning, and control rather than remaining confined to one subdomain. It organizes the field into three adoption routes: pretrained foundation models that give robots language understanding and zero-shot generalization; transformer variants integrated with deep reinforcement learning that treat decision-making as sequence modeling and enable long-horizon planning; and transformer-based systems that improve perception, planning, and control performance. The paper argues that these routes share a common engine, the self-attention mechanism, which lets a model relate every part of an input sequence to every other part regardless of distance. If the picture is correct, roboticists should expect transformer-based architectures to become the default scaffold for new autonomous systems.","feed_headline":"Transformers are remaking robot perception, planning, and control","feed_subtitle":"Three adoption routes: pretrained models, reinforcement learning, and control pipelines.","key_machinery":"The load-bearing object is the self-attention mechanism introduced by the original Transformer, which lets every element of an input sequence attend to every other element in parallel, independent of sequence length. In this review it is the shared engine behind all three adoption routes: attention over language tokens gives robots instruction-following, attention over visual tokens gives perception and grounding, and attention over sequences of states, actions, and rewards gives reinforcement learning and planning. The review's organizing distinction among foundation models, deep-reinforcement-learning hybrids, and perception-planning-control systems is the map that shows this one mechanism spreading across robotics.","core_discovery":"The paper's central claim is that in robotics, Transformers are being adopted in three major ways: as pretrained foundation models facilitating human-robot interaction and generalization, as transformer variants integrated with deep reinforcement learning for long-horizon planning, and as components that enhance perception, planning, and control systems. It presents evidence from generalist robot policies such as RT-1, RT-2, OpenVLA, Octo, and π0, from sequence-modeling RL methods such as the Decision Transformer and Trajectory Transformer, and from perception and control systems such as CLIP, SAM, ViNT, and TransformerMPC. The paper reports representative results, including a 92.73% detection rate for faulty instructions in construction human-robot interaction, a 20% success-rate improvement for MarineFormer, and 6.8x to 34.9x speedups from TransformerMPC, as evidence of this trend.","pith_inferences":["Going beyond the paper: if this trend continues, robotics may follow the same trajectory as natural language processing, converging on a few large generalist policies pretrained on pooled data and then fine-tuned per task.","The headline numbers in the review come from heterogeneous tasks, metrics, and baselines, so direct comparison across the three routes is not possible without a standardized evaluation protocol.","Because the review selected papers informally, the most direct test of its broad-adoption thesis is a systematic literature search; a search that found transformer use concentrated in manipulation and largely absent from field robotics would require qualifying that thesis."],"forward_implications":["If the central claim is correct, transformer-based components will continue displacing CNNs, RNNs, and classical planners as the default building blocks for robotic systems.","Shared multi-embodiment datasets such as Open X-Embodiment become a critical resource, so data collection, standardization, and embodiment balance are where the field's bottlenecks now sit.","Sequence-modeling formulations of reinforcement learning make long-horizon tasks more tractable, allowing robot learning systems to be pretrained offline and then fine-tuned online.","Transformer-based control accelerators such as TransformerMPC can cut runtime by an order of magnitude while preserving constraint satisfaction, making real-time deployment on robots more practical.","Zero-shot and few-shot generalization is treated as achievable for manipulation in controlled settings, while the review also warns that it is not yet guaranteed in unstructured, outdoor, or safety-critical environments."],"supporting_citations":[{"why":"Introduces the original Transformer and the self-attention mechanism that every reviewed application builds on.","marker":"[144]"},{"why":"Supplies the pooled Open X-Embodiment dataset used to train the generalist robot policies discussed in the foundation-model section.","marker":"[21]"},{"why":"Supplies the Decision Transformer formulation that casts reinforcement learning as sequence modeling, the basis of the long-horizon planning claims.","marker":"[17]"},{"why":"Provides RT-1, an example of a transformer-based real-world control policy trained at scale.","marker":"[12]"},{"why":"Provides RT-2, the vision-language-action model cited for transferring web knowledge to robotic control.","marker":"[11]"},{"why":"Provides OpenVLA, an open-source vision-language-action model that the review uses to compare performance against larger closed models.","marker":"[62]"},{"why":"Supplies the contrastive language-image pretraining method that the review cites for aligning visual and textual modalities in robotic perception.","marker":"[112]"},{"why":"Supplies the Segment Anything Model used as the foundation for segmentation-based robotic grasping and visual servoing.","marker":"[64]"},{"why":"Supplies the TransformerMPC speedup figures (6.8x to 34.9x) that anchor the control-performance claims.","marker":"[172]"},{"why":"Introduces the Vision Transformer that the review identifies as the basis for transformer-based perception in manipulation and navigation.","marker":"[29]"}],"fun_headline_variants":["Transformers rewrite robot autonomy: perception to control","Robot transformers: three routes to smarter autonomy","Transformers boost robot planning, perception, and control","From pretrained models to RL: transformers in robotics","How transformers are driving robot intelligence forward"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The review runs no experiments and uses no systematic search criteria, so its central picture rests on the assumption that the quantitative results reported in the cited primary papers are accurate and that the informally chosen set of papers fairly represents the field.","fun_headline_variants_meta":{"raw":{"variants":["Transformers rewrite robot autonomy: perception to control","Robot transformers: three routes to smarter autonomy","Transformers boost robot planning, perception, and control","From pretrained models to RL: transformers in robotics","How transformers are driving robot intelligence forward"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000284,"raw_usage":{"total_tokens":1641,"prompt_tokens":875,"completion_tokens":766,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":696}},"tokens_in":491,"tokens_out":766,"duration_ms":6998,"temperature":1.0,"reasoning_tokens":696,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:47:18.003363+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a systematic search of robotics papers from 2022 through 2024 using the review's topical keywords and check whether transformer-based methods appear in a substantial share of perception, planning, and control systems; then attempt to reproduce the headline numbers on the original benchmarks, including TransformerMPC's 6.8x to 34.9x speedups, MarineFormer's 20% success-rate gain, and the 92.73% fault-instruction detection rate. If transformer use is concentrated in a narrow slice of robotics or those numbers do not reproduce, the review's characterization of adoption is unsubstantiated.","supporting_citations":[{"cited_title":"Transformermpc: Accelerating model predictive control via transformers, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the TransformerMPC speedup figures (6.8x to 34.9x) that anchor the control-performance claims."}],"review_version":1}