Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RoboChemist's dual-loop VLM+VLA design lifts average success by 23.57% and compliance by 0.298 on chemistry lab tasks.

desk verdict Solid VLM+VLA integration for robotic chemistry with a genuinely useful visual-prompting contribution, but the headline success-rate claim compares apples to oranges because the closed-loop system gets retries and the baselines don't. read the letter →

arxiv 2509.08820 v1 pith:6A7IIFI4 submitted 2025-09-10 cs.RO

Zongzheng Zhang , Chenghao Yue , Haobo Xu , Minwen Liao , Xianglin Qi , Huan-ang Gao , Ziwei Wang , Hao Zhao This is my paper
classification cs.RO
keywords roboticchemistryvision-language-actionmodelvisualpromptingclosed-loopcontrollong-horizonmanipulationsafetycompliancedual-armlabautomation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that robotic chemistry requires both high-level reasoning about experimental norms and low-level dexterity, and that neither alone suffices. RoboChemist is a dual-loop system: a vision-language model (VLM) decomposes a long-horizon experiment into primitive steps, draws bounding boxes and keypoints onto the live camera image to mark safe grasp and target regions, and then, after each step, judges from the image whether the step succeeded; a vision-language-action (VLA) model executes each primitive using the prompted image. The authors report that this combination raises average success by 23.57 percentage points and compliance by 0.298 over strong VLA baselines on seven primitive tasks and five complete multi-step experiments, and that the trained primitives generalize to unseen objects and workflows. If correct, the work points toward lab automation that is compositional and safety-aware rather than rigidly scripted.

What carries the argument

The load-bearing mechanism is instruction-aware visual prompting: the VLM converts a subtask description plus safety guidelines into explicit 2D annotations (bounding boxes for regions of interest, points for grasp or target locations) overlaid on the RGB image, and these annotated images are used both during VLA fine-tuning and at inference as an extra input channel. The second component is the outer closed loop: after each primitive, the VLM inspects the current image and returns a success verdict, triggering re-execution when the step is judged incomplete, so a sequence of discrete actions can emulate contingent behavior such as 'pour until colorless.'

What would settle it

Run a pouring task with three visually similar transparent beakers containing colorless liquids in a cluttered scene, annotate ground-truth target containers by hand, and count how often the VLM's generated bounding box matches the intended container. If, without retraining, the prompt's spatial accuracy is near chance, the reported success-rate advantage should not transfer to scenes outside the training distribution. Alternatively, ablate the outer loop entirely (no monitor retry) and compare success rates: if the gap between the full system and the no-monitor version shrinks to near zero, t

Watch

Extended reading notes

Core claim

RoboChemist's central claim is that a VLM can serve three roles at once: planner, visual prompt generator, and monitor. Feeding the VLA model an image annotated with the VLM's bounding boxes and keypoints closes the gap between semantic instructions and precise manipulation. The VLA model is fine-tuned with prompted reference images alongside the usual camera views and text instructions, and the outer loop re-executes a primitive until the VLM's monitor confirms success. Evaluated on seven primitives (grasp, heat, pour, stir, transfer solid, insert, press button) and on complete protocol-like experiments such as acid-base neutralization and flame tests, the full system attains success rates

Load-bearing premise

The whole cascade depends on the vision-language model reliably placing bounding boxes and keypoints on transparent, deformable, and cluttered labware; if a prompt points to the wrong container or wrong grasp point, the action model will execute confidently on the wrong target, and the closed loop will then verify the wrong thing.

Editorial extensions

If this is right

  • A single system can execute multi-step chemistry protocols composed from a small set of trained primitives, with no task-specific programming beyond a natural-language description of apparatus and reagents.
  • The closed-loop monitor converts a fixed script into condition-based behavior: retrying a grasp until it lands at the compliant position, or re-pouring acid until an indicator changes color.
  • Visual prompts generated by a grounded vision-language model avoid the need for depth reconstruction of transparent labware, which previously caused failures in transparent-object manipulation.
  • Reported generalization means a robot trained on seven primitives can be repurposed to new reagents, containers, and entire experiments by changing only the task description given to the VLM.
  • The compliance metric shows that procedural adherence (not just task completion) can be evaluated and trained, which is necessary for real laboratory safety.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference not in the paper: the reported gains may come disproportionately from the retry loop rather than the prompts; an ablation that removes only the monitor while keeping prompted images would isolate the causal contribution of each loop.
  • Inference not in the paper: if prompt accuracy is the bottleneck, combining the VLM's 2D marks with depth or segmentation only where transparency defeats RGB is a natural next step for cluttered scenes.
  • Inference not in the paper: the same dual-loop architecture could transfer to other long-horizon, safety-critical manipulation domains, such as surgical assistance or assembly, where a semantic monitor can judge step completion from images.
  • Inference not in the paper: a concrete stress test would vary the number of visually similar transparent containers and measure the VLM's prompt IoU against human labels; if prompt spatial accuracy degrades faster than downstream success, that would pinpoint where the cascade's reliability limit lives.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RoboChemist, a dual-loop robotic chemistry system that couples a VLM (Qwen2.5-VL) with a VLA policy (π0). The VLM acts as planner, as generator of image-space visual prompts (bounding boxes and keypoints), and as monitor that verifies subtask completion and triggers re-execution. The VLA is fine-tuned on 400 demonstrations per primitive task, with a mixture of successful and second-attempt trajectories. Experiments cover seven primitive tasks and five complete chemistry protocols, comparing against ACT, RDT, and π0, plus visual-prompt baselines ReKep and MOKA. The headline claim is a 23.57 percentage-point higher average success rate and a 0.298 higher compliance rate over prior VLA baselines, with additional generalization results for unseen objects and workflows.

Significance. If the quantitative claims are robust, RoboChemist is a useful step toward closed-loop, safety-aware laboratory automation: it addresses transparent/deformable labware without depth reconstruction, and it integrates semantic monitoring into a VLA loop. The paper has real strengths: real-robot evaluation on a diverse chemistry task suite, a w/o-CL ablation showing that visual prompting alone improves over π0, and qualitative generalization to reaction types not seen in training. However, the current evaluation does not yet rigorously support the headline quantitative claim. The main comparison conflates the outer-loop retry mechanism with policy quality, the training-data mixture appears to be selected after seeing evaluation results, and the absence of statistical uncertainty makes the reported margins difficult to interpret. These are fixable with additional experiments and reporting, so the contribution is defensible in principle but needs a major revision.

major comments (4)
  1. [§3.3, Table 2] The headline 23.57 pp SR gain is not a like-for-like comparison. RoboChemist w/ CL re-executes a failed primitive until the VLM monitor declares success, whereas ACT/RDT/π0 are evaluated as single-pass policies (the paper states 'the loop would end after a failed attempt'). No baseline is augmented with the same monitor/retry wrapper, and no attempt counts or timeouts are reported. The w/o CL row (avg 82.14 vs π0's 70) shows visual prompting alone helps, but it does not decompose how much of the remaining 11.43 pp comes from retries versus policy quality. Add at least a π0+monitor ablation and report retry statistics.
  2. [§4.1/A.6, Table 7] The 300/100 training-data mixture (Config 2) used in the main experiments was selected after inspecting Table 7's evaluation results. This is test-set-based model selection and can inflate the reported numbers. The paper must either use a held-out validation set for this choice or report all configurations' end-to-end performance (with visual prompting and closed loop) so the reader can assess selection bias.
  3. [§4.1, Tables 2–3] The evaluation has 20 trials per task and no error bars, confidence intervals, or significance tests. Several SR differences are within binomial noise (e.g., 80 vs 85 in Table 2; 18/20 vs 17/20 in Table 1). Report 95% CIs or exact binomial tests for at least the headline averages, and for the compliance-rate differences, to support the claimed margins.
  4. [§3.2/A.8] The method's success depends on Qwen2.5-VL placing bounding boxes and keypoints correctly on transparent/deformable labware, but prompt accuracy is never measured independently. A.8 reports 'Prompting 35%' of 20 failures without defining the criterion or denominator, and the monitor's false positive/negative rates are unknown. Provide a human-annotated accuracy metric for generated prompts and monitor decisions on at least a subset of trials.
minor comments (5)
  1. [A.3.1, task 3] The task-decomposition prompt says 'flame test of copper(II) hydroxide' while A.2 defines the task as a CuSO4 flame test; this should be corrected.
  2. [Table 4, π0 row] The 'Press the Button' CR is 0.363, inconsistent with 0.575 in Table 2.
  3. [A.4] 'Manganese(II) hydroxide' should be the intended catalyst/species; as written the species/equation mismatch is confusing. Also, 'breaker' is a typo for 'beaker' in A.3.1 item 5.
  4. [Figure 18] The figure lacks axis labels and a legend; the text mentions seven variations but the figure shows only six labels.
  5. [A.6/Table 7] The relationship between Config 2's 70% average and the w/o CL average of 82.14% in Table 2 is unexplained; clarify whether visual prompts are included and whether Table 7 uses the same trial set as Table 2.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central comparisons use external human rubric and controlled ablations.

full rationale

The paper's headline SR/CR gains are measured against external baselines (ACT, RDT, pi0) fine-tuned on the same data and scored with a fixed human rubric in Appendix A.1. The visual prompting contribution is isolated by the w/o CL vs w/ CL comparison, and the closed-loop contribution is isolated by the w/o CL ablation. The VLM serves as planner/prompt-generator/monitor, which creates a potential self-referential loop, but the SR/CR metrics are externally defined in A.1, so the system's success is not defined as its own monitor's verdict. The retry-loop difference between RoboChemist w/CL and single-pass baselines is a fairness/experimental-design issue (missing baseline-with-monitor), not a circular reduction to the paper's inputs. Self-citations to related work (e.g., [16], [17], [33], [52], [81]) are contextual and not load-bearing. Therefore no circular step can be exhibited with a specific reduction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a handful of freely chosen evaluation parameters (rubric weights, trial counts, training mixture) and several domain assumptions about VLM grounding and VLA conditioning. No new physical entities are introduced.

free parameters (3)
  • Compliance rubric weights = 0, 0.25, 0.5, 0.75, 1 per task
    Hand-assigned scores in A.1 define the CR metric; the choice of weights is arbitrary and affects all reported compliance rates.
  • Training data mixture (successful vs second-attempt) = 300/100
    A.6 selects Config 2 because it yields the highest average SR on the evaluation tasks, a post-hoc choice.
  • Number of trials per task = 20
    Evaluation uses 20 trials per task with no power analysis or error bars.
assumptions (4)
  • domain assumption Qwen2.5-VL reliably grounds grasp and target points on transparent and cluttered scenes
    Section 3.2 relies on this for all visual prompts; if false, the VLA receives wrong targets.
  • domain assumption Fine-tuned pi0 VLA can condition on the prompted image as an additional input channel
    Section A.7 states the prompted image is added to the image encoder input; the method assumes this conditioning is effective.
  • domain assumption Human-defined compliance rubric reflects procedural safety norms
    The CR metric in A.1 is the only measure of safety compliance; the paper provides no external validation of the rubric.
  • domain assumption The 7 primitives compose into the 5 complete tasks
    Complete tasks are evaluated as sequences of primitives; failure to cover needed operations would invalidate the long-horizon claims.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation." pith.science (2026). https://pith.science/paper/6A7IIFI4

@misc{pith2026250908820,
  author       = {Pith},
  title        = {Pith review of: RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6A7IIFI4}},
  note         = {Machine review of arXiv:2509.08820}
}
read the original abstract

Robotic chemists promise to both liberate human experts from repetitive tasks and accelerate scientific discovery, yet remain in their infancy. Chemical experiments involve long-horizon procedures over hazardous and deformable substances, where success requires not only task completion but also strict compliance with experimental norms. To address these challenges, we propose \textit{RoboChemist}, a dual-loop framework that integrates Vision-Language Models (VLMs) with Vision-Language-Action (VLA) models. Unlike prior VLM-based systems (e.g., VoxPoser, ReKep) that rely on depth perception and struggle with transparent labware, and existing VLA systems (e.g., RDT, pi0) that lack semantic-level feedback for complex tasks, our method leverages a VLM to serve as (1) a planner to decompose tasks into primitive actions, (2) a visual prompt generator to guide VLA models, and (3) a monitor to assess task success and regulatory compliance. Notably, we introduce a VLA interface that accepts image-based visual targets from the VLM, enabling precise, goal-conditioned control. Our system successfully executes both primitive actions and complete multi-step chemistry protocols. Results show 23.57% higher average success rate and a 0.298 average increase in compliance rate over state-of-the-art VLA baselines, while also demonstrating strong generalization to objects and tasks.

Figures

Figures reproduced from arXiv: 2509.08820 by the authors.

Figure 1
Figure 1. (a) Overview of RoboChemist. The VLM in our system acts as the planner, decomposing high-level tasks into subtasks. Based on each subtask, the VLM generates prompted images through visual prompting and provides them, along with other relevant information, to the VLA models. The VLM also functions as the monitor, assessing the completion status of subtasks, thus ensuring a complete feedback loop in the system. (b) Ro… view at source ↗
Figure 2
Figure 2. Illustration of the acid-base neutralization reaction experiment. 1) The Beginning: The researcher provides the complete task instructions, initial scene setup and available primitive tasks, followed by RoboChemist’s task decomposition. 2) Step 2: During the glass rod grasping step, RoboChemist uses visual prompting to highlight the bbox and grasp point to adhere to safety guidelines, providing reference for subsequ… view at source ↗
Figure 3
Figure 3. In this experiment, ReKep [25] struggles to reconstruct the transparent test tube, leading to a failed grasp. Methods that pre-compute grasp candidates without considering textual instruc￾tions are generally effective in standard settings but may fail to account for safety and procedural requirements crucial in chemistry experiments. As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Visualization of Complete Chemical Experiments. (a) Complexation reaction: mixing NaCl and CuSO4 to form a light green complex; (b) Copper flame test: dipping a platinum wire into CuSO4 solution and observing a green flame upon heating; (c) Acid-base neutralization: pr…
Figure 5
Figure 5. Figure 5: Visualization of Generalization in Complete Tasks. (a) Flame tests: Ca2+ (brick-red), Li+ (purplish-red), and Na+ (yellow); (b) Displacement reaction: Fe displaces Cu from CuSO4 so￾lution; (c) Double displacement reaction: NaOH and CuSO4 form Cu(OH)2 precipitate; (d) D…
Figure 6
Figure 6. Figure 6: Visualization of primitive tasks. 4. Pouring Liquid: • Objective: Pour a liquid from one container into another. • Compliance Criteria: – Fails to grasp: 0 – Grasped but spills completely: 0.25 – Grasped but spills slightly: 0.75 – Successfully grasps and pours liquid …
Figure 7
Figure 7. Figure 7: Visualization of mixing NaCl and CuSO4 solutions. 2. Thermal Decomposition of Cu(OH)2 • Objective: Heat a test tube containing solid Cu(OH)2 and observe the resulting decompo￾sition process. • Primitive tasks: Grasp the test tube containing Cu(OH)2 → Heat over flame. •…
Figure 8
Figure 8. Figure 8: Visualization of thermal decomposition of Cu(OH) [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Visualization of flame test of CuSO4 solution. 4. Evaporation of NaCl Solution • Objective: Evaporate an impure NaCl solution to separate soluble salt from insoluble im￾purities. • Primitive tasks: Transfer solid NaCl into a beaker of water → Press the heater button to…
Figure 10
Figure 10. Figure 10: Visualization of evaporation of NaCl solution. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Visualization of acid-base neutralization with phenolphthalein indicator. [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 12
Figure 12. Figure 12: Visualization of primitive task generalization. [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: Visualization of combination reaction. (b) Decomposition Reaction: RoboChemist performs a catalytic decomposition of hydrogen perox￾ide (H2O2) by introducing Manganese(II) hydroxide (Mn(OH)2) as a catalyst. The reaction proceeds as: 2H2O2 Mn(OH)2 −−−−−→ 2H2O + O2 ↑ Up…
Figure 14
Figure 14. Figure 14: Visualization of decomposition reaction. [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: Visualization of displacement reaction between Zn and HCl. [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]
Figure 16
Figure 16. Figure 16: Visualization of primitive task generalization. [PITH_FULL_IMAGE:figures/full_fig_p031_16.png]
Figure 17
Figure 17. Figure 17: Error breakdown. Platform Clutter Occlusion Light Background Spatial 0.2 0.4 0.6 0.8 1.0 Ours 0 [PITH_FULL_IMAGE:figures/full_fig_p032_17.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AEGIS: Assay-Aware Protocol Validation and Runtime Monitoring for Open-Source Liquid Handling Robots

    cs.RO 2026-07 conditional novelty 6.0 of 10

    AEGIS combines a rule-guided LLM protocol validator with a PCA/VLM visual runtime monitor to catch silent liquid-handling failures on the Opentrons OT-2, reporting adjusted F1 0.97 and average precision 0.89 on small ...

  2. Long-Term Memory for VLA-based Agents in Open-World Task Execution

    cs.RO 2026-04 unverdicted novelty 6.0 of 10

    ChemBot adds dual-layer memory and future-state asynchronous inference to VLA models, enabling better long-horizon success in chemical lab automation on collaborative robots.

Reference graph

Works this paper leans on

129 extracted references · 35 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Burger, P

    B. Burger, P. M. Maffettone, V . V . Gusev, C. M. Aitchison, Y . Bai, X. Wang, X. Li, B. M. Alston, B. Li, R. Clowes, et al. A mobile robotic chemist.Nature, 583(7815):237–241, 2020

    2020

  2. [2]

    N. J. Szymanski, B. Rendy, Y . Fei, R. E. Kumar, T. He, D. Milsted, M. J. McDermott, M. Gal- lant, E. D. Cubuk, A. Merchant, et al. An autonomous laboratory for the accelerated synthesis of novel materials.Nature, 624(7990):86–91, 2023

    2023

  3. [3]

    T. Dai, S. Vijayakrishnan, F. T. Szczypi ´nski, J.-F. Ayme, E. Simaei, T. Fellowes, R. Clowes, L. Kotopanov, C. E. Shields, Z. Zhou, et al. Autonomous mobile robots for exploratory syn- thetic chemistry.Nature, pages 1–8, 2024

    2024

  4. [4]

    D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes. Autonomous chemical research with large language models.Nature, 624(7992):570–578, 2023

    2023

  5. [5]

    Steiner, J

    S. Steiner, J. Wolf, S. Glatzel, A. Andreou, J. M. Granda, G. Keenan, T. Hinkley, G. Aragon- Camarasa, P. J. Kitson, D. Angelone, et al. Organic synthesis in a modular robotic system driven by a chemical programming language.Science, 363(6423):eaav2211, 2019

    2019

  6. [6]

    S. H. M. Mehr, M. Craven, A. I. Leonov, G. Keenan, and L. Cronin. A universal system for digitization and automatic execution of the chemical synthesis literature.Science, 370(6512): 101–108, 2020

    2020

  7. [7]

    C. W. Coley, D. A. Thomas III, J. A. Lummiss, J. N. Jaworski, C. P. Breen, V . Schultz, T. Hart, J. S. Fishman, L. Rogers, H. Gao, et al. A robotic platform for flow synthesis of organic compounds informed by ai planning.Science, 365(6453):eaax1566, 2019

    2019

  8. [8]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

Show all 129 references
  1. [9]

    H. Liu, C. Li, Y . Li, and Y . J. Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  2. [10]

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, page...

    2024

  3. [11]

    S. Tong, E. Brown, P. Wu, S. Woo, M. Middepogu, S. C. Akula, J. Yang, S. Yang, A. Iyer, X. Pan, A. Wang, R. Fergus, Y . LeCun, and S. Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal llms.Advances in Neural Information Processing Systems, 37: 87310–87356, 2024

    2024

  4. [12]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  5. [13]

    X. Liu, B. Tian, Z. Wang, R. Wang, K. Sheng, B. Zhang, H. Zhao, and G. Zhou. Delving into shape-aware zero-shot semantic segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2999–3009, 2023

    2023

  6. [14]

    P. Li, B. Tian, Y . Shi, X. Chen, H. Zhao, G. Zhou, and Y .-Q. Zhang. Toist: Task oriented instance segmentation transformer with noun-pronoun distillation.Advances in Neural Infor- mation Processing Systems, 35:17597–17611, 2022

    2022

  7. [15]

    B. Jin, Y . Zheng, P. Li, W. Li, Y . Zheng, S. Hu, X. Liu, J. Zhu, Z. Yan, H. Sun, et al. Tod3cap: Towards 3d dense captioning in outdoor scenes. InEuropean Conference on Computer Vision, pages 367–384. Springer, 2024

    2024

  8. [16]

    Chi, H.-a

    H. Chi, H.-a. Gao, Z. Liu, J. Liu, C. Liu, J. Li, K. Yang, Y . Yu, Z. Wang, W. Li, et al. Impromptu vla: Open weights and open data for driving vision-language-action models.arXiv preprint arXiv:2505.23757, 2025

    2025 arXiv

  9. [17]

    K. Ding, B. Chen, Y . Su, H.-a. Gao, B. Jin, C. Sima, W. Zhang, X. Li, P. Barsch, H. Li, et al. Hint-ad: Holistically aligned interpretability in end-to-end autonomous driving.arXiv preprint arXiv:2409.06702, 2024

    2024 arXiv

  10. [18]

    M. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. Openvla: An open-source vision-language-action model.arXiv preprint arXi...

    2024 arXiv

  11. [19]

    Ghosh, H

    Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, T. Kreiman, Y . Tan, L. Y . Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine. Octo: An open-source generalist robot policy. InProceedings of Robotics: ...

    2024

  12. [20]

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation.International Conference on Learning Representations, 2025

    2025

  13. [21]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Haus- man, B. Ichter, et al.π 0: A vision-language-action flow model for general robot control. In Conference on Robot Learning. PMLR, 2024

    2024

  14. [22]

    Pertsch, K

    K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    2025 arXiv

  15. [23]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al.π 0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025

    2025 arXiv

  16. [24]

    Huang, C

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models.arXiv preprint arXiv:2307.05973, 2023

    2023 arXiv

  17. [25]

    Huang, C

    W. Huang, C. Wang, Y . Li, R. Zhang, and L. Fei-Fei. Rekep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation.arXiv preprint arXiv:2409.01652, 2024. 10

    2024 arXiv

  18. [26]

    Y . R. Wang, Y . Zhao, H. Xu, S. Eppel, A. Aspuru-Guzik, F. Shkurti, and A. Garg. Mv- trans: Multi-view perception of transparent objects. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 3771–3778. IEEE, 2023

    2023

  19. [27]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020

    1901

  20. [28]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  21. [29]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi `ere, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  22. [30]

    Touvron, L

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  23. [31]

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  24. [32]

    X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu, et al. Deepseek llm: Scaling open-source language models with longtermism.arXiv preprint arXiv:2401.02954, 2024

    2024 arXiv

  25. [33]

    Zhang, X

    Z. Zhang, X. Li, S. Zou, G. Chi, S. Li, X. Qiu, G. Wang, G. Zheng, L. Wang, H. Zhao, et al. Chameleon: Fast-slow neuro-symbolic lane topology extraction.arXiv preprint arXiv:2503.07485, 2025

    2025 arXiv

  26. [34]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, Q. Vuong, V . Vanhoucke, H. Tran, R. Soricut, A. Singh, J. Singh, P. Sermanet, P. R. San- keti, G. Salazar, M. S. Ryoo, K. Reymann, K. Rao, K. Pertsch, I. Mordatch, H. Michalewski, Y ....

    2023

  27. [35]

    Mandlekar, Y

    A. Mandlekar, Y . Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay, et al. Roboturk: A crowdsourcing platform for robotic skill learning through imita- tion. InConference on Robot Learning, pages 879–893. PMLR, 2018

    2018

  28. [36]

    Ebert, Y

    F. Ebert, Y . Yang, K. Schmeckpeper, B. Bucher, G. Georgakis, K. Daniilidis, C. Finn, and S. Levine. Bridge data: Boosting generalization of robotic skills with cross-domain datasets. InRobotics: Science and Systems, New York City, USA, 2022

    2022

  29. [37]

    O. X.-E. Collaboration, A. O’Neill, A. Rehman, A. Gupta, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, A. Tung, A. Bewley, A. Her- zog, A. Irpan, A. Khazatsky, A. Rai, A. Gupta, A. Wang, A. Kolobov, A. Singh, A. Garg, A. Kembhavi, A. ...

    2024

  30. [38]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, P. D. Fagan, J. Hejna, M. Itkina, M. Lepert, Y . J. Ma, P. T. Miller, J. Wu, S. Belkhale, S. Dass, H. Ha, A. Jain, A. Lee, Y . Lee, M. Memmel, S. Pa...

    2024

  31. [39]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. Joshi, R. Ju- lian, D. Kalashnikov, Y . Kuang, I. Leal, K.-H. Lee, S. Levine, Y . Lu, U. Malla, D. ...

    2022 arXiv

  32. [40]

    AgiBot-World-Contributors, Q. Bu, J. Cai, L. Chen, X. Cui, Y . Ding, S. Feng, S. Gao, X. He, X. Hu, X. Huang, S. Jiang, Y . Jiang, C. Jing, H. Li, J. Li, C. Liu, Y . Liu, Y . Lu, J. Luo, P. Luo, Y . Mu, Y . Niu, Y . Pan, J. Pang, Y . Qiao, G. Ren, C. Ruan, J. Shan, Y . Shen, C...

    2025 arXiv

  33. [41]

    X. Li, M. Liu, H. Zhang, C. Yu, J. Xu, H. Wu, C. Cheang, Y . Jing, W. Zhang, H. Liu, et al. Vision-language foundation models as effective robot imitators.International Conference on Learning Representations, 2024

    2024

  34. [42]

    Huang, S

    J. Huang, S. Yong, X. Ma, X. Linghu, P. Li, Y . Wang, Q. Li, S.-C. Zhu, B. Jia, and S. Huang. An embodied generalist agent in 3D world. InProceedings of the 41st International Conference on Machine Learning, pages 20413–20451. PMLR, 2024

    2024

  35. [43]

    Durante, B

    Z. Durante, B. Sarkar, R. Gong, R. Taori, Y . Noda, P. Tang, E. Adeli, S. K. Lakshmikanth, K. Schulman, A. Milstein, et al. An interactive agent foundation model.arXiv preprint arXiv:2402.05929, 2024

    2024 arXiv

  36. [44]

    Zheng, Y

    R. Zheng, Y . Liang, S. Huang, J. Gao, H. Daum ´e III, A. Kolobov, F. Huang, and J. Yang. Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies.arXiv preprint arXiv:2412.10345, 2024

    2024 arXiv

  37. [45]

    Zheng, J

    J. Zheng, J. Li, D. Liu, Y . Zheng, Z. Wang, Z. Ou, Y . Liu, J. Liu, Y .-Q. Zhang, and X. Zhan. Universal actions for enhanced embodied foundation models.arXiv preprint arXiv:2501.10105, 2025

    2025 arXiv

  38. [46]

    X. Li, P. Li, M. Liu, D. Wang, J. Liu, B. Kang, X. Ma, T. Kong, H. Zhang, and H. Liu. Towards generalist robot policies: What matters in building vision-language-action models. arXiv preprint arXiv:2412.14058, 2024

    2024 arXiv

  39. [47]

    J. Wen, Y . Zhu, J. Li, M. Zhu, Z. Tang, K. Wu, Z. Xu, N. Liu, R. Cheng, C. Shen, et al. Tinyvla: Towards fast, data-efficient vision-language-action models for robotic manipulation. IEEE Robotics and Automation Letters, 2025

    2025

  40. [48]

    Z. Wu, Y . Zhou, X. Xu, Z. Wang, and H. Yan. Momanipvla: Transferring vision-language- action models for general mobile manipulation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  41. [49]

    Jiang, Y

    A. Jiang, Y . Gao, Z. Sun, Y . Wang, J. Wang, J. Chai, Q. Cao, Y . Heng, H. Jiang, Y . Dong, et al. Diffvla: Vision-language guided diffusion planning for autonomous driving.arXiv preprint arXiv:2505.19381, 2025

    2025 arXiv

  42. [50]

    Q. Li, Y . Liang, Z. Wang, L. Luo, X. Chen, M. Liao, F. Wei, Y . Deng, S. Xu, Y . Zhang, et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650, 2024

    2024 arXiv

  43. [51]

    D. Qu, H. Song, Q. Chen, Y . Yao, X. Ye, Y . Ding, Z. Wang, J. Gu, B. Zhao, D. Wang, et al. Spatialvla: Exploring spatial representations for visual-language-action model.arXiv preprint arXiv:2501.15830, 2025

    2025 arXiv

  44. [52]

    K. Ding, B. Chen, R. Wu, Y . Li, Z. Zhang, H.-a. Gao, S. Li, G. Zhou, Y . Zhu, H. Dong, et al. Preafford: Universal affordance-based pre-grasping for diverse objects and environments. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 727...

    2024

  45. [53]

    J. Liu, M. Liu, Z. Wang, P. An, X. Li, K. Zhou, S. Yang, R. Zhang, Y . Guo, and S. Zhang. Robomamba: Efficient vision-language-action model for robotic reasoning and manipulation. Advances in Neural Information Processing Systems, 37:40085–40110, 2024

    2024

  46. [54]

    H. Zhen, X. Qiu, P. Chen, J. Yang, X. Yan, Y . Du, Y . Hong, and C. Gan. 3D-VLA: A 3D vision- language-action generative world model. InProceedings of the 41st International Conference on Machine Learning, pages 61229–61245. PMLR, 2024. 13

    2024

  47. [55]

    Bjorck, F

    J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    2025 arXiv

  48. [56]

    G. R. Team, S. Abeyruwan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, T. Armstrong, A. Balakr- ishna, R. Baruch, M. Bauza, M. Blokzijl, et al. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    2025 arXiv

  49. [57]

    J. Liu, H. Chen, P. An, Z. Liu, R. Zhang, C. Gu, X. Li, Z. Guo, S. Chen, M. Liu, et al. Hy- bridvla: Collaborative diffusion and autoregression in a unified vision-language-action model. arXiv preprint arXiv:2503.10631, 2025

    2025 arXiv

  50. [58]

    A. Bar, Y . Gandelsman, T. Darrell, A. Globerson, and A. Efros. Visual prompting via image inpainting.Advances in Neural Information Processing Systems, 35:25005–25017, 2022

    2022

  51. [59]

    M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim. Visual prompt tuning. InEuropean conference on computer vision, pages 709–727. Springer, 2022

    2022

  52. [60]

    J. Yang, H. Zhang, F. Li, X. Zou, C. Li, and J. Gao. Set-of-mark prompting unleashes extraor- dinary visual grounding in gpt-4v.arXiv preprint arXiv:2310.11441, 2023

    2023 arXiv

  53. [61]

    S. Yoo, E. Kim, D. Jung, J. Lee, and S. Yoon. Improving visual prompt tuning for self- supervised vision transformers. InInternational Conference on Machine Learning, pages 40075–40092. PMLR, 2023

    2023

  54. [62]

    W. Liu, X. Shen, C.-M. Pun, and X. Cun. Explicit visual prompting for low-level structure segmentations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19434–19445, 2023

    2023

  55. [63]

    F. Li, Q. Jiang, H. Zhang, T. Ren, S. Liu, X. Zou, H. Xu, H. Li, J. Yang, C. Li, et al. Visual in-context prompting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12861–12871, 2024

    2024

  56. [64]

    M. Cai, H. Liu, S. K. Mustikovela, G. P. Meyer, Y . Chai, D. Park, and Y . J. Lee. Vip-llava: Making large multimodal models understand arbitrary visual prompts. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12914–12923, 2024

    2024

  57. [65]

    C. Xu, Y . Zhu, H. Shen, B. Chen, Y . Liao, X. Chen, and L. Wang. Progressive visual prompt learning with contrastive feature re-formation.International Journal of Computer Vision, 133 (2):511–526, 2025

    2025

  58. [66]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  59. [67]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  60. [68]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021

    2021

  61. [69]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. De- hghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transform- ers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    2010 arXiv

  62. [70]

    Moenning and N

    C. Moenning and N. A. Dodgson. Fast marching farthest point sampling. Technical report, University of Cambridge, Computer Laboratory, 2003. 14

    2003

  63. [71]

    Krishna and M

    K. Krishna and M. N. Murty. Genetic k-means algorithm.IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 29(3):433–439, 1999

    1999

  64. [72]

    Z. Liu, M. Zhang, and Y . Li. Kuda: Keypoints to unify dynamics learning and visual prompting for open-vocabulary robotic manipulation.arXiv preprint arXiv:2503.10546, 2025

    2025 arXiv

  65. [73]

    K. Fang, F. Liu, P. Abbeel, and S. Levine. Moka: Open-world robotic manipulation through mark-based visual prompting.Robotics: Science and Systems (RSS), 2024

    2024

  66. [74]

    Harazono, H

    Y . Harazono, H. Shimono, K. Hata, T. Mitsuyama, and T. Horinouchi. Evaluation of microplate handling accuracy for applying robotic arms in laboratory automation.SLAS technology, 29 (6):100200, 2024

    2024

  67. [75]

    Yoshikawa, A

    N. Yoshikawa, A. Z. Li, K. Darvish, Y . Zhao, H. Xu, A. Kuramshin, A. Aspuru-Guzik, A. Garg, and F. Shkurti. Chemistry lab automation via constrained task and motion planning.arXiv preprint arXiv:2212.09672, 2022

    2022 arXiv

  68. [76]

    Darvish, M

    K. Darvish, M. Skreta, Y . Zhao, N. Yoshikawa, S. Som, M. Bogdanovic, Y . Cao, H. Hao, H. Xu, A. Aspuru-Guzik, et al. Organa: a robotic assistant for automated chemistry experimentation and characterization.Matter, 8(2), 2025

    2025

  69. [77]

    Fakhruldeen, G

    H. Fakhruldeen, G. Pizzuto, J. Glowacki, and A. I. Cooper. Archemist: Autonomous robotic chemistry system architecture. In2022 International Conference on Robotics and Automation (ICRA), pages 6013–6019. IEEE, 2022

    2022

  70. [78]

    Knobbe, H

    D. Knobbe, H. Zwirnmann, M. Eckhoff, and S. Haddadin. Core processes in intelligent robotic lab assistants: Flexible liquid handling. In2022 IEEE/RSJ international conference on intelli- gent robots and systems (IROS), pages 2335–2342. IEEE, 2022

    2022

  71. [79]

    Schober, R

    D. Schober, R. G ¨uldenring, J. Love, and L. Nalpantidis. Vision-based robot manipulation of transparent liquid containers in a laboratory setting. In2025 IEEE/SICE International Sympo- sium on System Integration (SII), pages 1193–1200. IEEE, 2025

    2025

  72. [80]

    S. Li, Y . Huang, C. Guo, T. Wu, J. Zhang, L. Zhang, and W. Ding. Chemistry3d: Robotic interaction benchmark for chemistry experiments.arXiv preprint arXiv:2406.08160, 2024

    2024 arXiv

  73. [81]

    W. Zhao, P. Ding, Z. Min, Z. Gong, S. Bai, H. Zhao, and D. Wang. Vlas: Vision-language- action model with speech instructions for customized robot manipulation. InThe Thirteenth International Conference on Learning Representations

  74. [82]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705, 2023

    2023 arXiv

  75. [83]

    Hurst, A

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  76. [84]

    W. Kuo, Y . Cui, X. Gu, A. Piergiovanni, and A. Angelova. F-vlm: Open-vocabulary object detection upon frozen vision and language models.arXiv preprint arXiv:2209.15639, 2022. 15 RoboChemist: Long-Horizon and Safety-Compliant Robotic Chemical Experimentation A Appendix In this...

    2022 arXiv

  77. [85]

    •Primitive tasks:Pour one beaker of liquid into another

    Mixing NaCl and CuSO4 Solutions •Objective:Mix sodium chloride solution with copper sulfate solution and observe the resulting color change. •Primitive tasks:Pour one beaker of liquid into another. •Observation and ExplanationThis is a coordination reaction in which hydrated c...

  78. [86]

    •Primitive tasks:Grasp the test tube containing Cu(OH) 2 →Heat over flame

    Thermal Decomposition of Cu(OH)2 •Objective:Heat a test tube containing solid Cu(OH) 2 and observe the resulting decompo- sition process. •Primitive tasks:Grasp the test tube containing Cu(OH) 2 →Heat over flame. •Explanation and Observation:Copper(II) hydroxide (Cu(OH) 2) is ...

  79. [87]

    •Primitive tasks:Grasp platinum wire→Dip into CuSO 4 solution→Heat platinum wire in flame

    Flame Test of CuSO4 Solution •Objective:Identify the presence of Cu 2+ ions in copper sulfate through a flame test, inves- tigating its physical property via characteristic flame emission. •Primitive tasks:Grasp platinum wire→Dip into CuSO 4 solution→Heat platinum wire in flam...

  80. [88]

    •Primitive tasks:Transfer solid NaCl into a beaker of water→Press the heater button to initiate evaporation

    Evaporation of NaCl Solution •Objective:Evaporate an impure NaCl solution to separate soluble salt from insoluble im- purities. •Primitive tasks:Transfer solid NaCl into a beaker of water→Press the heater button to initiate evaporation. 4 •Explanation and Observation:The prima...

  81. [89]

    Acid-Base Neutralization with Phenolphthalein Indicator •Objective:Neutralize a sodium hydroxide (NaOH) solution by gradually adding hy- drochloric acid (HCl) until the solution reaches neutrality, as indicated by phenolphthalein. •Primitive tasks:Transfer solid NaOH into a be...

  82. [90]

    The items, from left to right, are: •A beaker with sodium chloride solution

    Mixing NaCl and CuSO4 Solutions You are a lab assistant tasked with mixing NaCl and CuSO4 solutions to form sodium tetrachlorocuprate using the materials shown in the image. The items, from left to right, are: •A beaker with sodium chloride solution. •A beaker with copper sulf...

  83. [91]

    The items, from left to right, are: •A lit alcohol lamp

    Thermal Decomposition of Cu(OH)2 You are a lab assistant tasked with performing the thermal decomposition of copper(II) hydroxide using the materials shown in the image. The items, from left to right, are: •A lit alcohol lamp. •A test tube containing copper(II) hydroxide. Task...

  84. [92]

    The items, from left to right, are: •A lit alcohol lamp

    Flame Test of CuSO4 Solution 6 You are a lab assistant tasked with performing the flame test of copper(II) hydroxide using the materials shown in the image. The items, from left to right, are: •A lit alcohol lamp. •Platinum wire. •A test tube containing copper(II) hydroxide. T...

  85. [93]

    The items, from left to right, are: •An evaporator with a power button

    Evaporation of NaCl Solution You are a lab assistant tasked with evaporating an impure NaCl solution to separate soluble salt from insoluble impurities using the materials shown in the image. The items, from left to right, are: •An evaporator with a power button. •A breaker wi...

  86. [94]

    type": "box

    Acid-Base Neutralization with Phenolphthalein Indicator You are a lab assistant tasked with performing an acid-base neutrali- zation reaction using the materials shown in the image. The items, from left to right, are: •A beaker with NaOH solid. •A beaker with water and a glass...

  87. [95]

    The [COLOR] bounding box surrounds the glass rod, indicating the region of interest

    "In the image input, the last image is used as a reference image, with the [COLOR] target point being the location where the robotic gripper grasps the glass rod. The [COLOR] bounding box surrounds the glass rod, indicating the region of interest. Using the right arm of the ro...

  88. [96]

    The [COLOR] bounding box encloses the glass rod, marking the region to focus on

    "In the last image of the input sequence, the [COLOR] target point indicates the designated grasp location on the glass rod. The [COLOR] bounding box encloses the glass rod, marking the region to focus on. Using the right manipulator, precisely approach and grasp the rod at th...

  89. [97]

    The [COLOR] bounding box highlights the glass rod’s region of interest

    "Refer to the last image provided, in which the [COLOR] target point specifies the grasp location on the glass rod. The [COLOR] bounding box highlights the glass rod’s region of interest. The right robotic arm should be used to perform a precise and stable grasp at the indicat...

  90. [98]

    The [COLOR] bounding box defines the region of the glass rod

    "As shown in the final image input, the [COLOR] point represents the target location for grasping the glass rod. The [COLOR] bounding box defines the region of the glass rod. Utilize the right arm of the robot to perform a careful and firm grasp at this location, ensuring the ...

  91. [99]

    The [COLOR] bounding box clearly identifies the region of the glass rod

    "The last image in the input sequence provides the reference for grasping, with the [COLOR] point indicating the target position on the glass rod. The [COLOR] bounding box clearly identifies the region of the glass rod. The task is to control the robot’s right arm to grasp the...

  92. [100]

    Using the 12 right robotic arm, hold the platinum wire and carefully extend it into the outer flame of the Bunsen burner until it glows red-hot."

    "In the image input, the last image is used as a reference image, with the [COLOR] target point for the platinum wire head to extend into the alcohol burner flame. Using the 12 right robotic arm, hold the platinum wire and carefully extend it into the outer flame of the Bunsen...

  93. [101]

    The right robotic manipulator is used to securely hold the wire and extend it into the outer flame region until red-hot."

    "The final image in the input serves as a reference, with the [COLOR] marker specifying the target location for introducing the platinum wire tip into the Bunsen burner flame. The right robotic manipulator is used to securely hold the wire and extend it into the outer flame re...

  94. [102]

    The robot’s right arm should be used to hold the wire and steadily guide it into the outer flame until visible incandescence is achieved."

    "Refer to the last input image, where the [COLOR] target point marks the location for inserting the platinum wire tip into the flame of the Bunsen burner. The robot’s right arm should be used to hold the wire and steadily guide it into the outer flame until visible incandescen...

  95. [103]

    The right robotic arm is tasked with holding the wire and positioning it within the outer flame zone until it becomes red-hot."

    "In the last image provided, the [COLOR] point indicates the desired position for extending the platinum wire tip into the Bunsen burner flame. The right robotic arm is tasked with holding the wire and positioning it within the outer flame zone until it becomes red-hot."

  96. [104]

    The right arm of the robot is employed to grasp and insert the wire into the outer flame carefully, heating it until it glows red." 3.Inserting Platinum Wire into Solution:

    "The [COLOR] marker in the final input image denotes the target region for positioning the platinum wire head within the Bunsen burner flame. The right arm of the robot is employed to grasp and insert the wire into the outer flame carefully, heating it until it glows red." 3.I...

  97. [105]

    Using the right robotic arm, carefully grasp the platinum wire and gently extend it into the beaker to dip it into the liquid."

    "In the last image, the [COLOR] bounding box surrounds the beaker and the [COLOR] target point marks the liquid level inside it. Using the right robotic arm, carefully grasp the platinum wire and gently extend it into the beaker to dip it into the liquid."

  98. [106]

    The right robotic arm is used to securely hold the platinum wire and gently insert it into the liquid up to the specified depth."

    "The last image in the input sequence serves as a reference, where the [COLOR] bounding box outlines the beaker and the [COLOR] marker denotes the target liquid level inside it. The right robotic arm is used to securely hold the platinum wire and gently insert it into the liqu...

  99. [107]

    The robot’s right manipulator is employed to grasp the platinum wire and immerse it into the liquid to the designated level."

    "As shown in the final input image, the [COLOR] bounding box highlights the beaker and the [COLOR] target point indicates the liquid surface level. The robot’s right manipulator is employed to grasp the platinum wire and immerse it into the liquid to the designated level."

  100. [108]

    The platinum wire is held by the right robotic arm and is carefully dipped into the liquid accordingly."

    "Refer to the last image in the input, where the [COLOR] bounding box encloses the beaker and the [COLOR] target point represents the desired immersion depth corresponding to the liquid level. The platinum wire is held by the right robotic arm and is carefully dipped into the ...

  101. [109]

    The right robotic arm is used to delicately lower the wire into the beaker until the required depth is reached." 4.Pouring Liquid: 13

    "In the final image of the input, the [COLOR] bounding box frames the beaker and the [COLOR] point indicates the liquid level to which the platinum wire should be submerged. The right robotic arm is used to delicately lower the wire into the beaker until the required depth is ...

  102. [110]

    The [COLOR] point on the right beaker indicates the position for the robotic arm to grasp the beaker

    "In the last image, the [COLOR] bounding box around the left beaker and the [COLOR] bounding box around the right beaker are shown, each containing its respective [COLOR] grasp point (one on the left beaker, one on the right). The [COLOR] point on the right beaker indicates th...

  103. [111]

    The [COLOR] marker on the right beaker denotes the designated grasp location

    "The last image serves as a reference, showing the [COLOR] bounding box around the left beaker, the [COLOR] bounding box around the right beaker, and their corresponding [COLOR] grasp points. The [COLOR] marker on the right beaker denotes the designated grasp location. The rob...

  104. [112]

    The [COLOR] point on the right beaker indicates where to grasp

    "In the final image, you can see the [COLOR] bounding box around the left beaker and the [COLOR] bounding box around the right beaker, each highlighting a [COLOR] grasp point. The [COLOR] point on the right beaker indicates where to grasp. The robot uses its left manipulator t...

  105. [113]

    The [COLOR] marker on the right beaker identifies the designated grasp position

    "Refer to the last image, which shows a [COLOR] bounding box around the left beaker and a [COLOR] bounding box around the right beaker, each with an associated [COLOR] point. The [COLOR] marker on the right beaker identifies the designated grasp position. The dual-arm system c...

  106. [114]

    The [COLOR] point on the right beaker indicates where to grasp

    "In the final image of the input, the [COLOR] bounding box around each beaker and their corresponding [COLOR] grasp points are displayed (one on the left, one on the right). The [COLOR] point on the right beaker indicates where to grasp. The robot is instructed to use its left...

  107. [115]

    In the last image, the [COLOR] bounding box highlights the beaker. Use the right arm to grasp the spatula and stir inside that box

    "In the last image, the [COLOR] bounding box highlights the beaker. Use the right arm to grasp the spatula and stir inside that box."

  108. [116]

    The final image shows a [COLOR] box around the beaker. Command the right arm to pick up the spatula and stir within this box

    "The final image shows a [COLOR] box around the beaker. Command the right arm to pick up the spatula and stir within this box." 14

  109. [117]

    In the last frame, a [COLOR] bounding box encloses the beaker. Have the right manipulator grasp the spatula and stir inside that region

    "In the last frame, a [COLOR] bounding box encloses the beaker. Have the right manipulator grasp the spatula and stir inside that region."

  110. [118]

    Referencing the last image, you’ll see a [COLOR] box around the beaker. Instruct the right arm to hold the spatula and stir within the boxed area

    "Referencing the last image, you’ll see a [COLOR] box around the beaker. Instruct the right arm to hold the spatula and stir within the boxed area."

  111. [119]

    In the final image, a single [COLOR] bounding box marks the beaker. Use the right arm to grasp the spatula and stir inside the box

    "In the final image, a single [COLOR] bounding box marks the beaker. Use the right arm to grasp the spatula and stir inside the box." 6.Transferring Solid:

  112. [120]

    The left [COLOR] point is where to scoop solid; the right [COLOR] point marks the liquid surface

    "In the last image, the [COLOR] boxes highlight the left (solid) and right (liquid) cups, each with a [COLOR] point. The left [COLOR] point is where to scoop solid; the right [COLOR] point marks the liquid surface. Use the right arm to grasp the spatula, scoop at the left cup’...

  113. [121]

    With the right arm, grasp the spatula, scoop at the left [COLOR] point, then deposit into the right [COLOR] point."

    "The last image shows [COLOR] boxes around both cups and [COLOR] markers for scoop and pour points|the left for solid, the right for liquid. With the right arm, grasp the spatula, scoop at the left [COLOR] point, then deposit into the right [COLOR] point."

  114. [122]

    The right manipulator holds the spatula, scoops at the left [COLOR] point, and pours at the right [COLOR] point."

    "In the final image, two [COLOR] boxes enclose the cups, each with a [COLOR] point: left for scooping solid, right for the liquid level. The right manipulator holds the spatula, scoops at the left [COLOR] point, and pours at the right [COLOR] point."

  115. [123]

    The right arm grabs the spatula, scoops at the left [COLOR] point, and delivers into the right [COLOR] point."

    "Refer to the last image’s [COLOR] boxes and [COLOR] points|left at the solid’s scoop location, right at the liquid level. The right arm grabs the spatula, scoops at the left [COLOR] point, and delivers into the right [COLOR] point."

  116. [124]

    Use the right arm to pick up the spatula, scoop at the left [COLOR] point, and transfer into the right [COLOR] point." 7.Pressing a Button:

    "In the final image, [COLOR] boxes and points mark the scoop (left) and pour (right) locations. Use the right arm to pick up the spatula, scoop at the left [COLOR] point, and transfer into the right [COLOR] point." 7.Pressing a Button:

  117. [125]

    Using the right arm of the robotic arm, carefully extend to the red switch and flick it to the left to turn it on."

    "In the image input, the last image is used as a reference image, with the [COLOR] target point indicating the location of the switch. Using the right arm of the robotic arm, carefully extend to the red switch and flick it to the left to turn it on."

  118. [126]

    Using the right robotic arm, the system carefully extends toward the switch and flicks it to the left to activate it."

    "In the final image of the input, the [COLOR] target point indicates the location of the switch. Using the right robotic arm, the system carefully extends toward the switch and flicks it to the left to activate it."

  119. [127]

    The right manipulator is employed to approach the switch and toggle it leftward to turn it on."

    "The last image in the input sequence serves as a reference, where the [COLOR] marker denotes the switch position. The right manipulator is employed to approach the switch and toggle it leftward to turn it on."

  120. [128]

    The robot’s right arm is tasked with extending to the switch and flipping it to the left to power it on." 15

    "Refer to the last image in the input, where the [COLOR] point marks the switch location. The robot’s right arm is tasked with extending to the switch and flipping it to the left to power it on." 15

  121. [129]

    "In the final reference image, the [COLOR] marker identifies the location of the switch. The robotic system extends its right arm to engage the switch by flicking it to the left, thereby switching it on." A.8 Error Breakdown We identify core failure sources across five modules...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.