Pith. sign in

REVIEW 4 major objections 6 minor 34 references

Iterative Shaping of Multi-Particle Aggregates based on Action Trees and VLM

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The authors claim that a dual-arm robot can herd dispersed particles through a gate as one cohesive group by representing the pile's contour with a truncated Fourier series and planning pushes with an iterative action tree.

desk verdict Real integrated system and a novel action-tree planner, but the headline cohesion numbers don't match the stated metric; worth a careful revision rather than a desk reject. read the letter →

arxiv 2501.13507 v1 pith:AY7W4WAP submitted 2025-01-23 cs.RO

Hoi-Yin Lee , Peng Zhou , Anqing Duan , Chenguang Yang , David Navarro-Alarcon This is my paper · ORCID
classification cs.RO
keywords multi-particlemanipulationnon-prehensileFouriershaperepresentationactiontreepathplanningvision-languagemodelherdingcohesivenessmetricdual-armrobot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a robot can herd a dispersed pile of passive particles through a gate the way a sheepdog herds a flock: by working on the group's outline and cohesion, not on each particle. The authors represent the aggregate's contour with a truncated Fourier series, compute a cohesiveness score from shape regularity and density, and generate push trajectories from an iterative action tree built from triangle centroids between the farthest particles and the gate. A vision-language model supplies high-level actions and checks for leftover particles, while an MPC refines the tool path around obstacles. Real-world trials with 74 to 140 particles show the ensemble staying connected and reaching a measured cohesiveness of about 68%, close to the 70% measured for human performance. The payoff, if true, is a model-free route to manipulating many objects at once without needing the dynamics of granular material.

What carries the argument

The central mechanism is the coupling of a Fourier-descriptor contour model with an iterative action tree. The contour of the particle group is written as a complex-valued Fourier series $f(\tau)=\sum_{n=0}^{N} c_n e^{i n 2\pi \tau/\rho}$, with $N$ kept small (5 in the experiments) so that only the macro-scale shape is represented. Cohesiveness is quantified by $\zeta$, which multiplies a shape-regularity factor (how close the contour points are to a circle around the centroid) by a density factor (particle area over group area). The action tree then converts the spatial distribution into waypoints: the five farthest particles are paired with the gate to form centroids, new centroids are recursively averaged until two remain, and Dijkstra's algorithm connects them into candidate trajectories. This hierarchy is what lets one tool push on the group's outside rather than through it, and the same contour representation feeds back into the planner after each push.

What would settle it

Repeat the experiment with a fresh pile of 100 to 140 particles and record the contour and the cohesiveness score $\zeta$ after every push. The central claim fails if the aggregate splits into two or more disconnected clusters before all particles pass the gate, or if the measured $\zeta$ is consistently no better than the direct-pushing baseline across repeated trials.

Watch

Extended reading notes

Core claim

On its own terms, the paper reports that a bimanual robot, using two T-shaped tools, can autonomously gather scattered particles and guide the whole group through a narrow gate while keeping it as one connected aggregate. The working hypothesis is that the macro-scale shape of the particle ensemble is what matters: once the contour is captured from vision and compressed into a small set of Fourier coefficients, the system can treat the group as a single deformable object. The push directions come from an iterative action tree: pick the five particles that most hurt cohesion, form triangle centroids with the gate, recursively average consecutive centroids until two remain, and connect the resulting waypoints with Dijkstra's algorithm. The paper's experiments compare this herding strategy with direct pushing, with a model predictive control baseline, and with a landmark-guided shape-formation baseline; it reports that direct pushing splits the pile into subgroups, while the proposed method completes the task with a final cohesiveness of 68%, against 70% for manual human aggregation.

Load-bearing premise

The method assumes that moving the tool along the centroid-based waypoints will push the whole particle group toward the gate as one cohesive unit, even though the paper does not model how the tool contacts or pushes the particles.

Editorial extensions

If this is right

  • A robot can transport large numbers of particles without learning a material model, as long as the group's contour can be tracked reliably.
  • Because the action tree operates on centroid geometry rather than contact physics, the same planner should transfer to other piles, such as debris, grains, or mixed-size objects, without retraining.
  • The cohesiveness score gives a scalar feedback signal that can drive the choice of which particles to push next, closing the loop between shape observation and action.
  • Using a VLM to confirm leftover particles makes the task robust to failures of color-based detection when only a few particles remain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not explore it, but the same action-tree geometry could be run in simulation with a known granular dynamics model to map when the centroid-waypoint assumption breaks down, giving a quantitative failure boundary for the method.
  • Because cohesion is computed from the contour alone, the metric could be reused as a reward for learning-based pile-manipulation policies, a use the paper does not mention.
  • The reported gap between the robot's 68% and the human's 70% may be dominated by the way the tool is moved between waypoints, not by the planning itself; testing longer MPC horizons or different tool shapes would separate these factors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a bimanual robot system for herding multi-particle aggregates through a gate by combining high-level VLM/LLM task planning with a low-level shape-control pipeline. The aggregate contour is represented by truncated Fourier series, an iterative action tree generates pushing waypoints from the centroids of triangles formed by outlying particles and the gate, an MPC refines the tool trajectory to avoid obstacles, and a cohesiveness metric ζ is introduced. The system is evaluated in real-world experiments with 74–140 particles, compared to direct pushing, an MPC baseline, a landmark-inspired method, and manual human aggregation, with the conclusion reporting a cohesiveness of 68% versus 70% for humans.

Significance. If the claims are substantiated, the paper would make a useful contribution to non-prehensile manipulation of granular aggregates: it integrates modern VLM-based symbolic planning with a compact Fourier shape descriptor and a centroid-based action tree, and it provides real-robot demonstrations with several particle counts. Strengths include the use of a physical dual-arm platform, an obstacle-avoiding MPC with a clear cost formulation, and explicit comparisons against multiple baselines and a manual human result. The significance is limited by the quantitative fragility discussed below: the headline cohesion numbers depend on a metric whose stated formula does not reproduce the reported table, and the experimental comparison lacks repetition and statistical detail. The qualitative video evidence is suggestive but does not by itself support the precise numerical claim.

major comments (4)
  1. [§II-A2, Eq. (4), and Table II] The cohesion metric as typeset cannot reproduce the tabulated regularity and cohesion values for the stated regular shapes. For a circular contour of radius R with density ratio α/β = 0.5, Eq. (4) gives ζ = (sqrt(α/π)/((1/n)Σ||p_i − mean(p)||)) × (α/β) × 100% = (R/√2)/R × 0.5 × 100% ≈ 35.4%, not the 50.0% shown in all three circle rows. The same mismatch appears in the square and rectangle rows: the tabulated regularity column is not the first factor of Eq. (4). The numbers in Table II are consistent with Cohesiveness = Regularity × Density × 100%, but the text does not define the regularity factor that way. Because Eq. (4) is also used in §II-A3 to select outlying particles for the action tree, the inconsistency propagates into both planning and evaluation. Please correct the formula, the definitions of α and β, and the table, or explicitly state the exact computation used and show that it reproduces the reported values.
  2. [§III-B and Table II] The quantitative comparison is based on single numbers with no trial counts, error bars, or statistical significance tests. The conclusion that the method 'achieves a cohesiveness of 68%, while the human performance is slightly higher at 70%' rests on a single 'Ours' column in Table II; it is not stated which of the four experimental conditions in Fig. 5 corresponds to that number, nor how many independent runs were performed for any method. The 1.5% margin over manual aggregation is smaller than the run-to-run variation expected in granular manipulation. Please provide repeated trials with means and standard deviations (or individual trial data) for the proposed method and each baseline, and identify which condition each reported number refers to.
  3. [§II-A3 and §III] The cohesion metric is used both as a planning cue and as the main success criterion: §II-A3 states that the system selects outlying particles using the cohesion metric, and §III then reports the same metric as the principal outcome. This creates a circularity burden: an algorithm that explicitly acts to maximize ζ will naturally show improvement in ζ, so the favorable cohesion results are partly by construction. The gate-crossing task provides an independent measure, but the paper should report additional task-level metrics (e.g., number of particles delivered, time to completion, obstacle collisions, whether the aggregate ever splits) and ideally blind human ratings of the video, so that the evaluation is not dominated by the very quantity the planner optimizes.
  4. [Algorithm 1 and §II-A3] The action-tree path planning is described at a high level, but Algorithm 1 omits the cohesion-based selection rule described in the text: P is an input 'set of farthest points,' but the algorithm does not say how P is chosen, how the statement 'we utilize the cohesion metric' is realized algorithmically, or how the tool-length constraint ('the distance between the points taken is less than the length of the tool segment') is enforced. The paper also provides no dynamics model or simulation supporting the central assumption that executing pushes along the computed centroid waypoints will herd the ensemble coherently. Please specify the exact selection procedure, the parameter values (e.g., the tool length used to set the number of points to 5), and provide at least a simulation or an ablation study that isolates the contribution of the action-tree ordering.
minor comments (6)
  1. [§II-A1, Eqs. (1) and (3)] Equation (1) uses ρ for the period with ρ = 2π, but Eq. (3) writes the same exponent with P, which is not defined; please use a consistent symbol. Also, the summation indices in the extracted PDF appear as 'NX' and 'Pn'; please check the typesetting.
  2. [§II-C] In the first sentence of §II-C, 'spit' should be 'split.'
  3. [§II-B and §III] The MPC formulation in Eq. (5) is clear, but the numerical values of Q and R are not reported, and the prediction horizon is given only once as H=50. Please state all tuning parameters for reproducibility.
  4. [Table II] The table would be easier to read with explicit column headers and a note explaining how 'Regularity' was computed for the regular shapes; the current layout makes it difficult to connect the density, regularity, and cohesion columns to Eq. (4).
  5. [References] Reference [28] appears to have a typo in the author name ('Navarro-Alarco' instead of 'Navarro-Alarcon'), and several references are arXiv preprints rather than published versions; please update where possible.
  6. [§III-A] The text says 'we tested various particle group sizes and shapes, ranging from small (74 particles) to large (140 particles),' but the experimental section does not state how many trials were performed for each configuration or how representative the shown runs are; please add this information.

Circularity Check

1 steps flagged · score 4.0 of 10

The cohesion metric in Eq. (4) is used as the planning objective and then reported as the headline evaluation metric; the gate-delivery task and human baseline remain independent.

  1. self definitional [Sec. II-A3 (Path Planning: Iterative Action Tree), with evaluation in Sec. III-A and Table II]
    "We utilize the cohesion metric to analyze which specific particles are contributing to a lower overall cohesion of the group. If the cohesion is observed to be low, we select the particles that are farther away from the group’s centroid for targeted manipulation."

    The same zeta defined in Eq. (4) is both the control cue and the success metric. The planner selects waypoints to raise zeta, and Sec. III then states: "we compare the cohesiveness metrics calculated using Eq. (4)" and concludes that "our methodology achieves a cohesiveness of 68%." Thus the headline cohesion number is the very quantity the action-tree rule was designed to increase; it is not an independent validation of cohesion. The independent evidence is the successful delivery through the gate and the comparison with a human baseline, which keeps the circularity partial rather than total.

full rationale

The derivation chain is mostly self-contained: the Fourier-contour representation, action-tree centroid construction, and MPC refinement do not reduce to their inputs. The main circularity burden is that Eq. (4) is used twice: once to decide which outlying particles to push (Sec. II-A3) and once to report success (Sec. III-A, Table II, conclusion). Because the planner explicitly aims to improve this metric, the 68.5% cohesion figure is partly the objective being optimized, not an independent measurement. The task completion (particles entering the gate) and the manual comparison are independent, so the paper retains meaningful non-circular content. The self-citations to [28] and [33] support the task-planner implementation but are not load-bearing for the central derivation. Separately, I note a reproducibility defect rather than circularity: Eq. (4) as typeset uses sqrt(alpha/pi) in the regularity factor, which for a circle at density 0.5 gives about 0.707, while Table II lists 1.0; the table appears to require the contour-area version sqrt(beta/pi). This inconsistency should be corrected but does not change the circularity verdict.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The system rests on several stated but unvalidated modeling choices: a camera-extracted contour is taken as the true shape, particle dynamics are ignored in planning, the tool length is assumed sufficient to sweep the farthest points, and the VLM is trusted to detect leftover particles. The action planner and the evaluator share the same cohesion metric, which is the main circularity surface. No free parameter is fitted to data in the classic sense, but N, P, H, and MPC weights are hand-chosen and not sensitivity-analyzed.

free parameters (4)
  • N: number of Fourier harmonics = 5
    Chosen in Sec III as a balance between shape fidelity and smoothness; affects how much contour detail the planner sees.
  • P: number of farthest particles selected = 5
    Set in Sec II-A3 based on tool dimensions; determines how many target points seed the action tree.
  • H: MPC prediction horizon = 50
    Set in Sec III; no sensitivity study is reported.
  • Q and R: MPC weight matrices = not reported
    Declared positive definite in Eq. (5) but values are never given, so the trajectory refinement is not exactly reproducible.
assumptions (4)
  • domain assumption The top-down camera contour faithfully encloses all particles and is a valid boundary for the aggregate.
    The entire Fourier representation and centroid computations in Sec II-A1 depend on this contour; detection errors directly change the shape and the cohesion metric.
  • domain assumption Particle dynamics can be ignored: pushing the tool along action-tree waypoints will move the aggregate without scattering.
    Sec II-A1 explicitly avoids a dynamical model and Sec II-A3 assumes the swept waypoints herd the 'sheep'; no contact or friction model validates this.
  • domain assumption The tool length is large enough to cover the selected farthest points in one sweep.
    Sec II-A3 states the distance between selected points is kept below the tool segment length to avoid losing particles, but no margin analysis is given.
  • domain assumption The VLM/LLM provides correct action selection and reliable detection of remaining particles.
    Sec II-C depends on GPT-4o for high-level actions and final check; no ablation measures the planner's error rate.
invented entities (1)
  • Cohesiveness metric zeta (Eq. 4)
    purpose: Quantifies group compactness; used both as planning objective and evaluation metric.
    It is introduced in Eq. (4) and used both to guide the planner and to score the outcome; no external benchmark connects it to task success.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Iterative Shaping of Multi-Particle Aggregates based on Action Trees and VLM." pith.science (2026). https://pith.science/paper/AY7W4WAP

@misc{pith2026250113507,
  author       = {Pith},
  title        = {Pith review of: Iterative Shaping of Multi-Particle Aggregates based on Action Trees and VLM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AY7W4WAP}},
  note         = {Machine review of arXiv:2501.13507}
}
read the original abstract

In this paper, we address the problem of manipulating multi-particle aggregates using a bimanual robotic system. Our approach enables the autonomous transport of dispersed particles through a series of shaping and pushing actions using robotically-controlled tools. Achieving this advanced manipulation capability presents two key challenges: high-level task planning and trajectory execution. For task planning, we leverage Vision Language Models (VLMs) to enable primitive actions such as tool affordance grasping and non-prehensile particle pushing. For trajectory execution, we represent the evolving particle aggregate's contour using truncated Fourier series, providing efficient parametrization of its closed shape. We adaptively compute trajectory waypoints based on group cohesion and the geometric centroid of the aggregate, accounting for its spatial distribution and collective motion. Through real-world experiments, we demonstrate the effectiveness of our methodology in actively shaping and manipulating multi-particle aggregates while maintaining high system cohesion.

Figures

Figures reproduced from arXiv: 2501.13507 by the authors.

Figure 1
Figure 1. Setup of the addressed multi-particle shaping task. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. System Structure: The system consists of a Task Planner with a Shape Manipulation and Control module. An ontological knowledge graph and [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Conceptual representation of the proposed iterative action tree. (1)–(3): [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Before the micro-scale shape refinement, the tooltip (the purple dashed [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Experiments: (a) Different shapes and amounts of particles are used to evaluate the performance of the proposed shape representation method. The [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Evolution of the cohesion: (i)–(iv) reflect the changes in density, [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 8
Figure 8. Figure 8: Performance of adopting direct pushing method: (a) 134 particles; (b) [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 24 canonical work pages

  1. [1]

    Shepherding behaviors,

    J.-M. Lien, O. B. Bayazit, R. T. Sowell, S. Rodriguez, and N. M. Amato, “Shepherding behaviors,” in IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04. 2004 , vol. 4. IEEE, 2004, pp. 4159–4164

    work page 2004

  2. [2]

    A distributed outmost push approach for multi-robot herding,

    S. Zhang, X. Lei, M. Duan, X. Peng, and J. Pan, “A distributed outmost push approach for multi-robot herding,” IEEE Transactions on Robotics, 2024

    work page 2024

  3. [3]

    Heterogeneous targets trapping with swarm robots by using adaptive density-based interaction,

    S. Zhang, X. Lei, X. Peng, and J. Pan, “Heterogeneous targets trapping with swarm robots by using adaptive density-based interaction,” IEEE Transactions on Robotics , 2024

    work page 2024

  4. [4]

    Landmark-guided shape formation by a swarm of robots,

    A. Vardy, “Landmark-guided shape formation by a swarm of robots,” in Distributed Autonomous Robotic Systems: The 14th International Symposium. Springer, 2019, pp. 371–383

    work page 2019

  5. [5]

    Collecting a flock with multiple sub-groups by using multi-robot system,

    S. Zhang and J. Pan, “Collecting a flock with multiple sub-groups by using multi-robot system,” IEEE Robotics and Automation Letters , vol. 7, no. 3, pp. 6974–6981, 2022

    work page 2022

  6. [6]

    Reactive shepherding along a dynamic path,

    S. Van Havermaet, Y . Khaluf, and P. Simoens, “Reactive shepherding along a dynamic path,” Scientific Reports, vol. 14, no. 1, p. 14915, 2024

    work page 2024

  7. [7]

    Robotic shepherding in cluttered and unknown environments using control barrier functions,

    M. Hamandi, F. Khorrami, and A. Tzes, “Robotic shepherding in cluttered and unknown environments using control barrier functions,” arXiv preprint arXiv:2407.15701 , 2024

    arXiv 2024

  8. [8]

    Dynamic- resolution model learning for object pile manipulation,

    Y . Wang, Y . Li, K. Driggs-Campbell, L. Fei-Fei, and J. Wu, “Dynamic- resolution model learning for object pile manipulation,” arXiv preprint arXiv:2306.16700, 2023

    arXiv 2023

Show all 34 references
  1. [9]

    Automated robotic manipulation of individual colloidal particles using vision-based con- trol,

    S. Zimmermann, T. Tiemerding, and S. Fatikow, “Automated robotic manipulation of individual colloidal particles using vision-based con- trol,” IEEE/ASME Transactions on Mechatronics , vol. 20, no. 5, pp. 2031–2038, 2014

    2014

  2. [10]

    The surprising effectiveness of linear models for visual foresight in object pile manipulation,

    H. T. Suh and R. Tedrake, “The surprising effectiveness of linear models for visual foresight in object pile manipulation,” in Algorithmic Foundations of Robotics XIV: Proceedings of the Fourteenth Workshop on the Algorithmic Foundations of Robotics 14 . Springer, 2021, pp. 347–363

    2021

  3. [11]

    Learning robotic manipulation of granular media,

    C. Schenck, J. Tompson, S. Levine, and D. Fox, “Learning robotic manipulation of granular media,” in Conference on Robot Learning . PMLR, 2017, pp. 239–248

    2017

  4. [12]

    Manipulation of granular materials by learning particle interactions,

    N. Tuomainen, D. Blanco-Mulero, and V . Kyrki, “Manipulation of granular materials by learning particle interactions,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 5663–5670, 2022

    2022

  5. [13]

    Excavation learning for rigid objects in clutter,

    Q. Lu and L. Zhang, “Excavation learning for rigid objects in clutter,” IEEE Robotics and Automation Letters , vol. 6, no. 4, pp. 7373–7380, 2021

    2021

  6. [14]

    Uncertainty- aware self-supervised target-mass grasping of granular foods,

    K. Takahashi, W. Ko, A. Ummadisingu, and S.-i. Maeda, “Uncertainty- aware self-supervised target-mass grasping of granular foods,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 2620–2626

    2021

  7. [15]

    Model-free vision-based shaping of deformable plastic materials,

    A. Cherubini, V . Ortenzi, A. Cosgun, R. Lee, and P. Corke, “Model-free vision-based shaping of deformable plastic materials,” The International Journal of Robotics Research , vol. 39, no. 14, pp. 1739–1759, 2020

    2020

  8. [16]

    A data-driven approach for fast simulation of robot locomotion on granular media,

    Y . Zhu, L. Abdulmajeid, and K. Hauser, “A data-driven approach for fast simulation of robot locomotion on granular media,” in2019 international conference on robotics and automation (ICRA). IEEE, 2019, pp. 7653– 7659

    2019

  9. [17]

    Inferring the material properties of granular media for robotic tasks,

    C. Matl, Y . Narang, R. Bajcsy, F. Ramos, and D. Fox, “Inferring the material properties of granular media for robotic tasks,” in 2020 ieee international conference on robotics and automation (icra) . IEEE, 2020, pp. 2770–2777

    2020

  10. [18]

    Robot collection and transport of objects: A biomimetic process,

    D. Str ¨ombom and A. J. King, “Robot collection and transport of objects: A biomimetic process,” Frontiers in Robotics and AI, vol. 5, p. 48, 2018

    2018

  11. [19]

    Learning visible connectivity dynamics for cloth smoothing,

    X. Lin, Y . Wang, Z. Huang, and D. Held, “Learning visible connectivity dynamics for cloth smoothing,” in Conference on Robot Learning . PMLR, 2022, pp. 256–266

    2022

  12. [20]

    Day- dreamer: World models for physical robot learning,

    P. Wu, A. Escontrela, D. Hafner, P. Abbeel, and K. Goldberg, “Day- dreamer: World models for physical robot learning,” in Conference on robot learning. PMLR, 2023, pp. 2226–2240

    2023

  13. [21]

    Deep visual foresight for planning robot motion,

    C. Finn and S. Levine, “Deep visual foresight for planning robot motion,” in 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017, pp. 2786–2793

    2017

  14. [22]

    Robocraft: Learning to see, simulate, and shape elasto-plastic objects in 3d with graph networks,

    H. Shi, H. Xu, Z. Huang, Y . Li, and J. Wu, “Robocraft: Learning to see, simulate, and shape elasto-plastic objects in 3d with graph networks,” The International Journal of Robotics Research, vol. 43, no. 4, pp. 533– 549, 2024

    2024

  15. [23]

    Robouniview: Visual-language model with unified view representation for robotic manipulation,

    F. Liu, F. Yan, L. Zheng, C. Feng, Y . Huang, and L. Ma, “Robouniview: Visual-language model with unified view representation for robotic manipulation,” arXiv preprint arXiv:2406.18977 , 2024

    2024 arXiv

  16. [24]

    Continuous object state recognition for cooking robots using pre-trained vision-language models and black-box optimization,

    K. Kawaharazuka, N. Kanazawa, Y . Obinata, K. Okada, and M. Inaba, “Continuous object state recognition for cooking robots using pre-trained vision-language models and black-box optimization,”IEEE Robotics and Automation Letters, 2024

    2024

  17. [25]

    Gpt-4v (ision) for robotics: Multimodal task planning from human demonstration,

    N. Wake, A. Kanehira, K. Sasabuchi, J. Takamatsu, and K. Ikeuchi, “Gpt-4v (ision) for robotics: Multimodal task planning from human demonstration,” IEEE Robotics and Automation Letters , 2024

    2024

  18. [26]

    Visual language maps for robot navigation,

    C. Huang, O. Mees, A. Zeng, and W. Burgard, “Visual language maps for robot navigation,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 10 608–10 615

    2023

  19. [27]

    Replanvlm: Replanning robotic tasks with visual language models,

    A. Mei, G.-N. Zhu, H. Zhang, and Z. Gan, “Replanvlm: Replanning robotic tasks with visual language models,” IEEE Robotics and Automa- tion Letters, 2024

    2024

  20. [28]

    Non-prehensile tool-object manipulation by integrating llm- based planning and manoeuvrability-driven controls,

    H.-Y . Lee, P. Zhou, A. Duan, W. Ma, C. Yang, and D. Navarro- Alarco, “Non-prehensile tool-object manipulation by integrating llm- based planning and manoeuvrability-driven controls,” arXiv preprint arXiv:2412.06931, 2024

    2024

  21. [29]

    A survey of robot intelligence with large language models,

    H. Jeong, H. Lee, C. Kim, and S. Shin, “A survey of robot intelligence with large language models,” Applied Sciences, vol. 14, no. 19, p. 8868, 2024

    2024

  22. [30]

    Large language models for robotics: Opportunities, challenges, and perspectives,

    J. Wang, Z. Wu, Y . Li, H. Jiang, P. Shu, E. Shi, H. Hu, C. Ma, Y . Liu, X. Wang et al. , “Large language models for robotics: Opportunities, challenges, and perspectives,” arXiv preprint arXiv:2401.04334 , 2024

    2024 arXiv

  23. [31]

    Physically grounded vision-language models for robotic manipulation,

    J. Gao, B. Sarkar, F. Xia, T. Xiao, J. Wu, B. Ichter, A. Majumdar, and D. Sadigh, “Physically grounded vision-language models for robotic manipulation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 12 462–12 469

    2024

  24. [32]

    Large language models for robotics: A survey,

    F. Zeng, W. Gan, Y . Wang, N. Liu, and P. S. Yu, “Large language models for robotics: A survey,” arXiv preprint arXiv:2311.07226 , 2023

    2023

  25. [33]

    A distributed dynamic frame- work to allocate collaborative tasks based on capability matching in heterogeneous multirobot systems,

    H.-Y . Lee, P. Zhou, B. Zhang, L. Qiu, B. Fan, A. Duan, J. Tang, T. L. Lam, and D. Navarro-Alarcon, “A distributed dynamic frame- work to allocate collaborative tasks based on capability matching in heterogeneous multirobot systems,” IEEE Transactions on Cognitive and Developm...

    2023

  26. [34]

    Model predictive interaction control for robotic manipulation tasks,

    T. Gold, A. V ¨olz, and K. Graichen, “Model predictive interaction control for robotic manipulation tasks,” IEEE Transactions on Robotics, vol. 39, no. 1, pp. 76–89, 2022

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.