Pith. sign in

REVIEW 3 major objections 6 minor 76 references

Learning to Plan with Personalized Preferences

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that an explicit preference-inference stage between demonstrations and action planning significantly improves an embodied agent's ability to produce plans that match a user's demonstrated habits, and supports this with a…

desk verdict Useful new benchmark with a clear two-stage preference recipe; the Levenshtein-to-oracle metric overstates the headline claim about personalized planning. read the letter →

arxiv 2502.00858 v3 pith:SKK6YRCS submitted 2025-02-02 cs.AI cs.HC

Manjie Xu , Xinyi Yang , Wei Liang , Chi Zhang , Yixin Zhu This is my paper · ORCID
classification cs.AIcs.HC
keywords preference-basedplanningfew-shotlearningfromdemonstrationembodiedAIpersonalizationhouseholdroboticsLevenshteindistanceevaluationtwo-stageegocentricvideobenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that an embodied assistant can learn a person's preferences from a few demonstrations and then plan actions accordingly, and that making the learned preference explicit as an intermediate label is the key to doing this well. To test the idea, they build the Preference-based Planning (PbP) benchmark, with 290 hierarchically organized preferences over daily household activities, rendered across 50 scenes, and 5,000 test instances. They compare end-to-end models, which map demonstrations directly to action sequences, with a two-stage pipeline that first predicts a preference label and then plans using it. Across vision-based and symbol-based models, the two-stage pipeline consistently yields lower Levenshtein distance to the user's demonstrated action sequence, and with ground-truth preference labels top models approach near-perfect alignment. The paper concludes that preferences are a valuable abstraction layer between goals and actions for adaptive, personalized planning.

What carries the argument

The load-bearing mechanism is a two-stage decomposition: first learn the preference representation $\hat{p}=f(\mathcal{O};\theta_f)$ from few demonstrations $\mathcal{O}=\{(S_i,A_i,M)\}$, then optimize the planning loss $\mathcal{L}=\sum_i \ell(g(s_i,\hat{p};\theta_g), a_i)$ with the preference fed into the planner. The preferences themselves are organized into three tiers defined in the benchmark: 75 action-level preferences (fine execution details), 135 option-level preferences (spatial and object choices), and 80 sequence-level preferences (task ordering). The discrete preference label functions as a bottleneck that forces the model to abstract a general rule from the demonstrations before generating actions, and the paper's ablations show that performance degrades sharply without demonstrations, indicating models rely on demonstrations to recognize the pattern.

What would settle it

Re-score the PBP test set with a metric that checks only whether generated plans satisfy the stated preference (for example, whether the preferred object category is placed in the preferred location, in any valid order), and test whether the two-stage advantage over end-to-end survives; if it does not, the reported gains are artifacts of exact-sequence scoring rather than evidence of preference-guided planning.

Watch

Extended reading notes

Core claim

The paper's central claim is that preferences, made explicit as labels, are the right intermediate abstraction for personalized embodied planning. On the authors' PBP benchmark, end-to-end models that map demonstrations directly to action sequences score close to the average ground-truth sequence length (overall Levenshtein distances of 16.95 to 25.35 across models), meaning they largely fail to capture the demonstrated preference. Adding a first stage that predicts the preference label and then planning from that label lowers overall distances to 8.22 for GPT-4.1 and 10.90 for DeepSeek-R1, with consistent gains across option-level and sequence-level tasks. When the ground-truth preference label is supplied instead of the predicted one, top models reach near-perfect alignment (GPT-4.1 overall 5.23; GPT-4V option-level 1.26 and sequence-level 11.36). The authors interpret this as evidence that preferences are a valuable abstraction layer, while also finding that video-based models struggle to infer preferences from egocentric footage and that symbol-based models generalize across scenes more robustly.

Load-bearing premise

The evaluation assumes that each preference determines exactly one correct action sequence, the one produced by the benchmark's rule-based planner, and scores every deviation from that exact sequence equally; a plan that satisfies the preference but orders actions differently or picks a different valid object is penalized as heavily as a plan that violates the preference.

Editorial extensions

If this is right

  • Two-stage preference-label planning is a practical recipe: for symbol-based models, giving predicted or ground-truth preference labels cuts Levenshtein distance to less than half of end-to-end performance on overall tasks.
  • Progress on personalized embodied assistance will likely require stronger few-shot preference inference from visual input, since video-based models lag far behind symbol-based models at predicting preference labels (e.g., GPT-4V 42.99% overall vs. GPT-4.1 79.60%).
  • The benchmark's preference hierarchy gives a reusable protocol for comparing future methods on action-level, option-level, and sequence-level personalization.
  • Ground-truth labels near-perfect planning (GPT-4.1 option-level Levenshtein 0.15) suggests that once preferences are known, current models already plan well; the bottleneck is perception and abstraction of preferences from raw observations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to have human raters judge whether plans respect a stated preference, decoupling style from correctness; the paper's Levenshtein scoring may understate models that produce valid but differently ordered plans.
  • The same two-stage abstraction could transfer beyond household tasks to any domain where few demonstrations reveal a stable user trait, such as document formatting, route selection, or interface customization, because the method is domain-agnostic in principle.
  • The vision-based models' anchoring to scene-specific features suggests a concrete recipe for improvement: pretrain or fine-tune video models on object- and scene-invariant action abstractions, or combine video input with symbolic action sequences, to close the gap with symbol-based planners.
  • Real-world uptake will require handling noisier, unlabeled human video; the benchmark's clean egocentric demonstrations likely overstate how quickly preferences can be extracted from natural human behavior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces PBP, an OmniGibson-based benchmark for few-shot preference learning and personalized planning, containing 290 hand-defined preferences at three levels (action, option, sequence), 15,000 demonstrations, 5,000 test points, and 50 scenes. It evaluates end-to-end and two-stage pipelines (preference prediction followed by planning) across vision-based and symbol-based models. The headline result is that two-stage planning with predicted preference labels substantially reduces Levenshtein distance to the benchmark's rule-based ground-truth action sequences relative to end-to-end generation, and that providing ground-truth preference labels yields near-perfect sequence matching for the strongest models.

Significance. If the central claim is established, PBP and the two-stage abstraction would be a useful contribution: the benchmark is substantially larger and more hierarchically organized than prior preference benchmarks such as NeatNet and SAND, and the systematic baseline evaluation, generalization analysis, and demonstration-number ablations provide a reusable testbed. The paper also includes a detailed dataset card and promises code and data release. The benchmark itself is valuable even if the current evaluation metric is imperfect. However, the claim that two-stage preference labels 'significantly improve' personalized planning is currently supported only by a metric that measures imitation of a single planner's action sequence, not direct satisfaction of the user's preference; the significance of the work is therefore conditional on the metric concern being addressed.

major comments (3)
  1. [§5.2, Table 1] The evaluation metric, Levenshtein distance to a single rule-based ground-truth action sequence, conflates preference satisfaction with planner style. The case study in Table A3 for the preference 'Put fruit on the bed' shows this directly: the oracle plan places the grape and banana on the bed and leaves the pencil on the table, while Llama3's plan additionally places the pencil on the bed. The stated preference is satisfied by any plan that puts the fruit on the bed; the pencil action and the order of the two fruit placements are stylistic choices, not violations. Because Levenshtein distance penalizes every extra, missing, or reordered token equally, the two-stage improvements in Table 1 (e.g., GPT-4.1 Overall 16.95 to 8.22) could reflect better imitation of the demonstration/planner style rather than better satisfaction of user preferences. The paper needs a direct preference-satisfaction metric, such as checking final object placements and ordering constraints per preference, or a human evaluation, before the headline conclusion is supported.
  2. [§5.3, Second-stage (gt) rows of Table 1] The ground-truth preference labels used in the Second-stage (gt) condition are the same latent variables that the rule-based generator in §4.2 used to produce the ground-truth action sequences. Giving the model the generative label and asking it to reproduce the generated sequence is partly an inversion of the benchmark's own generation process, so near-zero distances (e.g., GPT-4.1: 0.15 at option level, 10.31 at sequence level) are expected to the extent that the model can imitate the rule-based planner. This row should be framed as an oracle upper bound on style matching, not as standalone evidence that explicit preference labels are the correct abstraction for personalized planning.
  3. [Table 1 and Abstract] The abstract and §5.3 describe the two-stage improvements as 'significant,' but Table 1 reports only means and standard deviations with no paired significance tests, confidence intervals, or effect sizes across the 5,000 test points. Some comparisons are within one standard deviation (e.g., EILEV option-level end-to-end 12.88±2.20 vs. two-stage 12.89±3.74), so the strength of the claim is not verifiable from the reported numbers. Paired tests or bootstrap confidence intervals should be reported for the central end-to-end versus two-stage comparison.
minor comments (6)
  1. [Throughout and Appendix D] There are numerous typos in the prompt templates and main text, including 'Quesiton', 'prevous', 'tet file', and 'EILVE' in §5.5; these should be corrected.
  2. [§5.5 and Figure 5] The text refers to 'Figure 5 (c)-(d)' for the demonstration-number ablation, but Figure 5 contains only panels (a) and (b); the ablation is actually shown in Figure 7.
  3. [§4.2] The test point construction says 'typically three' demonstrations, but the default setting elsewhere and the ablation in Figure 7 vary the number from 1 to 5; the paper should state the exact default number of demonstrations used in Table 1.
  4. [Table 2] The 'Ablative' rows of Table 2 are not clearly defined in the main text; please state explicitly whether all demonstrations are omitted and what 'isolated test sequences' means for each model family.
  5. [Appendix A.2(i)] The dataset card answers 'No' to whether there are errors, sources of noise, or redundancies; for a procedurally generated synthetic dataset, a categorical 'No' is too strong and should be qualified with the known limitations of the rule-based planner and rendering pipeline.
  6. [Figure 1] Figure 1 is very dense and the right-hand pipeline is difficult to follow at print size; consider enlarging or splitting the figure.

Circularity Check

1 steps flagged · score 6.0 of 10

The Second-stage (gt) planning condition reduces by construction to inverting the benchmark's own preference-to-action generator; the predicted-label two-stage result retains independent empirical content.

  1. self definitional [Section 4.2 (test-set construction) and Section 5.3 / Table 1 (Second-stage (gt) row)]
    "To generate each demonstration, we randomly assign a preference from our defined primitives to one of 50 OmniGibson scenes, then sample relevant objects within the chosen scene. We generate multi-perspective observations using rule-based planners for high-level planning and predefined scripts for low-level execution (e.g., Inverse Kinematics (IK) for grasping, A* for movement). ... For comprehensive evaluation, we include planning results using ground truth preference labels (Second-stage (gt) row)."

    The ground-truth action sequences scored in Table 1 are the outputs of the rule-based planner that took the randomly assigned preference as its high-level input. The Second-stage (gt) condition feeds that exact preference label back to the model and evaluates the result by Levenshtein distance against the planner's own output sequence. Near-perfect alignment in this row therefore certifies that the model can invert the benchmark's preference-to-action generator, not that preferences independently determine planning: that mapping was installed when the demonstrations were rendered. The two-stage rows with predicted labels are not circular, because the label must be inferred from demonstrations with measurable error (Table 2), giving the central comparison independent empirical content.

full rationale

The paper's central two-stage claim rests on the improvement from end-to-end to two-stage planning using predicted preference labels; that comparison is empirical and not automatically forced, since the models must first infer the label from demonstrations and Table 2 shows imperfect inference. The circular element is confined to the Second-stage (gt) condition: Section 4.2 generates every ground-truth action sequence by randomly assigning a preference and running a rule-based planner, and Section 5.3 then treats a model that receives that same preference label as achieving 'almost perfect alignment' when it reproduces the planner's output. This is a self-definitional step in the argument that preferences are a valuable abstraction layer, because the label-to-action mapping was built into the data-generation process rather than discovered. The separate concern that Levenshtein distance to the single planner-produced sequence penalizes valid alternative plans that satisfy the same preference (e.g., Table A3) is a validity risk for the benchmark metric, not a circularity of the derivation. No load-bearing self-citations or imported uniqueness arguments appear; earlier work by the authors is cited only as methodological background. Overall, partial circularity is present in the oracle-condition evidence, but the predicted-label two-stage result still has independent content, yielding a score of 6.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The benchmark rests on hand-designed preferences, template-generated demonstrations, and a metric that equates exact match with correctness. No parameters are fitted to data in the paper's own method; the free parameters are the experimental choices (e.g., 3 demonstrations, 512x512 at 8 fps), which are not fitted. The listed axioms are the domain assumptions that must hold for the empirical conclusions to transfer to real-world personalized planning.

assumptions (4)
  • domain assumption Preferences can be inferred from a small number of demonstrations and generalize across scenes and objects.
    This is the premise of the entire benchmark, stated in Section 3 and tested in Section 5.4; if false, the few-shot formulation collapses.
  • domain assumption The rule-based planner's action sequences are the correct embodiment of each preference.
    Section 4.2 generates ground-truth demonstrations using rule-based planners and predefined scripts; the evaluation assumes these sequences are the only correct plans.
  • domain assumption Levenshtein distance between generated and ground-truth action sequences measures preference satisfaction.
    Invoked in Section 5.2 as the primary metric without justification that different but valid plans are equivalent.
  • ad hoc to paper The 290 hand-defined preferences adequately cover the space of household preferences.
    Section 4.1 defines the hierarchy manually; there is no external validation that this set is complete or representative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Plan with Personalized Preferences." pith.science (2026). https://pith.science/paper/SKK6YRCS

@misc{pith2026250200858,
  author       = {Pith},
  title        = {Pith review of: Learning to Plan with Personalized Preferences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SKK6YRCS}},
  note         = {Machine review of arXiv:2502.00858}
}
read the original abstract

Effective integration of AI agents into daily life requires them to understand and adapt to individual human preferences, particularly in collaborative roles. Although recent studies on embodied intelligence have advanced significantly, they typically adopt generalized approaches that overlook personal preferences in planning. We address this limitation by developing agents that not only learn preferences from few demonstrations but also learn to adapt their planning strategies based on these preferences. Our research leverages the observation that preferences, though implicitly expressed through minimal demonstrations, can generalize across diverse planning scenarios. To systematically evaluate this hypothesis, we introduce Preference-based Planning (PbP) benchmark, an embodied benchmark featuring hundreds of diverse preferences spanning from atomic actions to complex sequences. Our evaluation of SOTA methods reveals that while symbol-based approaches show promise in scalability, significant challenges remain in learning to generate and execute plans that satisfy personalized preferences. We further demonstrate that incorporating learned preferences as intermediate representations in planning significantly improves the agent's ability to construct personalized plans. These findings establish preferences as a valuable abstraction layer for adaptive planning, opening new directions for research in preference-guided plan generation and execution.

Figures

Figures reproduced from arXiv: 2502.00858 by the authors.

Figure 1
Figure 1. An example of preference-based planning in a food preparation scenario. When the assistant receives a natural language instruction for a food preparation task, it can follow one of two approaches: (Left, traditional methods) The assistant verifies details with the user at each step through exhaustive communication; or (Right, our personalized approach) it first learns from previous user action sequences to infer exp… view at source ↗
Figure 2
Figure 2. Hierarchical organization of user preferences. Our framework organizes preferences in a three-tiered structure, visualized through sunburst diagrams: (a) Action level captures fine-grained execution details within specific tasks, from quantity preferences in “Contain” (e.g., “half a cup” vs. “full cup”) to environmental controls (e.g., lighting and window operations). (b) Option level represents spatial preferences … view at source ↗
Figure 3
Figure 3. Example of preferences and their corresponding actions in PBP. At the primitive action level, we demonstrate preferences through basic tasks: (a) cooking using microwave, (b) washing in the sink, and (c) cutting into halves. At the option level, we showcase different approaches to object rearrangement, where users can prefer either (d) grouping objects by their categories (v1) or (e) placing them on the same layer o… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Example of a demonstration in PBP. The robot in the demonstration is executing the task “Pick Apple from Fridge and place on Table”. Top: A third-person view video provides an overhead perspective of the entire scene. Middle: The bird’s-eye-view map displays the robot’…
Figure 5
Figure 5. Figure 5: Levenshtein distance between generated and ground truth action sequences. Results shown for both (a) option level and (b) sequence level under two conditions: End-to-end bars represent direct sequence generation from previous observations, while Second-stage bars show …
Figure 6
Figure 6. Figure 6: Analysis of test samples in direct and generalization settings. Lines represent distinct scenes, with grid colors indicating different sample statuses. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Ablation study on the number of demonstrations. Models are evaluated across both of the two stages within PBP task: (a) first-stage preference learning and (b) second-stage action planning. We evaluate both Option Level and Sequence Level tasks. The number of few-shot …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

76 extracted references · 63 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  3. [3]

    Abdo, N., Stachniss, C., Spinello, L., and Burgard, W. (2015). Robot, organize my shelves! tidying up objects by predicting user preferences. In International Conference on Robotics and Automation (ICRA)

    work page 2015

  4. [4]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  5. [5]

    G., Gopalakrishnan, K., Hausman, K., Ichter, B., Irpan, A., Joshi, N., Julian, R., et al

    Ahn, M., Dwibedi, D., Finn, C., Arenas, M. G., Gopalakrishnan, K., Hausman, K., Ichter, B., Irpan, A., Joshi, N., Julian, R., et al. (2024). Autort: Embodied foundation models for large scale orchestration of robotic agents. arXiv preprint arXiv:2401.12963

    arXiv 2024

  6. [6]

    Akgun, B., Cakmak, M., Jiang, K., and Thomaz, A. L. (2012). Keyframe-based learning from demonstration: Method and evaluation. International Journal of Social Robotics , 4:343--355

    work page 2012

  7. [7]

    Aleskerov, F., Bouyssou, D., and Monjardet, B. (2007). Utility maximization, choice and preference , volume 16. Springer Science & Business Media

    work page 2007

  8. [8]

    Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., S \"u nderhauf, N., Reid, I., Gould, S., and Van Den Hengel, A. (2018). Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR)

    work page 2018

Show all 76 references
  1. [9]

    Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lu c i \'c , M., and Schmid, C. (2021). Vivit: A video vision transformer. In Proceedings of International Conference on Computer Vision (ICCV)

    2021

  2. [10]

    Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, J. (2023). Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966

    2023 arXiv

  3. [11]

    A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M

    Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258

    2021 arXiv

  4. [12]

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., et al. (2022). Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817

    2022 arXiv

  5. [13]

    Chao, C., Cakmak, M., and Thomaz, A. L. (2011). Towards grounding concepts for transfer in goal learning from demonstration. In 2011 IEEE International Conference on Development and Learning (ICDL) , volume 2, pages 1--6. IEEE

    2011

  6. [14]

    Chen, H., Suhr, A., Misra, D., Snavely, N., and Artzi, Y. (2019). Touchdown: Natural language navigation and spatial reasoning in visual street environments. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR)

    2019

  7. [15]

    and Luo, Y

    Choi, Y. and Luo, Y. (2023). Understanding preferences in infancy. Wiley Interdisciplinary Reviews: Cognitive Science , 14(4):e1643

    2023

  8. [16]

    Coppin, G., Delplanque, S., Cayeux, I., Porcherot, C., and Sander, D. (2010). I’m no longer torn after choice: How explicit choices implicitly shape preferences of odors. Psychological science , 21(4):489--493

    2010

  9. [17]

    Ding, Y., Zhang, X., Amiri, S., Cao, N., Yang, H., Kaminski, A., Esselink, C., and Zhang, S. (2023). Integrating action knowledge and llms for task planning and situation handling in open worlds. Autonomous Robots , 47(8):981--997

    2023

  10. [18]

    S., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al

    Driess, D., Xia, F., Sajjadi, M. S., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023). Palm-e: An embodied multimodal language model. In Proceedings of International Conference on Machine Learning (ICML)

    2023

  11. [19]

    Epstein, S. (1994). Integration of the cognitive and the psychodynamic unconscious. American psychologist , 49(8):709

    1994

  12. [20]

    Fawcett, C. A. and Markson, L. (2010). Children reason about shared preferences. Developmental psychology , 46(2):299

    2010

  13. [21]

    W., Wallach, H., Iii, H

    Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Iii, H. D., and Crawford, K. (2021). Datasheets for datasets. Communications of the ACM , 64(12):86--92

    2021

  14. [22]

    A., Bekkering, H., and Hunnius, S

    Gerson, S. A., Bekkering, H., and Hunnius, S. (2017). Do you do as i do?: Young toddlers prefer and copy toy choices of similarly acting others. Infancy , 22(1):5--22

    2017

  15. [23]

    Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022). Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR)

    2022

  16. [24]

    G., Rao, K., Yu, W., Fu, C., Gopalakrishnan, K., Xu, Z., et al

    Gu, J., Kirmani, S., Wohlhart, P., Lu, Y., Arenas, M. G., Rao, K., Yu, W., Fu, C., Gopalakrishnan, K., Xu, Z., et al. (2023). Rt-trajectory: Robotic task generalization via hindsight trajectory sketches. arXiv preprint arXiv:2311.01977

    2023 arXiv

  17. [25]

    Jiang, K., Dahmani, A., Stacy, S., Jiang, B., Rossano, F., Zhu, Y., and Gao, T. (2022). What is the point? a theory of mind model of relevance. In Annual Meeting of the Cognitive Science Society (CogSci)

    2022

  18. [26]

    Jiang, K., Stacy, S., Wei, C., Chan, A., Rossano, F., Zhu, Y., and Gao, T. (2021). Individual vs. joint perception: a pragmatic model of pointing as communicative smithian helping. In Annual Meeting of the Cognitive Science Society (CogSci)

    2021

  19. [27]

    Kahneman, D. (1982). The psychology of preferences. Scientific American

    1982

  20. [28]

    Kant, Y., Ramachandran, A., Yenamandra, S., Gilitschenski, I., Batra, D., Szot, A., and Agrawal, H. (2022). Housekeep: Tidying virtual households using commonsense reasoning. In Proceedings of European Conference on Computer Vision (ECCV)

    2022

  21. [29]

    and Johns, E

    Kapelyukh, I. and Johns, E. (2022). My house, my rules: Learning tidying preferences with graph neural networks. In Conference on Robot Learning (CoRL)

    2022

  22. [30]

    Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Deitke, M., Ehsani, K., Gordon, D., Zhu, Y., et al. (2017). Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv:1712.05474

    2017 arXiv

  23. [31]

    Leal, I., Choromanski, K., Jain, D., Dubey, A., Varley, J., Ryoo, M., Lu, Y., Liu, F., Sindhwani, V., Vuong, Q., et al. (2023). Sara-rt: Scaling up robotics transformers with self-adaptive robust attention. arXiv preprint arXiv:2312.01990

    2023 arXiv

  24. [32]

    K., Forlizzi, J., Kiesler, S., Rybski, P., Antanitis, J., and Savetsila, S

    Lee, M. K., Forlizzi, J., Kiesler, S., Rybski, P., Antanitis, J., and Savetsila, S. (2012). Personalization in hri: A longitudinal field experiment. In ACM/IEEE International Conference on Human-Robot Interaction (HRI)

    2012

  25. [33]

    Levinson, S. C. (1983). Pragmatics. Cambridge UP

    1983

  26. [34]

    Leyzberg, D., Spaulding, S., and Scassellati, B. (2014). Personalizing robot tutors to individuals' learning differences. In ACM/IEEE International Conference on Human-Robot Interaction (HRI)

    2014

  27. [35]

    Li, C., Zhang, R., Wong, J., Gokmen, C., Srivastava, S., Mart \' n-Mart \' n, R., Wang, C., Levine, G., Lingelbach, M., Sun, J., et al. (2023). Behavior-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on Robot Learning (CoRL)

    2023

  28. [36]

    D., and Woodward, A

    Liberman, Z., Kinzler, K. D., and Woodward, A. L. (2021). Origins of homophily: Infants expect people with shared preferences to affiliate. Cognition , 212:104695

    2021

  29. [37]

    and Slovic, P

    Lichtenstein, S. and Slovic, P. (2006). The construction of preference: An overview. The construction of preference , 1:1--40

    2006

  30. [38]

    Liu, H., Li, C., Li, Y., and Lee, Y. J. (2024). Improved baselines with visual instruction tuning. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  31. [39]

    Mongin, P. (1997). Expected utility theory. Handbook of Economic Methodology , pages 342--350

    1997

  32. [40]

    Mu, Y., Zhang, Q., Hu, M., Wang, W., Ding, M., Jin, J., Wang, B., Dai, J., Qiao, Y., and Luo, P. (2023). Embodiedgpt: Vision-language pre-training via embodied chain of thought. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS)

    2023

  33. [41]

    and Chernova, S

    Patel, M. and Chernova, S. (2023). Proactive robot assistance via spatio-temporal object modeling. In Conference on Robot Learning (CoRL)

    2023

  34. [42]

    G., and Chernova, S

    Patel, M., Prakash, A. G., and Chernova, S. (2023). Predicting routine object usage for proactive robot assistance. In Conference on Robot Learning (CoRL)

    2023

  35. [43]

    Peng, Y., Han, J., Zhang, Z., Fan, L., Liu, T., Qi, S., Feng, X., Ma, Y., Wang, Y., and Zhu, S.-C. (2024). The tong test: Evaluating artificial general intelligence through dynamic embodied physical and social interactions. Engineering , 34:12--22

    2024

  36. [44]

    Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., and Wei, F. (2023). Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824

    2023 arXiv

  37. [45]

    B., Fidler, S., and Torralba, A

    Puig, X., Shu, T., Li, S., Wang, Z., Liao, Y.-H., Tenenbaum, J. B., Fidler, S., and Torralba, A. (2021a). Watch-and-help: A challenge for social perception and human-ai collaboration. In Proceedings of International Conference on Learning Representations (ICLR)

    2021

  38. [46]

    B., Fidler, S., and Torralba, A

    Puig, X., Shu, T., Li, S., Wang, Z., Liao, Y.-H., Tenenbaum, J. B., Fidler, S., and Torralba, A. (2021b). Watch-and-help: A challenge for social perception and human-ai collaboration. In Proceedings of International Conference on Learning Representations (ICLR)

    2021

  39. [47]

    B., and Torralba, A

    Puig, X., Shu, T., Tenenbaum, J. B., and Torralba, A. (2023a). Nopa: Neurally-guided online probabilistic assistance for building socially intelligent home assistants. In International Conference on Robotics and Automation (ICRA)

    2023

  40. [48]

    D., Yang, T.-Y., Partsey, R., Desai, R., Clegg, A., Hlavac, M., Min, S

    Puig, X., Undersander, E., Szot, A., Cote, M. D., Yang, T.-Y., Partsey, R., Desai, R., Clegg, A., Hlavac, M., Min, S. Y., et al. (2023b). Habitat 3.0: A co-habitat for humans, avatars, and robots. In Proceedings of International Conference on Learning Representations (ICLR)

    2023

  41. [49]

    W., Schydlo, P., Tarr, M

    Sarch, G., Fang, Z., Harley, A. W., Schydlo, P., Tarr, M. J., Gupta, S., and Fragkiadaki, K. (2022). Tidee: Tidying up novel rooms using visuo-semantic commonsense priors. In Proceedings of European Conference on Computer Vision (ECCV)

    2022

  42. [50]

    Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., et al. (2019). Habitat: A platform for embodied ai research. In Proceedings of International Conference on Computer Vision (ICCV)

    2019

  43. [51]

    Scott, J. et al. (2000). Rational choice theory. Understanding contemporary society: Theories of the present , 129:126--138

    2000

  44. [52]

    Sen, A. (1973). Behaviour and the concept of preference. Economica , 40(159):241--259

    1973

  45. [53]

    Shridhar, M., Thomason, J., Gordon, D., Bisk, Y., Han, W., Mottaghi, R., Zettlemoyer, L., and Fox, D. (2020). Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR)

    2020

  46. [54]

    and Tian, Y

    Shu, T. and Tian, Y. (2019). M\^ 3rl: Mind-aware multi-agent management reinforcement learning. In International Conference on Learning Representations

    2019

  47. [55]

    Simonson, I. (2008). Will i like a “medium” pillow? another look at constructed and inherent preferences. Journal of Consumer Psychology , 18(3):155--169

    2008

  48. [56]

    Singh, I., Blukis, V., Mousavian, A., Goyal, A., Xu, D., Tremblay, J., Fox, D., Thomason, J., and Garg, A. (2023). Progprompt: Generating situated robot task plans using large language models. In International Conference on Robotics and Automation (ICRA)

    2023

  49. [57]

    Slovic, P. (1995). The construction of preference. American Psychologist , 50(5):364

    1995

  50. [58]

    H., Wu, J., Washington, C., Sadler, B

    Song, C. H., Wu, J., Washington, C., Sadler, B. M., Chao, W.-L., and Su, Y. (2023). Llm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of International Conference on Computer Vision (ICCV)

    2023

  51. [59]

    A., Moll, M., and Kavraki, L

    Sucan, I. A., Moll, M., and Kavraki, L. E. (2012). The open motion planning library. IEEE Robotics & Automation Magazine , 19(4):72--82

    2012

  52. [60]

    Taniguchi, A., Isobe, S., El Hafi, L., Hagiwara, Y., and Taniguchi, T. (2021). Autonomous planning based on spatial concepts to tidy up home environments with service robots. Advanced Robotics , 35(8):471--489

    2021

  53. [61]

    Thomason, J., Murray, M., Cakmak, M., and Zettlemoyer, L. (2020). Vision-and-dialog navigation. In Conference on Robot Learning (CoRL)

    2020

  54. [62]

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023). Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  55. [63]

    Wu, J., Antonova, R., Kan, A., Lepert, M., Zeng, A., Song, S., Bohg, J., Rusinkiewicz, S., and Funkhouser, T. (2023). Tidybot: Personalized robot assistance with large language models. Autonomous Robots , 47(8):1087--1102

    2023

  56. [64]

    Xu, M., Jiang, G., Liang, W., Zhang, C., and Zhu, Y. (2024a). Active reasoning in an open-world environment. Advances in Neural Information Processing Systems , 36

    2024

  57. [65]

    Xu, M., Jiang, G., Liang, W., Zhang, C., and Zhu, Y. (2024b). Interactive visual reasoning under uncertainty. Advances in Neural Information Processing Systems , 36

    2024

  58. [66]

    P., Zhang, Z., Hu, F., and Chai, J

    Yu, K. P., Zhang, Z., Hu, F., and Chai, J. (2023). Efficient in-context learning in vision-language models for egocentric videos. arXiv preprint arXiv:2311.17041

    2023 arXiv

  59. [67]

    N., Rossano, F., Lu, H., Zhu, Y., and Zhu, S.-C

    Yuan, L., Gao, X., Zheng, Z., Edmonds, M., Wu, Y. N., Rossano, F., Lu, H., Zhu, Y., and Zhu, S.-C. (2022). In situ bidirectional human-robot value alignment. Science Robotics , 7(68)

    2022

  60. [68]

    Yuan, T., Liu, H., Fan, L., Zheng, Z., Gao, T., Zhu, Y., and Zhu, S.-C. (2020). Joint inference of states, robot knowledge, and human (false-)beliefs. In International Conference on Robotics and Automation (ICRA)

    2020

  61. [69]

    Yuan, Y., Wang, H., Ding, J., Jin, D., and Li, Y. (2023). Learning to simulate daily activities via modeling dynamic human needs. In Proceedings of the ACM Web Conference

    2023

  62. [70]

    Zey, M. (1998). Rational choice theory and organizational theory: A critique . Sage

    1998

  63. [71]

    Zhang, C., Jia, B., Edmonds, M., Zhu, S.-C., and Zhu, Y. (2021). Acre: Abstract causal reasoning beyond covariation. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR)

    2021

  64. [72]

    B., Shu, T., and Gan, C

    Zhang, H., Du, W., Shan, J., Zhou, Q., Du, Y., Tenenbaum, J. B., Shu, T., and Gan, C. (2024). zhang2023building. In Proceedings of International Conference on Learning Representations (ICLR)

    2024

  65. [73]

    V., et al

    Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022). Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

    2022 arXiv

  66. [74]

    K., and Xing, E

    Zheng, X., Aragam, B., Ravikumar, P. K., and Xing, E. P. (2018). Dags with no tears: Continuous optimization for structure learning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS)

    2018

  67. [75]

    Zhu, Y., Jiang, C., Zhao, Y., Terzopoulos, D., and Zhu, S.-C. (2016). Inferring forces and learning human utilities from videos. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR)

    2016

  68. [76]

    Zitkovich, B., Yu, T., Xu, S., Xu, P., Xiao, T., Xia, F., Wu, J., Wohlhart, P., Welker, S., Wahid, A., et al. (2023). Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL)

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.