RLS3: RL-Based Synthetic Sample Selection to Enhance Spatial Reasoning in Vision-Language Models for Indoor Autonomous Perception

Chinmay Hegde; Joshua R. Waite; Md. Zahid Hasan; Qisai Liu; Soumik Sarkar; Zhanhong Jiang

REVIEW 4 major objections 6 minor 42 references

RLS3: RL-Based Synthetic Sample Selection to Enhance Spatial Reasoning in Vision-Language Models for Indoor Autonomous Perception

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A reinforcement-learning agent that arranges objects to expose a vision-language model's spatial weaknesses produces synthetic fine-tuning data that improves spatial reasoning more efficiently than random data.

desk verdict A novel RL-guided synthetic data loop for VLM spatial reasoning, but the evaluation never leaves the simulator; sim-to-real transfer is acknowledged but untested. read the letter →

arxiv 2501.18880 v1 pith:CO3YFY7N submitted 2025-01-31 cs.CV cs.LG

Joshua R. Waite , Md. Zahid Hasan , Qisai Liu , Zhanhong Jiang , Chinmay Hegde , Soumik Sarkar This is my paper · ORCID

classification cs.CVcs.LG

keywords spatialreasoningvision-languagemodelssyntheticdatagenerationreinforcementlearningsamplingsoftactor-criticCLIPPaliGemma

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RLS3 is a framework for fine-tuning vision-language models (VLMs) with synthetic data chosen by a reinforcement-learning agent. The authors try to establish that if an RL agent moves objects in a simulated indoor scene and receives a reward equal to the current VLM's mistakes, the resulting image-caption pairs fine-tune the model more efficiently than randomly sampled scenes. They report that both PaliGemma and CLIP improve faster and reach higher spatial-reasoning scores with RL-selected data, and that CLIP with RLS3 matches the accuracy of the NegCLIP baseline, which uses hand-crafted negative captions. The wider point is that VLM fine-tuning is often bottlenecked by data quality and balance; if a model's own errors can steer data generation, then targeted reasoning skills might be improved without large manually curated datasets.

What carries the argument

The load-bearing mechanism is the combined reward $J = J_1 + \beta J_2$, where $J_1$ is the intrinsic feasibility reward for valid object placements and $J_2$ is the extrinsic VLM-performance reward: the contrastive loss for CLIP or $(6 - S_{PG})^2$ for PaliGemma's rubric score. Maximizing this reward makes the SAC agent propose object configurations that are physically valid yet difficult for the current VLM, biasing the fine-tuning batch toward hard spatial relations. The prompt generator supports this by dividing the space around a primary object into eight 45-degree horizontal sectors plus vertical angle bands, producing captions with one to three spatial terms so that difficulty has a consistent, quantified meaning.

What would settle it

Take RLS3 and the random baseline through the same fine-tuning budget, then evaluate both on a fixed set of real indoor photographs with manually verified spatial captions; if RLS3 does not outperform random sampling on that real-image set, the claim that VLM-loss-guided synthetic sampling improves spatial reasoning for indoor perception is falsified.

Watch

Extended reading notes

Core claim

The central claim is that feeding a soft actor-critic agent the VLM's own performance signal turns it into an informative data sampler for spatial reasoning. In each episode the agent places three objects on surfaces in a Unity scene, receives an intrinsic reward for physically feasible placements, and then an extrinsic reward based on how badly the VLM describes the resulting scene: the contrastive loss for CLIP or the inverted rubric score for PaliGemma. The agent therefore gravitates toward scenes the VLM currently gets wrong, and those hard samples are collected into a fine-tuning batch that is refreshed every iteration. On a held-out synthetic test set, the paper reports consistently higher PaliGemma scores and higher CLIP accuracy for this agent than for a random sampler, with the gap widest in the early fine-tuning iterations, and CLIP reaching performance comparable to NegCLIP.

Load-bearing premise

The claim depends on synthetic Unity scenes being a faithful stand-in for real indoor environments, so that the scenes the VLM gets wrong in simulation are also the scenes it would get wrong in reality.

Editorial extensions

If this is right

An equal number of RL-generated samples produces larger early gains in spatial reasoning score than randomly generated samples, for both PaliGemma and CLIP.
CLIP fine-tuned with RLS3 reaches test accuracy comparable to NegCLIP, which is trained with hand-crafted negative captions.
For PaliGemma, RL-guided fine-tuning improves or preserves scores on every spatial term examined, whereas random fine-tuning causes 'left' and 'above' to decline.
The RL agent is more sample-efficient: it yields fewer invalid object placements, so reaching a given performance level requires less generated data.
Validation-based early stopping terminates the loop when gains stall, and the validation curve tracks the test curve.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Our inference: the same reward loop should transfer to other VLM reasoning failures, such as counting or attribute binding, by swapping the prompt generator and the evaluation rubric, since the method is not tied to spatial prepositions.
Our inference: in expensive simulation or physical robot data collection, the observed data-efficiency advantage implies RL-guided sampling could cut the number of scenes needed to reach a target performance level.
Our inference: the agent's heavy preference for 'left' and 'right' scenes suggests the reward exploits an easy-to-generate failure region, and adding a diversity or coverage term over spatial relations would likely produce a more balanced and more robust fine-tuning set.
Our inference: because the data distribution is coupled to the model's current weaknesses, the advantage over random sampling should shrink as the VLM saturates on the synthetic distribution, so reweighting the reward or refreshing the scene distribution would be needed to sustain early gains.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

RLS3 is a genuinely new idea: an SAC agent moves objects in a Unity scene to generate synthetic image-caption pairs that are hard for the current VLM, using the VLM's loss or rubric score as reward, then those batches fine-tune the VLM iteratively. The comparison to a random agent is the right minimal experiment, and the per-term and prompt-complexity analyses are thoughtful. Credit where due: the framework is described clearly, the early-stopping protocol is sensible, and the paper openly concedes that sim-to-real transfer is not yet addressed.

The soft spots are exactly where the reader's report says they are. Training, validation, and test data all come from the same Unity engine, the same object assets, and the same angle-based prompt template; the held-out test set only changes scene layout. Because the RL agent is rewarded on this distribution, its "hard" samples may exploit quirks of the template or renderer rather than improving general spatial reasoning. The paper's Section 7 says sim2real transfer needs investigation, but the abstract and title claim improvements for indoor autonomous perception. That overstates the evidence. There is also no comparison to uncertainty-sampling or other active-learning baselines on the same generated pool, and no code or data is released, so the result is not independently reproducible. Hyperparameters like beta and T0 are tuned but not ablated.

None of this kills the central idea. The RL-guided generation loop is plausible, and the synthetic results do show a consistent advantage for SAC over random. But the current evidence supports only a scoped claim: RL-guided generation improves VLM spatial reasoning on synthetic scenes from the same simulator. Whether that transfers to real indoor perception remains open.

I'd send it to review — the idea deserves scrutiny, and the authors have a clear path to strengthen it with real-image benchmarks, additional baselines, and code release. I wouldn't cite it yet in its current form.

Referee Report

4 major / 6 minor

Summary. The paper proposes RLS3, a framework that couples a Soft Actor-Critic reinforcement learning agent with a Unity-based simulator to generate synthetic image-caption pairs for fine-tuning vision-language models (CLIP and PaliGemma) on spatial reasoning tasks. The RL agent receives a reward signal derived from the VLM's inference loss or rubric score on each generated batch, so that it learns to propose object arrangements that are challenging for the current model. The VLM is then iteratively fine-tuned on batches of these RL-selected samples. Experiments compare RL-guided generation against a random generation baseline on a held-out synthetic test set, reporting faster early improvement and better or comparable final performance for the RL-guided approach, including per-spatial-term and prompt-complexity analyses.

Significance. If the reported gains transfer beyond the synthetic simulator, the framework is an interesting step toward using a VLM's own error signal to drive targeted synthetic data generation for spatial reasoning, with potential relevance to autonomous perception. The paper's strengths include a clearly described pipeline, multiple random-seed runs with variance reporting, and a direct comparison to a random generation baseline. The authors are also candid about the sim-to-real limitation in the future-work section. However, the significance for the stated application area of 'Indoor Autonomous Perception' is currently limited because all evaluations share the same rendering engine, object assets, and prompt generator as the training data, and no comparison is made with active-learning or loss-based filtering baselines. The framework's novelty and practical value will remain unproven until these gaps are addressed.

major comments (4)

[§5.3, §7 (and abstract)] The evaluation is entirely on synthetic data produced by the same Unity engine, object-asset pool, and prompt generator used for training; the held-out test set only changes scene layout. Because the RL agent is rewarded for generating samples that maximize VLM loss within this specific synthetic distribution (Eq. 3–4), the observed advantage of SAC over the random agent may reflect overfitting to distribution-specific failure modes (e.g., prompt phrasing, texture, or limited layout manifold) rather than improved general spatial reasoning. The paper itself concedes in Section 7 that sim2real transfer 'necessitates further investigation.' Since the title and abstract claim improvement for indoor autonomous perception, this is a load-bearing gap. A concrete remedy would be to evaluate on real indoor images or an established spatial-reasoning benchmark (e.g., VSR, What's Up, or a real-robot grounding task), or to explicitly reframe the claims as being about synthetic-distribution performance only.
[§2.4, §6] The paper motivates RLS3 against active learning methods (uncertainty sampling, core-set, BatchBALD, BADGE) and stresses that it generates new samples rather than selecting from a fixed pool. However, all experiments compare only against a random agent. Without a comparison to a simpler informative-sampling baseline—such as random generation followed by loss-based filtering, or uncertainty sampling over the same Unity environment—the specific benefit of the RL-guided exploration over simpler difficulty-mining strategies is not established. The claim that the RL agent yields 'informative' samples would be strengthened by showing that it outperforms a non-RL selector that also optimizes for the same informativeness criterion.
[§5.2, Algorithm 1] The central hyperparameters (T0=200, eta=0.5, E=20, beta=10, K=256/10) are said to be chosen via 'extensive hyperparameter search,' but no search details, ablation study, or sensitivity analysis are reported. The reward-balancing factor beta and the feedback frequency E directly control the RL-VLM interaction, so the reader cannot assess how robust the reported gains are to these choices. At minimum, an ablation varying beta and E (or a table of searched values and selection criteria) is needed to support the claim that the framework is 'generalizable' and to enable reproducibility.
[§6.1, Fig. 5] The random-agent comparison is confounded by the early-stopping procedure. The text states that early stopping is applied only when the RL agent is used, and the resulting total number of generated samples is used as the budget for random-agent runs. This makes the comparison depend on the chosen stopping rule; if the random agent would continue improving after the RL agent has plateaued, the budget-limited comparison could understate the random agent's eventual performance. The authors should either apply symmetric early stopping based on validation performance for both agents or report full learning curves without budget truncation.

minor comments (6)

[§5.2] The text contains a typo: 'PlaiGemma' should be 'PaliGemma.'
[§4.2] The sentence 'It can be regarded as a key hyperparameter to tune in practice' is vague; please specify which quantity is being tuned (T0, the number of valid images per episode, or the episodic length T) and how it interacts with the requirement T >= T0.
[§4.4] The rubric for PaliGemma scoring is defined, but the operational mapping from the generated caption to a score is not fully specified—for example, how are extra or missing spatial terms counted when the prompt has only one term, and how are 'opposite' terms identified automatically? Clarify the rubric's implementation.
[§6.2] The per-term performance analysis is provided only for PaliGemma. Since the paper claims improvements for both PaliGemma and CLIP, adding the corresponding per-term results for CLIP would make the comparison more complete.
[General] The paper does not state whether code or data will be released. Providing the Unity scene configuration, the prompt generator, and hyperparameter search code would greatly aid reproducibility of this RL+VLM pipeline.
[References] References [3] and [29] appear incomplete or placeholder-style ('Anonymous Author et al.' and 'Sengupta et al.'); please update them to complete citations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the held-out test set and random-agent baseline make the central comparison empirical rather than definitional.

full rationale

The paper's derivation chain is: (i) an RL agent moves objects in a Unity environment; (ii) the VLM's loss or rubric score on the generated image-text pairs forms an extrinsic reward; (iii) the VLM is fine-tuned on a sampled subset of those pairs; (iv) performance is measured on a fixed, pre-generated test set from a separate Unity environment. No step equates the target claim with its inputs. The test set is static and generated before evaluation, so the reported improvement of the SAC agent over the random agent is not forced by construction. The coupling between VLM-based reward and fine-tuning data is the intended active-learning/adversarial feedback loop, not a self-definitional reduction. The paper's self-citations appear only in related-work or prior-application contexts (e.g., GENESIS-RL is cited as a similar approach and is explicitly acknowledged rather than used as load-bearing justification), so they do not make the argument circular. The main limitation is external validity: training, validation, and test all share the same Unity rendering pipeline, assets, and prompt templates, and the paper itself concedes in Section 7 that sim2real transfer 'necessitates further investigation.' That is a correctness or generalization risk, not a circularity, because the held-out test result is still an empirical outcome rather than an algebraic consequence of the reward definition.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The framework rests on standard RL and VLM machinery plus domain-specific assumptions about the synthetic environment and prompt generation. The main free parameters are training-loop hyperparameters chosen by search. No new physical entities are postulated.

free parameters (7)

beta = 10
Scaling factor balancing intrinsic feasibility reward and extrinsic VLM loss reward (Eq. 4); chosen by hyperparameter search.
T0 = 200
Minimum number of valid samples per RL episode; determines batch size; chosen by hyperparameter search.
eta = 0.5
Sampling rate applied to generated data per episode to form the fine-tuning batch; chosen by hyperparameter search.
E = 20
Number of RL episodes per fine-tuning iteration; controls feedback frequency; chosen by hyperparameter search.
K = 256 for PaliGemma, 10 for CLIP
Number of fine-tuning steps/epochs per iteration; chosen by hyperparameter search.
Early stopping thresholds = 10 iterations no improvement after 15 (PaliGemma); 5 after 10 (CLIP)
Hand-chosen criteria to stop the loop; affect total number of generated samples.
Angle thresholds for prompt generation = 20 and 75 degrees
Thresholds defining when vertical terms are appended or used alone; hand-chosen, affecting prompt complexity.

assumptions (5)

standard math Standard MDP formulation for the RL agent
Used in Section 3.1 to frame the data-generating agent as a Markov Decision Process.
standard math Contrastive loss for VLM fine-tuning (Eq. 1)
Assumed as the objective for CLIP and PaliGemma fine-tuning, with a temperature parameter.
domain assumption Unity simulator provides accurate ground-truth geometry and feasible scenes
The entire data generation relies on Unity's ability to place objects feasibly and provide exact coordinates for prompt generation.
ad hoc to paper Prompt generator template produces unambiguous spatial captions
The 8-region horizontal and 3-region vertical discretization defines what counts as a spatial relation and what the VLM is evaluated on.
ad hoc to paper Rubric-based score SPG reflects spatial reasoning quality
The scoring rubric in Section 4.4 maps VLM outputs to a 1-5 score and is used as an evaluation metric and reward signal.

how reviews work

0 comments

Cite this review

Pith. "Pith review of RLS3: RL-Based Synthetic Sample Selection to Enhance Spatial Reasoning in Vision-Language Models for Indoor Autonomous Perception." pith.science (2026). https://pith.science/paper/CO3YFY7N

@misc{pith2026250118880,
  author       = {Pith},
  title        = {Pith review of: RLS3: RL-Based Synthetic Sample Selection to Enhance Spatial Reasoning in Vision-Language Models for Indoor Autonomous Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CO3YFY7N}},
  note         = {Machine review of arXiv:2501.18880}
}

read the original abstract

Vision-language model (VLM) fine-tuning for application-specific visual grounding based on natural language instructions has become one of the most popular approaches for learning-enabled autonomous systems. However, such fine-tuning relies heavily on high-quality datasets to achieve successful performance in various downstream tasks. Additionally, VLMs often encounter limitations due to insufficient and imbalanced fine-tuning data. To address these issues, we propose a new generalizable framework to improve VLM fine-tuning by integrating it with a reinforcement learning (RL) agent. Our method utilizes the RL agent to manipulate objects within an indoor setting to create synthetic data for fine-tuning to address certain vulnerabilities of the VLM. Specifically, we use the performance of the VLM to provide feedback to the RL agent to generate informative data that efficiently fine-tune the VLM over the targeted task (e.g. spatial reasoning). The key contribution of this work is developing a framework where the RL agent serves as an informative data sampling tool and assists the VLM in order to enhance performance and address task-specific vulnerabilities. By targeting the data sampling process to address the weaknesses of the VLM, we can effectively train a more context-aware model. In addition, generating synthetic data allows us to have precise control over each scene and generate granular ground truth captions. Our results show that the proposed data generation approach improves the spatial reasoning performance of VLMs, which demonstrates the benefits of using RL-guided data generation in vision-language tasks.

Figures

Figures reproduced from arXiv: 2501.18880 by the authors.

**Figure 1.** Overview of RLS3 to generate a feasible and informative batch for fine-tuning a VLM to improve spatial reasoning. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** A detailed overview of our proposed RLS3 framework. The scheduler acts as a synchronizer between the processes. The [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** A diagram showing the Unity environment struc [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Illustration of our template-based prompt genera [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Combined figures of PaliGemma score and CLIP performance vs cumulative generated data. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: PaliGemma score (𝑎𝑣𝑔 ± 𝑠𝑡𝑑) separated by spatial term for RLS3 with an SAC and random agent. Cumulative term counts of data generated for fine-tuning are given in ( ) for SAC and [ ] for random agents. importance of intelligent sample selection to account for underrep…

**Figure 7.** Figure 7: Average PaliGemma score by prompt complexity [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

**Figure 8.** Figure 8: Concatenated PaliGemma loss plots for iterative [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 20 canonical work pages

[1]

Alekh Agarwal, Nan Jiang, Sham M Kakade, and Wen Sun. 2019. Reinforcement learning: Theory and algorithms. CS Dept., UW Seattle, Seattle, W A, USA, Tech. Rep 32 (2019), 96

2019
[2]

Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal

Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. 2020. Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds. In 8th International Conference on Learning Represen- tations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net. https://openreview.net/forum?id=ryghZJBKPS

work page 2020
[3]

Anonymous Author et al. 2023. Pushing RL Boundaries: Integrating Foundational Models, e.g. LLMs and VLMs, into Reinforcement Learning.Towards Data Science (2023)

work page 2023
[4]

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Waite et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022)

arXiv 2022
[5]

Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschan- nen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Kop- pula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias...

arXiv 2024
[6]

Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024. SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 14455–14465

work page 2024
[7]

An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. 2024. SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models. In NeurIPS

work page 2024
[8]

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep Reinforcement Learning from Human Preferences. In Ad- vances in Neural Information Processing Systems , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https://proceedings.neu...

work page 2017

Show all 42 references

[9]

Yarin Gal, Riashat Islam, and Zoubin Ghahramani. 2017. Deep Bayesian active learning with image data. In Proceedings of the 34th International Conference on Machine Learning - Volume 70 (Sydney, NSW, Australia)(ICML’17). JMLR.org, 1183–1192

2017
[10]

Jensen Gao, Bidipta Sarkar, Fei Xia, Ted Xiao, Jiajun Wu, Brian Ichter, Anirudha Majumdar, and Dorsa Sadigh. 2024. Physically Grounded Vision-Language Models for Robotic Manipulation. arXiv:2309.02561 [cs.RO] https://arxiv.org/ abs/2309.02561

2024 arXiv
[12]

Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning . PMLR, 1861– 1870

2018
[13]

Zahid Hasan, Jiajing Chen, Jiyang Wang, Mohammed Shaiqur Rahman, Ameya Joshi, Senem Velipasalar, Chinmay Hegde, Anuj Sharma, and Soumik Sarkar

Md. Zahid Hasan, Jiajing Chen, Jiyang Wang, Mohammed Shaiqur Rahman, Ameya Joshi, Senem Velipasalar, Chinmay Hegde, Anuj Sharma, and Soumik Sarkar. 2024. Vision-Language Models Can Identify Distracted Driver Behavior From Naturalistic Videos. IEEE Transactions on Intelligent T...

2024
[14]

Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel
[15]

Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. 2017. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern reco...

2017
[16]

Arthur Juliani, Vincent-Pierre Berges, Ervin Teng, Andrew Cohen, Jonathan Harper, Chris Elion, Chris Goy, Yuan Gao, Hunter Henry, Marwan Mattar, and Danny Lange. 2020. Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627 (2020). https://arxiv.org/p...

2020 arXiv
[17]

Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Rein- forcement learning: A survey. Journal of artificial intelligence research 4 (1996), 237–285

1996
[18]

Chow Jun Kang, Wong Cho Hin Peter, Tan Pin Siang, Tan Tun Jian, Li Zhaofeng, and Wang Yu-Hsing. 2023. An active learning framework fea- tured Monte Carlo dropout strategy for deep learning-based semantic seg- mentation of concrete cracks from images. Structural Health Monitor-...

2023 doi
[19]

Yeachan Kim and Bonggun Shin. 2022. In Defense of Core-set: A Density-aware Core-set Selection for Active Learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Washington DC, USA) (KDD ’22). Association for Computing Machinery, New ...

2022
[20]

Peter Kulits, Haiwen Feng, Weiyang Liu, Victoria Abrevaya, and Michael J. Black. 2024. Re-Thinking Inverse Graphics With Large Language Models. arXiv:2404.15228 [cs.CV] https://arxiv.org/abs/2404.15228

2024 arXiv
[21]

Junnan Li, Dongxu Li, Caiming Xiong, and Steven CH Hoi. 2022. Blip: Boot- strapped language-image pre-training for unified vision-language understanding and generation. arXiv preprint arXiv:2201.12086 (2022)

2022 arXiv
[22]

Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. 2023. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499 (2023)

2023 arXiv
[23]

Trung Quoc Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. 2024. ReFT: Reasoning with Reinforced Fine-Tuning. arXiv:2401.08967 [cs.CL] https://arxiv.org/abs/2401.08967

2024 arXiv
[24]

Pablo Martinez-Gonzalez, Sergiu Oprea, Alberto Garcia-Garcia, Alvaro Jover- Alvarez, Sergio Orts-Escolano, and Jose Garcia-Rodriguez. 2020. UnrealROX: an extremely photorealistic virtual reality environment for robotics simulations and synthetic data generation. Virtual Realit...

2020 doi
[25]

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, a...

2022
[26]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pam Mishkin, Jack Clark, et al
[27]

Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernes- tus, and Noah Dormann. 2021. Stable-Baselines3: Reliable Reinforcement Learn- ing Implementations. Journal of Machine Learning Research 22, 268 (2021), 1–8. http://jmlr.org/papers/v22/20-1364.html

2021
[28]

Gupta, Xiaojiang Chen, and Xin Wang

Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Brij B. Gupta, Xiaojiang Chen, and Xin Wang. 2021. A Survey of Deep Active Learning. arXiv:2009.00236 [cs.LG] https://arxiv.org/abs/2009.00236

2021 arXiv
[29]

Soumyadip Sengupta et al. 2023. Neural Inverse Rendering of an Indoor Scene From a Single Image. arXiv preprint arXiv:2312.03275 (2023)

2023 arXiv
[30]

Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano

Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2022. Learning to summarize from human feedback. arXiv:2009.01325 [cs.CL] https://arxiv.org/ abs/2009.01325

2022 arXiv
[31]

Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Yang Wang, Zhiyong Zhao, Kun Zhan, Peng Jia, XianPeng Lang, and Hang Zhao. 2024. DriveVLM: The Conver- gence of Autonomous Driving and Large Vision-Language Models. In8th Annual Conference on Robot Learning . https://openreview.ne...

2024
[32]

Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, and Stan Birchfield. 2018. Training Deep Networks With Synthetic Data: Bridging the Reality Gap by Domain Randomization. In Proceedings of the IEEE C...

2018
[33]

2022.Unity

Unity Technologies. 2022.Unity. https://unity.com/ Game development platform

2022
[34]

Waite, Jiale Feng, Riley Tavassoli, Laura Harris, Sin Yong Tan, Sub- hadeep Chakraborty, and Soumik Sarkar

Joshua R. Waite, Jiale Feng, Riley Tavassoli, Laura Harris, Sin Yong Tan, Sub- hadeep Chakraborty, and Soumik Sarkar. 2023. Active shooter detection and robust tracking utilizing supplemental synthetic data. arXiv:2309.03381 [cs.CV] https://arxiv.org/abs/2309.03381

2023 arXiv
[35]

Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Am- manabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi. 2023. Fine-grained human feedback gives better rewards for language model training. Advances in Neural Information Processing Systems 36 (2...

2023
[36]

Hsin-Jung Yang, Joe Beck, Md Zahid Hasan, Ekin Beyazit, Subhadeep Chakraborty, Tichakorn Wongpiromsarn, and Soumik Sarkar. 2024. GENESIS-RL: GEnerating Natural Edge-cases with Systematic Integration of Safety considera- tions and Reinforcement Learning. arXiv preprint arXiv:24...

2024 arXiv
[37]

Donggeun Yoo and In So Kweon. 2019. Learning Loss for Active Learning. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

2019
[38]

Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. 2023. When and why Vision-Language Models behave like Bags-of-Words, and what to do about it?. In International Conference on Learning Representations . https://openreview.net/forum?id=KRLUvxh8uaX

2023
[39]

Yuexiang Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Shengbang Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, and Sergey Levine. 2024. Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning. arXiv:2405.10292 [cs.AI] https://arxiv...

2024 arXiv
[40]

Wenshuai Zhao, Jorge Peña Queralta, and Tomi Westerlund. 2020. Sim-to- Real Transfer in Deep Reinforcement Learning for Robotics: a Survey. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI) . 737–744. https: //doi.org/10.1109/SSCI47803.2020.9308468

2020
[41]

Xingcheng Zhou, Mingyu Liu, Ekim Yurtsever, Bare Luka Zagar, Walter Zimmer, Hu Cao, and Alois C. Knoll. 2024. Vision Language Models in Autonomous Driving: A Survey and Outlook. IEEE Transactions on Intelligent Vehicles (2024), 1–20. https://doi.org/10.1109/TIV.2024.3402136

2024
[2011]

arXiv:1112.5745 [stat.ML] https://arxiv.org/abs/1112.5745

Bayesian Active Learning for Classification and Preference Learning. arXiv:1112.5745 [stat.ML] https://arxiv.org/abs/1112.5745

arXiv
[2021]

arXiv preprint arXiv:2103.00020 (2021)

Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020 (2021)

2021 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Alekh Agarwal, Nan Jiang, Sham M Kakade, and Wen Sun. 2019. Reinforcement learning: Theory and algorithms. CS Dept., UW Seattle, Seattle, W A, USA, Tech. Rep 32 (2019), 96

2019

[2] [2]

Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal

Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. 2020. Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds. In 8th International Conference on Learning Represen- tations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net. https://openreview.net/forum?id=ryghZJBKPS

work page 2020

[3] [3]

Anonymous Author et al. 2023. Pushing RL Boundaries: Integrating Foundational Models, e.g. LLMs and VLMs, into Reinforcement Learning.Towards Data Science (2023)

work page 2023

[4] [4]

Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Waite et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022)

arXiv 2022

[5] [5]

Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschan- nen, Emanuele Bugliarello, Thomas Unterthiner, Daniel Keysers, Skanda Kop- pula, Fangyu Liu, Adam Grycner, Alexey Gritsenko, Neil Houlsby, Manoj Kumar, Keran Rong, Julian Eisenschlos, Rishabh Kabra, Matthias...

arXiv 2024

[6] [6]

Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. 2024. SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 14455–14465

work page 2024

[7] [7]

An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. 2024. SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models. In NeurIPS

work page 2024

[8] [8]

Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep Reinforcement Learning from Human Preferences. In Ad- vances in Neural Information Processing Systems , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https://proceedings.neu...

work page 2017