REVIEW 7 cited by
OMNI: Open-endedness via Models of human Notions of Interestingness
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Open-ended algorithms aim to learn new, interesting behaviors forever. That requires a vast environment search space, but there are thus infinitely many possible tasks. Even after filtering for tasks the current agent can learn (i.e., learning progress), countless learnable yet uninteresting tasks remain (e.g., minor variations of previously learned tasks). An Achilles Heel of open-endedness research is the inability to quantify (and thus prioritize) tasks that are not just learnable, but also $\textit{interesting}$ (e.g., worthwhile and novel). We propose solving this problem by $\textit{Open-endedness via Models of human Notions of Interestingness}$ (OMNI). The insight is that we can utilize foundation models (FMs) as a model of interestingness (MoI), because they $\textit{already}$ internalize human concepts of interestingness from training on vast amounts of human-generated data, where humans naturally write about what they find interesting or boring. We show that FM-based MoIs improve open-ended learning by focusing on tasks that are both learnable $\textit{and interesting}$, outperforming baselines based on uniform task sampling or learning progress alone. This approach has the potential to dramatically advance the ability to intelligently select which tasks to focus on next (i.e., auto-curricula), and could be seen as AI selecting its own next task to learn, facilitating self-improving AI and AI-Generating Algorithms. Project website at https://www.jennyzhangzt.com/omni/
Forward citations
Cited by 7 Pith papers
-
Computational models of pragmatic reasoning with flexible generation of meaning and expression alternatives
Combining language-model generation with rule-based selection reproduces several pragmatic phenomena, but the language models only worked reliably as idea generators, not as judges of formal linguistic properties.
-
Expedition & Expansion: Leveraging Semantic Representations for Goal-Directed Exploration in Continuous Cellular Automata
A hybrid exploration algorithm that alternates semantic novelty search with VLM-generated linguistic goals discovers more diverse Flow Lenia behaviors than novelty search alone.
-
LLM-First Search: Self-Guided Exploration of the Solution Space
LLM-First Search, where the model itself decides whether to continue or backtrack during reasoning, outperforms MCTS, BestFS, and ToT-BFS on harder Countdown and Sudoku tasks while using fewer tokens.
-
Self-Challenging Language Model Agents
A language model agent can generate its own verifiable training tasks and improve its tool-use success rate by about 2x without human-annotated data.
-
Fusing Multi- and Hyperspectral Satellite Data for Harmful Algal Bloom Monitoring with Self-Supervised and Hierarchical Deep Learning
SIT-FUSE maps harmful algal bloom concentration and species from fused VIIRS/MODIS/Sentinel-3/PACE/TROPOMI data using self-supervised hierarchical clustering, with limited in-situ validation.
-
Automated Skill Discovery for Language Agents through Exploration and Iterative Feedback
EXIF repeatedly has a teacher agent explore an environment, relabel the exploration as tasks, train a student agent on it, and use the student's failures to guide the next round, improving 7B-8B agents in Webshop and Crafter.
-
VoyagerVision: Investigating the Role of Multi-modal Information for Open-ended Learning Systems
VoyagerVision combines GPT-4o with point-of-view screenshots in the Voyager Minecraft agent, producing 18 verified unit-test building successes out of 50 attempts and 2.75 unique structures per 50-step open-ended run.
Discussion (0). Sign in to comment.