REVIEW 12 cited by
CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory
read the original abstract
We propose CLIP-Fields, an implicit scene model that can be used for a variety of tasks, such as segmentation, instance identification, semantic search over space, and view localization. CLIP-Fields learns a mapping from spatial locations to semantic embedding vectors. Importantly, we show that this mapping can be trained with supervision coming only from web-image and web-text trained models such as CLIP, Detic, and Sentence-BERT; and thus uses no direct human supervision. When compared to baselines like Mask-RCNN, our method outperforms on few-shot instance identification or semantic segmentation on the HM3D dataset with only a fraction of the examples. Finally, we show that using CLIP-Fields as a scene memory, robots can perform semantic navigation in real-world environments. Our code and demonstration videos are available here: https://mahis.life/clip-fields
Forward citations
Cited by 12 Pith papers
-
ST-BiBench: Benchmarking Multi-Stream Multimodal Coordination in Bimanual Embodied Tasks for MLLMs
ST-BiBench reveals a coordination paradox in which MLLMs show strong high-level strategic reasoning yet fail at fine-grained 16-dimensional bimanual action synthesis and multi-stream fusion.
-
In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer
ICEdit achieves state-of-the-art instructional image editing in Diffusion Transformers via in-context generation, requiring only 0.1% of prior training data and 1% trainable parameters.
-
EmbodiedLGR: Integrating Lightweight Graph Representation and Retrieval for Semantic-Spatial Memory in Robotic Agents
A hybrid semantic graph and retrieval-augmented system with parameter-efficient VLMs achieves state-of-the-art inference and querying speeds on embodied navigation tasks with competitive accuracy.
-
GeGS-PCR: Effective and Robust 3D Point Cloud Registration with Two-Stage Color-Enhanced Geometric-3DGS Fusion
GeGS-PCR combines a color encoder, Geometric-3DGS module, LORA optimization, and joint photometric loss to achieve 99.9% registration recall and sub-0.03 rotation/translation errors on colorized 3DMatch and Kitti benchmarks.
-
Lifting Unlabeled Internet-level Data for 3D Scene Understanding
Unlabeled web videos processed by designed data engines generate effective training data that yields strong zero-shot and finetuned performance on 3D detection, segmentation, VQA, and navigation.
-
RADSeg: Unleashing Parameter and Compute Efficient Zero-Shot Open-Vocabulary Segmentation Using Agglomerative Models
RADSeg adapts the RADIO model with targeted enhancements to deliver 6-30% higher mIoU in zero-shot OVSS while using 2.5x fewer parameters and running 3.95x faster than prior large-model combinations.
-
LEEVLA: Seeing What Matters in Latent Environment Evolution for Vision-Language-Action
LEEVLA improves VLA robot policies by training-time prioritization of dynamic instruction-relevant patches plus structured latent future-feature prediction with topology constraints.
-
PhysGraph: A Physics-aware 3D Scene Graph for Perception and Reasoning
PhysGraph reconstructs object-centric 3D geometry from RGB-D, decomposes objects into parts, infers materials and articulations via visual reasoning, and reports SOTA results on semantic segmentation, multi-object mas...
-
Beyond Waypoints: Dual-Heatmap Grounding for Cross-Embodiment Semantic Navigation
A vision-language model outputs dual heatmaps for navigation affordance and facing to ground semantic instructions into executable free space, achieving higher affordance rates than waypoint regression across simulate...
-
SmartPhotoCrafter: Unified Reasoning, Generation and Optimization for Automatic Photographic Image Editing
SmartPhotoCrafter performs automatic photographic image editing by coupling an Image Critic module that identifies deficiencies with a Photographic Artist module that generates edits, trained via multi-stage pretraini...
-
Vision-Language Models for Deployable Social Robot Navigation: Bridging Semantic Reasoning and Low-Level Control
Survey organizing VLM-based social robot navigation into reasoning, planning, and bridging components with a proposed roadmap for hybrid deployable systems.
-
EL3DD: Extended Latent 3D Diffusion for Language Conditioned Multitask Manipulation
EL3DD extends latent 3D diffusion with language inputs and reference demonstrations to improve success rates on sequential manipulation tasks in the CALVIN dataset.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.