REVIEW 9 cited by
From data to functa: Your data point is a function and you can treat it like one
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
It is common practice in deep learning to represent a measurement of the world on a discrete grid, e.g. a 2D grid of pixels. However, the underlying signal represented by these measurements is often continuous, e.g. the scene depicted in an image. A powerful continuous alternative is then to represent these measurements using an implicit neural representation, a neural function trained to output the appropriate measurement value for any input spatial location. In this paper, we take this idea to its next level: what would it take to perform deep learning on these functions instead, treating them as data? In this context we refer to the data as functa, and propose a framework for deep learning on functa. This view presents a number of challenges around efficient conversion from data to functa, compact representation of functa, and effectively solving downstream tasks on functa. We outline a recipe to overcome these challenges and apply it to a wide range of data modalities including images, 3D shapes, neural radiance fields (NeRF) and data on manifolds. We demonstrate that this approach has various compelling properties across data modalities, in particular on the canonical tasks of generative modeling, data imputation, novel view synthesis and classification. Code: https://github.com/deepmind/functa
Forward citations
Cited by 9 Pith papers
-
On the Expressive Power of Permutation-Equivariant Weight-Space Networks
Permutation-equivariant weight-space networks are all equally expressive, and universality holds when hidden-layer biases are pairwise distinct.
-
FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fields
MDIR detects LLM weight homology from embedding matrices alone using polar decomposition and permutation matching, achieving perfect AUC and accuracy on LeaFBench and reconstructing layer-level transformations.
-
Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weights
ProbeLog represents each classifier output by its responses to fixed probe images and uses CLIP to answer text queries, achieving 43.8% top-1 accuracy when searching 1,500 ImageNet-trained models for a concept.
-
Picasso: Holistic Scene Reconstruction with Physics-Constrained Sampling
Picasso produces multi-object scene reconstructions that are both geometrically accurate and physically plausible by using physics-constrained rejection sampling over an inferred contact graph, outperforming prior met...
-
VidFuncta: Towards Generalizable Neural Representations for Ultrasound Videos
VidFuncta encodes ultrasound videos into static and time-varying latent vectors, improving reconstruction over 2D and 3D baselines while enabling efficient downstream analysis.
-
Deep Active Inference Agents for Delayed and Long-Horizon Environments
A policy-conditional world model trained under active inference enables single-lookahead planning over hundreds of steps and beats a DQN baseline on energy-efficient control of parallel machines.
-
Equivariant Eikonal Neural Networks: Grid-Free, Scalable Travel-Time Prediction on Homogeneous Spaces
E-NES uses Lie-group point-cloud conditioning and equivariant neural fields to make grid-free eikonal travel-time prediction steerable under rotations and translations, with complete invariant features and competitive...
-
PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling
A Fourier-based weight modulation for shared INR networks improves reconstruction of high-frequency PDE fields and enables bidirectional inference between paired solution spaces.
-
MINR: Implicit Neural Representations with Masked Image Modelling
A hybrid of implicit neural representations and masked image modeling, called MINR, reconstructs masked image patches better than MAE in the reported in-domain and out-of-distribution tests with fewer parameters.
Discussion (0). Continue with ORCID to comment.