Feedback in Imitation Learning: The Three Regimes of Covariate Shift
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:BR2KXGU2record.jsonopen to challenge →
read the original abstract
Imitation learning practitioners have often noted that conditioning policies on previous actions leads to a dramatic divergence between "held out" error and performance of the learner in situ. Interactive approaches can provably address this divergence but require repeated querying of a demonstrator. Recent work identifies this divergence as stemming from a "causal confound" in predicting the current action, and seek to ablate causal aspects of current state using tools from causal inference. In this work, we argue instead that this divergence is simply another manifestation of covariate shift, exacerbated particularly by settings of feedback between decisions and input features. The learner often comes to rely on features that are strongly predictive of decisions, but are subject to strong covariate shift. Our work demonstrates a broad class of problems where this shift can be mitigated, both theoretically and practically, by taking advantage of a simulator but without any further querying of expert demonstration. We analyze existing benchmarks used to test imitation learning approaches and find that these benchmarks are realizable and simple and thus insufficient for capturing the harder regimes of error compounding seen in real-world decision making problems. We find, in a surprising contrast with previous literature, but consistent with our theory, that naive behavioral cloning provides excellent results. We detail the need for new standardized benchmarks that capture the phenomena seen in robotics problems.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
Memory Retrieval in Visuomotor Policies for Long-Horizon Robot Control
HALO distills VLM priors via question-answering objectives and applies sparse attention to enable reliable memory retrieval from up to eight minutes of history in imitation-learned visuomotor policies.
-
Behavior Cloning Under PD Control: A Finite-Horizon Theory of Gain-Dependent Error Amplification
Nonasymptotic analysis shows compliant overdamped PD controllers minimize position error tails in behavior cloning by bounding gain-dependent amplification of sub-Gaussian action errors.
-
Behavior Cloning Under PD Control: A Finite-Horizon Theory of Gain-Dependent Error Amplification
Nonasymptotic analysis shows sub-Gaussian action errors in behavior cloning propagate through gain-dependent closed-loop dynamics to produce sub-Gaussian position errors whose tail is governed by a proxy matrix and am...
-
Difference-Aware Retrieval Policies for Imitation Learning
DARP reparameterizes imitation learning around local neighborhood structure using k-NN expert states, actions, and relative distance vectors, delivering 15-46% gains over behavior cloning in control and manipulation tasks.
-
Beyond Self-Play: Hierarchical Reasoning for Continuous Motion in Closed-Loop Traffic Simulation
A hierarchical Stackelberg MARL plus continuous-motion architecture with hybrid co-training produces smoother and safer closed-loop traffic behavior than standard self-play methods.
-
Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulation
An instance-centric representation with local frames, relative positional encodings, and adaptive reward transformation in adversarial IRL yields scalable, accurate, and robust behavior models for multi-agent driving ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.