REVIEW 2 cited by
Avoiding Tampering Incentives in Deep RL via Decoupled Approval
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
How can we design agents that pursue a given objective when all feedback mechanisms are influenceable by the agent? Standard RL algorithms assume a secure reward function, and can thus perform poorly in settings where agents can tamper with the reward-generating mechanism. We present a principled solution to the problem of learning from influenceable feedback, which combines approval with a decoupled feedback collection procedure. For a natural class of corruption functions, decoupled approval algorithms have aligned incentives both at convergence and for their local updates. Empirically, they also scale to complex 3D environments where tampering is possible.
Forward citations
Cited by 2 Pith papers
-
Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI
A post-hoc audit framework finds that many agent benchmark scores are inflated by protocol exposures (public answers, readable hidden state, generator regularities, feedback, or scoring flaws), with measured inflation...
-
RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation
RIVAL iteratively re-trains a reward model adversarially against the current translator and adds a BLEU-predicting head, improving in-domain WMT and subtitle translation over SFT baselines.
Discussion (0). Continue with ORCID to comment.