pith. sign in

arxiv: 2402.05110 · v1 · pith:3UYCR576new · submitted 2024-02-07 · 💻 cs.LG

Opening the AI black box: program synthesis via mechanistic interpretability

Eric J. Michaud , Isaac Liao , Vedang Lad , Ziming Liu , Anish Mudide , Chloe Loughridge , Zifan Carl Guo , Tara Rezaei Kheirkhah
show 2 more authors
Mateja Vukeli\'c Max Tegmark
This is my paper
classification 💻 cs.LG
keywords mipslearnedprogramsynthesisalgorithmcodegpt-4integer
0
0 comments X
read the original abstract

We present MIPS, a novel method for program synthesis based on automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the learned algorithm into Python code. We test MIPS on a benchmark of 62 algorithmic tasks that can be learned by an RNN and find it highly complementary to GPT-4: MIPS solves 32 of them, including 13 that are not solved by GPT-4 (which also solves 30). MIPS uses an integer autoencoder to convert the RNN into a finite state machine, then applies Boolean or integer symbolic regression to capture the learned algorithm. As opposed to large language models, this program synthesis technique makes no use of (and is therefore not limited by) human training data such as algorithms and code from GitHub. We discuss opportunities and challenges for scaling up this approach to make machine-learned models more interpretable and trustworthy.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Interpreting Neural Combinatorial Optimization via Evolving Programmatic Bottlenecks

    cs.AI 2026-06 unverdicted novelty 7.0

    EPB distills NCO models into evolving program portfolios via LLM-driven textual-numerical optimization, matching original performance while exposing stage-dependent heuristic-like behavior.

  2. Explaining Attention with Program Synthesis

    cs.LG 2026-06 unverdicted novelty 7.0

    Language-model-guided program synthesis can approximate transformer attention heads with over 75% IoU fidelity on held-out data and allow replacing 25% of heads with only 16% average perplexity increase.

  3. From I/O to Code with Discovery Agent

    cs.LG 2026-05 unverdicted novelty 7.0

    DIO-Agent frames IO2Code as LLM-driven evolutionary search over programs with a Transformation Priority Premise to favor simple hypotheses, outperforming baselines on a new IO2CodeBench.

  4. Explaining Attention with Program Synthesis

    cs.LG 2026-06 unverdicted novelty 6.0

    Fewer than 1000 synthesized Python programs can replicate attention head behavior in GPT-2, TinyLlama, and Llama-3B at >75% IoU on TinyStories, and replacing 25% of heads raises perplexity by only 16% while preserving...

  5. Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces

    cs.CL 2026-06 unverdicted novelty 5.0

    Reasoning in large output spaces proceeds via shortlisting then fine-grained reasoning; this characterization enables a mechanistic distillation strategy that outperforms standard distillation.

  6. Probing for Representation Manifolds in Superposition

    cs.LG 2026-05 unverdicted novelty 5.0

    Introduces the Manifold Probe to discover representation manifolds in superposition and demonstrates causal steering on time concepts in Llama 2-7b.