Pith. sign in

REVIEW 3 cited by

Automatically Identifying Local and Global Circuits with Linear Computation Graphs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2405.13868 v2 pith:6EOFZBL7 submitted 2024-05-22 cs.LG cs.CL

Xuyang Ge , Fukang Zhu , Wentao Shu , Junxuan Wang , Zhengfu He , Xipeng Qiu This is my paper · ORCID
classification cs.LGcs.CL
keywords circuitslinearmodelcalledcircuitcomputationgraphlocal
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Circuit analysis of any certain model behavior is a central task in mechanistic interpretability. We introduce our circuit discovery pipeline with Sparse Autoencoders (SAEs) and a variant called Transcoders. With these two modules inserted into the model, the model's computation graph with respect to OV and MLP circuits becomes strictly linear. Our methods do not require linear approximation to compute the causal effect of each node. This fine-grained graph identifies both end-to-end and local circuits accounting for either logits or intermediate features. We can scalably apply this pipeline with a technique called Hierarchical Attribution. We analyze three kinds of circuits in GPT-2 Small: bracket, induction, and Indirect Object Identification circuits. Our results reveal new findings underlying existing discoveries.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Verbalizable Representations Form a Global Workspace in Language Models

    cs.CL 2026-07 conditional novelty 7.0 of 10

    Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.

  2. Analyze Feature Flow to Enhance Interpretation and Steering in Language Models

    cs.LG 2025-02 conditional novelty 5.0 of 10

    Cosine similarity between sparse autoencoder features across layers and modules builds flow graphs that explain feature evolution and enable multi-layer steering of language model generation.

  3. Hierarchical Sparse Circuit Extraction from Billion-Parameter Language Models through Scalable Attribution Graph Decomposition

    cs.LG 2026-01 reject novelty 4.0 of 10

    HAGD claims to extract sparse circuits from billion-parameter LMs by hierarchical graph coarsening and GNN-guided search, but the O(n^2 log n) complexity guarantee rests on an unproven greedy-optimality assumption.

Pith tools