Averaging top-K attention heads selected on 10 labeled proteins recovers contacts in one forward pass and outperforms the multi-pass Categorical Jacobian on leakage-clean data for bidirectional protein LMs.
Title resolution pending
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.LG 2years
2026 2verdicts
UNVERDICTED 2representative citing papers
Case study applies SAE probing with enstrophy triage to a continuum-dynamics foundation model and reports intermittent feature consistency that does not align with standard physics while linking some output discrepancies to specific feature changes.
citing papers explorer
-
Protein contacts are already in the attention: a single-forward-pass alternative to the Categorical Jacobian
Averaging top-K attention heads selected on 10 labeled proteins recovers contacts in one forward pass and outperforms the multi-pass Categorical Jacobian on leakage-clean data for bidirectional protein LMs.
-
Sparse probes and murky physics: a case study of interpretability challenges in a foundation model for continuum dynamics
Case study applies SAE probing with enstrophy triage to a continuum-dynamics foundation model and reports intermittent feature consistency that does not align with standard physics while linking some output discrepancies to specific feature changes.