IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers

Shashank Hegde; Vaishnavi B Mohan; Vijayakrishna Naganoor; Yashas Annadani

REVIEW 4 major objections 3 minor 17 references

IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers

T0 review · 4 major / 3 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Orientation selectivity in vision transformers is set by the pretraining objective, and its profile predicts where fine-tuning should begin.

desk verdict Careful descriptive study of orientation selectivity in ViTs, but its headline fine-tuning-depth predictor rests on a peak-alignment claim that the paper's own grids contradict. read the letter →

arxiv 2608.05122 v2 pith:UYDPTH7M submitted 2026-08-05 cs.CV

Vaishnavi B Mohan , Vijayakrishna Naganoor , Yashas Annadani , Shashank Hegde This is my paper · ORCID

classification cs.CV

keywords orientationselectivityvisiontransformersmechanisticinterpretabilityrepresentationalsimilarityfine-tuningdepthLoRAtrainingparadigmsparseautoencoders

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether vision transformers, which mix information globally rather than pooling local patches, nevertheless build the kind of low-level orientation code that the visual cortex builds first. To answer, it introduces a suite of neuroscience-inspired metrics—representational similarity score (RSS), orientation recruitment score (ORS), and tuning bandwidth—and applies them to frozen transformers from five pretraining paradigms at matched relative depths. Its central findings are that the training objective, not architecture or data, governs where orientation selectivity peaks, that orientation-selective units are recruited early in training in shallow layers and then handed off to semantic features in deep layers, and that the RSS curve read off a frozen backbone predicts, almost exactly, which layer to start unfreezing for best downstream fine-tuning. A sympathetic reader would take away a cheap, compute-free rule for choosing a LoRA start layer and a mechanistic account of how low-level features form in transformers.

What carries the argument

The carrying machinery is a relative-depth coordinate plus three metrics. Relative depth maps block index via $\mathrm{block}(d,L) = \mathrm{round}(d(L-1))$ over nine depths, so models with 12 to 48 blocks can be compared at the same processing stage. RSS is a representational-similarity score: it correlates pairwise cosine dissimilarities of pooled activations with the circular distance of grating orientations, so a high value means orientation geometry is preserved in the representation, and it is basis-invariant, which is why it is the primary cross-model comparison. ORS counts the fraction of units—raw channels, MLP neurons, or sparse-autoencoder features—whose tuning is significant against a permutation null, and bandwidth is the von Mises half-width at half-maximum of each selective unit's tuning curve. The argument runs on RSS: its shape tracks frozen-backbone generalization, and its peak marks the empirically best LoRA start layer.

What would settle it

Run a held-out ViT backbone, not among the six studied, through the same RSS probe and a full LoRA start-layer sweep on Taskonomy: if the RSS peak differs from the performance peak by more than one depth-grid step, the prediction claim fails. Separately, train two same-family backbones with identical objectives but widely different depths and widths: if their RSS curves do not overlay when aligned by relative depth but do align by absolute block index or parameter count, the relative-depth claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that orientation selectivity is a real, measurable organization of ViT representations, that its depth profile is determined primarily by the pretraining paradigm, and that this profile has operational value. Concretely: across five pretraining families, RSS follows an inverted-U over depth but the location of the peak differs—DINOv2/DINOv3 sustain it to relative depth 0.6, AIMv2 collapses by 0.1, and MAE keeps rising to the final layer; within a family, tiers overlay almost exactly when aligned by relative depth; and over training, shallow layers mature orientation selectivity early while deep layers trade it for semantic content. The load-bearing application claim is that RSS, computed once on the frozen backbone before any fine-tuning, predicts the best unfreeze depth for LoRA adaptation almost exactly, beating nine other a-priori predictors and the default of adapting the whole model. If right, this gives a compute-free criterion for a decision that otherwise requires an expensive sweep.

Load-bearing premise

The paper's cross-model comparisons line up different models by relative depth, assuming that two blocks at the same fraction of their network's depth are genuinely at the same processing stage; if the right coordinate is parameter count, compute, or absolute block index, the conclusions about training objectives could change.

Editorial extensions

If this is right

When choosing a frozen encoder, its RSS curve—not just its final accuracy—predicts how well it will transfer across a set of divergent visual tasks.
For LoRA fine-tuning, the RSS peak read from the frozen backbone gives the start layer without a sweep, and adapting the whole model is often worse than starting at that peak.
Backbones that sustain a low-level orientation code deep into the network (the DINO family) are the strongest frozen encoders, while backbones that drop it early or never hand it off transfer worse.
Pretraining objective is a first-order determinant of internal representation structure, so two models with similar accuracy can have very different layer-wise features.
Because RSS is basis-invariant and comparable across scales, the same probe can rank fine-tuning depth for a newly released ViT backbone without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: The same RSS readout could plausibly rank not just where to start LoRA but how much of the network to freeze during continued pretraining, turning the orientation handoff into a layer-completion signal; the paper only conjectures this.
Editorial inference: If the relative-depth alignment holds, RSS offers a task-agnostic representation-maturity probe that could be applied to non-ViT architectures or to other low-level features like spatial frequency and color, which the paper lists as future work.
Editorial inference: The finding that training paradigm dominates suggests that benchmark comparisons among ViT backbones should control for pretraining objective before attributing differences to architecture.
Editorial inference: A testable extension is to use the RSS peak to initialize a small search around nearby layers, potentially reducing fine-tuning sweeps from nine depths to one or two runs.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, the descriptive core is good. The RSS depth profiles, the ORS null that mirrors the frequency-selection step, the linear-decoding control, and the across-paradigm/scale/checkpoint sweep are careful and reproducible enough to re-implement. The observation that training objective, not data or architecture, dominates where the orientation code lives is interesting and plausibly right. Second, the practical claim in Section 5—that RSS measured once on the frozen backbone predicts the optimal LoRA start depth 'almost exactly'—is not currently supported by the paper's own numbers.

The stress-test note about the grid mismatch is correct. For DINOv3 ViT-S/16, the main text says both RSS and pooled performance 'peak at relative depth d=0.25' (layer 4). But the LoRA sweep in App. A.4 uses a nine-point grid d = {0, .09, .27, .36, ...}, with no point at 0.25. The reported argopt-delta in App. A.5 is 0 at k=4, which is d=0.36. The RSS grid (Sec. 3.2) has 0.25, but the LoRA grid has 0.27 and 0.36 nearby. So the two curves were not actually compared at the same depths, and the 'almost exactly' claim has no joint support. Also, block(d,L)=round(d(L-1)) with L=12 maps d=0.25 to block 3, not layer 4, unless you use 1-indexing, but then it's still not the k=4/d=0.36 reported.

This matters because the prescriptive rule stands or falls on peak coincidence. The descriptive measurements don't depend on it, and they hold up. The Section 4.1 takeaway ('training paradigm has the strongest effect') is based on one backbone per family with data and architecture varying simultaneously, so it's suggestive, not airtight. No code or artifacts are shipped, which makes re-implementation harder to verify. And the fine-tuning validation uses six backbones, one tiny Taskonomy split, and a coarse depth grid.

Verdict: this deserves a serious referee, but the referee should ask for the peak-coincidence claim to be either fixed with proper grid alignment and interpolation or downgraded to a qualitative observation. The descriptive analysis is worth engaging with. I'd want to see the paper again after revision.

Referee Report

4 major / 3 minor

Summary. The paper introduces IRIS, a suite of three proposed orientation-selectivity metrics—RSS, ORS, and von Mises bandwidth—and probes frozen vision transformers with oriented gratings and natural images across five pretraining paradigms, multiple scales, and training checkpoints. The authors report that (1) the training paradigm is the strongest determinant of orientation selectivity, with models sharing an objective peaking at comparable relative depths; (2) orientation selectivity is recruited early in training in early-to-middle layers and lost in deeper layers; and (3) the RSS curve measured on a frozen backbone predicts the best depth at which to start LoRA fine-tuning, beating nine other a-priori predictors. The load-bearing claim is the third one, stated in Section 5 and App. A.4-A.5.

Significance. If the central claims hold, the paper would provide a compute-free, mechanistically motivated rule for choosing a LoRA start layer and a biologically grounded lens on how ViTs organize low-level features. The measurement design has genuine strengths: the ORS null in Eq. (5) mirrors the frequency-selection step in Eq. (2), so the significance test is not inflated by selecting the best frequency; RSS in Eq. (3) is basis-invariant; and the decoding control in App. A.2 shows that orientation remains linearly decodable even where RSS declines. The frozen-before-fine-tuning evaluation in Section 5 is also a real a-priori prediction rather than a post-hoc fit. However, the central practical claim currently rests on inconsistent peak coordinates and incompatible depth grids, and the cross-paradigm attribution is under-identified by the experimental design.

major comments (4)

[Section 5, App. A.4-A.5.] The claim that RSS's peak coincides 'almost exactly' with the optimal LoRA start depth is not supported as stated, because the two curves are evaluated on different depth grids and the reported peak coordinates disagree. For DINOv3 ViT-S/16, App. A.4 fixes the LoRA grid to k in {0,1,3,4,6,7,8,10,11}, i.e., d in {0,.09,.27,.36,.55,.64,.73,.91,1}; the empirical optimum reported in App. A.5 is k=4, which is d=0.36, yet Section 5 states that both RSS and performance peak at d=0.25. Under the mapping block(d,L)=round(d(L-1)) of Section 3.2, d=0.25 maps to block 3 for L=12, not to block 4 unless a one-based layer count is intended; and for L=24 it maps to block 6, which is not 'layer 6' on a one-based count. The RSS grid in Section 3.2 contains d=0.25 but not d=0.36, so no comparison is reported at the empirical optimum without interpolation. Please report RSS on the same depth grid as the LoRA sweep, or provide interpolation with an explicit tolerance, and reconcile the layer-number convention.
[Section 4.1 and Table 2.] The design with one representative backbone per paradigm cannot support the takeaway that 'training paradigm has the strongest effect, not data or architecture', because paradigm, architecture, training data, patch size, and likely training compute vary jointly across the five families. The sentence in Section 4.1 claiming that any difference 'reflects the objective rather than architecture, scale, or the probe' is not established by the experiment; the within-contrastive control (OpenCLIP, MetaCLIP, SigLIP2) varies data and loss but does not vary architecture. This is load-bearing for contribution (1) and should either be softened to a correlational statement or supplemented with additional backbones per paradigm.
[Section 3.2 and Section 4.2.] The alignment of all models by relative depth is an assumption that the paper never tests, and the cross-scale conclusion that 'relative depth, not block number, governs the strength of the orientation code' depends on it. If the correct cross-architecture coordinate is, for example, parameter count, width, or training compute, the close overlay of tiers in Figure 3 could be an artifact of the chosen mapping. Please test robustness with at least one alternative alignment, such as absolute block index or a parameter-count-based coordinate, or explicitly report the comparison under those alternatives.
[Section 5 and App. A.4.] The empirical performance optimum is estimated from 5 seeds on a single 983-train/157-val Taskonomy split, and the paper reports no confidence interval or tolerance on the location of the performance peak. Given that adjacent LoRA start depths for ViT-S/16 differ by 0.09 in relative depth, statements like 'almost exactly' and 'lands exactly' require either a variance estimate or an explicit tolerance on the peak location before the practical rule can be considered reliable.

minor comments (3)

[References.] The Vaswani et al. reference contains a stray '/suppress Lukasz Kaiser' in the author list; please correct it.
[Figure 5.] The panel labels 'DINOv3_B', 'MAE_B', and so on are not defined in the caption; clarify whether 'B' denotes Base tier and 'L' denotes Large tier.
[App. A.5.] The statement that argopt-delta is 0 with 'k=4 exactly' is hard to reconcile with the Section 5 claim that DINOv3 peaks at d=0.25; please align the depth conventions throughout the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: metrics are fixed probes; the RSS-to-LoRA-depth claim is an empirical alignment compared against an independent sweep.

full rationale

Walking the claimed derivation chain, I find no step that reduces to its own inputs by definition or by self-citation. The core metrics (RSS, OSI, ORS, HWHM) are fixed, closed-form functions of frozen-model activations and grating stimuli: RSS is a Pearson correlation between activation dissimilarities and orientation dissimilarities (Eq. 3), OSI is a circular-variance measure (Eq. 4), ORS uses a permutation null that matches the frequency-selection step (Eq. 5), and HWHM is a von Mises fit to each unit's own tuning curve (Eq. 6). No constant in any of these metrics is fit to downstream performance, LoRA results, or the generalization ranking. The Section 5 claim that RSS predicts the best LoRA start depth is an empirical alignment test: the empirical optimum comes from a separate 450-run Taskonomy fine-tuning sweep (App. A.4) across 9 tasks and 5 seeds, while RSS is computed once on the frozen backbone before any fine-tuning. The performance curve is not defined, normalized, or selected using RSS; the comparison is post-hoc correlation of two independently measured curves. The App. A.5 ranking of RSS against 9 other a-priori predictors likewise scores each frozen-backbone metric against the same independent performance sweep, which is model comparison rather than circular reasoning. The paper's cited prior work (SAEs, RSA, LoRA) is external and is used as standard methodology; there are no load-bearing self-citations by the present authors. The internal inconsistencies flagged in the skeptic analysis, e.g., the DINOv3 RSS peak reported as d=0.25 in the main text versus k=4 (d=0.36) in App. A.4, and the mismatch between the RSS probe grid and the LoRA sweep grid, are accuracy and internal-consistency concerns about how precisely the peak coincidence is established; they do not show that the prediction is equivalent to its inputs by construction. Similarly, the assumption that relative depth rather than absolute block index is the correct alignment coordinate is a modeling choice that could be wrong, but it is not a circular reduction of the cross-scale claim. The paper also explicitly states its own scope limits, e.g., that progressive-freezing guidance is 'not validated' (Sec. 4.3) and that 'No single statistic here is a formal significance test' (App. A.5), which are honest limitations rather than disguised circularity.

Assumptions & free parameters 6 free parameters · 8 assumptions · 0 invented entities

The central claims rest on three families of assumptions: (i) RSA and energy-model conventions borrowed from neuroscience, (ii) SAE unitization borrowed from mechanistic interpretability, and (iii) paper-specific choices, most notably linear relative-depth alignment and one-model-per-paradigm attribution. The paper-specific choices are the least defended: the first is never tested against alternative alignments, and the second is only partially controlled by the within-contrastive comparison. No new physical or architectural entities are postulated; the four metrics are measurement constructs defined on activations.

free parameters (6)

SAE sparsity coefficient lambda = 1e-3
Hand-chosen for all SAEs; ORS recruitment fractions and their cross-model comparability depend on it (App. A.1.1, Eq. 8).
SAE dictionary size = 2048 features
Fixed per (model, block) pair; the pool of candidate units for ORS is set by this choice.
SAE training budget = 2000 steps, batch 1024, 256 Imagenette images
If dictionaries converge unevenly across models or blocks, ORS differences could reflect SAE quality rather than true selectivity.
LoRA rank and scale = r=8, alpha=16
Adapter capacity could shift the optimal start depth; no sweep over rank or scale is reported (App. A.4).
ORS significance threshold = alpha=0.05
Conventional choice; the recruitment fraction scales with this threshold (Eq. 5).
Von Mises fit parameters (b, A, kappa, theta_pref) = fit per selective unit, individual values not reported
Descriptive fit used only to summarize bandwidth (Eq. 6); HWHM statistics depend on fit quality.

assumptions (8)

standard math Cosine dissimilarity on pooled, l2-normalized activations is a valid basis-free geometry for comparing representations (RSA assumption).
Underpins RSS (Eq. 3); standard practice since Kriegeskorte et al. 2008.
domain assumption The energy model (root-mean-square over phases) makes tuning phase-invariant (Eq. 2).
Copied from V1 modeling; assumes phase pooling is the correct combination rule for transformer units.
ad hoc to paper Linear relative-depth alignment block(d,L)=round(d(L-1)) is the correct cross-architecture coordinate (Sec. 3.2).
Load-bearing for the paradigm and scale claims; never validated against alignment by parameter count, absolute block, or training compute.
domain assumption SAE features trained on 256 Imagenette images are fair monosemantic units across all models and blocks (App. A.1).
ORS comparability presumes equally well-trained dictionaries at every (model, block) pair.
ad hoc to paper One representative backbone per family isolates the training objective (Sec. 4.1).
Data, architecture, patch size, and compute vary together across the five families; only the within-contrastive set varies fewer factors.
ad hoc to paper AIMv2's deviating RSS scale behavior is explained by width-scaling (Sec. 4.2).
Evidence from a single family; no controlled width-versus-depth experiment is run.
domain assumption Tuning curves are von Mises shaped (Eq. 6).
Used for the closed-form HWHM; capped at 90 degrees for kappa below ln 2.
domain assumption MLP neurons provide a stable per-neuron identity across checkpoints because GELU fixes a basis (Sec. 4.3).
Supports the training-trajectory analysis; the residual stream is rotationally uncommitted.

how reviews work

0 comments

Cite this review

Pith. "Pith review of IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers." pith.science (2026). https://pith.science/paper/UYDPTH7M

@misc{pith2026260805122,
  author       = {Pith},
  title        = {Pith review of: IRIS: A Visual Cortex-Inspired Framework for Analyzing Orientation Selectivity in Vision Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UYDPTH7M}},
  note         = {Machine review of arXiv:2608.05122}
}

read the original abstract

Vision transformers (ViTs) have become the de facto standard for image encoding across many perception tasks. Despite their empirical success, it remains mechanistically unclear how they encode low-level features, given their lack of inductive biases: ViTs process information globally rather than relying on local structure. Biological visual systems, in contrast, build low-level features, such as orientation selectivity in the primary visual cortex, by combining information from small, localized regions of the visual field. These features are general-purpose representations, shared and required across multiple specialized neural pathways, unlike higher-level, task-specific semantic features. This raises the question if such biologically-grounded features arise in ViTs. In this work, we systematically study how orientation selectivity emerges in ViTs by introducing a suite of neuroscience-inspired metrics: representational similarity score (RSS), orientation recruitment score (ORS), and orientation tuning bandwidth to quantify how orientation is encoded in representational geometry and as a function of model depth. Through extensive analysis, we find that: (1) the training paradigm is the strongest determinant of orientation selectivity, with models sharing an objective, peaking at comparable relative depths regardless of scale (2) many units are orientation-selective early in training, with early-to-middle layers recruiting more such units over time, while deeper layers lose selectivity and broaden their tuning toward semantic encoding and (3) our metrics offer a mechanistic heuristic for how many layers to unfreeze for best downstream generalization. Our framework presents a way to track biologically-grounded features during ViT training, probes how desired properties are encoded in transformer representations, and builds a systematic understanding of how ViTs generalize across tasks.

Figures

Figures reproduced from arXiv: 2608.05122 by the authors.

**Figure 2.** Orientation selectivity across training paradigms. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** We plot RSS vs. relative depth, for each family, across tiers. Tiers overlay almost exactly within [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (7 more)

**Figure 3.** Figure 3: We plot RSS vs. relative depth, for each family, across tiers. Tiers overlay almost exactly within [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]

**Figure 4.** Figure 4: Orientation selectivity over OpenCLIP ViT-H/14’s training trajectory. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Pooled downstream performance (solid) against a-priori RSS on the frozen backbone (dashed), [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: We plot mean min–max normalized performance [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

**Figure 6.** Figure 6: Mean min–max normalized performance S(b) (Eq. 11) for the six L-tier frozen backbones, pooled across all 9 tasks: within each task the weakest backbone scores 0 and the strongest 1, and S(b) averages this per-task rank across tasks – a relative ranking, not an absolute…

**Figure 7.** Figure 7: Protocol schematic. Top: the frozen backbone is probed once with the RSA battery at every block, giving an a-priori RSS-vs-block curve, no fine-tuning required. Bottom: for each block k, the same backbone is adapted with LoRA on blocks [k :] and fine-tuned across the 9…

**Figure 7.** Figure 7: Protocol schematic. Top: the frozen backbone is probed once, giving an a-priori RSS-vs-block curve. Bottom: the same backbone is adapted with LoRA on blocks [k :] and fine-tuned across the 9 Taskonomy-style tasks (5 seeds), giving an empirical performance-vs-k curve. R…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 3 canonical work pages

[2]

across all 9 benchmark tasks (dataset and decoder head for each task in Table 3; full protocol in App. A.3). Each entry is the best checkpoint’s score on that task’s own validation split and primary metric. Green indicates the best- performing backbone on a task and terracotta the worst; retrieval is a tie between two backbones. Metrics differ in scale ac...

work page 1982
[6]

Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang

Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views.arXiv preprint arXiv:2511.10647,

arXiv
[7]

For each of six backbones, we adapt progressively more of the network with LoRA from a range of starting depthsk, measure downstream performance as a function ofk, and compare it to RSS’s a-priori, no-fine-tuning-required curve on the same frozen backbone – testing whether RSS’s peak predicts where fine-tuning performance actually peaks. These six backbon...

work page 2022
[11]

Griffiths

Shikhar Tuli, Ishita Dasgupta, Erin Grant, and Thomas L. Griffiths. Are convolutional neural networks or transformers more like human vision?arXiv preprint arXiv:2105.07197,

arXiv
[13]

arXiv:2502.20578. Amir R. Zamir, Alexander Sax, William Shen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Taskon- omy: Disentangling task transfer learning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

arXiv
[14]

A large- scale study of representation learning with the visual task adaptation benchmark.arXiv preprint arXiv:1910.04867,

Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, and Neil Houlsby. A large- scale study of representation learning with the visual task a...

arXiv 1910
[1962]

Prisma: An open source toolkit for mechanistic interpretability in vision and video.arXiv preprint arXiv:2504.19475,

Sonia Joseph, Praneet Suresh, Lorenz Hufe, Edward Stevinson, Robert Graham, Yash Vadi, Danilo Bzdok, Sebastian Lapuschkin, Lee Sharkey, and Blake Aaron Richards. Prisma: An open source toolkit for mechanistic interpretability in vision and video.arXiv preprint arXiv:2504.19475,

arXiv
[1993]

13-12-05251.1993

doi: 10.1523/JNEUROSCI. 13-12-05251.1993. Taco Cohen and Max Welling. Group equivariant convolutional networks. InInternational Conference on Machine Learning, pages 2990–2999,

arXiv 1993

Show all 17 references

[2001]

Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson

doi: 10.1038/35082568. Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? InNeurIPS,

doi
[2002]

Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J

doi: 10.1523/ JNEUROSCI.22-13-05639.2002. Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J. Majaj, Rishi Rajalingham, Elias B. Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Kailyn Schmidt, Franziska Geiger, Daniel L. K. Yamins, and James J. DiCarlo. Brain-score: W...

2002
[2009]

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer

doi: 10.1109/CVPR.2009.5206848. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. InNeurIPS,

2009
[2018]

Oriane Sim´ eoni, Huy V

doi: 10.1101/407007. Oriane Sim´ eoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha¨ el Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timoth´ ee Darcet, Th´ eo M...

doi
[2019]

oblique effect

A Supplementary Materials A.1 Feature Dictionary This section details the SAE readout basis introduced as ‘SAE features’ in Sec. 3.2. Why a dictionary.We use the sparse dictionary specifically for computing ORS (recruitment): determin- ing which units are genuinely orientation...

2023
[2021]

Steering clip’s vision transformer with sparse autoencoders.arXiv preprint arXiv:2504.08729,

Sonia Rao et al. Steering clip’s vision transformer with sparse autoencoders.arXiv preprint arXiv:2504.08729,

arXiv
[2023]

Horace B

doi: 10.1016/j.neunet.2023.04.032. Horace B. Barlow. Possible principles underlying the transformation of sensory messages. In Walter A. Rosenblith, editor,Sensory Communication, pages 217–234. MIT Press, Cambridge, MA,

2023 doi
[2024]

Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey

doi: 10.1038/s41467-024-53147-y. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600,

arXiv
[2025]

Vo, Camille Couprie, Patrick Labatut, Piotr Bojanowski, Valentin Wyart, and Jean-R´ emi King

Jos´ ephine Raugel, Marc Szafraniec, Huy V. Vo, Camille Couprie, Patrick Labatut, Piotr Bojanowski, Valentin Wyart, and Jean-R´ emi King. Disentangling the factors of convergence between brains and computer vision models.arXiv preprint arXiv:2508.18226,

arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [2]

across all 9 benchmark tasks (dataset and decoder head for each task in Table 3; full protocol in App. A.3). Each entry is the best checkpoint’s score on that task’s own validation split and primary metric. Green indicates the best- performing backbone on a task and terracotta the worst; retrieval is a tie between two backbones. Metrics differ in scale ac...

work page 1982

[2] [6]

Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang

Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views.arXiv preprint arXiv:2511.10647,

arXiv

[3] [7]

For each of six backbones, we adapt progressively more of the network with LoRA from a range of starting depthsk, measure downstream performance as a function ofk, and compare it to RSS’s a-priori, no-fine-tuning-required curve on the same frozen backbone – testing whether RSS’s peak predicts where fine-tuning performance actually peaks. These six backbon...

work page 2022

[4] [11]

Griffiths

Shikhar Tuli, Ishita Dasgupta, Erin Grant, and Thomas L. Griffiths. Are convolutional neural networks or transformers more like human vision?arXiv preprint arXiv:2105.07197,

arXiv

[5] [13]

arXiv:2502.20578. Amir R. Zamir, Alexander Sax, William Shen, Leonidas Guibas, Jitendra Malik, and Silvio Savarese. Taskon- omy: Disentangling task transfer learning. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),

arXiv

[6] [14]

A large- scale study of representation learning with the visual task adaptation benchmark.arXiv preprint arXiv:1910.04867,

Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, and Neil Houlsby. A large- scale study of representation learning with the visual task a...

arXiv 1910

[7] [1962]

Prisma: An open source toolkit for mechanistic interpretability in vision and video.arXiv preprint arXiv:2504.19475,

Sonia Joseph, Praneet Suresh, Lorenz Hufe, Edward Stevinson, Robert Graham, Yash Vadi, Danilo Bzdok, Sebastian Lapuschkin, Lee Sharkey, and Blake Aaron Richards. Prisma: An open source toolkit for mechanistic interpretability in vision and video.arXiv preprint arXiv:2504.19475,

arXiv

[8] [1993]

13-12-05251.1993

doi: 10.1523/JNEUROSCI. 13-12-05251.1993. Taco Cohen and Max Welling. Group equivariant convolutional networks. InInternational Conference on Machine Learning, pages 2990–2999,

arXiv 1993