Pith. sign in

REVIEW 4 major objections 6 minor 25 references

Position: We Need An Algorithmic Understanding of Generative AI

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs don't match BFS or DFS when navigating graphs

desk verdict A useful position paper on algorithmic understanding of LLMs whose empirical case study overclaims: the BFS/DFS negative result is underdetermined without a positive control or chance baseline. read the letter →

arxiv 2507.07544 v1 pith:6JRSGEU7 submitted 2025-07-10 cs.AI cs.LG

Oliver Eberle , Thomas McGee , Hamza Giaffar , Taylor Webb , Ida Momennejad This is my paper
classification cs.AIcs.LG
keywords algorithmicunderstandinggenerativeAIlargelanguagemodelsinterpretabilityemergentalgorithmsgraphnavigationattentionanalysisrepresentationalsimilarity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that the machine-learning community should prioritize algorithmic understanding of generative AI—uncovering the step-by-step procedures, or algorithmic primitives, that LLMs actually learn—and proposes AlgEval, a framework for doing so. The paper grounds this argument in a case study: on a simple tree-graph navigation task, instruction-tuned Llama-3.1 (8B and 70B) does not appear to implement classical breadth-first or depth-first search. Instead, attention and representation analyses show incremental attention toward the correct path, a late peak of attention to the goal token, and progressive separation of goal from non-goal representations. The authors conclude that current models use a policy-dependent, non-exhaustive strategy rather than building a full world model or running explicit search, and they sketch how algorithmic explanations could yield more sample-efficient training, lower emissions, and better multi-agent designs.

What carries the argument

The central object is AlgEval, a research framework that treats algorithms as compositions of reusable algorithmic primitives (a 'vocabulary and grammar') and prescribes identification of primitives, construction of simple tasks with verifiable ground truth, cataloging of mechanisms, composition analysis, and ablations. The case study's operative machinery is a two-step analytic procedure: (1) cascading attention analysis, which traces how the final token's attention and each node's attention to predecessors evolve across layers, and (2) representational similarity analysis, which extracts a candidate algorithmic step per layer by finding the node pair with highest representational similarity in that layer's activation matrix and compares the resulting sequence to BFS/DFS rollouts via edit distance and longest-correct-subsequence measures. These tools convert the abstract question 'what algorithm does the model run?' into a concrete comparison between layer-wise internal states and verifiable search trajectories.

What would settle it

Train a small transformer on the same two-step tree-graph task with supervision that explicitly installs a known search rollout (BFS or DFS), then run the layer-wise representational-similarity extraction and check whether the inferred step sequence matches the installed rollout. If the method fails to recover a known algorithm, the reported mismatch for Llama-3.1 cannot by itself rule out search-like computation; if it succeeds, the negative result gains force.

Watch

Extended reading notes

Core claim

The paper's central discovery is that on a graph-navigation task with verifiable ground truth, the step-by-step sequence of node pairs inferred from Llama-3.1's layer-wise representations matches neither breadth-first search nor depth-first search (mean proportion of correct matching steps: 0.18 for BFS, 0.24 for DFS), even under a maximally permissive comparison that does not require adjacent layers. Complementing this, the final token's attention rises significantly toward rooms on the correct path in 14 of 32 layers, with only three layers favoring the incorrect path, and attention to the goal node peaks in middle layers before the final token commits to the correct response. In representation space, the goal node and its closest competitor separate progressively across layers, while non-goal nodes cluster together. The authors read these patterns as evidence for a policy-dependent, incremental algorithm—attending to predecessors along the goal-directed path—rather than exhaustive search or a full world model, and they take the case study as a demonstration of how top-down algorithmic hypotheses can be tested bottom-up through attention and representation analysis.

Load-bearing premise

The negative result rests on the assumption that one transformer layer corresponds to one visited node or node pair in the search, and that the node pair with the highest representational similarity in a layer reveals which step the algorithm took.

Editorial extensions

If this is right

  • If the case study generalizes, scale alone is unlikely to produce principled search on navigation tasks; algorithmic-level interventions in architecture, training, or inference-time compute would be needed.
  • AlgEval's five-step loop—identifying primitives, building tasks with verifiable ground truth, cataloging mechanisms, analyzing composition, and ablating—offers a concrete way to turn interpretability findings into falsifiable algorithmic claims.
  • Inference-time compute (chain-of-thought, tree search, agentic systems) should be analyzed with the same primitive/composition lens, and its outputs must be causally linked to actual performance because chain-of-thought can be unfaithful.
  • Algorithmic understanding promises more sample-efficient training, lower emissions, and principled design of multi-agent systems, because it targets the computations that actually drive behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the case study's conclusion would be to run the same layer-wise extraction on a model explicitly trained to perform BFS/DFS on graph navigation; recovering the known rollout would strengthen the negative result, while failing to recover it would suggest the mismatch is an artifact of the layer-to-step mapping assumption.
  • The progressive separation between goal and competitor representations resembles winner-take-all competition observed in other settings; if it is a reusable algorithmic primitive, it should appear across tasks and models and could serve as a building block for reasoning beyond navigation.
  • If algorithmic evaluation were added to standard benchmarks, two models with identical task accuracy could be distinguished by the algorithms they implement, changing how model comparison and 'emergent ability' claims are made.
  • The framework implies that an 'algorithmic similarity' measure between models (how much their primitive vocabularies and compositions overlap) could predict transfer and complementarity in multi-agent systems.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This position paper argues that the ML community should prioritize an algorithmic understanding of generative AI, and proposes a research framework called AlgEval. The framework is built around algorithmic primitives, their composition via algorithmic grammars, and a battery of evaluation methods (attention analysis, representational similarity, circuit discovery, inference-time compute). To ground the proposal, the authors present a case study on graph navigation with Llama-3.1-8B and 70B, in which they test whether LLMs implement classical search algorithms (BFS/DFS). The case study reports that attention patterns favor correct paths and that layer-wise extracted node-pair sequences do not match BFS/DFS rollouts (mean proportion of correct steps 0.18 for BFS, 0.24 for DFS), concluding that current models do not construct a full world model or perform exhaustive search. The paper also discusses alternative views, theoretical directions, and broader impacts.

Significance. If the AlgEval agenda is adopted, it could reorient interpretability research from isolated circuit discovery toward a systematic, hypothesis-driven account of computation in LLMs, with potential benefits for sample efficiency, architecture design, and multi-agent systems. The paper's emphasis on top-down algorithmic hypotheses combined with bottom-up testing is a valuable corrective to purely exploratory interpretability. The case study is a concrete demonstration of the proposed pipeline, and the authors are transparent about some assumptions. However, the empirical grounding is thin and currently insufficient to support the strong negative claim about exhaustive search and world models; the significance of the paper therefore rests more on the position and framework than on the case study results.

major comments (4)
  1. [Section 4.2, 'Comparing LLM vs. hypothesis sequences'] The central negative claim that the model does not perform BFS/DFS rests on the low mean proportion of correct matching steps (0.18 for BFS, 0.24 for DFS). This result is reported without any chance baseline or error bars. Because the matching criterion is maximally permissive (steps need only appear in order, not in adjacent layers), a random sequence of node pairs on the 7-node tree could plausibly match a substantial fraction of a rollout. To support the negative conclusion, the authors should report (a) the distribution of match scores under a null model (e.g., permuted edge sequences or random node pairs with the same permissiveness), and (b) a positive control: a setting where the layer-to-step extraction is known to detect a search algorithm if one is present, such as a synthetic transformer trained to implement BFS/DFS or a known algorithmically-aligned model. Without these, the low match could be an artifact of the decoding pipeline rather than evidence against search-style computation.
  2. [Section 4.2, 'Analysis of Feedforward Representations' and Appendix Figure 6] The layer-to-step mapping assumes that each transformer layer corresponds to one visited node pair and that the highest-similarity pair indicates the algorithm's next step. The authors acknowledge this assumption, but it is not validated. Notably, Appendix Figure 6 shows that representational similarity between neighboring layers remains high (above 0.95) with no step-like discontinuities, which is difficult to reconcile with the premise that each layer implements a distinct search step. If the mapping is wrong (e.g., search steps are distributed over multiple layers, or multiple edges are evaluated in parallel), the low match to BFS/DFS says nothing about whether the model implements some other search-like algorithm. The paper should either validate the layer-to-step correspondence (e.g., through intervention experiments or comparison on a model with a known algorithm) or substantially soften the interpretation.
  3. [Section 4.2, 'Interpretation'] The conclusion that 'current models do not construct a full world model or perform exhaustive search' is a logical overreach relative to the evidence. The analyses show that the model's attention and representations do not align with BFS/DFS rollouts, but absence of evidence for exhaustive search is not evidence for absence of a world model. A model could internally represent the graph structure (a world model) and still fail to match the specific rollout decoding used here. The paper should either provide a more direct test of world-model representation (e.g., probing for graph structure in hidden states) or restrict the claim to 'the extracted step sequences do not match BFS/DFS.'
  4. [Section 4, case study setup] The empirical case study is based on a single 2-2 tree graph with one prompt template and four goal nodes (W, Q, V, M), evaluated on two model sizes. The interpretation in Section 4.2 generalizes to 'current models' without supporting evidence. While the paper is a position piece and the case study is explicitly illustrative, the strength of the negative conclusion should be calibrated to the narrowness of the task; at minimum, the authors should acknowledge this limitation in the main text (not only in future-work remarks) and avoid generalization beyond the specific graph size and structure.
minor comments (6)
  1. [Section 1, last paragraph] The phrase 'Algorithmic explanations of LLMs, therefore, involves uncovering...' has a subject-verb agreement error; change 'involves' to 'involve'.
  2. [Appendix A.1] The heading says 'Paired-sample t-tests for attention from the final token to correct vs. incorrect pathways,' but the reported analysis is a linear mixed-effects model (Table 1). Please align the heading with the actual method or clarify why both are described.
  3. [Section 4.2, t-SNE analysis] The t-SNE-based claims about 'progressive separation' and 'clustering' are qualitative and depend on hyperparameters; consider adding quantitative metrics (e.g., representational distances with confidence intervals) or explicitly labeling these as qualitative observations.
  4. [Section 4.1, Figure 3] The attention heatmaps in Figure 3 would benefit from a unified color scale across panels and a brief description of what the rows and columns represent, especially the distinction between 'node token W' and 'final token' attention.
  5. [Abstract and Section 6] The phrase 'enabling comprehension of the model's internal reasoning performance measures' is awkward and the meaning is unclear; consider rephrasing to something like 'enabling comprehension of the model's internal reasoning and performance measures.'
  6. [Impact Statement] The claim about GPT-3's water/carbon usage per interaction cites a 2025 paper but uses pre-2023 data; please update or contextualize the numbers to avoid outdated impressions.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the empirical case study is observational and the framework's claims do not reduce to fitted inputs or to self-citations.

full rationale

The paper is primarily a position paper advocating an algorithmic-understanding research program, and its central thesis is not derived from any fitted quantity or self-citation chain. The case study in Section 4 uses fixed, pretrained Llama-3.1 models and compares observed attention and representation structure against classical BFS/DFS rollouts; no parameter is fit to make the comparison come out one way or the other, so the reported low match proportions (0.18 BFS, 0.24 DFS) are not forced by construction. The layer-to-node-pair mapping is explicitly stated as an assumption ('This analysis assumes that each layer corresponds to one visited node or node pair and that multiple connections may be evaluated simultaneously') rather than imported as an external theorem, so the negative result is underdetermined as an empirical inference but not circular. Self-citations such as Momennejad et al. (2023) for the prompt and for motivating LLM planning failures, Webb et al. (2024), and Yousefi et al. (2024) are contextual and not load-bearing: the case study's observations rest on the models' own attention weights and hidden states, not on the truth of those prior papers. No uniqueness theorem from the authors is invoked, no ansatz is smuggled in via citation, and no known empirical pattern is merely renamed. Concerns about missing positive controls or chance baselines are validity or underdetermination issues, not circularity, and therefore do not raise the circularity score.

Assumptions & free parameters 0 free parameters · 5 assumptions · 2 invented entities

The case study fits no free parameters; it analyzes public models with fixed weights. The framework introduces conceptual constructs (primitives and grammars) that lack independent falsifiable handles. The main load-bearing assumptions are domain assumptions about the interpretability of attention and representations, the layer-to-step mapping, and the suitability of the graph task as a testbed for search algorithms.

assumptions (5)
  • domain assumption LLM computations can be decomposed into domain-general algorithmic primitives that compose.
    Invoked in Section 3.1 to define the vocabulary and grammar of AlgEval; this is the foundational postulate of the framework, not demonstrated.
  • domain assumption Attention patterns and representational geometry are valid evidence about the algorithm implemented by the model.
    Used throughout Section 3.3 and Section 4 to infer algorithmic steps from attention and t-SNE; this is a standard interpretability assumption but unproven.
  • domain assumption Each layer corresponds to one visited node or node pair in the search.
    Stated in Section 4: 'This analysis assumes that each layer corresponds to one visited node or node pair.' This assumption is load-bearing for the BFS/DFS negative result.
  • domain assumption The graph navigation prompt and task are a valid testbed for search algorithms.
    Section 4 uses one two-step tree graph and four goal prompts; the validity of this as a test of search algorithms is assumed.
  • standard math Statistical tests assume independence of attention observations.
    The mixed-effects model in Section 4.1 uses t-statistics on attention values that may be correlated across tokens and layers; the model includes a random intercept for layer but does not account for other dependencies.
invented entities (2)
  • Algorithmic primitives
    purpose: Basic units of computation that compose into algorithms.
    Defined conceptually in Section 3.1; no independent falsifiable handle outside the framework.
  • Algorithmic grammars
    purpose: Composition rules for combining primitives.
    Proposed in Section 3.1 as the 'grammar' for combining primitives; not yet operationalized or tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Position: We Need An Algorithmic Understanding of Generative AI." pith.science (2026). https://pith.science/paper/6JRSGEU7

@misc{pith2026250707544,
  author       = {Pith},
  title        = {Pith review of: Position: We Need An Algorithmic Understanding of Generative AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6JRSGEU7}},
  note         = {Machine review of arXiv:2507.07544}
}
read the original abstract

What algorithms do LLMs actually learn and use to solve problems? Studies addressing this question are sparse, as research priorities are focused on improving performance through scale, leaving a theoretical and empirical gap in understanding emergent algorithms. This position paper proposes AlgEval: a framework for systematic research into the algorithms that LLMs learn and use. AlgEval aims to uncover algorithmic primitives, reflected in latent representations, attention, and inference-time compute, and their algorithmic composition to solve task-specific problems. We highlight potential methodological paths and a case study toward this goal, focusing on emergent search algorithms. Our case study illustrates both the formation of top-down hypotheses about candidate algorithms, and bottom-up tests of these hypotheses via circuit-level analysis of attention patterns and hidden states. The rigorous, systematic evaluation of how LLMs actually solve tasks provides an alternative to resource-intensive scaling, reorienting the field toward a principled understanding of underlying computations. Such algorithmic explanations offer a pathway to human-understandable interpretability, enabling comprehension of the model's internal reasoning performance measures. This can in turn lead to more sample-efficient methods for training and improving performance, as well as novel architectures for end-to-end and multi-agent systems.

Figures

Figures reproduced from arXiv: 2507.07544 by the authors.

Figure 1
Figure 1. AlgEval: A methodological path to prioritizing algorith￾mic evaluation and understanding of LLMs. cific algorithmic hypotheses and for which ground truth responses and strategies are available. An example is goal￾directed navigation on deterministic graphs, [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Case study. A graph navigation task, algorithmic hy￾potheses (possible rollouts for DFS and BFS are shown), and potential methods for algorithmic evaluation. how rooms (nodes) connect to one another (edges) and tasks the model with determining whether a direct path from the start to the goal node exists ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Case study on algorithmic discovery. (a) Attention heatmap: from goal to the correct vs. incorrect pathway. (b) Attention is mostly allocated to the goal location M, followed by V and G on the correct path. (c) Separation of room representations across layers. subsets among them as shown in Appendix [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Attention heatmaps from the goal token to all graph nodes in the tree graph, when each final node is specified as the goal location. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]
Figure 5
Figure 5. Figure 5: T-SNE projections of activations in a graph search setting. Plots correspond to prompts (1)–(4), corresponding to a changing goal node in a tree graph of n=7 nodes (see [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]
Figure 6
Figure 6. Figure 6: Left: We reasoned that discrete steps in a BFS or DFS rollout might be identified as substantial changes in representational geometry between subsequent layers. To explore this possibility, we computed representational similarity matrices using two similarity measures:…
Figure 8
Figure 8. Figure 8: Evaluation of breadth-first search (BFS) and depth￾first search (DFS) hypotheses in Llama3.1-70B. (A) Proportion of correct steps identified in the layer by layer hidden activations. Each data point represents a single rollout of the BFS or DFS algorithm. (B) Represent…
Figure 9
Figure 9. Figure 9: Attention heatmaps for Llama-3.1-70B. Left: Average attention per layer directed from the final token to the correct vs. incorrect pathways when node W is specified as the goal location. Right: Average attention from final token to different graph node and response tok…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 7 canonical work pages

  1. [8]

    Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Car- ney, A., et al

    URL https://proceedings.mlr.press/ v235/huh24a.html. Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Car- ney, A., et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024. Jafari, F. R., Montavon, G., M ¨uller, K.-R., and Eberle, O. Mambalrp: Explaining selective state space sequence ...

    arXiv 2024

  2. [11]

    Collective Innovation in Groups of Large Language Models

    URL https://www.sciencedirect.com/ science/article/pii/S1051200417302385. Morris, C., Ritzert, M., Fey, M., Hamilton, W. L., Lenssen, J. E., Rattan, G., and Grohe, M. Weisfeiler and leman go neural: higher-order graph neural networks. In Pro- ceedings of the Thirty-Third AAAI Conference on Ar- tificial Intelligence and Thirty-First Innovative Applica- tio...

    work page Pith review arXiv 2019

  3. [20]

    Villalobos, P., Ho, A., Sevilla, J., Besiroglu, T., Heim, L., and Hobbhahn, M

    URL https://proceedings.mlr.press/ v235/vilas24a.html. Villalobos, P., Ho, A., Sevilla, J., Besiroglu, T., Heim, L., and Hobbhahn, M. Position: will we run out of data? limits of llm scaling based on human-generated data. In Proceedings of the 41st International Conference on Ma- chine Learning, ICML’24. JMLR.org, 2024. von Oswald, J., Schlegel, M., Meule...

    arXiv 2024

  4. [21]

    Webb, T., Mondal, S

    URL https://openreview.net/forum? id=NpsVSN6o4ul. Webb, T., Mondal, S. S., and Momennejad, I. Improving planning with large language models: A modular agen- tic architecture, 2024. URL https://arxiv.org/ abs/2310.00194. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V ., Zhou, D., et al. Chain-of-thought prompting elicits reasoning...

    arXiv 2024

  5. [22]

    blackboxnlp-1.22/

    URL https://aclanthology.org/2024. blackboxnlp-1.22/. Merullo, J., Eickhoff, C., and Pavlick, E. Language models implement simple Word2Vec-style vector arith- metic. In Duh, K., Gomez, H., and Bethard, S. (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies...

    work page doi:10.18653/v1/2024.na 2024

  6. [23]

    cc/paper_files/paper/2021/file/ 252a3dbaeb32e7690242ad3b556e626b-Paper

    URL https://proceedings.neurips. cc/paper_files/paper/2021/file/ 252a3dbaeb32e7690242ad3b556e626b-Paper. pdf. Wu, Q., Bansal, G., Zhang, J., Wu, Y ., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A. H., White, R. W., Burger, D., and Wang, C. Autogen: Enabling next- gen llm applications via multi-agent conversation, 2023a. URL https...

    arXiv 2021

  7. [24]

    Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y ., and Narasimhan, K

    URL https://openreview.net/forum? id=DRWCDFsb2e. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y ., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024. Ye, T., Xu, Z., Li, Y ., and Allen-Zhu, Z. Physics of Language Models: Part 2.1, Grade-Sch...

    arXiv 2024

  8. [25]

    blackboxnlp-1.25/

    URL https://aclanthology.org/2024. blackboxnlp-1.25/. Talon, D., Lippe, P., James, S., Bue, A. D., and Maglia- cane, S. Towards the reusability and compositionality of causal representations. In Locatello, F. and Didelez, V . (eds.), Proceedings of the Third Conference on Causal Learning and Reasoning, volume 236 of Proceedings of Machine Learning Researc...

    arXiv 2024

Show all 25 references
  1. [281]

    naacl-long.281/

    URL https://aclanthology.org/2024. naacl-long.281/. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. Equation of state cal- culations by fast computing machines. The Journal of Chemical Physics , 21(6):1087–1092, 1953. doi: 10.1063/1.1699114....

    2024 arXiv

  2. [385]

    acl-main.385/

    URL https://aclanthology.org/2020. acl-main.385/. Achtibat, R., Hatefi, S. M. V ., Dreyer, M., Jain, A., Wiegand, T., Lapuschkin, S., and Samek, W. AttnLRP: Attention- aware layer-wise relevance propagation for transform- ers. In Salakhutdinov, R., Kolter, Z., Heller, K., Well...

    2020

  3. [427]

    emnlp-main.427/

    URL https://aclanthology.org/2023. emnlp-main.427/. Edelman, E., Tsilivis, N., Edelman, B. L., eran malach, and Goel, S. The evolution of statistical induction heads: In-context learning markov chains. In The Thirty-eighth Annual Conference on Neural Information Processing Sys...

    2023 arXiv

  4. [435]

    naacl-long.435/

    URL https://aclanthology.org/2024. naacl-long.435/. Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017. Veliˇckovi´c, P. and Blundell, C. Neural algorithmic reasoning. Patterns, 2(7), 2021. Vig, J. and Belinkov, Y . Analyzing the str...

    2024 arXiv

  5. [446]

    emnlp-main.446/

    URL https://aclanthology.org/2021. emnlp-main.446/. Giaffar, H., Rull ´an Bux ´o, C., and Aoi, M. The effective number of shared dimensions between paired datasets. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics , volume 238 of Pr...

    2021

  6. [615]

    emnlp-main.615/

    URL https://aclanthology.org/2023. emnlp-main.615/. Zekri, O., Odonnat, A., Benechehab, A., Bleistein, L., Boull´e, N., and Redko, I. Large language models as markov chains, 2024. URL https://arxiv.org/ abs/2410.02724. Zhong, Z., Liu, Z., Tegmark, M., and Andreas, J. The clock...

    2023 arXiv

  7. [1936]

    Turpin, M., Michael, J., Perez, E., and Bowman, S

    URL http://www.cs.helsinki.fi/u/ gionis/cc05/OnComputableNumbers.pdf. Turpin, M., Michael, J., Perez, E., and Bowman, S. Lan- guage models don’t always say what they think: unfaith- ful explanations in chain-of-thought prompting.Advances in Neural Information Processing System...

    2024

  8. [1985]

    Chefer, H., Gur, S., and Wolf, L

    doi: 10.1007/BF00940812. Chefer, H., Gur, S., and Wolf, L. Transformer interpretabil- ity beyond attention visualization. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 782–791, 2021. doi: 10.1109/CVPR46437. 2021.00084. Choi, D., Huang, V .,...

    2021

  9. [2017]

    cc/paper_files/paper/2017/file/ 8a20a8621978632d76c43dfd28b67767-Paper

    URL https://proceedings.neurips. cc/paper_files/paper/2017/file/ 8a20a8621978632d76c43dfd28b67767-Paper. pdf. Malach, E. Auto-regressive next-token predictors are uni- versal learners. arXiv preprint arXiv:2309.06979, 2023. Manvi, R., Singh, A., and Ermon, S. Adaptive inferenc...

    2017 arXiv

  10. [2020]

    Momennejad, I

    URL https://www.sciencedirect.com/ science/article/pii/S2352154620300371. Momennejad, I. Memory and planning in brains and ma- chines: Multiscale predictive representations. In Nadel, L. and Aronovitz, S. (eds.), Space, Time, and Memory. Oxford University Press, forthcoming. M...

    2017 arXiv

  11. [2021]

    Wiedemer, T., Mayilvahanan, P., Bethge, M., and Brendel, W

    URL https://proceedings.mlr.press/ v139/weiss21a.html. Wiedemer, T., Mayilvahanan, P., Bethge, M., and Brendel, W. Compositional generalization from first principles. In Thirty-seventh Conference on Neural Information Pro- cessing Systems, 2023. URL https://openreview. net/for...

    2023

  12. [2022]

    Power, A., Burda, Y ., Edwards, H., Babuschkin, I., and Misra, V

    https://transformer-circuits.pub/2022/in-context- learning-and-induction-heads/index.html. Power, A., Burda, Y ., Edwards, H., Babuschkin, I., and Misra, V . Grokking: Generalization beyond overfit- 15 Algorithmic Understanding of LLMs ting on small algorithmic datasets. arXiv...

    2022 arXiv

  13. [2023]

    Galke, L., Ram, Y ., and Raviv, L

    URL https://openreview.net/forum? id=IEMLNF4gK4. Galke, L., Ram, Y ., and Raviv, L. Deep neural networks and humans both benefit from compositional language struc- ture. Nature Communications, 15:10816, 2024. ISSN 2041-1723. doi: 10.1038/s41467-024-55158-1. Gandhi, K., Lee, D....

    2024 arXiv

  14. [2024]

    Ali, A., Schnake, T., Eberle, O., Montavon, G., M ¨uller, K.-R., and Wolf, L

    URL https://proceedings.mlr.press/ v235/achtibat24a.html. Ali, A., Schnake, T., Eberle, O., Montavon, G., M ¨uller, K.-R., and Wolf, L. XAI for transformers: Better explana- tions through conservative propagation. In International Conference on Machine Learning, ICML 2022, 17-...

    2022 arXiv

  15. [3529]

    Kriegeskorte, N., Mur, M., and Bandettini, P

    PMLR, 2019. Kriegeskorte, N., Mur, M., and Bandettini, P. A. Repre- sentational similarity analysis-connecting the branches of systems neuroscience. Frontiers in systems neuroscience, 2:249, 2008. Lehnert, L., Sukhbaatar, S., Su, D., Zheng, Q., McVay, P., Rabbat, M., and Tian,...

    2019 arXiv

  16. [4257]

    URL https://proceedings

    PMLR, 2024. URL https://proceedings. mlr.press/v238/giaffar24a.html. Gurnee, W. and Tegmark, M. Language models represent space and time, 2024. URL https://arxiv.org/ abs/2310.02207. Hanna, M., Pezzelle, S., and Belinkov, Y . Have faith in faithfulness: Going beyond circuit ov...

    2024 arXiv

  17. [9179]

    doi: 10.18653/v1/2022

    ACL, December 2022. doi: 10.18653/v1/2022. emnlp-main.624. Radev, S. T., V oss, A., Wieschen, E. M., and B¨urkner, P.- C. Amortized bayesian inference for models of cogni- tion, 2020. URL https://arxiv.org/abs/2005. 03899. Ren, J., Guo, Q., Yan, H., Liu, D., Zhang, Q., Qiu, X....

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.