Recognition: unknown
Use and usability: concepts of representation in philosophy, neuroscience, cognitive science, and computer science
Pith reviewed 2026-05-10 12:31 UTC · model grok-4.3
The pith
Representations are understood at three levels based on whether their information is useful, in usable format, and actually used.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Building on four aspects of information and use, existing perspectives on neural representations are organized into three levels: Representations as Information (Level 1); Representations as Usable (Level 2); and Representations as Used (Level 3). The account helps appreciate the diversity of notions of neural representation and clarify the appropriate notion for specific investigations.
What carries the argument
The taxonomy of three levels of representation supported by the four aspects of information, usefulness, usable format, and downstream use.
If this is right
- Studies of neural codes can be classified by whether they address mere information presence or actual usability.
- Models in AI can be evaluated on which level of representation they implement.
- Philosophical arguments about representation can be mapped to these levels to resolve apparent disagreements.
- Investigators can choose the level appropriate to their research question to avoid mismatch.
Where Pith is reading between the lines
- This organization could reveal gaps in current research where certain levels are understudied.
- It might suggest new experiments that test the transition from usable to used representations.
- Connecting to neighboring problems like the role of representations in learning or decision making could extend the framework.
Load-bearing premise
The four aspects comprehensively capture the relevant dimensions of representation across the disciplines without significant omissions.
What would settle it
Discovery of a conception of representation in one of the fields that fits none of the three levels or requires a fifth aspect of use.
read the original abstract
Representations play a central role in the study of both biological and artificial intelligence, as well as philosophy of mind. Across neuroscience, computer science, and philosophy, a recurring theme is that representations not only carry information but should be ``useful'' for or ``usable'' by an agent in some sense. Here, we review how the ``usefulness'' of representations has been conceptualized and how it figures into different conceptions of representation. We identify and explore four aspects of use and usability: representations generally carry \textit{information}; that information may or may not be \textit{useful} and it may or may not be encoded in a usable \textit{format}; and the representations may or may not be \textit{used downstream}. Building on these four aspects of information and use, we then organize existing perspectives on neural representations into three levels: Representations as Information (Level 1); Representations as Usable (Level 2); and Representations as Used (Level 3). Our account is meant to give readers an appreciation for the diversity of notions of ``neural representation,'' help them navigate the vast and multi-disciplinary literature on the topic, and help them clarify the appropriate notion of representation for their own investigations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript reviews how usefulness and usability figure into conceptions of representation across philosophy, neuroscience, cognitive science, and computer science. It identifies four aspects—representations carry information; that information may or may not be useful; it may or may not be encoded in a usable format; and representations may or may not be used downstream—and employs these to organize existing perspectives on neural representations into three levels: Representations as Information (Level 1), Representations as Usable (Level 2), and Representations as Used (Level 3). The account is intended to help readers appreciate the diversity of notions and navigate the multidisciplinary literature.
Significance. If the taxonomy holds, the paper supplies a coherent organizational framework for a sprawling interdisciplinary topic without introducing new parameters, axioms, or empirical claims. Its value lies in the explicit linkage of the four aspects to the three levels and the cross-field synthesis, which can assist researchers in selecting defensible notions of representation for their own work. The absence of self-referential derivations or fitted quantities is a strength for a review of this type.
minor comments (2)
- A summary table mapping the four aspects to the three levels, with one or two canonical citations per cell, would improve readability and make the organizational claim easier to apply.
- The manuscript should explicitly note any disciplinary literatures (e.g., embodied cognition or social epistemology) that fall outside the four-aspect framing, even if only to delimit scope.
Simulated Author's Rebuttal
We thank the referee for their positive assessment of the manuscript, including the recognition of its value as a cross-disciplinary organizational framework for notions of neural representation. We appreciate the recommendation for minor revision and note that no specific major comments were raised in the report.
Circularity Check
No significant circularity in this conceptual review and taxonomy
full rationale
This paper is a review and synthesis that identifies four aspects of representation (information carried, usefulness, usable format, downstream use) drawn from existing multi-disciplinary literature and uses them to organize prior perspectives into three levels. The central contribution is the coherence and utility of this framing rather than any derivation, model, or empirical claim. No equations, fitted parameters, or self-citation chains appear; the argument rests on external citations and does not reduce any claim to its own inputs by construction. This is the expected non-finding for a self-contained organizational review.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Representations carry information that can be evaluated for usefulness and usability.
Reference graph
Works this paper leans on
-
[1]
Fallon FT, Ryan TJ, Krakauer JW, What Are We T alking about? Clarifying the Fuzzy Concept of Representation in Neuroscience and Beyond; 2023
2023
-
[2]
Three Aspects of Representation in Neuroscience
Baker B, Lansdell B, Kording KP . Three Aspects of Representation in Neuroscience. T rends in Cognitive Sciences 2022 Nov;26(11):942–958
2022
-
[3]
Investigating the Concept of Representation in the Neural and Psychological Sciences
Favela LH, Machery E. Investigating the Concept of Representation in the Neural and Psychological Sciences. Frontiers in Psychology 2023 Jun;14:1–13
2023
-
[4]
Commentary: Investigating the Concept of Representation in the Neural and Psychological Sciences
Richmond A. Commentary: Investigating the Concept of Representation in the Neural and Psychological Sciences. Frontiers in Psychology 2023;14
2023
-
[5]
Putting Representations to Use
Cao R. Putting Representations to Use. Synthese 2022 Apr;200(2):151
2022
-
[6]
Deflating Mental Representation
Egan F . Deflating Mental Representation. The Jean Nicod Lectures, Cambridge, Massachusetts: The MIT Press; 2025
2025
-
[7]
The Extended Mind
Clark A, Chalmers D. The Extended Mind. Analysis 1998;58(1):7–19
1998
-
[8]
The Embodied Mind: Cognitive Science and Human Experience
Varela FJ, Thompson E, Rosch E. The Embodied Mind: Cognitive Science and Human Experience. 14. print. ed. Cam- bridge, Mass.: MIT Press; 1993
1993
-
[9]
Untangling Invariant Object Recognition
DiCarlo JJ, Cox DD. Untangling Invariant Object Recognition. T rends in Cognitive Sciences 2007;11(8):333–341
2007
-
[10]
Interpreting Encoding and Decoding Models
Kriegeskorte N, Douglas PK. Interpreting Encoding and Decoding Models. Current Opinion in Neurobiology 2019 Apr;55:167–179
2019
-
[11]
arXiv; 2022
Ivanova AA, Schrimpf M, Anzellotti S, Zaslavsky N, Fedorenko E, Isik L, Beyond Linear Regression: Mapping Models in Cognitive Neuroscience Should Align with Research Goals. arXiv; 2022
2022
-
[12]
Representation Learning: A Review and New Perspectives
Bengio Y, Courville A, Vincent P . Representation Learning: A Review and New Perspectives. IEEE T ransactions on Pattern Analysis and Machine Intelligence 2013;35(8):1798–1828
2013
-
[13]
A Framework for the Quantitative Evaluation of Disentangled Representations
Eastwood C, Williams CKI. A Framework for the Quantitative Evaluation of Disentangled Representations. ICLR 2018
2018
-
[14]
T owards a Definition of Disentangled Represen- tations
Higgins I, Amos D, Pfau D, Racaniere S, Matthey L, Rezende D, et al. T owards a Definition of Disentangled Represen- tations. arXiv 2018;p. 1–29
2018
-
[15]
Deep Learning and the Information Bottleneck Principle
Tishby N, Zaslavsky N. Deep Learning and the Information Bottleneck Principle. arXiv 2015
2015
-
[16]
Emergence of Invariance and Disentanglement in Deep Representations
Achille A, Soatto S. Emergence of Invariance and Disentanglement in Deep Representations. Information Theory and Applications Workshop 2018;19:1–34
2018
-
[17]
Clarifying the Conceptual Dimensions of Representation in Neuroscience
Pohl S, Walker EY, Barack DL, Lee J, Denison RN, Block N, et al. Clarifying the Conceptual Dimensions of Representation in Neuroscience. Nature Reviews Neuroscience 2026 Mar;p. 1–16
2026
-
[18]
Grounded Cognition: Past, Present, and Future
Barsalou LW. Grounded Cognition: Past, Present, and Future. T opics in Cognitive Science 2010 Oct;2(4):716–724
2010
-
[19]
The Modal—Amodal Distinction in the Debate on Conceptual Format
Haimovici S. The Modal—Amodal Distinction in the Debate on Conceptual Format. Philosophies 2018 Jun;3(2):7–19
2018
-
[20]
Grounding Action Representations
Weber AM, Vosgerau G. Grounding Action Representations. Review of Philosophy and Psychology 2012 Mar;3(1):53– 69
2012
-
[21]
Representation Reconsidered
Ramsey W. Representation Reconsidered. Cambridge: Cambridge University Press; 2007
2007
-
[22]
arXiv; 2018
Alain G, Bengio Y, Understanding Intermediate Layers Using Linear Classifier Probes. arXiv; 2018. 26 Baker, Lange, Richmond et al
2018
-
[23]
Fast Readout of Object Identity from Macaque Inferior T emporal Cortex
Hung CP , Kreiman G, Poggio T, DiCarlo JJ. Fast Readout of Object Identity from Macaque Inferior T emporal Cortex. Science 2005 Nov;310(5749):863–866
2005
-
[24]
Decoding the Brain: Neural Representation and the Limits of Multivariate Pattern Analysis in Cognitive Neuroscience
Ritchie JB, Kaplan DM, Klein C. Decoding the Brain: Neural Representation and the Limits of Multivariate Pattern Analysis in Cognitive Neuroscience. The British Journal for the Philosophy of Science 2019 Jun;70(2):581–607
2019
-
[25]
Representational Models: A Common Framework for Understanding Encoding, Pattern- Component, and Representational-Similarity Analysis, vol
Diedrichsen J, Kriegeskorte N. Representational Models: A Common Framework for Understanding Encoding, Pattern- Component, and Representational-Similarity Analysis, vol. 13; 2017
2017
-
[26]
What Representational Similarity Measures Imply about Decodable Information
Harvey SE, Lipshutz D, Williams AH. What Representational Similarity Measures Imply about Decodable Information. In: UniReps: 2nd Edition of the Workshop on Unifying Representations in Neural Models; 2024
2024
-
[27]
Peeling the Onion of Brain Representations
Kriegeskorte N, Diedrichsen J. Peeling the Onion of Brain Representations. Annual Review of Neuroscience 2019;42:407–432
2019
-
[28]
Decoding the Cognitive Map: Ensemble Hippocampal Sequences and Decision Making
Wikenheiser AM, Redish AD. Decoding the Cognitive Map: Ensemble Hippocampal Sequences and Decision Making. Current Opinion in Neurobiology 2015 Jun;32:8–15
2015
-
[29]
Neuroscience Needs Behavior: Correcting a Reductionist Bias
Krakauer JW, Ghazanfar AA, Gomez-Marin A, Maciver MA, Poeppel D. Neuroscience Needs Behavior: Correcting a Reductionist Bias. Neuron Perspective 2017;93(3):480–490
2017
-
[30]
Contents, Vehicles, and Complex Data Analysis in Neuroscience
Burnston DC. Contents, Vehicles, and Complex Data Analysis in Neuroscience. Synthese An International Journal for Epistemology, Methodology and Philosophy of Science 2020;199:1617–1639
2020
-
[31]
Single-T rial Neural Dynamics Are Dominated by Richly Varied Movements
Musall S, Kaufman MT, Juavinett AL, Gluf S, Churchland AK. Single-T rial Neural Dynamics Are Dominated by Richly Varied Movements. Nature Neuroscience 2019 Oct;22(10):1677–1686
2019
-
[32]
Context-Dependent Computation by Recurrent Dynamics in Prefrontal Cortex
Mante V, Sussillo D, Shenoy KV, Newsome WT. Context-Dependent Computation by Recurrent Dynamics in Prefrontal Cortex. Nature 2013;503(7474):78–84
2013
-
[33]
What Is a Theory of Neural Representation For? Synthese An International Journal for Epistemology, Methodology and Philosophy of Science 2025;205(14)
Richmond A. What Is a Theory of Neural Representation For? Synthese An International Journal for Epistemology, Methodology and Philosophy of Science 2025;205(14)
2025
-
[34]
Multitask Learning
Caruana R. Multitask Learning. Machine Learning 1997 Jul;28(1):41–75
1997
-
[35]
Exploiting T ask Relatedness for Multiple T ask Learning
Ben-David S, Schuller R. Exploiting T ask Relatedness for Multiple T ask Learning. In: Goos G, Hartmanis J, Van Leeuwen J, Schölkopf B, Warmuth MK, editors. Learning Theory and Kernel Machines, vol. 2777 Berlin, Heidelberg: Springer Berlin Heidelberg; 2003.p. 567–580
2003
-
[36]
Examples of Gibsonian Affordances in Legged Robotics Research Using an Empirical, Generative Framework
Roberts SF, Koditschek DE, Miracchi LJ. Examples of Gibsonian Affordances in Legged Robotics Research Using an Empirical, Generative Framework. Frontiers in Neurorobotics 2020;14(February)
2020
-
[37]
The Affordance Competition Hypothesis: A Framework for Embodied Behavior
Cisek P . The Affordance Competition Hypothesis: A Framework for Embodied Behavior. In: Klatzky RL, MacWhinney B, Behrmann M, editors. Embodiment, Ego-Space, and Action, 0 ed. Psychology Press; 2008
2008
-
[38]
arXiv; 2019
Frankle J, Carbin M, The Lottery Ticket Hypothesis: Finding Sparse, T rainable Neural Networks. arXiv; 2019
2019
-
[39]
T emporal Structure of Motor Variability Is Dynamically Regulated and Predicts Motor Learning Ability
Wu HG, Miyamoto YR, Castro LNG, Ölveczky BP , Smith MA. T emporal Structure of Motor Variability Is Dynamically Regulated and Predicts Motor Learning Ability. Nature Neuroscience 2014;17(2):312–321
2014
-
[40]
Neural Spiking for Causal Inference and Learning
Lansdell BJ, Kording KP . Neural Spiking for Causal Inference and Learning. PLOS Computational Biology 2023 Apr;19(4):e1011005
2023
-
[41]
The Information Bottleneck Method
Tishby N, Pereira FC, Bialek W. The Information Bottleneck Method. Proceedings of the 37th Allerton Conference on Communication, Control, and Computing 1999;p. 368–377
1999
-
[42]
Visual Representations: Defining Properties and Deep Approximations
Soatto S, Chiuso A. Visual Representations: Defining Properties and Deep Approximations. ICLR 2016;p. 1–20. Baker, Lange, Richmond et al. 27
2016
-
[43]
Natural Image Statistics and Neural Representation
Simoncelli EP , Olshausen BA. Natural Image Statistics and Neural Representation. Annual Review of Neuroscience 2001 Mar;24(Volume 24, 2001):1193–1216
2001
-
[44]
Efficient Representation as a Design Principle for Neural Coding and Computation
Bialek W, de Ruyter van Stevenick RR, Tishby N. Efficient Representation as a Design Principle for Neural Coding and Computation. IEEE International Symposium on Information Theory 2007;p. 659–663
2007
-
[45]
Sparse Coding with an Incomplete Basis Set: A Strategy Employed by V1? Vision Research 1997;37(23):3311–3325
a Olshausen B, Field DJ. Sparse Coding with an Incomplete Basis Set: A Strategy Employed by V1? Vision Research 1997;37(23):3311–3325
1997
-
[46]
Possible Principles Underlying the T ransformations of Sensory Messages
Barlow HB. Possible Principles Underlying the T ransformations of Sensory Messages. Sensory Communication 1961;p. 216–234
1961
-
[47]
Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems
Dayan P , Abbott LF . Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Se- jnowski TJ, Poggio T, editors, London: MIT Press; 2001
2001
-
[48]
T oward a Unified Theory of Efficient, Predictive, and Sparse Coding
Chalk M, Marre O, Tkačik G. T oward a Unified Theory of Efficient, Predictive, and Sparse Coding. Proceedings of the National Academy of Sciences 2018;115(1):186–191
2018
-
[49]
The Foundations of Statistics
Savage LJ. The Foundations of Statistics. Courier Corporation; 1972
1972
-
[50]
Decision Theory: What "Should" the Nervous System Do? Science 2008;606(2007)
Körding KP . Decision Theory: What "Should" the Nervous System Do? Science 2008;606(2007)
2008
-
[51]
Probabilistic Interpretation of Population Codes
Zemel RS, Dayan P , Pouget A. Probabilistic Interpretation of Population Codes. Neural Computation 1998;10(2):403– 430
1998
-
[52]
Statistically Optimal Perception and Learning: From Behavior to Neural Repre- sentations
Fiser J, Berkes P , Orbán G, Lengyel M. Statistically Optimal Perception and Learning: From Behavior to Neural Repre- sentations. T rends in cognitive sciences 2010 Mar;14(3):119–30
2010
-
[53]
Probabilistic Brains: Knowns and Unknowns
Pouget A, Beck JM, Ma WJ, Latham PE. Probabilistic Brains: Knowns and Unknowns. Nature Neuroscience 2013;16(9):1170–1178
2013
-
[54]
Psychosemantics: The Problem of Meaning in the Philosophy of Mind
Fodor JA. Psychosemantics: The Problem of Meaning in the Philosophy of Mind. The MIT Press; 1987
1987
-
[55]
Knowledge & the Flow of Information
Dretske FI. Knowledge & the Flow of Information. 4. print ed. A Bradford Book, Cambridge, Mass: MIT Pr; 1986
1986
-
[56]
Language, Thought, and Other Biological Categories: New Foundations for Realism
Millikan RG. Language, Thought, and Other Biological Categories: New Foundations for Realism. MIT Press; 1987
1987
-
[57]
Knowledge in Perspective: Selected Essays in Epistemology
Sosa E. Knowledge in Perspective: Selected Essays in Epistemology. Cambridge University Press; 1991
1991
-
[58]
Virtues of the Mind: An Inquiry into the Nature of Virtue and the Ethical Foundations of Knowledge
Zagzebski LT. Virtues of the Mind: An Inquiry into the Nature of Virtue and the Ethical Foundations of Knowledge. Cambridge University Press; 1996
1996
-
[59]
towards Reconstruction in Epistemology
Susan H, William D, Evidence and Inquiry. towards Reconstruction in Epistemology. Oxford: Blackwell Publishers; 1993
1993
-
[60]
How Does the Brain Solve Visual Object Recognition? Neuron 2012;73(3):415–434
DiCarlo JJ, Zoccolan D, Rust NC. How Does the Brain Solve Visual Object Recognition? Neuron 2012;73(3):415–434
2012
-
[61]
Explicit Information for Category-Orthogonal Object Properties Increases along the Ventral Stream
Hong H, Y amins DLK, Majaj NJ, DiCarlo JJ. Explicit Information for Category-Orthogonal Object Properties Increases along the Ventral Stream. Nature Neuroscience 2016;19(4)
2016
-
[62]
Memory Capacity of Linear vs
Poirazi P , Mel B. Memory Capacity of Linear vs. Nonlinear Models of Dendritic Integration. In: Advances in Neural Information Processing Systems, vol. 12 MIT Press; 1999
1999
-
[63]
Normalization as a Canonical Neural Computation
Carandini M, Heeger DJ. Normalization as a Canonical Neural Computation. Nature Reviews Neuroscience 2012;13:51–62
2012
-
[64]
Conjectures Regarding the Nonlinear Geometry of Visual Neurons
Golden JR, Vilankar KP , Wu MCK, Field DJ. Conjectures Regarding the Nonlinear Geometry of Visual Neurons. Vision Research 2016 Mar;120:74–92. 28 Baker, Lange, Richmond et al
2016
-
[65]
Are Dendrites Conceptually Useful? Neuroscience 2022
Larkum ME. Are Dendrites Conceptually Useful? Neuroscience 2022
2022
-
[66]
Soft Matching Distance: A Metric on Neural Representations That Captures Single-Neuron T uning
Khosla M, Williams AH. Soft Matching Distance: A Metric on Neural Representations That Captures Single-Neuron T uning. In: Proceedings of UniReps: The First Workshop on Unifying Representations in Neural Models PMLR; 2024. p. 326–341
2024
-
[67]
Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies
Achille A, Eccles T, Matthey L, Burgess CP , Watters N, Lerchner A, et al. Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies. Advances in Neural Information Processing Systems 2018;2018- Decem(Mdl):9873–9883
2018
-
[68]
A Sober Look at the Unsupervised Learning of Disen- tangled Representations and Their Evaluation
Locatello F, Bauer S, Lucic M, Rätsch G, Gelly S, Schölkopf B, et al. A Sober Look at the Unsupervised Learning of Disen- tangled Representations and Their Evaluation. The Journal of Machine Learning Research 2020 Jan;21(1):209:8629– 209:8690
2020
-
[69]
Are Disentangled Representations Helpful for Abstract Visual Reasoning? Advances in Neural Information Processing Systems 2019;32(NeurIPS)
van Steenkiste S, Locatello F, Schmidhuber J, Bachem O. Are Disentangled Representations Helpful for Abstract Visual Reasoning? Advances in Neural Information Processing Systems 2019;32(NeurIPS)
2019
-
[70]
The Role of Disentanglement in Generalization
Montero ML, Ludwig CJ, Costa RP , Malhotra G, Bowers J. The Role of Disentanglement in Generalization. ICLR 2021
2021
-
[71]
arXiv; 2024
Lippl S, Stachenfeld K, When Does Compositional Structure Yield Compositional Generalization? A Kernel Theory. arXiv; 2024
2024
-
[72]
arXiv; 2023
Hyvarinen A, Khemakhem I, Morioka H, Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning. arXiv; 2023
2023
-
[73]
T oward Causal Representation Learning
Scholkopf B, Locatello F, Bauer S, Ke NR, Kalchbrenner N, Goyal A, et al. T oward Causal Representation Learning. Proceedings of the IEEE 2021;109(5):612–634
2021
-
[74]
Learning Optimal Representations with the Decodable Information Bottle- neck
Dubois Y, Kiela D, Vedantam R, Schwab DJ. Learning Optimal Representations with the Decodable Information Bottle- neck. arXiv 2020
2020
-
[75]
Harvard University Press; 1975
Fodor J, The Language of Thought. Harvard University Press; 1975
1975
-
[76]
The Language of Thought Hypothesis
Aydede M. The Language of Thought Hypothesis. In: Zalta E, editor. Stanford Encyclopedia of Philosophy; 2010
2010
-
[77]
The Best Game in T own: The Re-emergence of the Language of Thought Hypothesis across the Cognitive Sciences
Quilty-Dunn J, Porot N, Mandelbaum E. The Best Game in T own: The Re-emergence of the Language of Thought Hypothesis across the Cognitive Sciences. Behavioral and Brain Sciences 2022 Dec;p. 1–55
2022
-
[78]
The Medium and the Message in Mental Imagery: A Theory
Kosslyn SM. The Medium and the Message in Mental Imagery: A Theory. Psychological Review 1981;88(1):46–66
1981
-
[79]
Mental Rotation of Three-Dimensional Objects
Shepard RN, Metzler J. Mental Rotation of Three-Dimensional Objects. Science 1971;171(972):701–703
1971
-
[80]
Iconic Representation: Maps, Pictures, and Perception
Burge T. Iconic Representation: Maps, Pictures, and Perception. In: Wuppuluri S, Doria FA, editors. The Map and the T erritory Cham: Springer International Publishing; 2018.p. 79–100
2018
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.