Pith. sign in

REVIEW 3 major objections 6 minor 74 references

Geospatial Mechanistic Interpretability of Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Spatial autocorrelation exposes geospatial structure inside LLM place-name representations.

desk verdict A fresh framework for spatial interpretability of LLMs, but the main empirical claim is confounded by the always-present admin region token in the prompts. read the letter →

arxiv 2505.03368 v2 pith:ZHXGLBZO submitted 2025-05-06 cs.LG

Stef De Sabbata , Stefano Mizzaro , Kevin Roitero This is my paper · ORCID
classification cs.LG
keywords mechanisticinterpretabilitygeospatialanalysisspatialautocorrelationsparseautoencoderslargelanguagemodelsinternalrepresentationsAIprobing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the internal representations a large language model forms for place names are geographically structured, and that this structure can be reverse-engineered with spatial statistics. Using spatial autocorrelation on post-attention activations, it argues that nearby places evoke similar internal patterns, while distant or different kinds of places diverge. It further claims that most geographic encoding is polysemantic, meaning spread across many neurons that each encode several concepts, and that sparse autoencoders can disentangle a small number of monosemantic, spatially interpretable features such as 'southern Italy' and 'city'. If right, this gives geography a window into the model's hidden geography: a way to inspect, rather than merely query, what LLMs know about place.

What carries the argument

The central mechanism is spatial autocorrelation, measured with global and local Moran's I, applied to vectors that summarize the model's state: 4,096 post-attention, mean-pooled activations per layer for each placename, and then 32,768 sparse features obtained from a sparse autoencoder with a TopK/ReLU activation that retains the 2,048 largest embeddings per input. Autocorrelation tests whether high values for one placename tend to be surrounded by high values for nearby placenames; local Moran's I then maps significant high-high and low-low clusters. The sparse autoencoder is the disentangling instrument: it projects the 4,096-dimensional activation space into a wider, sparsely activated feature space in which individual directions can become interpretable, and the same autocorrelation test is applied to those feature directions to see whether they are geospatially coherent. The whole argument thus rides on Moran's I as a way to turn a causal-sounding question, 'does the model encode geography?', into a measurable spatial-statistical one.

What would settle it

Take the same placenames and run the identical spatial-autocorrelation analysis with prompts that omit the country, province, or state suffix, or with those administrative-region tokens removed from the activation vector; if the clustered spatial patterns (Moran's I >= 0.3) disappear or drop sharply, the spatial structure is an artifact of the shared region token rather than evidence of encoded geographic knowledge.

Watch

Extended reading notes

Core claim

The paper claims that internal activations evoked by placenames are not geographically inert: neurons and sparse-autoencoder features carrying high values for nearby places cluster in space, consistent with Tobler's first law operating inside the model. Across layers 7, 15, and 31 of a 7-billion-parameter instruction-tuned model, about 15% of neurons show significant spatial autocorrelation (p < .01, Moran's I >= 0.3), with individual neurons activating for multiple regions at once, such as New York State and Northern Ireland, which supports a polysemantic, superposition-style encoding. A sparse autoencoder trained on layer-15 activations decomposes these into 32,768 sparse features, of which 0.2% show significant spatial autocorrelation; some of these are clearly monosemantic, such as a feature for southern Italy that does not fire for northern Italy, the UK, or the US, and a 'city' feature firing for London, Manchester, Milan, New York, and Philadelphia. The small count of spatially coherent features is read as evidence that geographic information is distributed sparsely and diffusely across the representation, entangled with non-geographic concepts.

Load-bearing premise

The analysis assumes the spatial autocorrelation in activations reflects the model's geographic knowledge rather than the administrative region name (for example 'England', a province, or a US state) that is appended to every placename in the prompt; because every placename from the same region shares that token, a neuron could cluster spatially simply by tracking that token, and the paper does not test this alternative.

Editorial extensions

If this is right

  • Placenames that are geographic neighbors tend to have similar internal representations, so geographic structure is present in LLM hidden states, not only in outputs.
  • Geographic knowledge is mostly distributed across polysemantic neurons; single-neuron readings will mislead, and superposition is a real obstacle for geographic interpretability.
  • Sparse autoencoders can recover a small number of monosemantic geographic features such as 'southern Italy' and 'city', suggesting that disentangled geographic directions exist in representation space.
  • Because only 0.2% of sparse features show spatial autocorrelation, geographic content is encoded sparsely and diffusely; future work must improve feature extraction to surface more of it.
  • The framework opens a research path on geospatial mechanistic interpretability for foundation models beyond the specific model and three test regions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural control experiment follows from the paper's own setup: hold the administrative suffix constant but shuffle placenames across coordinates, or drop the suffix entirely; if autocorrelation disappears, the spatial signal lives in the token, not in the place.
  • The same spatial-autocorrelation test could be applied to features from larger models, multilingual models, or different sparse-autoencoder variants to map how geographic scale and cultural context shift the features that fire.
  • The 'city' feature hints at a path toward geospatial feature atlases, dictionaries that connect model-internal directions to regions, urbanity, and other geographic concepts, which could make model geographies auditable for bias and diversity.
  • The paper's observation of a feature shared by northeastern Italy and northwestern England suggests feature-level analysis could reveal non-cartographic, relational geographies that coordinate probes miss.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This manuscript proposes a framework for geospatial mechanistic interpretability of large language models. The authors extract internal activations from Mistral-7B-Instruct-v0.2 for placenames in the UK, Italy, and four US states, using prompts that append a country/province/state qualifier. They compute global and local Moran's I on per-neuron activations and report that 1,841 of 12,288 neurons (14.98%) show significant spatial autocorrelation. They then train a sparse autoencoder on layer-15 activations to decompose the representations into 32,768 features, of which 67 are reported to show significant spatial autocorrelation, and present examples of features that appear to correspond to regions such as Wales, southern Italy, and 'city'. The central claim is that these results show LLMs encode geographic information in a spatially structured, largely polysemantic way, and that sparse autoencoders can disentangle this structure into more monosemantic features.

Significance. If the claims were supported, the paper would offer a novel methodological bridge between geographic information science and mechanistic interpretability, providing a concrete toolset (Moran's I on internal activations, SAE decomposition) applicable to questions about bias, diversity, and knowledge representation in LLMs. The paper is clearly written, situates the work well in both literatures, and makes code and data available. However, the significance is conditional: the central geospatial interpretation is confounded by the administrative tokens always present in the prompts, and the SAE statistical summary is misleading. With appropriate controls and corrected statistics, the contribution could be solid and of interest to the geoAI and interpretability communities.

major comments (3)
  1. [Section 2.3.1 (prompt design); Section 2.3.2 (results)] The prompt design confounds geographic location with administrative-token identity. Every prompt is of the form '[placename], [country/province/state]', and the reported activations are mean-pooled across all tokens in the prompt (Section 2.3.1). Consequently, a neuron that simply responds to the token 'England' will produce near-identical activations for all English placenames, yielding spatial autocorrelation that mirrors administrative boundaries by construction. The manuscript acknowledges this risk explicitly (Section 2.3.1) but never controls for it; Figure 2(a) even displays a neuron whose high values align with the admin-area tokens in the prompts. To support the claim that the spatial patterns are 'related to their geographic location' (abstract), the authors should run a control condition, e.g., prompts with the qualifier omitted, prompts with mismatched qualifiers (e.g., 'Liverpool, California'), or a neuron-level baseline obtained by scrambling the qualifier across places. Without such a control, the 1,841-neuron statistic cannot distinguish encoding of geographic location from encoding of token identity.
  2. [Section 2.3.2 and Section 3.2.2 (statistical summarization)] The statistical interpretation is problematic in two ways. First, the 1,841 significant neurons out of 12,288 are assessed at p < .01 without any multiple-comparison correction or permutation test; at that threshold roughly 123 false positives are expected, and the paper does not report how many neurons survive a corrected threshold, so the strength of the rejection of the a-spatial null is not quantified. Second, the SAE result is mis-stated: the paper reports 'only 67 of the 32,768 features (0.2%) displayed a significant spatial autocorrelation', but 99.53% of features are always zero (Section 3.2.2), so only about 154 features are active. The meaningful rate is therefore 67/154 ≈ 43%, far above the ~1.5 false positives expected among active features at p < .01. The paper's conclusion that geospatial information is 'sparse and diffuse' is the opposite of what this calculation suggests; the correct denominator should be active features, not all 32,768.
  3. [Section 3.2.1 (SAE training and selection)] The sparse autoencoder is trained on the full set of pooled activations, and k = 2,048 is selected based on the final training loss on that same dataset. The subsequent Moran's I analysis is then performed on the very representations used for training, with no held-out split, seed averaging, or stability analysis. This does not make the spatial analysis circular (the SAE objective is reconstruction, not spatial autocorrelation), but it does mean the reported counts of spatially autocorrelated features are not protected against overfitting to the training representations, and the quantitative claims may not generalize. The paper should either validate the SAE on a held-out set or report that the results are stable across random initializations.
minor comments (6)
  1. [Section 2.3.1 (methodology)] The spatial weights matrix used for the global and local Moran's I (e.g., k-nearest neighbors or distance band, row standardization) is not specified, which makes the reported p-values and Moran's I values impossible to verify from the text alone.
  2. [Figures 2 and 3] The maps in Figures 2 and 3 would be easier to interpret with color bars, explicit layer/neuron or feature labels, and the corresponding Moran's I values for the displayed examples.
  3. [Section 3.2.1 (training details)] Training hyperparameters for the sparse autoencoder (learning rate, optimizer, batch size, normalization) are not reported beyond the 300 epochs and k values; these details should be given or referenced to the code repository.
  4. [Section 1] The phrase 'an a-spatial manner' is awkward; consider 'aspatial' or 'non-spatial' for clarity.
  5. [Section 4] Reference [65] is cited for 'a new research question'; the intended noun is likely 'questions'.
  6. [Section 3.2.2] The interpretation of the feature in Figure 3(c) as relating to ports and industrial development is speculative and not backed by a validation experiment; it should be flagged as such in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: SAE is trained for reconstruction only, Moran's I is applied post hoc, and no parameter is fitted to the spatial outcome; the admin-token confound is a validity issue, not a circularity issue.

full rationale

The derivation chain is not circular. The sparse autoencoder is trained on layer-15 condensed activations for reconstruction (Section 3.2.1) with no spatial objective, and the spatial-autocorrelation statistics are computed post hoc on raw activations and on the resulting features (Sections 2.3.2, 3.2.2). No parameter is fitted to Moran's I, to latitude/longitude, or to any spatial outcome, and no reported 'prediction' is constructed from the quantity being predicted. The only self-citation (Ref. [54], sharing one author) supports expectations about placename coverage and is not load-bearing for the geospatial-interpretability claim. The repeated administrative qualifier in the prompts ('[placename], [country/province/state]', Section 2.3.1) is a genuine experimental confound, because a neuron that simply tracked the repeated admin token would show spatially clustered activations by construction; the paper acknowledges this risk in Section 2.3.1 but does not run a control condition. However, that threatens the internal validity of the geospatial interpretation, not the circularity of the derivation: the reported autocorrelation statistics are not equivalent to the inputs by construction, and the sparse-autoencoder features are not fitted to geographic labels. Under the stated standard requiring an exhibitable reduction from claimed result to input, no circular step is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its load-bearing assumptions concern data representativeness, prompt design, faithfulness of activations, and the applicability of Moran's I. The main free parameters are thresholds and SAE hyperparameters chosen by the authors.

free parameters (4)
  • Moran's I thresholds = p<0.01 and I>=0.3
    Chosen thresholds for flagging spatially autocorrelated neurons and features; not theory-derived, and no multiple-comparison correction across 12,288 tests.
  • SAE expansion and k = k=2048, expansion 8x
    Sparse autoencoder size and sparsity level selected by inspecting final training loss on the full dataset, not by a held-out validation or by the spatial task.
  • Layer set = layers 7, 15, 31
    Hand-picked early/middle/late layers following prior work; different layer choices could change spatial signal.
  • Population filters = Italy >500; UK/US >0
    Data inclusion criteria for placenames; may bias coverage and the spatial distributions compared across regions.
assumptions (5)
  • domain assumption GeoNames populated places with population criteria provide a representative set of placenames for the three regions.
    Invoked in Section 2.3.1; coverage expectation drawn from [54].
  • domain assumption Mean-pooled post-attention normalized activations at a layer are a faithful stand-in for the LLM's internal representation of the prompt.
    Assumed throughout Section 2.3.1; no test of alternate pooling or normalization.
  • domain assumption The administrative region string in the prompt does not dominate the spatial autocorrelation signal.
    Load-bearing because prompts embed state/province/country names; the paper acknowledges this and argues for additional patterns, but does not control for it.
  • standard math Spatial autocorrelation (global and local Moran's I) is an appropriate statistic for detecting geographic structure in high-dimensional activations.
    Standard spatial statistics applied to activation values; validity depends on the spatial weighting scheme, which is not detailed in the text.
  • domain assumption Sparse autoencoder features trained on the same dataset reconstruct the model's internal representations well enough to be interpreted.
    SAE trained on the full dataset without a held-out split; k chosen by training loss; no evaluation of reconstruction quality on unseen inputs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Geospatial Mechanistic Interpretability of Large Language Models." pith.science (2026). https://pith.science/paper/ZHXGLBZO

@misc{pith2026250503368,
  author       = {Pith},
  title        = {Pith review of: Geospatial Mechanistic Interpretability of Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZHXGLBZO}},
  note         = {Machine review of arXiv:2505.03368}
}
read the original abstract

Large Language Models (LLMs) have demonstrated unprecedented capabilities across various natural language processing tasks. Their ability to process and generate viable text and code has made them ubiquitous in many fields, while their deployment as knowledge bases and "reasoning" tools remains an area of ongoing research. In geography, a growing body of literature has been focusing on evaluating LLMs' geographical knowledge and their ability to perform spatial reasoning. However, very little is still known about the internal functioning of these models, especially about how they process geographical information. In this chapter, we establish a novel framework for the study of geospatial mechanistic interpretability - using spatial analysis to reverse engineer how LLMs handle geographical information. Our aim is to advance our understanding of the internal representations that these complex models generate while processing geographical information - what one might call "how LLMs think about geographic information" if such phrasing was not an undue anthropomorphism. We first outline the use of probing in revealing internal structures within LLMs. We then introduce the field of mechanistic interpretability, discussing the superposition hypothesis and the role of sparse autoencoders in disentangling polysemantic internal representations of LLMs into more interpretable, monosemantic features. In our experiments, we use spatial autocorrelation to show how features obtained for placenames display spatial patterns related to their geographic location and can thus be interpreted geospatially, providing insights into how these models process geographical information. We conclude by discussing how our framework can help shape the study and use of foundation models in geography.

Figures

Figures reproduced from arXiv: 2505.03368 by the authors.

Figure 1
Figure 1. An example illustrating the extraction of the activations from an LLM (top); the use of the activations in a linear probe to predict the latitude and longitude of the place mentioned in the input (bottom-left) and a sparse autoencoder (bottom-right); and the use of spatial autocorrelation to analyse the activations and the sparse features (centre). Our approach encompasses the latter two components. high output valu… view at source ↗
Figure 2
Figure 2. Activations captured for the input placenames at different layers of the LLM (left for each region) and their local spatial autocorrelation (local Moran’s I clusters, p < .01, right for each region), illustrating the polysemantic nature of its internal representations. Two neurons at layers 7 (a) and 15 (b) show high values for the State of New York and Northern Ireland and very low values for northern Italy. A neur… view at source ↗
Figure 3
Figure 3. Features extracted from layer 15 through a sparse autoencoder (left for each region) and their local spatial autocorrelation (local Moran’s I clusters, p < .01, right for each region): (a) Wales as a region part of prompt; (b) south of Italy as a region activating a seemingly monosemantic feature; (c) north-east of Italy and north-west of England as regions activating a seemingly polysemantic feature; and (d) a repr… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 55 canonical work pages

  1. [1]

    On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper)

    Mai G, Huang W, Sun J, Song S, Mishra D, Liu N, et al. On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper). ACM Trans Spatial Algorithms Syst. 2024 Jul;10(2)

    work page 2024

  2. [2]

    GPT, large language models (LLMs) and generative artificial intelligence (GAI) models in geospatial science: a systematic review

    Wang S, Hu T, Xiao H, Li Y , Zhang C, Ning H, et al. GPT, large language models (LLMs) and generative artificial intelligence (GAI) models in geospatial science: a systematic review. International Journal of Digital Earth. 2024;17(1):2353122

    work page 2024

  3. [3]

    Correctness Comparison of ChatGPT-4, Gemini, Claude-3, and Copilot for Spatial Tasks

    Hochmair HH, Juh ´asz L, Kemp T. Correctness Comparison of ChatGPT-4, Gemini, Claude-3, and Copilot for Spatial Tasks. Transactions in GIS. 2024 Aug

    work page 2024

  4. [4]

    Evaluating Large Language Models on Spatial Tasks: A Multi-Task Benchmarking Study; 2024

    Xu L, Zhao S, Lin Q, Chen L, Luo Q, Wu S, et al.. Evaluating Large Language Models on Spatial Tasks: A Multi-Task Benchmarking Study; 2024. Available from: https://arxiv.org/abs/2408.14438

    arXiv 2024

  5. [5]

    Dialectical language model evaluation: An initial appraisal of the com- monsense spatial reasoning abilities of LLMs; 2023

    Cohn AG, Hernandez-Orallo J. Dialectical language model evaluation: An initial appraisal of the com- monsense spatial reasoning abilities of LLMs; 2023. Available from: https://arxiv.org/abs/ 2304.11164

    arXiv 2023

  6. [6]

    Evaluating the Ability of Large Language Models to Reason About Cardinal Directions

    Cohn AG, Blackwell RE. Evaluating the Ability of Large Language Models to Reason About Cardinal Directions. In: Adams B, Griffin AL, Scheider S, McKenzie G, editors. 16th International Conference on Spatial Information Theory (COSIT 2024). vol. 315 of Leibniz International Proceedings in Informatics (LIPIcs). Dagstuhl, Germany: Schloss Dagstuhl – Leibniz-...

    work page 2024

  7. [7]

    Advancing spatial reasoning in large language models: an in-depth evalu- ation and enhancement using the StepGame benchmark

    Li F, Hogg DC, Cohn AG. Advancing spatial reasoning in large language models: an in-depth evalu- ation and enhancement using the StepGame benchmark. In: Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Ar- tificial Intelligence and Fourteenth Symposium on Educational Adva...

    work page 2024

  8. [8]

    Toponym resolution leveraging lightweight and open-source large language models and geo-knowledge

    Hu X, Kersten J, Klan F, Farzana SM. Toponym resolution leveraging lightweight and open-source large language models and geo-knowledge. International Journal of Geographical Information Science. 2024;0(0):1-28

    work page 2024

Show all 74 references
  1. [9]

    Autonomous GIS: the next-generation AI-powered GIS

    Li Z, Ning H. Autonomous GIS: the next-generation AI-powered GIS. International Journal of Digital Earth. 2023;16(2):4668-86

    2023

  2. [10]

    GeoGPT: An assistant for understanding and processing geospatial tasks

    Zhang Y , Wei C, He Z, Yu W. GeoGPT: An assistant for understanding and processing geospatial tasks. International Journal of Applied Earth Observation and Geoinformation. 2024;131:103976

    2024

  3. [11]

    BB-GeoGPT: A framework for learning a large language model for geographic information science

    Zhang Y , Wang Z, He Z, Li J, Mai G, Lin J, et al. BB-GeoGPT: A framework for learning a large language model for geographic information science. Information Processing & Management. 2024;61(5):103808

    2024

  4. [12]

    MapGPT: an autonomous framework for mapping by integrating large language model and cartographic tools

    Zhang Y , He Z, Li J, Lin J, Guan Q, Yu W. MapGPT: an autonomous framework for mapping by integrating large language model and cartographic tools. Cartography and Geographic Information Science. 2024;0(0):1-27

    2024

  5. [13]

    GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilots

    Singh S, Fore M, Stamoulis D. GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilots. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

  6. [14]

    On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applications

    Tan C, Cao Q, Li Y , Zhang J, Yang X, Zhao H, et al.. On the Promises and Challenges of Multimodal Foundation Models for Geographical, Environmental, Agricultural, and Urban Planning Applications

  7. [15]

    PlanGPT: Enhancing Urban Planning with Tailored Language Model and Efficient Retrieval; 2024

    Zhu H, Zhang W, Huang N, Li B, Niu L, Fan Z, et al.. PlanGPT: Enhancing Urban Planning with Tailored Language Model and Efficient Retrieval; 2024. Available from: https://arxiv.org/abs/ 2402.19273

    2024 arXiv

  8. [16]

    Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messages

    Hu Y , Mai G, Cundy C, Choi K, Lao N, Liu W, et al. Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messages. International Journal of Geographical Information Science. 2023;37(11):2289-318

    2023

  9. [17]

    Charting New Territories: Exploring the Geo- graphic and Geospatial Capabilities of Multimodal LLMs

    Roberts J, L ¨uddecke T, Sheikh R, Han K, Albanie S. Charting New Territories: Exploring the Geo- graphic and Geospatial Capabilities of Multimodal LLMs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops; 2024. p. 554-63

    2024

  10. [18]

    CityGPT: Empowering Urban Spatial Cognition of Large Language Models; 2024

    Feng J, Du Y , Liu T, Guo S, Lin Y , Li Y . CityGPT: Empowering Urban Spatial Cognition of Large Language Models; 2024. Available from: https://arxiv.org/abs/2406.13948

    2024 arXiv

  11. [19]

    Distortions in Judged Spatial Relations in Large Language Models

    Fulman N, Memduho ˘glu A, Zipf A. Distortions in Judged Spatial Relations in Large Language Models. The Professional Geographer. 2024;76(6):703-11

    2024

  12. [20]

    Where to move next: Zero-shot generalization of llms for next poi recommendation

    Feng S, Lyu H, Li F, Sun Z, Chen C. Where to move next: Zero-shot generalization of llms for next poi recommendation. In: 2024 IEEE Conference on Artificial Intelligence (CAI). IEEE; 2024. p. 1530-5. May 2025

    2024

  13. [21]

    Do Sentence Transformers Learn Quasi-Geospatial Concepts from General Text?; 2024

    Ilyankou I, Lipani A, Cavazzi S, Gao X, Haworth J. Do Sentence Transformers Learn Quasi-Geospatial Concepts from General Text?; 2024. Available from: https://arxiv.org/abs/2404.04169

    2024 arXiv

  14. [22]

    GPT4GEO: How a Language Model Sees the World’s Geography; 2023

    Roberts J, L ¨uddecke T, Das S, Han K, Albanie S. GPT4GEO: How a Language Model Sees the World’s Geography; 2023. Available from: https://arxiv.org/abs/2306.00020

    2023 arXiv

  15. [23]

    Are Large Language Models Geospatially Knowledgeable? In: Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems

    Bhandari P, Anastasopoulos A, Pfoser D. Are Large Language Models Geospatially Knowledgeable? In: Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems. SIGSPATIAL ’23. New York, NY , USA: Association for Computing Machinery; 2023

    2023

  16. [24]

    Evaluation of Geographical Dis- tortions in Language Models: A Crucial Step Towards Equitable Representations

    Decoupes R, Interdonato R, Roche M, Teisseire M, Valentin S. Evaluation of Geographical Dis- tortions in Language Models: A Crucial Step Towards Equitable Representations. arXiv preprint arXiv:240417401. 2024

    2024

  17. [25]

    Measuring Geographic Diversity of Foundation Models with a Natural Language–based Geo-guessing Experiment on GPT-4

    Liu Z, Janowicz K, Currier K, Shi M. Measuring Geographic Diversity of Foundation Models with a Natural Language–based Geo-guessing Experiment on GPT-4. AGILE: GIScience Series. 2024;5:38

    2024

  18. [26]

    Making Geographic Space Explicit In Probing Multimodal Large Lan- guage Models For Cul-Tural Subjects

    Liu Z, Currier K, Janowicz K. Making Geographic Space Explicit In Probing Multimodal Large Lan- guage Models For Cul-Tural Subjects. In: Global AI Cultures Workshop of ICLR 2024; 2024

    2024

  19. [27]

    Mapping Great Britain’s semantic footprints through a large language model analysis of Reddit comments

    Berragan C, Singleton A, Calafiore A, Morley J. Mapping Great Britain’s semantic footprints through a large language model analysis of Reddit comments. Computers, Environment and Urban Systems. 2024;110:102121

    2024

  20. [28]

    Deep learning

    LeCun Y , Bengio Y , Hinton G. Deep learning. nature. 2015;521(7553):436-44

    2015

  21. [29]

    Rectified linear units improve restricted boltzmann machines

    Nair V , Hinton GE. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10); 2010. p. 807-14

    2010

  22. [30]

    Attention is All you Need

    Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is All you Need. In: Guyon I, Luxburg UV , Bengio S, Wallach H, Fergus R, Vishwanathan S, et al., editors. Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc.; 2017

    2017

  23. [31]

    Representation Learning: A Review and New Perspectives

    Bengio Y , Courville A, Vincent P. Representation Learning: A Review and New Perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2013;35(8):1798-828

    2013

  24. [32]

    Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

    Templeton A, Conerly T, Marcus J, Lindsey J, Bricken T, Chen B, et al. Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Transformer Circuits Thread. 2024

    2024

  25. [33]

    Backpropagation and the brain

    Lillicrap TP, Santoro A, Marris L, Akerman CJ, Hinton G. Backpropagation and the brain. Nature Reviews Neuroscience. 2020;21(6):335-46

    2020

  26. [34]

    Fine-Tuning Language Models from Human Preferences; 2020

    Ziegler DM, Stiennon N, Wu J, Brown TB, Radford A, Amodei D, et al.. Fine-Tuning Language Models from Human Preferences; 2020. Available from: https://arxiv.org/abs/1909.08593

    2020 arXiv

  27. [35]

    A Computer Movie Simulating Urban Growth in the Detroit Region

    Tobler WR. A Computer Movie Simulating Urban Growth in the Detroit Region. Economic Geography. 1970;46:234-40

    1970

  28. [36]

    Do Language Models Know the Way to Rome? arXiv preprint arXiv:210907971

    Li ´etard B, Abdou M, Søgaard A. Do Language Models Know the Way to Rome? arXiv preprint arXiv:210907971. 2021

    2021

  29. [37]

    Language Models Represent Space and Time

    Gurnee W, Tegmark M. Language Models Represent Space and Time. arXiv; 2024

    2024

  30. [38]

    On the Scaling Laws of Geographical Representation in Language Models

    Godey N, de la Clergerie ´E, Sagot B. On the Scaling Laws of Geographical Representation in Language Models. arXiv; 2024

    2024

  31. [39]

    More than Correlation: Do Large Language Models Learn Causal Representations of Space?; 2023

    Chen Y , Gan Y , Li S, Yao L, Zhao X. More than Correlation: Do Large Language Models Learn Causal Representations of Space?; 2023. Available from: https://arxiv.org/abs/2312.16257

    2023 arXiv

  32. [40]

    Geographic information analysis

    O’Sullivan D, Unwin D. Geographic information analysis. John Wiley & Sons; 2010

    2010

  33. [41]

    Mechanistic Interpretability for AI Safety – A Review; 2024

    Bereska L, Gavves E. Mechanistic Interpretability for AI Safety – A Review; 2024. Available from: https://arxiv.org/abs/2404.14082

    2024 arXiv

  34. [42]

    Investigating causal understanding in LLMs

    Hobbhahn M, Lieberum T, Seiler D. Investigating causal understanding in LLMs. In: NeurIPS ML Safety Workshop; 2022

    2022

  35. [43]

    Do NLP models know numbers? probing numeracy in embeddings

    Wallace E, Wang Y , Li S, Singh S, Gardner M. Do NLP models know numbers? probing numeracy in embeddings. arXiv preprint arXiv:190907940. 2019

    2019

  36. [44]

    Probing what different NLP tasks teach machines about function word comprehension

    Kim N, Patel R, Poliak A, Wang A, Xia P, McCoy RT, et al. Probing what different NLP tasks teach machines about function word comprehension. arXiv preprint arXiv:190411544. 2019

    2019

  37. [45]

    Probing classifiers: Promises, shortcomings, and advances

    Belinkov Y . Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics. 2022;48(1):207-19

    2022

  38. [46]

    Discourse probing of pretrained language models

    Koto F, Lau JH, Baldwin T. Discourse probing of pretrained language models. arXiv preprint arXiv:210405882. 2021

    2021

  39. [47]

    Probing for constituency structure in neural language models

    Arps D, Samih Y , Kallmeyer L, Sajjad H. Probing for constituency structure in neural language models. arXiv preprint arXiv:220406201. 2022

    2022

  40. [48]

    From Pretraining Data to Language Models to Downstream Tasks: May 2025 Tracking the Trails of Political Biases Leading to Unfair NLP Models

    Feng S, Park CY , Liu Y , Tsvetkov Y . From Pretraining Data to Language Models to Downstream Tasks: May 2025 Tracking the Trails of Political Biases Leading to Unfair NLP Models. In: Rogers A, Boyd-Graber J, Okazaki N, editors. Proceedings of the 61st Annual Meeting of the As...

    2025

  41. [49]

    Probing pretrained language models for lex- ical semantics

    Vuli ´c I, Ponti EM, Litschko R, Glava ˇs G, Korhonen A. Probing pretrained language models for lex- ical semantics. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP); 2020. p. 7222-40

    2020

  42. [50]

    Syntactic Perturbations Reveal Repre- sentational Correlates of Hierarchical Phrase Structure in Pretrained Language Models; 2021

    Alleman M, Mamou J, Del Rio MA, Tang H, Kim Y , Chung S. Syntactic Perturbations Reveal Repre- sentational Correlates of Hierarchical Phrase Structure in Pretrained Language Models; 2021. Available from: https://arxiv.org/abs/2104.07578

    2021 arXiv

  43. [51]

    Identifying Linear Relational Concepts in Large Language Models

    Chanin D, Hunter A, Camburu OM. Identifying Linear Relational Concepts in Large Language Models

  44. [52]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Burstein J, Doran C, Solorio T, editors. Proceedings of the 2019 Con- ference of the North American Chapter of the Association for Computational Lingui...

    2019

  45. [53]

    Spatial autocorrelation

    Getis A. Spatial autocorrelation. In: Handbook of applied spatial analysis: Software tools, methods and applications. Springer; 2009. p. 255-78

    2009

  46. [54]

    A quantitative analysis of global gazetteers: Patterns of coverage for common feature types

    Acheson E, De Sabbata S, Purves RS. A quantitative analysis of global gazetteers: Patterns of coverage for common feature types. Computers, Environment and Urban Systems. 2017;64:309-20

    2017

  47. [55]

    Mistral 7B

    Jiang AQ, Sablayrolles A, Mensch A, Bamford C, Chaplot DS, Casas Ddl, et al. Mistral 7B. arXiv preprint arXiv:231006825. 2023

    2023

  48. [56]

    Training language mod- els to follow instructions with human feedback

    Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, et al. Training language mod- els to follow instructions with human feedback. Advances in neural information processing systems. 2022;35:27730-44

    2022

  49. [57]

    Toy Models of Superposition

    Elhage N, Hume T, Olsson C, Schiefer N, Henighan T, Kravec S, et al.. Toy Models of Superposition

  50. [58]

    Pooling methods in deep neural networks, a review

    Gholamalinezhad H, Khosravi H. Pooling methods in deep neural networks, a review. arXiv preprint arXiv:200907485. 2020

    2020

  51. [59]

    Notes on Continuous Stochastic Phenomena

    Moran PAP. Notes on Continuous Stochastic Phenomena. Biometrika. 1950;37(1/2):17-23

    1950

  52. [60]

    Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs

    Singh AK, Strouse D. Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs

  53. [61]

    Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters; 2024

    Snell C, Lee J, Xu K, Kumar A. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters; 2024. Available from: https://arxiv.org/abs/2408.03314

    2024 arXiv

  54. [62]

    Towards Monosemantic- ity: Decomposing Language Models With Dictionary Learning

    Bricken T, Templeton A, Batson J, Chen B, Jermyn A, Conerly T, et al. Towards Monosemantic- ity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread. 2023. Https://transformer-circuits.pub/2023/monosemantic-features/index.html

    2023

  55. [63]

    Sparse autoencoder

    Ng A, et al. Sparse autoencoder. CS294A Lecture notes. 2011;72(2011):1-19

    2011

  56. [64]

    Available from: https://arxiv.org/abs/2402.14903

    arXiv

  57. [65]

    Open Problems in Mechanistic Interpretability; 2025

    Sharkey L, Chughtai B, Batson J, Lindsey J, Wu J, Bushnaq L, et al.. Open Problems in Mechanistic Interpretability; 2025. Available from: https://arxiv.org/abs/2501.16496

    2025 arXiv

  58. [66]

    On the Opportunities and Risks of Foundation Models; 2022

    Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, et al.. On the Opportunities and Risks of Foundation Models; 2022. Available from: https://arxiv.org/abs/2108.07258

    2022 arXiv

  59. [67]

    Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small; 2022

    Wang K, Variengien A, Conmy A, Shlegeris B, Steinhardt J. Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small; 2022. Available from:https://arxiv.org/abs/2211. 00593

    2022

  60. [68]

    Scaling and evaluating sparse autoen- coders; 2024

    Gao L, la Tour TD, Tillman H, Goh G, Troll R, Radford A, et al.. Scaling and evaluating sparse autoen- coders; 2024. Available from: https://arxiv.org/abs/2406.04093

    2024 arXiv

  61. [69]

    Modelling vague places with knowledge from the Web

    Jones CB, Purves RS, Clough PD, Joho H. Modelling vague places with knowledge from the Web. International Journal of Geographical Information Science. 2008;22(10):1045-65

    2008

  62. [70]

    GeoAI: spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond

    Janowicz K, Gao S, McKenzie G, Hu Y , Bhaduri B. GeoAI: spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond. International Journal of Geographical Information Science. 2020;34(4):625-36

    2020

  63. [72]

    Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models; 2024

    Marks S, Rager C, Michaud EJ, Belinkov Y , Bau D, Mueller A. Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models; 2024. Available from:https://arxiv. org/abs/2403.19647

    2024 arXiv

  64. [2022]

    Available from: https://arxiv.org/abs/2209.10652

    arXiv

  65. [2023]

    Available from: https://arxiv.org/abs/2312.17016

    arXiv

  66. [2024]

    Available from: https://arxiv.org/abs/2311.08968

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.