Pith. sign in

REVIEW 7 cited by

GeoLLM: Extracting Geospatial Knowledge from Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.06213 v2 pith:32RVIDFB submitted 2023-10-10 cs.CL cs.LG

Rohin Manvi , Samar Khanna , Gengchen Mai , Marshall Burke , David Lobell , Stefano Ermon This is my paper
classification cs.CLcs.LG
keywords geospatialgeollmllmstasksacrossinformationknowledgelanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

The application of machine learning (ML) in a range of geospatial tasks is increasingly common but often relies on globally available covariates such as satellite imagery that can either be expensive or lack predictive power. Here we explore the question of whether the vast amounts of knowledge found in Internet language corpora, now compressed within large language models (LLMs), can be leveraged for geospatial prediction tasks. We first demonstrate that LLMs embed remarkable spatial information about locations, but naively querying LLMs using geographic coordinates alone is ineffective in predicting key indicators like population density. We then present GeoLLM, a novel method that can effectively extract geospatial knowledge from LLMs with auxiliary map data from OpenStreetMap. We demonstrate the utility of our approach across multiple tasks of central interest to the international community, including the measurement of population density and economic livelihoods. Across these tasks, our method demonstrates a 70% improvement in performance (measured using Pearson's $r^2$) relative to baselines that use nearest neighbors or use information directly from the prompt, and performance equal to or exceeding satellite-based benchmarks in the literature. With GeoLLM, we observe that GPT-3.5 outperforms Llama 2 and RoBERTa by 19% and 51% respectively, suggesting that the performance of our method scales well with the size of the model and its pretraining dataset. Our experiments reveal that LLMs are remarkably sample-efficient, rich in geospatial information, and robust across the globe. Crucially, GeoLLM shows promise in mitigating the limitations of existing geospatial covariates and complementing them well. Code is available on the project website: https://rohinmanvi.github.io/GeoLLM

Discussion (0). Sign in to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Around the World in 24 Hours: Probing LLM Knowledge of Time and Place

    cs.CL 2025-06 conditional novelty 6.0 of 10

    On GeoTemp, the best open model answers only 56% of two-city time questions and 33% when an hour shift is added, despite near-perfect scores on pure time arithmetic.

  2. MapStory: Prototyping Editable Map Animations with LLM Agents

    cs.HC 2025-05 conditional novelty 6.0 of 10

    Natural language scripts can be turned into editable, geospatially grounded map animations through MapStory's dual-agent LLM architecture.

  3. Foundation Models for Geospatial Reasoning: Assessing Capabilities of Large Language Models in Understanding Geometries and Topological Spatial Relations

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Large language models, especially GPT-4 with few-shot prompts, can classify topological spatial relations between WKT-encoded geometries with roughly 0.6 to 0.66 accuracy, though errors cluster near conceptually simil...

  4. POI-Enhancer: An LLM-based Semantic Enhancement Framework for POI Representation Learning

    cs.AI 2025-02 conditional novelty 6.0 of 10

    POI-Enhancer uses LLM-generated text features and attention-based fusion to improve POI embeddings from six classic models, gaining consistent accuracy on three real-world mobility datasets.

  5. Mitigating Geospatial Knowledge Hallucination in Large Language Models: Benchmarking and Dynamic Factuality Aligning

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A new benchmark called GEOHALUBENCH measures how often LLMs invent, omit, or confuse real-world places and relations, and a dynamic-beta KTO method reduces these errors on the benchmark.

  6. A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applications

    cs.AI 2025-06 conditional novelty 4.0 of 10

    A survey of 80+ Deep Research systems that proposes a four-layer taxonomy (foundation models, tool use, planning, synthesis) and compares commercial and open-source implementations.

  7. The World As Large Language Models See It: Exploring the reliability of LLMs in representing geographical features

    cs.CY 2025-05 conditional novelty 4.0 of 10

    GPT-4o and Gemini 2.0 Flash approximate the geography of Austria but show systematic biases in coordinates and elevations and frequent errors in assigning federal states.

Pith tools