Pith. sign in

REVIEW 8 cited by

DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.01412 v5 pith:JAE6EOJ6 submitted 2023-10-02 cs.CV cs.RO

Zhenhua Xu , Yujia Zhang , Enze Xie , Zhen Zhao , Yong Guo , Kwan-Yee. K. Wong , Zhenguo Li , Hengshuang Zhao This is my paper · ORCID
classification cs.CVcs.RO
keywords drivegpt4autonomousdrivingend-to-endinterpretabledatadatasetlanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multimodal large language models (MLLMs) have emerged as a prominent area of interest within the research community, given their proficiency in handling and reasoning with non-textual data, including images and videos. This study seeks to extend the application of MLLMs to the realm of autonomous driving by introducing DriveGPT4, a novel interpretable end-to-end autonomous driving system based on LLMs. Capable of processing multi-frame video inputs and textual queries, DriveGPT4 facilitates the interpretation of vehicle actions, offers pertinent reasoning, and effectively addresses a diverse range of questions posed by users. Furthermore, DriveGPT4 predicts low-level vehicle control signals in an end-to-end fashion.These advanced capabilities are achieved through the utilization of a bespoke visual instruction tuning dataset, specifically tailored for autonomous driving applications, in conjunction with a mix-finetuning training strategy. DriveGPT4 represents the pioneering effort to leverage LLMs for the development of an interpretable end-to-end autonomous driving solution. Evaluations conducted on the BDD-X dataset showcase the superior qualitative and quantitative performance of DriveGPT4. Additionally, the fine-tuning of domain-specific data enables DriveGPT4 to yield close or even improved results in terms of autonomous driving grounding when contrasted with GPT4-V.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MentalThink: Shaping Thoughts in Mental SVG World

    cs.AI 2026-07 conditional novelty 7.0 of 10

    MLLMs that generate and render SVG sketches as multi-turn intermediate reasoning steps reach 55.1% on VSIBench and 76.0% on MindCube, far above the Qwen2.5-VL-7B backbone.

  2. PDB-Eval: An Evaluation of Large Multimodal Models for Description and Explanation of Personalized Driving Behavior

    cs.CV 2025-07 conditional novelty 6.0 of 10

    Introduces PDB-Eval, a dual-view benchmark for fine-grained driver behavior description and explanation, and shows fine-tuning on it boosts performance on driving QA and downstream intention and recognition tasks.

  3. AD^2-Bench: A Hierarchical CoT Benchmark for MLLM in Autonomous Driving under Adverse Conditions

    cs.CV 2025-06 conditional novelty 6.0 of 10

    AD^2-Bench is a new adverse-weather driving benchmark with hierarchical chain-of-thought annotations and LLM-based quality metrics; 12 MLLMs all scored below 60%.

  4. Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models

    cs.CV 2025-05 conditional novelty 6.0 of 10

    A new 80K-clip dataset of unstructured driving scenarios with Q&A annotations improves VLA performance on NeuroNCAP and nuScenes benchmarks.

  5. CoopReflect: Towards Natural Language Communication for Cooperative Autonomous Driving via Multi-Agent Learning

    cs.RO 2025-05 conditional novelty 6.0 of 10

    Post-episode multi-agent debriefing lets LLM driving agents learn concise natural-language coordination protocols that avoid collisions and merge traffic, and distillation makes the policy fast enough for near-real-time use.

  6. Leveraging LLMs for Mission Planning in Precision Agriculture

    cs.RO 2025-06 conditional novelty 5.0 of 10

    ChatGPT can generate valid behavior-tree mission plans for agricultural robots from natural-language requests, but spatial and route-optimization tasks still require an external stochastic-orienteering solver.

  7. One For All: LLM-based Heterogeneous Mission Planning in Precision Agriculture

    cs.RO 2025-06 conditional novelty 4.0 of 10

    An LLM-based planner with XML-schema validation generates behavior-tree missions in plain language for both a Husky rover and a Kinova arm in agricultural scenarios.

  8. 2nd Place Solution for CVPR2024 E2E Challenge: End-to-End Autonomous Driving Using Vision Language Model

    cs.CV 2025-09 conditional novelty 3.0 of 10

    A single-camera vision-language-model system scored 0.8747 on the CVPR 2024 E2E driving benchmark, the best camera-only result.

Pith tools