Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems

Hao Zhou; Yuhang Wang

REVIEW 5 major objections 6 minor 27 references

Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LKAlert predicts lane-keeping failures 1-3 seconds ahead and explains them in plain language.

desk verdict Useful new dataset and a sensible VLM+LoRA recipe, but the evaluation does not support the 1–3 second prediction claim because the labels and the model inputs share the same CAN/Openpilot signals. read the letter →

arxiv 2505.11535 v1 pith:NI2ARDGO submitted 2025-05-14 cs.RO cs.CVcs.LG

Yuhang Wang , Hao Zhou This is my paper · ORCID

classification cs.ROcs.CVcs.LG

keywords lanekeepingassistvision-languagemodelspredictivealertingexplainableartificialintelligenceadvanceddriverassistancesystemslow-rankadaptationsegmentationoversight

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LKAlert aims to close the gap between black-box lane-keeping assistance and the driver who must supervise it. The paper claims that a vision-language model, fed dash-cam video, lane-segmentation masks, and vehicle CAN data, can flag an upcoming LKA failure one to three seconds before it happens and explain in words why the alert was raised. To show this, the authors build the OpenLKA-Alert benchmark from real driving data and fine-tune Qwen2.5-VL-7B with LoRA, keeping the vision encoder frozen and using interpretable lane masks as guiding attention. If the result holds, drivers would get an early, understandable warning instead of a sudden disengagement, and the same recipe could be applied to other opaque automation systems.

What carries the argument

The central mechanism is LKAlert's multimodal conditioning set: the RGB frame, two LaneNet segmentation masks (binary and instance), and CAN/Openpilot text, all passed through a frozen Qwen2.5-VL encoder whose decoder is adapted by LoRA. The LaneNet masks are the interpretable surrogate features that make lane geometry explicit to the model without retraining the vision backbone; LoRA injects low-rank updates into the decoder so the model learns to correlate those geometric cues with imminent failures while maintaining the base model's inference latency. The output is a two-field sequence: a Yes/No alert and a natural-language explanation.

What would settle it

Run the trained model with the CAN/Openpilot text inputs removed or shifted forward in time, and measure whether it still flags failures at least one second before the annotated disengagement; if predictive accuracy collapses, the claimed lead-time advantage is an artifact of reading the vehicle's current state rather than genuine anticipation.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a VLM adapted with LoRA and guided by surrogate lane-segmentation masks can serve as a practical supervisory layer for a commercial LKA system. The fine-tuned Qwen2.5-VL-7B reaches 69.8% accuracy and 58.6% F1 on alert prediction, generates explanations scoring 71.7 ROUGE-L against human-authored justifications, and runs at about two samples per second. The authors also report that adding the two LaneNet masks improves F1 by 12.4 points and ROUGE-L by 68.2 points over the unguided baseline, and that LoRA keeps inference speed stable while training only a small fraction of parameters. Together these numbers are offered as evidence that a VLM can forecast LKA disengagement with enough lead time to warn a driver, and can articulate the reason in language.

Load-bearing premise

The model's input includes the same CAN and Openpilot signals that define the ground-truth failure labels, so the reported anticipation could just be the model reading the vehicle's current lane-centering state from its text input rather than truly foreseeing a failure from visual or geometric cues.

Editorial extensions

If this is right

A VLM supervised with surrogate masks and LoRA can be a real-time alerting layer for black-box ADAS, producing both a binary warning and a human-readable justification at roughly 2 Hz.
Adding interpretable lane-segmentation masks consistently improves predictive accuracy and explanation quality across model scales, with the largest gains on the 7B model.
LoRA-based decoder-only adaptation is sufficient to transfer a general VLM to this safety task, avoiding retraining of the vision encoder and adding no extra inference latency.
The OpenLKA-Alert dataset provides a first benchmark for predictive and explainable LKA failure warnings, enabling future comparisons and extensions.
The same methodological recipe—frozen encoder, surrogate feature guidance, LoRA—is proposed as a template for interpreting other opaque automation systems beyond lane keeping.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A clean test of genuine anticipation would hold out the time dimension: train on earlier frames of each failure window and test only on frames at least one second before disengagement, since the current split may mix pre-failure and post-failure frames.
Because the paper's offline validation is acknowledged in the conclusion, a simulator-based user study measuring whether LKAlert changes driver reaction time or trust would be the natural next experiment.
The choice of LaneNet as a surrogate deliberately mirrors the perception limits of 2019-2022 production vehicles; this suggests the same system could be re-aimed at whatever perception model a given car actually runs, turning the alerting layer into a vehicle-specific monitor.
Combining the 2B or 3B unguided variants with the 7B guided model could yield a two-stage design—cheap pre-filtering on the vehicle, then a deeper VLM explanation only when risk is elevated—although the paper does not explore this.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This one is worth a look, but the headline claim needs to be treated with caution. What is genuinely new: OpenLKA-Alert dataset, first purpose-built for predictive and explainable LKA warnings, with synchronized video/CAN/Openpilot and human-authored explanations. The idea of feeding LaneNet binary/instance masks as surrogate geometric guidance to a frozen Qwen2.5-VL with LoRA decoder adaptation is a reasonable application-level combination. The guided-vs-unguided ablation shows consistent gains, and the LoRA efficiency story (2 Hz) is plausible.

The soft spot is exactly where the reader's report points. Section III-A defines the 'Yes' labels by locating LKA disengagement or large lane-centering deviation using CAN and Openpilot signals, then extending a window 3.5s before and 2.5s after. The model input includes those same CAN dynamics and Openpilot outputs as text. So for any frame at or after the failure instant, the model can read the current LKA state from the text and answer 'Yes' correctly. That is detection, not prediction. Validation draws 1,000 samples uniformly from this windowed data with no breakdown by temporal offset, and the manual screening that keeps only frames resembling the 7th failure frame makes it worse. No ablation removes the text channel.

The paper itself admits offline validation, but the lack of temporal stratification is the deeper problem. I don't think this is fraud; it reads like an oversight. The fix is straightforward: ablate CAN/Openpilot text inputs, report performance by time-to-failure (and after failure separately), and split by video rather than uniformly sampled frames. If the visual+mask signal alone still gives a useful lead time, the paper becomes solid.

The citation pattern is fine; related work is covered and the self-citation to OpenLKA is appropriate since this dataset builds on it. No invented entities. The free parameters listed (thresholds, window sizes, LoRA rank) are not tuned adversarially, but the thresholds define the label, so they matter more than typical hyperparameters.

Who is this for? Researchers working on ADAS supervision and VLM-based driver warning systems. The dataset is a real contribution if released cleanly, even if the current numbers are not believable as evidence of anticipation. I would send it to a serious referee because the dataset and framework deserve an airing, but the referee should require the ablation and temporal re-analysis before acceptance.

My recommendation: engage, but treat '69.8% / 58.6% / 1–3s' as unverified until the re-run.

Referee Report

5 major / 6 minor

Summary. This paper presents LKAlert, a vision-language-model-based system intended to warn drivers 1-3 seconds before lane-keeping assist (LKA) failures. The system takes dashcam RGB frames, LaneNet binary and instance segmentation masks, and CAN/Openpilot-derived signals as text, and is fine-tuned with LoRA on the decoder of Qwen2.5-VL to output a Yes/No alert and a natural-language explanation. The authors also introduce OpenLKA-Alert, derived from the OpenLKA dataset, in which all frames in 6-second windows around detected LKA disengagements or large lane-centering deviations are labeled Yes. On a held-out set of 1,000 samples, the best model is reported to achieve 69.8% accuracy, 58.6% F1, 71.7 ROUGE-L, and about 2 Hz inference, which the paper interprets as successful predictive alerting.

Significance. The contribution is well motivated: a low-cost, interpretable early-warning mechanism for black-box LKA systems would have clear practical value, and a public benchmark with synchronized video, CAN, and explanation annotations would be a useful resource. The technical recipe of keeping the vision encoder frozen, adapting only the decoder with LoRA, and conditioning on surrogate segmentation masks is simple, efficient, and likely transferable to other ADAS monitoring tasks. However, the current evaluation does not establish the central predictive claim because the label-defining signals are also model inputs and the validation set is not temporally stratified; the paper is therefore best regarded as a system and dataset proposal whose alerting performance remains unvalidated.

major comments (5)

[Section III-A and Eq. (1)] The alert labels are computed from vehicle position, LKA state, attitude, and lane-centering outputs in the CAN and Openpilot streams, and those same streams are concatenated into the model's text input. Because every frame in the 3.5 s pre/2.5 s post window is labeled Yes, any frame at or after the failure moment can be classified correctly by reading the current LKA state or lane-centering deviation directly from the text. The paper neither ablates this text channel nor reports exactly which fields it contains, so the reported accuracy cannot be attributed to visual or geometric anticipation.
[Section III-D] Dval is drawn uniformly from OpenLKA-Failure and OpenLKA-Normal, with 456 Yes samples, and OpenLKA-Failure includes frames up to 2.5 s after the detected failure. No results are stratified by the frame's temporal offset from the failure, so the '1-3 s ahead' claim in the abstract and Section I is not tested. Additionally, no event-level train/test separation is reported, leaving open the possibility that frames from the same failure event appear in both training and validation.
[Section III-A] The manual screening that retains only frames 'with the same features as the 7th frame image' is not quantified and conditions the evaluation on frames that visually resemble the failure frame. This can inflate both the alerting and explanation metrics relative to an unselected deployment stream, and it makes it difficult to know how the method would behave on the original OpenLKA distribution.
[Table II and Section IV-C] The ablation study varies only the presence of the segmentation masks; it never removes or corrupts the CAN/Openpilot text channel, which is the most direct carrier of the label-defining state information. A decisive experiment would evaluate the model on pre-failure frames only, with and without the text channel, to separate genuine anticipation from reading the current failure state.
[Table I] Even on the current split, the best model's recall is 46.71%, meaning it misses over half of the failure frames; the conclusion that LKAlert is suitable for real-time, safety-relevant deployment is not supported by the reported operating point, and the paper should at minimum report the false-negative cost and a threshold analysis.

minor comments (6)

[Abstract] 'Empirical results correctly predicts upcoming LKA failures' is ungrammatical and should read 'The empirical results correctly predict upcoming LKA failures'.
[Section III-A] The direction of the labeling window is ambiguous ('3.5 seconds forward and 2.5 seconds backward'); clarify that the window extends before and after the failure timestamp and state the number of sampled frames per event.
[Section III-B and III-D] The prompt template and the exact list of CAN/Openpilot fields fed to the text encoder are not specified, which prevents replication of the central experiment.
[Section IV-A] No sample of the generated explanations is shown, so the reported ROUGE-L of 71.7 is hard to interpret; a table of representative ground-truth and generated explanation pairs would help.
[Conclusion and Figure 2] The conclusion contains the typo 'attention huiding' (should be 'guiding'), and Figure 2's caption contains 'indecates' for 'indicates'.
[Section III-D] The dataset split is described only by label counts (456 Yes, 544 No); report the number of unique failure events and vehicles in train and validation to allow assessment of temporal leakage.

Circularity Check

2 steps flagged · score 6.0 of 10

Alert labels are thresholds on the same CAN/Openpilot state fed as model input; validation mixes pre- and post-failure frames, so reported accuracy may detect current failure rather than predict 1–3 s ahead.

self definitional [Section III-A (Data Preparation), Section III-B (Inputs and Outputs), Eq. (1)]
"OpenLKA-Failure combines the vehicle position information from CAN and Openpilot, LKA state information, vehicle attitude information and the model outputs of the vehicle’s position in the lane to locate the LKA facing a larger Lane Centering Deviation and LKA disengagement. The specific location of the LKA disengagement is obtained by extending it according to a window of 3.5 seconds forward and 2.5 seconds backward. ..."

The label is, by construction, a threshold applied to CAN/Openpilot-derived quantities (Lane Centering Deviation, LKA state, vehicle attitude). Those same quantities are part of the multimodal input S = {Irgb, Mbin, Mins, c} and are encoded into X in Eq. (1). For any frame at or after the failure moment, the model can emit 'Yes' by reading the current LKA state / lane-centering value from the text input, which is the identical signal used to set the label. Thus the reported alert 'prediction' is partly a detection of the label-defining state, not an independent 1–3 s forecast.
fitted input called prediction [Section III-A (Data Preparation), Section III-D (Validation Methods), Table I]
"We annotate all the image frames of OpenLKA-Failure and OpenLKA-Normal with whether or not we should let the Alert System do the alerting for the current image, and we always annotate the Failure dataset with ‘Yes’ and the Normal dataset with ‘No.’ ... A held-out set Dval of 1,000 samples is drawn uniformly from OpenLKA-Failure and OpenLKA-Normal that not contained in the training set (456 Yes and 544 No cases)."

Dval is drawn uniformly from the same OpenLKA-Failure 6-second windows (every frame labeled 'Yes') and OpenLKA-Normal ('No'), with no temporal stratification by offset before/after the failure moment. Since frames at and after failure are labeled 'Yes' and the post-failure CAN/Openpilot state is included in the model input, the evaluation cannot separate genuine look-ahead from reading the current failure state. The 69.8% accuracy / 58.6% F1 numbers therefore do not, as claimed, demonstrate 1–3 s anticipation.

full rationale

The central predictive claim of the paper is that LKAlert 'forecast[s] potential LKA failure 1–3 seconds in advance.' However, the labels are defined by the very signals that are fed into the model: OpenLKA-Failure is located using CAN and Openpilot vehicle-position, LKA-state, attitude, and lane-centering outputs, and the model input explicitly includes CAN dynamics and Openpilot outputs as text (Fig. 1, Eq. 1). Because every frame in the 6-second failure window is labeled 'Yes', and because Dval is drawn uniformly from those windows without temporal stratification, the reported accuracy can be achieved for post-failure frames by reading the current LKA state directly from the input—this is detection, not prediction. No ablation removes the CAN/Openpilot text channel; the only ablation (Table II) removes the segmentation masks. The manual screening that retains only frames resembling the 7th failure frame further narrows the visual signal to failure-like frames, but that alone does not establish look-ahead. The paper's offline-validation caveat addresses deployment realism, not this label-input overlap. There is still independent content in the mask-guided visual reasoning and explanation generation, and not every 'Yes' frame is necessarily post-failure, so the circularity is partial rather than total; however, the headline anticipation claim is not supported by the reported evaluation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-set dataset construction parameters (deviation threshold, window sizes, sampling interval), on the assumption that CAN/Openpilot inputs do not leak the label, and on the assumption that manual screening preserves the pre-failure signal. None of these are externally validated.

free parameters (4)

Lane Centering Deviation threshold
Used to detect LKA failure events from Openpilot/CAN outputs; the specific threshold value is not reported, and it determines which frames become 'Yes' labels.
Pre- and post-event window sizes = 3.5 s before, 2.5 s after
Defines the 6-second window around each LKA event; all frames in this window are labeled 'Yes', so this hand-set window defines the effective prediction horizon.
Frame sampling interval = 0.5 s
Video frames are sampled every 0.5 s, which sets the temporal resolution and makes frame 7 the event moment.
LoRA rank and alpha
LoRA hyperparameters are not reported; they affect training but are not fitted to data. Listed for completeness.

assumptions (4)

domain assumption OpenLKA labels provide a valid ground truth for LKA failure.
The dataset defines failure via large lane centering deviation or disengagement as detected from CAN/Openpilot; whether this matches true safety-relevant failures is assumed.
domain assumption LaneNet segmentation masks are a useful surrogate for the vehicle's lane detection.
The system assumes LaneNet's binary and instance masks provide task-relevant geometric guidance for the VLM, without validating against the actual on-board lane detection of the tested vehicles.
ad hoc to paper CAN and Openpilot text inputs do not already encode the alert label.
Since the labels are computed from CAN and Openpilot signals, the model may read the label directly from these inputs; the paper implicitly assumes this is not happening but never ablates these channels.
ad hoc to paper Manually screened frames are representative of the pre-failure state.
Frames are retained only if they have the same features as the failure frame (Section III-A), which assumes away the temporal evolution that a predictive system would need to capture.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems." pith.science (2026). https://pith.science/paper/NI2ARDGO

@misc{pith2026250511535,
  author       = {Pith},
  title        = {Pith review of: Bridging Human Oversight and Black-box Driver Assistance: Vision-Language Models for Predictive Alerting in Lane Keeping Assist Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NI2ARDGO}},
  note         = {Machine review of arXiv:2505.11535}
}

read the original abstract

Lane Keeping Assist systems, while increasingly prevalent, often suffer from unpredictable real-world failures, largely due to their opaque, black-box nature, which limits driver anticipation and trust. To bridge the gap between automated assistance and effective human oversight, we present LKAlert, a novel supervisory alert system that leverages VLM to forecast potential LKA risk 1-3 seconds in advance. LKAlert processes dash-cam video and CAN data, integrating surrogate lane segmentation features from a parallel interpretable model as automated guiding attention. Unlike traditional binary classifiers, LKAlert issues both predictive alert and concise natural language explanation, enhancing driver situational awareness and trust. To support the development and evaluation of such systems, we introduce OpenLKA-Alert, the first benchmark dataset designed for predictive and explainable LKA failure warnings. It contains synchronized multimodal inputs and human-authored justifications across annotated temporal windows. We further contribute a generalizable methodological framework for VLM-based black-box behavior prediction, combining surrogate feature guidance with LoRA. This framework enables VLM to reason over structured visual context without altering its vision backbone, making it broadly applicable to other complex, opaque systems requiring interpretable oversight. Empirical results correctly predicts upcoming LKA failures with 69.8% accuracy and a 58.6\% F1-score. The system also generates high-quality textual explanations for drivers (71.7 ROUGE-L) and operates efficiently at approximately 2 Hz, confirming its suitability for real-time, in-vehicle use. Our findings establish LKAlert as a practical solution for enhancing the safety and usability of current ADAS and offer a scalable paradigm for applying VLMs to human-centered supervision of black-box automation.

Figures

Figures reproduced from arXiv: 2505.11535 by the authors.

**Figure 2.** Visual Data From OpenLKA-Failure Dataset and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Although the current laneline is clear, there is a sharp [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Semantic word cloud of the LKA Failure Explanation [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: LKAlert Architecture: Multimodal inputs (Image [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 7.** Figure 7: The Validation Results of Different Models During [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 11 canonical work pages

[1]

OpenLKA: an open dataset of lane keeping assist from market autonomous vehicles

Y . Wang, A. Alhuraish, S. Yuan, S. Wang, and H. Zhou, “Openlka: an open dataset of lane keeping assist from market autonomous vehicles,” arXiv preprint arXiv:2501.03287 , 2025

work page Pith review arXiv 2025
[2]

Effectiveness of advanced driver assistance systems in preventing system-relevant crashes,

R. Spicer, A. Vahabaghaie, D. Murakhovsky, G. Bahouth, B. Drayer, and S. S. Lawrence, “Effectiveness of advanced driver assistance systems in preventing system-relevant crashes,” SAE International Journal of Advances and Current Practices in Mobility , vol. 3, no. 2021-01-0869, pp. 1697–1701, 2021

work page 2021
[3]

Novice and young drivers and advanced driver assistant systems: A review,

F. Mansourifar, N. Nadimi, and F. Golbabaei, “Novice and young drivers and advanced driver assistant systems: A review,” Future Transportation, vol. 5, no. 1, p. 32, 2025

work page 2025
[4]

Exploratory analysis of injury severity under different levels of driving automation (sae levels 2 and 4) using multi-source data,

S. Ding, M. Abdel-Aty, N. Barbour, D. Wang, Z. Wang, and O. Zheng, “Exploratory analysis of injury severity under different levels of driving automation (sae levels 2 and 4) using multi-source data,” Accident Analysis & Prevention , vol. 206, p. 107692, 2024

work page 2024
[5]

On the road safety benefits of advanced driver assistance systems in different driving contexts,

L. Masello, G. Castignani, B. Sheehan, F. Murphy, and K. McDonnell, “On the road safety benefits of advanced driver assistance systems in different driving contexts,” Transportation research interdisciplinary perspectives, vol. 15, p. 100670, 2022

work page 2022
[6]

Regiongpt: Towards region understanding vision language model,

Q. Guo, S. De Mello, H. Yin, W. Byeon, K. C. Cheung, Y . Yu, P. Luo, and S. Liu, “Regiongpt: Towards region understanding vision language model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 13 796–13 806

2024
[7]

Minivlm: A smaller and faster vision-language model,

J. Wang, X. Hu, P. Zhang, X. Li, L. Wang, L. Zhang, J. Gao, and Z. Liu, “Minivlm: A smaller and faster vision-language model,” arXiv preprint arXiv:2012.06946, 2020

arXiv 2012
[8]

Self-adapting large visual-language models to edge devices across visual modalities,

K. Cai, Z. Duan, G. Liu, C. Fleming, and C. X. Lu, “Self-adapting large visual-language models to edge devices across visual modalities,” in European Conference on Computer Vision . Springer, 2024, pp. 301–318

work page 2024

Show all 27 references

[9]

Lora: Low-rank adaptation of large language models

E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

2022
[10]

Low-rank few-shot adaptation of vision- language models,

M. Zanella and I. Ben Ayed, “Low-rank few-shot adaptation of vision- language models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 1593–1603

2024
[11]

Lora+: Efficient low rank adaptation of large models,

S. Hayou, N. Ghosh, and B. Yu, “Lora+: Efficient low rank adaptation of large models,” arXiv preprint arXiv:2402.12354 , 2024

2024 arXiv
[12]

Parameter- efficient fine-tuning methods for pretrained language models: A critical review and assessment,

L. Xu, H. Xie, S.-Z. J. Qin, X. Tao, and F. L. Wang, “Parameter- efficient fine-tuning methods for pretrained language models: A critical review and assessment,” arXiv preprint arXiv:2312.12148 , 2023

2023 arXiv
[13]

Drivellava: Human- level behavior decisions via vision language model,

R. Zhao, Q. Yuan, J. Li, Y . Fan, Y . Li, and F. Gao, “Drivellava: Human- level behavior decisions via vision language model,” Sensors (Basel, Switzerland), vol. 24, no. 13, p. 4113, 2024

2024
[14]

Carllava: Vision lan- guage models for camera-only closed-loop driving,

K. Renz, L. Chen, A.-M. Marcu, J. H ¨unermann, B. Hanotte, A. Karn- sund, J. Shotton, E. Arani, and O. Sinavski, “Carllava: Vision lan- guage models for camera-only closed-loop driving,” arXiv preprint arXiv:2406.10165, 2024

2024 arXiv
[15]

Vision-language models for autonomous driving: Clip-based dynamic scene understanding,

M. Elhenawy, H. I. Ashqar, A. Rakotonirainy, T. I. Alhadidi, A. Jaber, and M. A. Tami, “Vision-language models for autonomous driving: Clip-based dynamic scene understanding,” Electronics, vol. 14, no. 7, p. 1282, 2025

2025
[16]

Mul- timodal instruction tuning with conditional mixture of lora,

Y . Shen, Z. Xu, Q. Wang, Y . Cheng, W. Yin, and L. Huang, “Mul- timodal instruction tuning with conditional mixture of lora,” arXiv preprint arXiv:2402.15896, 2024

2024 arXiv
[17]

When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis,

R. Zhang, B. Wang, J. Zhang, Z. Bian, C. Feng, and K. Ozbay, “When language and vision meet road safety: leveraging multimodal large language models for video-based traffic accident analysis,” arXiv preprint arXiv:2501.10604, 2025

2025 arXiv
[18]

Driving with llms: Fusing object- level vector modality for explainable autonomous driving,

L. Chen, O. Sinavski, J. H ¨unermann, A. Karnsund, A. J. Willmott, D. Birch, D. Maund, and J. Shotton, “Driving with llms: Fusing object- level vector modality for explainable autonomous driving,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, ...

2024
[19]

Automated evaluation of large vision-language models on self-driving corner cases,

K. Chen, Y . Li, W. Zhang, Y . Liu, P. Li, R. Gao, L. Hong, M. Tian, X. Zhao, Z. Li et al., “Automated evaluation of large vision-language models on self-driving corner cases,” in 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2025, pp. 7817–7826

2025
[20]

Vision language models in autonomous driving: A survey and outlook,

X. Zhou, M. Liu, E. Yurtsever, B. L. Zagar, W. Zimmer, H. Cao, and A. C. Knoll, “Vision language models in autonomous driving: A survey and outlook,” IEEE Transactions on Intelligent Vehicles , 2024

2024
[21]

A survey on multimodal large language models for autonomous driving,

C. Cui, Y . Ma, X. Cao, W. Ye, Y . Zhou, K. Liang, J. Chen, J. Lu, Z. Yang, K.-D. Liao et al. , “A survey on multimodal large language models for autonomous driving,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 958– 979

2024
[22]

Vlp: Vision language planning for autonomous driving,

C. Pan, B. Yaman, T. Nesti, A. Mallik, A. G. Allievi, S. Velipasalar, and L. Ren, “Vlp: Vision language planning for autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14 760–14 769

2024
[23]

Drivegpt4: Interpretable end-to-end autonomous driving via large language model,

Z. Xu, Y . Zhang, E. Xie, Z. Zhao, Y . Guo, K.-Y . K. Wong, Z. Li, and H. Zhao, “Drivegpt4: Interpretable end-to-end autonomous driving via large language model,” IEEE Robotics and Automation Letters , 2024

2024
[24]

Senna: Bridging large vision-language models and end-to-end autonomous driving,

B. Jiang, S. Chen, B. Liao, X. Zhang, W. Yin, Q. Zhang, C. Huang, W. Liu, and X. Wang, “Senna: Bridging large vision-language models and end-to-end autonomous driving,”arXiv preprint arXiv:2410.22313, 2024

2024 arXiv
[25]

Evaluation of safety cognition capability in vision-language models for autonomous driving,

E. Zhang, P. Gong, X. Dai, Y . Lv, and Q. Miao, “Evaluation of safety cognition capability in vision-language models for autonomous driving,” arXiv preprint arXiv:2503.06497 , 2025

2025
[26]

Towards end-to-end lane detection: an instance segmen- tation approach,

D. Neven, B. De Brabandere, S. Georgoulis, M. Proesmans, and L. Van Gool, “Towards end-to-end lane detection: an instance segmen- tation approach,” in 2018 IEEE intelligent vehicles symposium (IV) . IEEE, 2018, pp. 286–291

2018
[27]

Qwen2. 5-vl technical report,

S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang et al., “Qwen2. 5-vl technical report,” arXiv preprint arXiv:2502.13923, 2025

2025 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

OpenLKA: an open dataset of lane keeping assist from market autonomous vehicles

Y . Wang, A. Alhuraish, S. Yuan, S. Wang, and H. Zhou, “Openlka: an open dataset of lane keeping assist from market autonomous vehicles,” arXiv preprint arXiv:2501.03287 , 2025

work page Pith review arXiv 2025

[2] [2]

Effectiveness of advanced driver assistance systems in preventing system-relevant crashes,

R. Spicer, A. Vahabaghaie, D. Murakhovsky, G. Bahouth, B. Drayer, and S. S. Lawrence, “Effectiveness of advanced driver assistance systems in preventing system-relevant crashes,” SAE International Journal of Advances and Current Practices in Mobility , vol. 3, no. 2021-01-0869, pp. 1697–1701, 2021

work page 2021

[3] [3]

Novice and young drivers and advanced driver assistant systems: A review,

F. Mansourifar, N. Nadimi, and F. Golbabaei, “Novice and young drivers and advanced driver assistant systems: A review,” Future Transportation, vol. 5, no. 1, p. 32, 2025

work page 2025

[4] [4]

Exploratory analysis of injury severity under different levels of driving automation (sae levels 2 and 4) using multi-source data,

S. Ding, M. Abdel-Aty, N. Barbour, D. Wang, Z. Wang, and O. Zheng, “Exploratory analysis of injury severity under different levels of driving automation (sae levels 2 and 4) using multi-source data,” Accident Analysis & Prevention , vol. 206, p. 107692, 2024

work page 2024

[5] [5]

On the road safety benefits of advanced driver assistance systems in different driving contexts,

L. Masello, G. Castignani, B. Sheehan, F. Murphy, and K. McDonnell, “On the road safety benefits of advanced driver assistance systems in different driving contexts,” Transportation research interdisciplinary perspectives, vol. 15, p. 100670, 2022

work page 2022

[6] [6]

Regiongpt: Towards region understanding vision language model,

Q. Guo, S. De Mello, H. Yin, W. Byeon, K. C. Cheung, Y . Yu, P. Luo, and S. Liu, “Regiongpt: Towards region understanding vision language model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 13 796–13 806

2024

[7] [7]

Minivlm: A smaller and faster vision-language model,

J. Wang, X. Hu, P. Zhang, X. Li, L. Wang, L. Zhang, J. Gao, and Z. Liu, “Minivlm: A smaller and faster vision-language model,” arXiv preprint arXiv:2012.06946, 2020

arXiv 2012

[8] [8]

Self-adapting large visual-language models to edge devices across visual modalities,

K. Cai, Z. Duan, G. Liu, C. Fleming, and C. X. Lu, “Self-adapting large visual-language models to edge devices across visual modalities,” in European Conference on Computer Vision . Springer, 2024, pp. 301–318

work page 2024