Pith. sign in

REVIEW 3 major objections 6 minor 46 references

SignEye: Traffic Sign Interpretation from Vehicle First-Person View

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Traffic signs become driving plans with 87 percent accuracy

desk verdict TSI-FPV is a sensible new task and Traffic-CN is a useful dataset, but the headline TGA accuracy measures consistency with the authors' own label generator, not traffic-law correctness. read the letter →

arxiv 2411.11507 v1 pith:UBSJ6ZKT submitted 2024-11-18 cs.CV

Chuang Yang , Xu Han , Tao Han , Yuejiao SU , Junyu Gao , Hongyuan Zhang , Yi Wang , Lap-Pui Chau This is my paper · ORCID
classification cs.CV
keywords trafficsigninterpretationfirst-personviewautonomousdrivingvision-languagemodelegocentricrelativepositionplangenerationstructureddescriptionguidanceassistant
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that traffic signs can do more than be detected: they can be read from the vehicle's own viewpoint and converted into driving-plan suggestions. To make that leap, it introduces a task, TSI-FPV, in which each sign unit is described in structured natural language and assigned to the lane or road it governs from an egocentric perspective (current, left, right, or all). It builds a dataset, Traffic-CN, with 20k+ images, 50k+ sign descriptions, and 1.2M+ plan samples, and a stepwise vision-language pipeline, SignEye. The headline result is 87.0% overall accuracy on the traffic guidance assistant (TGA) task, with the largest gains on position-sensitive lane-change and speed choices. If the claim holds, an autonomous driving system can use sign interpretations, not just obstacle perception, for regulation assessment and direction navigation, and can complement electronic map navigation when maps are offline or signs are temporarily adjusted.

What carries the argument

The load-bearing mechanism is EgoRPD (egocentric relative position definition), which labels every lane and road as L (left), C (current), R (right), or A (all) relative to the vehicle's position, and Algorithm 1, which assigns sign boxes to these labels by comparing x-axis midpoints at two-thirds image height. This is what ties a sign to the vehicle's actual lane. The other central component is SSLM, a vision-language model that follows a fixed instruction chain—recognize symbols and text, describe the sign unit in a structured format, locate its lane and road, then make a plan—and is trained with a permutation-invariant loss so that reordering keywords or destinations does not count as error. Together these pieces turn a first-person road image into a closed-form driving plan.

What would settle it

Take a held-out set of first-person road images, have licensed driving instructors independently annotate the correct road, lane, speed, and other plan for each image under the applicable traffic law, and compare SignEye's choices against those annotations; if agreement drops well below the reported 87.0%, the claim that TGA performs regulation assessment and direction navigation is not established.

Watch

Extended reading notes

Core claim

The central claim is that interpreting traffic signs from the vehicle's first-person view, with signs and lanes labeled relative to the vehicle under the EgoRPD strategy, makes a traffic guidance plan-making task achievable by a single vision-language model rather than a staged detection, classification, and natural-language pipeline. SignEye decomposes the reasoning into symbol and text extraction, structured description, lane localization, road localization, and plan-making; it assigns each sign description to egocentrically labeled lanes and roads using Algorithm 1, then combines the descriptions with vehicle attributes and a route graph to choose a road, lane, speed, and other plan. On Traffic-CN, SignEye reaches 87.0% overall accuracy and 85.6% accuracy when the 'none' option is removed, surpassing general vision-language models by at least 7.3 percentage points (9.7 points when 'none' is dropped). The paper interprets this as evidence that egocentric spatial position, not just sign semantics, is the missing ingredient in sign-based assistance for autonomous driving.

Load-bearing premise

The ground-truth driving plans were generated automatically by splitting up the sign descriptions the system itself is trained to write, then matching those pieces to vehicle attributes and a route graph; if those automatically made labels do not match what traffic law or a competent driver would actually advise, the reported 87% accuracy measures agreement with the label generator, not correctness of the driving advice.

Editorial extensions

If this is right

  • Sign-based driving assistance can be produced from sparse natural-language descriptions of sign units, without requiring dense symbol and text bounding-box annotations.
  • Because EgoRPD considers only lanes around the vehicle, it avoids recognition interference from perspective distortion of distant lanes and can support single-step lane-change decisions.
  • TGA accuracy is strongest on 'none' options; when those are removed, the egocentric position information contributes most of the remaining value, with a 9.7 percentage point gain over general vision-language models.
  • The data engine's combination of manual labeling, model-assisted automatic annotation, and manual adjustment makes building large-scale TSI-FPV and TGA datasets feasible.
  • A TGA-style system can complement electronic map navigation under offline conditions, temporary sign adjustments, and road construction, rather than replacing maps.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 87.0% figure measures agreement with rule-generated plan labels, not with independently certified driving advice; a natural next step is auditing outputs against licensed driving instructors or official traffic-law annotations.
  • The egocentric relative-position idea generalizes beyond roads: any first-person assistant, such as indoor wayfinding or delivery-robot navigation, could use L/C/R/A labeling to attach instructions to the agent's current corridor or lane.
  • Because the structured descriptions and plan options are anchored to Chinese traffic sign criteria, transferring SignEye to other countries would require replacing the description templates and plan options with local regulations.
  • The permutation-invariant loss should help any vision-language structured-description task where the order of destinations or keywords is semantically free, not just traffic signs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces TSI-FPV, a task that interprets traffic sign units from a vehicle's first-person view as structured descriptions in an egocentric relative position definition (EgoRPD), and TGA, a driving-plan application built on TSI-FPV using vehicle attributes and a route graph. The authors construct SignEye, a stepwise reasoning pipeline combining SAM, SignDet, UFLD, and a SigLIP/Qwen2-based SSLM trained with LoRA, and build the Traffic-CN dataset with 20k+ images, 50k+ descriptions, and 1.2M+ TGA samples via a semi-automatic data engine. Experiments report that SignEye outperforms general VLMs on TSI metrics (e.g., BLEU-4 56.2 vs. the best baseline 53.9) and on TGA plan accuracy (87.0% overall vs. the best baseline 79.7%, and 85.6% when the 'none' option is removed). The paper claims TGA can provide traffic regulation assessment and direction navigation as a complement to electronic map navigation.

Significance. If validated, the paper would make a useful contribution by introducing an egocentric, position-aware traffic sign interpretation task and a corresponding dataset, and by demonstrating a vision-language pipeline that can link signs to lane/road positions for downstream plan-making. Strengths include the clear framing of the TSI-FPV task, the structured description format, the stepwise instruction design, and the comparison against several strong general VLMs. The TSI description results are less affected by circularity because the description ground truths pass through a manual adjustment stage. However, the TGA and EgoRPD results are currently not sufficient to establish the central claim that SignEye performs traffic regulation assessment and direction navigation, because the evaluation labels are generated by the same rules used in inference and no external validation against traffic-law correctness is provided. The paper therefore needs a major revision that adds independent validation rather than only additional baselines.

major comments (3)
  1. [Section 4, Table 4] The TGA plan labels are generated automatically by decomposing the EgoRPD structured descriptions and matching vehicle attributes and waypoints to the resulting instructions. Because those structured descriptions are exactly what SignEye is trained to emit, the reported 87.0% overall TGA accuracy largely measures how well the model reproduces the authors' rule-based label generator, not whether the resulting plans are correct driving advice under traffic regulations. Please provide independent expert annotations of plan correctness or a traffic-law-based evaluation on a held-out set; without this, the central claim that TGA achieves regulation assessment and direction navigation is not established.
  2. [Section 3.1, Algorithm 1, Table 5] EgoRPD localization ground truth is generated in Section 4 using Algorithm 1, and the same Algorithm 1 is used in the inference path for assigning sign units to lanes and roads. The large improvements on lane-change options in Table 5 (30% and 21% relative gains) may therefore reflect the model learning the nearest-region geometric heuristic, rather than genuine spatial understanding of the scene. Please validate the lane/road assignment with manually labeled spatial relations or an independent route/lane ground truth source.
  3. [Section 5.1-5.2] No train/test split, number of runs, or variance is reported for any of the TSI or TGA results. SSLM is trained and evaluated on the same Traffic-CN corpus, and the SignDet model used for sign-region proposals is also trained on Traffic-CN. Please report a fixed split and repeated-run statistics, and ideally also evaluate with perturbed or independent region proposals, so that the headline numbers can be interpreted as generalization measurements rather than fitting to the evaluation set.
minor comments (6)
  1. [Abstract and Introduction] The TGA abbreviation is defined as 'traffic guidance assistant' in the Abstract but as 'traffic regulation assistant' in the Introduction; please standardize the terminology.
  2. [Algorithm 1] The break condition 'theta <= 90' is not explained: please specify the units, the reference axis, and why this condition separates lane segments correctly.
  3. [Table 5] The column headers in Table 5 are difficult to parse; in particular, 'Or', 'Ol', 'Os', and 'Oo' should be explicitly labeled in the table rather than only in the text.
  4. [Equation (2)] The permutation-invariant loss enumerates M! times the product of V_j! possible orderings; the paper does not discuss the computational cost or an efficient approximation when the number of sub-sentences or keywords is large.
  5. [Figure 3] Figure 3 is extremely dense and the callouts are difficult to read at the current resolution; please enlarge or split the figure into separate pipeline and architecture diagrams.
  6. [References] The 'Road Traffic Signs and Markings Criteria' mentioned in Section 3.1 is not cited; please provide a reference or a formal specification.

Circularity Check

2 steps flagged · score 6.0 of 10

TGA plan labels are generated by decomposing the very structured descriptions SignEye is trained to emit, and EgoRPD localization labels are produced by Algorithm 1, which is also the inference-time assignment rule; the headline plan accuracies therefore measure consistency with the authors' rule-based labeler, not correctness under traffic law.

  1. fitted input called prediction [Section 3.1 (Structured sign description) and Section 4 (Traffic-CN Dataset, Data engine)]
    "structured descriptions are the key to generating plenty of TGA samples to help our model learn to accomplish traffic regulation assessment and direction navigation... The plan-making data is generated by combining structured descriptions in EgoRPD from the localization task with vehicle attributes and the route graph, where the description is decomposed into smaller traffic instructions following its structure, and vehicle attributes and waypoints are matched with the instruction for generating suitable advice from options."

    The TGA 'suitable advice' labels are produced by deterministically decomposing the structured descriptions in EgoRPD, which is exactly the output representation SignEye is trained to generate in TSI-FPV, and matching the decomposed instructions to vehicle attributes and a route graph. Consequently the reported 87.0% overall plan accuracy (85.6% after removing the 'none' option) measures how faithfully SignEye reproduces the authors' rule-based labeler applied to its own description format, not whether the resulting plans are legally correct driving advice or correct navigation.

  2. self definitional [Section 3.1 (Structured description in EgoRPD), Section 4 (Data engine), Section 5.2 (Main Results)]
    "The trainable ground truth data for this assignment stage is generated via our data engine based on Algorithm 1... SignEye determines the current lane by analyzing the geometry of lane lines and connects the egocentric view with the corresponding sign unit based on their spatial relationship (see Algorithm 1)."

    The same geometric EgoRPD algorithm (Algorithm 1) both generates the lane/road localization ground truth in Traffic-CN and is used in SignEye's inference path to determine the current lane and assign sign units to it. The large lane-change improvements in Table 5 (e.g., +30.0% on lane option #3 and +21.5% on #4 with EgoRPD) therefore may simply reflect the model learning to imitate the geometric heuristic on labels produced by that same heuristic. The comparison does not establish independent egocentric spatial reasoning; the evaluation target is defined by the method being tested.

full rationale

The TSI-FPV description task itself has independent visual grounding: the structured descriptions are derived from the Road Traffic Signs and Markings Criteria, the images are real first-person driving scenes, and the data engine's final labels pass through a manual adjustment stage, so the description results in Table 3 are not circular by construction. However, the paper's headline downstream claim, that TGA is achievable as traffic regulation assessment and direction navigation, rests on plan labels that are auto-generated by decomposing the same structured description representation SignEye is trained to produce and matching it to vehicle attributes and a route graph. The reported plan accuracies therefore certify consistency with the authors' rule-based labeler, not correctness under traffic law or real navigation, and no independent expert validation of the plans is provided. Similarly, the EgoRPD localization ground truth is generated by Algorithm 1, which is also referenced in the inference path, so the large position-sensitive gains are at least partly a self-consistency artifact. These two steps are load-bearing for the conclusion that TGA complements ADS, making the circularity partial rather than complete; the central TSI-FPV description results retain independent content. Score 6 reflects one or more predictions that reduce by construction to their own label-generation rules.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The central evaluation rests on a fixed set of structured description templates, a hand-coded geometric assignment rule (Algorithm 1), a rule-based plan label generator, and pretrained detection and vision-language components. These are needed to produce Traffic-CN labels and to run SignEye; none of them are independently validated against an external standard, which raises the circularity burden described in the red flags.

free parameters (5)
  • EgoRPD y-coordinate for lane assignment = 2H/3
    Algorithm 1 uses x-midpoint distance at y = 2/3 image height to assign sign boxes to left, current, or right lanes; this value is chosen by hand without sensitivity analysis.
  • Lane-line angle break condition = theta <= 90 degrees
    Algorithm 1 splits lane-line points into left, current, and right using an arctangent and polyfit angle threshold; no justification or ablation is provided.
  • LoRA rank = 64
    Section 5.1 sets rank r = 64 for low-rank adaptation; no sweep or sensitivity study is reported.
  • Image partition grid = 2 x 3 slices at 1344 x 1344 pixels
    Section 3.2 divides the input into six slices to handle high resolution; this fixed grid affects which sign features are visible, with no ablation.
  • Smoother hidden dimension M = not reported
    Section 3.2 reduces SigLIP feature dimension from D = 1096 to M << D to fit on 24 GB GPUs, but the actual M is not stated, so the memory and performance trade-off is not reproducible.
assumptions (5)
  • domain assumption Structured description templates in Table 1 fully cover the semantic content of Chinese Road Traffic Signs and Markings Criteria
    SSLM generates descriptions according to these templates, and the TGA label generator decomposes them into instructions; no independent audit against the standard is provided.
  • ad hoc to paper Algorithm 1's nearest-region rule correctly encodes which signs apply to the ego vehicle
    The same Algorithm 1 generates EgoRPD localization ground truth in the data engine and is referenced in the inference path, so systematic rule errors are baked into both labels and predictions.
  • ad hoc to paper Rule-based plan labels from structured descriptions, vehicle attributes, and route graph equal correct driving advice
    Section 4 states plan-making data is generated automatically by decomposing structured descriptions and matching attributes and waypoints; TGA accuracy is therefore a consistency check with this generator.
  • domain assumption Pretrained components (SigLIP, Qwen2, SAM, SignDet, UFLD) provide reliable image, text, and region representations
    SignDet is trained on Traffic-CN itself, and upstream detector errors are not separated from SignEye's mistakes in the reported end-to-end accuracy.
  • domain assumption Permutation-invariant likelihood over all keyword orderings defines the correct description equivalence
    Equations (1) and (2) assume every permutation of sub-sentences and keywords is an equally valid answer and uses this to define the training loss.
invented entities (1)
  • EgoRPD (egocentric relative position definition)
    purpose: Labels roads and lanes as left, current, or right relative to the ego vehicle and assigns sign units to them for TSI-FPV and TGA.
    The only support is the paper's own ablation on the auto-generated Traffic-CN; no external benchmark or independent human-annotated positions validate the assignment rule.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SignEye: Traffic Sign Interpretation from Vehicle First-Person View." pith.science (2026). https://pith.science/paper/UBSJ6ZKT

@misc{pith2026241111507,
  author       = {Pith},
  title        = {Pith review of: SignEye: Traffic Sign Interpretation from Vehicle First-Person View},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UBSJ6ZKT}},
  note         = {Machine review of arXiv:2411.11507}
}
read the original abstract

Traffic signs play a key role in assisting autonomous driving systems (ADS) by enabling the assessment of vehicle behavior in compliance with traffic regulations and providing navigation instructions. However, current works are limited to basic sign understanding without considering the egocentric vehicle's spatial position, which fails to support further regulation assessment and direction navigation. Following the above issues, we introduce a new task: traffic sign interpretation from the vehicle's first-person view, referred to as TSI-FPV. Meanwhile, we develop a traffic guidance assistant (TGA) scenario application to re-explore the role of traffic signs in ADS as a complement to popular autonomous technologies (such as obstacle perception). Notably, TGA is not a replacement for electronic map navigation; rather, TGA can be an automatic tool for updating it and complementing it in situations such as offline conditions or temporary sign adjustments. Lastly, a spatial and semantic logic-aware stepwise reasoning pipeline (SignEye) is constructed to achieve the TSI-FPV and TGA, and an application-specific dataset (Traffic-CN) is built. Experiments show that TSI-FPV and TGA are achievable via our SignEye trained on Traffic-CN. The results also demonstrate that the TGA can provide complementary information to ADS beyond existing popular autonomous technologies.

Figures

Figures reproduced from arXiv: 2411.11507 by the authors.

Figure 1
Figure 1. Illustration of the SignEye, where the TSI-FPV part in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. TGA estimates the structured sign descriptions in EgoRPD the vehicle attribute and route graph for achieving traffic regulation [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overall pipeline of SignEye. It takes a road image from the vehicle’s first-person view, and object (sign, lane, and road) regions [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Workflow of the data engine for building the Traffic-CN. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Visualization of the egocentric vehicle corresponding description in the TSI-FPV task. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 19 canonical work pages

  1. [6]

    Learning to understand traffic signs

    Yunfei Guo, Wei Feng, Fei Yin, Tao Xue, Shuqi Mei, and Cheng-Lin Liu. Learning to understand traffic signs. In Pro- ceedings of the 29th ACM International Conference on Mul- timedia, pages 2076–2084, 2021. 1, 3, 4

    work page 2021

  2. [7]

    Visual traffic knowledge graph generation from scene images

    Yunfei Guo, Fei Yin, Xiao-hui Li, Xudong Yan, Tao Xue, Shuqi Mei, and Cheng-Lin Liu. Visual traffic knowledge graph generation from scene images. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 21604–21613, 2023. 1, 3, 4, 5

    work page 2023

  3. [39]

    Traffic sign interpretation via natural language description

    Chuang Yang, Kai Zhuang, Mulin Chen, Haozhao Ma, Xu Han, Tao Han, Changxing Guo, Han Han, Bingxuan Zhao, and Qi Wang. Traffic sign interpretation via natural language description. IEEE Transactions on Intelligent Transportation Systems, 2024. 1, 3, 4, 5

    work page 2024

  4. [1]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966, 2023. 3

    arXiv 2023

  5. [2]

    Introducing our multimodal models, 2023

    Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, and Sa ˘gnak Tas ¸ırlar. Introducing our multimodal models, 2023. 3

    2023

  6. [3]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhang- wei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821, 2024. 3

    arXiv 2024

  7. [4]

    Road traffic sign de- tection and classification

    Arturo De La Escalera, Luis E Moreno, Miguel Angel Salichs, and Jos ´e Mar ´ıa Armingol. Road traffic sign de- tection and classification. IEEE transactions on industrial electronics, 44(6):848–859, 1997. 1

    work page 1997

  8. [5]

    Regiongpt: Towards region understanding vision lan- guage model

    Qiushan Guo, Shalini De Mello, Hongxu Yin, Wonmin Byeon, Ka Chun Cheung, Yizhou Yu, Ping Luo, and Sifei Liu. Regiongpt: Towards region understanding vision lan- guage model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13796– 13806, 2024. 3

    work page 2024

Show all 46 references
  1. [8]

    Sign- parser: An end-to-end framework for traffic sign understand- ing

    Yunfei Guo, Wei Feng, Fei Yin, and Cheng-Lin Liu. Sign- parser: An end-to-end framework for traffic sign understand- ing. International Journal of Computer Vision, 132(3):805– 821, 2024. 3, 5

    2024

  2. [9]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 3

    2017

  3. [10]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 6

    2021 arXiv

  4. [11]

    Selfocc: Self-supervised vision-based 3d oc- cupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Borui Zhang, Jie Zhou, and Jiwen Lu. Selfocc: Self-supervised vision-based 3d oc- cupancy prediction. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 19946–19956, 2024. 4

    2024

  5. [12]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 4

    2023

  6. [13]

    Geochat: Grounded large vision-language model for remote sensing

    Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, and Fahad Shahbaz Khan. Geochat: Grounded large vision-language model for remote sensing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27831– 2784...

    2024

  7. [14]

    Otterhd: A high-resolution multi- modality model

    Bo Li, Peiyuan Zhang, Jingkang Yang, Yuanhan Zhang, Fanyi Pu, and Ziwei Liu. Otterhd: A high-resolution multi- modality model. arXiv preprint arXiv:2311.04219, 2023. 3

    2023 arXiv

  8. [15]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 3

    2022

  9. [16]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 3

    2023

  10. [17]

    Mon- key: Image resolution and text label are important things for large multi-modal models

    Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xiang Bai. Mon- key: Image resolution and text label are important things for large multi-modal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  11. [18]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 3

    2024

  12. [19]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024. 3

    2024

  13. [20]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 7

    2017 arXiv

  14. [21]

    Deepseek-vl: towards real-world vision- language understanding

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Yaofeng Sun, et al. Deepseek-vl: towards real-world vision- language understanding. arXiv preprint arXiv:2403.05525,

    arXiv

  15. [22]

    Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction

    Qihang Ma, Xin Tan, Yanyun Qu, Lizhuang Ma, Zhizhong Zhang, and Yuan Xie. Cotr: Compact occupancy transformer for vision-based 3d occupancy prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19936–19945, 2024. 4

    2024

  16. [23]

    Road-sign detection and recognition based on support vector machines.IEEE transactions on intelligent transportation systems, 8(2):264–278, 2007

    Saturnino Maldonado-Basc ´on, Sergio Lafuente-Arroyo, Pe- dro Gil-Jimenez, Hilario G ´omez-Moreno, and Francisco L´opez-Ferreras. Road-sign detection and recognition based on support vector machines.IEEE transactions on intelligent transportation systems, 8(2):264–278, 2007

    2007

  17. [24]

    Ultra fast structure- aware deep lane detection

    Zequn Qin, Huanyu Wang, and Xi Li. Ultra fast structure- aware deep lane detection. In The European Conference on Computer Vision (ECCV), 2020. 4

    2020

  18. [25]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  19. [26]

    You only look once: Unified, real-time object de- tection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016. 3

    2016

  20. [27]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information process- ing systems, 28, 2015. 3

    2015

  21. [28]

    Deep learning for large-scale traffic-sign detection and recognition

    Domen Tabernik and Danijel Sko ˇcaj. Deep learning for large-scale traffic-sign detection and recognition. IEEE transactions on intelligent transportation systems , 21(4): 1427–1440, 2019. 3

    2019

  22. [29]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving. Advances in Neural Information Processing Systems, 36, 2024. 4

    2024

  23. [30]

    https://github.com/TuSimple/tusimple- benchmark

    Tusimple. https://github.com/TuSimple/tusimple- benchmark. 4

  24. [31]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  25. [32]

    All you need is boundary: Toward arbitrary-shaped text spotting

    Hao Wang, Pu Lu, Hui Zhang, Mingkun Yang, Xiang Bai, Yongchao Xu, Mengchao He, Yongpan Wang, and Wenyu Liu. All you need is boundary: Toward arbitrary-shaped text spotting. In Proceedings of the AAAI conference on artificial intelligence, pages 12160–12167, 2020

    2020

  26. [33]

    The all-seeing project: Towards panop- tic visual recognition and understanding of the open world

    Weiyun Wang, Min Shi, Qingyun Li, Wenhai Wang, Zhen- hang Huang, Linjie Xing, Zhe Chen, Hao Li, Xizhou Zhu, Zhiguo Cao, et al. The all-seeing project: Towards panop- tic visual recognition and understanding of the open world. arXiv preprint arXiv:2308.01907, 2023. 3

    2023 arXiv

  27. [34]

    The all-seeing project v2: Towards general relation comprehension of the open world

    Weiyun Wang, Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang, Qingyun Li, Lewei Lu, Xizhou Zhu, et al. The all-seeing project v2: Towards general relation comprehension of the open world. arXiv preprint arXiv:2402.19474, 2024. 3

    2024 arXiv

  28. [35]

    Chain-of-thought prompting elicits reasoning in large lan- guage models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  29. [36]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jian- wei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzhe...

    2024 arXiv

  30. [37]

    Cm-net: Concentric mask based arbitrary-shaped text detection

    Chuang Yang, Mulin Chen, Zhitong Xiong, Yuan Yuan, and Qi Wang. Cm-net: Concentric mask based arbitrary-shaped text detection. IEEE Transactions on Image Processing, 31: 2864–2877, 2022. 3

    2022

  31. [38]

    Text growing on leaf

    Chuang Yang, Mulin Chen, Yuan Yuan, and Qi Wang. Text growing on leaf. IEEE Transactions on Multimedia , 25: 9029–9043, 2023. 3

    2023

  32. [40]

    Towards real-time traffic sign detection and classification

    Yi Yang, Hengliang Luo, Huarong Xu, and Fuchao Wu. Towards real-time traffic sign detection and classification. IEEE Transactions on Intelligent transportation systems, 17 (7):2022–2031, 2015. 3

    2022

  33. [41]

    Minicpm-v: A gpt-4v level mllm on your phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800, 2024. 3

    2024 arXiv

  34. [42]

    Deepsolo: Let transformer decoder with explicit points solo for text spot- ting

    Maoyuan Ye, Jing Zhang, Shanshan Zhao, Juhua Liu, Tongliang Liu, Bo Du, and Dacheng Tao. Deepsolo: Let transformer decoder with explicit points solo for text spot- ting. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 19348–19357,

  35. [43]

    Yi: Open foundation models by 01

    Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, et al. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.04652, 2024. 3

    2024 arXiv

  36. [44]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11975–11986, 2023. 5, 6

    2023

  37. [45]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 3

    2023 arXiv

  38. [46]

    Traffic-sign detection and classifica- tion in the wild

    Zhe Zhu, Dun Liang, Songhai Zhang, Xiaolei Huang, Baoli Li, and Shimin Hu. Traffic-sign detection and classifica- tion in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2110–2118,

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.