Pith. sign in

REVIEW 5 major objections 6 minor 86 references

CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CRUISE claims that guiding multi-sensor fusion with vision-language-model uncertainty maps beats current methods by an average of 4.87% in 3D object detection and 4.23% in semantic segmentation.

desk verdict Genuinely new VLM-guided uncertainty fusion, but the headline gains don't survive arithmetic against the paper's own tables. read the letter →

arxiv 2608.09202 v1 pith:FD6QUV27 submitted 2026-08-10 cs.AI

Junyao Wang , Yulin Xu , Yu Li , Pramod Khargonekar , Mohammad Abdullah Al Faruque This is my paper · ORCID
classification cs.AI
keywords uncertaintyquantificationvision-languagemodelsensorfusionautonomousdrivingout-of-distributionrobustnessbird's-eyeview3Dobjectdetectionsemanticsegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CRUISE proposes that autonomous-vehicle perception becomes more robust if sensor fusion is guided by pixel-level uncertainty maps produced by a vision-language model, instead of coarse modality-level confidence scores. The paper claims this works because a VLM fine-tuned on driving data turns a raw distance-based uncertainty estimate into a context-aware heatmap that pinpoints which image, LiDAR, or radar regions are unreliable. Across three datasets under simulated fog, blur, exposure shifts, and sensor noise, CRUISE reports average gains of 4.87% in 3D object detection and 4.23% in semantic segmentation over current fusion and uncertainty quantification methods. The result matters because it suggests a practical route to keeping multi-sensor perception dependable when conditions shift away from training data.

What carries the argument

The load-bearing object is the VLM-guided uncertainty heatmap $W_m^r = \lambda M + \gamma A_{\mathrm{text}}$, where $M$ is the visual-stream localization map from AnomalyGPT's ImageBind encoder plus transformer decoder, and $A_{\mathrm{text}}$ is the attention-based uncertainty map from the Vicuna-7B textual stream. A prompt adapter converts the raw distance scores $Q_m$ and thresholded region hints $B_m$ into structured prompts that feed the textual stream. The heatmap plays two roles: it acts as a multiplicative mask in Eq. (4) to suppress unreliable spatial regions, and it feeds the dynamic adaptation weighting in Eq. (9) that adjusts whole-modality contributions using aggregated uncertainty and cross-modal variance. The feature pool, built with core sampling, supplies the raw uncertainty estimate by nearest-feature distance, while the UQ loss and modality consistency loss train representations to stay stable in high-uncertainty regions and aligned in low-uncertainty regions.

What would settle it

Compare the high-uncertainty regions of $W_m^r$ against ground-truth degradation masks for each simulated OOD perturbation, for example the motion-blurred and over-exposed regions shown in Figure 4: if the VLM heatmap does not localize the actually corrupted pixels substantially better than the raw $Q_m$ distance map, the central claim fails. A second test would replace the VLM output with $Q_m$ processed through an equally sized learned network; if the gain disappears, the VLM's contextual prior, not added capacity, is what matters.

Watch

Extended reading notes

Core claim

The paper's central claim is that fine-grained, context-aware uncertainty is the right signal for cross-modal fusion, and that a VLM can supply it. In CRUISE, each modality's bird's-eye-view feature map is compared against a pool of representative training features to get a per-location Euclidean-distance uncertainty map; AnomalyGPT, fine-tuned with LoRA, then enriches that map with local visual detail and global textual reasoning to produce a heatmap $W_m^r = \lambda M + \gamma A_{\mathrm{text}}$. Fusion multiplies each modality's features by $(1-W_m^r)$ to mute unreliable pixels, applies spatial, channel, and pixel attention, and weights each modality by an exponential function of its aggregated uncertainty plus inter-modal variance. The authors report that this combination outperforms current state-of-the-art methods by 4.87% in 3D detection and 4.23% in segmentation, and that the VLM component alone accounts for an average 3.42% gain over using the raw distance map. They also claim the UQ module is model-agnostic and can be dropped into existing fusion frameworks.

Load-bearing premise

The whole gain rests on the claim that the VLM's heatmap is genuinely context-aware uncertainty rather than a re-render of the raw distance map; if it is only a re-render, the masking and weighting could suppress useful signal.

Editorial extensions

If this is right

  • If the VLM heatmap is genuinely informative, fusion frameworks can rely on a pretrained VLM's contextual prior to localize novel degradation types instead of learning uncertainty heuristics from scratch.
  • Pixel-level masking plus modality-level weighting means the benefit compounds: corrupted regions inside a still-trusted sensor are muted, while an entire unreliable sensor is downweighted.
  • The model-agnostic claim implies that existing BEV fusion methods could adopt CRUISE's UQ module and see gains without retraining the full fusion stack.
  • The reported latency overhead of roughly 0.01 to 0.06 seconds suggests the framework could fit into near-real-time autonomous driving stacks.
  • Because the prompt adapter can be fine-tuned per modality, extending CRUISE to new sensors such as event cameras or thermal imaging only requires corresponding fine-tuning data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The evaluation uses simulated OOD perturbations rather than real sensor failures; a natural next test is whether the VLM heatmap localizes real faults as well as synthetic blur and noise.
  • The 3.42% gain attributed to the VLM could come partly from added network capacity rather than semantic context; replacing the VLM with a same-capacity non-VLM regressor on $Q_m$ would separate those factors.
  • If the heatmap is calibrated, it could be surfaced to a planner or driver-assistance system as a region-level reliability monitor, not just a fusion weight.
  • The inter-modality variance term in the weighting function should matter most when one sensor is clearly worse; when all sensors degrade together, only the VLM's prior can carry the fusion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes CRUISE, a sensor-fusion framework for autonomous driving perception that combines a vision-language-model-guided uncertainty-quantification (UQ) module with a dynamic cross-modal adaptation mechanism. The UQ module computes per-pixel distance-based uncertainty estimates, refines them with a fine-tuned AnomalyGPT VLM, and uses the resulting heatmaps to mask and weight LiDAR, camera, and radar features during fusion. Experiments on aiMotive, DeLiVER, and MFNet are reported for 3D object detection and semantic segmentation, with the abstract claiming average improvements of 4.87% and 4.23% over state-of-the-art methods, and 4.16% over state-of-the-art UQ methods. The paper also reports ablations for the VLM, the prompt adapter, the core-sampling feature pool, and the fusion components.

Significance. The core idea is timely and potentially useful: using a VLM's contextual priors to produce fine-grained, pixel-level uncertainty maps for guiding cross-modal fusion is a plausible alternative to coarse modality-level uncertainty estimates, and the dynamic adaptation mechanism addresses a real gap in fusion design. If the reported gains were reproducible, the contribution could be of interest to the autonomous-driving perception community. The paper also includes useful ablations and a model-agnostic UQ design. However, the headline numerical claims do not reproduce from the tables in the manuscript, and the central evidence for the VLM's added value is thin; the claimed significance therefore cannot currently be evaluated.

major comments (5)
  1. [Abstract / §I Contributions; Tables I and IV] The abstract and the contribution list claim average improvements of 4.87% in 3D object detection and 4.23% in semantic segmentation over SOTA methods, and 4.16% over SOTA UQ. These numbers do not follow from the tables. In Table I, CRUISE achieves 68.37/66.76 AP versus the best baseline MSMDFusion at 67.76/65.96, a gain of only 0.61/0.80 AP. In Table IV, CRUISE achieves 65.89/64.62 versus InfNoise at 63.98/63.10, a gain of 1.91/1.52 AP. No aggregation convention is stated that would reconcile these figures with 4.87%, 4.23%, or 4.16%. The authors must either correct the claims to match the tabulated results or provide the exact averaging formula and per-scenario scores used for the headline numbers.
  2. [§IV.C.1, Tables II and III] The text states that on DeLiVER CRUISE achieves 6.71% higher mIoU on the test set and 4.41% higher mIoU on the validation set over its baseline CMNeXt, and that on MFNet it achieves 2.24% higher mIoU over the second-best result. Table II shows gains of 3.31/2.40 mIoU over CMNeXt, and Table III shows gains of 1.31 mIoU over CMNeXt without UQ and 0.92 mIoU over CMNeXt+InfNoise. These discrepancies are too large to be rounding effects and must be corrected. The same issue appears in §IV.D.1: Table VI gives average gains over DPP of about 0.17 percentage points and over VQ-VAE of about 2.29 percentage points, while the text claims 0.28% and 3.67%, and the reported latency improvement of 7.6% does not match the tabulated latencies (about 6.5%).
  3. [§IV.C.2, Tables IV and V] The claimed UQ-comparison improvements are also unsupported. The text says CRUISE outperforms the second-best UQ method by 2.98% AP and 2.20% 11-point AP on aiMotive and by 4.24%/4.08% mIoU on DeLiVER. Table IV shows a mean gain of 1.91/1.52 over InfNoise, and Table V shows a mean gain of 1.89/2.10 over InfNoise. The paper should either recompute these claims or explicitly state the baseline and metric convention used; as written, the reported values do not follow from the tables.
  4. [§IV.D.2, Table VII; §III.D and Eqs. (3)-(4), (9)] The central claim that the VLM-guided heatmap W_m^r provides informative uncertainty beyond the raw distance map Q_m is not quantitatively validated. The only controlled evidence, Table VII, shows average gains of 1.77 percentage points for the VLM over the w/o VLM variant (2.33, 1.87, and 1.12 on the three datasets), not the 3.42% claimed in the text; the prompt-adapter gain is 0.48 on average, not 0.81%. In addition, W_m^r is never compared against ground-truth degradation masks or any calibration metric; Figure 4 is qualitative only. Since Eq. (4) and Eq. (9) use W_m^r to suppress and reweight features, the authors should provide a quantitative analysis (e.g., correlation with known degradation regions, or an uncertainty-calibration curve) demonstrating that W_m^r carries context-aware signal beyond Q_m.
  5. [§II.B and §IV.A] The comparisons omit several recent methods that the related-work section explicitly names, including HydraFusion, EcoFusion, uncertainty-encoded fusion, Cocoon, SAMFusion, CAFuser, and ContextualFusion. The paper also reports only single runs without standard deviations, seeds, or code, so the magnitude of the claimed improvements cannot be assessed for statistical significance. The authors should add the named baselines where feasible and provide at least mean and variance over multiple runs, or justify their absence.
minor comments (6)
  1. [§IV.D.1] The phrase "detrimental point processes" should be "determinantal point processes" (DPP); the current wording appears to be a typo.
  2. [§III.C and §IV.B] The feature-pool size N' is introduced as an important hyperparameter (N' << N), but its value is never reported in the implementation details; please state the value used for each dataset.
  3. [References] The aiMotive dataset is cited twice as references [60] and [67]; these should be consolidated into a single citation.
  4. [Table II] The table header describes the setting as "RGB-LIDAR," but the DeLiVER dataset used in the experiments includes additional modalities (event and depth sensors); please clarify which modalities were actually used in the fusion for each experiment.
  5. [§III.E, Eqs. (11)-(12)] The threshold tau is fixed at 0.5 for the UQ and consistency losses, but no sensitivity analysis or justification is provided; a short study or a statement of how tau was chosen would improve reproducibility.
  6. [§IV.B, Model Specification] For the 2D datasets DeLiVER and MFNet, the paper describes a "3D-to-2D migration" by extracting ResNet-50 features, but it is unclear how these features are aligned with the BEV-based 3D framework; please specify the projection and alignment procedure in more detail.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CRUISE's central comparisons are empirical and evaluated against external baselines; self-cited prior work is background, not load-bearing.

full rationale

I examined the paper's derivation chain for reductions to its own inputs. The uncertainty estimate begins at Eq. (2) as a Euclidean distance to a feature pool, is refined in Eq. (3) as W = lambda*M + gamma*A_text, and is then used in Eq. (4) as a mask and in Eqs. (9)-(10) as a weighting. None of these equations defines the claimed downstream performance in terms of the same quantity by construction; the masking and weighting are applications of the UQ signal, and the final numbers are empirical benchmark results. The UQ loss Eq. (11) and consistency loss Eq. (12) are training regularizers; using the heatmap to select regions for a loss does not make the heatmap's evaluation circular. The w/o VLM ablation in Table VII compares two trained variants, so it is a controlled empirical test rather than a construction. Self-citations (e.g., refs. [1], [10], [36], [46]) appear in related-work and motivation contexts, but no load-bearing claim rests on a self-cited uniqueness theorem or an unverified self-citation chain; the central performance claims are against external baselines such as BEVFusion, MSMDFusion, CMNeXt, and InfNoise. I therefore find no circular step. Separate arithmetic inconsistencies between the abstract's aggregate gains and the table entries are a reproducibility and correctness concern, not a circularity finding.

Assumptions & free parameters 8 free parameters · 5 assumptions · 2 invented entities

The framework is an empirical engineering system; the only mathematical input is standard Euclidean distance. The load-bearing assumptions are domain transfer of the VLM, calibration of the heatmap, representativeness of simulated OOD, and heuristic fusion weighting. Multiple hyperparameters are hand-set with no sensitivity analysis, and one key design parameter (pool size N') is unreported.

free parameters (8)
  • lambda (Eq. 3) = 1
    Weight for visual localization map M in the final uncertainty heatmap W = lambda*M + gamma*A_text; set by hand with no sensitivity analysis.
  • gamma_heatmap (Eq. 3) = 0.5
    Weight for textual attention map A_text in the final heatmap; set by hand.
  • nu (Eq. 9) = 1
    Sensitivity to per-modality uncertainty in dynamic weighting; set by hand.
  • gamma_weight (Eq. 9) = 0.5
    Sensitivity to inter-modality variance in dynamic weighting; note the same symbol gamma is also used in Eq. 3 for a different role.
  • alpha (Eq. 13) = 1
    Weight for L_uncertainty in the total loss; set by hand.
  • beta (Eq. 13) = 1
    Weight for L_consistency in the total loss; set by hand.
  • tau (Eq. 11 and 12) = 0.5
    Threshold for defining high-uncertainty regions in UQ and consistency losses; set by hand.
  • feature pool size N' = not specified
    Core sampling pool size is never reported, although it controls the reference distribution for uncertainty estimates.
assumptions (5)
  • domain assumption Greedy core sampling (Eq. 1) produces a feature pool that faithfully represents the training distribution, making nearest-neighbor distance in Eq. (2) a meaningful uncertainty score.
    Section III-C.1. The paper assumes farthest-point sampling preserves diversity; no coverage or error analysis is provided.
  • domain assumption AnomalyGPT, pretrained on industrial anomaly detection, transfers via LoRA to BEV feature maps of LiDAR, radar, and camera and produces context-aware pixel-level uncertainty that improves on raw distance.
    Section III-D. Core premise of the VLM-guided UQ; supported only by the ablation in Table VII, not by analysis of what the VLM learned.
  • domain assumption The final uncertainty heatmap W_m^r = lambda*M + gamma*A_text (Eq. 3) is sufficiently calibrated to guide feature masking and fusion.
    Section III-D.3 and Fig. 4. Only qualitative heatmap examples are shown; no calibration or ground-truth evaluation of the heatmap is provided.
  • domain assumption Simulated OOD perturbations (motion blur, exposure shifts, LiDAR noise and jitter) are representative of real-world adverse conditions.
    Section IV-A and IV-B.2. OOD sets are generated by perturbing validation frames; DeLiVER's built-in OOD split is the only real OOD data used.
  • ad hoc to paper The fusion weighting in Eq. (9), combining exponential uncertainty penalty and inter-modality variance, is a valid model of cross-modal dependencies.
    Section III-E.2. The functional form is heuristic; hyperparameters nu and gamma are set to 1 and 0.5 without sensitivity analysis.
invented entities (2)
  • Prompt adapter
    purpose: Converts uncertainty estimates Q_m, region hints B_m, and localization maps M into structured prompts E_prompt for the VLM textual stream.
    Only evaluated in a single ablation (Table VII, +0.81% over no-prompt-adapter variant); no external validation or theoretical grounding.
  • Dynamic adaptation mechanism
    purpose: Weighting function in Eq. (9) that combines per-modality uncertainty Q'_m with inter-modality variance to compute fusion weights for Eq. (10).
    Supported only by ablation Table VIII; the formula is heuristic with hand-set hyperparameters nu and gamma.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving." pith.science (2026). https://pith.science/paper/FD6QUV27

@misc{pith2026260809202,
  author       = {Pith},
  title        = {Pith review of: CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FD6QUV27}},
  note         = {Machine review of arXiv:2608.09202}
}
read the original abstract

Modern autonomous vehicles are equipped with multiple sensors, such as cameras, LiDAR, and radar, for comprehensive environmental perception. However, robust cross-modal feature fusion remains a critical challenge, as the reliability of each sensor varies significantly across diverse real-world driving conditions, including poor visibility and adverse weather. While uncertainty quantification (UQ) mitigates this issue by allowing models to prioritize reliable signals, existing uncertainty-aware fusion methods typically rely on simple feature-level uncertainty estimates and thus often fail to generalize effectively in complex, out-of-distribution scenarios. To address this limitation, we propose CRUISE, a novel uncertainty-aware cross-modal sensor fusion framework. CRUISE integrates a vision-language model (VLM)-guided UQ module that generates fine-grained, pixel-level uncertainty estimates. By leveraging the VLM's rich prior knowledge and superior contextual reasoning, our approach provides a highly informative guide for the fusion process. Furthermore, we introduce a dynamic adaptive mechanism that explicitly models and captures cross-modal dependencies, ensuring the framework fully exploits the inherent complementary nature of multi-sensor inputs.

Figures

Figures reproduced from arXiv: 2608.09202 by the authors.

Figure 1
Figure 1. Overview of CRUISE. A VLM-guided UQ module provides pixel￾level uncertainty maps to guide sensor fusion, leading to more robust object detection and semantic segmentation. Existing uncertainty-aware multi-modal sensor fusion typ￾ically adopt popular UQ techniques, such as Bayesian infer￾ence [11], [12], ensemble methods [13], [14], and Monte Carlo dropout, to improve robustness [15]. However, a fundamental challenge… view at source ↗
Figure 2
Figure 2. 3D architecture of our uncertainty-aware multi-modal sensor fusion framework for AV perception tasks. The framework comprises three states: feature [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Original and Simulated Out-of-Distribution Samples: The figure [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visualization of uncertainty heatmaps (W mr ) generated by the VLM￾guided UQ module for OOD samples. The figure shows uncertainty heatmaps for two camera OOD samples (motion blur and over-exposure). Brighter areas in the heatmap correspond to high-uncertainty regions, …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

86 extracted references · 63 canonical work pages

  1. [1]

    Rs2g: Data-driven scene-graph extraction and embed- ding for robust autonomous perception and scenario understanding,

    J. Wanget al., “Rs2g: Data-driven scene-graph extraction and embed- ding for robust autonomous perception and scenario understanding,” inIEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 7493–7502

    2024

  2. [2]

    Deep learning sensor fusion for autonomous vehicle perception and localization: A review,

    J. Fayyad, M. A. Jaradat, D. Gruyer, and H. Najjaran, “Deep learning sensor fusion for autonomous vehicle perception and localization: A review,”Sensors, vol. 20, no. 15, p. 4220, 2020

    2020

  3. [3]

    Emi-lidar: Uncovering vulnerabilities of lidar sensors in autonomous driving setting using electromagnetic interference,

    S. H. V . Bhupathiraju, J. Sheldon, L. A. Bauer, V . Bindschaedler, T. Sugawara, and S. Rampazzi, “Emi-lidar: Uncovering vulnerabilities of lidar sensors in autonomous driving setting using electromagnetic interference,” inProceedings of the 16th ACM Conference on Security and Privacy in Wireless and Mobile Networks, 2023, pp. 329–340

    2023

  4. [4]

    Lidar for autonomous driving: The principles, challenges, and trends for automotive lidar and perception systems,

    Y . Li and J. Ibanez-Guzman, “Lidar for autonomous driving: The principles, challenges, and trends for automotive lidar and perception systems,”IEEE Signal Processing Magazine, 2020

    2020

  5. [5]

    Futr3d: A unified sensor fusion framework for 3d detection,

    X. Chen, T. Zhang, Y . Wang, Y . Wang, and H. Zhao, “Futr3d: A unified sensor fusion framework for 3d detection,” inIEEE/CVF conference on computer vision and pattern recognition, 2023

    2023

  6. [6]

    Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges,

    D. Feng, C. Haase-Sch ¨utz, L. Rosenbaum, H. Hertlein, C. Glaeser, F. Timm, W. Wiesbeck, and K. Dietmayer, “Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges,”IEEE Transactions on Intelligent Transporta- tion Systems, vol. 22, no. 3, pp. 1341–1360, 2020

    2020

  7. [7]

    Uncertainty-encoded multi-modal fusion for robust object detection in autonomous driving,

    Y . Lou, Q. Song, Q. Xu, R. Tan, and J. Wang, “Uncertainty-encoded multi-modal fusion for robust object detection in autonomous driving,” inECAI 2023. IOS Press, 2023, pp. 1593–1600

    work page 2023

  8. [8]

    Disthd: A learner-aware dynamic encoding method for hyperdimensional classification,

    J. Wang, S. Huang, and M. Imani, “Disthd: A learner-aware dynamic encoding method for hyperdimensional classification,” in2023 60th ACM/IEEE Design Automation Conference (DAC). IEEE, 2023

    work page 2023

Show all 86 references
  1. [9]

    Squeezesegv3: Spatially-adaptive convolution for effi- cient point-cloud segmentation,

    C. Xu, B. Wu, Z. Wang, W. Zhan, P. Vajda, K. Keutzer, and M. Tomizuka, “Squeezesegv3: Spatially-adaptive convolution for effi- cient point-cloud segmentation,” inComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXVIII 16. ...

    2020

  2. [10]

    Hyperdimensional uncertainty quantification for multimodal uncer- tainty fusion in autonomous vehicles perception,

    L. Chen, J. Wang, T. Mortlock, P. Khargonekar, and M. A. Al Faruque, “Hyperdimensional uncertainty quantification for multimodal uncer- tainty fusion in autonomous vehicles perception,” in2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025, p...

    2025

  3. [11]

    Simple and scalable predictive uncertainty estimation using deep ensembles,

    B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,”Advances in neural information processing systems, vol. 30, 2017

    2017

  4. [12]

    Late breaking results: Scalable and efficient hyperdimensional computing for network intrusion detection,

    J. Wang, H. Chen, M. Issa, S. Huang, and M. Imani, “Late breaking results: Scalable and efficient hyperdimensional computing for network intrusion detection,” in2023 60th ACM/IEEE Design Automation Con- ference (DAC). IEEE, 2023, pp. 1–2

    2023

  5. [13]

    Uncertainty quan- tification using bayesian neural networks in classification: Application to biomedical image segmentation,

    Y . Kwon, J.-H. Won, B. J. Kim, and M. C. Paik, “Uncertainty quan- tification using bayesian neural networks in classification: Application to biomedical image segmentation,”Computational Statistics & Data Analysis, vol. 142, p. 106816, 2020

    2020

  6. [14]

    Domino: Domain-invariant hyperdimensional classification for multi-sensor time series data,

    J. Wang, L. Chen, and M. A. Al Faruque, “Domino: Domain-invariant hyperdimensional classification for multi-sensor time series data,” in 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD). IEEE, 2023, pp. 1–9

    2023

  7. [15]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning,

    Y . Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” ininternational conference on machine learning. PMLR, 2016, pp. 1050–1059

    2016

  8. [16]

    A decentralized sensor fusion scheme for multi sensorial fault resilient pose estimation,

    M. Mukherjeeet al., “A decentralized sensor fusion scheme for multi sensorial fault resilient pose estimation,”Sensors, 2021

    2021

  9. [17]

    A variational bayesian approach to robust sensor fusion based on student-t distribution,

    H. Zhu, H. Leung, and Z. He, “A variational bayesian approach to robust sensor fusion based on student-t distribution,”Information Sciences, vol. 221, pp. 201–214, 2013

    2013

  10. [18]

    Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,

    M. Bijelicet al., “Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

    2020

  11. [19]

    Transformer-based contrastive meta- learning for low-resource generalizable activity recognition,

    J. Wang and M. A. Al Faruque, “Transformer-based contrastive meta- learning for low-resource generalizable activity recognition,” inICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    2025

  12. [20]

    Beyond unimodal: Generalising neural processes for multimodal uncertainty estimation,

    M. C. Jung, H. Zhao, J. Dipnall, and L. Du, “Beyond unimodal: Generalising neural processes for multimodal uncertainty estimation,” Advances in Neural Information Processing Systems, 2023

    2023

  13. [21]

    Quantifying & modeling multimodal interactions: An information decomposition framework,

    P. P. Lianget al., “Quantifying & modeling multimodal interactions: An information decomposition framework,”Advances in Neural Information Processing Systems, 2023

    2023

  14. [22]

    Hyperdetect: A real-time hyperdimensional solution for intrusion de- tection in iot networks,

    J. Wang, H. Xu, Y . G. Achamyeleh, S. Huang, and M. A. Al Faruque, “Hyperdetect: A real-time hyperdimensional solution for intrusion de- tection in iot networks,”IEEE Internet of Things Journal, vol. 11, no. 8, pp. 14 844–14 856, 2023

    2023

  15. [23]

    Smore: Similarity-based hyperdimensional domain adaptation for multi-sensor time series classification,

    J. Wang and M. Al Faruque, “Smore: Similarity-based hyperdimensional domain adaptation for multi-sensor time series classification,” inPro- ceedings of the 61st ACM/IEEE Design Automation Conference, 2024, pp. 1–6

    2024

  16. [24]

    Anomalygpt: Detecting industrial anomalies using large vision-language models,

    Z. Gu, B. Zhu, G. Zhu, Y . Chen, M. Tang, and J. Wang, “Anomalygpt: Detecting industrial anomalies using large vision-language models,” arXiv preprint arXiv:2308.15366, 2023

    2023 arXiv

  17. [25]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022

    2022

  18. [26]

    A survey of uncertainty in deep neural networks,

    J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, P. Jung, R. Roscheret al., “A survey of uncertainty in deep neural networks,”Artificial Intelligence Review, 2023

    2023

  19. [27]

    Bayesian fusion: Modeling and application,

    J. Sander and J. Beyerer, “Bayesian fusion: Modeling and application,” in2013 Workshop on Sensor Data Fusion: Trends, Solutions, Applica- tions (SDF). IEEE, 2013, pp. 1–6

    2013

  20. [28]

    Choosing the best sensor fusion method: A machine- learning approach,

    R. F. Brena, A. A. Aguileta, L. A. Trejo, E. Molino-Minero-Re, and O. Mayora, “Choosing the best sensor fusion method: A machine- learning approach,”Sensors, vol. 20, no. 8, p. 2350, 2020

    2020

  21. [29]

    Uncertainty quantification and deep ensembles,

    R. Rahamanet al., “Uncertainty quantification and deep ensembles,” Advances in neural information processing systems, 2021

    2021

  22. [30]

    Trustworthy sensor fusion against inaudible command attacks in advanced driver- assistance systems,

    J. Guan, L. Pan, C. Wang, S. Yu, L. Gao, and X. Zheng, “Trustworthy sensor fusion against inaudible command attacks in advanced driver- assistance systems,”IEEE Internet of Things Journal, 2023

    2023

  23. [31]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational conference on machine learning. PmLR, 2021, pp. 8748–8763

    2021

  24. [32]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Liet al., “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” inInternational conference on machine learning. PMLR, 2023, pp. 19 730–19 742

    2023

  25. [33]

    Overcoming the pitfalls of vision-language model finetuning for ood generalization,

    Y . Zang, H. Goh, J. Susskind, and C. Huang, “Overcoming the pitfalls of vision-language model finetuning for ood generalization,”arXiv preprint arXiv:2401.15914, 2024

    2024 arXiv

  26. [34]

    Leveraging vision-language models for improving domain generalization in image classification,

    S. Addepalli, A. R. Asokan, L. Sharma, and R. V . Babu, “Leveraging vision-language models for improving domain generalization in image classification,” inProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2024, pp. 23 922–23 932. JOURNAL OF L...

    2024

  27. [35]

    Practicaldg: Perturbation distillation on vision-language models for hybrid domain generalization,

    Z. Chen, W. Wang, Z. Zhao, F. Su, A. Men, and H. Meng, “Practicaldg: Perturbation distillation on vision-language models for hybrid domain generalization,” inProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2024, pp. 23 501–23 511

    2024

  28. [36]

    Hydrafu- sion: Context-aware selective sensor fusion for robust and efficient autonomous vehicle perception,

    A. V . Malawade, T. Mortlock, and M. A. Al Faruque, “Hydrafu- sion: Context-aware selective sensor fusion for robust and efficient autonomous vehicle perception,” in2022 ACM/IEEE 13th International Conference on Cyber-Physical Systems. IEEE, 2022, pp. 68–79

    2022

  29. [37]

    Sensor fusion and object localization,

    S. Shekhar, O. Khatib, and M. Shimojo, “Sensor fusion and object localization,” inProceedings. 1986 IEEE International Conference on Robotics and Automation, vol. 3. IEEE, 1986, pp. 1623–1628

    1986

  30. [38]

    Multi-level fusion based 3d object detection from monocular images,

    B. Xu and Z. Chen, “Multi-level fusion based 3d object detection from monocular images,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2345–2353

    2018

  31. [39]

    Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding,

    Z. Huang, C. Lv, Y . Xing, and J. Wu, “Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding,”IEEE Sensors Journal, 2020

    2020

  32. [40]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,

    Z. Liuet al., “Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation,” inICRA, 2023

    2023

  33. [41]

    Transfusion: Robust lidar-camera fusion for 3d object detection with transformers,

    X. Bai, Z. Hu, X. Zhu, Q. Huang, Y . Chen, H. Fu, and C.-L. Tai, “Transfusion: Robust lidar-camera fusion for 3d object detection with transformers,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 1090–1099

    2022

  34. [42]

    The multi-modal fusion in visual question answering: a review of attention mechanisms,

    S. Lu, M. Liu, L. Yin, Z. Yin, X. Liu, and W. Zheng, “The multi-modal fusion in visual question answering: a review of attention mechanisms,” PeerJ Computer Science, vol. 9, p. e1400, 2023

    2023

  35. [43]

    Multimodal token fusion for vision transformers,

    Y . Wang, X. Chen, L. Cao, W. Huang, F. Sun, and Y . Wang, “Multimodal token fusion for vision transformers,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022

    2022

  36. [44]

    Perception-aware multi-sensor fusion for 3d lidar semantic segmentation,

    Z. Zhuang, R. Li, K. Jia, Q. Wang, Y . Li, and M. Tan, “Perception-aware multi-sensor fusion for 3d lidar semantic segmentation,” inIEEE/CVF international conference on computer vision, 2021

    2021

  37. [45]

    Sensor fusion for joint 3d object detection and semantic segmentation,

    G. P. Meyer, J. Charland, D. Hegde, A. Laddha, and C. Vallespi- Gonzalez, “Sensor fusion for joint 3d object detection and semantic segmentation,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2019

    2019

  38. [46]

    Ecofusion: Energy-aware adaptive sensor fusion for efficient autonomous vehicle perception,

    A. V . Malawade, T. Mortlock, and M. A. A. Faruque, “Ecofusion: Energy-aware adaptive sensor fusion for efficient autonomous vehicle perception,” inACM/IEEE Design Automation Conference, 2022

    2022

  39. [47]

    Cocoon: Robust multi-modal perception with uncertainty-aware sensor fusion,

    M. Choet al., “Cocoon: Robust multi-modal perception with uncertainty-aware sensor fusion,” inICLR, 2024

    2024

  40. [48]

    Samfusion: Sensor-adaptive multimodal fusion for 3d object detection in adverse weather,

    E. Palladinet al., “Samfusion: Sensor-adaptive multimodal fusion for 3d object detection in adverse weather,” inEuropean Conference on Computer Vision. Springer, 2024

    2024

  41. [49]

    Cafuser: Condition-aware multimodal fusion for robust semantic perception of driving scenes,

    T. Br ¨odermann, C. Sakaridis, Y . Fu, and L. Van Gool, “Cafuser: Condition-aware multimodal fusion for robust semantic perception of driving scenes,”IEEE Robotics and Automation Letters, 2025

    2025

  42. [50]

    Contextualfusion: Context- based multi-sensor fusion for 3d object detection in adverse operating conditions,

    S. Sural, N. Sahu, and R. R. Rajkumar, “Contextualfusion: Context- based multi-sensor fusion for 3d object detection in adverse operating conditions,” inIntelligent vehicles symposium. IEEE, 2024

    2024

  43. [51]

    V oxelnet: End-to-end learning for point cloud based 3d object detection,

    Y . Zhou and O. Tuzel, “V oxelnet: End-to-end learning for point cloud based 3d object detection,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4490–4499

    2018

  44. [52]

    Large kernel sparse convnet weighted by multi- frequency attention for remote sensing scene understanding,

    J. Wanget al., “Large kernel sparse convnet weighted by multi- frequency attention for remote sensing scene understanding,”IEEE Transactions on Geoscience and Remote Sensing, 2023

    2023

  45. [53]

    Resnet 50,

    B. Koonceet al., “Resnet 50,”Convolutional neural networks with swift for tensorflow: image recognition and dataset categorization, 2021

    2021

  46. [54]

    Pointpillars: Fast encoders for object detection from point clouds,

    A. H. Langet al., “Pointpillars: Fast encoders for object detection from point clouds,” inCVPR, 2019

    2019

  47. [55]

    Submanifold sparse convolutional networks,

    B. Graham and L. Van der Maaten, “Submanifold sparse convolutional networks,”arXiv preprint arXiv:1706.01307, 2017

    2017 arXiv

  48. [56]

    Towards total recall in industrial anomaly detection,

    K. Rothet al., “Towards total recall in industrial anomaly detection,” in CVPR, 2022

    2022

  49. [57]

    Imagebind: One embedding space to bind them all,

    R. Girdharet al., “Imagebind: One embedding space to bind them all,” inCVPR, 2023

    2023

  50. [58]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, march 2023,

    W.-L. Chiang, Z. Li, Z. Lin, Y . Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y . Zhuang, J. E. Gonzalezet al., “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, march 2023,”URL https://lmsys. org/blog/2023-03-30-vicuna, vol. 3, no. 5, 2023

    2023

  51. [59]

    Gradient-based learning applied to document recogni- tion,

    Y . LeCunet al., “Gradient-based learning applied to document recogni- tion,”Proceedings of the IEEE

  52. [60]

    aimotive dataset: A multimodal dataset for ro- bust autonomous driving with long-range perception,

    T. Matuszkaet al., “aimotive dataset: A multimodal dataset for ro- bust autonomous driving with long-range perception,”arXiv preprint arXiv:2211.09445, 2022

    2022 arXiv

  53. [61]

    Delivering arbitrary-modal semantic segmentation,

    J. Zhanget al., “Delivering arbitrary-modal semantic segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1136–1147

    2023

  54. [62]

    Mfnet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes,

    Q. Haet al., “Mfnet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes,” inInternational Con- ference on Intelligent Robots and Systems. IEEE, 2017

    2017

  55. [63]

    Focalformer3d: Focusing on hard instance for 3d object detection,

    Y . Chenet al., “Focalformer3d: Focusing on hard instance for 3d object detection,” 2023

    2023

  56. [64]

    Multimodal virtual point 3d detection,

    T. Yin, X. Zhou, and P. Kr ¨ahenb¨uhl, “Multimodal virtual point 3d detection,”Advances in Neural Information Processing Systems, 2021

    2021

  57. [65]

    Cross modal transformer via coordinates encoding for 3d object dectection,

    J. Yanet al., “Cross modal transformer via coordinates encoding for 3d object dectection,”arXiv preprint arXiv:2301.01283, 2023

    2023 arXiv

  58. [66]

    Msmdfusion: Fusing lidar and camera at multiple scales with multi-depth seeds for 3d object detection,

    Y . Jiao, Z. Jie, S. Chen, J. Chen, L. Ma, and Y .-G. Jiang, “Msmdfusion: Fusing lidar and camera at multiple scales with multi-depth seeds for 3d object detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  59. [67]

    aimotive dataset: A multimodal dataset for robust autonomous driving with long-range perception,

    T. Matuszka, “aimotive dataset: A multimodal dataset for robust autonomous driving with long-range perception,” inInternational Conference on Learning Representations 2023 Workshop on Scene Representations for Autonomous Driving, 2023. [Online]. Available: https://openreview.n...

    2023

  60. [68]

    Stitchfusion: Weaving any visual modalities to enhance multimodal semantic segmentation,

    B. Li, D. Zhang, Z. Zhao, J. Gao, and X. Li, “Stitchfusion: Weaving any visual modalities to enhance multimodal semantic segmentation,”arXiv preprint arXiv:2408.01343, 2024

    2024 arXiv

  61. [69]

    Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers,

    J. Zhang, H. Liu, K. Yang, X. Hu, R. Liu, and R. Stiefelhagen, “Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers,” IEEE Transactions on Intelligent Transportation Systems, 2023

    2023

  62. [70]

    Hrfuser: A multi-resolution sensor fusion architecture for 2d object detection,

    T. Broedermannet al., “Hrfuser: A multi-resolution sensor fusion architecture for 2d object detection,” inInternational Conference on Intelligent Transportation Systems. IEEE, 2023

    2023

  63. [71]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022

    2021

  64. [72]

    Segformer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,”Advances in neural information processing systems, vol. 34, pp. 12 077–12 090, 2021

    2021

  65. [73]

    Acnet: Attention based network to exploit complementary features for rgbd semantic segmentation,

    X. Hu, K. Yang, L. Fei, and K. Wang, “Acnet: Attention based network to exploit complementary features for rgbd semantic segmentation,” in International conference on image processing. IEEE, 2019

    2019

  66. [74]

    Fuseseg: Semantic segmentation of urban scenes based on rgb and thermal data fusion,

    Y . Sun, W. Zuo, P. Yun, H. Wang, and M. Liu, “Fuseseg: Semantic segmentation of urban scenes based on rgb and thermal data fusion,” Transactions on Automation Science and Engineering, 2020

    2020

  67. [75]

    Feanet: Feature-enhanced attention network for rgb-thermal real-time semantic segmentation,

    F. Deng, H. Feng, M. Liang, H. Wang, Y . Yang, Y . Gao, J. Chen, J. Hu, X. Guo, and T. L. Lam, “Feanet: Feature-enhanced attention network for rgb-thermal real-time semantic segmentation,” inInternational con- ference on intelligent robots and systems (IROS). IEEE, 2021

    2021

  68. [76]

    Multi-interactive feature learning and a full-time multi-modality bench- mark for image fusion and segmentation,

    J. Liu, Z. Liu, G. Wu, L. Ma, R. Liu, W. Zhong, Z. Luo, and X. Fan, “Multi-interactive feature learning and a full-time multi-modality bench- mark for image fusion and segmentation,” inInternational Conference on Computer Vision, 2023

    2023

  69. [77]

    Training-free uncertainty estimation for dense regression: Sensitivity as a surrogate,

    L. Mi, H. Wang, Y . Tian, H. He, and N. N. Shavit, “Training-free uncertainty estimation for dense regression: Sensitivity as a surrogate,” inProceedings of the AAAI Conference on Artificial Intelligence, 2022

    2022

  70. [78]

    Latent discriminant deterministic uncertainty,

    G. Franchi, X. Yu, A. Bursuc, E. Aldea, S. Dubuisson, and D. Filliat, “Latent discriminant deterministic uncertainty,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 243–260

    2022

  71. [79]

    Posterior network: Uncertainty estimation without ood samples via density-based pseudo- counts,

    B. Charpentier, D. Z ¨ugner, and S. G ¨unnemann, “Posterior network: Uncertainty estimation without ood samples via density-based pseudo- counts,”Advances in neural information processing systems, 2020

    2020

  72. [80]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,”arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  73. [81]

    Sgd: General analysis and improved rates,

    R. M. Goweret al., “Sgd: General analysis and improved rates,” in International conference on machine learning. PMLR, 2019

    2019

  74. [82]

    scikit-image: image processing in python,

    S. Van der Walt, J. L. Sch ¨onberger, J. Nunez-Iglesias, F. Boulogne, J. D. Warner, N. Yager, E. Gouillart, and T. Yu, “scikit-image: image processing in python,”PeerJ, vol. 2, p. e453, 2014

    2014

  75. [83]

    Automatic differentiation in pytorch,

    A. Paszkeet al., “Automatic differentiation in pytorch,” inNIPS-W, 2017

    2017

  76. [84]

    Neural discrete representation learning,

    A. Van Den Oord, O. Vinyalset al., “Neural discrete representation learning,”Advances in neural information processing systems, 2017

    2017

  77. [85]

    Determinantal point processes for machine learning,

    A. Kulesza, B. Taskaret al., “Determinantal point processes for machine learning,”Foundations and Trends in Machine Learning, 2012

    2012

  78. [86]

    Kmc 3: counting and manipulating k-mer statistics,

    M. Kokot, M. Długosz, and S. Deorowicz, “Kmc 3: counting and manipulating k-mer statistics,”Bioinformatics, 2017

    2017

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.