Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training-time modality ranking lets one segmentation backbone handle any sensor subset, with large gains on MUSES and DELIVER.

desk verdict The missing test-time fusion rule makes the headline numbers irreproducible, but the training-time robustness idea is worth a revision. read the letter →

arxiv 2412.16876 v1 pith:3NKXT2IT submitted 2024-12-22 cs.CV

Xu Zheng , Yuanhuiyi Lyu , Lutao Jiang , Jiazhou Zhou , Lin Wang , Xuming Hu This is my paper · ORCID
classification cs.CV
keywords SemanticSegmentationMulti-modalLearningModality-agnosticModalitySelectionSensorFailureRobustnessMulti-scaleFeatureFusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that semantic segmentation can be made resilient to arbitrary combinations of available sensors by training a single backbone that learns, at each feature scale, which modalities are reliable and which are fragile. It proposes MAGIC++, adding a multi-modal interaction module and a multi-scale arbitrary-modal selection module during training, and reports state-of-the-art results in the standard fully-combined setting alongside large margins in the modality-agnostic setting: +2.19% mean mIoU on MUSES and +7.25% on DELIVER across all modality subsets. A sympathetic reader would care because real autonomous systems lose sensors or face degraded conditions, and a model that treats every sensor as potentially valuable rather than RGB-centric would be safer in those situations.

What carries the argument

The load-bearing mechanism is the Multi-scale Arbitrary-modal Selection Module (MASM), which at each transformer-block scale computes the cosine similarity of each modality's feature to their mean feature, ranks them, and forwards the top-ranked (robust) and bottom-ranked (fragile) features to the Multi-modal Interaction Module (MIM). MIM then rectifies these selected features with channel-wise and spatial-wise attention and produces a fused feature that also serves as a surrogate for aligning the remaining modality features through a consistency loss. This forces the encoder to learn from both reliable and degraded inputs, which the paper claims is what transfers modality-agnostic robustness to the plain backbone at inference.

What would settle it

Train the same backbone on DELIVER with identical batches and losses but replace the similarity-based selection in MASM with a random modality choice; if random selection reproduces the reported mean mIoU (47.74 for the four-modality training), the ranking is not the cause. The minimal observation is the exact inference rule: the paper should state how predictions from available sensors are combined, since the results cannot be attributed without it.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that training a hierarchical backbone with a cross-modal similarity ranking over each scale's features—merging the most robust and most fragile modalities—makes the backbone itself modality-agnostic, so that at test time it can consume whatever sensors are available without retraining or extra fusion modules. The reported evidence is that MAGIC++ outperforms previous arbitrary-modality methods in mean mIoU across all possible modality combinations, with the largest gains appearing precisely in the sparse-sensor cases (e.g., Event-only from 0.52 to 19.03 on DELIVER and Frame/Event gains on MUSES). The modules are used only during training; inference runs on the backbone alone.

Load-bearing premise

The paper says the interaction and selection modules are used only in training and that inference is achieved by the backbone, but it never specifies how the backbone processes an arbitrary subset of modalities at test time—if the rule is simple averaging of per-modality predictions, the gains may be an ensemble effect rather than the new selection mechanism.

Editorial extensions

If this is right

  • A vehicle or robot equipped with several sensors could be trained once on all of them, then continue segmenting accurately when some sensors fail or are unavailable, without per-scenario retraining.
  • Because the auxiliary modules are discarded at test time, deployment can run the same lightweight backbone as a unimodal model while keeping the robustness gains.
  • The approach transfers across at least three hierarchical backbones (SegFormer, PVTv2, Swin), suggesting the training-time selection recipe is not tied to one encoder.
  • Adding the cross-modal consistency loss improves mean mIoU by a small but consistent margin on DELIVER (47.10 to 47.74), including on all-sensor input.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not state how the backbone combines outputs from multiple sensors at inference; if that rule is prediction averaging, part of the margin over prior methods may be an ensemble effect that those methods could also exploit—an editor-level concern, not a claim of the paper.
  • The ranking-by-similarity idea is a generic training-time regularizer and could be tested on other dense tasks such as depth estimation or object detection that must cope with missing sensors.
  • A direct way to check the mechanism is to replace the similarity ranking with random selection during training; if performance is unchanged, the specific ranking signal is not the source of the gains.
  • The reported single-modality gains suggest the backbone learns to extract usable signal from weak modalities; probing feature quality during training could tell whether that is learned or merely inherited from the fusion losses.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. MAGIC++ proposes a training-time framework for modality-agnostic semantic segmentation, built on a shared backbone such as SegFormer, with two auxiliary modules: a Multi-modal Interaction Module (MIM) and a Multi-scale Arbitrary-modal Selection Module (MASM). During training, MIM fuses features across modalities, and MASM ranks per-scale modality features by cosine similarity to their mean, then supervises the backbone with losses from the top/fragile features plus a consistency loss. The paper reports MaSS evaluations on MUSES and DELIVER and claims state-of-the-art results in the common multi-modal setting and large gains in the arbitrary-modality setting (+2.19% and +7.25% mean mIoU vs. MAGIC). The central contribution, as stated, is a single backbone that at inference is robust to any available subset of modalities, without using MIM/MASM.

Significance. If fully supported, the paper would address a real and timely problem: training a single segmentation model that behaves gracefully under arbitrary sensor availability. The idea of using training-time auxiliary losses to make a backbone robust to missing modalities is potentially useful, and the paper explores multiple backbones and two datasets. The paper also gives clear credit to previous work and extends a prior ECCV paper with explicit incremental contributions. However, the significance is severely undercut by three load-bearing problems: (1) the test-time inference rule for an arbitrary modality subset is never defined, making the reported numbers unverifiable; (2) the abstract's claim of state-of-the-art performance in the common multi-modal setting is contradicted by the paper's own full-modality columns; and (3) the main comparison is against the authors' own prior MAGIC, with only one external modality-agnostic baseline (Any2Seg, also by the same group), so the external validation is thin.

major comments (4)
  1. [Section III-C (Training paragraph), Section III-A/B] The test-time inference procedure is unspecified, and this omission is load-bearing for every result in Tables I-IV. The paper states: "The MIM and MASM is only utilized in training while the inference is achieved by the backbone model, i.e., SegFormer." At test time, given a subset such as {R,D}, the paper does not say whether each available modality is passed independently through the shared backbone and the resulting softmax maps are averaged, whether features from multiple modalities are averaged before the segmentation head, whether one modality is selected, or whether the subset is concatenated and fed as a single multi-channel input. None of these rules is described or justified, and each yields different predictions. Since MIM and MASM are removed at inference, the "hierarchical modality selection" named in the title and abstract is not actually performed at test time. Without an explicit inference rule, the reported modality-agnostic results are not reproducible, and the gains over a simple ensemble of per-modality backbones cannot be assessed. This is not a presentation issue; it directly undermines the central claim.
  2. [Abstract, Section I, Table I, Table II, Table III, Section VI.A] The claim that "Under the common multi-modal setting, our method achieves state-of-the-art performance" is contradicted by the full-modality columns in the paper's own tables. On MUSES (Table I), with all three modalities (FEL), MAGIC++ obtains 40.58 mIoU, whereas Any2Seg reaches 50.00 and MAGIC reaches 49.02. On DELIVER (Table II), with all four modalities (RDEL), MAGIC++ obtains 61.67 compared to MAGIC's 63.40. In Table III, the full-combination RDL scores are 61.80 (MAGIC++) vs. 62.64 (MAGIC), and RDE scores are 61.83 vs. 62.49. Section VI.A explicitly acknowledges that "MAGIC++ under-performs in some controlled multi-modal evaluation settings." The abstract and introduction therefore overstate the results; the gains are captured in the mean over all arbitrary combinations, not in the standard full-modality setting. The authors should revise the central claim to be consistent with their own data.
  3. [Table IV, Swin-tiny RDL row] The Swin-tiny RDL row for MAGIC++ lists RD=61.39, RL=49.59, DL=53.97, RDL=61.80, which are exactly identical to the corresponding values for the SegFormer-B0 backbone in Table III (RDL block). Since the backbone is different, identical mIoU values to two decimal places for these multi-modality combinations are implausible and strongly suggest a copy-paste error. If these numbers are incorrect, the cross-backbone generalization claims in Section IV.B.4 are unsupported; the authors should verify and correct this row or explain why the values coincide.
  4. [Section III-C3, Eq. (4)] The definition of the consistency loss is unclear. The scalars c1 and c2 are defined with features f^1_rm and f^2_rm, but the summation runs over K-1, and it is not specified whether the index is over semantic classes, spatial locations, or feature scales. In addition, Eq. (5) introduces β as the only trade-off weight, while the text mentions both λ and β; λ is never defined. This ambiguity prevents the reader from understanding how the loss is computed and how the hyper-parameters are set in the ablations of Table V.
minor comments (6)
  1. [Section III-C1] The text still refers to the previous version's "MAM" and to a semantic feature "fse derived from the MAM"; the new module is MIM, and the notation should be updated for consistency.
  2. [Eq. (5)] The sentence mentions "λ and β are hyper-parameters," but λ does not appear in Eq. (5). Use only β or define λ.
  3. [Tables I and II headers] The "Training" column is not self-explanatory; for example, CMNeXt shows "FEL" under Training while MAGIC++ shows "-". Please clarify what training configuration is used for each method.
  4. [Figure 4 and Figure 5 captions] Typo: "propsoed" should be "proposed" in both figure captions.
  5. [Footnote 1] "DELVIER" should be "DELIVER".
  6. [Section V.B] In the sentence "all the features are resized for better visualization," a period is missing after "visualization".

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the central claims are empirical benchmark results with external baselines, though the training-only MIM/MASM leaves the inference protocol underspecified.

full rationale

The central claim (state-of-the-art MaSS on MUSES and DELIVER) is an empirical result evaluated on public benchmarks against external baselines including Any2Seg and CMNeXt, so it does not reduce to its input definitions. Equation (2)'s ranking uses a mean-feature anchor and selects top/bottom features by construction, but this is a training heuristic, not a predicted quantity; it is self-referential in spirit but not circular in the sense of fitting a target and then reporting it as a prediction. The main self-citation, MAGIC (ECCV 2024), is the predecessor used as a baseline and as a starting point; the actual selection in this paper replaces MAGIC's trainable MAM with a parameter-free mean anchor, so the argument does not rest on unverified self-cited results. The paper's own sentence in Section III-C ('The MIM and MASM is only utilized in training while the inference is achieved by the backbone model, i.e., SegFormer') is an explicit limitation: the inference-time rule for an arbitrary subset of modalities is not specified, so the +2.19%/+7.25% gains are not fully derived from the described architecture. That is a completeness and reproducibility gap, not a circular derivation, and it is weighed here as a correctness risk rather than as evidence of circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical entities are introduced. The method relies on standard deep learning assumptions plus three task-specific assumptions listed above. The only explicit hyperparameters are the consistency loss weight and the rank selection count.

free parameters (2)
  • consistency loss weight beta = not reported
    Total loss in Eq. (5) is L = LM + beta * LC, but the paper never states the value of beta or any sensitivity analysis over it.
  • rank selection k (top-1, bottom-1) = 1
    MASM always selects exactly one most robust and one most fragile feature at each scale; no ablation varies this choice.
assumptions (3)
  • ad hoc to paper Cosine similarity between a modality feature and the mean feature indicates that modality's robustness for segmentation at each scale.
    MASM ranks features by Cos(., fm) in Eq. (2) and treats top-1 and bottom-1 as robust and fragile; no evidence or ablation supports this proxy.
  • domain assumption Scene semantics are consistent across modalities, so the consistency loss LC can align remaining features through the MIM output.
    Eq. (4) assumes the remaining modality features should have similar cosine correlations to fmim; this may not hold for sensors capturing different physical cues.
  • ad hoc to paper The backbone alone, without MIM or MASM, is sufficient for arbitrary-modal inference after training with the auxiliary modules.
    The paper asserts MIM and MASM are training-only, but provides no ablation showing inference behavior is unchanged when they are removed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection." pith.science (2026). https://pith.science/paper/3NKXT2IT

@misc{pith2026241216876,
  author       = {Pith},
  title        = {Pith review of: MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3NKXT2IT}},
  note         = {Machine review of arXiv:2412.16876}
}
read the original abstract

In this paper, we address the challenging modality-agnostic semantic segmentation (MaSS), aiming at centering the value of every modality at every feature granularity. Training with all available visual modalities and effectively fusing an arbitrary combination of them is essential for robust multi-modal fusion in semantic segmentation, especially in real-world scenarios, yet remains less explored to date. Existing approaches often place RGB at the center, treating other modalities as secondary, resulting in an asymmetric architecture. However, RGB alone can be limiting in scenarios like nighttime, where modalities such as event data excel. Therefore, a resilient fusion model must dynamically adapt to each modality's strengths while compensating for weaker inputs.To this end, we introduce the MAGIC++ framework, which comprises two key plug-and-play modules for effective multi-modal fusion and hierarchical modality selection that can be equipped with various backbone models. Firstly, we introduce a multi-modal interaction module to efficiently process features from the input multi-modal batches and extract complementary scene information with channel-wise and spatial-wise guidance. On top, a unified multi-scale arbitrary-modal selection module is proposed to utilize the aggregated features as the benchmark to rank the multi-modal features based on the similarity scores at hierarchical feature spaces. This way, our method can eliminate the dependence on RGB modality at every feature granularity and better overcome sensor failures and environmental noises while ensuring the segmentation performance. Under the common multi-modal setting, our method achieves state-of-the-art performance on both real-world and synthetic benchmarks. Moreover, our method is superior in the novel modality-agnostic setting, where it outperforms prior arts by a large margin.

Figures

Figures reproduced from arXiv: 2412.16876 by the authors.

Figure 1
Figure 1. (a) MAGIC++ framework with multi-scale arbitrary modality selection [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall framework of MAGIC++ framework, incorporates plug-and-play multi-modal interaction module (MIM) and multi-scale arbitrary-modal [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the proposed plug-and-play multi-modal aggregation and multi-scale arbitrary-modal selection modules. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative results of arbitrary inputs evaluation with CMNeXt [ [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results of arbitrary inputs evaluation with CMNeXt [ [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Visualization of multi-scale multi-modal features and the fused MAGIC++ features. The scales correspond to: (1) [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: t-SNE visualization of multi-modal features and the fused MAGIC++ features. Each column corresponds to a specific modality: RGB, Depth, Event, [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BiXFormer: A Robust Framework for Maximizing Modality Effectiveness in Multi-Modal Semantic Segmentation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A multi-modal semantic segmentation framework that processes RGB and non-RGB sensors separately, matches labels in two stages, and aligns cross-modal queries with a VAE refiner.

  2. Partial CLIP is Enough: Chimera-Seg for Zero-shot Semantic Segmentation

    cs.CV 2025-06 conditional novelty 5.0 of 10

    A partial, frozen CLIP block mounted on a segmentation backbone, plus selective distillation to CLIP's CLS token, improves zero-shot semantic segmentation by about 1 hIoU point on two datasets.

  3. EGFormer: Towards Efficient and Generalizable Multimodal Semantic Segmentation

    cs.CV 2025-05 conditional novelty 4.0 of 10

    EGFormer dynamically scores and drops the least useful sensor modality at each processing stage, cutting parameters by up to 91 percent and GFLOPs by half while keeping segmentation accuracy competitive.

Reference graph

Works this paper leans on

107 extracted references · 74 canonical work pages · cited by 3 Pith papers

  1. [1]

    Centering the value of every modality: Towards efficient and resilient modality-agnostic semantic segmentation,

    X. Zheng, Y . Lyu, J. Zhou, and L. Wang, “Centering the value of every modality: Towards efficient and resilient modality-agnostic semantic segmentation,” in ECCV, pp. 192–212, Springer, 2025. 1, 2, 4, 5, 6, 7, 8

    2025

  2. [2]

    Multimodal sensors and ml-based data fusion for advanced robots,

    S. Duan, Q. Shi, and J. Wu, “Multimodal sensors and ml-based data fusion for advanced robots,” Advanced Intelligent Systems , vol. 4, no. 12, p. 2200213, 2022. 1

    2022

  3. [3]

    Recent advancements in multimodal human–robot interaction,

    H. Su, W. Qi, J. Chen, C. Yang, J. Sandoval, and M. A. Laribi, “Recent advancements in multimodal human–robot interaction,” Frontiers in Neurorobotics, vol. 17, p. 1084000, 2023. 1

    2023

  4. [4]

    Eventbind: Learning a unified representation to bind them all for event-based open-world understanding,

    J. Zhou, X. Zheng, Y . Lyu, and L. Wang, “Eventbind: Learning a unified representation to bind them all for event-based open-world understanding,” 2024. 1

    2024

  5. [5]

    Multi-modal 3d object detection in autonomous driving: a survey,

    Y . Wang, Q. Mao, H. Zhu, J. Deng, Y . Zhang, J. Ji, H. Li, and Y . Zhang, “Multi-modal 3d object detection in autonomous driving: a survey,” IJCV, pp. 1–31, 2023. 1

    2023

  6. [6]

    Ev-segnet: Semantic segmentation for event-based cameras,

    I. Alonso and A. C. Murillo, “Ev-segnet: Semantic segmentation for event-based cameras,” in Proceedings of IEEE/CVF CVPR Workshops, pp. 0–0, 2019. 1, 3

    2019

  7. [7]

    Event-based semantic segmentation with posterior attention,

    Z. Jia, K. You, W. He, Y . Tian, Y . Feng, Y . Wang, X. Jia, Y . Lou, J. Zhang, G. Li, et al. , “Event-based semantic segmentation with posterior attention,” IEEE TIP, vol. 32, pp. 1829–1842, 2023. 1

    2023

  8. [8]

    Customize segment anything model for multi-modal semantic segmentation with mixture of lora experts,

    C. Zhu, B. Xiao, L. Shi, S. Xu, and X. Zheng, “Customize segment anything model for multi-modal semantic segmentation with mixture of lora experts,” arXiv:2412.04220, 2024. 1

    arXiv 2024

Show all 107 references
  1. [9]

    Learning modality-agnostic representation for semantic segmentation from any modalities,

    X. Zheng, Y . Lyu, and L. Wang, “Learning modality-agnostic representation for semantic segmentation from any modalities,” arXiv:2407.11351, 2024. 1, 6, 7

    2024 arXiv

  2. [10]

    Survey on deep multi-modal data analytics: Collaboration, rivalry, and fusion,

    Y . Wang, “Survey on deep multi-modal data analytics: Collaboration, rivalry, and fusion,” ACM TOMM, vol. 17, no. 1s, pp. 1–25, 2021. 1

    2021

  3. [11]

    Deep learning for event-based vision: A comprehensive survey and benchmarks,

    X. Zheng, Y . Liu, Y . Lu, T. Hua, T. Pan, W. Zhang, D. Tao, and L. Wang, “Deep learning for event-based vision: A comprehensive survey and benchmarks,” arXiv:2302.08890, 2023. 1

    2023 arXiv

  4. [12]

    Delivering arbitrary-modal semantic segmenta- tion,

    J. Zhang, R. Liu, H. Shi, K. Yang, S. Reiß, K. Peng, H. Fu, K. Wang, and R. Stiefelhagen, “Delivering arbitrary-modal semantic segmenta- tion,” in Proceedings of IEEE/CVF CVPR , pp. 1136–1147, 2023. 1, 2, 3, 4, 5, 6, 7, 8, 10

    2023

  5. [13]

    Muses: The multi-sensor semantic perception dataset for driving under uncertainty,

    T. Br ¨odermann, D. Bruggemann, C. Sakaridis, K. Ta, O. Liagouris, J. Corkill, and L. Van Gool, “Muses: The multi-sensor semantic perception dataset for driving under uncertainty,” in ECCV, pp. 21– 38, Springer, 2025. 1, 2, 5, 6

    2025

  6. [14]

    Improving rgb-d salient object detection via modality-aware decoder,

    M. Song, W. Song, G. Yang, and C. Chen, “Improving rgb-d salient object detection via modality-aware decoder,” IEEE TIP , vol. 31, pp. 6124–6138, 2022. 1, 3

    2022

  7. [15]

    Mseg3d: Multi-modal 3d semantic segmentation for autonomous driving,

    J. Li, H. Dai, H. Han, and Y . Ding, “Mseg3d: Multi-modal 3d semantic segmentation for autonomous driving,” in Proceedings of IEEE/CVF CVPR, pp. 21694–21704, 2023. 1, 3

    2023

  8. [16]

    Issafe: Improving seman- tic segmentation in accidents by fusing event-based data,

    J. Zhang, K. Yang, and R. Stiefelhagen, “Issafe: Improving seman- tic segmentation in accidents by fusing event-based data,” in 2021 IEEE/RSJ IROS, pp. 1132–1139, IEEE, 2021. 1, 3

    2021

  9. [17]

    Bridging search region interaction with template for rgb-t tracking,

    T. Hui, Z. Xun, F. Peng, J. Huang, X. Wei, X. Wei, J. Dai, J. Han, and S. Liu, “Bridging search region interaction with template for rgb-t tracking,” in Proceedings of IEEE/CVF CVPR, pp. 13630–13639, 2023. 1, 3

    2023

  10. [18]

    Visual prompt multi- modal tracking,

    J. Zhu, S. Lai, X. Chen, D. Wang, and H. Lu, “Visual prompt multi- modal tracking,” in Proceedings of IEEE/CVF CVPR , pp. 9516–9526,

  11. [19]

    Cmx: Cross-modal fusion for rgb-x semantic segmentation with transform- ers,

    J. Zhang, H. Liu, K. Yang, X. Hu, R. Liu, and R. Stiefelhagen, “Cmx: Cross-modal fusion for rgb-x semantic segmentation with transform- ers,” arXiv:2203.04838, 2022. 1, 3

    2022 arXiv

  12. [20]

    Segformer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,” NeurIPS, vol. 34, pp. 12077–12090, 2021. 2, 4, 6

    2021

  13. [21]

    Swin transformer v2: Scaling up capacity and resolution,

    Z. Liu, H. Hu, Y . Lin, Z. Yao, Z. Xie, Y . Wei, J. Ning, Y . Cao, Z. Zhang, L. Dong, et al., “Swin transformer v2: Scaling up capacity and resolution,” in Proceedings of IEEE/CVF CVPR, pp. 12009–12019,

  14. [22]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,

    W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,” in Proceedings of the IEEE/CVF ICCV, pp. 568–578, 2021. 2

    2021

  15. [23]

    Distilling efficient vision transformers from cnns for semantic segmentation,

    X. Zheng, Y . Luo, P. Zhou, and L. Wang, “Distilling efficient vision transformers from cnns for semantic segmentation,” arXiv:2310.07265,

    arXiv

  16. [24]

    Semantics distortion and style matter: Towards source-free uda for panoramic segmentation,

    X. Zheng, P. Zhou, A. V . Vasilakos, and L. Wang, “Semantics distortion and style matter: Towards source-free uda for panoramic segmentation,” in Proceedings of IEEE/CVF CVPR , pp. 27885–27895, 2024. 2

    2024

  17. [25]

    Frozen is better than learning: A new design of prototype-based classifier for semantic segmentation,

    J. Chen, D. Deguchi, C. Zhang, X. Zheng, and H. Murase, “Frozen is better than learning: A new design of prototype-based classifier for semantic segmentation,” Pattern Recognition, vol. 152, p. 110431,

  18. [26]

    Clip is also a good teacher: A new learning framework for inductive zero-shot semantic segmentation,

    J. Chen, D. Deguchi, C. Zhang, X. Zheng, and H. Murase, “Clip is also a good teacher: A new learning framework for inductive zero-shot semantic segmentation,” arXiv:2310.02296, 2023. 2

    2023 arXiv

  19. [27]

    Both style and distortion matter: Dual-path unsupervised domain adaptation for panoramic semantic segmentation,

    X. Zheng, J. Zhu, Y . Liu, Z. Cao, C. Fu, and L. Wang, “Both style and distortion matter: Dual-path unsupervised domain adaptation for panoramic semantic segmentation,” in Proceedings of IEEE/CVF CVPR, pp. 1285–1295, 2023. 2

    2023

  20. [28]

    Look at the neigh- bor: Distortion-aware unsupervised domain adaptation for panoramic semantic segmentation,

    X. Zheng, T. Pan, Y . Luo, and L. Wang, “Look at the neigh- bor: Distortion-aware unsupervised domain adaptation for panoramic semantic segmentation,” in Proceedings of the IEEE/CVF ICCV , pp. 18687–18698, 2023. 2

    2023

  21. [29]

    A good student is cooperative and reliable: Cnn-transformer collaborative learning for semantic segmentation,

    J. Zhu, Y . Luo, X. Zheng, H. Wang, and L. Wang, “A good student is cooperative and reliable: Cnn-transformer collaborative learning for semantic segmentation,” in Proceedings of the IEEE/CVF ICCV , pp. 11720–11730, 2023. 2

    2023

  22. [30]

    Transformer-cnn cohort: Semi-supervised semantic segmentation by the best of both students,

    X. Zheng, Y . Luo, H. Wang, C. Fu, and L. Wang, “Transformer-cnn cohort: Semi-supervised semantic segmentation by the best of both students,” arXiv:2209.02178, 2022. 2

    2022 arXiv

  23. [31]

    Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges,

    D. Feng, C. Haase-Sch ¨utz, L. Rosenbaum, H. Hertlein, C. Glaeser, F. Timm, W. Wiesbeck, and K. Dietmayer, “Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges,” IEEE TITS, vol. 22, no. 3, pp. 1341–1360,

  24. [32]

    A comparative study of real-time semantic segmentation for autonomous driving,

    M. Siam, M. Gamal, M. Abdel-Razek, S. Yogamani, M. Jagersand, and H. Zhang, “A comparative study of real-time semantic segmentation for autonomous driving,” in Proceedings of the IEEE conference on CVPR workshops, pp. 587–597, 2018. 2

    2018

  25. [33]

    Vision-based semantic segmentation in scene understanding for autonomous driving: Recent achievements, challenges, and outlooks,

    K. Muhammad, T. Hussain, H. Ullah, J. Del Ser, M. Rezaei, N. Kumar, M. Hijji, P. Bellavista, and V . H. C. de Albuquerque, “Vision-based semantic segmentation in scene understanding for autonomous driving: Recent achievements, challenges, and outlooks,” IEEE TITS, 2022. 2

    2022

  26. [34]

    Sfnet-n: An improved sfnet algorithm for semantic segmentation of low-light autonomous driving road scenes,

    H. Wang, Y . Chen, Y . Cai, L. Chen, Y . Li, M. A. Sotelo, and Z. Li, “Sfnet-n: An improved sfnet algorithm for semantic segmentation of low-light autonomous driving road scenes,” IEEE TITS, vol. 23, no. 11, pp. 21405–21417, 2022. 2

    2022

  27. [35]

    Self-distillation for robust lidar semantic segmentation in autonomous driving,

    J. Li, H. Dai, and Y . Ding, “Self-distillation for robust lidar semantic segmentation in autonomous driving,” in ECCV, pp. 659–676, Springer,

  28. [36]

    Baseg: Boundary aware semantic segmentation for autonomous driv- ing,

    X. Xiao, Y . Zhao, F. Zhang, B. Luo, L. Yu, B. Chen, and C. Yang, “Baseg: Boundary aware semantic segmentation for autonomous driv- ing,” Neural Networks, vol. 157, pp. 460–470, 2023. 2

    2023

  29. [37]

    Feddrive: Generalizing federated learning to semantic segmentation in autonomous driving,

    L. Fantauzzo, E. Fan `ı, D. Caldarola, A. Tavera, F. Cermelli, M. Ci- ccone, and B. Caputo, “Feddrive: Generalizing federated learning to semantic segmentation in autonomous driving,” in 2022 IEEE/RSJ IROS, pp. 11504–11511, IEEE, 2022. 2 JOURNAL OF LATEX CLASS FILES, VOL. 14, ...

    2022

  30. [38]

    Evalu- ating the robustness of semantic segmentation for autonomous driving against real-world adversarial patch attacks,

    F. Nesti, G. Rossolini, S. Nair, A. Biondi, and G. Buttazzo, “Evalu- ating the robustness of semantic segmentation for autonomous driving against real-world adversarial patch attacks,” in Proceedings of the IEEE/CVF WACV, pp. 2280–2289, 2022. 2

    2022

  31. [39]

    Cenet: Toward concise and efficient lidar semantic segmentation for autonomous driving,

    H.-X. Cheng, X.-F. Han, and G.-Q. Xiao, “Cenet: Toward concise and efficient lidar semantic segmentation for autonomous driving,” in 2022 IEEE ICME, pp. 01–06, IEEE, 2022. 2

    2022

  32. [40]

    Fully convolutional networks for semantic segmentation,

    J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on CVPR, pp. 3431–3440, 2015. 2

    2015

  33. [41]

    Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,

    L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE TPAMI, vol. 40, no. 4, pp. 834–848, 2017. 2

    2017

  34. [42]

    Encoder-decoder with atrous separable convolution for semantic im- age segmentation,

    L.-C. Chen, Y . Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic im- age segmentation,” in Proceedings of the ECCV (ECCV), pp. 801–818,

  35. [43]

    Strip pooling: Rethink- ing spatial pooling for scene parsing,

    Q. Hou, L. Zhang, M.-M. Cheng, and J. Feng, “Strip pooling: Rethink- ing spatial pooling for scene parsing,” in Proceedings of IEEE/CVF CVPR, pp. 4003–4012, 2020. 2

    2020

  36. [44]

    Pyramid scene parsing network,

    H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in Proceedings of the IEEE conference on CVPR , pp. 2881– 2890, 2017. 2

    2017

  37. [45]

    Cars can’t fly up in the sky: Improving urban-scene segmentation via height-driven attention networks,

    S. Choi, J. T. Kim, and J. Choo, “Cars can’t fly up in the sky: Improving urban-scene segmentation via height-driven attention networks,” in Proceedings of IEEE/CVF CVPR , pp. 9373–9383, 2020. 2

    2020

  38. [46]

    Dual attention network for scene segmentation,

    J. Fu, J. Liu, H. Tian, Y . Li, Y . Bao, Z. Fang, and H. Lu, “Dual attention network for scene segmentation,” in Proceedings of IEEE/CVF CVPR , pp. 3146–3154, 2019. 2

    2019

  39. [47]

    Ccnet: Criss-cross attention for semantic segmentation,

    Z. Huang, X. Wang, L. Huang, C. Huang, Y . Wei, and W. Liu, “Ccnet: Criss-cross attention for semantic segmentation,” in Proceedings of the IEEE/CVF ICCV, pp. 603–612, 2019. 2

    2019

  40. [48]

    Ocnet: Object context for semantic segmentation,

    Y . Yuan, L. Huang, J. Guo, C. Zhang, X. Chen, and J. Wang, “Ocnet: Object context for semantic segmentation,” IJCV, vol. 129, no. 8, pp. 2375–2398, 2021. 2

    2021

  41. [49]

    Inverseform: A loss function for structured boundary-aware segmentation,

    S. Borse, Y . Wang, Y . Zhang, and F. Porikli, “Inverseform: A loss function for structured boundary-aware segmentation,” in Proceedings of IEEE/CVF CVPR , pp. 5901–5911, 2021. 2

    2021

  42. [50]

    Boundary-aware feature propagation for scene segmentation,

    H. Ding, X. Jiang, A. Q. Liu, N. M. Thalmann, and G. Wang, “Boundary-aware feature propagation for scene segmentation,” in Pro- ceedings of the IEEE/CVF ICCV , pp. 6819–6829, 2019. 2

    2019

  43. [51]

    Boundary- aware geometric encoding for semantic segmentation of point clouds,

    J. Gong, J. Xu, X. Tan, J. Zhou, Y . Qu, Y . Xie, and L. Ma, “Boundary- aware geometric encoding for semantic segmentation of point clouds,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, pp. 1424–1432, 2021. 2

    2021

  44. [52]

    Improving semantic segmentation via decoupled body and edge supervision,

    X. Li, X. Li, L. Zhang, G. Cheng, J. Shi, Z. Lin, S. Tan, and Y . Tong, “Improving semantic segmentation via decoupled body and edge supervision,” in ECCV , Proceedings, Part XVII 16, pp. 435–452, Springer, 2020. 2

    2020

  45. [53]

    Gated-scnn: Gated shape cnns for semantic segmentation,

    T. Takikawa, D. Acuna, V . Jampani, and S. Fidler, “Gated-scnn: Gated shape cnns for semantic segmentation,” in Proceedings of the IEEE/CVF ICCV, pp. 5229–5238, 2019. 2

    2019

  46. [54]

    Acnet: Attention based network to exploit complementary features for rgbd semantic segmentation,

    X. Hu, K. Yang, L. Fei, and K. Wang, “Acnet: Attention based network to exploit complementary features for rgbd semantic segmentation,” in 2019 IEEE ICIP , pp. 1440–1444, IEEE, 2019. 2

    2019

  47. [55]

    Refinenet: Multi-path refinement networks for high-resolution semantic segmentation,

    G. Lin, A. Milan, C. Shen, and I. Reid, “Refinenet: Multi-path refinement networks for high-resolution semantic segmentation,” in Proceedings of the IEEE CVPR , pp. 1925–1934, 2017. 2

    1925

  48. [56]

    Context prior for scene segmentation,

    C. Yu, J. Wang, C. Gao, G. Yu, C. Shen, and N. Sang, “Context prior for scene segmentation,” in Proceedings of IEEE/CVF CVPR , pp. 12416– 12425, 2020. 2

    2020

  49. [57]

    Context encoding for semantic segmentation,

    H. Zhang, K. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi, and A. Agrawal, “Context encoding for semantic segmentation,” in Pro- ceedings of the IEEE conference on CVPR , pp. 7151–7160, 2018. 2

    2018

  50. [58]

    Segmenter: Trans- former for semantic segmentation,

    R. Strudel, R. Garcia, I. Laptev, and C. Schmid, “Segmenter: Trans- former for semantic segmentation,” in Proceedings of the IEEE/CVF ICCV, pp. 7262–7272, 2021. 2

    2021

  51. [59]

    Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,

    S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y . Wang, Y . Fu, J. Feng, T. Xiang, P. H. Torr, et al. , “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in Proceedings of IEEE/CVF CVPR , pp. 6881–6890, 2021. 2

    2021

  52. [60]

    Multi-scale high-resolution vision transformer for semantic segmentation,

    J. Gu, H. Kwon, D. Wang, W. Ye, M. Li, Y .-H. Chen, L. Lai, V . Chan- dra, and D. Z. Pan, “Multi-scale high-resolution vision transformer for semantic segmentation,” in Proceedings of IEEE/CVF CVPR , pp. 12094–12103, 2022. 2

    2022

  53. [61]

    Topformer: Token pyramid transformer for mobile semantic segmentation,

    W. Zhang, Z. Huang, G. Luo, T. Chen, X. Wang, W. Liu, G. Yu, and C. Shen, “Topformer: Token pyramid transformer for mobile semantic segmentation,” in Proceedings of IEEE/CVF CVPR , pp. 12083–12093,

  54. [62]

    A unified efficient pyramid transformer for semantic segmentation,

    F. Zhu, Y . Zhu, L. Zhang, C. Wu, Y . Fu, and M. Li, “A unified efficient pyramid transformer for semantic segmentation,” in Proceedings of the IEEE/CVF ICCV, pp. 2667–2677, 2021. 2

    2021

  55. [63]

    Rtformer: Efficient design for real-time semantic segmentation with transformer,

    J. Wang, C. Gou, Q. Wu, H. Feng, J. Han, E. Ding, and J. Wang, “Rtformer: Efficient design for real-time semantic segmentation with transformer,” NeurIPS, vol. 35, pp. 7423–7436, 2022. 2

    2022

  56. [64]

    Multi- class token transformer for weakly supervised semantic segmentation,

    L. Xu, W. Ouyang, M. Bennamoun, F. Boussaid, and D. Xu, “Multi- class token transformer for weakly supervised semantic segmentation,” in Proceedings of IEEE/CVF CVPR , pp. 4310–4319, 2022. 2

    2022

  57. [65]

    Segvit: Semantic segmentation with plain vision transformers,

    B. Zhang, Z. Tian, Q. Tang, X. Chu, X. Wei, C. Shen, et al., “Segvit: Semantic segmentation with plain vision transformers,” NeurIPS, vol. 35, pp. 4971–4982, 2022. 2

    2022

  58. [66]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF ICCV , pp. 10012– 10022, 2021. 2

    2021

  59. [67]

    Pvt v2: Improved baselines with pyramid vision transformer,

    W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “Pvt v2: Improved baselines with pyramid vision transformer,” Computational Visual Media, vol. 8, no. 3, pp. 415–424,

  60. [68]

    Omnibind: Teach to build unequal-scale modality interaction for omni-bind of all,

    Y . Lyu, X. Zheng, D. Kim, and L. Wang, “Omnibind: Teach to build unequal-scale modality interaction for omni-bind of all,” arXiv:2405.16108, 2024. 3

    2024 arXiv

  61. [69]

    Unibind: Llm-augmented unified and balanced representation space to bind them all,

    Y . Lyu, X. Zheng, J. Zhou, and L. Wang, “Unibind: Llm-augmented unified and balanced representation space to bind them all,” in Pro- ceedings of IEEE/CVF CVPR , pp. 26752–26762, 2024. 3

    2024

  62. [70]

    Image anything: Towards reasoning-coherent and training-free multi-modal image generation,

    Y . Lyu, X. Zheng, and L. Wang, “Image anything: Towards reasoning-coherent and training-free multi-modal image generation,” arXiv:2401.17664, 2024. 3

    2024 arXiv

  63. [71]

    Learning deep multimodal fea- ture representation with asymmetric multi-layer fusion,

    Y . Wang, F. Sun, M. Lu, and A. Yao, “Learning deep multimodal fea- ture representation with asymmetric multi-layer fusion,” in Proceedings of the 28th ACM MM , pp. 3902–3910, 2020. 3

    2020

  64. [72]

    Rgb-d co-attention network for semantic segmentation,

    H. Zhou, L. Qi, Z. Wan, H. Huang, and X. Yang, “Rgb-d co-attention network for semantic segmentation,” in Proceedings of the ACCV ,

  65. [73]

    Deep mul- timodal fusion by channel exchanging,

    Y . Wang, W. Huang, F. Sun, T. Xu, Y . Rong, and J. Huang, “Deep mul- timodal fusion by channel exchanging,” NeurIPS, vol. 33, pp. 4835– 4845, 2020. 3

    2020

  66. [74]

    Shapeconv: Shape-aware convolutional layer for indoor rgb-d seman- tic segmentation,

    J. Cao, H. Leng, D. Lischinski, D. Cohen-Or, C. Tu, and Y . Li, “Shapeconv: Shape-aware convolutional layer for indoor rgb-d seman- tic segmentation,” in Proceedings of the IEEE/CVF ICCV , pp. 7088– 7097, 2021. 3

    2021

  67. [75]

    Spatial information guided convolution for real-time rgbd semantic segmenta- tion,

    L.-Z. Chen, Z. Lin, Z. Wang, Y .-L. Yang, and M.-M. Cheng, “Spatial information guided convolution for real-time rgbd semantic segmenta- tion,” IEEE TIP, vol. 30, pp. 2313–2324, 2021. 3

    2021

  68. [76]

    Uctnet: Uncertainty-aware cross-modal transformer network for indoor rgb-d semantic segmentation,

    X. Ying and M. C. Chuah, “Uctnet: Uncertainty-aware cross-modal transformer network for indoor rgb-d semantic segmentation,” in ECCV, pp. 20–37, Springer, 2022. 3

    2022

  69. [77]

    Spsn: Superpixel prototype sam- pling network for rgb-d salient object detection,

    M. Lee, C. Park, S. Cho, and S. Lee, “Spsn: Superpixel prototype sam- pling network for rgb-d salient object detection,” in ECCV, pp. 630– 647, Springer, 2022. 3

    2022

  70. [78]

    Cir-net: Cross-modality interaction and refinement for rgb-d salient object detection,

    R. Cong, Q. Lin, C. Zhang, C. Li, X. Cao, Q. Huang, and Y . Zhao, “Cir-net: Cross-modality interaction and refinement for rgb-d salient object detection,” IEEE TIP, vol. 31, pp. 6800–6815, 2022. 3

    2022

  71. [79]

    Dmra: Depth-induced multi-scale recurrent attention network for rgb- d saliency detection,

    W. Ji, G. Yan, J. Li, Y . Piao, S. Yao, M. Zhang, L. Cheng, and H. Lu, “Dmra: Depth-induced multi-scale recurrent attention network for rgb- d saliency detection,” IEEE TIP, vol. 31, pp. 2321–2336, 2022. 3

    2022

  72. [80]

    Learning discriminative cross- modality features for rgb-d saliency detection,

    F. Wang, J. Pan, S. Xu, and J. Tang, “Learning discriminative cross- modality features for rgb-d saliency detection,” IEEE TIP , vol. 31, pp. 1285–1297, 2022. 3

    2022

  73. [81]

    Pst900: Rgb-thermal calibration, dataset and segmenta- tion network,

    S. S. Shivakumar, N. Rodrigues, A. Zhou, I. D. Miller, V . Kumar, and C. J. Taylor, “Pst900: Rgb-thermal calibration, dataset and segmenta- tion network,” in 2020 IEEE ICRA , pp. 9441–9447, IEEE, 2020. 3

    2020

  74. [82]

    Abm- drnet: Adaptive-weighted bi-directional modality difference reduction network for rgb-t semantic segmentation,

    Q. Zhang, S. Zhao, Y . Luo, D. Zhang, N. Huang, and J. Han, “Abm- drnet: Adaptive-weighted bi-directional modality difference reduction network for rgb-t semantic segmentation,” in Proceedings of IEEE/CVF CVPR, pp. 2633–2642, 2021. 3

    2021

  75. [83]

    Complementarity-aware cross-modal feature fusion network for rgb-t semantic segmentation,

    W. Wu, T. Chu, and Q. Liu, “Complementarity-aware cross-modal feature fusion network for rgb-t semantic segmentation,” Pattern Recognition, vol. 131, p. 108881, 2022. 3

    2022

  76. [84]

    Cross-collaborative fusion-encoder network for robust rgb-thermal salient object detection,

    G. Liao, W. Gao, G. Li, J. Wang, and S. Kwong, “Cross-collaborative fusion-encoder network for robust rgb-thermal salient object detection,” IEEE TCSVT, vol. 32, no. 11, pp. 7646–7661, 2022. 3 JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 14

    2022

  77. [85]

    Mmsmcnet: Modal memory sharing and morphological complementary networks for rgb-t urban scene semantic segmentation,

    W. Zhou, H. Zhang, W. Yan, and W. Lin, “Mmsmcnet: Modal memory sharing and morphological complementary networks for rgb-t urban scene semantic segmentation,” IEEE TCSVT, 2023. 3

    2023

  78. [86]

    Cross-modality double bidirectional interaction and fusion network for rgb-t salient object detection,

    Z. Xie, F. Shao, G. Chen, H. Chen, Q. Jiang, X. Meng, and Y .-S. Ho, “Cross-modality double bidirectional interaction and fusion network for rgb-t salient object detection,” IEEE TCSVT, 2023. 3

    2023

  79. [87]

    Modality-induced transfer-fusion network for rgb-d and rgb-t salient object detection,

    G. Chen, F. Shao, X. Chai, H. Chen, Q. Jiang, X. Meng, and Y .-S. Ho, “Modality-induced transfer-fusion network for rgb-d and rgb-t salient object detection,” IEEE TCSVT, vol. 33, no. 4, pp. 1787–1801, 2022. 3

    2022

  80. [88]

    Caver: Cross-modal view- mixed transformer for bi-modal salient object detection,

    Y . Pang, X. Zhao, L. Zhang, and H. Lu, “Caver: Cross-modal view- mixed transformer for bi-modal salient object detection,” IEEE TIP , vol. 32, pp. 892–904, 2023. 3

    2023

  81. [89]

    Efficient rgb-t tracking via cross-modality distillation,

    T. Zhang, H. Guo, Q. Jiao, Q. Zhang, and J. Han, “Efficient rgb-t tracking via cross-modality distillation,” in Proceedings of IEEE/CVF CVPR, pp. 5404–5413, 2023. 3

    2023

  82. [90]

    Glass segmentation using intensity and spectral polarization cues,

    H. Mei, B. Dong, W. Dong, J. Yang, S.-H. Baek, F. Heide, P. Peers, X. Wei, and X. Yang, “Glass segmentation using intensity and spectral polarization cues,” in Proceedings of IEEE/CVF CVPR , pp. 12622– 12631, 2022. 3

    2022

  83. [91]

    Polarization-driven semantic segmentation via efficient attention-bridged fusion,

    K. Xiang, K. Yang, and K. Wang, “Polarization-driven semantic segmentation via efficient attention-bridged fusion,” Optics Express , vol. 29, no. 4, pp. 4802–4820, 2021. 3

    2021

  84. [92]

    Eventdance: Unsupervised source-free cross- modal adaptation for event-based object recognition,

    X. Zheng and L. Wang, “Eventdance: Unsupervised source-free cross- modal adaptation for event-based object recognition,” in Proceedings of IEEE/CVF CVPR , pp. 17448–17458, 2024. 3

    2024

  85. [93]

    Chasing day and night: Towards robust and efficient all-day object detection guided by an event camera,

    J. Cao, X. Zheng, Y . Lyu, J. Wang, R. Xu, and L. Wang, “Chasing day and night: Towards robust and efficient all-day object detection guided by an event camera,” arXiv:2309.09297, 2023. 3

    2023 arXiv

  86. [94]

    Exact: Language-guided conceptual reasoning and uncertainty estimation for event-based action recognition and more,

    J. Zhou, X. Zheng, Y . Lyu, and L. Wang, “Exact: Language-guided conceptual reasoning and uncertainty estimation for event-based action recognition and more,” in Proceedings of IEEE/CVF CVPR, pp. 18633– 18643, 2024. 3

    2024

  87. [95]

    Perception- aware multi-sensor fusion for 3d lidar semantic segmentation,

    Z. Zhuang, R. Li, K. Jia, Q. Wang, Y . Li, and M. Tan, “Perception- aware multi-sensor fusion for 3d lidar semantic segmentation,” in Proceedings of the IEEE/CVF ICCV , pp. 16280–16290, 2021. 3

    2021

  88. [96]

    2dpass: 2d priors assisted semantic segmentation on lidar point clouds,

    X. Yan, J. Gao, C. Zheng, C. Zheng, R. Zhang, S. Cui, and Z. Li, “2dpass: 2d priors assisted semantic segmentation on lidar point clouds,” in ECCV, pp. 677–695, Springer, 2022. 3

    2022

  89. [97]

    Multimodal token fusion for vision transformers,

    Y . Wang, X. Chen, L. Cao, W. Huang, F. Sun, and Y . Wang, “Multimodal token fusion for vision transformers,” in Proceedings of IEEE/CVF CVPR, pp. 12186–12195, 2022. 3

    2022

  90. [98]

    Deepfusion: Lidar-camera deep fusion for multi-modal 3d object detection,

    Y . Li, A. W. Yu, T. Meng, B. Caine, J. Ngiam, D. Peng, J. Shen, Y . Lu, D. Zhou, Q. V . Le, et al., “Deepfusion: Lidar-camera deep fusion for multi-modal 3d object detection,” in Proceedings of IEEE/CVF CVPR, pp. 17182–17191, 2022. 3

    2022

  91. [99]

    X-align: Cross-modal cross-view alignment for bird’s-eye-view segmentation,

    S. Borse, M. Klingner, V . R. Kumar, H. Cai, A. Almuzairee, S. Yo- gamani, and F. Porikli, “X-align: Cross-modal cross-view alignment for bird’s-eye-view segmentation,” in Proceedings of the IEEE/CVF WACV, pp. 3287–3297, 2023. 3

    2023

  92. [100]

    Mx2m: Masked cross-modality modeling in domain adaptation for 3d semantic segmentation,

    B. Zhang, Z. Wang, Y . Ling, Y . Guan, S. Zhang, and W. Li, “Mx2m: Masked cross-modality modeling in domain adaptation for 3d semantic segmentation,” in Proceedings of the AAAI , vol. 37, pp. 3401–3409,

  93. [101]

    Camliflow: Bidirectional camera-lidar fusion for joint optical flow and scene flow estimation,

    H. Liu, T. Lu, Y . Xu, J. Liu, W. Li, and L. Chen, “Camliflow: Bidirectional camera-lidar fusion for joint optical flow and scene flow estimation,” in Proceedings of IEEE/CVF CVPR, pp. 5791–5801, 2022. 3

    2022

  94. [102]

    Multimodal ma- terial segmentation,

    Y . Liang, R. Wakaki, S. Nobuhara, and K. Nishino, “Multimodal ma- terial segmentation,” in Proceedings of IEEE/CVF CVPR , pp. 19800– 19808, 2022. 3

    2022

  95. [103]

    Hrfuser: A multi-resolution sensor fusion architecture for 2d object detection,

    T. Broedermann, C. Sakaridis, D. Dai, and L. Van Gool, “Hrfuser: A multi-resolution sensor fusion architecture for 2d object detection,” arXiv:2206.15157, 2022. 3

    2022 arXiv

  96. [104]

    Mmanet: Margin-aware distillation and modality-aware regularization for incomplete multimodal learning,

    S. Wei, C. Luo, and Y . Luo, “Mmanet: Margin-aware distillation and modality-aware regularization for incomplete multimodal learning,” in Proceedings of IEEE/CVF CVPR , pp. 20039–20049, 2023. 3

    2023

  97. [105]

    Bev-guided multi-modality fusion for driving perception,

    Y . Man, L.-Y . Gui, and Y .-X. Wang, “Bev-guided multi-modality fusion for driving perception,” in Proceedings of IEEE/CVF CVPR , pp. 21960–21969, 2023. 3

    2023

  98. [106]

    Fourier prompt tuning for modality- incomplete scene segmentation,

    R. Liu, J. Zhang, K. Peng, Y . Chen, K. Cao, J. Zheng, M. S. Sarfraz, K. Yang, and R. Stiefelhagen, “Fourier prompt tuning for modality- incomplete scene segmentation,” arXiv:2401.16923, 2024. 3

    2024 arXiv

  99. [107]

    Cmx: Cross-modal fusion for rgb-x semantic segmentation with transform- ers,

    J. Zhang, H. Liu, K. Yang, X. Hu, R. Liu, and R. Stiefelhagen, “Cmx: Cross-modal fusion for rgb-x semantic segmentation with transform- ers,” IEEE TITS, 2023. 5, 6

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.