Pith. sign in

REVIEW 4 major objections 6 minor 22 references

Minimizing Occlusion Effect on Multi-View Camera Perception in BEV with Multi-Sensor Fusion

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fusing radar and LiDAR with occluded cameras cuts the relative degradation of bird's-eye-view vehicle segmentation under realistic lens soiling from 27.6% to 15.5%.

desk verdict Useful first measurement of occlusion effects on BEV segmentation, but the synthetic blur protocol is unvalidated, so the headline numbers are illustrative, not transferable. read the letter →

arxiv 2501.05997 v1 pith:S32LMQ4Q submitted 2025-01-10 cs.CV

Sanjay Kumar , Hiep Truong , Sushil Sharma , Ganesh Sistu , Tony Scanlan , Eoin Grua , Ciarán Eising This is my paper · ORCID
classification cs.CV
keywords multi-sensorfusionbird'seyeviewvehiclesegmentationcameraocclusionsyntheticsoilingnuScenesSimple-BEVautonomousdriving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests how camera lens occlusions—simulated dirt, moisture, and fog from the WoodScape soiling dataset applied to nuScenes multi-view images—hurt bird's-eye-view (BEV) vehicle segmentation. It reports that realistic occlusion is the most damaging of three occlusion types, dropping camera-only intersection-over-union (IoU) from 47.4 to 34.3. Fusing the occluded cameras with radar and LiDAR in the Simple-BEV architecture recovers most of the loss, raising IoU to 54.5 and cutting relative degradation from 27.6% to 15.5%. The central claim is that multi-sensor fusion is an effective practical countermeasure to lens soiling for BEV perception.

What carries the argument

The load-bearing mechanism is the Simple-BEV fusion pipeline [15]: a ResNet-101 camera encoder extracts 2D features, bilinear sampling lifts them to a 3D BEV plane, and voxelized radar and LiDAR occupancy grids are flattened and merged into the same BEV representation before a ResNet-18 encoder and segmentation head. The other key piece is the occlusion synthesis: WoodScape [17] soiling patterns are binarized and blurred with a 251x251 Gaussian kernel to produce realistic lens occlusions on the nuScenes [16] camera views.

What would settle it

Train or evaluate the same Simple-BEV configurations on a real soiling dataset—cameras with physically dirty or wet lenses in a controlled driving scene—and compare the camera-only and camera-radar-LiDAR IoU drops. If the relative degradation under realistic soiling is not substantially larger than the degradation from random-box occlusion, or if fusing radar and LiDAR does not reduce the relative degradation below the camera-only level, the paper's central claim would be contradicted.

Watch

Extended reading notes

Core claim

The paper's central discovery is that realistic camera occlusion, modeled as binary WoodScape soiling masks blurred with a 251x251 Gaussian kernel, degrades BEV vehicle segmentation more than random or overlap occlusion, and that this degradation is systematically mitigated by adding radar and LiDAR. In the camera-only baseline, IoU falls to 34.3; with camera, radar, and LiDAR fused it stays at 54.5, so the relative degradation falls from 27.6% to 15.5%. The result is presented as evidence that complementary sensors compensate for occluded camera views in the BEV domain.

Load-bearing premise

The whole measurement of degradation and mitigation assumes that WoodScape soiling patterns, converted to binary masks and blurred with a 251x251 Gaussian kernel, faithfully reproduce real-world lens occlusion from dirt, rain, and fog.

Editorial extensions

If this is right

  • Under realistic camera soiling, BEV vehicle segmentation degrades substantially; camera-only IoU drops by 27.6%, so systems relying on cameras alone need a mitigation strategy.
  • Adding radar alone reduces the relative degradation to 22.6%, adding LiDAR alone to 17.2%, and adding both to 15.5%, so each extra modality contributes measurably.
  • The fusion benefit holds for all three occlusion types tested, with the largest absolute gains on the hardest, realistic occlusion.
  • The pattern suggests that multi-sensor fusion is a general robustness mechanism for BEV perception under lens contamination, not just for the specific soiling patterns used in the paper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same fusion approach would likely reduce occlusion-driven degradation for other BEV tasks such as object detection and lane segmentation, since the BEV representation is shared; this extrapolates beyond the paper's vehicle-segmentation experiments.
  • The degradation numbers may underestimate real-world soiling because the synthesis uses static binary masks and does not model glare, transparent droplets, or temporal accretion; real-world field tests could show different magnitudes.
  • The monotonic improvement with added sensors suggests that even radar-only fusion is a worthwhile fallback when LiDAR is unavailable, a cost-benefit conclusion the paper does not draw.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies how camera lens occlusion affects bird's-eye-view (BEV) vehicle segmentation and whether multi-sensor fusion with radar and LiDAR mitigates the degradation. The authors start from the Simple-BEV architecture on nuScenes and create three occlusion conditions: random boxes, occlusion over overlapping camera fields of view, and a so-called "realistic" occlusion obtained by binarizing soiling patterns from the WoodScape dataset and blurring the occluded regions with a 251x251 Gaussian kernel. They report IoU values for camera-only (C), camera+radar (C+R), camera+LiDAR (C+L), and camera+radar+LiDAR (C+R+L) under each condition, with the central quantitative result in Table 1: under "realistic" occlusion, IoU drops to 34.3 for C but only to 54.5 for C+R+L, and the relative degradation decreases from 27.6% to 15.5%. The paper concludes that fusing clean radar and LiDAR data with occluded cameras substantially restores BEV segmentation accuracy.

Significance. If the result holds, the paper provides a useful empirical measurement of how sensor fusion can compensate for camera soiling in BEV perception, which is relevant for autonomous driving robustness. The authors use public datasets (nuScenes, WoodScape) and a public architecture (Simple-BEV), and the core numbers in Table 1 are direct measurements rather than fitted quantities, which is a strength. However, the current evidence is preliminary: the synthetic occlusion protocol is not validated against real dirt, rain, or fog; the baseline IoU values for C, C+R, and C+L are quoted from the Simple-BEV paper rather than retrained under the same protocol; and no error bars, repeated runs, or statistical tests are provided. These issues directly affect the external validity and reliability of the headline claim, so the contribution is not yet established at the level expected for a journal publication. The paper is a reasonable starting point for a more rigorous empirical study, but it currently reads more like a short report or workshop contribution than a complete journal article.

major comments (4)
  1. [Quantitative Analysis, Table 1] The "realistic" occlusion protocol is not validated, and the specific numbers in Table 1 depend on an uncalibrated 251x251 Gaussian blur applied to binarized WoodScape patterns. WoodScape images are fisheye while nuScenes cameras are not, and the paper does not describe any geometric or photometric remapping. More importantly, a blurred copy of the original image content is not equivalent to opaque dirt, raindrops, or fog: scene information remains partially visible, and the amount of information loss is an artifact of the arbitrary kernel size. Since radar and LiDAR inputs are always clean, sufficiently strong image corruption will make fusion appear beneficial, so the 27.6% to 15.5% degradation reduction is not shown to transfer to real-world soiling. The paper should validate the protocol against real occluded or rainy frames (e.g., nuScenes rain sequences or WoodScape soiled images) or at least include a sensitivity analysis over kernel size and blur opacity, and ideally provide real-occlusion test results.
  2. [Quantitative Analysis, Table 1, Implementation Details] The baseline values for C, C+R, and C+R+L are not obtained under a common protocol. The text states that C, C+R, and C+L IoU values (47.4, 55.7, 60.8) are reported from the Simple-BEV paper, while C+R+L (64.5) was trained by the authors because pretrained weights were not available. The degradation percentages in Table 1 are computed against this mixed set of baselines, so the comparison is only meaningful if the quoted Simple-BEV numbers are directly comparable to the authors' training setup (80 epochs, batch size 12, input resolution 448x800) and evaluation protocol. The authors should retrain all four configurations under identical conditions, or clearly justify the transferability of the quoted numbers. Otherwise, even a monotonic trend in Table 1 could be an artifact of differing training protocols rather than of fusion.
  3. [Table 1, Quantitative Analysis] All IoU values in Table 1 are single numbers with no error bars, no repeated runs, and no significance tests. The conclusion relies on relatively small differences among conditions, e.g., the realistic condition C+R+L at 54.5 versus C+L at 50.3, and degradation percentages of 15.5% versus 17.2%. Without variance estimates or a stated number of evaluation samples, these differences may be within run-to-run noise, especially given that the self-trained C+R+L model could have a different random seed or data split from the quoted baselines. The authors should report mean and standard deviation over multiple runs (or at least over evaluation subsets) and state the number of test scenes used.
  4. [Dataset section, Implementation Details] The experimental protocol is under-specified, which makes the results difficult to reproduce or interpret. The paper does not state whether the nuScenes training or validation split is used for occlusion evaluation, how many samples are occluded and evaluated, whether all six cameras are occluded simultaneously, how patterns are selected and placed per image, and crucially whether the models are trained on clean images and tested on occluded images, or trained with occlusion augmentation. The text says "we create an occluded version of the nuScenes dataset" but also describes training a C+R+L baseline, so the reader cannot tell if the occlusion is applied only at test time. The authors should provide a complete protocol description, including the exact evaluation split and sample count.
minor comments (6)
  1. [Abstract] The abstract contains the raw YouTube URL "https://youtu.be/OmX2NEeOzAE" in the text body; this should be moved to a footnote or removed.
  2. [Throughout] The paper uses inconsistent capitalization and spelling for the same terms, e.g., "Woodscape" versus "WoodScape" and "simple-BEV" versus "Simple-BEV". Please standardize these names.
  3. [Equation (1)] The BCEWithLogitsLoss formula in Equation (1) is written with a sum over an unspecified N; please clarify whether the average is over pixels or over samples, and ensure the notation matches the implementation referenced in [15].
  4. [Figure 4 caption] The caption says the third column shows "a binary mask indicating the areas of occlusion in red," but the main text describes the third column as a BEV projection; please align the column descriptions with what is actually displayed.
  5. [Table 1] The first column header "Baseline Architecture [15]" is misleading because the rows are occlusion conditions, not baselines. Consider renaming it to "Occlusion Condition" and using the first row as "No occlusion (Simple-BEV)."
  6. [References] The reference for the soiling patterns is given as [17], the general WoodScape dataset paper, but the text refers to the "WoodScape Soiled Dataset." Please clarify whether a separate soiling dataset was used and cite it specifically if so.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity: the Table 1 IoU values and degradation percentages are direct measurements under a stated synthetic occlusion protocol, not fitted inputs renamed as predictions.

full rationale

The paper's central empirical claim is Table 1: occluded camera-only IoU drops to 34.3 while C+R+L stays at 54.5, with degradation improving from 27.6% to 15.5%. These numbers are measured outputs of a fixed Simple-BEV inference pipeline; they are not obtained by fitting a parameter to the reported outcomes. The degradation percentages are arithmetic ratios of the IoU values in the same table (baseline vs. realistic occlusion), so they reduce to direct measurements rather than to an input. The only configurable elements in the pipeline are Simple-BEV's published architecture/weights from the external reference [15] and the occlusion synthesis kernel (a 251x251 Gaussian blur applied to WoodScape binary masks); neither is fitted to the target IoUs. The WoodScape citation is a data-source citation to an external dataset, and although one author overlaps with the present paper, the dataset itself is independently published and is not invoked as an unverified theorem to force a conclusion. The prior self-citations ([5], [18], [19]) appear only as related-work context and are not load-bearing. The substantive weakness identified by a skeptical reader—whether blurred binary masks faithfully reproduce real dirt, rain, or fog—is an external-validity concern, not a circularity concern, because the measurements are well-defined for the stated protocol. No step in the paper's argument reduces by construction to its own inputs, so no circular step can be exhibited.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the realism of the occlusion synthesis and on the comparability of cited baselines, but the paper introduces no new physical or architectural entities.

free parameters (1)
  • Gaussian blur kernel size = 251x251
    Hand-chosen in the Dataset section to control the severity of simulated moist or fog occlusion; no ablation is reported, and this value directly affects the magnitude of IoU degradation in Table 1.
assumptions (4)
  • domain assumption Simple-BEV is a valid and representative BEV vehicle segmentation architecture.
    The paper uses Simple-BEV [15] as the sole baseline without re-evaluating other BEV methods.
  • ad hoc to paper WoodScape soiling patterns blurred with a Gaussian filter approximate real-world camera occlusion.
    The entire synthetic-occlusion methodology in the Dataset section rests on this assumption; no real-world occlusion validation is provided.
  • domain assumption The nuScenes validation annotations used for IoU are correct and consistent.
    Assumed throughout the quantitative evaluation.
  • ad hoc to paper The authors' trained C+R+L model (64.5 IoU) is comparable to the Simple-BEV pretrained models for the other modalities.
    The paper trains only the C+R+L baseline itself (no pretrained weights released) while citing C, C+R, C+L numbers from [15].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Minimizing Occlusion Effect on Multi-View Camera Perception in BEV with Multi-Sensor Fusion." pith.science (2026). https://pith.science/paper/S32LMQ4Q

@misc{pith2026250105997,
  author       = {Pith},
  title        = {Pith review of: Minimizing Occlusion Effect on Multi-View Camera Perception in BEV with Multi-Sensor Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S32LMQ4Q}},
  note         = {Machine review of arXiv:2501.05997}
}
read the original abstract

Autonomous driving technology is rapidly evolving, offering the potential for safer and more efficient transportation. However, the performance of these systems can be significantly compromised by the occlusion on sensors due to environmental factors like dirt, dust, rain, and fog. These occlusions severely affect vision-based tasks such as object detection, vehicle segmentation, and lane recognition. In this paper, we investigate the impact of various kinds of occlusions on camera sensor by projecting their effects from multi-view camera images of the nuScenes dataset into the Bird's-Eye View (BEV) domain. This approach allows us to analyze how occlusions spatially distribute and influence vehicle segmentation accuracy within the BEV domain. Despite significant advances in sensor technology and multi-sensor fusion, a gap remains in the existing literature regarding the specific effects of camera occlusions on BEV-based perception systems. To address this gap, we use a multi-sensor fusion technique that integrates LiDAR and radar sensor data to mitigate the performance degradation caused by occluded cameras. Our findings demonstrate that this approach significantly enhances the accuracy and robustness of vehicle segmentation tasks, leading to more reliable autonomous driving systems.

Figures

Figures reproduced from arXiv: 2501.05997 by the authors.

Figure 1
Figure 1. Left: Multi-view cameras and their mounting positions on the ego vehicle. Middle: Camera lens obstructed by mud, raindrops, dust, etc. Right: Occluded image adversely affect the performance of vision-based algorithms, ultimately compromising the safety and reliability of AVs [7]. As visual data is crucial for generating the semantic representation of the environment, any occlusion can have a cascading effect on the … view at source ↗
Figure 2
Figure 2. The process of generating occlusion patterns from the Wood [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Architecture Flow for Simple-BEV: The diagram illustrates the integration of multi-view camera, radar, and LiDAR sensor data for vehicle segmentation. Occluded data from the cameras is processed and transformed into Bird’s-Eye View (BEV) features. These BEV features, along with radar and LiDAR features, are fused to enhance perception and mitigate the effects of occlusion. The model leverages the combined sensor dat… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative results: The sequence from left to right illustrates the multi-view cameras affected by occlusions, followed by the bird’s-eye view (BEV) projection of the occluded cameras. Next, we have a binary mask indicating the areas of occlusion in red. Following tha…
Figure 5
Figure 5. Figure 5: Types of occlusions Left: Random Box occlusion. Middle: Overlapping region occlusion. Right: Realistic Woodscape Pattern. multi-view cameras with realistic occlusion patterns. The second and third columns display the bird’s-eye view (BEV) projection and the correspondi…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 21 canonical work pages

  1. [1]

    M. M. Rana and K. Hossain, ”Connected and autonomous vehicles and infrastructures: A literature review,” International Journal of Pavement Research and Technology , vol. 16, no. 2, pp. 264–284, Springer, 2023

    work page 2023

  2. [2]

    J. Zhao, W. Zhao, B. Deng, Z. Wang, F. Zhang, W. Zheng, W. Cao, J. Nan, Y . Lian, and A. F. Burke, ”Autonomous driving system: A comprehensive survey,”Expert Systems with Applications, p. 122836, Elsevier, 2023

    work page 2023

  3. [3]

    H. Shao, L. Wang, R. Chen, H. Li, and Y . Liu, ”Safety-enhanced autonomous driving using interpretable sensor fusion transformer,” in Conference on Robot Learning, PMLR, 2023, pp. 726–737

    work page 2023

  4. [4]

    Z. Zou, K. Chen, Z. Shi, Y . Guo, and J. Ye, ”Object detection in 20 years: A survey,” Proceedings of the IEEE, vol. 111, no. 3, pp. 257– 276, 2023

    work page 2023

  5. [5]

    Sharma, A

    S. Sharma, A. Das, G. Sistu, M. Halton, and C. Eising, ”BEVSeg2TP: Surround View Camera Bird’s-Eye-View Based Joint Vehicle Seg- mentation and Ego Vehicle Trajectory Prediction,” in Proceedings of the 19th International Joint Conference on Computer Vision, Imag- ing and Computer Graphics Theory and Applications - Volume 4: VISAPP, 2024, pp. 25–34

    work page 2024

  6. [6]

    Y . Liu, J. Yan, F. Jia, S. Li, A. Gao, T. Wang, and X. Zhang, ”Petrv2: A unified framework for 3d perception from multi-camera images,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3262–3272

    work page 2023

  7. [7]

    S. Yao, R. Guan, X. Huang, Z. Li, X. Sha, Y . Yue, E. G. Lim, H. Seo, K. L. Man, X. Zhu, et al., ”Radar-camera fusion for object detection and semantic segmentation in autonomous driving: A comprehensive review,”IEEE Transactions on Intelligent Vehicles, IEEE, 2023

    work page 2023

  8. [8]

    Zhang, A

    Y . Zhang, A. Carballo, H. Yang, and K. Takeda, ”Perception and sens- ing for autonomous vehicles under adverse weather conditions: A survey,”ISPRS Journal of Photogrammetry and Remote Sensing, vol. 196, pp. 146–177, Elsevier, 2023

    work page 2023

Show all 22 references
  1. [9]

    U ˇriˇc´aˇr, J

    M. U ˇriˇc´aˇr, J. Uli ˇcn´y, G. Sistu, H. Rashed, P. K ˇr´ıˇzek, D. Hurych, A. V obeck´y, and S. Yogamani, ”Desoiling dataset: Restoring soiled ar- eas on automotive fisheye cameras,” inProceedings of the IEEE/CVF International Conference on Computer Vision Workshops , IEEE, ...

    2019

  2. [10]

    U ˇriˇc´aˇr, P

    M. U ˇriˇc´aˇr, P. Kˇr´ıˇzek, G. Sistu, and S. Yogamani, ”SoilingNet: Soil- ing detection on automotive surround-view cameras,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC) , IEEE, 2019, pp. 67–72

    2019

  3. [11]

    Yahiaoui, M

    L. Yahiaoui, M. U ˇriˇc´aˇr, A. Das, and S. Yogamani, ”Let the sunshine in: Sun glare detection on automotive surround-view cameras,” in Electronic Imaging, vol. 2020, no. 16, Society for Imaging Science and Technology, 2020, pp. 80–1

    2020

  4. [12]

    A. Das, P. Kˇr´ıˇzek, G. Sistu, F. B¨urger, S. Madasamy, M. Uˇriˇc´aˇr, V . R. Kumar, and S. Yogamani, ”Tiledsoilingnet: Tile-level soiling detec- tion on automotive surround-view cameras using coverage metric,” in 2020 IEEE 23rd International Conference on Intelligent Transpo...

    2020

  5. [13]

    U ˇriˇc´aˇr, G

    M. U ˇriˇc´aˇr, G. Sistu, L. Yahiaoui, and S. Yogamani, ”Ensemble- based semi-supervised learning to improve noisy soiling annota- tions in autonomous driving,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) , IEEE, 2021, pp. 2925– 2930

    2021

  6. [14]

    U ˇriˇc´aˇr, G

    M. U ˇriˇc´aˇr, G. Sistu, H. Rashed, A. V obeck´y, V . R. Kumar, P. Kˇr´ıˇzek, F. B¨urger, and S. Yogamani, ”Let’s get dirty: GAN based data aug- mentation for camera lens soiling detection in autonomous driving,” in Proceedings of the IEEE/CVF Winter Conference on Application...

    2021

  7. [15]

    A. W. Harley, Z. Fang, J. Li, R. Ambrus, and K. Fragkiadaki, ”Simple-BEV: What really matters for multi-sensor BEV perception,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2023, pp. 2759–2765

    2023

  8. [16]

    Caesar, V

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, ”nuScenes: A multimodal dataset for autonomous driving,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11621–11631

    2020

  9. [17]

    Yogamani, C

    S. Yogamani, C. Hughes, J. Horgan, G. Sistu, P. Varley, D. O’Dea, M. Uˇriˇc´aˇr, S. Milz, M. Simon, K. Amende, et al., ”WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 20...

    2019

  10. [18]

    Sharma, A

    S. Sharma, A. Das, G. Sistu, M. Halton, and C. Eising, ”BEVSeg2GTA: Joint Vehicle Segmentation and Graph Neural Net- works for Ego Vehicle Trajectory Prediction in Bird’s-Eye-View,” IEEE Access, 2024, pp. 1–1, doi: 10.1109/ACCESS.2024.3459595

    2024

  11. [19]

    Hayes, S

    S. Hayes, S. Sharma, and C. Eising, ”Velocity Driven Vision: Asyn- chronous Sensor Fusion Birds Eye View Models for Autonomous Ve- hicles,” arXiv preprint arXiv:2407.16636, 2024

    2024 arXiv

  12. [20]

    Zhou, B., & Kr ¨ahenb¨uhl, P. (2022). Cross-view transformers for real-time map-view semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13760–13769)

    2022

  13. [21]

    Philion, J., & Fidler, S. (2020). Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3D. In Com- puter Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV (pp. 194–210). Springer

    2020

  14. [22]

    Xu, R., Tu, Z., Xiang, H., Shao, W., Zhou, B., & Ma, J. (2022). CoBEVT: Cooperative bird’s eye view semantic segmentation with sparse transformers. arXiv preprint arXiv:2207.02202

    2022 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.