Pith. sign in

REVIEW 5 major objections 6 minor 56 references

Project-and-Fuse: Improving RGB-D Semantic Segmentation via Graph Convolution Networks

T0 review · 5 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RGB-D segmentation improves consistently when depth geometry is routed through an RGB-guided semantic graph, with mIoU gains of 1.7-2.3 points on NYUDv2 over the paper's baselines.

desk verdict Plausible incremental late-fusion RGB-D graph method; the empirical claim needs error bars and code before it is fully credible. read the letter →

arxiv 2501.18851 v3 pith:CF4LV2WX submitted 2025-01-31 cs.CV

Xiaoyan Jiang , Bohan Wang , Xinlong Wan , Shanshan Chen , Hamido Fujita , Hanan Abd.Al Juaid This is my paper · ORCID
classification cs.CV
keywords RGB-DsemanticsegmentationgraphconvolutionalnetworkslatefusionnormalmapencodingprojectionmatrixconstraintsKLdivergencelocality-awareadjacencyindoorsceneunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper seeks to establish that RGB-D semantic segmentation can be improved by fusing the two modalities late, inside a pixel-node-pixel graph, rather than through elaborate cross-modal fusion modules. Depth is first encoded as a surface normal map so a relatively light CNN can extract geometric tendencies; the graph's projection matrix is learned from the RGB branch alone, on the premise that texture should lead and depth should supplement. Two constraints on graph construction are the core fixes: a Kullback-Leibler loss that stops the projection from assigning many pixels to the same nodes while leaving others unassigned, and a locality-aware adjacency matrix that weights edges by inverse Euclidean distance between region centroids. On NYUDv2 the full method raises mIoU from 44.3 to 47.0 for the 2D baseline and from 48.9 to 50.6 when the normal-map depth branch is added, with similar patterns on SUN RGB-D. The point of the work is to reduce counter-intuitive segmentation patches by reasoning about region relationships rather than isolated pixels.

What carries the argument

The load-bearing object is the shared projection matrix $P \in \mathbb{R}^{N \times H \times W}$, produced by a $1\times 1$ convolution from the RGB feature map, that assigns every pixel to one of $N$ semantic nodes; the same $P$ projects both the RGB features and the depth-branch features into node space, which is what makes the late fusion a simple summation. Around it sit two supporting mechanisms: the KL-divergence loss $\mathcal{L}_{KL}$ that keeps $P$ from collapsing into a sparse one-hot-like assignment, and the locality-aware adjacency matrix whose edge weight between nodes $i$ and $j$ is the inverse Euclidean distance between learned region centroids, optionally supervised by an MSE loss $\mathcal{L}_{MSE}$ against centroids computed from the projection matrix and depth. A graph convolutional network then updates the fused node features, and the transpose $P^T$ re-projects them to the pixel grid. The depth branch is fed a normal map computed from the pinhole-camera point cloud via least-squares plane fitting, which lets a ResNet18 match the geometric information a much heavier depth encoder would provide.

What would settle it

Take the classes where RGB texture is least informative (wall, floor, ceiling, glass) on NYUDv2 and compare per-class IoU for the shared RGB-generated projection $P$ against a variant that generates $P$ from depth features as well; if the depth-generated projection improves those classes by more than a small margin, the claim that RGB texture should lead the assignment is falsified.

Watch

Extended reading notes

Core claim

The discovery the paper argues for is that the pixel-node-pixel graph pipeline can serve as a fusion mechanism for RGB and depth, provided the graph construction is constrained. Depth is converted into a normal map so a comparatively light CNN can extract surface tendencies; the projection matrix that maps pixels to semantic nodes is generated from the RGB feature map only, on the premise that RGB texture should lead and depth should supplement. The paper identifies two failure modes in unconstrained graph construction: Biased-Assignment, where soft projection becomes sparse and some pixels are assigned to multiple nodes while others to none, and Ambiguous-Locality, where edge weights depend only on semantic similarity and ignore position. The fixes are a KL-divergence loss pushing the projection matrix toward uniform assignment and an adjacency matrix whose weights combine semantic similarity with inverse Euclidean distance between per-node region centroids. With these constraints, the fused graph yields more consistent segmentation and consistent mIoU gains on both evaluated datasets.

Load-bearing premise

The load-bearing premise is that the RGB image's texture alone can decide where the depth-derived geometry belongs in the semantic graph; on textureless walls, glass, or occluded boundaries the depth features may be sent to the wrong places and the fusion then loses the geometry it was meant to add.

Editorial extensions

If this is right

  • On NYUDv2, adding the full graph module to the 2D baseline raises mIoU from 44.3 to 47.0, while the normal-map depth branch raises the RGB-D baseline from 48.9 to 50.6.
  • The KL constraint on the projection matrix accounts for about 1.2 mIoU points by itself, and combining it with hard assignment yields a total 1.3-point gain, suggesting that assignment collapse is a measurable source of error.
  • Locality-aware edge weights from region centroids outperform semantic-only edges and edges from the raw projection matrix, implying that positional information should be part of graph construction for segmentation.
  • On SUN RGB-D, replacing raw depth with normal maps or with the HHA encoding (a standard depth-derived representation) raises mIoU from 47.9 to 49.1 or 49.3, consistent with the claim that CNNs extract surface tendencies better from encoded depth.
  • Because the module is appended to the encoder-decoder output, it can be stacked on different backbones without changing the base segmentation architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the texture-prior premise holds, the method's gains should concentrate where RGB already supplies trustworthy grouping; a confidence-gated projection that falls back to depth when RGB texture is weak would be a natural extension.
  • The success of the normal-map encoding with a much lighter depth backbone suggests that representing geometry by surface orientation, rather than raw distance, is the transferable lesson for other depth-sensor tasks.
  • The KL-loss reading of the projection matrix as hard-pixel mining is not specific to RGB-D; it could be tested as a regularizer in any pixel-to-node graph segmentation pipeline to prevent node collapse.
  • A direct next experiment would be to replace the shared RGB-generated projection with two modality-specific projections aligned by a consistency loss, and check whether ambiguous boundaries benefit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Project-and-Fuse, a late-fusion RGB-D semantic segmentation method built on the pixel-node-pixel graph reasoning pipeline. The main components are: encoding the depth map as a normal map before 3D feature extraction; generating a single projection matrix from RGB features and using it to project both RGB and depth features into a shared graph; adding a KL-divergence loss on the projection matrix to mitigate 'Biased-Assignment'; and building a locality-aware adjacency matrix from region centroids to address 'Ambiguous-Locality'. The fused graph is processed with GCNs and re-projected to the pixel space. Experiments on NYUDv2 and SUN RGB-D report consistent mIoU improvements over the authors' baselines, with e.g. 2D branch improving from 44.3 to 47.0 and 3D branch from 48.9 to 50.6 on NYUDv2. The paper also includes ablations and visualizations.

Significance. If the claims hold, the paper offers a relatively simple and interpretable way to fuse RGB and depth for indoor segmentation while addressing unintuitive patch errors. The identified problems in graph construction—Biased-Assignment and Ambiguous-Locality—are plausible and the proposed fixes are straightforward. The strength of the paper is its broad ablation coverage and evaluations on two public datasets. However, the central empirical claim, 'consistently boost', is not yet established because all results are from single runs without error bars, and key hyperparameters are undisclosed. More importantly, the mathematical formulation of the normal-map encoding is incorrect, which undermines a stated contribution. The shared-projection design is also a risk on textureless regions. With corrected normals, proper statistical reporting, and complete implementation details, the work could become a solid contribution; in its current form it requires major revision.

major comments (5)
  1. [Section 3.2.3, Eq. (2)–(3)] The normal vector estimation is mathematically flawed. Setting An = b with b = 1 solves for a plane n·x = 1, which fixes the signed distance from the origin to unity for every plane; the standard surface normal should instead satisfy (p_i − centroid)·n = 0, i.e., a homogeneous least-squares or PCA fit. Furthermore, the normalization in Eq. (3) divides by the squared norm ||·||_2^2, so the result is not a unit vector. Since the normal map is a stated contribution and is used as input to the 3D branch, this error must be corrected and the experiments re-run with a valid normal-estimation method.
  2. [Section 4.3, Tables 1–2] All reported mIoU numbers are from single runs with no error bars, number of seeds, or statistical significance. The improvements are 0.6–2.3 points, which can easily be within run-to-run variance for deep segmentation models. The abstract's claim of 'consistently boost' is therefore not supported by the presented evidence. Please report mean ± std over at least three seeds, or otherwise justify that the differences are outside optimization noise.
  3. [Section 4.2 and Eq. (17)] Key hyperparameters are not reported: the loss weights α and β, the number of graph nodes N, the node feature dimension D, and the depth discontinuity threshold γ defined in Section 3.2.2. The neighborhood size k is mentioned as 9, but that is not tied to any sensitivity analysis. Without these values the experiments are not reproducible, and the ablation tables cannot be interpreted quantitatively. Please provide a complete hyperparameter table and, ideally, a sensitivity study for the most influential ones.
  4. [Section 3.3.3] The shared projection matrix P is generated from RGB features only and then used to project depth features as well. This design assumes that RGB texture is a reliable predictor of where depth-derived geometry should be assigned in the semantic graph. The paper does not test this assumption, and failure cases such as textureless walls or glass could misroute depth information to wrong nodes. Please add an experiment with a separate projection matrix for the depth branch, or a qualitative/quantitative evaluation on textureless regions, to verify that the 'RGB leads, depth supplements' premise actually holds.
  5. [Section 3.3.4 and 3.4] The paper defines two adjacency matrices, semantic-aware As and locality-aware Al, but never specifies how they are combined into the single adjacency matrix E used by the GCN in Eq. (11)–(14). The text describes computing ws_ij and wl_ij but does not give a formula for the final edge weights. This omission makes the core graph-construction module underspecified. Please state explicitly how As and Al are fused (e.g., sum, concatenation, or learned weights) and verify that the chosen combination is used in all reported experiments.
minor comments (6)
  1. [Tables 1 and 2] The table formatting is broken: column headers are merged with row labels, making it impossible to tell which configuration corresponds to which Y marks. For example, in Table 1 the text refers to 'first three lines' but the row layout is ambiguous. Please re-render the tables with clear column headers and per-row checkmarks.
  2. [Equation (17)] The sentence 'where LKL and LMSE have beed introduced in Equation 5 and 9 and 16' is incorrect: Equation 16 defines LCE, not LMSE. Please correct the cross-references.
  3. [Section 3.3.3] The phrase 'RGB-D images are rich in texture' should be 'RGB images are rich in texture', because the projection matrix is derived from the 2D RGB feature map, not from the RGB-D input.
  4. [Section 3.3.1] The interpretation of the KL loss as 'hard pixel mining' is not self-evident: minimizing KL divergence between P and a uniform distribution spreads the assignment scores, whereas hard-pixel mining typically concentrates effort on difficult pixels. Please either provide a formal connection or rephrase this post-hoc explanation.
  5. [Section 3.3.4] The text introduces 'two different approaches' for locality-aware adjacency—direct multiplication P·P^T and the centroid-based method—but the first is mentioned only in passing and no ablation quantifies its performance. Please clarify which approach is used in the main results and why the centroid method is preferred.
  6. [Notation] The symbol Fg is used both for projection generation in Section 3.3.1 and for 'Graph Reasoning or Graph Convolution' in Eq. (11). Please disambiguate to avoid confusion.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central mIoU claims are tested on external benchmarks; the one self-citation is contextual and not load-bearing.

full rationale

The paper's central claim is an empirical mIoU improvement, evaluated in Tables 1-4 against its own DeepLabV3+ baselines and published methods on the fixed NYUDv2 and SUN RGB-D benchmarks; the claimed boost is therefore not defined into existence by the method. The projection matrix P is learned from the RGB feature map (Eq. 4) and reused for depth features (Sec 3.3.3), but that is a deliberate architectural choice rather than a circular reduction, since depth features still come from a separate backbone and the final fused output is supervised by cross-entropy against ground-truth labels (Eq. 16). The KL constraint (Eq. 5) and MSE centroid supervision (Eq. 9) are regularizers whose effect is tested by ablation, not quantities that presuppose the final accuracy. The only overlapping-author citation, [35] (Yao, Jiang, Fujita, Fang) in Sec 2.3, is used merely as one example of GNN applications in person re-identification; no load-bearing claim or uniqueness theorem is imported from it. The absence of error bars, seed counts, and reported hyperparameters (alpha, beta, N, D) is a reproducibility concern, not a circularity concern: single-run evidence can be weak without being definitionally forced. The score of 2 reflects only the presence of one minor non-load-bearing self-citation; no circular step could be exhibited.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical entities, forces, or particles. Its contributions are architectural: a shared projection matrix, a KL constraint, and a locality-aware adjacency matrix. The free parameters listed above are all chosen by hand or left unspecified, and the axioms are domain assumptions about modality fusion and graph reasoning.

free parameters (7)
  • alpha (KL loss weight)
    Weight of L_KL in Eq. 17; affects how strongly the projection matrix is forced toward uniformity; no value or sweep reported.
  • beta (MSE loss weight)
    Weight of L_MSE for the positional encoding in Eq. 17; no value or sweep reported.
  • N (number of graph nodes)
    Pre-defined node count in the graph; Section 4.3 says a threshold exists but does not give the number.
  • D (node feature dimension)
    Feature dimension after the transformation module; mentioned in Sec 3.3.2 but no value is reported.
  • k (neighborhood size for normal estimation) = 9
    Empirically chosen: 'we can get better results when k = 9' in Sec 3.2.3.
  • gamma (depth discontinuity threshold)
    Threshold in |zi - zj| < gamma * zi for defining the neighbor set in Sec 3.2.2; value not given.
  • GCN layer count
    Number of stacked graph convolutional layers in Sec 3.4 is not specified in the experiments.
assumptions (6)
  • domain assumption A projection matrix computed from RGB features alone is sufficient to assign pixels of both modalities to shared semantic nodes.
    Sec 3.3.3: 'we can use RGB-D to lead and the depth to supplement'. The entire late-fusion design depends on this.
  • ad hoc to paper Regularizing the projection matrix toward a uniform distribution does not destroy the semantic structure needed for segmentation.
    Sec 3.3.1 introduces L_KL to push P toward a uniform matrix; the paper does not prove this preserves discriminative assignments, and the 'hard pixel mining' explanation is informal.
  • domain assumption Euclidean distance between region centers is a valid measure of positional relationship for improving segmentation.
    Sec 3.3.4, Eq. 10 sets edge weight as 1/Euclid(i,j), assuming closer regions are more related; no evidence that this is better than learned affinities.
  • ad hoc to paper The least-squares fit with b=1 in Eq. 3 produces a surface normal map.
    The stated equation minimizes ||A n - 1||, which does not correspond to the standard plane-normal estimation; as written, the computed vector is not the surface normal.
  • domain assumption Depth maps are texture-less, so CNNs benefit from a normal or HHA encoding.
    Standard in the RGB-D literature and supported by the paper's Table 2 ablation, but still an assumption about modality characteristics.
  • domain assumption Graph convolution on the fused region graph can reduce irregular patches in the final segmentation.
    Sec 3.4: the paper assumes that region-level message passing corrects pixel-level label noise; the visualizations support it but no formal argument is given.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Project-and-Fuse: Improving RGB-D Semantic Segmentation via Graph Convolution Networks." pith.science (2026). https://pith.science/paper/CF4LV2WX

@misc{pith2026250118851,
  author       = {Pith},
  title        = {Pith review of: Project-and-Fuse: Improving RGB-D Semantic Segmentation via Graph Convolution Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CF4LV2WX}},
  note         = {Machine review of arXiv:2501.18851}
}
read the original abstract

Most existing RGB-D semantic segmentation methods focus on the feature level fusion, including complex cross-modality and cross-scale fusion modules. However, these methods may cause misalignment problem in the feature fusion process and counter-intuitive patches in the segmentation results. Inspired by the popular pixel-node-pixel pipeline, we propose to 1) fuse features from two modalities in a late fusion style, during which the geometric feature injection is guided by texture feature prior; 2) employ Graph Neural Networks (GNNs) on the fused feature to alleviate the emergence of irregular patches by inferring patch relationship. At the 3D feature extraction stage, we argue that traditional CNNs are not efficient enough for depth maps. So, we encode depth map into normal map, after which CNNs can easily extract object surface tendencies.At projection matrix generation stage, we find the existence of Biased-Assignment and Ambiguous-Locality issues in the original pipeline. Therefore, we propose to 1) adopt the Kullback-Leibler Loss to ensure no missing important pixel features, which can be viewed as hard pixel mining process; 2) connect regions that are close to each other in the Euclidean space as well as in the semantic space with larger edge weights so that location informations can been considered. Extensive experiments on two public datasets, NYU-DepthV2 and SUN RGB-D, have shown that our approach can consistently boost the performance of RGB-D semantic segmentation task.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 48 canonical work pages

  1. [1]

    Neurocomputing 493, 626–646 (2022)

    Mo, Y., Wu, Y., Yang, X., Liu, F., Liao, Y.: Review the state-of-the-art tech- nologies of semantic segmentation based on deep learning. Neurocomputing 493, 626–646 (2022)

    work page 2022

  2. [2]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Wei, Y., Xiao, H., Shi, H., Jie, Z., Feng, J., Huang, T.S.: Revisiting dilated convolution: A simple approach for weakly-and semi-supervised semantic segmen- tation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7268–7277 (2018)

    work page 2018

  3. [3]

    arXiv preprint arXiv:1409.1556 (2014)

    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

    arXiv 2014

  4. [4]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

    2018

  5. [5]

    In: Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

    work page 2018

  6. [6]

    In: European Conference on Computer Vision, pp

    Yuan, Y., Chen, X., Wang, J.: Object-contextual representations for semantic segmentation. In: European Conference on Computer Vision, pp. 173–190 (2020). Springer

    work page 2020

  7. [7]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Li, X., Zhao, H., Han, L., Tong, Y., Tan, S., Yang, K.: Gated fully fusion for semantic segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11418–11425 (2020)

    work page 2020

  8. [8]

    In: European Conference on Computer Vision, pp

    Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End- to-end object detection with transformers. In: European Conference on Computer Vision, pp. 213–229 (2020). Springer

    2020

Show all 56 references
  1. [9]

    arXiv preprint arXiv:2010.11929 (2020)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    2020 arXiv

  2. [10]

    Displays 70, 102080 (2021) 21

    Wang, C., Wang, C., Li, W., Wang, H.: A brief survey on rgb-d semantic segmentation using deep learning. Displays 70, 102080 (2021) 21

    2021

  3. [11]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Liu, M., Schonfeld, D., Tang, W.: Exploit visual dependency relations for semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9726–9735 (2021)

    2021

  4. [12]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Wu, Y., Zhang, G., Gao, Y., Deng, X., Gong, K., Liang, X., Lin, L.: Bidirec- tional graph reasoning network for panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9080–9089 (2020)

    2020

  5. [13]

    In: International Conference on Image and Graphics, pp

    Zhou, L., Liu, Y., Bai, X., Wang, X., Wang, C., Zhang, L., Gu, L.: Relation- aware reasoning with graph convolutional network. In: International Conference on Image and Graphics, pp. 52–64 (2021). Springer

    2021

  6. [14]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Wang, C., Fu, H., Ling, C.X., Du, P., Ma, H.: Region-based global reasoning networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12136–12143 (2020)

    2020

  7. [15]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    2015

  8. [16]

    In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 234–241 (2015). Springer

    2015

  9. [17]

    IEEE transactions on pattern analysis and machine intelligence 40(4), 834–848 (2017)

    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40(4), 834–848 (2017)

    2017

  10. [18]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Cheng, B., Collins, M.D., Zhu, Y., Liu, T., Huang, T.S., Adam, H., Chen, L.- C.: Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12475–12485 (2020)

    2020

  11. [19]

    CVPR (2018)

    Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. CVPR (2018)

    2018

  12. [20]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., Lu, H.: Dual attention network for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3146–3154 (2019)

    2019

  13. [21]

    Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: Ccnet: Criss-cross attention for semantic segmentation (2019)

    2019

  14. [22]

    Image and Vision Computing 105, 22 104042 (2021)

    Zhang, Y., Sidib´ e, D., Morel, O., M´ eriaudeau, F.: Deep multimodal fusion for semantic image segmentation: A survey. Image and Vision Computing 105, 22 104042 (2021)

    2021

  15. [23]

    Neurocomputing 462, 568–580 (2021)

    Cao, J., Leng, H., Cohen-Or, D., Lischinski, D., Chen, Y., Tu, C., Li, Y.: Rgb × d: Learning depth-weighted rgb patches for rgb-d indoor semantic segmentation. Neurocomputing 462, 568–580 (2021)

    2021

  16. [24]

    In: Asian Confer- ence on Computer Vision, pp

    Hazirbas, C., Ma, L., Domokos, C., Cremers, D.: Fusenet: Incorporating depth into semantic segmentation via fusion-based cnn architecture. In: Asian Confer- ence on Computer Vision, pp. 213–228 (2016). Springer

    2016

  17. [25]

    In: Proceedings of the IEEE International Conference on Computer Vision, pp

    Park, S.-J., Hong, K.-S., Lee, S.: Rdfnet: Rgb-d multi-level residual feature fusion for indoor semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4980–4989 (2017)

    2017

  18. [26]

    In: Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Cheng, Y., Cai, R., Li, Z., Zhao, X., Huang, K.: Locality-sensitive deconvolution networks with gated fusion for rgb-d indoor semantic segmentation. In: Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3029–3037 (2017)

    2017

  19. [27]

    Advances in Neural Information Processing Systems 33, 4835–4845 (2020)

    Wang, Y., Huang, W., Sun, F., Xu, T., Rong, Y., Huang, J.: Deep multi- modal fusion by channel exchanging. Advances in Neural Information Processing Systems 33, 4835–4845 (2020)

    2020

  20. [28]

    arXiv preprint arXiv:2101.10801 (2021)

    Chen, S., Zhu, X., Liu, W., He, X., Liu, J.: Global-local propagation network for rgb-d semantic segmentation. arXiv preprint arXiv:2101.10801 (2021)

    2021 arXiv

  21. [29]

    In: Proceed- ings of the European Conference on Computer Vision (ECCV), pp

    Wang, W., Neumann, U.: Depth-aware cnn for rgb-d segmentation. In: Proceed- ings of the European Conference on Computer Vision (ECCV), pp. 135–150 (2018)

    2018

  22. [30]

    In: European Conference on Computer Vision, pp

    Xing, Y., Wang, J., Zeng, G.: Malleable 2.5 d convolution: Learning receptive fields along the depth-axis for rgb-d scene parsing. In: European Conference on Computer Vision, pp. 555–571 (2020). Springer

    2020

  23. [31]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp

    Cao, J., Leng, H., Lischinski, D., Cohen-Or, D., Tu, C., Li, Y.: Shapeconv: Shape- aware convolutional layer for indoor rgb-d semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7088–7097 (2021)

    2021

  24. [32]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Jiao, J., Wei, Y., Jie, Z., Shi, H., Lau, R.W., Huang, T.S.: Geometry-aware dis- tillation for indoor semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2869–2878 (2019)

    2019

  25. [33]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Liu, Z., Qi, X., Fu, C.-W.: 3d-to-2d distillation for indoor scene parsing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4464–4474 (2021) 23

    2021

  26. [34]

    arXiv preprint arXiv:2203.04838 (2022)

    Liu, H., Zhang, J., Yang, K., Hu, X., Stiefelhagen, R.: Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers. arXiv preprint arXiv:2203.04838 (2022)

    2022 arXiv

  27. [35]

    Pattern Recognition129, 108708 (2022)

    Yao, Y., Jiang, X., Fujita, H., Fang, Z.: A sparse graph wavelet convolution neural network for video-based person re-identification. Pattern Recognition129, 108708 (2022)

    2022

  28. [36]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp

    Zhao, G., Ge, W., Yu, Y.: Graphfpn: Graph feature pyramid network for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2763–2772 (2021)

    2021

  29. [37]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Shi, W., Rajkumar, R.: Point-gnn: Graph neural network for 3d object detection in a point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1711–1719 (2020)

    2020

  30. [38]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Zhang, Y., Huang, D., Wang, Y.: Pc-rgnn: Point cloud completion and graph neural network for 3d object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 3430–3437 (2021)

    2021

  31. [39]

    In: Proceedings of the IEEE International Conference on Computer Vision, pp

    Qi, X., Liao, R., Jia, J., Fidler, S., Urtasun, R.: 3d graph neural networks for rgbd semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5199–5208 (2017)

    2017

  32. [40]

    In: 2019 International Conference on 3D Vision (3DV), pp

    Chen, Y., Mensink, T., Gavves, E.: 3d neighborhood convolution: learning depth- aware features for rgb-d and rgb semantic segmentation. In: 2019 International Conference on 3D Vision (3DV), pp. 173–182 (2019). IEEE

    2019

  33. [41]

    Advances in Neural Information Processing Systems 31 (2018)

    Li, Y., Gupta, A.: Beyond grids: Learning graph representations for visual recognition. Advances in Neural Information Processing Systems 31 (2018)

    2018

  34. [42]

    Advances in Neural Information Processing Systems 31 (2018)

    Liang, X., Hu, Z., Zhang, H., Lin, L., Xing, E.P.: Symbolic graph reasoning meets convolutions. Advances in Neural Information Processing Systems 31 (2018)

    2018

  35. [43]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Chen, Y., Rohrbach, M., Yan, Z., Shuicheng, Y., Feng, J., Kalantidis, Y.: Graph- based global reasoning networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 433–442 (2019)

    2019

  36. [44]

    arXiv preprint arXiv:1609.02907 (2016)

    Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

    2016 arXiv

  37. [45]

    arXiv preprint arXiv:2206.00272 (2022)

    Han, K., Wang, Y., Guo, J., Tang, Y., Wu, E.: Vision gnn: An image is worth graph of nodes. arXiv preprint arXiv:2206.00272 (2022)

    2022 arXiv

  38. [46]

    In: Proceedings of the ACM Web Conference 2022, pp

    Wei, L., Zhao, H., He, Z.: Designing the topology of graph neural networks: A novel feature fusion perspective. In: Proceedings of the ACM Web Conference 2022, pp. 1381–1391 (2022) 24

    2022

  39. [47]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Yin, Z., Shi, J.: Geonet: Unsupervised learning of dense depth, optical flow and camera pose. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1983–1992 (2018)

    2018

  40. [48]

    In: IEEE International Conference on Image Processing, pp

    Yajie, X., Jingbo, W., Xiaokang, C., Gang, Z.: 2.5d convolution for rgb-d semantic segmentation. In: IEEE International Conference on Image Processing, pp. 1410– 1414 (2019). IEEE

    2019

  41. [49]

    In: European Conference on Computer Vision, pp

    Chen, X., Lin, K.-Y., Wang, J., Wu, W., Qian, C., Li, H., Zeng, G.: Bi-directional cross-modality feature propagation with separation-and-aggregation gate for rgb- d semantic segmentation. In: European Conference on Computer Vision, pp. 561– 577 (2020). Springer

    2020

  42. [50]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Borse, S., Wang, Y., Zhang, Y., Porikli, F.: Inverseform: A loss function for structured boundary-aware segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5901–5911 (2021)

    2021

  43. [51]

    Pattern Recognition 124, 108468 (2022)

    Zhou, H., Qi, L., Huang, H., Yang, X., Wan, Z., Wen, X.: Canet: Co-attention network for rgb-d semantic segmentation. Pattern Recognition 124, 108468 (2022)

    2022

  44. [52]

    In: European Conference on Computer Vision, pp

    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: European Conference on Computer Vision, pp. 746–760 (2012). Springer

    2012

  45. [53]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Song, S., Lichtenberg, S.P., Xiao, J.: Sun rgb-d: A rgb-d scene understanding benchmark suite. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 567–576 (2015)

    2015

  46. [54]

    In: Consumer Depth Cameras for Computer Vision, pp

    Janoch, A., Karayev, S., Jia, Y., Barron, J.T., Fritz, M., Saenko, K., Darrell, T.: A category-level 3d object dataset: Putting the kinect to work. In: Consumer Depth Cameras for Computer Vision, pp. 141–165. Springer, ??? (2013)

    2013

  47. [55]

    In: Proceedings of the IEEE International Conference on Computer Vision, pp

    Xiao, J., Owens, A., Torralba, A.: Sun3d: A database of big spaces reconstructed using sfm and object labels. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1625–1632 (2013)

    2013

  48. [56]

    In: 2021 IEEE Interna- tional Conference on Robotics and Automation (ICRA), pp

    Seichter, D., K¨ ohler, M., Lewandowski, B., Wengefeld, T., Gross, H.-M.: Efficient rgb-d semantic segmentation for indoor scene analysis. In: 2021 IEEE Interna- tional Conference on Robotics and Automation (ICRA), pp. 13525–13531 (2021). IEEE 25

    2021

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.