Pith. sign in

REVIEW 4 major objections 4 minor 46 references

Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A self-supervised framework aligns misaligned infrared-visible image pairs without ground-truth correspondence or image translation, and the paper reports that it beats eight prior methods on every measured metric across three datasets.

desk verdict Plausible self-supervised registration idea for IR-VIS fusion, but the alignment claim is under-supported because no direct registration error is reported and the main-text Lepr loss looks mis-specified. read the letter →

arxiv 2505.06920 v1 pith:WCKKXU6C submitted 2025-05-11 cs.CV

Timing Li , Bing Cao , Pengfei Zhu , Bin Xiao , Qinghua Hu This is my paper · ORCID
classification cs.CV
keywords infrared-visibleimagefusionregistrationself-supervisedlearningmulti-modalalignmentdeformationfieldproxydatageneratormisalignedpairs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes B-SR, a fully self-supervised framework for aligning and fusing infrared-visible image pairs that are spatially misaligned. It creates its own training signal with a proxy data generator that randomly crops, flips, rotates, and re-stitches images, and an inverse generator that undoes those operations on the model's outputs. The registration network is trained so that processing the permuted image and then un-permuting the result matches processing the original image, providing global and local alignment supervision without any ground-truth displacement. The paper reports that B-SR beats eight prior methods on all six evaluation metrics for the RoadScene, DroneVehicle, and TNO datasets, covering both simulated and real misalignment. This matters because existing alignment-based fusion methods rely on image translation, which injects noise, whereas B-SR aligns without it.

What carries the argument

The load-bearing mechanism is the proxy/inverse proxy data generator pair (PDG and IPDG). PDG cuts an image into N×N patches, records random flips and rotations per patch, and re-stitches them into a new image; IPDG applies the inverse operations to the model's predicted aligned images and deformation fields. The registration module is a U-Net that predicts bi-directional deformation fields, and a spatial transformer network warps each modality accordingly. The self-supervision couples the two branches through L1 and L2 losses on aligned images, deformation fields, and reconstructions, enforcing that registration commutes with the PDG permutation. Two additional losses address the modality gap directly: the neighborhood dynamic alignment loss (Lnda) matches edge distance and angle within a local neighborhood so that the current modality aligns to valid edges of the other modality, and the edge pixel retention loss (Lepr) preserves the pre-alignment structural differences around edges. A reconstruction decoder, trained jointly with registration, provides feature-level supervision, and the fusion module later reuses the frozen encoder.

What would settle it

Create a test set with known ground-truth displacements, such as infrared images shifted by a known amount, and compute the endpoint error of the predicted deformation fields against the true displacement. If removing the inter-branch self-supervised loss does not increase this endpoint error, the PDG/IPDG branch is not the source of alignment; conversely, if the loss stays low while endpoint error remains high, the equivariance assumption alone does not guarantee correct registration.

Watch

Extended reading notes

Core claim

The paper's central claim is that misaligned infrared-visible image pairs can be registered in a fully self-supervised way using bi-directional deformation fields, and that this registration improves downstream fusion quality. The supervisory signal is a consistency constraint between two branches: one branch processes the original images, the other processes images that have been cut into patches, randomly flipped and rotated, and re-stitched, and then applies the inverse operation to the aligned outputs and predicted deformation fields. The paper argues that these two branches should agree, so the loss enforces global-local difference consistency. It further claims that the proposed neighborhood dynamic alignment loss aligns cross-modal edges without image translation, and that joint optimization of registration with a reconstruction module sharpens both. The empirical claim is that B-SR obtains the highest scores on all six metrics across RoadScene with simulated 5-pixel shifts, DroneVehicle with large real misalignment, and TNO with small misalignment, and that it stays robust under dilation-and-crop misalignment up to 50 pixels.

Load-bearing premise

The training signal depends on the registration network being equivariant to the proxy data generator's random patch flips, rotations, and re-stitching: processing a permuted image and then undoing the permutation must equal processing the original image, and if the loss can be satisfied without true spatial alignment, the self-supervision gives no correctness signal.

Editorial extensions

If this is right

  • If B-SR is correct, multi-modal registration for fusion no longer needs ground-truth alignment or image translation, removing a major source of noise in prior pipelines.
  • The reported across-the-board gains on six metrics imply that alignment quality directly translates into measurable fusion quality on both simulated and real misaligned data.
  • The robustness experiments suggest the method can tolerate large unknown misalignment amplitudes, such as drone-view footage, without retraining per magnitude.
  • The ablation results indicate that each component—neighborhood dynamic alignment, joint registration-reconstruction, bi-directional fields, and inter-branch self-supervision—contributes to the final improvement, which means removing any one of them should degrade fusion quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension beyond the paper is to measure the registration error directly on synthetic warps with known ground truth; this would separate alignment accuracy from fusion quality and test the equivariance assumption more directly.
  • The same PDG/IPDG equivariance strategy could transfer to other registration tasks where paired ground truth is scarce, such as medical image alignment across modalities or multi-sensor remote sensing, since it only requires the network to be equivariant to the chosen proxy transformations.
  • The paper's choice of N=2 for the PDG is empirical; a principled study of how equivariance error grows with N could turn this hyperparameter into an architecture-aware calibration, possibly with curriculum scheduling of N during training.
  • Because the framework imposes consistency without ground truth, it will be most convincing when paired with an explicit calibration of the equivariance error of the U-Net backbone; architectures with padding or positional biases may require explicit equivariance guarantees.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes B-SR, a self-supervised bi-directional registration and fusion framework for misaligned infrared-visible image pairs. It uses a proxy data generator (PDG) and its inverse (IPDG) to construct inter-branch self-supervision, bi-directional deformation fields for intra-branch alignment, and a neighborhood dynamic alignment loss to handle the modality gap without explicit image translation. The method is evaluated on RoadScene, DroneVehicle, and TNO datasets with six fusion metrics, reporting the highest scores on all metrics in all datasets. The authors also provide ablation studies and robustness experiments under simulated dilation and center-cropping misalignment.

Significance. If the claims hold, B-SR addresses a practical problem: fusion of misaligned multi-modal data without ground-truth registration, and it does so without relying on image translation, which the paper argues avoids noise from translation and model dependence. The design is interesting and the experimental scope covers both simulated and real misalignment. The promise of public code is a positive factor for reproducibility. However, the registration claim is not directly verified: the paper never reports registration error, and the self-supervision target may be ill-posed as written. The fusion metrics alone are indirect evidence of alignment and can improve through contrast or sharpness effects. The central contribution needs stronger validation before it can be accepted.

major comments (4)
  1. [Section 3.4, Eq. (7)] The edge pixel retention loss Lepr = (Ta − V)^2 − (T − V)^2 is problematic as written. Its gradient with respect to Ta is 2(Ta − V), so minimizing it drives the aligned infrared image Ta toward the visible image V, erasing modality-specific infrared information. This contradicts the stated purpose (preserving structural differences) and would be a load-bearing flaw if the loss is implemented as in Eq. (7). Please either correct the formula, clarify the intended variable for Ta, or provide a derivation showing why this loss does not cause modality collapse.
  2. [Algorithm 1 and Section 3.1] Algorithm 1 writes 'Vnew,Tnew = P(V),P(T)', which suggests that the stochastic PDG transform is applied independently to V and T. But the inter-branch consistency loss requires that IPDG invert the same spatial transform that was applied to both modalities; otherwise the pseudo-global differences do not correspond to the original global difference, and the self-supervision target is incorrect. The paper must state explicitly that a single shared random transform (same patch partition, flips, rotations, stitching) is used for both images, and implement it as a joint operation. If independent transforms are intended, the method needs a different justification.
  3. [Sections 4.1 and 4.2] No direct registration error is reported anywhere. For the RoadScene experiment the misalignment is a known 5-pixel horizontal shift, so the displacement residual (e.g., mean endpoint error or EPE of the deformation field) is computable and would directly validate the registration module. The six fusion metrics used (Qabf, VIFF, SF, AG, MG, EI) are quality measures that can improve from increased contrast or sharpness without geometric alignment. Without a direct alignment metric, the claim in Section 5 that B-SR 'improves the registration' is not supported by the quantitative evidence.
  4. [Tables 1-3 and Section 4.2] All quantitative results are reported as single point estimates without standard deviations, confidence intervals, or significance tests across repeated runs or test splits. Some comparisons are very close (e.g., Qabf on DroneVehicle: B-SR 0.56 vs SuperFusion 0.53; VIFF on TNO: B-SR 0.75 vs CDDFuse 0.74). Given that the paper's central claim is consistent superiority, the lack of statistical evidence is a concern that should be addressed, for instance by reporting mean and standard deviation over at least three runs or by performing a paired significance test.
minor comments (4)
  1. [Section 1, Related Work] There are several typographical errors: 'singer or multiple alignment modules' should be 'single or multiple', and the title contains 'Self-Registrationtion' in the running text. Please proofread the manuscript.
  2. [Section 3.4, Eq. (12)] Equation (12) appears to have a missing parenthesis: the denominator 'cc(Fl_T,(Fl_V) + epsilon' is not syntactically well-formed. Please fix the notation.
  3. [Table 4] The column headers in Table 4 ('Ls', 'Lnda', 'JO', 'Lepr', 'SS', 'SD') are not all defined in the main text. 'Ls' and 'SD' are unclear. Please define every abbreviation in the table caption or in the ablation descriptions.
  4. [Algorithm 1] In the line 'Lss(TR, TR, VR'new, TR'new, ...)' the first two arguments appear to be identical; presumably one should be the visible reconstruction. Please correct the pseudocode so it matches the described loss between global and pseudo-global branches.

Circularity Check

1 steps flagged · score 5.0 of 10

The PDG/IPDG self-supervision is a self-consistency constraint that an identity-deformation solution satisfies by construction; it cannot by itself certify spatial alignment, and no direct registration error is reported.

  1. self definitional [Section 3.1 (PDG and IPDG), Eqs. (1)-(2); Section 3.4 Eq. (8); Algorithm 1]
    "Based on the characteristics of deep learning networks, it is known that the result obtained from inputting Inew into the model and then applying an inverse transformation will be the same as the result obtained from inputting I into the model. ... IPDG converts the obtained local differences into pseudo-global differences, which are used to perform global-local difference consistency with the global differences."

    The pseudo-global differences are obtained by applying IPDG to the output of the same registration network on the PDG-transformed input. If the network outputs the identity deformation (no registration), then T_hat=T and V_hat=V; after PDG and IPDG the transformed branch returns exactly the original branch, so the inter-branch consistency loss in Eq. (8) and Algorithm 1 is zero by construction. The loss therefore only enforces equivariance to the synthetic patch transforms, not geometric alignment, and cannot rule out a solution that leaves the input pair misaligned. Since no direct registration error is measured, the claimed self-supervised signal for the alignment task is a self-referential consistency target rather than an independent constraint.

full rationale

The paper's central novelty is the PDG/IPDG self-supervision, but this supervision is defined through the network's own outputs: the pseudo-global target is the inverse-transformed output of the same registration module on a synthetically transformed input. Because an identity-deformation solution satisfies this consistency loss exactly, the loss does not, by itself, provide evidence that spatial alignment has improved. The paper does not report any direct registration error or residual-displacement statistics, so the fusion metrics are the only quantitative evidence, and those metrics do not isolate geometric alignment. This constitutes a partial circularity in the claim that the self-supervised framework 'solves the problem of lacking constraints for the alignment task.' That said, the paper is not wholly circular: the neighborhood dynamic alignment loss Lnda, the edge-pixel retention loss Lepr, and the reconstruction and fusion losses provide external, content-bearing objectives, and the ablation study indicates the self-supervised branch contributes to the reported fusion scores. There is no load-bearing self-citation or imported uniqueness theorem, and the equivariance assumption, though unverified, is an ordinary self-supervised learning premise. The score reflects one central self-referential constraint that reduces to a consistency target by construction, without collapsing the entire method.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on a self-supervised consistency generated from the network's own outputs on transformed inputs, plus edge-based losses on original images. No external alignment ground truth is used. The key hyperparameters (N, mu, w1, w2) are hand-chosen, and the evaluation treats fusion metrics as evidence of alignment quality rather than measuring registration directly.

free parameters (4)
  • PDG patch count N = 2
    Selected as 'an optimal balance' between global and local features based on feature-response analysis and performance (Sections 4.4 and 7.4). It controls the strength of the self-supervision and is not derived from theory.
  • Effective edge threshold mu = 1e-4
    Set in Section 4.1 to define which Sobel edges count as effective in the neighborhood dynamic alignment loss. It is hand-chosen and no sensitivity analysis is reported.
  • Self-supervised loss weights w1, w2 = w1=5, w2=1
    Weights in Equations 5, 6, and 8, stated in Section 4.1. They are hand-chosen and their values are not justified beyond the implementation details.
  • Simulated horizontal shift in RoadScene = 5 pixels
    Section 4.1: visible images are paired with infrared images shifted horizontally by 5 pixels to create misaligned training and testing data. This modeling choice defines the benchmark task and is not itself derived from real sensor statistics.
assumptions (3)
  • domain assumption The registration network is equivariant to PDG's random patch flips, rotations, and stitching, so that applying IPDG to the model output on transformed input equals the model output on the original input.
    Stated in Section 3.1 and 3.2 with reference to feature symmetry invariance in [34,35]. This is not guaranteed for a standard U-Net registration module and is the load-bearing premise of the self-supervision.
  • domain assumption Sobel edge correspondence within a pixel neighborhood is a sufficient cross-modal alignment signal.
    The neighborhood dynamic alignment loss in Section 3.4 assumes that effective edges, computed with a threshold, are reliable features for aligning infrared and visible images within a local search window.
  • domain assumption Fusion-quality metrics (Qabf, VIFF, SF, AG, MG, EI) are appropriate proxies for registration accuracy.
    The paper only reports fusion-quality metrics in Tables 1, 2, 3, 5, and 6 to support the claim of improved registration. It never measures pointwise alignment error against ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion." pith.science (2026). https://pith.science/paper/WCKKXU6C

@misc{pith2026250506920,
  author       = {Pith},
  title        = {Pith review of: Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WCKKXU6C}},
  note         = {Machine review of arXiv:2505.06920}
}
read the original abstract

Acquiring accurately aligned multi-modal image pairs is fundamental for achieving high-quality multi-modal image fusion. To address the lack of ground truth in current multi-modal image registration and fusion methods, we propose a novel self-supervised \textbf{B}i-directional \textbf{S}elf-\textbf{R}egistration framework (\textbf{B-SR}). Specifically, B-SR utilizes a proxy data generator (PDG) and an inverse proxy data generator (IPDG) to achieve self-supervised global-local registration. Visible-infrared image pairs with spatially misaligned differences are aligned to obtain global differences through the registration module. The same image pairs are processed by PDG, such as cropping, flipping, stitching, etc., and then aligned to obtain local differences. IPDG converts the obtained local differences into pseudo-global differences, which are used to perform global-local difference consistency with the global differences. Furthermore, aiming at eliminating the effect of modal gaps on the registration module, we design a neighborhood dynamic alignment loss to achieve cross-modal image edge alignment. Extensive experiments on misaligned multi-modal images demonstrate the effectiveness of the proposed method in multi-modal image alignment and fusion against the competing methods. Our code will be publicly available.

Figures

Figures reproduced from arXiv: 2505.06920 by the authors.

Figure 1
Figure 1. (a) Self-registration paradigm in B-SR. (b) The first column shows infrared and visible image pairs with 5 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Architecture of the B-SR method. (a) The training process of the bi-directional self-registration framework [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparisons of fusion results with a 5-pixel difference in the horizontal direction for " [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparison of results in a DroneVehicle dataset based on drone views. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The analysis of the ablation experiment was conducted using the RoadScene dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Comparison of fusion results on the TNO dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Visualisation of shallow features and deep features after PDG with different parameters [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: The proposed bi-directional self-registration framework is compared with existing single-alignment frame [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Comparison of RoadScene results with misaligned data after dilation and center cropping. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: The results of registration and fusion under unaligned data with different levels of dilation and center [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Percentage of high response areas in the features. [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 40 canonical work pages

  1. [1]

    A survey on region based image fusion methods.Information Fusion, 48:119–132, 2019

    Bikash Meher, Sanjay Agrawal, Rutuparna Panda, and Ajith Abraham. A survey on region based image fusion methods.Information Fusion, 48:119–132, 2019

    work page 2019

  2. [2]

    Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion.IEEE TCSVT, 32(1):105–119, 2021

    Jinyuan Liu, Xin Fan, Ji Jiang, Risheng Liu, and Zhongxuan Luo. Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion.IEEE TCSVT, 32(1):105–119, 2021

    work page 2021

  3. [3]

    Spectral-depth imaging with deep learning based reconstruction.Optics Express, 27(26):38312–38325, 2019

    Mingde Yao, Zhiwei Xiong, Lizhi Wang, Dong Liu, and Xuejin Chen. Spectral-depth imaging with deep learning based reconstruction.Optics Express, 27(26):38312–38325, 2019

    work page 2019

  4. [4]

    Rffnet: Towards robust and flexible fusion for low-light image denoising

    Qiang Wang, Yuning Cui, Yawen Li, Yaping Ruan, Ben Zhu, and Wenqi Ren. Rffnet: Towards robust and flexible fusion for low-light image denoising. InProceedings of the 32nd ACM International Conference on Multimedia, pages 836–845, 2024

    work page 2024

  5. [5]

    Deep material-aware cross-spectral stereo matching

    Tiancheng Zhi, Bernardo R Pires, Martial Hebert, and Srinivasa G Narasimhan. Deep material-aware cross-spectral stereo matching. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1916–1925, 2018

    work page 1916

  6. [6]

    Unsupervised multi-modal medical image registration via discriminator-free image-to- image translation

    ZK Chen, J Wei, and R Li. Unsupervised multi-modal medical image registration via discriminator-free image-to- image translation. InIJCAI. IJCAI Proceedings, 2022

    work page 2022

  7. [7]

    Multimodal mri reconstruction assisted with spatial alignment network.IEEE TMI, 41(9):2499–2509, 2022

    Kai Xuan, Lei Xiang, Xiaoqian Huang, Lichi Zhang, Shu Liao, Dinggang Shen, and Qian Wang. Multimodal mri reconstruction assisted with spatial alignment network.IEEE TMI, 41(9):2499–2509, 2022

    work page 2022

  8. [8]

    Equivariant multi-modality image fusion

    Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Kai Zhang, Shuang Xu, Dongdong Chen, Radu Timofte, and Luc Van Gool. Equivariant multi-modality image fusion. InCVPR, pages 25912–25921, 2024

    work page 2024

Show all 46 references
  1. [9]

    Unified information fusion network for multi-modal rgb-d and rgb-t salient object detection.IEEE TCSVT, 32(4):2091–2106, 2021

    Wei Gao, Guibiao Liao, Siwei Ma, Ge Li, Yongsheng Liang, and Weisi Lin. Unified information fusion network for multi-modal rgb-d and rgb-t salient object detection.IEEE TCSVT, 32(4):2091–2106, 2021

    2021

  2. [10]

    Feature dynamic alignment and refinement for infrared–visible image fusion: Translation robust fusion.Information Fusion, 95:26–41, 2023

    Huafeng Li, Junzhi Zhao, Jinxing Li, Zhengtao Yu, and Guangming Lu. Feature dynamic alignment and refinement for infrared–visible image fusion: Translation robust fusion.Information Fusion, 95:26–41, 2023

    2023

  3. [11]

    Matr: Multimodal medical image fusion via multiscale adaptive transformer.IEEE TIP, 31:5134–5149, 2022

    Wei Tang, Fazhi He, Yu Liu, and Yansong Duan. Matr: Multimodal medical image fusion via multiscale adaptive transformer.IEEE TIP, 31:5134–5149, 2022

    2022

  4. [12]

    Promoting single-modal optical flow network for diverse cross-modal flow estimation

    Shili Zhou, Weimin Tan, and Bo Yan. Promoting single-modal optical flow network for diverse cross-modal flow estimation. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3562–3570, 2022

    2022

  5. [13]

    Darkvisionnet: Low-light imaging via rgb-nir fusion with deep inconsistency prior

    Shuangping Jin, Bingbing Yu, Minhao Jing, Yi Zhou, Jiajun Liang, and Renhe Ji. Darkvisionnet: Low-light imaging via rgb-nir fusion with deep inconsistency prior. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 1104–1112, 2022

    2022

  6. [14]

    Vifnet: An end-to-end visible-infrared fusion network for image dehazing.Neurocomputing, page 128105, 2024

    Meng Yu, Te Cui, Haoyang Lu, and Yufeng Yue. Vifnet: An end-to-end visible-infrared fusion network for image dehazing.Neurocomputing, page 128105, 2024. 11 APREPRINT- SEPTEMBER7, 2025

    2024

  7. [15]

    Infrared and visible image fusion methods and applications: A survey

    Jiayi Ma, Yong Ma, and Chang Li. Infrared and visible image fusion methods and applications: A survey. Information fusion, 45:153–178, 2019

    2019

  8. [16]

    Current advances and future perspectives of image fusion: A comprehensive review.Information Fusion, 90:185–217, 2023

    Shahid Karim, Geng Tong, Jinyang Li, Akeel Qadir, Umar Farooq, and Yiting Yu. Current advances and future perspectives of image fusion: A comprehensive review.Information Fusion, 90:185–217, 2023

    2023

  9. [17]

    Discrimina- tive unsupervised feature learning with exemplar convolutional neural networks.IEEE TPAMI, 38(9):1734–1747, 2016

    Dosovitskiy Alexey, Philipp Fischer, Jost Tobias, Martin Riedmiller Springenberg, and Thomas Brox. Discrimina- tive unsupervised feature learning with exemplar convolutional neural networks.IEEE TPAMI, 38(9):1734–1747, 2016

    2016

  10. [18]

    Sc2-pcr: A second order spatial compatibility for efficient and robust point cloud registration

    Zhi Chen, Kun Sun, Fan Yang, and Wenbing Tao. Sc2-pcr: A second order spatial compatibility for efficient and robust point cloud registration. InCVPR, pages 13221–13231, 2022

    2022

  11. [19]

    Unsupervised multi-modal image registration via geometry preserving image-to-image translation

    Moab Arar, Yiftach Ginger, Dov Danon, Amit H Bermano, and Daniel Cohen-Or. Unsupervised multi-modal image registration via geometry preserving image-to-image translation. InCVPR, pages 13410–13419, 2020

    2020

  12. [20]

    Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration.arXiv preprint arXiv:2205.11876, 2022

    Di Wang, Jinyuan Liu, Xin Fan, and Risheng Liu. Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration.arXiv preprint arXiv:2205.11876, 2022

    2022 arXiv

  13. [21]

    Murf: Mutually reinforcing multi-modal image registration and fusion.IEEE TPAMI, 45(10):12148–12166, 2023

    Han Xu, Jiteng Yuan, and Jiayi Ma. Murf: Mutually reinforcing multi-modal image registration and fusion.IEEE TPAMI, 45(10):12148–12166, 2023

    2023

  14. [22]

    Rfnet: Unsupervised network for mutually reinforcing multi-modal image registration and fusion

    Han Xu, Jiayi Ma, Jiteng Yuan, Zhuliang Le, and Wei Liu. Rfnet: Unsupervised network for mutually reinforcing multi-modal image registration and fusion. InCVPR, pages 19679–19688, 2022

    2022

  15. [23]

    Mri cross-modality image-to- image translation.Scientific reports, 10(1):3753, 2020

    Qianye Yang, Nannan Li, Zixu Zhao, Xingyu Fan, Eric I-Chao Chang, and Yan Xu. Mri cross-modality image-to- image translation.Scientific reports, 10(1):3753, 2020

    2020

  16. [24]

    Ganmcc: A generative adversarial network with multiclassification constraints for infrared and visible image fusion.TIM, 70:1–14, 2020

    Jiayi Ma, Hao Zhang, Zhenfeng Shao, Pengwei Liang, and Han Xu. Ganmcc: A generative adversarial network with multiclassification constraints for infrared and visible image fusion.TIM, 70:1–14, 2020

    2020

  17. [25]

    Rfn-nest: An end-to-end residual fusion network for infrared and visible images.Information Fusion, 73:72–86, 2021

    Hui Li, Xiao-Jun Wu, and Josef Kittler. Rfn-nest: An end-to-end residual fusion network for infrared and visible images.Information Fusion, 73:72–86, 2021

    2021

  18. [26]

    Unsupervised deep image fusion with structure tensor representations.TIP, 29:3845–3858, 2020

    Hyungjoo Jung, Youngjung Kim, Hyunsung Jang, Namkoo Ha, and Kwanghoon Sohn. Unsupervised deep image fusion with structure tensor representations.TIP, 29:3845–3858, 2020

    2020

  19. [27]

    Fusiongan: A generative adversarial network for infrared and visible image fusion.Information Fusion, 48:11–26, 2019

    Jiayi Ma, Wei Yu, Pengwei Liang, Chang Li, and Junjun Jiang. Fusiongan: A generative adversarial network for infrared and visible image fusion.Information Fusion, 48:11–26, 2019

    2019

  20. [28]

    Infrared and visible image fusion via detail preserving adversarial learning.Information Fusion, 54:85–98, 2020

    Jiayi Ma, Pengwei Liang, Wei Yu, Chen Chen, Xiaojie Guo, Jia Wu, and Junjun Jiang. Infrared and visible image fusion via detail preserving adversarial learning.Information Fusion, 54:85–98, 2020

    2020

  21. [29]

    Didfuse: deep image decomposition for infrared and visible image fusion

    Zixiang Zhao, Shuang Xu, Chunxia Zhang, Junmin Liu, Jiangshe Zhang, and Pengfei Li. Didfuse: deep image decomposition for infrared and visible image fusion. InIJCAI, pages 976–976, 2021

    2021

  22. [30]

    Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion

    Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Shuang Xu, Zudi Lin, Radu Timofte, and Luc Van Gool. Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion. InCVPR, pages 5906–5916, 2023

    2023

  23. [31]

    Fusionvae: A deep hierarchical variational autoencoder for rgb image fusion

    Fabian Duffhauss, Ngo Anh Vien, Hanna Ziesche, and Gerhard Neumann. Fusionvae: A deep hierarchical variational autoencoder for rgb image fusion. InECCV, pages 674–691. Springer, 2022

    2022

  24. [32]

    U2fusion: A unified unsupervised image fusion network.IEEE TPAMI, 44(1):502–518, 2020

    Han Xu, Jiayi Ma, Junjun Jiang, Xiaojie Guo, and Haibin Ling. U2fusion: A unified unsupervised image fusion network.IEEE TPAMI, 44(1):502–518, 2020

    2020

  25. [33]

    Ifcnn: A general image fusion framework based on convolutional neural network.Information Fusion, 54:99–118, 2020

    Yu Zhang, Yu Liu, Peng Sun, Han Yan, Xiaolin Zhao, and Li Zhang. Ifcnn: A general image fusion framework based on convolutional neural network.Information Fusion, 54:99–118, 2020

    2020

  26. [34]

    Scalars are universal: Equivariant machine learning, structured like classical physics.NIPS, 34:28848–28863, 2021

    Soledad Villar, David W Hogg, Kate Storey-Fisher, Weichi Yao, and Ben Blum-Smith. Scalars are universal: Equivariant machine learning, structured like classical physics.NIPS, 34:28848–28863, 2021

    2021

  27. [35]

    Imaging with equivariant deep learning: From unrolled network design to fully unsupervised learning.IEEE Signal Processing Magazine, 40(1):134–147, 2023

    Dongdong Chen, Mike Davies, Matthias J Ehrhardt, Carola-Bibiane Schönlieb, Ferdia Sherry, and Julián Tachella. Imaging with equivariant deep learning: From unrolled network design to fully unsupervised learning.IEEE Signal Processing Magazine, 40(1):134–147, 2023

    2023

  28. [36]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMICCAI, pages 234–241. Springer, 2015

    2015

  29. [37]

    Spatial transformer networks.NIPS, 28, 2015

    Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks.NIPS, 28, 2015

    2015

  30. [38]

    V oxelmorph: a learning framework for deformable medical image registration.IEEE TMI, 38(8):1788–1800, 2019

    Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. V oxelmorph: a learning framework for deformable medical image registration.IEEE TMI, 38(8):1788–1800, 2019. 12 APREPRINT- SEPTEMBER7, 2025

    2019

  31. [39]

    Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network.Information Fusion, 82:28–42, 2022

    Linfeng Tang, Jiteng Yuan, and Jiayi Ma. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network.Information Fusion, 82:28–42, 2022

    2022

  32. [40]

    Fusiondn: A unified densely connected network for image fusion

    Han Xu, Jiayi Ma, Zhuliang Le, Junjun Jiang, and Xiaojie Guo. Fusiondn: A unified densely connected network for image fusion. InAAAI, 2020

    2020

  33. [41]

    Progress in color night vision.Optical Engineering, 51(1):010901– 010901, 2012

    Alexander Toet and Maarten A Hogervorst. Progress in color night vision.Optical Engineering, 51(1):010901– 010901, 2012

    2012

  34. [42]

    Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning.IEEE TCSVT, 32(10):6700–6713, 2022

    Yiming Sun, Bing Cao, Pengfei Zhu, and Qinghua Hu. Drone-based rgb-infrared cross-modality vehicle detection via uncertainty-aware learning.IEEE TCSVT, 32(10):6700–6713, 2022

    2022

  35. [43]

    Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5, 2017

    Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5, 2017

    2017 arXiv

  36. [44]

    Reconet: Recurrent correction network for fast and efficient multi-modality image fusion

    Zhanbo Huang, Jinyuan Liu, Xin Fan, Risheng Liu, Wei Zhong, and Zhongxuan Luo. Reconet: Recurrent correction network for fast and efficient multi-modality image fusion. InECCV, pages 539–555. Springer, 2022

    2022

  37. [45]

    Superfusion: A versatile image registration and fusion network with semantic awareness.IEEE/CAA Journal of Automatica Sinica, 9(12):2121–2137, 2022

    Linfeng Tang, Yuxin Deng, Yong Ma, Jun Huang, and Jiayi Ma. Superfusion: A versatile image registration and fusion network with semantic awareness.IEEE/CAA Journal of Automatica Sinica, 9(12):2121–2137, 2022

    2022

  38. [46]

    Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion

    Di Wang, Jinyuan Liu, Long Ma, Risheng Liu, and Xin Fan. Improving misaligned multi-modality image fusion with one-stage progressive dense registration.IEEE TCSVT, 2024. 13 APREPRINT- SEPTEMBER7, 2025 Algorithm 1Pseudocode for B-SR training phase. Input:origin multi-modal imag...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.