pith. sign in

arxiv: 2606.29870 · v1 · pith:AZ5CWDD3new · submitted 2026-06-29 · ⚛️ physics.optics

Ultrasensitive infrared-to-visible artificial vision via self-evolving projection guided by single-pixel detection

Yao Wang , Baolei Liu , Muchen Zhu , Linjun Zhai , Dajing Wang , Zhaohua Yang , Fan Wang This is my paper

Pith reviewed 2026-06-30 05:26 UTC · model grok-4.3

classification ⚛️ physics.optics
keywords infrared-to-visible upconversionsingle-pixel detectionself-evolving projectionphoton-starved imagingartificial visionreal-time visualizationdigital micromirror devicelow-light sensing
0
0 comments X

The pith

A single-pixel infrared detector iteratively guides visible-light projection to reconstruct and display targets at 0.11 photons per pixel per frame.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents SIVIS, a system that combines a single-pixel infrared detector with a digital micromirror device to create a closed feedback loop. The detector measures the infrared response to successive projected patterns, allowing the system to evolve the illumination until the target's shape is reconstructed. At the same time a co-modulated visible beam is projected, making the infrared object directly visible to the naked eye. This occurs in real time and without post-processing even when the infrared flux drops to 0.11 photons per pixel per frame. The approach replaces pixelated infrared cameras with a simpler, lower-cost architecture suited to photon-starved settings such as night vision or tissue imaging.

Core claim

SIVIS achieves sensing and projection without latency under an ultra-low infrared detection limit of 0.11 photons per pixel per frame by integrating self-evolving projection with single-pixel sensing. The system iteratively optimizes illumination patterns with a digital micromirror device based on real-time feedback from a single-pixel infrared detector, enabling autonomous reconstruction of the target's geometric profile while simultaneously projecting a co-modulated visible beam onto the object or an adjacent screen.

What carries the argument

The iterative feedback loop that uses single-pixel infrared measurements to evolve DMD illumination patterns for simultaneous geometric reconstruction and visible projection.

If this is right

  • Real-time upconverted visualization becomes possible in photon-starved conditions without traditional infrared camera arrays.
  • Infrared-encoded anti-counterfeiting features can be decrypted by the same feedback-driven process.
  • Vascular-like structures embedded in biological tissues can be visualized directly.
  • The architecture scales to other low-light sensing tasks by replacing full detector arrays with a single-pixel sensor.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same single-pixel feedback principle could be tested with detectors sensitive to other spectral bands to create analogous visible projections.
  • Because the loop operates without post-processing, it may support continuous tracking of slowly moving targets if the reconstruction speed remains adequate.
  • Power and size savings relative to focal-plane arrays could enable portable or wearable versions of the system for extended low-light use.

Load-bearing premise

The iterative feedback loop from the single-pixel detector can reliably reconstruct the target's geometric profile and co-modulate a visible projection in real time without accumulating errors or requiring post-processing, even at the stated photon-starved levels.

What would settle it

An experiment in which the visible projection visibly deviates from the true target shape or exhibits measurable latency when the infrared input is restricted to 0.11 photons per pixel per frame and no post-processing is applied.

Figures

Figures reproduced from arXiv: 2606.29870 by Baolei Liu, Dajing Wang, Fan Wang, Linjun Zhai, Muchen Zhu, Yao Wang, Zhaohua Yang.

Figure 2
Figure 2. Figure 2: Demonstration of SIVIS with static and dynamic targets in transmission/reflection mode. (a) Schematic diagram of SIVIS in transmissive configuration. The infrared (IR) and visible (VIS) beams are combined by a dichroic mirror (DM), and incident on the DMD simultaneously. The modulated patterns are projected onto the target with a projection lens. Transmitted IR photons are collected by a convex lens, spect… view at source ↗
read the original abstract

Infrared detection and visualization are essential for augmenting human perception across diverse fields, ranging from night vision to industrial inspection and bio-imaging. Conventional infrared cameras are often hindered by high cost, bulky architecture, and complex fabrication requirements. Upconversion sensing systems offer a pixel-free and cost-effective alternative solution by upconverting infrared photons into visible-light signals. However, existing upconversion systems suffer from limitations such as high operating voltages, low quantum efficiency, which prevent their applications in photon-starved environments. Here, we report self-evolving infrared-to-visible upconversion with single-pixel detection (SIVIS) that enables real-time upconverted visualization under photon-starved conditions by integrating self-evolving projection with single-pixel sensing. SIVIS iteratively optimizes illumination patterns with a digital micromirror device based on real-time feedback from a single-pixel infrared detector. This self-evolving process enables the autonomous reconstruction of the target's geometric profile. Simultaneously, it projects a co-modulated visible beam onto the object itself or an adjacent screen, rendering the infrared target directly perceptible to the naked eye in real-time. SIVIS achieves sensing and projection without latency under an ultra-low infrared detection limit of 0.11 photons per pixel per frame (sub-pW -cm2 level) benefited from the high sensitivity. Furthermore, we also validate SIVIS to decrypt infrared-encoded anti-counterfeiting features and visualize vascular-like structures embedded within biological tissues. This photon-feedback-driven artificial vision framework offers a scalable and adaptive solution for ultrasensitive infrared vision, opening promising avenues for night vision, biomedical imaging, and sensing under extreme low-light conditions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The manuscript presents SIVIS, a system that integrates single-pixel infrared detection with iterative DMD-based projection optimization to enable real-time infrared-to-visible upconversion visualization. It claims autonomous reconstruction of target geometry and co-modulated visible projection without latency at an ultra-low detection limit of 0.11 photons per pixel per frame (sub-pW/cm² level), with demonstrations for anti-counterfeiting decryption and vascular imaging in tissue.

Significance. If the performance claims are substantiated with data, the approach could offer a pixel-free, adaptive alternative to conventional IR cameras for photon-starved environments, with relevance to night vision and bio-imaging.

major comments (1)
  1. [Abstract] Abstract: The central claim of real-time operation without latency at 0.11 photons/pixel/frame is stated, but the text provides no iteration count, update rule for DMD patterns, convergence criterion, or explicit noise-handling method for the feedback loop. At this flux, Poisson statistics yield SNR ≪ 1 per measurement, so the absence of these details leaves the reliability of reconstruction and zero-latency assertion unevaluable.
minor comments (1)
  1. [Abstract] The notation 'sub-pW -cm2 level' contains a spacing/formatting inconsistency and should be written consistently as sub-pW cm^{-2}.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the detailed and constructive review. We address the single major comment below and are willing to revise the manuscript to improve clarity.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim of real-time operation without latency at 0.11 photons/pixel/frame is stated, but the text provides no iteration count, update rule for DMD patterns, convergence criterion, or explicit noise-handling method for the feedback loop. At this flux, Poisson statistics yield SNR ≪ 1 per measurement, so the absence of these details leaves the reliability of reconstruction and zero-latency assertion unevaluable.

    Authors: The abstract is necessarily concise, but we agree that the absence of key methodological parameters makes the central performance claims difficult to evaluate from the abstract alone. The full manuscript (Methods section) specifies the iterative DMD optimization, which employs a gradient-based update rule driven by the single-pixel detector feedback, a convergence criterion based on stabilization of the reconstructed profile (typically 30–80 iterations), and explicit noise mitigation via multi-frame temporal averaging combined with adaptive thresholding to handle Poisson-dominated statistics. The zero-latency claim refers to the continuous, on-the-fly projection update without offline reconstruction or buffering delays. We will revise the abstract to include a concise statement of these elements so that the performance claims can be properly assessed. revision: yes

Circularity Check

0 steps flagged

No circularity: experimental hardware demonstration with no derivation chain

full rationale

The paper presents an experimental system (SIVIS) that uses DMD pattern optimization driven by single-pixel IR feedback to reconstruct geometry and project visible light. No equations, fitted parameters, uniqueness theorems, or mathematical derivations are claimed or present in the abstract or described methods. All performance claims (0.11 photons/pixel/frame, real-time operation) rest on hardware measurements and iterative feedback rather than any reduction of a predicted quantity to its own inputs by construction. Self-citations, if present, are not load-bearing for any derivation. The work is therefore self-contained as an empirical demonstration.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is an experimental demonstration; the central claim rests on the assumption that the feedback algorithm converges accurately at the stated photon flux and that the optical components function as described. No free parameters, axioms, or invented entities are extractable from the abstract alone.

pith-pipeline@v0.9.1-grok · 5844 in / 1213 out tokens · 25647 ms · 2026-06-30T05:26:14.927426+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references

  1. [1]

    Introduction The spectral sensitivity of the human retina is intrinsically confined to the visible (VIS) regime (400 –700 nm), rendering the information -rich infrared (IR) spectrum imperceptible to the naked eye . Despite its invisibility, the IR band carries critical thermal and chemical signatures, fueling a surge of interest in fields ranging from nig...

  2. [2]

    butterfly

    Results The conceptual framework of the proposed SIVIS is illustrated in Fig. 1a, in which the imperceptible IR target is visualized by projecting visible patterns that mirror the target’s geometry onto either the target surface or a display screen , enabling direct observation by the naked eye or a standard silicon-based camera. Unlike conventional infra...

    1951

  3. [3]

    Discussion In summary, we have developed SIVIS, a self-evolving infrared-to-visible protype, to upconvert IR vision to visible light projections, based on the single -pixel detection feedback and evolved illumination patterns. In contrast to existing infrared-to-visible methods, which face tremendous challenges in photon-starving conditions due to the int...

  4. [4]

    2 a and Supplementary Fig

    Methods 4.1 Experimental setup All experiments were conducted with the purpose -built SIVIS (Fig. 2 a and Supplementary Fig. S2). The IR and VIS lasers are a polarization-maintaining single- mode continuous-wave 852 nm laser (MDL-III-852L, CNI) and a free-space output 532 nm laser (MGL-FN-532-500 Mw, CNI). The collimated IR and VIS laser beams have diamet...

  5. [5]

    Kä llhammer, Imaging: The road ahead for car night-vision

    J.-E. Kä llhammer, Imaging: The road ahead for car night-vision. Nat. Photonics 5, 12–13 (2006)

    2006

  6. [6]

    F. Bao, X. Wang, S. H. Sureshbabu, G. Sreekumar, L. Yang, V. Aggarwal, V. N. Boddeti, Z. Jacob, Heat -assisted detection and ranging. Nature 619, 743–748 (2023)

    2023

  7. [7]

    D. Y. Kim, D. W. Song, N. Chopra, P. De Somer, F. So, Organic infrared upconversion device. Adv. Mater. 22, 2260–2263 (2010)

    2010

  8. [8]

    Y. Chen, S. Wang, F. Zhang, Near -infrared luminescence high-contrast in vivo biomedical imaging. Nat. Rev. Bioeng. 1, 60–78 (2023)

    2023

  9. [9]

    C. Li, G. Chen, Y. Zhang, F. Wu, Q. Wang, Advanced fluorescence imaging technology in the near -infrared-II window for biomedical applications. J. Am. Chem. Soc. 142, 14789–14804 (2020)

    2020

  10. [10]

    Z. Deng, L. Li, P. Tang, C. Jiao, Z. Z. Yu, C. M. Koo, H. Bin Zhang, Controllable surface-grafted MXene inks for electromagnetic wave modulation and infrared anti-counterfeiting applications. ACS Nano 16, 16976–16986 (2022)

    2022

  11. [11]

    Pasquini, Near infrared spectroscopy: A mature analytical technique with new perspectives – A review

    C. Pasquini, Near infrared spectroscopy: A mature analytical technique with new perspectives – A review. Anal. Chim. Acta 1026, 8–36 (2018)

    2018

  12. [12]

    B. Xu, C. Jin, J. S. Park, H. Liu, X. Lin, J. Cui, D. Chen, J. Qiu, Emerging near- infrared luminescent materials for next -generation broadband optical communications. InfoMat 6, 1–30 (2024)

    2024

  13. [13]

    Y. Li, J. Ibanez -Guzman, Lidar for autonomous driving: The principles, challenges, and trends for automotive lidar and perception systems. IEEE Signal Process. Mag. 37, 50–61 (2020)

    2020

  14. [14]

    M. Tian, B. Liu, Z. Lu, Y. Wang, Z. Zheng, J. Song, X. Zhong, F. Wang, Miniaturized on-chip spectrometer enabled by electrochromic modulation. Light Sci. Appl. 13, 278 (2024)

    2024

  15. [15]

    J. Chen, J. Tao, D. Ban, M. G. Helander, Z. Wang, J. Qiu, Z. Lu, Hybrid organic/inorganic optical up-converter for pixel-less near-infrared imaging. Adv. Mater. 24, 3138–3142 (2012)

    2012

  16. [16]

    Valencia Molina, R

    L. Valencia Molina, R. Camacho Morales, J. Zhang, R. Schiek, I. Staude, A. A. Sukhorukov, D. N. Neshev, Enhanced infrared vision by nonlinear up - conversion in nonlocal metasurfaces. Adv. Mater. 36, 2402777 (2024)

    2024

  17. [17]

    T. Rao, M. Chen, G. Mu, X. Tang, Infrared -to-visible upconversion devices. Coatings 12, 456 (2022)

    2022

  18. [18]

    X. I. N. Hu, M. Ren, X. I. Luo, Y. Lu, Q. Chen, Infrared-to-visible upconversion imagers: Recent advances and future trends. IEEE Open J. Immersive Displays 1, 107–118 (2024)

    2024

  19. [19]

    A. Barh, P. J. Rodrigo, L. Meng, C. Pedersen, P. Tidemand -Lichtenberg, Parametric upconversion imaging and its applications. Adv. Opt. Photonics 11, 952–1019 (2019)

    2019

  20. [20]

    Camacho-Morales, D

    R. Camacho-Morales, D. Rocco, L. Xu, V. F. Gili, N. DImitrov, L. Stoyanov, Z. Ma, A. Komar, M. Lysevych, F. Karouta, A. Dreischuh, H. H. Tan, G. Leo, C. De Angelis, C. Jagadish, A. E. Miroshnichenko, M. Rahmani, D. N. Neshev, Infrared upconversion imaging in nonlinear metasurfaces. Adv. Photonics 3, 36002 (2021)

    2021

  21. [21]

    Y. Wang, K. Huang, J. Fang, M. Yan, E. Wu, H. Zeng, Mid-infrared single-pixel imaging at the single-photon level. Nat. Commun. 14, 1073 (2023)

    2023

  22. [22]

    M. Wu, D. N. Congreve, M. W. B. Wilson, J. Jean, N. Geva, M. Welborn, T. Van Voorhis, V. Bulovic, M. G. Bawendi, M. A. Baldo, Solid-state infrared-to- visible upconversion sensitized by colloidal nanocrystals. Nat. Photonics 10, 31– 34 (2016)

    2016

  23. [23]

    Bharmoria, H

    P. Bharmoria, H. Bildirir, K. Moth -Poulsen, Triplet -triplet annihilation based near infrared to visible molecular photon upconversion. Chem. Soc. Rev. 49, 6529–6554 (2020)

    2020

  24. [24]

    Y. Ma, Y. Chen, S. Wang, Z. H. Chen, Y. Zhang, L. Huang, X. Zhang, F. Yin, Y. Wang, M. Yang, Z. Li, K. Huang, X. Fang, Z. Li, M. Wang, W. Liu, J. N. Li, L. Li, H. Zhao, M. Wei, Y. Shi, R. Liu, M. Zhang, J. Chen, J. Shen, J. Meng, Y. Yang, F. Zhang, X. Gong, G. Han, T. Xue, Near -infrared spatiotemporal color vision in humans enabled by upconversion contac...

    2025

  25. [25]

    B. Liu, C. Chen, X. Di, J. Liao, S. Wen, Q. P. Su, X. Shan, Z. Q. Xu, L. A. Ju, C. Mi, F. Wang, D. Jin, Upconversion nonlinear structured illumination microscopy. Nano Lett. 20, 4775–4781 (2020)

    2020

  26. [26]

    Y. Wang, B. Liu, L. Ding, C. Chen, X. Shan, D. Wang, M. Tian, J. Song, Z. Zheng, X. Xu, X. Zhong, F. Wang, Multi -photon super-linear image scanning microscopy using upconversion nanoparticles. Laser Photon. Rev. 18, 2400746 (2024)

    2024

  27. [27]

    Y. Ma, J. Bao, Y. Zhang, Z. Li, X. Zhou, C. Wan, L. Huang, Y. Zhao, G. Han, T. Xue, Mammalian near -infrared image vision through injectable and self - powered retinal nanoantennae. Cell 177, 243–255 (2019)

    2019

  28. [28]

    D. Wang, Q. Cheng, W. Xia, Y. Tan, K. Yu, L. Zhang, G. Li, R. Liu, W. Wang, L. Xu, S. Wu, Y. Liu, Y. Wang, H. Zhou, Ultralow -threshold upconversion infrared vision via a microsphere-mediated directional photofield. Nat. Commun. 16, 5064 (2025)

    2025

  29. [29]

    H. Yu, D. Kim, J. Lee, S. Baek, J. Lee, R. Singh, F. So, High -gain infrared-to- visible upconversion light -emitting phototransistors. Nat. Photonics 10, 129 – 134 (2016)

    2016

  30. [30]

    C. J. Shih, Y. Z. Li, M. Z. Li, S. Biring, B. C. Huang, C. W. Liu, T. H. Yeh, D. Luo, J. H. Lee, Y. H. Huang, K. T. Wong, S. W. Liu, Transparent organic upconversion device targeting high - grade infrared visual image. Nano Energy 86, 106043 (2021)

    2021

  31. [31]

    H. Luo, D. Ban, H. C. Liu, P. J. Poole, M. Buchanan, Pixelless imaging device using optical up-converter. IEEE Electron Device Lett. 25, 129–131 (2004)

    2004

  32. [32]

    Zheng, P

    W. Zheng, P. Huang, Z. Gong, D. Tu, J. Xu, Q. Zou, R. Li, W. You, J. C. G. Bü nzli, X. Chen, Near-infrared-triggered photon upconversion tuning in all - inorganic cesium lead halide perovskite quantum dots. Nat. Commun. 9, 3462 (2018)

    2018

  33. [33]

    S. W. Liu, C. C. Lee, C. H. Yuan, W. C. Su, S. Y. Lin, W. C. Chang, B. Y. Huang, C. F. Lin, Y. Z. Lee, T. H. Su, K. T. Chen, Transparent organic upconversion devices for near-infrared sensing. Adv. Mater. 27, 1217–1222 (2015)

    2015

  34. [34]

    Z. Shi, H. Ding, H. Hong, D. Cheng, K. Rajabi, J. Yang, Y. Wang, L. Wang, Y. Luo, K. Liu, X. Sheng, Ultrafast and low -power optoelectronic infrared -to- visible upconversion devices. Photonics Res. 7, 1161 (2019)

    2019

  35. [35]

    W. Zhou, Y. Shang, F. P. Garcí a de Arquer, K. Xu, R. Wang, S. Luo, X. Xiao, X. Zhou, R. Huang, E. H. Sargent, Z. Ning, Solution -processed upconversion photodetectors based on quantum dots. Nat. Electron. 3, 251–258 (2020)

    2020

  36. [36]

    Bruschini, H

    C. Bruschini, H. Homulle, I. M. Antolovic, S. Burri, E. Charbon, Single-photon avalanche diode imagers in biophotonics: review and outlook. Light Sci. Appl. 8, 87 (2019)

    2019

  37. [37]

    M. P. Edgar, G. M. Gibson, M. J. Padgett, Principles and prospects for single - pixel imaging. Nat. Photonics 13, 13–20 (2019)

    2019

  38. [38]

    B. Liu, F. Wang, C. Chen, F. Dong, D. McGloin, Self -evolving ghost imaging. Optica 8, 1340–1349 (2021)

    2021

  39. [39]

    J. Song, B. Liu, Y. Wang, C. Chen, X. Shan, X. Zhong, L. -A. Wu, F. Wang, Computational and dark-field ghost imaging with ultraviolet light. Photonics Res. 12, 226–234 (2024)

    2024

  40. [40]

    D. Wang, B. Liu, J. Song, Y. Wang, X. Shan, X. Zhong, F. Wang, Dual -mode adaptive-SVD ghost imaging. Opt. Express 31, 14225–14239 (2023)

    2023

  41. [41]

    M. Zhu, B. Liu, Y. Wang, L. Zhai, J. Song, N. Liu, Z. Yang, L. Ding, F. Wang, Physics-informed neural network enhanced multispectral single -pixel imaging with a chip spectral detector. Opt. Laser Technol. 194, 114430 (2026)

    2026

  42. [42]

    J. Li, W. Zhang, W. Ji, J. Wang, N. Wang, W. Wu, Q. Wu, X. Hou, W. Hu, L. Li, Near infrared photothermal conversion materials: mechanism, preparation, and photothermal cancer therapy applications. J. Mater. Chem. B 9, 7909–7926 (2021)

    2021

  43. [43]

    A. V. Paterova, S. M. Maniam, H. Yang, G. Grenci, L. A. Krivitsky, Hyperspectral infrared microscopy with visible light. Sci. Adv. 6, eabd0460 (2020)

    2020

  44. [44]

    M. Wu, T. A. Lin, J. O. Tiepelt, V. Bulović, M. A. Baldo, Nanocrystal-sensitized infrared-to-visible upconversion in a microcavity under subsolar flux. Nano Lett. 21, 1011–1016 (2021)

    2021

  45. [45]

    S. Song, Y. Wang, Y. Zhao, S. Fu, Article Molecular engineering of AIE luminogens for NIR-II / IIb bioimaging and surgical navigation of lymph nodes Molecular engineering of AIE luminogens for NIR -II / IIb bioimaging and surgical navigation of lymph nodes. Matter 5, 2847–2863 (2022)

    2022

  46. [46]

    M. H. Choi, W. Han, K. Min, D. Min, G. Han, K. S. Shin, M. Kim, J. H. Park, Recent applications of optical elements in augmented and virtual reality displays: A review. ACS Appl. Opt. Mater. 2, 1247–1268 (2024)

    2024

  47. [47]

    R. I. Stantchev, B. Sun, S. M. Hornett, P. A. Hobson, G. M. Gibson, M. J. Padgett, E. Hendry, Noninvasive, near-field terahertz imaging of hidden objects using a single-pixel detector. Sci. Adv. 2, 1600190 (2016)

    2016

  48. [48]

    T. Wang, S. Y. Ma, L. G. Wright, T. Onodera, B. C. Richard, P. L. McMahon, An optical neural network using less than 1 photon per multiplication. Nat. Commun. 13, 123 (2022)

    2022