Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Intelligent Sensing-to-Action for Robust Autonomy at the Edge: Opportunities and Challenges

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A perspective argues that autonomy can thrive by sensing only 8–10% of a scene and reconstructing the rest, cutting sensing-and-reconstruction energy by 9.11× while preserving detection accuracy.

desk verdict A useful synthesis of the group's prior work into a 'sensing-to-action' framework, but the headline energy numbers overstate what has been demonstrated. read the letter →

arxiv 2502.02692 v1 pith:KY5C6Q6R submitted 2025-02-04 cs.RO cs.CVcs.LG

Amit Ranjan Trivedi , Sina Tayebati , Hemant Kumawat , Nastaran Darabi , Divake Kumar , Adarsh Kumar Kosta , Yeshwanth Venkatesha , Dinithi Jayasuriya
show 4 more authors
Nethmi Jayasinghe Priyadarshini Panda Saibal Mukhopadhyay Kaushik Roy
This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords sensing-to-actionloopgenerativesensingLiDARKoopmanoperatorneuromorphiccomputingfederatedlearningedgeautonomyreliabilitymonitoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This perspective article argues that the sensing-to-action loop—the closed cycle in which a robot senses, processes, and acts—can be made dramatically more efficient by adapting sensing and computation to the task and environment. The authors claim that a system does not need to sense everything: generative models can reconstruct unobserved regions, so a LiDAR can actively sample only 8–10% of a scene and still support object detection. They further claim that control actions can guide sensing through Koopman-based representations, that reliability can be maintained with lightweight anomaly monitoring, and that multi-agent coordination plus neuromorphic, event-driven processing can cut energy several-fold. If these claims hold at deployment scale, edge autonomy could shed much of its sensing and compute overhead while keeping real-time responsiveness.

What carries the argument

The central object is the sensing-to-action loop defined as the cyclic coupling of sensing, learning, and actuation modules. Three mechanisms carry the argument: R-MAE's range-aware radial masking, which samples a distance-dependent subset of LiDAR voxels and lets a sparse 3D encoder plus occupancy decoder reconstruct the unobserved scene; the contrastive spectral Koopman operator, which maps visual observations into a complex-valued linear embedding space whose stable eigenvalues let a Linear Quadratic Regulator derive control with few interactions; and STARNet's likelihood-regret metric, which uses a variational autoencoder over intermediate features to flag inputs that require large distributional adjustments. These are unified by the claim that neuromorphic, event-driven representations naturally fuse sensing and computation.

What would settle it

Run an autonomous vehicle in heavy snow or with a misaligned LiDAR while sensing only 8–10% of the scene; if object-detection precision falls below the full-scan baseline by more than the safety margin despite R-MAE and STARNet, the central energy-savings claim fails. Also measure end-to-end energy of a real robot loop; if combined sensing-plus-reconstruction energy is not close to nine times lower than full sensing, the headline figure is not representative.

Watch

Extended reading notes

Core claim

The paper's central claim is that the tight, bidirectional coupling in sensing-to-action loops is an opportunity, not just a constraint. It reports that generative sensing with Radially Masked Autoencoding (R-MAE) lets an autonomous system reconstruct the bulk of a LiDAR point cloud from under 10% of actively sensed voxels, cutting combined sensing-plus-reconstruction energy by 9.11× while sustaining object detection accuracy on KITTI, Waymo, and nuScenes. It reports that a contrastive spectral Koopman encoder can condition visual representations on control objectives, learning linear dynamics with fewer interactions; that a likelihood-regret monitor (STARNet) detects corrupted sensor streams and restores detection accuracy; and that event-driven spiking networks plus federated, hardware-aware co-design cut energy and latency in distributed settings by about threefold. The authors present these as evidence that end-to-end co-design of sensing, computation, and actuation, rather than component-level optimization, is the route to robust energy-efficient autonomy.

Load-bearing premise

The quantitative gains assume that benchmark results on KITTI, Waymo, nuScenes, and CIFAR-10 transfer to real dynamic deployments without unacceptable accuracy loss when reconstruction quality or distribution shifts degrade.

Editorial extensions

If this is right

  • If generative sensing generalizes, LiDAR-equipped robots could operate with as little as 8–10% active sensing, cutting the dominant energy cost of depth perception.
  • Action-to-sensing conditioning via Koopman embeddings could let robots adjust sensor focus, field of view, or sampling rate based on control goals, reducing redundant data acquisition.
  • STARNet-style monitoring could filter corrupted or out-of-distribution sensor streams at the edge, keeping feedback loops stable under weather, crosstalk, or sensor degradation.
  • Multi-agent hardware-aware federated learning (DC-NAS and HaLo-FL) could distribute sensing and computation across heterogeneous devices with about threefold lower energy on vision tasks.
  • Neuromorphic event-driven optical flow (Adaptive-SpikeNet and Fusion-FlowNet) shows competitive accuracy with roughly tenfold lower energy, pointing to spike-based loops as a path for low-latency control.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 9.11× figure counts sensing plus reconstruction; if reconstruction runs on more powerful hardware or fails on rare objects, the trade-off narrows. A testable extension is measuring task-relevant accuracy, not just reconstruction, under distribution shift while varying the sensing mask ratio.
  • The radial-masking principle could transfer to radar and acoustic sensing, where energy also scales with range; the paper mentions this possibility but does not quantify it.
  • The Koopman action-to-sensing loop could be merged with STARNet so that the monitor's confidence modulates both sensing aggressiveness and control authority, closing the reliability loop adaptively.
  • The multi-agent results on CIFAR-10 need demonstration on actual sensing-action loops; if the threefold saving persists there, federated edge autonomy becomes substantially more plausible.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This perspective article argues that closed-loop 'sensing-to-action' designs can make edge autonomy more energy-efficient and responsive by sensing only task-relevant parts of the environment and reconstructing the rest, by using control feedback (e.g., Koopman representations) to steer sensing, by monitoring sensor-feature reliability (STARNet), by employing neuromorphic SNN/ANN processing, and by coordinating multi-agent federated learning and hardware-aware model compression. The main quantitative support is a 9.11x lower sensing-plus-reconstruction energy for R-MAE on LiDAR, a threefold energy reduction in distributed settings, and accuracy restoration of about 15% under corruptions. The paper positions these as evidence for a broader co-design agenda for edge robotics and autonomous vehicles.

Significance. The paper offers a useful synthesis and a coherent research agenda, and it deserves credit for clearly naming the reliability and cascading-error risks of closed-loop adaptation, for providing concrete architectural diagrams, and for grounding each proposed direction in specific prior results rather than hand-waving. If the quantitative claims held at system level, the impact would be substantial for edge autonomy. However, the two headline quantitative claims (9.11x and threefold) are not yet shown to hold end-to-end: the first omits downstream detection energy, and the second is demonstrated on CIFAR-10 image classification, not on a sensing-action loop. The paper also relies heavily on the authors' own prior work, with no independent validation, so the evidence base is narrower than the text suggests. These are fixable within the scope of the paper by re-labeling claims and adding caveats.

major comments (4)
  1. [Sec. III, Table II] The paper reports a 9.11x energy saving by comparing 72 mJ conventional sensing per scan with 792 uJ plus 7.1 mJ for R-MAE. Because the task in Table I is 3D object detection, both pipelines must also execute a detection network (e.g., SECOND or PV-RCNN) on the dense reconstructed volume (R-MAE) or on the full point cloud (conventional). Adding detector energy E_det to both arms changes the ratio to (72 + E_det)/(7.892 + E_det) mJ, which falls to 1.59x for E_det=100 mJ and to 1.25x for E_det=250 mJ. The 9.11x figure should be labeled 'sensing-plus-reconstruction vs sensing-only' or, preferably, be replaced by an end-to-end energy comparison that includes the detector and any reconstruction-to-dense-grid conversion.
  2. [Sec. III, R-MAE masking paragraph] The text says R-MAE addresses LiDAR's energy-accuracy-range trade-offs 'without hardware modifications' and notes that long-range measurements scale with the fourth power of distance. Yet Table II reduces energy per laser pulse from 50 uJ to 5.5 uJ. For a diffuse target, maximum range scales as the fourth root of pulse energy, so this reduction shortens maximum range by roughly 42% unless aperture, detector sensitivity, or other optical parameters improve. The claim of no hardware modifications is therefore not established; the paper needs a range-aware analysis or a qualification that the reduced pulse energy applies at a shorter range or requires receiver-side improvements.
  3. [Sec. VII, Fig. 11 and Sec. VIII] The conclusion states that 'multi-agent sensing-to-action loops ... can achieve a threefold reduction in energy consumption,' but Fig. 11 reports DC-NAS/HaLo-FL results on CIFAR-10 image classification in a federated learning setup. This is a static vision benchmark, not a closed-loop sensing-action task with state feedback, actuation, or temporal dynamics. The threefold number should be reported as a federated-learning image-classification result and its transferability to sensing-action loops stated as an open hypothesis, not a demonstrated property.
  4. [Sec. III and Sec. V] The paper presents generative sensing as a key enabler for robust autonomy, but all R-MAE accuracy numbers in Table I are from clean KITTI/Waymo/nuScenes benchmarks. No evaluation is shown under the very corruptions and sensor failures that motivate STARNet (snow, beam missing, crosstalk, cross-sensor interference), nor is the effect of reconstruction error on downstream detection accuracy discussed quantitatively. Since the central claim depends on reconstructing 90% of the scene from sparse sensing, the behavior of generative sensing under distribution shift should be quantified in Sec. III or explicitly identified as an open robustness problem in Sec. VIII.
minor comments (6)
  1. [Abstract] The phrase 'enabling optimized resource allocation, reduce communication latency, and faster responsiveness' mixes a participle with bare verbs; revise for parallel structure.
  2. [Sec. VI] The sentence 'bringing artificial an biological systems closer' should read 'artificial and biological systems.'
  3. [Sec. III vs. Sec. VIII] The conclusion says 'only 8% of the environment needs to be actively sensed' while Sec. III states '8-10%' and '90% of the scene masked'; unify the active-sensing fraction.
  4. [Throughout] Section references are inconsistent: Sec. II uses 'Sec. 3' and 'Sec. 4' while later sections use 'Sec. III' and 'Sec. VII'; use a single citation style consistently.
  5. [Fig. 5] Fig. 5(b) shows performance under external disturbances, but the axes are not described in the caption; please make the figure self-contained.
  6. [Secs. V and VIII] The monitoring framework is spelled both 'STARNet' (Sec. V, Fig. 6) and 'STAR-Net' (Sec. VIII); unify the spelling.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a perspective that supports its claims with externally benchmarked prior results rather than reducing predictions to their inputs.

full rationale

I found no circular step that meets the required standard. The paper is an expository perspective article: it asserts that sensing-to-action loops can improve edge autonomy and illustrates this with generative sensing (R-MAE), Koopman-based action-to-sensing (RoboKoop), reliability monitoring (STARNet), neuromorphic optical-flow networks, and federated multi-agent methods (DC-NAS, HaLo-FL). In each case the paper cites the authors' own prior work, but that prior work was evaluated on external benchmarks (KITTI, Waymo, nuScenes, MVSEC, CIFAR-10), so the citations are independent evidence rather than a self-citation chain that makes the conclusion true by construction. The central claims are not defined in terms of their conclusions: 'generative sensing' is not defined as 'whatever achieves 9.11x savings,' and the 8-10% active-sensing figure is a design/masking parameter evaluated against detection accuracy, not a fitted parameter renamed as a prediction. The closest candidate for concern is the Sec. III energy comparison, where conventional LiDAR counts sensing energy only while R-MAE counts sensing plus reconstruction and neither arm includes downstream detector energy; that is an accounting/boundary issue that could weaken the headline quantitative claim, but it is not circular. No uniqueness theorem is imported from the authors, no ansatz is smuggled in solely by citation, and no known result is merely renamed. The paper does not present a formal derivation whose output equals its input, so the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper is a perspective; it introduces no new free parameters or invented entities. Its central claims rest on domain assumptions and on borrowed quantitative results from the authors' own prior work. The free parameters listed below are inherited from that prior work and determine the headline efficiency numbers.

free parameters (2)
  • Scene sensing ratio (radial mask fraction) = 8-10%
    Chosen in R-MAE [3] to balance reconstruction quality and energy; the paper's headline energy savings depend on it.
  • Energy per LiDAR pulse (conventional vs R-MAE) = 50 µJ vs 5.5 µJ
    Model-based values from [3] used in Table II to compute the 9.11x reduction; not experimentally validated in this paper.
assumptions (4)
  • domain assumption The sensing-to-action loop is the right abstraction for edge autonomy.
    Adopted in Sec. I and used to structure the whole perspective; not argued for against alternative architectures.
  • domain assumption Koopman operators can linearize control-relevant visual dynamics with a small set of modes.
    Used in Sec. IV (RoboKoop) to justify action-to-sensing efficiency; if the dynamics are not well captured, the efficiency claim collapses.
  • domain assumption Event-driven neuromorphic processing is energy- and latency-efficient for the considered optical flow and detection tasks.
    Assumed in Sec. VI; energy numbers are from prior chips and simulations, not measured on the full sensing-action loop.
  • domain assumption The authors' prior experimental results are correct and transferable.
    All quantitative claims depend on the correctness and generality of [3], [18], [34], [49], [51], [76], [77], none of which are independently verified in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Intelligent Sensing-to-Action for Robust Autonomy at the Edge: Opportunities and Challenges." pith.science (2026). https://pith.science/paper/KY5C6Q6R

@misc{pith2026250202692,
  author       = {Pith},
  title        = {Pith review of: Intelligent Sensing-to-Action for Robust Autonomy at the Edge: Opportunities and Challenges},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KY5C6Q6R}},
  note         = {Machine review of arXiv:2502.02692}
}
read the original abstract

Autonomous edge computing in robotics, smart cities, and autonomous vehicles relies on the seamless integration of sensing, processing, and actuation for real-time decision-making in dynamic environments. At its core is the sensing-to-action loop, which iteratively aligns sensor inputs with computational models to drive adaptive control strategies. These loops can adapt to hyper-local conditions, enhancing resource efficiency and responsiveness, but also face challenges such as resource constraints, synchronization delays in multi-modal data fusion, and the risk of cascading errors in feedback loops. This article explores how proactive, context-aware sensing-to-action and action-to-sensing adaptations can enhance efficiency by dynamically adjusting sensing and computation based on task demands, such as sensing a very limited part of the environment and predicting the rest. By guiding sensing through control actions, action-to-sensing pathways can improve task relevance and resource use, but they also require robust monitoring to prevent cascading errors and maintain reliability. Multi-agent sensing-action loops further extend these capabilities through coordinated sensing and actions across distributed agents, optimizing resource use via collaboration. Additionally, neuromorphic computing, inspired by biological systems, provides an efficient framework for spike-based, event-driven processing that conserves energy, reduces latency, and supports hierarchical control--making it ideal for multi-agent optimization. This article highlights the importance of end-to-end co-design strategies that align algorithmic models with hardware and environmental dynamics and improve cross-layer interdependencies to improve throughput, precision, and adaptability for energy-efficient edge autonomy in complex environments.

Figures

Figures reproduced from arXiv: 2502.02692 by the authors.

Figure 1
Figure 1. Opportunities for Intelligent Sensing-to-Action: In sensing-to￾action loops, significant gains can be achieved by selectively sensing critical environmental regions while predicting less critical areas based on training data. This frugal sensing strategy is especially beneficial for resource-intensive modalities, such as LiDAR, enhancing task accuracy without unnecessary overhead. Similarly, action-to-sensing optimi… view at source ↗
Figure 2
Figure 2. An end-to-end computing pipeline comparison sensing-processing-action loop between a biological and a neuromorphic system. In a biological system, inputs are perceived as changes in intensity (events and frames) and color (frames) by the eye. In contrast, a neuromorphic system uses frame cameras to capture analog intensity at low rates and event cameras to detect motion-induced variations, generating events. The bra… view at source ↗
Figure 3
Figure 3. Generative Sensing: Sense only what you really need: Generative sensing optimizes resource use by focusing on essential environmental features, reducing unnecessary data collection and enhancing real-time responsiveness. For LiDAR proessing, in this approach, the input point cloud is voxelized and radially masked based on voxel distance from the sensor to minimize redundant information. A 3D spatially sparse convolu… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Our approach conditions visual representations on the task policy [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 7
Figure 7. Figure 7: Object Detection Accuracy for KITTI Dataset: The VAE-based approach analyzes LiDAR point clouds and object labels, producing bounding boxes for cars, pedestrians, and cyclists. The network was tested under challenging conditions, such as varying snow intensities and ot…
Figure 8
Figure 8. Figure 8: Neuromorphic sensing-action loop architectures. a) Full-ANN [48], Full-SNN [49], and Hybrid SNN-ANN [50] models for optical flow estimation using event data. b) Fusion-FlowNet [51] integrates event-based and frame-based modalities for enhanced feature extraction. Outpu…
Figure 9
Figure 9. Figure 9: Average Endpoint Error (AEE) comparison for Optical flow estimation on the MVSEC [66] dataset. The left shows the Average Endpoint Error (AEE) for baseline models, EvFlow-Net (EvF) [48], Spike-FlowNet (SpF) [50], and Fusion-FlowNet (FF) [51]). The right showcases how A…
Figure 10
Figure 10. Figure 10: Key aspects of dynamic multi-agent systems: resource heterogeneity, [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Performance comparison of DC-NAS and HaLo-FL on the CIFAR-10 [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Edge Intelligence with Spiking Neural Networks

    cs.DC 2025-07 conditional novelty 4.0 of 10

    A comprehensive review of spiking neural networks for edge computing, covering neuron models, learning algorithms, hardware, deployment, security, and evaluation, with a claim to be the first survey on this specific i...

Reference graph

Works this paper leans on

78 extracted references · 68 canonical work pages · cited by 1 Pith paper

  1. [1]

    Lidar–a new (self-driving) vehicle for introducing optics to broader engineering and non-engineering audiences,

    C. Rablau, “Lidar–a new (self-driving) vehicle for introducing optics to broader engineering and non-engineering audiences,” in Education and training in optics and photonics . Optica Publishing Group, 2019, p. 11143 138

    work page 2019

  2. [2]

    Long-range, high-resolution camera optical design for assisted and autonomous driving,

    F. E. Sahin, “Long-range, high-resolution camera optical design for assisted and autonomous driving,” in photonics. MDPI

    work page

  3. [3]

    Sense Less, Generate More: Pre-training LiDAR Perception with Masked Autoencoders for Ultra-Efficient 3D Sensing

    S. Tayebati, T. Tulabandhula, and A. R. Trivedi, “Sense less, generate more: Pre-training lidar perception with masked autoencoders for ultra- efficient 3d sensing,” arXiv preprint arXiv:2406.07833 , 2024

    work page Pith review arXiv 2024

  4. [4]

    Second: Sparsely embedded convolutional detection,

    Y . Yan, Y . Mao, and B. Li, “Second: Sparsely embedded convolutional detection,” Sensors, vol. 18, no. 10, p. 3337, 2018

    2018

  5. [5]

    Occupancy-mae: Self-supervised pre-training large-scale lidar point clouds with masked occupancy autoencoders,

    C. Min, L. Xiao, D. Zhao, Y . Nie, and B. Dai, “Occupancy-mae: Self-supervised pre-training large-scale lidar point clouds with masked occupancy autoencoders,” IEEE Transactions on Intelligent Vehicles , 2023

    work page 2023

  6. [6]

    Also: Auto- motive lidar self-supervision by occupancy estimation,

    A. Boulch, C. Sautier, B. Michele, G. Puy, and R. Marlet, “Also: Auto- motive lidar self-supervision by occupancy estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 13 455–13 465

    work page 2023

  7. [7]

    Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,

    S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 10 529–10 538

    work page 2020

  8. [8]

    Spconv: Spatially sparse convolution library,

    S. Contributors, “Spconv: Spatially sparse convolution library,” https:// github.com/traveller59/spconv, 2022

    work page 2022

Show all 78 references
  1. [9]

    Mv-jar: Masked voxel jigsaw and reconstruction for lidar-based self-supervised pre-training,

    R. Xu, T. Wang, W. Zhang, R. Chen, J. Cao, J. Pang, and D. Lin, “Mv-jar: Masked voxel jigsaw and reconstruction for lidar-based self-supervised pre-training,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 13 445–13 454

    2023

  2. [10]

    Masked autoencoder for self-supervised pre-training on lidar point clouds,

    G. Hess, J. Jaxing, E. Svensson, D. Hagerman, C. Petersson, and L. Svensson, “Masked autoencoder for self-supervised pre-training on lidar point clouds,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision , 2023, pp. 350–359

    2023

  3. [11]

    Mc 2ram: Markov chain monte carlo sampling in sram for fast bayesian inference,

    P. Shukla, A. Shylendra, T. Tulabandhula, and A. R. Trivedi, “Mc 2ram: Markov chain monte carlo sampling in sram for fast bayesian inference,” in 2020 IEEE International Symposium on Circuits and Systems (ISCAS)

    2020

  4. [12]

    Ultralow-power localization of insect-scale drones: Interplay of probabilistic filtering and compute-in-memory,

    P. Shukla, A. Muralidhar, N. Iliev, T. Tulabandhula, S. B. Fuller, and A. R. Trivedi, “Ultralow-power localization of insect-scale drones: Interplay of probabilistic filtering and compute-in-memory,” IEEE transactions on very large scale integration (VLSI) systems , vol. 30, n...

    2021

  5. [13]

    Mf- net: Compute-in-memory sram for multibit precision inference using memory-immersed data conversion and multiplication-free operators,

    S. Nasrin, D. Badawi, A. E. Cetin, W. Gomes, and A. R. Trivedi, “Mf- net: Compute-in-memory sram for multibit precision inference using memory-immersed data conversion and multiplication-free operators,” IEEE Transactions on Circuits and Systems I: Regular Papers , vol. 68, no...

    1966

  6. [14]

    Low power unsupervised anomaly detection by nonparametric modeling of sensor statistics,

    A. Shylendra, P. Shukla, S. Mukhopadhyay, S. Bhunia, and A. R. Trivedi, “Low power unsupervised anomaly detection by nonparametric modeling of sensor statistics,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 28, no. 8, pp. 1833–1843, 2020

    2020

  7. [15]

    Ultra-low power electronics with si/ge tunnel fet,

    A. R. Trivedi, M. F. Amir, and S. Mukhopadhyay, “Ultra-low power electronics with si/ge tunnel fet,” in 2014 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2014, pp. 1–6

    2014

  8. [16]

    Two- dimensional materials for bio-realistic neuronal computing networks,

    V . K. Sangwan, S. E. Liu, A. R. Trivedi, and M. C. Hersam, “Two- dimensional materials for bio-realistic neuronal computing networks,” Matter, vol. 5, no. 12, pp. 4133–4152, 2022

    2022

  9. [17]

    Roadmap for unconventional computing with nanotechnology,

    G. Finocchio, J. A. C. Incorvia, J. S. Friedman, Q. Yang, A. Giordano, J. Grollier, H. Yang, F. Ciubotaru, A. V . Chumak, A. J. Naeemi et al. , “Roadmap for unconventional computing with nanotechnology,” Nano Futures, vol. 8, no. 1, p. 012001, 2024

    2024

  10. [18]

    Robokoop: Efficient control conditioned representations from visual input in robotics using koopman operator,

    H. Kumawat, B. Chakraborty, and S. Mukhopadhyay, “Robokoop: Efficient control conditioned representations from visual input in robotics using koopman operator,” 2024. [Online]. Available: https: //arxiv.org/abs/2409.03107

    2024 arXiv

  11. [19]

    A methodology for understanding the origins of false negatives in dnn based object detectors,

    K. Samal, H. Kumawat, M. Wolf, and S. Mukhopadhyay, “A methodology for understanding the origins of false negatives in dnn based object detectors,” in 2022 International Joint Conference on Neural Networks (IJCNN), 2022, pp. 1–8

    2022

  12. [20]

    Cognitive sensing for energy-efficient edge intelli- gence,

    M. Lee, S. Sharma, W. C. Wang, H. Kumawat, N. M. Rahman, and S. Mukhopadhyay, “Cognitive sensing for energy-efficient edge intelli- gence,” in 2024 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2024, pp. 1–6

    2024

  13. [21]

    Task-driven rgb-lidar fusion for object tracking in resource-efficient autonomous system,

    K. Samal, H. Kumawat, P. Saha, M. Wolf, and S. Mukhopadhyay, “Task-driven rgb-lidar fusion for object tracking in resource-efficient autonomous system,” IEEE Transactions on Intelligent Vehicles , vol. 7, no. 1, pp. 102–112, 2022

    2022

  14. [22]

    Radar guided dynamic visual attention for resource-efficient rgb object detection,

    H. Kumawat and S. Mukhopadhyay, “Radar guided dynamic visual attention for resource-efficient rgb object detection,” in2022 International Joint Conference on Neural Networks (IJCNN) , 2022, pp. 1–8

    2022

  15. [23]

    STAGE net: Spatio-temporal attention-based graph encoding for learning multi-agent interactions in the presence of hidden agents,

    H. Kumawat, B. Chakraborty, and S. Mukhopadhyay, “STAGE net: Spatio-temporal attention-based graph encoding for learning multi-agent interactions in the presence of hidden agents,” 2024. [Online]. Available: https://openreview.net/forum?id=tsj6rDzI0V

    2024

  16. [24]

    STEMFold: Stochastic temporal manifold for multi-agent interactions in the presence of hidden agents,

    ——, “STEMFold: Stochastic temporal manifold for multi-agent interactions in the presence of hidden agents,” in Proceedings of the 6th Annual Learning for Dynamics & Control Conference, ser. Proceedings of Machine Learning Research, A. Abate, M. Cannon, K. Margellos, and A. Pap...

    2024

  17. [25]

    Adacred: Adaptive causal decision transformers with feature crediting,

    H. Kumawat and S. Mukhopadhyay, “Adacred: Adaptive causal decision transformers with feature crediting,” 2024. [Online]. Available: https://arxiv.org/abs/2412.15427

    2024 arXiv

  18. [26]

    Curl: Contrastive unsupervised representations for reinforcement learning,

    A. Srinivas, M. Laskin, and P. Abbeel, “Curl: Contrastive unsupervised representations for reinforcement learning,” 2020

    2020

  19. [27]

    Task-oriented koopman-based control with contrastive encoder,

    X. Lyu, H. Hu, S. Siriya, Y . Pu, and M. Chen, “Task-oriented koopman-based control with contrastive encoder,” in 7th Annual Conference on Robot Learning , 2023. [Online]. Available: https: //openreview.net/forum?id=q0V AoefCI2

    2023

  20. [28]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2023

    2023

  21. [29]

    Decision transformer: Reinforcement learning via sequence modeling,

    L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch, “Decision transformer: Reinforcement learning via sequence modeling,” 2021

    2021

  22. [30]

    Dream to control: Learning behaviors by latent imagination,

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to control: Learning behaviors by latent imagination,” 2020

    2020

  23. [31]

    Weather influence and classification with automotive lidar sensors,

    R. Heinzler, P. Schindler, J. Seekircher, W. Ritter, and W. Stork, “Weather influence and classification with automotive lidar sensors,” in 2019 IEEE intelligent vehicles symposium (IV) . IEEE, 2019, pp. 1527–1534

    2019

  24. [32]

    Event- based vision: A survey,

    G. Gallego, T. Delbr ¨uck, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis et al. , “Event- based vision: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 1, pp. 154–180, 2020

    2020

  25. [33]

    Solid or not solid: Vision for radar target validation,

    A. Sole, O. Mano, G. P. Stein, H. Kumon, Y . Tamatsu, and A. Shashua, “Solid or not solid: Vision for radar target validation,” in IEEE Intelligent Vehicles Symposium, 2004. IEEE, 2004, pp. 819–824

    2004

  26. [34]

    Starnet: Sensor trustworthiness and anomaly recognition via approx- imated likelihood regret for robust edge autonomy,

    N. Darabi, S. Tayebati, S. Ravi, T. Tulabandhula, A. R. Trivedi et al. , “Starnet: Sensor trustworthiness and anomaly recognition via approx- imated likelihood regret for robust edge autonomy,” arXiv preprint arXiv:2309.11006, 2023

    2023 arXiv

  27. [35]

    Likelihood regret: An out-of-distribution detection score for variational auto-encoder,

    Z. Xiao, Q. Yan, and Y . Amit, “Likelihood regret: An out-of-distribution detection score for variational auto-encoder,” Advances in neural infor- mation processing systems , vol. 33, pp. 20 685–20 696, 2020

    2020

  28. [36]

    Stochastic first-and zeroth-order methods for nonconvex stochastic programming,

    S. Ghadimi and G. Lan, “Stochastic first-and zeroth-order methods for nonconvex stochastic programming,” SIAM Journal on Optimization , vol. 23, no. 4, pp. 2341–2368, 2013

    2013

  29. [37]

    Generalized simultaneous perturbation stochastic approximation with reduced estimator bias,

    S. Bhatnagar and L. Prashanth, “Generalized simultaneous perturbation stochastic approximation with reduced estimator bias,” in 2023 57th Annual Conference on Information Sciences and Systems (CISS) . IEEE, 2023, pp. 1–6

    2023

  30. [38]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  31. [39]

    Neural precision polarization: Simplifying neural network inference with dual- level precision,

    D. Jayasuriya, N. Darabi, M. B. Hashem, and A. R. Trivedi, “Neural precision polarization: Simplifying neural network inference with dual- level precision,” arXiv preprint arXiv:2411.05845 , 2024

    2024 arXiv

  32. [40]

    Robo3d: Towards robust and reliable 3d perception against corruptions,

    L. Kong, Y . Liu, X. Li, R. Chen, W. Zhang, J. Ren, L. Pan, K. Chen, and Z. Liu, “Robo3d: Towards robust and reliable 3d perception against corruptions,” arXiv preprint arXiv:2303.17597 , 2023

    2023 arXiv

  33. [41]

    Pedestrian protection systems: Issues, survey, and challenges,

    T. Gandhi and M. M. Trivedi, “Pedestrian protection systems: Issues, survey, and challenges,” IEEE Transactions on intelligent Transportation systems, vol. 8, no. 3, pp. 413–430, 2007

    2007

  34. [42]

    Mitigation of crosstalk effects in multi-lidar configurations,

    A. L. Diehm, M. Hammer, M. Hebel, and M. Arens, “Mitigation of crosstalk effects in multi-lidar configurations,” in Electro-Optical Remote Sensing XII, vol. 10796. SPIE, 2018, pp. 13–24

    2018

  35. [43]

    Robustness evaluation of electrical characteristics of sub-22 nm finfets affected by physical variability,

    B. M. Kalasapati and S. L. Tripathi, “Robustness evaluation of electrical characteristics of sub-22 nm finfets affected by physical variability,” Materials Today: Proceedings, vol. 49, pp. 2245–2252, 2022

    2022

  36. [44]

    Mutual information-calibrated conformal feature fusion for uncertainty- aware multimodal 3d object detection at the edge,

    A. C. Stutts, D. Erricolo, S. Ravi, T. Tulabandhula, and A. R. Trivedi, “Mutual information-calibrated conformal feature fusion for uncertainty- aware multimodal 3d object detection at the edge,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2...

    2024

  37. [45]

    Lightweight, uncertainty-aware conformalized visual odometry,

    A. C. Stutts, D. Erricolo, T. Tulabandhula, and A. R. Trivedi, “Lightweight, uncertainty-aware conformalized visual odometry,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 7742–7749

    2023

  38. [46]

    Navigating the unknown: Uncertainty-aware compute- in-memory autonomy of edge robotics,

    N. Darabi, P. Shukla, D. Jayasuriya, D. Kumar, A. C. Stutts, and A. R. Trivedi, “Navigating the unknown: Uncertainty-aware compute- in-memory autonomy of edge robotics,” in 2024 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2024, pp. 1–6

    2024

  39. [47]

    Conformalized multimodal uncertainty regression and reasoning,

    D. Parente, N. Darabi, A. C. Stutts, T. Tulabandhula, and A. R. Trivedi, “Conformalized multimodal uncertainty regression and reasoning,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024, pp. 6985–6989

    2024

  40. [48]

    Ev-flownet: Self- supervised optical flow estimation for event-based cameras,

    A. Z. Zhu, L. Yuan, K. Chaney, and K. Daniilidis, “Ev-flownet: Self- supervised optical flow estimation for event-based cameras,” arXiv preprint arXiv:1802.06898, 2018

    2018 arXiv

  41. [49]

    Adaptive-spikenet: event-based optical flow estimation using spiking neural networks with learnable neuronal dynam- ics,

    A. K. Kosta and K. Roy, “Adaptive-spikenet: event-based optical flow estimation using spiking neural networks with learnable neuronal dynam- ics,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 6021–6027

    2023

  42. [50]

    Spike-flownet: event-based optical flow estimation with energy-efficient hybrid neural networks,

    C. Lee, A. K. Kosta, A. Z. Zhu, K. Chaney, K. Daniilidis, and K. Roy, “Spike-flownet: event-based optical flow estimation with energy-efficient hybrid neural networks,” in European Conference on Computer Vision . Springer, 2020, pp. 366–382

    2020

  43. [51]

    Fusion-flownet: Energy-efficient optical flow estimation using sensor fusion and deep fused spiking-analog network architectures,

    C. Lee, A. K. Kosta, and K. Roy, “Fusion-flownet: Energy-efficient optical flow estimation using sensor fusion and deep fused spiking-analog network architectures,” in 2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022, pp. 6504–6510

    2022

  44. [52]

    Kinematics measurement and power requirements of fruitflies at various flight speeds,

    H. J. Zhu and M. Sun, “Kinematics measurement and power requirements of fruitflies at various flight speeds,” Energies, vol. 13, no. 16, p. 4271, 2020

    2020

  45. [53]

    Exploring neuromorphic computing based on spiking neural networks: Algorithms to hardware,

    N. Rathi, I. Chakraborty, A. Kosta, A. Sengupta, A. Ankit, P. Panda, and K. Roy, “Exploring neuromorphic computing based on spiking neural networks: Algorithms to hardware,” ACM Computing Surveys , vol. 55, no. 12, pp. 1–49, 2023

    2023

  46. [54]

    Random number generators and spiking neurons from metal oxide/small molecules heterojunction n-shape switching transistors,

    J. Seo, S. Kang, D. Kumar, W. Shin, J. Cho, T. Kim, Y . Kim, B. C. Jang, A. R. Trivedi, and H. Yoo, “Random number generators and spiking neurons from metal oxide/small molecules heterojunction n-shape switching transistors,” Advanced Functional Materials, p. 2411348, 2024

    2024

  47. [55]

    A 128 × 128 120 db 15 µs latency asynchronous temporal contrast vision sensor,

    P. Lichtsteiner, C. Posch, and T. Delbruck, “A 128 × 128 120 db 15 µs latency asynchronous temporal contrast vision sensor,” IEEE Journal of Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb 2008

    2008

  48. [56]

    A 240 ×180 130 db 3 µs latency global shutter spatiotemporal vision sensor,

    C. Brandli, R. Berner, M. Yang, S. Liu, and T. Delbruck, “A 240 ×180 130 db 3 µs latency global shutter spatiotemporal vision sensor,” IEEE Journal of Solid-State Circuits , vol. 49, no. 10, pp. 2333–2341, 2014

    2014

  49. [57]

    Data-driven feature tracking for event cameras,

    N. Messikommer, C. Fang, M. Gehrig, and D. Scaramuzza, “Data-driven feature tracking for event cameras,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 5642– 5651

    2023

  50. [58]

    Networks of spiking neurons: the third generation of neural network models,

    W. Maass, “Networks of spiking neurons: the third generation of neural network models,” Neural networks, vol. 10, no. 9, pp. 1659–1671, 1997

    1997

  51. [59]

    Towards spike-based machine intel- ligence with neuromorphic computing,

    K. Roy, A. Jaiswal, and P. Panda, “Towards spike-based machine intel- ligence with neuromorphic computing,” Nature, vol. 575, no. 7784, pp. 607–617, 2019

    2019

  52. [60]

    Spiking neural networks with improved inherent recurrence dynamics for sequential learning,

    W. Ponghiran and K. Roy, “Spiking neural networks with improved inherent recurrence dynamics for sequential learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 7, 2022, pp. 8001–8008

    2022

  53. [61]

    Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation,

    N. Rathi, G. Srinivasan, P. Panda, and K. Roy, “Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum? id=B1xSperKvH

    2020

  54. [62]

    Surrogate gradient learning in spiking neural networks,

    E. O. Neftci, H. Mostafa, and F. Zenke, “Surrogate gradient learning in spiking neural networks,” IEEE Signal Processing Magazine, vol. 36, pp. 61–63, 2019

    2019

  55. [63]

    Spiking deep convolutional neural networks for energy-efficient object recognition,

    Y . Cao, Y . Chen, and D. Khosla, “Spiking deep convolutional neural networks for energy-efficient object recognition,” International Journal of Computer Vision , vol. 113, pp. 54–66, 2015

    2015

  56. [64]

    Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization,

    N. Rathi and K. Roy, “Diet-snn: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization,” IEEE Transactions on Neural Networks and Learning Systems , 2021

    2021

  57. [65]

    Enabling spike-based backpropagation for training deep neural network architec- tures,

    C. Lee, S. S. Sarwar, P. Panda, G. Srinivasan, and K. Roy, “Enabling spike-based backpropagation for training deep neural network architec- tures,” Frontiers in Neuroscience, vol. 14, 2020

    2020

  58. [66]

    The multivehicle stereo event camera dataset: An event camera dataset for 3d perception,

    A. Z. Zhu, D. Thakur, T. ¨Ozaslan, B. Pfrommer, V . Kumar, and K. Dani- ilidis, “The multivehicle stereo event camera dataset: An event camera dataset for 3d perception,” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2032–2039, 2018

    2018

  59. [67]

    Dotie-detecting objects through temporal isolation of events using a spiking architecture,

    M. Nagaraj, C. M. Liyanagedera, and K. Roy, “Dotie-detecting objects through temporal isolation of events using a spiking architecture,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 4858–4864

    2023

  60. [68]

    Neuro-cim: A 310.4 tops/w neuromorphic computing-in-memory processor with low wl/bl activity and digital-analog mixed-mode neuron firing,

    S. Kim, S. Kim, S. Um, S. Kim, K. Kim, and H.-J. Yoo, “Neuro-cim: A 310.4 tops/w neuromorphic computing-in-memory processor with low wl/bl activity and digital-analog mixed-mode neuron firing,” in 2022 IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuit...

    2022

  61. [69]

    Impulse: A 65-nm digital compute-in-memory macro with fused weights and membrane potential for spike-based sequential learning tasks,

    A. Agrawal, M. Ali, M. Koo, N. Rathi, A. Jaiswal, and K. Roy, “Impulse: A 65-nm digital compute-in-memory macro with fused weights and membrane potential for spike-based sequential learning tasks,” IEEE Solid-State Circuits Letters , vol. 4, pp. 137–140, 2021

    2021

  62. [70]

    Resparc: A reconfigurable and energy-efficient architecture with memristive crossbars for deep spiking neural networks,

    A. Ankit, A. Sengupta, P. Panda, and K. Roy, “Resparc: A reconfigurable and energy-efficient architecture with memristive crossbars for deep spiking neural networks,” in Proceedings of the 54th Annual Design Automation Conference 2017 , 2017, pp. 1–6

    2017

  63. [71]

    Highly parallel and ultra-low- power probabilistic reasoning with programmable gaussian-like memory transistors,

    C. Lee, L. Rahimifard, J. Choi, J.-i. Park, C. Lee, D. Kumar, P. Shukla, S. M. Lee, A. R. Trivedi, H. Yoo et al., “Highly parallel and ultra-low- power probabilistic reasoning with programmable gaussian-like memory transistors,” Nature Communications, vol. 15, no. 1, p. 2439, 2024

    2024

  64. [72]

    Adc/dac-free analog acceleration of deep neural networks with frequency transformation,

    N. Darabi, M. B. Hashem, H. Pan, A. Cetin, W. Gomes, and A. R. Trivedi, “Adc/dac-free analog acceleration of deep neural networks with frequency transformation,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2024

    2024

  65. [73]

    Federated learning for healthcare: Systematic review and architecture proposal,

    R. S. Antunes, C. Andr ´e da Costa, A. K ¨uderle, I. A. Yari, and B. Eskofier, “Federated learning for healthcare: Systematic review and architecture proposal,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 13, no. 4, pp. 1–23, 2022

    2022

  66. [74]

    Federated learning for internet of things: Recent advances, taxonomy, and open challenges,

    L. U. Khan, W. Saad, Z. Han, E. Hossain, and C. S. Hong, “Federated learning for internet of things: Recent advances, taxonomy, and open challenges,” IEEE Communications Surveys & Tutorials, vol. 23, no. 3, pp. 1759–1799, 2021

    2021

  67. [75]

    Federated learning in robotic and autonomous systems,

    Y . Xianjia, J. P. Queralta, J. Heikkonen, and T. Westerlund, “Federated learning in robotic and autonomous systems,” Procedia Computer Sci- ence, vol. 191, pp. 135–142, 2021

    2021

  68. [76]

    Divide-and-conquer the nas puzzle in resource-constrained federated learning systems,

    Y . Venkatesha, Y . Kim, H. Park, and P. Panda, “Divide-and-conquer the nas puzzle in resource-constrained federated learning systems,” Neural Networks, vol. 168, pp. 569–579, 2023

    2023

  69. [77]

    Halo- fl: Hardware-aware low-precision federated learning,

    Y . Venkatesha, A. Bhattacharjee, A. Moitra, and P. Panda, “Halo- fl: Hardware-aware low-precision federated learning,” in 2024 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2024, pp. 1–6

    2024

  70. [78]

    Fast inference from transform- ers via speculative decoding,

    Y . Leviathan, M. Kalman, and Y . Matias, “Fast inference from transform- ers via speculative decoding,” in International Conference on Machine Learning. PMLR, 2023, pp. 19 274–19 286

    2023

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.