Pith. sign in

REVIEW 5 major objections 5 minor 23 references

Bio-inspired visual relative localization for large swarms of UAVs

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A CNN that regresses neighbor density over distance outperforms per-drone detection for UAV swarm relative localization.

desk verdict Fresh idea, but the paper's own Table I contradicts its central accuracy claim and the promised control law is missing. read the letter →

arxiv 2412.02393 v1 pith:HRATER5Q submitted 2024-12-03 cs.RO cs.CV

Martin Křížek , Matouš Vrba , Antonella Barišić Kulaš , Stjepan Bogdan , Martin Saska This is my paper · ORCID
classification cs.ROcs.CV
keywords UAVswarmsrelativelocalizationdensityoverdistancemonocularvisionconvolutionalneuralnetworkswarmstabilizationsim-to-realtransfercollisionavoidance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a UAV in a large swarm can localize its neighbors more accurately by regressing how many neighbors are at each distance in its camera image than by detecting and estimating the pose of each neighbor individually. This density-over-distance output is a small histogram with one bin per distance range, and it is insensitive to the number of visible neighbors and to each neighbor's orientation, allowing the perception network to be smaller and faster. The authors show on synthetic imagery with up to 150 drones that their regressor has lower per-bin distance error than a conventional bounding-box detector and even than a hypothetical perfect detector, and they report qualitative transfer to real photos after fine-tuning on 30 images. If the claim holds, dense collision-avoiding swarms become feasible with lightweight onboard monocular cameras.

What carries the argument

The central object is the distance-density histogram $l_{\mathrm{raw}}[d]$ from eq. (1): the count of UAVs in each distance bin $[d, d+\Delta d)$, capped at $d_{\max}$ by eq. (2). The network is trained against the partially smoothed label $l_{\mathrm{gt}}[d] = (l_{\mathrm{raw}} * g)[d]$, where $g[d]$ is the identity kernel for the closest $k$ bins and a Gaussian kernel $G_\sigma[d]$ otherwise; smoothing preserves total count while forgiving small distance misassignments. A convolutional feature extractor feeds a $1\times1$ convolutional layer that scores feature maps per distance bin, followed by average pooling and a fully connected layer, and the weighted Euclidean loss in eq. (7) gives larger weights to close bins so collision-relevant distances are learned despite their underrepresentation in the dataset. This machinery converts an image into a sparse distribution that scales with swarm size and avoids per-target pose estimation.

What would settle it

Fly a real multi-drone experiment with ground-truth positions from RTK GPS or motion capture, run the trained density regressor on the camera feed, compute the per-bin error $\bar{e}[d]$ and total integral error $\bar{T}$, and compare them with the synthetic test results in Table I; a large degradation at close ranges after fine-tuning on 30 real photos would refute the sim-to-real transfer claim.

Watch

Extended reading notes

Core claim

The central discovery is that relative localization for swarm stabilization can be solved as a one-dimensional regression of neighbor counts over distance instead of a multi-object detection problem. For each output grid cell the network predicts $l_o[d]$, an estimate of $l_{\mathrm{raw}}[d]$, the number of UAVs whose position in the camera frame satisfies $\lVert p \rVert \in [d, d+\Delta d)$. Training uses partially smoothed labels $l_{\mathrm{gt}}[d] = (l_{\mathrm{raw}} * g)[d]$ with a Gaussian kernel for distant bins and an identity kernel for the closest bins, plus a weighted Euclidean loss that emphasizes close distances. The paper reports that this representation removes the inherent bounding-box bias caused by target pose, keeps error low at high neighbor counts, and is suitable as the primary input to swarm stabilization.

Load-bearing premise

The load-bearing premise is that the synthetic training images are realistic enough that a network fine-tuned on a few dozen real photos will estimate distance distributions on actual drone imagery about as accurately as it does in simulation.

Editorial extensions

If this is right

  • The same trained network handles images containing from 1 to 150 drones, with per-bin error on the high-density test set lower than at lower densities, so swarm size is no longer a bottleneck for perception.
  • A drone observing a tilted or rotated neighbor no longer suffers the systematic distance bias that bounding-box width and height estimates produce, because the regressor learns distance from visual features directly.
  • The network's small parameter count and 18 ms per-frame inference on a laptop GPU make it deployable onboard SWaP-constrained UAVs as the main relative-localization source.
  • Because the output is a distribution rather than discrete detections, the method can be fine-tuned to real-world imagery with only tens of labeled photographs, and the paper shows qualitative improvement from 30 fine-tuning images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the density-over-distance output could be accumulated over time and across multiple cameras, giving a lightweight sensor-fusion front end for swarm state estimation without requiring data association.
  • Editorial inference: the approach should be naturally tolerant of partial occlusion and motion blur, since an occluded or blurred neighbor shifts a count between nearby distance bins instead of vanishing; augmenting the synthetic dataset with such degradations would test this directly.
  • Editorial inference: the distance-bin weight vector in the loss is a direct control lever: if a collision-avoidance potential is specified, the weights could be optimized to minimize expected collision risk rather than uniform per-bin error, coupling perception and swarm control more tightly than the paper does.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes replacing per-UAV detection with a convolutional neural network that regresses a histogram of neighbor density over distance from a focal UAV, inspired by animal swarms. The authors claim that this yields more accurate distance estimation than detector-based localization, better scalability with swarm size, and compatibility with a novel swarm control algorithm. Evaluation is performed on a synthetic dataset of 16,000 images (Sim2Air-based), an additional synthetic high-density dataset with up to 150 UAVs, and a set of 40 real-world photos. Baselines include a YOLOv4 Tiny/EfficientDet-style state-of-the-art detector, an ideal detector, and several ablations of the proposed architecture and label smoothing.

Significance. The density-over-distance formulation is an interesting and potentially useful idea for large heterogeneous swarms, and the paper has concrete strengths: the dataset is released, the synthetic pipeline is described in some detail, the comparison includes a hypothetical ideal detector, and the high-density scaling experiment targets a real gap in the literature. If the claims were supported, the paper would provide a lightweight vision pipeline for swarm relative localization. However, the paper's own Table I contradicts the headline accuracy claim, the promised swarm control algorithm is absent, and the real-world validation is qualitative and tiny. As a result, the central claims are not established by the evidence presented.

major comments (5)
  1. [§IV-C, Table I] The statement that 'Our method outperforms the state-of-the-art detector using all metrics' is contradicted by the paper's own Table I. Under Eqs. (8)-(10), lower values of T, E, and E' are better. Table I reports for the SotA detector T = 0.02, E = 1.31, E' = 0.93, while Ours (1x1) reports T = 0.142, E = 1.31, E' = 1.25, and Ours (3x3) reports T = 0.174, E = 1.30, E' = 1.27. The proposed method therefore has substantially higher total integral error and higher close-range error than the detector baseline. This directly undermines the abstract's claim of 'more accurate distance estimation.'
  2. [Abstract, §I-B, §V] The abstract and contributions list promise 'a novel swarm control algorithm' that is compatible with the density regression, but no control law, closed-loop simulation, or stability analysis appears in Sections II-V. The conclusion instead states: 'In the future, we aim to design a swarming rule intended to utilize the proposed relative localization method.' The paper therefore advertises a central contribution that is missing, and the claim that the method is 'suitable to be used as the main source of relative localization for swarm stabilization' is not supported by any closed-loop experiment.
  3. [§IV-E, §III-B] The real-world generalization evaluation is not strong enough to carry the load assigned to it. The dataset consists of 40 manually annotated photos; 30 are used for fine-tuning and only 10 for testing. The text itself describes the analysis as 'qualitative,' and no quantitative distance-error metric is reported on these 10 images. Since the central scalability and robustness claims are based on synthetic data, the sim-to-real transfer question is effectively open. A quantitative real-world comparison with the detector baseline is needed before the method can be claimed suitable for real swarm stabilization.
  4. [§II-B, §III, §IV-B, Eq. (10)] There is a partial circularity in the evaluation setup: the loss function is weighted to emphasize close distances, the dataset cropping is biased toward images with close UAVs, and the summary metric E' is defined specifically as close-distance per-bin error. Any method trained with these choices is expected to perform better on close-range metrics than an untuned detector baseline. Table II's high-density experiment also trains and tests only on synthetic data generated by the same pipeline, so it does not demonstrate scalability to real dense swarms. The paper should separate 'performance by construction' from an independent measure of task accuracy, for example by reporting metrics over the full distance range with the training distribution ablated, or by evaluating on real high-density footage.
  5. [Tables I and II] No error bars, confidence intervals, or repeated-run statistics are reported for any of the trained models. Several of the claimed differences are small (e.g., E = 1.31 for both SotA detector and Ours 1x1), and the central accuracy comparison relies on numbers that could plausibly change with random seeds, initialization, or hyperparameter choices. The authors should provide multiple training runs or statistical measures before claiming superiority on any metric.
minor comments (5)
  1. [§II-C] The training setup paragraph says 'Stochastic Gradient Descent (SGD) back-propagation' but then states 'We used the ADAM optimizer.' This is inconsistent; please specify the optimizer correctly.
  2. [§IV-B] The ablation study references 'partially smoothed labels lgt[d] (eq. (2))', but Eq. (2) defines the last-bin truncation, not the smoothed label. The correct reference is Eqs. (3)-(4).
  3. [§IV-C, Table I] The identity of the 'SotA detector' is unclear: §IV-C introduces a YOLOv4 Tiny detector, while the surrounding text mentions EfficientDet, and the table caption does not specify which detector is reported. This needs to be resolved for reproducibility.
  4. [§IV-E] The real-world photos are described as containing '1 to 8 UAVs,' but the paper does not report how the density ground truth was obtained for these images; the nearest-neighbor distance estimation method is mentioned, but the annotation procedure should be described more concretely.
  5. [General] Several reference entries are marked as 'submitted to T-RO' rather than as published or accepted works (e.g., [5], [7]); these should be updated or marked as preprints to avoid implying peer-reviewed status.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the density-regression results are empirical predictions measured against raw ground-truth labels and external detector baselines, not quantities forced by construction.

full rationale

The paper's derivation chain is empirical and externally anchored. The network is trained with a weighted Euclidean loss (Eq. 7) against labels obtained from raw histograms via fixed Gaussian smoothing (Eq. 3), and it is evaluated against the unsmoothed raw histogram l_raw[d] using the independent metrics in Eqs. (8)--(10). No parameter fitted to a subset of data is renamed as a prediction; the only tuned element is the non-learnable bin-weight vector w, which is not fitted to the test set and does not by itself determine the reported errors. The comparison against a YOLOv4 Tiny detector and a hypothetical ideal detector provides an external benchmark, and the high-density experiment trains and tests both methods on newly generated synthetic data. Sim2Air [22] is a self-cited prior dataset-generation pipeline, but it is used as an external data source rather than as a theorem that forces the paper's conclusions, so it is not load-bearing circularity. The paper does contain non-circular defects: Table I does not support the text's claim that the proposed method outperforms the state-of-the-art detector on all metrics (e.g., SotA detector E'=0.93 versus Ours 1.25 and T=0.02 versus 0.142), and the abstract's promised swarm control algorithm is absent and deferred to future work in the conclusion. These are correctness/completeness issues, not circularity, and they do not affect the circularity score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

This is an empirical ML paper; the central claim rests on a synthetic data generation pipeline, hand-tuned loss and label smoothing parameters, and the assumption that monocular appearance encodes distance. No new physical entities are introduced.

free parameters (5)
  • Distance discretization step and bin count = Delta_d = 1 m, nbin = 50
    Design choice setting the resolution of the density-over-distance output; values are chosen by hand and affect accuracy.
  • Label smoothing kernel parameters = k and sigma not reported in the paper
    Partial Gaussian smoothing of ground-truth histograms (Eq. 3-4) uses an identity kernel for k close bins and Gaussian sigma elsewhere; values are not given, which also hurts reproducibility.
  • Loss weight vector = not reported
    Per-bin weights w in Eq. 7 are hand-tuned to emphasize close distances, counteracting dataset bias; exact values omitted.
  • Dataset generation balancing threshold = 15 targets
    Balancing of the synthetic dataset is enforced only up to an empirically determined 15 targets; this is a data-generation free parameter.
  • Network architecture hyperparameters = VGG-like head, 1x1 conv, nbin filters
    Architecture choices (feature extractor depth, pooling, fully connected layer) are design decisions; not all details are specified.
assumptions (4)
  • domain assumption A monocular RGB image of a known UAV platform contains sufficient visual information to regress the number of nearby UAVs per distance bin.
    The whole method is built on this premise; Section I-C defines the problem and Section II describes the CNN, but no argument establishes that distance information is recoverable from appearance alone.
  • domain assumption The synthetic dataset generated with the Sim2Air pipeline is representative enough of real-world imagery that models trained on it transfer to real UAVs after fine-tuning.
    Section III generates 16,000 synthetic images; Section IV-E validates on only 40 real photos, 30 for fine-tuning and 10 for qualitative testing, so the transfer assumption is only weakly checked.
  • domain assumption All UAVs in the swarm are of the known MRS F450 platform, so their visual appearance and size are consistent with training.
    Section III selects the MRS F450 platform and textures; the method is not shown to generalize to other UAV types.
  • domain assumption Occlusions can be largely avoided in the training data, so the density regression does not need to explicitly model heavy occlusion.
    Section III states "we create only up to one group in close proximity to the camera... to avoid introducing too many occlusions"; real swarms may be more occluded.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bio-inspired visual relative localization for large swarms of UAVs." pith.science (2026). https://pith.science/paper/HRATER5Q

@misc{pith2026241202393,
  author       = {Pith},
  title        = {Pith review of: Bio-inspired visual relative localization for large swarms of UAVs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HRATER5Q}},
  note         = {Machine review of arXiv:2412.02393}
}
read the original abstract

We propose a new approach to visual perception for relative localization of agents within large-scale swarms of UAVs. Inspired by biological perception utilized by schools of sardines, swarms of bees, and other large groups of animals capable of moving in a decentralized yet coherent manner, our method does not rely on detecting individual neighbors by each agent and estimating their relative position, but rather we propose to regress a neighbor density over distance. This allows for a more accurate distance estimation as well as better scalability with respect to the number of neighbors. Additionally, a novel swarm control algorithm is proposed to make it compatible with the new relative localization method. We provide a thorough evaluation of the presented methods and demonstrate that the regressing approach to distance estimation is more robust to varying relative pose of the targets and that it is suitable to be used as the main source of relative localization for swarm stabilization.

Figures

Figures reproduced from arXiv: 2412.02393 by the authors.

Figure 1
Figure 1. Top: An illustration of the Gaussian smoothing and the CNN prediction of the distribution of UAVs in the image over distance. The input image is shown on the right. Bottom: The output of the CNN with wout = hout = 3 for the input image. The predicted density for a given grid cell and distance is marked with a circle of the corresponding size and color from blue to yellow. 300x300x32 150x150x64 75x75x128 37x37x128 18… view at source ↗
Figure 2
Figure 2. Visualization of the network architecture for [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison of synthetic (a) and real (b) images of the MRS F450 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Distribution of images in the dataset based on the total number of [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Distribution of UAVs in the dataset images based on their corre [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 7
Figure 7. Figure 7: Comparison of the average absolute per-bin error for the [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: Comparison of the output of the CNN on a real-wold photo for [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]
Figure 9
Figure 9. Figure 9: An example of a challenging image for bounding box-based distance [PITH_FULL_IMAGE:figures/full_fig_p005_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 19 canonical work pages

  1. [1]

    Understanding how animal groups achieve coor- dinated movement,

    J. E. Herbert-Read, “Understanding how animal groups achieve coor- dinated movement,” Journal of Experimental Biology, vol. 219, no. 19, pp. 2971–2983, 2016

    work page 2016

  2. [2]

    Efficientdet: Scalable and efficient object detection,

    M. Tan, R. Pang, and Q. V . Le, “Efficientdet: Scalable and efficient object detection,” in Conference on Computer Vision and Pattern Recognition, 2020, pp. 10 781–10 790

    work page 2020

  3. [3]

    CenterNet: Keypoint triplets for object detection,

    K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang, and Q. Tian, “CenterNet: Keypoint triplets for object detection,” in International Conference on Computer Vision, 2019, pp. 6569–6578

    work page 2019

  4. [4]

    YOLOv3: An incremental improvement,

    J. Redmon and A. Farhadi, “YOLOv3: An incremental improvement,” preprint, arXiv:1804.02767, 2018

    arXiv 2018

  5. [5]

    Distributed UAV Formation Control Robust to Relative Pose Measurement Noise

    V . Walter, M. Vrba, D. B. Licea, and M. Saska, “Distributed formation- enforcing control for uavs robust to observation noise in relative pose measurements,” preprint, arXiv:2304.03057, 2023, submitted to T-RO

    work page Pith review arXiv 2023

  6. [6]

    Flexible layouts for fiducial tags,

    M. Krogius, A. Haggenmiller, and E. Olson, “Flexible layouts for fiducial tags,” in International Conference on Intelligent Robots and Systems, 2019, pp. 1898–1903

    work page 2019

  7. [7]

    On Onboard LiDAR-based Flying Object Detection

    M. Vrba, V . Walter, and M. Saska, “On onboard LiDAR-based flying object detection,” preprint, arXiv 2303.05404 , 2023, submitted to T- RO

    work page Pith review arXiv 2023

  8. [8]

    Ultra-wideband and odometry-based co- operative relative localization with application to multi-uav formation control,

    K. Guo, X. Li, and L. Xie, “Ultra-wideband and odometry-based co- operative relative localization with application to multi-uav formation control,” Transactions on Cybernetics, vol. 50, no. 6, pp. 2590–2603, 2019

    work page 2019

Show all 23 references
  1. [9]

    Onboard marker-less detection and localization of non-cooperating drones for their safe interception by an autonomous aerial system,

    M. Vrba, D. He ˇrt, and M. Saska, “Onboard marker-less detection and localization of non-cooperating drones for their safe interception by an autonomous aerial system,” Robotics and Automation Letters , vol. 4, no. 4, pp. 3402–3409, 2019

    2019

  2. [10]

    Marker-less micro aerial vehicle detection and localization using convolutional neural networks,

    M. Vrba and M. Saska, “Marker-less micro aerial vehicle detection and localization using convolutional neural networks,” Robotics and Automation Letters, vol. 5, no. 2, pp. 2459–2466, 2020

    2020

  3. [11]

    Tracking and relative localization of drone swarms with a vision- based headset,

    M. Pavliv, F. Schiano, C. Reardon, D. Floreano, and G. Loianno, “Tracking and relative localization of drone swarms with a vision- based headset,” Robotics and Automation Letters , vol. 6, no. 2, pp. 1455–1462, 2021

    2021

  4. [12]

    Monoloco: Monocular 3D pedestrian localization and uncertainty estimation,

    L. Bertoni, S. Kreiss, and A. Alahi, “Monoloco: Monocular 3D pedestrian localization and uncertainty estimation,” in International Conference on Computer Vision , 2019, pp. 6861–6871

    2019

  5. [13]

    M3D-RPN: Monocular 3D region proposal net- work for object detection,

    G. Brazil and X. Liu, “M3D-RPN: Monocular 3D region proposal net- work for object detection,” in International Conference on Computer Vision, 2019, pp. 9287–9296

    2019

  6. [14]

    Target strength distributions of Pacific sardine schools: Model results at 500 Hz to 10 kHz,

    R. H. Love, J. M. Fialkowski, and T. H. Jagielo, “Target strength distributions of Pacific sardine schools: Model results at 500 Hz to 10 kHz,” The Journal of the Acoustical Society of America , vol. 140, no. 6, pp. 4456–4471, 2016

    2016

  7. [15]

    Crowd counting via scale-adaptive convolutional neural network,

    L. Zhang, M. Shi, and Q. Chen, “Crowd counting via scale-adaptive convolutional neural network,” in Winter Conference on Applications of Computer Vision , 2018, pp. 1113–1121

    2018

  8. [16]

    CNN- based density estimation and crowd counting: A survey,

    G. Gao, J. Gao, Q. Liu, Q. Wang, and Y . Wang, “CNN- based density estimation and crowd counting: A survey,” preprint, arXiv:2003.12783, 2020

    2003 arXiv

  9. [17]

    A survey of recent advances in CNN- based single image crowd counting and density estimation,

    V . A. Sindagi and V . M. Patel, “A survey of recent advances in CNN- based single image crowd counting and density estimation,” Pattern Recognition Letters, vol. 107, pp. 3–16, 2018

    2018

  10. [18]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” preprint, arXiv:1409.1556, 2014

    2014 arXiv

  11. [19]

    Single-image crowd counting via multi-column convolutional neural network,

    Y . Zhang, D. Zhou, S. Chen, S. Gao, and Y . Ma, “Single-image crowd counting via multi-column convolutional neural network,” in Conference on Computer Vision and Pattern Recognition , 2016, pp. 589–597

    2016

  12. [20]

    Adam: A method for stochastic optimiza- tion,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimiza- tion,” preprint, arXiv:1412.6980, 2014

    2014 arXiv

  13. [21]

    TensorFlow: A system for large-scale machine learning,

    M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V . Vasudevan, P. War- den, M. Wicke, Y . Yu, and X. Zheng, “TensorFlow: A system for large-s...

    2016

  14. [22]

    Sim2Air-synthetic aerial dataset for UA V monitoring,

    A. Barisic, F. Petric, and S. Bogdan, “Sim2Air-synthetic aerial dataset for UA V monitoring,”Robotics and Automation Letters , vol. 7, no. 2, pp. 3757–3764, 2022

    2022

  15. [23]

    MRS modular UA V hardware plat- forms for supporting research in real-world outdoor and indoor environments,

    D. Hert, T. Baca, P. Petracek, V . Kratky, V . Spurny, M. Petrlik, M. Vrba, D. Zaitlik, P. Stoudek, V . Walter, P. Stepan, J. Horyna, V . Pritzl, G. Silano, D. Bonilla Licea, P. Stibinger, R. Penicka, T. Nascimento, and M. Saska, “MRS modular UA V hardware plat- forms for supp...

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.