REVIEW 4 major objections 5 minor 2 references
YASMOT: Yet another stereo image multi-object tracker
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read yasmot claims that tracking, stereo linking, and detector consensus can all be built from Gaussian distances on bounding-box position and size alone, with no image content, no motion model, and almost no dependencies.
desk verdict A clean, functional detection-only tracker with stereo and ensemble niceties, but no quantitative evaluation; the code is solid, the claims are not. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the Gaussian affinity score over detection parameters. For every candidate pair of boxes, yasmot applies a Gaussian to the position coordinates (center x, center y) and a separate Gaussian to the size coordinates, combining them into a distance matrix; the sharpness, or temperature, of each Gaussian is a user parameter set by `--scale`. The Hungarian algorithm (Kuhn 1955) then solves the assignment problem on this matrix to find the optimal pairing of detections between two frames, which is the same operation used for temporal tracking (frame t to t+1) and for stereo linking (left to right view). For more than two simultaneous inputs, as in consensus mode with several detectors, exact multi-frame assignment becomes too costly, so a heuristic is used. `--max_age` allows a track to bridge frames with no detection, and `--interpolate` fills the gap with synthetic boxes whose probability is set to 0.0000 so they can be identified later.
What would settle it
Run yasmot on a crowded video with fast-moving, crossing objects and intermittent occlusions, with a fixed `--scale`, and count identity switches and track fragmentation against a Kalman-plus-IoU tracker such as SORT; if objects are lost in exactly the regimes where overlap fails but appearance or motion would succeed, the claim that Gaussian box distances alone suffice is refuted for that regime. Since the paper reports no quantitative benchmark evaluation, this comparison on a standard multi-object-tracking dataset is the direct way to settle the claim.
Extended reading notes
Core claim
The central claim is that a hand-configured Gaussian distance over bounding-box coordinates is a sufficient affinity measure for three association tasks: tracking detections over time, matching them between left and right stereo views, and merging simultaneous detections from several detectors into a consensus set. yasmot scores each candidate box pair by applying a Gaussian to the position coordinates and a separate Gaussian to the size coordinates, with the `--scale` parameters controlling the temperature, and then uses the Hungarian algorithm to find the optimal one-to-one pairing between the two frames. Because the score is a function of box parameters rather than overlap, detections can be associated even when their boxes do not intersect, which the paper argues matters at low frame rates and for stereo pairs of objects close to the camera. Tracks survive missed detections up to `--max_age` frames, optionally with interpolated boxes marked by zero probability; stereo mode uses the same machinery on left/right pairs, and consensus mode applies it across detectors, with an `--unknown` label to suppress uninformative consensus classes. The intended consequence is that any detection pipeline gains identity persistence, stereo depth, and ensemble agreement from its text output alone, without the tracker ever requiring the images.
Load-bearing premise
The load-bearing premise is that a hand-tuned Gaussian temperature on bounding-box position and size is a sufficient measure of whether two detections are the same object, without any motion prediction or appearance information; the paper leaves `--scale` as a user parameter to adjust by hand when objects move fast or frame rate is low.
Editorial extensions
If this is right
- Any detection pipeline gains identity persistence from its text output alone, even when the original video frames are no longer available.
- Stereo matching with the same Gaussian machinery yields depth information and object-size estimates from bounding boxes alone.
- Because scores measure proximity rather than overlap, detections can be associated even when boxes do not intersect, which matters at low frame rates and for close objects in stereo pairs.
- Ensemble consensus merges several detectors frame-by-frame or along tracks, with the `--unknown` flag preventing an unidentified class from becoming a track's consensus label.
- Missed detections do not break tracks: up to `--max_age` frames can be bridged, and `--interpolate` fills the gap with synthetic boxes carrying probability 0.0000.
Reading between the lines
- If the claim holds, appearance features are unnecessary in many practical tracking regimes; a direct ablation would be to remove the appearance embedding from an appearance-based tracker and measure how much it actually contributes on non-crowded video.
- The `--scale` temperature is the natural failure point of the design, so a testable extension is to estimate it automatically from the observed distribution of inter-frame box displacements instead of hand-tuning.
- Adding a constant-velocity prediction step to the Gaussian score would target the acknowledged weak regime of fast motion and low frame rates without reintroducing image content.
- Because the tracker deliberately ignores appearance, crowded scenes with frequent crossings should be its worst case; plotting identity-switch rate against crowd density would quantify the cost of the no-image-content design.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript describes yasmot, a Python multi-object tracker that operates purely on object-detector outputs rather than on image content. Temporal tracking and left/right stereo linking are performed by computing Gaussian distances on bounding-box position and size and solving pairwise assignments with the Hungarian algorithm; an ensemble mode merges detections from multiple detectors into consensus predictions. The paper documents the command-line options (--scale, --max_age, --interpolate, -s, -c), output formats, and several usage examples drawn from a test suite, but it contains no quantitative evaluation, no error metrics, and no comparison with the trackers cited in the related-work section.
Significance. Software that tracks objects from detector outputs alone is practically attractive, and yasmot's minimal dependencies, public availability on PyPI and GitHub, and included test suite are genuine strengths that make the implementation reproducible. If the central association claim were validated, the ability to connect non-overlapping detections via Gaussian distances would be a useful alternative to IoU-based trackers, particularly for low-frame-rate or stereo settings. However, the manuscript currently provides only illustrative command examples; the claims that tracking and stereo linking improve detection performance, that the method is suitable for low frame rates, and that the Gaussian affinity is sufficient for reliable identity linkage are unsupported by any experiment. The paper is a useful software note, but as a scientific contribution it is incomplete without a benchmark and an analysis of the failure modes inherent in its design.
major comments (4)
- [Statement of need] The central claims that tracking 'improves object detection performance' and that stereo linking 'further improves detection performance' are never quantified. The paper contains no detection or tracking metrics (e.g., MOTA, IDF1, HOTA, precision/recall) and no comparison against SORT, ByteTrack, BoT-SORT, or DeepSORT, even though these methods are cited as related work. Add a quantitative evaluation on a public benchmark or on the provided test data, reporting performance with and without tracking, stereo linking, and ensemble consensus, to substantiate the stated benefits.
- [Controlling tracking sensitivity] The affinity model is a single Gaussian over (x, y, w, h) with a global --scale and no motion prediction. In the low-frame-rate and crowded regimes that the paper explicitly motivates, a moving object can easily be closer in box-coordinate space to a neighboring same-class object than to its own next-frame detection, causing identity switches that are structural rather than incidental. Because yasmot deliberately avoids motion models and appearance features, the paper must demonstrate empirically that Gaussian-distance association is sufficient in those regimes, for example by reporting identity-switch counts as a function of --scale and frame rate on a benchmark with known ground-truth identities.
- [Tracking stereo images] Stereo matching uses the same Gaussian distance as temporal tracking without any epipolar constraint. For rectified stereo pairs, corresponding detections differ primarily in horizontal disparity and share nearly the same row, but the symmetric Gaussian penalty weights vertical and horizontal offsets equally. This design invites left/right mismatches whenever several detections lie along the same image row. Report stereo matching accuracy on rectified stereo data, and ideally compare against a variant that enforces the epipolar constraint, to show that this simplification is sound in practice.
- [Ensemble predictions] The consensus-detection mechanism is described only at the level of specifying '-c' and producing 'a consensus set of predictions'. The algorithm for combining multiple detectors -- how candidate boxes from different detectors are matched, how class labels and confidence scores are merged, and how the Gaussian assignment is applied across more than two inputs -- is not specified. Since ensemble consensus is one of the paper's three claimed contributions, the method needs a precise algorithmic description and an evaluation showing that the consensus output is more accurate than any individual detector.
minor comments (5)
- [Abstract and Summary] The opening sentence should read 'There now exist many popular object detectors' rather than 'There now exists many'.
- [Statement of need and Usage] Several formatting artifacts need correction: 'ontherelativepositionanddimensions', 'the--scale', 'tostdout', and 'aspip install' are missing spaces or contain stray hyphens.
- [Controlling tracking sensitivity] The description says to reduce --scale for large frame-to-frame changes, but it does not define the relationship between the Gaussian scale and the coordinate units. State whether --scale is global or per-coordinate, and give the exact formula used to compute the distance, so users can interpret the parameter meaningfully.
- [Examples] The example using --time_pattern frame_\{:d\}.txt does not explain the pattern syntax. A one-sentence note that this is a Python format string matching the frame names would make the option usable without inspecting the source code.
- [Related work] The reference to RetinaNet is cited as 'Focal Loss for Dense Object Detection' with an arXiv URL, while other references use a different style; this is a minor consistency issue, not a substantive one.
Circularity Check
No circularity: yasmot is a tool with no fitted parameters or derived predictions; its operation depends only on standard algorithms and user-set thresholds.
full rationale
The paper makes no attempt to derive a theoretical result or to validate a fitted model. Its central functionality is to link detections across time or stereo views using a Gaussian-weighted distance on bounding-box coordinates, with the sharpness controlled by a user-specified --scale parameter, and to solve the resulting assignment problem with the Hungarian algorithm. The Gaussian temperature is not fitted to any dataset and no quantity is predicted from data used to estimate that quantity; the tracking output is the algorithm's direct output, not a fitted value renamed as a prediction. The cited external works (Hungarian algorithm, object detectors, SORT, ByteTrack, etc.) are standard references and none is invoked as a load-bearing justification that reduces the paper's claim to a self-citation. The paper is self-contained as a software description, with test examples included. Whether a single Gaussian on box coordinates is sufficient for reliable tracking in all regimes is a legitimate robustness or correctness concern, but it is not circularity: the paper does not claim to have derived that sufficiency from evidence, nor does it fit parameters to conceal the limitation. Therefore, no circular step is present.
Assumptions & free parameters
free parameters (2)
- --scale
- --max_age
assumptions (3)
- standard math The Hungarian algorithm provides an optimal bipartite assignment between detections in two frames.
- domain assumption Gaussian distance on position and size is an appropriate affinity measure for linking detections across time and stereo views.
- domain assumption Detector outputs (bounding boxes, class labels, confidence scores) are accurate enough to support tracking.
Cite this review
Pith. "Pith review of YASMOT: Yet another stereo image multi-object tracker." pith.science (2026). https://pith.science/paper/TA3G5MTW
@misc{pith2026250617186,
author = {Pith},
title = {Pith review of: YASMOT: Yet another stereo image multi-object tracker},
year = {2026},
howpublished = {\url{https://pith.science/paper/TA3G5MTW}},
note = {Machine review of arXiv:2506.17186}
}
read the original abstract
There now exists many popular object detectors based on deep learning that can analyze images and extract locations and class labels for occurrences of objects. For image time series (i.e., video or sequences of stills), tracking objects over time and preserving object identity can help to improve object detection performance, and is necessary for many downstream tasks, including classifying and predicting behaviors, and estimating total abundances. Here we present yasmot, a lightweight and flexible object tracker that can process the output from popular object detectors and track objects over time from either monoscopic or stereoscopic camera configurations. In addition, it includes functionality to generate consensus detections from ensembles of object detectors.
Reference graph
Works this paper leans on
-
[2018]
Focal Loss for Dense Object Detection
“Focal Loss for Dense Object Detection.” https://arxiv.org/abs/1708.02002. Redmon, Joseph, Santosh Divvala, Ross Girshick, and Ali Farhadi
-
[2022]
BoT-SORT: Robust Associations Multi-Pedestrian Tracking
“BoT-SORT: Robust Associations Multi-Pedestrian Tracking.”arXiv Preprint arXiv:2206.14651 . Bergmann, Philipp, Tim Meinhardt, and Laura Leal-Taixe
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.