FSDC-DETR: A Frequency-Spatial Domain Collaborative DETR for Small Object Detection
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 01:07 UTCglm-5.2pith:O5X4TUDTrecord.jsonopen to challenge →
The pith
Frequency-Spatial Collaboration Lifts Small Object Detection by 6+ AP
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that the dominant bottleneck in small object detection is high-frequency information degradation during multi-scale feature processing, and that explicitly constructing, propagating, and preserving frequency-domain representations at each stage of the detection pipeline — backbone fusion, encoder interaction, and downsampling — yields large, consistent gains. The ablation study (Table 2) shows that the DBFSAF module alone accounts for 5.0 of the 6.4 AP improvement, suggesting that the initial alignment of heterogeneous ViT-CNN frequency characteristics is the single most impactful design choice, with the encoder fusion (SFS-FF) and wavelet-based downsampling (FSD
What carries the argument
Three modules form the frequency-spatial pipeline: (1) DBFSAF — applies Frequency Dynamic Convolution to concatenated ViT-CNN features, then partitions channels into a frequency-refinement path (FPU) and a spatial-refinement path (MKSRU) with multi-kernel depthwise dilated convolutions, recombining via softmax-weighted channel attention; (2) SFS-FF — splits encoder features into spatial (FFT-based 3x3 conv in frequency domain), frequency (spatial refine module), and residual streams, fusing them bidirectionally across scales; (3) FSD-Down — uses 2D discrete wavelet transform to decompose features into four sub-bands (LL, LH, HL, HH), with a learnable scaling factor and grouped convolution to
If this is right
- If high-frequency preservation is the primary lever for small object detection, then any detection architecture using dual-branch backbones with heterogeneous frequency properties should benefit from explicit frequency alignment before fusion, not just DETR variants.
- The wavelet-based downsampling module (FSD-Down) is architecturally independent of the DETR framework and could be dropped into any multi-scale feature pyramid (e.g., FPN, PAN) as a replacement for strided convolution, potentially benefiting CNN-based detectors as well.
- The finding that DBFSAF alone contributes 5.0 of 6.4 AP suggests that the backbone fusion stage is where frequency mismatch matters most, which could redirect architectural effort toward earlier-stage frequency alignment rather than encoder-level modifications.
- If the frequency-spatial modeling principle generalizes, it should produce comparable gains on other small-object-heavy domains such as medical lesion detection, industrial defect detection, and satellite imagery, beyond the drone and aerial benchmarks tested.
Where Pith is reading between the lines
- The paper does not report FLOPs or inference speed, so it is unclear whether the 8.3M additional parameters from the three modules incur a computational cost that would change the Pareto frontier relative to simply scaling up the baseline model.
- The ablation uses the same training recipe for all configurations, but the main comparison table compares FSDC-DETR (40.3M params) against baselines at different parameter counts, making it impossible to fully isolate whether the gains stem from the frequency-spatial principle or partly from increased model capacity.
- The wavelet decomposition in FSD-Down uses a fixed Haar-type DWT with a single learnable scaling factor; a more flexible learnable wavelet basis or per-sub-band adaptive weighting might yield further gains, though at increased parameter cost.
- The frequency response analysis in Figure 1 is qualitative and limited to four channels; a systematic quantitative comparison of spectral energy preservation across all channels and all stages would strengthen the causal claim that the modules work by preserving high frequencies rather than by some other mechanism.
Load-bearing premise
The paper assumes that the 6.4 AP gain over the baseline is attributable to the frequency-spatial modeling principle rather than to the 8.3M additional parameters the new modules introduce, and it does not report FLOPs or inference speed to rule out the possibility that simply scaling up the baseline would yield comparable improvements.
What would settle it
If one added 8.3M parameters to DEIMv2-L via non-frequency-aware modules (e.g., additional transformer layers or wider channels) using the same training recipe and achieved comparable AP gains, the central claim that explicit frequency-spatial modeling is the causal mechanism would be undermined.
Figures
read the original abstract
Small object detection (SOD) remains a challenging task in real-world applications. Despite recent advances, existing detectors remain limited by rigid processing that entangle spatial aggregation with implicit frequency aliasing and truncation, leading to inadequate preservation of high-frequency components for SOD. To tackle these limitations, we propose a Frequency-Spatial Domain Collaborative Detection Transformer (FSDC-DETR), a novel collaborative framework that explicitly models complementary spatial and frequency representations. Specifically, we first introduce Dual-Branch Frequency-Spatial Adaptive Fusion (DBFSAF) to enhance frequency diversity and adaptively capture frequency-spatial domain discriminative representations. Building on these representations, a frequency-spatial interaction scheme is further explored within the hybrid encoder to enable progressive feature propagation to the decoder. In particular, structure-aware frequency-spatial aggregation is achieved through Shunt Frequency-Spatial Feature Fusion (SFS-FF), establishing bidirectional interaction and progressive cross-scale propagation between frequency and spatial representations for coherent discriminative modeling. Meanwhile, informative high-frequency responses are preserved during scale transitions through Frequency-Spatial Dynamic Downsampling (FSD-Down), thereby minimizing frequency degradation throughout multi-scale fusion for the precise SOD. Experimental results demonstrate that FSDC-DETR achieves state-of-the-art performance, improving AP by 6.4 on VisDrone-DET2019 and 6.6 on AITODv2, with gains of 6.8 and 6.9 AP for small objects. The code is available at github.com/nevereverinsomnia/FSDC-DETR.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FSDC-DETR, a frequency-spatial domain collaborative DETR for small object detection (SOD). Built on DEIMv2, the approach introduces three modules: (1) DBFSAF, which replaces DEIMv2's concatenation-based fusion of CNN and ViT features with a frequency-dynamic convolution and partial refinement scheme; (2) SFS-FF, a shunt frequency-spatial feature fusion strategy within the hybrid encoder; and (3) FSD-Down, a wavelet-based downsampling operator that preserves high-frequency components during scale transitions. The method is evaluated on VisDrone-DET2019 and AITODv2, reporting AP improvements of 6.4 and 6.6 points over DEIMv2-L, with substantial small-object (APs) gains. The ablation study (Table 2) shows progressive contributions from each module. The code is publicly available.
Significance. The paper addresses a well-motivated problem: the frequency-domain mismatch between CNN and ViT branches in dual-backbone detectors and the loss of high-frequency information during multi-scale fusion. The reported gains on two SOD benchmarks are substantial, particularly the APs improvements (+6.8 and +6.9 over DEIMv2-L). The public code release and the structured ablation in Table 2 are commendable. The frequency response analysis in Fig. 1, while descriptive, provides useful intuition for the architectural design. However, the significance of these results is tempered by the absence of computational cost reporting (FLOPs, latency) and the lack of per-module parameter counts in the main text, which are needed to assess whether the gains are attributable to the frequency-spatial modeling principle specifically rather than to increased model capacity.
major comments (4)
- §4.4, Table 1: The main comparison table reports parameter counts but not FLOPs or inference latency. FSDC-DETR (40.3M) is compared against DEIMv2-L (32.0M) and other models with varying scales. Without FLOPs or latency, it is impossible to assess whether the 6.4 AP gain comes at a disproportionate computational cost. This is a standard requirement for detection papers claiming practical improvements. At minimum, FLOPs and FPS should be reported for FSDC-DETR and the directly compared baselines (DEIMv2-L, RT-DETRv4-L).
- §4.5, Table 2: The DBFSAF module alone adds +5.0 AP (24.7→29.7), which is the single largest contribution and is surprisingly large for a feature fusion module. The ablation does not report per-module parameter counts, so one cannot distinguish whether this gain reflects the frequency-spatial modeling principle or simply the increased capacity of a more expressive fusion module (FDConv + PFSR with MKSRU and FPU). A parameter-matched non-frequency-aware fusion control would strengthen the attribution claim. The paper references Supplement Sec. C for module-wise parameter statistics; these should be summarized in the main text, at least for DBFSAF.
- §4.5, Table 2 vs. Table 1: The ablation uses the same training recipe for all configurations, which is appropriate. However, the main comparison table compares FSDC-DETR (40.3M) against DEIMv2-L (32.0M) without isolating whether the additional 8.3M parameters account for a portion of the gain independent of the frequency-spatial principle. The comparison with DEIMv2-X (50.0M, 26.5 AP) partially mitigates the pure-scaling concern, but backbone scaling and fusion-layer scaling are not directly analogous. The paper should explicitly discuss this confound and, if possible, report a capacity-matched control.
- §4.4, Table 1: The comparison includes models at widely varying scales (2.9M to 64.0M) and architectures (YOLO, DETR). While FSDC-DETR achieves the highest AP, the lack of FLOPs makes it difficult to assess efficiency-accuracy trade-offs. Additionally, some compared methods (e.g., D-FINE-X at 62.0M, RT-DETRv4-X at 62.0M) are much larger yet achieve lower AP, which the paper should discuss as evidence that the gains are not purely from scaling. This would partially address the capacity confound.
minor comments (7)
- §3.2, Eq. (1): The notation F^P_L and F^P_H is introduced as low-frequency and high-frequency feature maps from HGNetv2 and DINOv3, but the subscripts L and H appear to refer to the branch identity (CNN vs. ViT) rather than frequency content. This is confusing given the paper's frequency-domain framing. Clarify whether L/H denote branches or frequency bands.
- §3.3, Eq. (10)-(11): The channel allocation for F_S, F_F, and F-bar is given as C/2, 3C/8, and C/8 respectively. These sum to C, which is consistent, but the rationale for this specific allocation is not discussed. A brief justification or a sensitivity study would help.
- §3.4, Eq. (12)-(13): The FSD-Down module uses both DWT and grouped convolution (GConv). The relationship between the GConv output and the DWT sub-bands is not fully clear. Clarify whether GConv is meant to complement DWT or serve as an alternative path, and how their outputs interact via the channel attention F_CA.
- Fig. 1: The frequency response analysis compares CNN, ViT, DEIMv2, and FSDC-DETR backbones. However, it is unclear at which stage of the pipeline these responses are measured and how they are computed. Adding methodological details (e.g., feature extraction stage, computation method) would improve reproducibility.
- §4.2: The training uses 8×NVIDIA H200 GPUs, which are not commonly available. While this does not affect correctness, reporting training time and whether the method can be trained on more modest hardware would broaden accessibility.
- Table 1: The AITODv2 column for FSDC-DETR reports AP=32.3, but the text in §4.4 states 'FSDC-DETR achieves 31.1 AP' on AITODv2. This appears to be a typo; the text should say 32.3 AP for AITODv2 (31.1 is the VisDrone result).
- References: Several references have 2025-2026 dates (e.g., [3] SAM3, [35] RT-DETRv4, [56] DINOv3). Ensure these are correctly cited and that arXiv preprints are labeled as such, not as published proceedings, unless they have been formally accepted.
Simulated Author's Rebuttal
We thank the referee for the careful and constructive review. The core concern is whether FSDC-DETR's gains are attributable to the frequency-spatial modeling principle or to increased model capacity, compounded by the absence of FLOPs/latency reporting. We agree that these are important points. In the revised manuscript, we will (1) add FLOPs and FPS for FSDC-DETR and key baselines (DEIMv2-L, RT-DETRv4-L) to Table 1; (2) summarize per-module parameter counts for DBFSAF (and the other modules) in the main text, moving the key figures from Supplement Sec. C; (3) add an explicit discussion of the capacity confound, including the DEIMv2-X comparison and the fact that larger models (D-FINE-X, RT-DETRv4-X) achieve lower AP; and (4) add a parameter-matched non-frequency-aware fusion control for DBFSAF to strengthen attribution. We believe these revisions substantially address the referee's concerns.
read point-by-point responses
-
Referee: §4.4, Table 1: Missing FLOPs and inference latency. FSDC-DETR (40.3M) vs DEIMv2-L (32.0M) — cannot assess whether 6.4 AP gain comes at disproportionate computational cost. At minimum, FLOPs and FPS for FSDC-DETR and directly compared baselines (DEIMv2-L, RT-DETRv4-L).
Authors: The referee is correct that FLOPs and latency are standard reporting requirements and their omission is a gap in the current manuscript. We will add FLOPs and FPS (measured on a single NVIDIA H200 GPU at 800×800 input) for FSDC-DETR and the directly compared baselines — at minimum DEIMv2-L and RT-DETRv4-L, and ideally all models in Table 1 — to the revised Table 1. We note that FSDC-DETR's modules are designed to be lightweight: the partial channel strategy in DBFSAF (γ=0.5) processes only half the channels through the refinement path, SFS-FF uses a channel-split design with only 3C/8 allocated to frequency processing, and FSD-Down leverages efficient wavelet decomposition rather than heavy learned convolutions. We expect the computational overhead to be modest relative to the 8.3M parameter increase, but we agree the actual numbers must be reported for the reader to judge. revision: yes
-
Referee: §4.5, Table 2: DBFSAF alone adds +5.0 AP (24.7→29.7), surprisingly large for a feature fusion module. No per-module parameter counts — cannot distinguish frequency-spatial principle from increased capacity. Parameter-matched non-frequency-aware fusion control would strengthen attribution. Per-module parameter statistics from Supplement Sec. C should be summarized in main text, at least for DBFSAF.
Authors: We agree that the +5.0 AP contribution from DBFSAF warrants careful attribution analysis, and the referee's suggestion for a parameter-matched control is well-taken. We will make two changes: (1) We will summarize the per-module parameter counts (currently in Supplement Sec. C) in the main text, at least for DBFSAF and preferably for all three modules, so the reader can immediately assess the capacity cost of each component. (2) We will add a parameter-matched non-frequency-aware fusion control: specifically, a fusion module with comparable parameter count to DBFSAF but using standard convolutional fusion (e.g., multi-kernel depthwise convolutions without FDConv or FPU) in place of the frequency-aware components. This will directly test whether the gain comes from the frequency-spatial modeling principle or from increased fusion capacity. We note that the ablation in Table 3 already provides partial evidence: setting γ=0 (disabling the partial refinement path entirely while retaining FDConv) yields 29.7 AP, and the refinement path adds only +1.4 AP (29.7→31.1) despite being the more parameter-heavy component, suggesting that the frequency-dynamic convolution itself — not the additional refinement capacity — drives the larger portion of the gain. However, we agree that a direct parameter-matched control is the cleaner experiment and will include it. revision: yes
-
Referee: §4.5, Table 2 vs. Table 1: FSDC-DETR (40.3M) vs DEIMv2-L (32.0M) — additional 8.3M parameters may account for portion of gain independent of frequency-spatial principle. DEIMv2-X (50.0M, 26.5 AP) partially mitigates scaling concern, but backbone scaling ≠ fusion-layer scaling. Paper should explicitly discuss this confound and, if possible, report a capacity-matched control.
Authors: The referee raises a valid concern about the capacity confound. We will add an explicit discussion of this issue to Section 4.4. The key points we will make: (1) DEIMv2-X (50.0M, 26.5 AP) has 9.7M more parameters than FSDC-DETR yet achieves 4.6 AP lower, indicating that naive scaling does not account for our gains. (2) D-FINE-X (62.0M, 26.6 AP) and RT-DETRv4-X (62.0M, 27.1 AP) are substantially larger yet also lower, further supporting that the gains are not purely from capacity. (3) We acknowledge the referee's point that backbone scaling and fusion-layer scaling are not directly analogous, so the DEIMv2-X comparison is suggestive but not definitive. (4) The parameter-matched control for DBFSAF (described in our response to the previous comment) will provide a more direct test. If feasible within the revision timeline, we will also report a full capacity-matched control (DEIMv2-L with a non-frequency-aware fusion module matched to FSDC-DETR's total parameter count). We commit to at least the DBFSAF-level control and the explicit discussion. revision: partial
-
Referee: §4.4, Table 1: Models at widely varying scales (2.9M to 64.0M) and architectures. Lack of FLOPs makes efficiency-accuracy trade-off assessment difficult. Some larger methods (D-FINE-X 62.0M, RT-DETRv4-X 62.0M) achieve lower AP — paper should discuss this as evidence gains are not purely from scaling.
Authors: We agree and will add both the FLOPs/FPS data (as addressed in our first response) and an explicit discussion of the scaling evidence. Specifically, we will note that D-FINE-X (62.0M, 26.6 AP) and RT-DETRv4-X (62.0M, 27.1 AP) are 1.54× and 1.54× larger than FSDC-DETR (40.3M, 31.1 AP) yet achieve 4.5 and 4.0 AP lower, respectively. This demonstrates that the frequency-spatial modeling principle provides gains beyond what parameter scaling alone achieves. We will also discuss the efficiency-accuracy trade-off in the context of the FLOPs data once collected, positioning FSDC-DETR relative to both smaller and larger models. This discussion will be integrated into the revised Section 4.4. revision: yes
Circularity Check
No circularity found: empirical systems paper with externally benchmarked results and no self-citation chain
full rationale
This is an empirical systems paper for small object detection. The central claim — that FSDC-DETR improves AP on VisDrone-DET2019 and AITODv2 — is measured against external benchmarks using standard COCO-style metrics (Section 4.3, Table 1). The proposed modules (DBFSAF, SFS-FF, FSD-Down) are trained end-to-end with standard detection losses; no module output is defined in terms of the evaluation metric, and no parameter is fitted to the target benchmark and then 'predicted' on the same data. The frequency response analysis in Fig. 1 is descriptive/motivational, not used to fit parameters or define the target metric. The ablation study (Table 2) uses the same training recipe across configurations and measures each variant on the held-out test split. While the paper cites prior work by overlapping authors (e.g., DEIMv2 [24] is by Huang et al., a different group; FDConv [9] and SFS-Conv [33] are by Chen et al. and Li et al. respectively, also different groups), none of these citations are load-bearing in the circularity sense: the paper does not invoke a self-authored 'uniqueness theorem' to forbid alternatives, nor does it smuggle in an ansatz via self-citation. The architectural components (wavelet downsampling, frequency dynamic convolution, partial channel partitioning) are each independently parameterized and trained, not defined circularly. The concern about whether the +5.0 AP from DBFSAF reflects frequency-spatial modeling versus added capacity is a valid correctness/attribution concern, but it is not circularity — the ablation does not reduce to its inputs by construction. The derivation chain is self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (5)
- gamma (partial ratio in DBFSAF) =
0.5
- alpha (learnable scalar in PFSR) =
learned during training
- beta (learnable scalar in PFSR) =
learned during training
- rho (learnable scaling factor in FSD-Down) =
learned during training
- Channel allocation ratios in SFS-FF =
C/2, 3C/8, C/8
axioms (4)
- domain assumption ViTs exhibit low-pass filtering behavior while CNNs preserve high-frequency components
- domain assumption High-frequency components are critical for small object detection
- domain assumption Standard convolution-based downsampling causes frequency aliasing that degrades small object features
- domain assumption DEIMv2's simple concatenation of dual-branch features is suboptimal for SOD
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.