REVIEW 6 cited by
Exploring Plain Vision Transformer Backbones for Object Detection
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We explore the plain, non-hierarchical Vision Transformer (ViT) as a backbone network for object detection. This design enables the original ViT architecture to be fine-tuned for object detection without needing to redesign a hierarchical backbone for pre-training. With minimal adaptations for fine-tuning, our plain-backbone detector can achieve competitive results. Surprisingly, we observe: (i) it is sufficient to build a simple feature pyramid from a single-scale feature map (without the common FPN design) and (ii) it is sufficient to use window attention (without shifting) aided with very few cross-window propagation blocks. With plain ViT backbones pre-trained as Masked Autoencoders (MAE), our detector, named ViTDet, can compete with the previous leading methods that were all based on hierarchical backbones, reaching up to 61.3 AP_box on the COCO dataset using only ImageNet-1K pre-training. We hope our study will draw attention to research on plain-backbone detectors. Code for ViTDet is available in Detectron2.
Forward citations
Cited by 6 Pith papers
-
SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images
SMARTIES, a single masked-autoencoder foundation model with spectrum-aware band projections and cross-sensor token mixup, handles multiple remote sensing sensors and transfers to unseen sensors via interpolation.
-
FUSEP: A Multi-Center Benchmark for Diverse Tasks in Early Pregnancy Fetal Ultrasound Screening
FUSEP is a new multi-center public benchmark with 4,017 early-pregnancy ultrasound images, 45,820 box annotations of 14 structures, and detection baselines across four learning paradigms.
-
ISRS-DETR: Detection-Guided Click Propagation for Remote Sensing Interactive Segmentation
ISRS-DETR combines a CrossCut segmenter with an RF-DETR detector and a dynamic top-K click selector to propagate a single user click to multiple same-class instances, reducing per-image click counts by 3–28 on three b...
-
Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces
VeBrain unifies perception, spatial reasoning, and robot control in one MLLM by representing control as keypoint detection plus skill selection, with a robotic adapter for deployment.
-
Manager: Aggregating Insights from Unimodal Experts in Two-Tower VLMs and MLLMs
Manager aggregates multi-layer unimodal representations and improves both two-tower VLMs (ManagerTower) and MLLMs (LLaVA-OV-Manager) on 24 downstream tasks.
-
Pan-Arctic Permafrost Landform and Human-built Infrastructure Feature Detection with Vision Transformers and Location Embeddings
Vision Transformers with SatCLIP location embeddings beat prior CNN baselines on thaw slump and ice-wedge polygon detection test splits, but not on infrastructure detection.
Discussion (0). Sign in to comment.