Pith. sign in

REVIEW 3 major objections 4 minor 32 references

Instance Scale Normalization for image understanding

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Restricting every object to a single size range [16,560] during both training and testing compresses the scale space of objects and improves detection, segmentation, and keypoint estimation over multi-scale baselines.

desk verdict A real but under-ablated extension of SNIP/SNIPER: the consistent scale range is a genuine idea, but the reported gains are confounded by inherited chip sampling. read the letter →

arxiv 1908.07323 v2 pith:27AWK3Y6 submitted 2019-08-20 cs.CV

Zewen He , He Huang , Yudong Wu , Guan Huang , Wensheng Zhang This is my paper · ORCID
classification cs.CV
keywords instancescalenormalizationobjectdetectionvariationmulti-scaletrainingfeaturepyramidnetworkCOCOsegmentationhumanposeestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the longstanding difficulty of detecting objects at very different sizes can be largely neutralized by forcing all objects into a single, fixed size range during both training and testing. The method, called Instance Scale Normalization (ISN), resizes each image through a fixed set of scales and then keeps only those objects whose size in the resized image falls between 16 and 560 pixels. ISN does not change the network architecture, yet the paper reports that it consistently outperforms standard multi-scale training and testing on object detection, instance segmentation, and human pose estimation across several backbones, including 46.5 mAP on COCO test-dev with a ResNet-101. The paper presents this as evidence that scale variation is not a property the network must learn to absorb, but a range of inputs that can be filtered.

What carries the argument

The load-bearing device is the ISN range [sl, su] = [16, 560] applied as a filter on object scales in resized images. It converts scale variation from a continuous wide spectrum into a bounded interval, and the same interval is reused for training and testing, so the distribution shift between phases is removed. The range is found by a greedy search on the COCO validation set: first the lower bound is raised from 0 to 16, then the upper bound is lowered from 640 to 560, each step chosen by local mAP. With FPN integrated, each pyramid stage receives objects in a narrower effective range, which the paper argues eases optimization and avoids the self-contradictory sampling it attributes to SNIP.

What would settle it

Train an ISN detector on COCO with the range [16,560] and evaluate it on a dataset whose object scales are mostly below 16 pixels or above 560 pixels, such as aerial or pathology images; if the method's accuracy falls below that of a standard multi-scale detector, it would show that the gains come from matching the tuned range rather than from the normalization principle itself.

Watch

Extended reading notes

Core claim

ISN's central claim is that a consistent scale interval applied identically in training and testing compresses the scale space of objects, letting a detector work in a moderate, uniform regime instead of facing extreme sizes. For a multi-resolution image pyramid with scaling factors {4.0, 2.0, 1.0, 0.5, 0.25}, ISN first resizes each image, then discards any proposal whose scale in that resized image lies outside [16,560]. The same rule is used at test time, so the model never sees objects outside this range in either phase. Because feature pyramid network (FPN) stages each specialize in a sub-range of scales, ISN integrates with FPN to keep a wide feasible range while still letting each stage train on objects of consistent size. The paper reports that this yields gains over multi-scale training and testing baselines in all three tasks, with the largest relative gains on small objects and on keypoint detection.

Load-bearing premise

The fixed ISN range [16,560] is chosen by a greedy search on the COCO validation set, and the paper assumes that this range transfers to the test distribution and to other tasks and backbones without re-tuning, so the reported gains may depend on matching the validation distribution.

Editorial extensions

If this is right

  • A detector trained with ISN works on a specified scale range [16,560] and can be tested at a single resolution, giving accuracy nearly as high as multi-scale testing while running faster.
  • ISN improves performance on small objects substantially; for example, APs on COCO val with ResNet-50-FPN rises from 26.9 with multi-scale training to 31.4 with ISN.
  • ISN transfers beyond detection: it improves instance segmentation mask AP by around 1 point and keypoint AP by more than 3.5 points over multi-scale training and testing.
  • ISN is backbone-agnostic, bringing the largest relative gains to small models like ResNet-18 and MobileNet-v2, making it a cheap accuracy boost for real-time applications.
  • The single ResNet-101 ISN model reaches 46.5 mAP on COCO test-dev, competitive with state-of-the-art detectors that use heavier machinery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mechanism is genuinely about compressing scale space rather than about the specific [16,560] bound, the same trick may apply to other continuous nuisance variations—such as rotation, aspect ratio, or illumination—where extreme values are rare but harmful.
  • Because the range is tuned on COCO val, a stress test on a dataset with a different scale distribution (for example, aerial imagery with mostly small objects) would show whether the gains come from the principle itself or from matching the evaluation range.
  • The reported success suggests a testable extension: instead of a fixed range, one could learn a per-dataset or per-class range that adapts to the object scale distribution, which might outperform the hand-tuned interval.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Instance Scale Normalization (ISN), a training and testing scheme that restricts object instances to a consistent scale range [sl, su] (set to [16, 560]) across all image-pyramid resolutions. Using the same chip-generation strategy as SNIPER, ISN discards objects outside the range in both training and testing, with the goal of compressing the scale space and reducing optimization difficulty. Experiments on COCO report consistent improvements over multi-scale training/testing baselines on object detection (Tables 4, 6), instance segmentation (Table 8), and human pose estimation (Table 9), with a single ResNet-101 model reaching 46.5 mAP on test-dev (Table 3).

Significance. If the improvements are attributable to ISN's scale-range filtering, the method would offer a simple, architecture-agnostic alternative to multi-scale training and testing, with practical benefits for small backbones and single-scale inference. The paper is honest about the validation-set greedy search for the ISN range (Section 4.2.1) and provides consistent, controlled baselines across three tasks and several backbones, which is a strength. The central empirical claim is plausible, but the paper currently does not separate the contribution of scale-range filtering from the contribution of the inherited SNIPER-style chip sampling, and the SOTA comparison in Table 3 is confounded by the use of FPN in ISN but not in SNIP/SNIPER. These issues require additional experiments before the claimed attribution can be accepted.

major comments (3)
  1. [Section 3.3.1; Tables 4, 6, 8, 9] The central comparisons do not isolate the effect of scale-range normalization from the effect of SNIPER-style chip sampling. Section 3.3.1 states that ISN uses 'the same chip generation strategy in SNIPER [27]', and Sections 4.2.4 and 4.3.2 refer to the patch size 576 and patches used in ISN training. The multi-scale baselines, by contrast, are trained on full images with image pyramids and no chip sampling. Since SNIPER already demonstrated that object-centric chip sampling improves detection, the reported +2 to +5 AP gains may be largely due to the inherited chip mechanism rather than the [16, 560] scale filter. An ablation is needed: e.g., (a) add SNIPER-style chips to the multi-scale baseline without scale filtering, and (b) run ISN on full images without chips, while keeping all other settings fixed. Without such an ablation, the causal claim that compressing the scale space drives the improvement is uncontrolled.
  2. [Table 3 (SOTA comparison)] The state-of-the-art comparison is confounded by FPN. SNIP and SNIPER are reported on ResNet-50/ResNet-101 without FPN, while ISN is reported on ResNet-*-FPN backbones. The text claims ISN 'surpasses previous methods hugely', but the comparison mixes a change in scale-sampling method with a change in network architecture. A fair comparison would include SNIP or SNIPER augmented with FPN, or ISN without FPN, or otherwise hold the feature-pyramid component fixed. This is important because the paper's own Section 3.3.2 argues that FPN integration is a deliberate and beneficial part of ISN; the reader cannot tell how much of the Table 3 margin comes from ISN versus from FPN.
  3. [Section 4.2.1 and Tables 4-9] The ISN range [16, 560] is selected by greedy search on the COCO validation set, and the same validation set is used to report most of the subsequent improvements (Tables 4, 6, 8, 9). This creates a risk of selection bias in the reported margins, since the hyperparameter is optimized on the evaluation set. The paper would be strengthened by reporting results on the validation set under a pre-registered default range (e.g., [16, 640] or [0, 560]) or by reporting test-dev results for all key configurations. The single test-dev result in Table 3 is welcome, but it is not accompanied by an ablation that removes the chip confound, so it does not resolve the attribution issue in the first major comment.
minor comments (4)
  1. [Section 3.2.1] There is a spelling error: 'normalizatoin' should be 'normalization'.
  2. [Section 4.2.1] The description of the third greedy-search step reads 'sl was set to 560 and su = 32 is evaluated continuously'. This appears to be a typo: the context indicates sl=32 and su=560 should be evaluated. Please correct.
  3. [Table 4] The row label 'ResNet-101-FP' should read 'ResNet-101-FPN'. Also, the footnote marks SNIPER as evaluated on COCO test-dev, while the '+ISN' row has no asterisk; if the ISN row is on COCO val2017, the comparison between SNIPER and +ISN within this table mixes evaluation sets and should be clarified or moved to Table 3.
  4. [Section 4.2.4] The claim that ISN is 'robust for scale variation' when changing from a predefined test resolution to the original resolution is based on a single set of numbers in Table 7; reporting the standard deviation across multiple runs or the per-scale breakdown would make the robustness claim more convincing.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: ISN's claims are empirical and externally evaluated; the scale range is a tuned hyperparameter, not an input that defines the reported metric.

full rationale

ISN's central claims are empirical: filtering objects outside [16,560] and testing on COCO improve measured AP compared with multi-scale baselines. The range [16,560] is found by greedy search on the validation set (Sec. 4.2.1), but it is a hyperparameter of the training procedure, not a term in the evaluation metric. The main headline result, 46.5 mAP on COCO test-dev, is an external benchmark not used for tuning. The paper explicitly credits the chip generation strategy to SNIPER [27], so no ansatz is smuggled through self-citation. The restricted metric AP[16,560] in Table 7 is deliberately matched to the ISN range, but the paper states this explicitly and uses it as a secondary, application-oriented evaluation; it does not feed back into the derivation of the primary detection claims. The principal weaknesses (lack of an ablation separating chip sampling from scale filtering, and validation-set tuning) are experimental-control concerns, not circular reductions. Consequently, no load-bearing step reduces to its own inputs by construction.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical entities. Its free parameters are the scale-range bounds and the pyramid scaling factors; the bounds are fitted on the validation set used for final scores. The background assumptions are standard for detection literature, plus the domain-specific belief that moderate scales are easier to learn.

free parameters (3)
  • ISN range lower bound sl = 16
    Chosen via greedy search on COCO val (Sec 4.2.1); objects below this size (in resized image) are excluded from training and testing.
  • ISN range upper bound su = 560
    Chosen via greedy search on COCO val (Sec 4.2.1); objects above this size are excluded.
  • Image pyramid scaling factors Omega = {4.0, 2.0, 1.0, 0.5, 0.25}
    Hand-selected; no ablation shown for this choice, and it largely defines the multi-scale setup.
assumptions (3)
  • domain assumption Objects in a moderate scale range are easier for CNNs to learn and generalize than extreme-scale objects.
    Motivates filtering; supported by SNIP and Table 2, but not proven.
  • domain assumption The receptive field and semantic scope of a RoI should be consistent, and this holds only for moderate object sizes.
    Cited from [30] in Sec 1; used to justify why extreme scales hurt.
  • domain assumption FPN's map from RoI scale to pyramid level remains reliable after ISN filtering.
    Sec 3.3.2 assumes FPN integration is beneficial and not disrupted by the filtering.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Instance Scale Normalization for image understanding." pith.science (2026). https://pith.science/paper/27AWK3Y6

@misc{pith2026190807323,
  author       = {Pith},
  title        = {Pith review of: Instance Scale Normalization for image understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/27AWK3Y6}},
  note         = {Machine review of arXiv:1908.07323}
}
read the original abstract

Scale variation remains a challenging problem for object detection. Common paradigms usually adopt multiscale training & testing (image pyramid) or FPN (feature pyramid network) to process objects in a wide scale range. However, multi-scale methods aggravate more variations of scale that even deep convolution neural networks with FPN cannot handle well. In this work, we propose an innovative paradigm called Instance Scale Normalization (ISN) to resolve the above problem. ISN compresses the scale space of objects into a consistent range (ISN range), in both training and testing phases. This reassures the problem of scale variation fundamentally and reduces the difficulty of network optimization. Experiments show that ISN surpasses multi-scale counterpart significantly for object detection, instance segmentation, and multi-task human pose estimation, on several architectures. On COCO test-dev, our single model based on ISN achieves 46.5 mAP with a ResNet-101 backbone, which is among the state-of-the-art (SOTA) candidates for object detection.

Figures

Figures reproduced from arXiv: 1908.07323 by the authors.

Figure 1
Figure 1. (e) gives the distribution of training objects in SNIP. As it shows, there are plentiful extremely tiny, and huge ob￾jects participating in the training phase. The [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. ISN plus FPN: Here shows ISN for multi-scale training & testing. In training, ISN firstly resizes original image ωi times to get i-th resolution. Then it select objects which scale falls in ISN range (marked by green boxes) as valid for training, otherwise discards invalid boxes (red). In testing, only the predicted boxes from different resolutions in ISN range are preserved and fused. 3.3.1 Instance scale normaliza… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 23 canonical work pages

  1. [27]

    Singh, M

    B. Singh, M. Najibi, and L. S. Davis. Sniper: Efficient multi- scale training. InAdvances in Neural Information Processing Systems, pages 9333–9343, 2018. 1, 2, 4, 5

    work page 2018

  2. [1]

    E. H. Adelson, C. H. Anderson, J. R. Bergen, P. J. Burt, and J. M. Ogden. Pyramid methods in image processing. RCA engineer, 29(6):33–41, 1984. 2

    work page 1984

  3. [2]

    Bodla, B

    N. Bodla, B. Singh, R. Chellappa, and L. S. Davis. Soft- nms–improving object detection with one line of code. In Proceedings of the IEEE international conference on com- puter vision, pages 5561–5569, 2017. 5

    work page 2017

  4. [3]

    Z. Cai, Q. Fan, R. S. Feris, and N. Vasconcelos. A unified multi-scale deep convolutional neural network for fast ob- ject detection. In Proceedings of the european conference on computer vision, pages 354–370. Springer, 2016. 2

    work page 2016

  5. [4]

    J. Dai, H. Qi, Y . Xiong, Y . Li, G. Zhang, H. Hu, and Y . Wei. Deformable convolutional networks. In Proceedings of the IEEE international conference on computer vision , pages 764–773, 2017. 5

    work page 2017

  6. [5]

    P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ra- manan. Object detection with discriminatively trained part- based models. IEEE transactions on pattern analysis and machine intelligence, 32(9):1627–1645, 2010. 1, 2

    work page 2010

  7. [6]

    Girshick

    R. Girshick. Fast r-cnn. In Proceedings of the IEEE inter- national conference on computer vision , pages 1440–1448,

  8. [7]

    Girshick, J

    R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich fea- ture hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 580–587,

Show all 32 references
  1. [8]

    Girshick, I

    R. Girshick, I. Radosavovic, G. Gkioxari, P. Doll ´ar, and K. He. Detectron. https://github.com/ facebookresearch/detectron, 2018. 5, 6

    2018

  2. [9]

    Goyal, P

    P. Goyal, P. Doll ´ar, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y . Jia, and K. He. Accurate, large minibatch sgd: training imagenet in 1 hour. arXiv preprint arXiv:1706.02677, 2017. 5

    2017 arXiv

  3. [10]

    Z. Hao, Y . Liu, H. Qin, J. Yan, X. Li, and X. Hu. Scale-aware face detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6186–6195,

  4. [11]

    K. He, G. Gkioxari, P. Doll ´ar, and R. Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on com- puter vision, pages 2980–2988. IEEE, 2017. 1, 2, 3, 4, 5, 7, 8

    2017

  5. [12]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learn- ing for image recognition. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 770–778, 2016. 1, 5

    2016

  6. [13]

    Law and J

    H. Law and J. Deng. Cornernet: Detecting objects as paired keypoints. In Proceedings of the european conference on computer vision, pages 734–750, 2018. 2

    2018

  7. [14]

    Y . Li, Y . Chen, N. Wang, and Z. Zhang. Scale-aware trident networks for object detection. arXiv preprint arXiv:1901.01892, 2019. 4

    1901 arXiv

  8. [15]

    T.-Y . Lin, P. Doll´ar, R. B. Girshick, K. He, B. Hariharan, and S. J. Belongie. Feature pyramid networks for object detec- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, volume 1, page 4, 2017. 1, 2, 6

    2017

  9. [16]

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar. Focal loss for dense object detection. arXiv preprint arXiv:1708.02002, 2017. 1, 2

    2017 arXiv

  10. [17]

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ra- manan, P. Doll´ar, and C. L. Zitnick. Microsoft coco: Com- mon objects in context. In Proceedings of the european con- ference on computer vision, pages 740–755. Springer, 2014. 1, 4, 7

    2014

  11. [18]

    W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y . Fu, and A. C. Berg. Ssd: Single shot multibox detector. In Proceedings of the european conference on computer vision, pages 21–37. Springer, 2016. 2

    2016

  12. [19]

    Najibi, P

    M. Najibi, P. Samangouei, R. Chellappa, and L. S. Davis. Ssh: Single stage headless face detector. In Proceedings of the IEEE international conference on computer vision, pages 4875–4884, 2017. 2

    2017

  13. [20]

    Najibi, B

    M. Najibi, B. Singh, and L. S. Davis. Autofocus: Efficient multi-scale inference. arXiv preprint arXiv:1812.01600 ,

    arXiv

  14. [21]

    Papandreou, T

    G. Papandreou, T. Zhu, N. Kanazawa, A. Toshev, J. Tomp- son, C. Bregler, and K. Murphy. Towards accurate multi- person pose estimation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 4903–4911, 2017. 8

    2017

  15. [22]

    Redmon, S

    J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You only look once: Unified, real-time object detection. In Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 779–788, 2016. 2

    2016

  16. [23]

    Redmon and A

    J. Redmon and A. Farhadi. Yolov3: An incremental improve- ment. arXiv preprint arXiv:1804.02767, 2018. 2

    2018 arXiv

  17. [24]

    S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems , pages 91–99, 2015. 1, 2, 3, 5

    2015

  18. [25]

    Sandler, A

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen. Mobilenetv2: Inverted residuals and linear bottle- necks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018. 5, 6

    2018

  19. [26]

    Singh and L

    B. Singh and L. S. Davis. An analysis of scale invariance in object detection–snip. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 3578–3587, 2018. 1, 2, 3, 5

    2018

  20. [28]

    Viola and M

    P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Proceedings of the IEEE conference on computer vision and pattern recognition, vol- ume 1, pages I–I. IEEE, 2001. 2

    2001

  21. [29]

    Viola and M

    P. Viola and M. J. Jones. Robust real-time face detection. International journal of computer vision , 57(2):137–154,

  22. [30]

    J. Wang, K. Chen, S. Yang, C. C. Loy, and D. Lin. Region proposal by guided anchoring. arXiv preprint arXiv:1901.03278, 2019. 1

    1901 arXiv

  23. [31]

    B. Xiao, H. Wu, and Y . Wei. Simple baselines for human pose estimation and tracking. InProceedings of the european conference on computer vision, pages 466–481, 2018. 8

    2018

  24. [32]

    C. Zhu, Y . He, and M. Savvides. Feature selective anchor- free module for single-shot object detection. arXiv preprint arXiv:1903.00621, 2019. 2

    1903 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.