REVIEW 4 major objections 5 minor 18 references
Let Human Sketches Help: Empowering Challenging Image Segmentation Task with Freehand Sketches
T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read Freehand sketches, not clicks or boxes, are the prompt that cracks camouflaged object segmentation: a sketch-guided model matches pixel-by-pixel annotation quality while cutting annotation time by up to 120 times.
desk verdict Promising sketch-guided COD dataset and prompt-comparison results, but the 120x annotation-time claim rests on a misread ANOVA and a train/test prompt mismatch. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is DeepSketchCamo, a SAM backbone adapted for sketch prompts: a ResNet-based sketch encoder extracts features from the freehand drawing, a multi-head cross-attention fusion mechanism modulates the image features with the sketch features, and SAM's mask decoder produces the final mask. Training relies on three supporting components: sketch augmentation, which fits a cubic Bezier curve to sketch strokes and randomly displaces its control points to synthesize imprecise contours that mimic human drawing; a boundary refinement loss that measures precision and recall near object edges; and an adaptive focal loss whose difficulty exponent adjusts with the model's global learning state. Together these parts are meant to let the network use the sketch as a spatial high-level cue while staying robust to the abstraction and sloppiness inherent in freehand drawing.
What would settle it
Collect a training set of real human freehand sketches for a subset of camouflaged images, train DeepSketchCamo on those human sketches, and test on the existing KOSCamo+ test split: if the human-trained model does not achieve performance at least as good as the edge-map-trained model, the paper's central claim that freehand sketch expressiveness drives the improvement would be falsified. A second check compares the same architecture trained with and without sketch augmentation on real human sketches, isolating whether the Bezier perturbation actually closes the synthetic-to-human domain gap.
Extended reading notes
Core claim
The authors claim that giving an interactive segmentation model a freehand sketch of a camouflaged object's rough outline yields better masks than giving it points, boxes, or scribbles, and that this advantage survives even when the sketch is imprecise or strays outside the object boundary. They show this both by plugging sketch input into existing models such as SAM, UniRef++, and RefCod, and by designing DeepSketchCamo, which fuses sketch features with image features in the SAM decoder and adds boundary refinement plus an adaptive focal loss to sharpen predictions. The strongest quantitative claim is that when DeepSketchCamo's predicted masks are used as pseudo-labels to train ZoomNet and UGTR on the NC4K dataset, the resulting models perform on par with the same models trained on human pixel-by-pixel masks, despite the sketch-based annotation being over 120 times faster. The paper frames this as transforming sketch input from a mere prompt into a practical annotation engine that could reduce dependence on labor-intensive pixel-level labeling.
Load-bearing premise
The load-bearing premise is that synthetic sketches generated from ground-truth edge maps with Bezier perturbation are a faithful training proxy for real human freehand sketches, including rough contours that may fall outside the object. Training is conducted on edge maps derived from ground-truth masks, while only test-time sketches come from the human-collected KOSCamo+ dataset, so the headline annotation-time claim rests on an untested transfer from synthetic contours to genuine human drawing behavior.
Editorial extensions
If this is right
- Sketch prompts can replace clicks and boxes in interactive segmentation tools for camouflage-heavy scenes, giving non-expert annotators a faster way to obtain accurate masks.
- Predicted masks from DeepSketchCamo can be used as pseudo ground truth to train downstream segmentation models, potentially reducing annotation cost for new datasets dramatically.
- The KOSCamo+ dataset provides a benchmark and training resource for studying how human-drawn contours encode attention in obscured-object scenes.
- The proposed sketch augmentation and boundary-focused losses are transferable to other prompt-based segmentation pipelines that need robustness to imprecise user input.
Reading between the lines
- A natural extension the authors do not test is whether the approach works for partially occluded or transparent objects, where the sketch contour would be even more ambiguous; the same architecture could be evaluated on those datasets.
- The annotation-time comparison embeds a strong assumption: the 120x figure compares sketch time to full pixel-mask time, but the real workflow gain depends on how often a predicted mask must be manually corrected, so a careful study of correction time would strengthen the practical claim.
- If human sketches carry systematically different information from edge-map-derived synthetic sketches, then the model may be learning to exploit edge geometry rather than true human contour abstraction; comparing performance after fine-tuning on real human sketches versus only synthetic ones would isolate this effect.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a sketch-guided interactive segmentation framework for camouflaged object detection (COD). The authors build DeepSketchCamo on the SAM architecture by adding a sketch encoder, a cross-attention fusion mechanism, domain-specific adapters, a Bezier-curve sketch augmentation method, a boundary refinement loss, and an adaptive focal loss. They introduce KOSCamo+, a freehand sketch dataset covering the test sets of CHAMELEON, CAMO, COD10K, and NC4K. Experiments compare sketch prompts with point, box, and scribble inputs on SAM, UniRef++, and RefCod, and report that DeepSketchCamo achieves state-of-the-art COD results. A further experiment trains ZoomNet and UGTR on DeepSketchCamo's predicted labels and claims the results are comparable to training on pixel-by-pixel annotations, with up to 120x annotation-time savings.
Significance. If the central claims were fully supported, this would be a valuable contribution to interactive segmentation and annotation efficiency. The idea of using freehand sketches as a more expressive prompt is intuitively appealing, and the KOSCamo+ dataset is a potentially useful resource for studying human attention in camouflaged scenes. The paper also ships a substantial set of experiments and ablation studies. However, the headline annotation-equivalence claim rests on a misread statistical test, and the training protocol uses synthetic rather than human sketches in a way that is not disclosed in the main text. Because these issues are load-bearing, the current version does not yet establish the paper's main selling points.
major comments (4)
- [Section 4.3 and Table 3] The ANOVA is misread. In ANOVA, p < 0.05 rejects the null hypothesis that group means are equal, so the reported p-values (p = 0.0323 for ZoomNet, p = 0.0068 for UGTR) indicate statistically significant differences between training on pixel-by-pixel annotations and training on the predicted labels, not consistency. To support the claim that the pseudo-labels are 'on par,' the paper needs an equivalence test with a pre-specified margin (e.g., TOST) or confidence intervals for the performance differences. The ANOVA design is also undocumented: no information is given about the dependent variable, the grouping factor, the unit of analysis, or how the four metrics and multiple datasets were pooled. The supplementary Table 5 reports per-dataset results, but the main-text Table 3 appears to show only one dataset (likely COD10K), so the scope of the statistical claim is unclear.
- [Supplementary D.6 and Section 4.1] The main text does not disclose that training uses synthetic sketches generated by applying the Canny edge detector to ground-truth masks, with Bezier perturbation, and that human sketches from KOSCamo+ are used only at test time. This is a load-bearing assumption for the paper's central claim that the network learns to exploit the expressiveness and error tolerance of human freehand sketches. The manuscript should state this explicitly in the main text and validate the assumption, for example by training on human sketches or by measuring the domain gap between synthetic edges and human drawings. Without such evidence, the annotation-time claim and the claim of learning from human sketches remain untested in the training regime that the model actually uses.
- [Section 4.2 and Section 4.1] The point and box prompts are derived from the sketch in the KOSCamo+ dataset (the bounding rectangle and its center), rather than being elicited under realistic annotation protocols. This could bias the comparison in favor of sketch input, because the derived box is tightly aligned with the sketched object and the point is guaranteed to fall inside the object. The paper should either use standard point/box annotation protocols (e.g., random or simulated user clicks) or explicitly justify why the derived inputs are fair representatives of those modalities.
- [Section 1 and Section 4.4] The introduction states that sketches are error-tolerant and that 'rough sketches within or outside of the object are both fine,' but the experiment in Section 4.4 and Figure 5 shows that drawing the sketch in the wrong region causes the network to fail. The paper should clarify that the claimed tolerance applies to contour roughness and placement near the object boundary, not to location errors, and should adjust the wording in the introduction and related work accordingly.
minor comments (5)
- [Abstract and Section 4.2] The abstract and introduction claim that sketch input outperforms 'text or bounding box annotations,' but no text-prompt baseline appears in Table 2 or elsewhere in the experiments. RefCod is a referring segmentation model, but it is evaluated with sketch and scribble prompts, not with its native text input.
- [Equation (5)] The summation in Eq. (5) is rendered as 'nX' due to a formatting error; it should display the summation from i=1 to n.
- [Figure 7 caption] The caption contains the typo 'Anallysis' instead of 'Analysis.'
- [Section 3.4, Eq. (6)] The variable C is described as the 'number of control unit rows of displacement deformation,' which is difficult to interpret. Please define it concretely or provide a diagram of how the sketch rows are partitioned.
- [Supplementary H] The annotation-time comparison in Figure 8 reports only the time to draw the input, not the time needed to verify or correct the automatically generated masks. A fair comparison of annotation pipelines should include QA or correction time.
Circularity Check
No significant circularity: the central sketch-segmentation and annotation-time claims rest on empirical benchmark comparisons against independent baselines and held-out datasets, not on self-citation or fitted inputs.
full rationale
The paper's derivation chain is self-contained. DeepSketchCamo is trained with ground-truth mask supervision and evaluated against independent baselines on standard COD benchmarks. The claim that sketch prompts outperform point, box, and scribble prompts is an empirical comparison in Table 2, not a construction. The pseudo-label annotation experiment in Section 4.3 trains ZoomNet and UGTR on model-generated labels for the held-out NC4K set and compares against the same architectures trained on pixel-by-pixel ground truth; this is a legitimate transfer test. Self-citations to the SAM-Adapter line provide a backbone and baseline, but no load-bearing theorem is imported from those papers; the architecture and losses are specified in this manuscript. Supplementary D.6 reveals that training sketches are derived from ground-truth mask edge maps with Bezier perturbation, which is a domain-gap and validity concern for the 'freehand sketch' claim, but it is not circular because the model still predicts masks from sketch inputs under ground-truth supervision. The Section 4.3 ANOVA sentence misuses the p-value, but this is a correctness issue rather than a circular reduction.
Assumptions & free parameters
free parameters (5)
- n (patch count in sketch augmentation) =
64
- K (sketch augmentation displacement multiplier) =
8
- theta1 and theta2 (boundary refinement pooling window sizes) =
3, 3
- theta (adaptive focal loss difficulty weight) =
0.5
- Loss coefficients lambda_mask, lambda_dice, lambda_adaptive, lambda_boundary =
2.0, 5.0, 5e-4, 1.0
assumptions (4)
- domain assumption SAM pretrained features transfer to camouflaged object detection after lightweight adapters.
- ad hoc to paper Synthetic sketches generated from Canny edge maps of GT masks plus Bezier perturbation are a faithful training proxy for human freehand sketches.
- domain assumption Rough sketches inside or outside the object are valid prompts and carry enough spatial and semantic signal.
- standard math Standard COD benchmarks and metrics (S-measure, E-measure, weighted F, MAE) are valid evaluation proxies.
Cite this review
Pith. "Pith review of Let Human Sketches Help: Empowering Challenging Image Segmentation Task with Freehand Sketches." pith.science (2026). https://pith.science/paper/BUJ47HTA
@misc{pith2026250119329,
author = {Pith},
title = {Pith review of: Let Human Sketches Help: Empowering Challenging Image Segmentation Task with Freehand Sketches},
year = {2026},
howpublished = {\url{https://pith.science/paper/BUJ47HTA}},
note = {Machine review of arXiv:2501.19329}
}
read the original abstract
Sketches, with their expressive potential, allow humans to convey the essence of an object through even a rough contour. For the first time, we harness this expressive potential to improve segmentation performance in challenging tasks like camouflaged object detection (COD). Our approach introduces an innovative sketch-guided interactive segmentation framework, allowing users to intuitively annotate objects with freehand sketches (drawing a rough contour of the object) instead of the traditional bounding boxes or points used in classic interactive segmentation models like SAM. We demonstrate that sketch input can significantly improve performance in existing iterative segmentation methods, outperforming text or bounding box annotations. Additionally, we introduce key modifications to network architectures and a novel sketch augmentation technique to fully harness the power of sketch input and further boost segmentation accuracy. Remarkably, our model' s output can be directly used to train other neural networks, achieving results comparable to pixel-by-pixel annotations--while reducing annotation time by up to 120 times, which shows great potential in democratizing the annotation process and enabling model training with less reliance on resource-intensive, laborious pixel-level annotations. We also present KOSCamo+, the first freehand sketch dataset for camouflaged object detection. The dataset, code, and the labeling tool will be open sourced.
Figures
Figures from the paper (11 more)
Reference graph
Works this paper leans on
-
[1]
Ablation study on the domain-specific info (Inj. Info.). Method Sm ↑ Em ↑ F ω β ↑ MAE↓ Baseline Inj.Info. CHAMELEON (76 images) ✓ × 0.906 0.966 0.881 0.023 ✓ ✓ 0.903 0.967 0.879 0.024 CAMO (250 images) ✓ × 0.856 0.942 0.833 0.049 ✓ ✓ 0.861 0.943 0.840 0.047 COD10K (2026 images) ✓ × 0.867 0.950 0.807 0.023 ✓ ✓ 0.872 0.954 0.817 0.021 Boundary Refinement, a...
work page 2026
-
[2]
The effectiveness of Adaptive Focal Loss, Boundary Refinement and Sketch Augmentation. Method Sm ↑ Em ↑ Fw β ↑ MAE↓ Adaptive Boundary SA CHAMELEON (76 images) × ✓ ✓ 0.903 0.964 0.878 0.023 ✓ × ✓ 0.904 0.967 0.883 0.022 ✓ ✓ × 0.904 0.962 0.883 0.023 ✓ ✓ ✓ 0.908 0.972 0.887 0.022 CAMO (250 images) × ✓ ✓ 0.859 0.941 0.839 0.049 ✓ × ✓ 0.850 0.928 0.826 0.049 ...
work page 2026
-
[3]
the better the sketch, the better the outcome
Sensitivity Analysis on n in SA on CHAMELEON, CAMO, and COD10K. n CHAMELEON (76 images) CAMO (250 images) COD10K (2026 images) Sm ↑ Em ↑ Fwβ ↑ MAE↓ Sm ↑ Em ↑ Fwβ ↑ MAE↓ Sm ↑ Em ↑ Fwβ ↑ MAE↓ 36 0.906 0.962 0.887 0.023 0.845 0.933 0.822 0.052 0.859 0.945 0.796 0.024 64 0.908 0.972 0.887 0.022 0.861 0.945 0.842 0.048 0.875 0.956 0.825 0.021 81 0.901 0.962 0....
work page 2026
-
[4]
Different Sketch Enhancement/Perturbations in training/testing data affect the performance. Training data Test Data CHAMELEON (76 images) CAMO (250 images) COD10K (2026 images) Sm ↑ Em ↑ F w β ↑ MAE↓ Sm ↑ Em ↑ F w β ↑ MAE↓ Sm ↑ Em ↑ F w β ↑ MAE↓ Augmented - 0.907 0.974 0.886 0.022 0.861 0.945 0.843 0.048 0.876 0.957 0.827 0.021 Augmented Weak 0.905 0.968 ...
work page 2026
-
[5]
Annotations generated by our method are used as pseudo labels for ZoomNet and UGTR model training. Trained with label CHAMELEON (76 images) CAMO (250 images) COD10K (2026 images) Sm↑ Em↑ Fωβ ↑ MAE↓ Sm↑ Em↑ Fωβ ↑ MAE↓ Sm↑ Em↑ Fωβ ↑ MAE↓ ZoomNet (Pixel-by-Pixel Annotated)0.845 0.919 0.758 0.034 0.769 0.836 0.668 0.083 0.818 0.884 0.688 0.034 ZoomNet (Our Pr...
work page 2026
-
[8]
Sam 2: Segment anything in images and videos
Ravi, N., Gabeur, V ., Hu, Y .-T., Hu, R., Ryali, C., Ma, T., Khedr, H., R ¨adle, R., Rolland, C., Gustafson, L., et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714,
-
[11]
Uniref++: Segment every reference object in spatial and temporal spaces
Wu, J., Jiang, Y ., Yan, B., Lu, H., Yuan, Z., and Luo, P. Uniref++: Segment every reference object in spatial and temporal spaces. arXiv preprint arXiv:2312.15715,
-
[1997]
Phraseclick: toward achieving flexible interactive segmentation by phrase and click
Ding, H., Cohen, S., Price, B., and Jiang, X. Phraseclick: toward achieving flexible interactive segmentation by phrase and click. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, pp. 417–435. Springer,
work page 2020
Show all 18 references
-
[2009]
A human-computer collaborative editing tool for conceptual diagrams
Pan, L., Yu, C., He, Z., and Shi, Y . A human-computer collaborative editing tool for conceptual diagrams. In Proceedings of the 2023 CHI Conference on Human Fac- tors in Computing Systems, pp. 1–29,
2023
-
[2010]
O., and Swamy, M
Esmaeilzehi, A., Ahmad, M. O., and Swamy, M. Fpnet: A deep light-weight interpretable neural network using for- ward prediction filtering for efficient single image super resolution. IEEE Transactions on Circuits and Systems II: Express Briefs, 69(3):1937–1941,
1937
-
[2011]
Perfor- mance of decamouflaging through exploratory image analysis
Sengottuvelan, P., Wahi, A., and Shanmugam, A. Perfor- mance of decamouflaging through exploratory image analysis. In 2008 First International Conference on Emerging Trends in Engineering and Technology , pp. 6–10. IEEE,
2008
-
[2016]
He, R., Dong, Q., Lin, J., and Lau, R. W. Weakly-supervised camouflaged object detection with scribble annotations. In Proceedings of the AAAI Conference on Artificial In- telligence, volume 37, pp. 781–789, 2023b. Hendrycks, D. and Gimpel, K. Gaussian error linear units (gelu...
-
[2017]
Reality3dsketch: rapid 3d modeling of objects from single freehand sketches
Chen, T., Ding, C., Zhu, L., Zang, Y ., Liao, Y ., Li, Z., and Sun, L. Reality3dsketch: rapid 3d modeling of objects from single freehand sketches. IEEE Transactions on Multimedia, 2023a. Chen, T., Fu, C., Zhu, L., Mao, P., Zhang, J., Zang, Y ., and Sun, L. Deep3dsketch: 3d mo...
2024 arXiv
-
[2018]
Deep3dsketch+: obtaining customized 3d model by single free-hand sketch through deep learning
Zang, Y ., Fu, C., Chen, T., Hu, Y ., Liu, Q., and Hu, W. Deep3dsketch+: obtaining customized 3d model by single free-hand sketch through deep learning. arXiv preprint arXiv:2310.18609, 2023a. Zang, Y ., Fu, C., Chen, T., Hu, Y ., Liu, Q., and Hu, W. Deep3dsketch+: obtaining c...
-
[2020]
M., and Song, Y .- Z
Hu, C., Li, D., Yang, Y ., Hospedales, T. M., and Song, Y .- Z. Sketch-a-segmenter: Sketch-based photo segmenter generation. IEEE transactions on image processing, 29: 9470–9481, 2020a. Hu, R., Rohrbach, M., and Darrell, T. Segmentation from natural language expressions. In Co...
2016
-
[2023]
Zoom in and out: A mixed-scale triplet network for camouflaged object detection
Pang, Y ., Zhao, X., Xiang, T., Zhang, L., and Lu, H. Zoom in and out: A mixed-scale triplet network for camouflaged object detection. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2150–2160, 2022a. Pang, Y ., Zhao, X., Xiang, T.-Z., Zhang, L., and Lu, H. ...
-
[2024]
F., and Llados, J
Riba, P., Dey, S., Biten, A. F., and Llados, J. Localizing infinity-shaped fishes: Sketch-guided object localization in the wild. arXiv preprint arXiv:2109.11874,
-
[2025]
H., Wei, Y ., Wei, S., and Zhao, Y
Zhang, S., Liew, J. H., Wei, Y ., Wei, S., and Zhao, Y . Inter- active object segmentation with inside-outside guidance. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12234–12244, 2020a. Zhang, S., Liew, J. H., Wei, Y ., Wei, S., and...
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.